AI coding with large codebases – Why understanding context is Especially Important in Enterprise Systems?

In Part 1, we looked at why an AI coding agent’s token usage can increase so quickly, and why the number of questions is not necessarily the real problem.

In Part 2, we explored how too much low-relevance information affects both cost and answer quality, and why targeted context works better.

Where does the business logic actually live?

In long-running enterprise software environments – whether IBM i RPG, Java, COBOL or even SAP – business logic is often not located in a single program.

In an IBM i environment, for example, rules may be spread across RPG programs, CL processes, DB2 for i tables, file structures, interfaces and old documentation. In many cases, part of how the system really works is best understood by experienced developers who have worked with it for years or even decades. That is why repository access alone may not be enough for an AI coding agent.

The agent may be able to read the code and still fail to find:

• where a specific business rule is actually implemented;
• which programs depend on a particular file or field;
• which relationships are critical when making a change;
• which seemingly strange details exist for a real business or operational reason.

This is also a recurring topic in the IBM i developer community: the real value of AI is not only in generating code, but also in understanding dependencies, uncovering relationships between programs and supporting impact analysis when changes are made.

Token Optimisation Is Not Just About Cutting Costs

Reducing token usage does not mean giving the AI less knowledge. The goal is to give it less unnecessary information and more relevant system context.

This can improve the way AI works in several areas:

1. Much better answers: if the model gets better input, the answer gets better too. Measurably better.
2. Faster responses: if the model does not have to process large amounts of irrelevant documentation or entire files for every question, it can respond faster.
3. Fewer repeated prompts: when context is incomplete, developers often have to keep correcting the agent: “Check this other program too.”, “That rule is not here.”, “Take the batch process into account.”, “This is a customer-specific exception.” Targeted retrieval and better-prepared context can reduce these extra rounds.
4. More accurate analysis: the agent is much more useful when it can see not only a piece of code, but also the relevant dependencies, documentation and business background for the task.
5. Better scalability: when one team, then hundreds of developers or multiple agents use the same AI infrastructure, poorly managed context can quickly become a cost and performance problem.
6. Less guessing in business-critical systems: the cost of a bad change is not measured only in API fees. A broken business process, a misunderstood dependency or a missed exception rule can have much bigger consequences.

Coming next: what does this mean in practice?