News
Enterprise AI Has a Token Problem. New Research Says More Context Isn't the Answer
Enterprise AI has spent the past few years chasing bigger models and larger context windows. New research from Jedify suggests that for some enterprise workloads, the better approach may be giving AI less information, provided it's the right information.
The enterprise AI company tested its context graph architecture against 100 business questions running on a production data warehouse. Each question was run twice, producing 200 data points across simple, moderate, and complex tasks.
Jedify's benchmark found that its system returned correct answers in 87 percent of those runs while averaging 25,036 tokens per SQL generation call. Perhaps more significantly, token consumption barely increased as the questions became more complicated.
Simple queries averaged 24,590 tokens, moderate questions averaged 24,980 tokens, and complex questions averaged 26,071 tokens, a difference of only about six percent between the simplest and most demanding categories.
The results underpin Jedify’s broader argument: enterprise AI may not need more context. It may need better context.
AI Doesn't Need to Read Everything
The traditional text-to-SQL approach is relatively straightforward. Give a large language model information about a database schema, then ask it to translate a user's natural-language question into SQL.
That becomes increasingly expensive as enterprise databases grow. Every additional table, column, definition, and piece of documentation potentially adds more information for the model to process. If that context is repeatedly included with every request, token consumption can grow alongside the database.
Jedify argues that simply giving a model more information can also make it harder to find what matters. Its study points to prior research on the "lost in the middle" problem, in which LLMs can struggle to use relevant information buried within very long contexts.
The company's alternative is a context graph. Jedify's Semantic Fusion architecture stores business concepts and metrics as entities containing definitions, validated SQL, relationships, filters, business rules, and synonyms. Instead of forcing the LLM to work out those relationships from scratch every time someone asks a question, much of that understanding is established beforehand.
When a question arrives, a profiler agent determines factors such as the timeframe, the query type, and the required entities. Only those relevant entities are then provided to the SQL generation process.
"The rest of the graph is not loaded," the study explains. "This is the token efficiency mechanism: question-time graph pruning driven by semantic classification, not brute-force schema inclusion."
In simpler terms, Jedify is trying to stop the AI from effectively rereading an entire enterprise database every time somebody asks it a question.
The company has described the semantic or context layer as the "load-bearing structure" connecting enterprise data with AI agents, with its knowledge graph mapping business entities to their definitions, relationships, and SQL implementations.
Fewer Tokens Without Sacrificing Accuracy
Reducing the amount of context an AI receives doesn't mean much if the resulting answers are wrong. That makes the accuracy portion of Jedify's benchmark particularly important.
The company tested 100 business questions across tasks including metric lookups, cohort analysis, revenue retention calculations, and multi-entity acquisition funnels. Each was run twice.
Of those 200 graded runs, 174 produced the correct answer, resulting in an overall accuracy rate of 87 percent. Outputs were compared with pre-validated reference answers.
Jedify contrasts these results with previously published enterprise baselines reporting roughly 60 to 70 percent accuracy for raw database schema injection and around 65 percent for the multi-agent CHESS approach.
Its token comparison is similarly striking. Jedify's architecture averaged 25,036 tokens per SQL generation call. Outside studies cited in its research estimate that raw schema injection for roughly 100-table enterprise environments can require between 50,000 and 150,000 tokens. CHESS (Contextual Harnessing for Efficient SQL Synthesis), an open-source LLM-based text-to-SQL framework, is listed at 339,965 tokens per request.
Based on those published estimates, Jedify calculates that its context graph architecture uses around 50 percent fewer tokens than the raw DDL baseline at 100-table scale. At 200 tables, the reduction exceeds 75 percent.
The important distinction is that Jedify isn't simply trying to squeeze the same enormous prompt into fewer tokens. Its architecture is designed to prevent irrelevant information from reaching the model in the first place.
The Numbers Come with a Caveat
There is an important qualification to Jedify's headline numbers.
The company did not test the competing architectures against the same production warehouse.
The 50,000- to 150,000-token estimates and competing accuracy figures come from previously published research using different schemas and question sets. This isn't, therefore, an apples-to-apples benchmark proving that Jedify beats those systems by the stated margins.
The study acknowledges that limitation directly, describing the comparisons as "juxtapositions, not experiments."
There is another limitation. All 100 questions used in Jedify's benchmark could be answered using entities already covered by its context graph. The test, therefore, doesn't establish how well the architecture would perform when an employee asks a question requiring business information that hasn't yet been encoded.
The 87 percent accuracy result was also graded by the benchmark's authors and has not been independently replicated. And those limitations don't negate Jedify's findings, but they do narrow what can reasonably be concluded from them.
Bigger Context Isn't Always Better Context
The more interesting takeaway from Jedify's research may ultimately be architectural rather than competitive.
AI developers have dramatically expanded the amount of information models can process at once. But the ability to fill a massive context window doesn't necessarily mean enterprises should do it.
A corporate database might contain hundreds of tables and thousands of columns, while an employee asking about quarterly recurring revenue may require information from only a small fraction of them.
Jedify's context graph approach changes that equation by moving some of the work outside the LLM.
Business definitions, relationships, and rules are established before the question is asked, leaving the model with a smaller and more focused reasoning problem.
That could also change which models enterprises need. Jedify's research argues that once the reasoning problem is sufficiently constrained, cheaper models may handle queries that would otherwise require more expensive frontier models.
There are still questions to answer, particularly how Jedify's context graphs perform against competing architectures under identical conditions and how reliably the system handles questions outside its existing coverage.
But the benchmark challenges a common assumption about enterprise AI: when a model struggles with a complicated business problem, simply giving it more information may not make it smarter.
As Jedify's researchers put it: "The fix is not a better model. It is a different architecture."