A production search system needs both: keyword search for exact terms, semantic search for meaning, and a merge step that combines their rankings into one list. That combination is called hybrid search, and it’s become the standard retrieval strategy for RAG systems.

We’ll walk through how each approach works, where each one breaks down, and how hybrid search merges them.

What is Keyword Search?

Keyword search, also called lexical search, matches the words in a query against the words in documents. Most implementations use the BM25 scoring algorithm, which ranks documents higher when they contain the query terms more often, when those terms are rare across the entire collection, and when the documents aren’t padded with unrelated text. A common word like “revenue” appears in nearly every page of an earnings report, so a match carries little weight. A string like ERROR_401_EXPIRED appears in one runbook, so BM25 treats that match as a strong signal.

This mechanical behavior is the strength. Keyword search finds exact identifiers, error codes, product names and quoted phrases every time they appear. It runs fast, and its results are easy to explain to users and auditors: a document matched because it contains the words you typed.

What is Semantic Search?

Semantic search compares meaning instead of spelling. It uses an embedding model to convert text into a vector, where vectors representing related ideas are positioned close together. The query is converted into a vector as well, and the engine returns the documents whose vectors are nearest to it, usually measured by cosine similarity.

Vocabulary stops being a constraint. A search for “authentication” retrieves the “login security” document because the two phrases occupy the same semantic neighborhood. Users can ask full questions in plain language, like “How do I let users sign in with their Google account?”, and retrieve OAuth documentation that shares almost no words with the query.

Where Each Approach Breaks Down

The two engines fail in nearly opposite situations, which is what makes them such good partners.

Keyword search breaks on vocabulary mismatch. Writers and searchers rarely pick the same words. A support agent types “refund policy” while the document says “returns and reimbursements,” and BM25 scores it near zero. Keyword search also carries no model of intent, so it can’t tell that “How do I cancel my plan?” and “subscription termination steps” want the same answer.

Semantic search breaks on exact strings. Embeddings compress text into general meaning, and rare tokens lose their identity in that compression. Error codes, part numbers, acronyms and person or product names tend to retrieve similar-looking content instead of the exact match. A query for “OpenEdge 12.8 release notes” may return notes for a nearby version, since the two documents are semantically near twins.

The second failure mode is the one we see teams underestimate. Demos run on natural language questions, so semantic search looks flawless until a real user pastes an invoice number or an error code into the search bar.

What is Hybrid Search?

Hybrid search runs both engines on every query and merges the two ranked lists into one. The merge step needs care because BM25 scores and cosine similarities live on different scales, so comparing them directly is meaningless. Reciprocal Rank Fusion sidesteps the scale problem by using positions instead of scores. Each document earns points based on where it ranks in each list, and a document that ranks high in both lists accumulates the most points and floats to the top.

Most implementations expose a weight between the two signals. A support knowledge base full of error codes benefits from a keyword tilt, while a policy wiki that people query in plain language benefits from a semantic tilt. We’d start balanced and adjust only after watching real queries fail.

Why Hybrid Search Matters for RAG

A RAG pipeline can only generate answers from the passages retrieval hands it. If retrieval misses the relevant passage, the LLM never sees it and can’t cite it. No amount of prompt engineering or model quality can recover a fact that never entered the context.

Consider a question like “What was revenue guidance for FY 2026?” Semantic retrieval surfaces passages discussing financial outlook and guidance, while keyword retrieval ensures the exact phrase “FY 2026” is matched so the model cites the correct fiscal year instead of a semantically similar one. Combined, the two approaches retrieve both the relevant passage and the precise details the LLM needs to generate an accurate answer.

Hybrid Search in Progress Agentic RAG

The Progress Agentic RAG platform runs keyword and semantic search together by default. The platform is powered by NucliaDB, which unifies semantic search, keyword search, metadata search and knowledge graph traversal in a single store, so we never maintain a separate keyword index next to a vector database or write our own merging logic.

The Search configuration panel exposes Reciprocal Rank Fusion directly. We can adjust the weighting to boost semantic results over keyword results, or the reverse, depending on how our users search and the types of information they need to retrieve.

Wrap Up

Keyword search matches exact words with speed and predictability, semantic search matches meaning across different vocabulary. Each one fails where the other succeeds. Hybrid search runs both and merges their rankings with Reciprocal Rank Fusion, which is why it has become the default retrieval strategy for RAG. Progress Agentic RAG ships this behavior out of the box, with the fusion weighting a single setting away.

For more details and to get started with Progress Agentic RAG, be sure to check out the following resources:

Frequently Asked Questions

Is semantic search always better than keyword search?

No. Semantic search wins when queries are phrased differently from the documents, and keyword search wins when the query contains an exact string such as an error code, a product name, or a quoted phrase. Each covers the other’s blind spot, which is why production systems merge both.

What is Reciprocal Rank Fusion in hybrid search?

Reciprocal Rank Fusion (RRF) is the algorithm that merges the ranked lists from keyword and semantic search. Since the two engines score documents on incompatible scales, RRF ignores the raw scores and awards each document points based on its rank position in each list. Documents that rank well in both lists rise to the top of the fused ranking, and many systems, including Progress Agentic RAG, let us weight one list more heavily than the other.

Do I need a separate keyword index and vector database to run hybrid search?

It depends on the stack. Many teams pair a keyword engine like Elasticsearch with a vector database and merge results in application code, which works but doubles the infrastructure and keeps two indexes in sync. Unified stores such as NucliaDB index text for both search modes at ingestion, so hybrid search runs as a single query against one index.

Here's the short answer: use RAG when the model needs knowledge it lacks and use fine-tuning when the model needs behavior it lacks. RAG retrieves facts from an external knowledge base at query time, so answers stay current and citable. Fine-tuning adjusts a model's weights with curated training examples, so it changes how the model responds (its tone, format, and style) rather than what it knows.

That one-line rule covers most decisions, but the details matter, especially around cost, hallucinations and keeping answers up to date. We'll walk through both approaches and the situations where each one fits.

What is RAG?

RAG connects an LLM to an external knowledge source and grounds every answer in retrieved content. The process runs in three steps. When a user asks a question, the system:

• Retrieves the most relevant passages from an indexed knowledge base
• Augments the prompt with those passages
• Generates a response grounded in that context

Suppose we've indexed Progress Software' Q2 2026 earnings report and someone asks, "What was revenue guidance for FY 2026?" The system pulls the guidance passage from the report and hands it to the model along with the question. The answer quotes the actual figures, and it can cite the exact source paragraph.

The defining trait of RAG is that the model itself never changes. Update a document in the knowledge base, and the very next answer reflects it.

What is Fine-Tuning?

Fine-tuning takes a pre-trained model and continues its training on a curated dataset of example inputs and outputs. The model's weights shift to match the patterns in those examples. Training data is typically provided as JSONL, where each line is a conversation showing the model exactly how it should respond:

{"messages": [
{"role": "system", "content": "You are a support assistant for FinTrack."},
{"role": "user", "content": "How do I export my transactions?"},
{"role": "assistant", "content": "Happy to help! Head to Settings > Data,
choose Export, and pick CSV or JSON. Exports cover the last 24 months."}
]}

Feed the model a few hundred examples like this and it learns the pattern: the greeting style, the concise step-by-step structure, the product terminology. After training, the model produces that style by default, without needing lengthy instructions in every prompt.

That's what fine-tuning is good at: tone, output format, domain-specific phrasing and following a house style. What it does poorly is act as a knowledge store. Facts from training examples aren't reliably memorized, and the model can blend or misremember them at generation time. Fine-tuning effectively freezes knowledge at training time. When the FY 2027 guidance replaces FY 2026, a fine-tuned model keeps answering with the old numbers until we retrain it.

One practical caveat: OpenAI is winding down its fine-tuning platform for new users. The technique itself remains available through other providers and on open-source models, but if you're on the OpenAI stack, RAG becomes an even clearer default.

How do RAG and Fine-Tuning Compare?

The differences show up across almost every practical dimension:

Our take: For question answering over company knowledge, which is what most teams are actually building, RAG should be the default. The ability to cite sources alone justifies it in any setting where users need to verify answers, and the update story (re-index a file versus schedule a training run) decides the matter for content that changes quarterly or faster. Fine-tuning still earns its keep when the output itself is the product, like a classifier that must emit a strict JSON schema or an assistant that must sound unmistakably like your brand—though availability and tooling vary by provider.

Can you combine RAG and fine-tuning?

Yes, and the combination is often stronger than either alone because they solve different problems. A support team might fine-tune a model on hundreds of its best-rated replies, so responses match the company’s voice and formatting conventions. That fine-tuned model then sits behind a RAG pipeline that retrieves from the current product documentation. Fine-tuning supplies the behavior, while retrieval supplies the facts. When the docs change, answers change with them, and no retraining is required.

Progress Agentic RAG

The most common objection to RAG isn’t conceptual; it’s infrastructural. Someone has to stand up embedding models, a vector database, chunking and indexing pipelines and retrieval logic—then keep all of it running. The Progress Agentic RAG solution removes that objection by delivering RAG-as-a-Service. Upload documents to a Knowledge Box, and the platform handles extraction, chunking, embeddings and indexing automatically.

Because Agentic RAG works with any LLM, the hybrid strategy above fits naturally. We can point the retrieval pipeline at a stock model today and swap in a fine-tuned one later, without touching the knowledge base. The knowledge layer and the behavior layer stay independent, which is exactly how we’d want them.

In addition to the above, the clearest wins for retrieval are the tasks where the answer lives in a document that someone edited recently:

• Customer support over live product docs. A release moves the export flow on Tuesday, and the answer has to move with it. A fine-tuned model keeps reciting the old steps until we rebuild the dataset and retrain.
• Investor and financial questions. When FY 2027 guidance replaces FY 2026, each figure needs the source paragraph attached to it. Model weights have nothing to cite.
• Policy and compliance lookups. An auditor asks which version of the expense policy was applied in March. Retrieval returns the clause itself, while a fine-tuned model produces an answer nobody can trace.

Fine-tuning can have a real job in all three, shaping how the reply reads, but the lookup underneath it belongs to retrieval.

FAQs

Is RAG cheaper than fine-tuning?

Usually, and especially over time. RAG has ongoing retrieval and indexing costs, but updating knowledge is as cheap as reindexing a document. Fine-tuning carries the cost of dataset curation, training runs and evaluation, and that full cycle repeats every time the model needs to learn something new. For fast-changing content, fine-tuning costs compound quickly.

Does fine-tuning stop a model from hallucinating?

No. Fine-tuning shapes how a model responds, not how truthful it is. A fine-tuned model will confidently produce wrong facts in a perfect brand voice. Grounding answers in retrieved documents, the way RAG does, is the more direct lever against hallucination because the model works from real text instead of its memory.

Can I use RAG with a fine-tuned model?

Yes. RAG is model-agnostic, so the generation step can use any LLM, including one you’ve fine-tuned. This pairing is common in production: the fine-tuned model handles tone and format while retrieval keeps the facts current. The Progress Agentic RAG solution supports this directly by letting you bring your own model.

Wrap-Up

RAG and fine-tuning answer different questions. RAG changes what a model can know by retrieving from a live knowledge base, which keeps answers current, citable and grounded. Fine-tuning changes how a model behaves by training it on curated examples, which is the right tool for tone, format and style. Reach for RAG first when the problem is knowledge, reach for fine-tuning when the problem is behavior, and combine them when you need both.

Progress Agentic RAG covers the retrieval side of this without the pipeline work. Start a free trial to index your own documents or book a live demo to walk through it with a Progress AI expert.

Retrieval-Augmented Generation (RAG) systems typically rely on embeddings to find semantically similar information. Embeddings help a system understand that phrases like "company earnings" and "financial results" refer to similar concepts. What they don't do is explicitly recognize that "YogeshGupta" is a person or that "August 31, 2025" is a date. When a query depends on a specific person, organization, product or point in time, that missing structure becomes important.

Named Entity Recognition (NER) is a natural language processing technique that locates named entities in unstructured text and classifies them into categories such as people, organizations, locations, dates and products. In a RAG pipeline, these entity labels become structured metadata that sits alongside embeddings in the search index, enabling more precise filtering, better retrieval for exact names and dates and features like entity-based autocomplete.

In this article, we'll unpack how NER works, why it matters for retrieval quality and how the Progress Agentic RAG solution applies entity detection to our content.

What is Named Entity Recognition?

Most organizational knowledge lives in unstructured text. Contracts, earnings reports, support tickets and internal wikis are full of facts written as sentences rather than stored in neat database columns. NER adds structure to that prose by doing two things. It locates each entity mention within the text. It then classifies each mention into a predefined category.

What are common named entities?

Most NER models ship with a standard set of categories:

• People: Executives, authors, customers and other named individuals.
• Organizations: Companies, government agencies, universities and internal departments.
• Locations: Countries, cities, regions and street addresses.
• Dates and times: "August 31, 2025," "Q3 2025" or "the quarter ended August 31."
• Events: Earnings calls, product launches, conferences and outages.
• Products: Product names, model numbers and SKUs.
• Numerical values: currencies, percentages and quantities.

Consider a sentence you might find in an earnings report:

Yogesh Gupta discussed Progress Software's results for the quarter ended August 31, 2025.

After NER processes it, the same sentence carries explicit labels:

[Yogesh Gupta](PERSON) discussed [Progress Software](ORGANIZATION)'s results for the quarter ended [August 31, 2025](DATE).

The text itself hasn't changed. What's new is that the system knows what those words represent, and that knowledge can be stored as metadata alongside the original text and its embedding. NER relies on context to make these classifications: it can determine that "Progress" refers to a company in this sentence, even though the same word can mean forward movement in other contexts.

Why Does NER Matter in a RAG Pipeline?

A typical RAG pipeline splits documents into chunks, converts each chunk into an embedding and stores everything in an index. Semantic search then matches queries against that index by meaning. This works well for conceptual questions, but similarity alone can blur the details. Earnings reports from different companies and quarters all look semantically similar, even though only one contains the fact a user is asking about.

NER adds a second, complementary signal. When entities are detected at ingestion time and stored as index metadata, the RAG system can use them at query time in several ways:

Suppose a knowledge base contains several years of quarterly reports and a user asks, "What did Yogesh Gupta say about AI in the Q3 2025 report?" An embedding-only search will surface passages about executives and AI from multiple quarters. With NER metadata, the system also knows which chunks mention the person "Yogesh Gupta" and the period "Q3 2025," so it can weight or filter results accordingly.

NER doesn't replace embeddings. Embeddings capture the meaning of a whole passage, while NER labels the individual people, companies and dates mentioned inside it. The two signals work best together.

How Does NER Provide Better Answers?

Entity awareness pays off most when the identity of a subject matters as much as the topic itself. Here are the situations where we find it makes the clearest difference.

Questions about a specific organization. If a user asks, "Which contracts mention Acme Corp?", semantic search will happily return contracts involving similar companies in similar industries. An organization filter narrows results to documents where Acme Corp actually appears.

Date-sensitive financial queries. Guidance, pricing and policies change over time. Recognizing "August 31, 2025" as a date prevents the system from blending figures from different reporting periods into one answer.

People lookups. Names carry little semantic content on their own, so embeddings often can't tell two executives apart. A person entity makes "What has Yogesh Gupta said about acquisitions?" resolve to the right speaker.

Similar or ambiguous names. Product names, project code names and short company names frequently collide with ordinary words. Classifying them as entities keeps unrelated passages out of the context sent to the LLM, which directly improves the quality of the generated answer.

How NER Helps Auto-Build Knowledge Graphs

One of the more interesting things NER unlocks is the knowledge graph, a structure that represents entities as nodes and the relationships between them as edges. From our example sentence, "Yogesh Gupta" and "Progress Software" become nodes, and a relation extraction step adds an edge—such as "is CEO of"—between them.

When this extraction runs across an entire knowledge base, facts scattered through separate documents connect into a single navigable graph. That makes it possible to answer questions that require following relationships rather than matching one passage.

NER in Progress Agentic RAG

The Progress Agentic RAG solution runs entity detection automatically as part of ingestion. When we upload a resource to a Knowledge Box, the platform identifies entities across 16 built-in entity types, covering people, organizations, dates, events, locations and more. We can also define our own entity types for domain-specific vocabulary.

For deeper customization, the Graph Extraction agent lets us describe the entities and relationships we care about, along with examples that guide the extraction. A legal team, for instance, might define plaintiffs, defendants, contracts and clauses, so the extracted entities reflect their domain rather than generic categories.

That separation is what raises search precision. With plaintiff and defendant as distinct types, the team can pull only the filings where a given company sat on the defense side, which one generic PERSON or ORGANIZATION label can't express. A compliance team gets the same effect by mapping internal regulation codes and control IDs, so a question about a single control returns the records governed by it instead of anything that reads like policy language.

Wrap-Up

Named Entity Recognition turns the names, organizations, dates and products buried in unstructured text into structured metadata. In a RAG pipeline, that metadata works alongside embeddings to sharpen filtering and retrieval, power entity-based autocomplete, support anonymization and lay the foundation for knowledge graphs. The Progress Agentic RAG solution handles all of this automatically at ingestion time.

To see entity detection running against your own documents, book a live demo with a Progress AI expert or start a free trial and upload a file to a Knowledge Box.

FAQs

What is the difference between NER and keyword extraction?

Keyword extraction surfaces the phrases that best summarize a document's topics, such as "financial results" or "revenue guidance." NER instead finds specific mentions and assigns them to categories, labeling "Yogesh Gupta" as a person and "August 31, 2025" as a date. In short, keywords describe what a document is about, while entities identify who and what it refers to.

Does NER replace embeddings in a RAG system?

No. Embeddings capture the meaning of an entire passage, which is what makes semantic retrieval possible. NER labels the entities inside that passage. A strong RAG pipeline uses embeddings to find related content and entity metadata to filter and rank it when a query depends on an exact name or date.

Can NER identify custom, domain-specific entities like product codes or internal project names?

Yes. Beyond built-in categories, NER systems can be configured to recognize custom entity types. With the Progress Agentic RAG solution, we can define our own entities and relationships for the Graph Extraction agent, with descriptions and examples that help the platform distinguish things like internal project names from ordinary text.

What lets an AI assistant go beyond quoting a contract and trace who signed it, which policy in your estate governs it and whether it is still current? The answer has to make business relationships explicit and preserve the evidence behind each step, even when two systems disagree. A knowledge graph is the structure that makes it possible.

What Is a Knowledge Graph?

Who signed a contract and which policy governs it: those questions start with the relationships your systems store. Applications often express those relationships through different tables and joins, with system-local or inconsistently shared identifiers. Cross-system questions force an application or AI assistant to recreate that logic. That is repeated plumbing. A knowledge graph makes those relationships reusable.

The change is straightforward: give entities a shared identifier and connect them through explicit, typed relationships. You can then follow one path instead of reproducing every application’s joins. The Resource Description Framework (RDF) provides that shared path through a W3C-standard data model, recording each relationship as a subject-predicate-object triple.

Take a customer who is party to a contract. The customer is the subject, the contract is the object and “is party to” is the predicate or named relationship. A query can now follow that relationship to the contract and then to the governing policy. No custom joins at every step.

That relationship only holds while both systems agree on which customer and which contract they mean. Shared identity is half of it. The relationship also needs shared meaning, which is where an ontology comes in. Ontologies define your key concepts, like customers, contracts and the policies that govern them, as well as how they connect. Ontologies stay flexible by design, so a missing detail doesn’t automatically make a record invalid. When you need to enforce required fields or data shapes, handle that separately with something like the Shapes Constraint Language (SHACL). Keeping meaning and validation apart sets up the real question: do the graph’s relationships actually support the answer?

Why Does Enterprise AI Need a Knowledge Graph?

Try a question that sounds simple: “Which policy governs a given customer’s current contract?” Similar passages about the customer and the policy, the kind a similarity search returns, can look persuasive, but they do not prove that the records belong together. Let the model bridge that gap, and you can get a polished answer that no source supports.

A defensible answer needs the full path: the customer, the contract, the governing-policy relationship and evidence for each step. Stable identifiers let an authorized caller traverse that path and inspect permitted facts. Confidence is not the goal; evidence is.

A knowledge-orchestration layer, such as the one in the Progress Agentic RAG solution, can assemble that traceable evidence from retrieved sources. It uses retrieval-augmented generation (RAG) to place retrieved evidence in a model’s context. But retrieval is only as trustworthy as the rules around it. It should require authorized, traceable evidence and define how the system behaves when evidence is incomplete.

How Do Graph and Vector Search Work Together?

Those evidence rules also shape how graph queries and vector search divide the work. Graph queries complement vector search rather than replace it. Vector search compares numeric text representations, called embeddings, to find passages with similar meaning. That helps when the wording is uncertain: “renewal risk” can surface passages about churn or approaching contract end dates. Graph queries handle a different job. They follow a defined path, such as every project assigned to a consultant under a specific agreement.

In a hybrid pipeline, vector search proposes passages while graph queries resolve their entities and governing relationships. A GraphRAG study applies a related pattern: a graph-based index built from source text supplies graph-derived summaries for answers. That related pattern still leaves you to route loose wording to vector search and defined paths to graph queries. Mixed questions can use both. For each query class, write down what runs first, then set an evidence threshold and spell out the fallback.

Those query rules still need a source-authority matrix. A vector result might mention the right customer in an expired contract, while the graph lacks the latest amendment text. Quote the expired terms, and the customer could act on the wrong number. So, decide which source owns the contract status and amendment text. Define how effective dates filter results and when facts expire. When sources conflict, choose which source wins. If neither can, present the disagreement or decline to answer. “Use both” is not a conflict policy.

Where Should an Enterprise Architect Start?

Start by turning that source-authority matrix into a narrow set of relationship-dependent questions. For each question, document the required entity types and relationships and source owner, then define the identity rule, freshness target, conflict policy and decision maker. Turn those requirements into acceptance tests for a current, authorized path to evidence.

Creating those relationships is the next thing you have to schedule, and it does not all have to be hand-modeled. Agentic RAG builds graph relations two ways: graph extraction agents derive them from your documents, or you write them directly through its ingestion API. Both paths build the same graph, and the API documents three relation shapes:

• One entity to another entity
• A document to an entity it mentions
• One entity declared equivalent to another entity

The third shape is where competing definitions get handled. You mark “master agreement” and “framework contract” as the same concept, rather than leaving each system’s label to stand alone. The documented gain is a more structured representation for search, not a record of which path an answer took, so keep your own evidence tests either way.

Whichever creation path you use, customers, contracts, governing policies and effective dates make a useful first slice. They expose inconsistent identifiers, competing definitions, missing ownership and stale relationships without forcing you to model the whole company. Expand only when the slice stays accurate. Your architecture review board should be able to compare the slice’s answer accuracy and running cost against your current retrieval setup. A smaller, trustworthy graph beats an enterprise graph nobody can explain.

Book a demo to see this run against your own content.

FAQ

How should graph authorization follow source-system permissions?

Carry source identity and access metadata along that first slice, then evaluate permissions before facts or passages enter the model context. Mixed-permission paths are where weak designs show up. Access to a customer record must not automatically grant access to a linked contract or policy. Keep denials observable without exposing the restricted value. Authorization decides who can traverse the path; temporal rules decide which path is current.

How should the graph represent relationships that change over time?

Represent the relationship’s effective interval or status and preserve its source timestamp. A query for “current” facts should apply an agreed business-time rule, not trust the latest ingestion timestamp. Then test late-arriving amendments and overlapping intervals. Test retroactive corrections separately because a correction can change which path was valid for a past question.

How do you know whether the graph justifies its governance cost?

Temporal rules add governance work, so make the graph earn it. Measure answer accuracy and the rate of unsupported answers. Then measure join work and identity resolution, including freshness failures. Continue only when the improvement justifies the ownership and validation work.

What Is Grounding in RAG?

Start with what you want: better answers, with fewer invented ones. Grounding is the means to that end. It tethers each answer to an authoritative source, so the model does not rely on training data alone. Retrieval-augmented generation (RAG) is the pattern most teams use. It fetches passages from your content at question time and tells the model to answer from them.

Grounding is broader than RAG. You can ground an answer through:

• A tool call that returns a live system value
• A lookup against a governed table
• Citation rules that reject unsupported claims
• A context API that supplies curated metadata

Keep the goal separate from the mechanism. When a vendor says “we do RAG,” ask whether the answers improve and how you can prove it.

How Does Grounding Prevent Hallucinations in RAG?

An AI hallucination happens when a model fills a knowledge gap with plausible text. The model is not lying. It predicts patterns without knowing whether they are true. Grounding narrows that gap by retrieving relevant facts before the model writes its answer. In one study, retrieval cut hallucinated responses by more than 60%.

The percentage matters less than the audit path. A grounded answer can cite the exact source it used, so you can inspect the supporting passage. Picture the alternative. A confident answer reaches a customer email or a board slide. A week later, someone asks where it came from. With a memory-only model, there is nothing to open.

Why Doesn’t a Bigger Model Mean Better Answers?

If hallucinations come from knowledge gaps, a bigger model should have fewer gaps, right? The data does not cooperate. OpenAI’s o3 reasoning model hallucinated on 33% of the PersonQA benchmark. The older o1 scored 16%. More capability did not buy more truth.

Your enterprise data creates a more basic limit. No public model knows your current contracts or last quarter’s pricing exception. A larger model arrives knowing more about the world and exactly as little about your company.

Warning: If model upgrades are your hallucination strategy, your reliability roadmap depends on a vendor’s release schedule. Grounding keeps that control on your side.

A larger context window does not solve that problem. It accepts more material, but it still cannot prove which passage drove an answer. It also cannot decide whether the person asking may see that source. Grounding adds traceable sources and query-time access checks. Model size adds neither.

How Do You Turn Grounding Into Durable Architecture?

Grounding now earns its place in the design. The next question is how to make it survive model changes and fit the estate you already run.

Build a shared, model-agnostic grounding layer between enterprise content and each AI experience. OpenAI-compatible APIs can make a model swap a configuration change. Embeddings create the deeper lock-in. Change the embedding model and you must re-index the corpus, so keep that choice open too.

Trace at least one real source-system path before you approve the design: SharePoint to an ingestion connector, then to a governed index, permission-filtered retrieval and the AI experience. That path exposes integration and access assumptions that a box labeled “enterprise data” hides.

The design also needs evaluation. A separate evaluator, often called a judge model, compares each answer with its retrieved context and assigns a generic groundedness score. Judge models can use different scales, so store the scoring method with the result. If score history must move between platforms, require a documented export format and verify it during procurement. Do not assume it travels.

Progress Agentic RAG provides the scoring function through REMi (RAG Evaluation Metrics). Its named Groundedness metric measures whether the generated answer is supported by retrieved context. That verified capability does not imply that score export is available.

Durable grounding must also honor who is asking. Filter candidate sources against each user’s permissions at query time. Log the retrieved content and source identity. When an auditor asks why the system produced an answer, that log separates evidence from guesswork.

What Should an Architect Do Now?

Ask a harder question than “which model should we use?” Check that every answer can point to its source and that retrieval preserves source permissions. Then confirm that the evaluation method survives a platform review. Design grounding as shared infrastructure with replaceable models and embeddings. Verify score portability instead of assuming it.

Book a demo to see Groundedness measured against your own content.

FAQ

Three practical questions follow once grounding becomes an architecture decision.

How Do You Prove an Answer Is Actually Grounded?

Return the source with every answer. Then check that the cited passage supports the claim. A real grounding layer exposes the retrieved content and its origin. A footnote label added after generation proves nothing.

Can Grounding Eliminate Hallucinations Completely?

No. Grounding reduces hallucinations by shrinking the gaps a model fills on its own. It also makes failures easier to detect because each answer points to a source you can inspect. The goal is an auditable system, not a perfect model.

Does Grounding Slow Answers Down or Add Cost?

Retrieval adds latency and compute cost. That tradeoff favors grounding when someone will act on the answer. For lower-stakes paths, use a lighter model. Reserve heavier models for decisions with larger consequences.