Build a research agent with source verification

A research agent that cites its sources looks trustworthy. But a citation is only a promise that "this page supports this sentence", and nothing in the format guarantees that the page exists, says what the sentence says, or says it without conditions.

This tutorial designs an agent that keeps those promises checkable. It searches, saves each source, drafts a brief in which every claim is tied to a short quoted passage and a URL, then runs a separate verification pass before a person signs the brief.

It applies the nine-step method from our pillar guide, build your first AI agent without code, to one domain, without re-teaching the steps. AI Chronicle has not built this agent in any product. The agent, brief and test cases are a labelled hypothetical design, and search tools are reported only as their documentation describes them, with the date we read it. For the basics of how agents use tools, see our explainer on what AI agents are.

Why citations need their own check

The clearest evidence comes from Nelson Liu, Tianyi Zhang and Percy Liang at Stanford, who audited four generative search engines (Bing Chat, NeevaAI, perplexity.ai and YouChat) in early 2023, with human annotators checking 1,450 queries per system. Citation recall is the share of statements fully supported by their citations; citation precision is the share of citations that support the statement they are attached to.[1]

On average, 51.5% of generated sentences were fully supported by their citations, and 74.5% of citations supported their associated sentence.[1] The authors' judgement was blunt:

“We believe that these results are concerningly low for systems that may serve as a primary tool for information-seeking users.”

— Liu, Zhang and Liang, Evaluating Verifiability in Generative Search Engines (2023).[1]

Two further findings shape the design. Across the four systems, citation precision was inversely correlated with perceived utility: the answers that looked most useful were often the least well cited. And the most precisely cited systems tended to copy or closely paraphrase their sources, sometimes without answering the question.[1] A reviewer cannot rely on how convincing a brief feels.

Princeton's ALCE benchmark, by Tianyu Gao and colleagues, automated those measures: a natural language inference (NLI) model judges whether cited passages entail each statement, which the authors report agreed well with human judgements.[2] ALCE also tested a common shortcut: write the answer first, then find citations. That combination, labelled ClosedBook plus PostCite, “delivers strong correctness but poor citation quality”.[2] Evidence attached after the fact often does not support the sentence it decorates.

Both studies examined 2023 systems, not current products. Their shared lesson is structural: generation and verification are different jobs, and verification needs its own stage. Our explainer on AI hallucinations covers why models produce confident, unsupported statements.

The job and its scope

The hypothetical agent answers one research question at a time and returns a draft brief of about a page, plus a claim ledger, to a named reviewer.

The last row matters more here than for most agents: every page it reads is untrusted input, as discussed below.

The claim ledger: the core data structure

The brief is a view; the ledger is the record. Each claim is one row with these fields:

• claim: one sentence, one fact;
• quote: the exact supporting passage, kept short;
• url and page title;
• retrieved: fetch date and time, plus a stored text copy;
• source type: primary (study, regulator, official document) or secondary;
• verdict: supported, partial, unsupported or unreachable, and which check set it;
• reviewer decision: approved, edited or cut, with initials.

Search tools now return much of this structure natively. Anthropic's documentation for its web search tool, read on 16 October 2026, describes citations that include the URL, the title and a cited_text field holding up to 150 characters of the cited passage, and notes that citations are always enabled for that tool.[3] OpenAI's web search documentation, read the same day, describes url_citation annotations carrying a URL, a title and the character positions in the answer that the source supports. It also describes a separate sources field listing every URL the model consulted, which it says is often larger than the set of citations.[4]

Neither feature does the verification for you. Anthropic's citations feature for supplied documents states that citations “are guaranteed to contain valid pointers to the provided documents”.[5] That guarantees the pointer is real, not that the passage supports the sentence. Keep the ledger, and check it.

Tools and permissions

Each tool gets an access level and a risk rating. Anything irreversible, which here means anything leaving the organisation, stays with a person.

Both search APIs document allowed and blocked domain lists, and Anthropic's describes a max_uses cap on searches per request.[3][4] Anthropic's web fetch documentation adds a constraint worth copying: the model can only fetch URLs that already appeared in the conversation, such as in search results, and a failed fetch returns a named error such as url_not_accessible.[6] That error is useful evidence for the verification pass.

The verification pass

Verification runs after drafting, as a separate stage with its own instructions, and checks every ledger row in four steps.

AI Chronicle

Check 1: the source resolves. Re-fetch the URL and compare it with the saved snapshot. A failed fetch, a redirect elsewhere, or a page that no longer contains the quote goes to the reviewer. Pew Research Center's 2024 study found that 38% of webpages that existed in 2013 were no longer accessible a decade later, and that 54% of the English Wikipedia pages it sampled had at least one reference link pointing to a page that no longer existed.[7] The dated snapshot records what the source said when it was read.

Check 2: the quote is really on the page. This is a string match, not a judgement, so give it to code. Normalise whitespace and quotation marks, then search the snapshot. A quote not found verbatim fails, even if a similar sentence exists.

Check 3: the quote supports the claim. Here a model can help, but only as a flagger. The test comes from the attribution framework of Hannah Rashkin and colleagues, Attributable to Identified Sources: a statement counts as supported only if a generic reader would accept it when framed as according to the cited page, [claim].[8][1] ALCE automated a version of this test with an NLI model.[2] Its authors also record the limit that matters most for a brief: their precision check “cannot detect the case of ‘partially support’”.[2] Partial support, such as a figure that holds for one region or year, is exactly what a reviewer needs to see. Treat any verdict other than a clear "supported" as a flag.

Check 4: the claim is no broader than the quote. Compare numbers, dates, units and named entities between claim and quote with code, and flag any mismatch. Then the reviewer looks for what code cannot: a dropped "up to", a single study stated as consensus, or a secondary report standing in where the primary document is available.

A claim that fails any check is marked with the reason and left for the person, not rewritten by the agent until the checker is satisfied.

Untrusted pages and prompt injection

A research agent reads text written by strangers, and some of that text may be written for it. OWASP's guidance describes indirect prompt injection as occurring when a model accepts input from external sources, such as websites or files, containing content that alters its behaviour.[9] One of its example scenarios is exactly this job: a user asks a model to summarise a webpage containing hidden instructions.[9]

The design limits the damage in three ways OWASP's mitigations also list: least-privilege tools with no publish or send action, a fixed output format that code validates, and human approval before anything leaves.[9] A page containing instructions aimed at an AI system is a reason to distrust the page; record that in the ledger.

A worked example (hypothetical): a policy brief

This example is fictional. The organisation, question, sources, quotes and verdicts were written by AI Chronicle to illustrate the process. No agent was run and no output was captured.

A small housing charity asks: "What do recent official sources say about households in temporary accommodation in our region?" The agent searches government and statistics domains, saves five sources and drafts four claims. Verification returns this ledger:

Acceptance criterion. The reviewer signs only when every remaining claim is "supported" or carries a recorded human decision. This is an editorial standard, not a measured result.

Test cases

Run each case several times after every change to instructions, tools or model. The Result column is empty on purpose: these are cases to run, not results we observed.

Failure modes specific to research agents

• Citation laundering. A real, relevant URL is attached to a claim it does not support. This is the gap Liu and colleagues measured as citation precision.[1]
• Quote drift. The "quote" is a paraphrase, or a sentence stitched from two places on the page.
• Synthesis without a source. A sentence combines two sources into a conclusion neither states. Split it, or label it as inference.
• Snippet citing. A search-result summary is cited without fetching the page.
• Partial support counted as full. Automatic checks are weakest here, so the reviewer sees every partial verdict.[2]
• Secondary for primary. A news summary stands in for an available study or regulation.

When not to build this agent

• The stakes demand a specialist. Legal, medical or financial advice needs a qualified reviewer whatever the tool; the agent may save them time, not replace them.
• The answer lives behind logins or paywalls. A design limited to public pages will over-rely on whatever is freely available.
• Nobody will review. Without a named person who signs, the verification pass produces flags that no one reads.
• The corpus is your own documents. A retrieval system over internal files is a different design; our RAG explainer covers how it works and fails.

Who signs the brief

Name the reviewer before the first run. Their job is not to redo the research but to read every flag, sample supported claims, confirm that high-stakes figures come from primary sources and record a decision on each row. The brief goes out under their name. Our guide on how to verify AI-generated research sets out the checks a reviewer can apply to any AI-assisted document.

Questions readers commonly ask

Can I use a model to verify its own citations?

As a flagger, yes; as the final check, no. Mechanical checks belong to code and the decision to a person. ALCE's automatic support checks agreed well with human judges overall, yet could not detect partial support.[2]

Why store quotes and snapshots instead of just links?

Pages change and disappear.[7] A dated quote shows what the source said when the claim was written, and lets a reviewer check it without repeating the search.

Do built-in citations from search APIs solve this?

They provide useful structure: cited text, titles and URLs.[3][4] They show where a statement came from, not whether the source fully supports it.

Which research tools already do some of this?

Our shortlist of the best AI research tools for evidence-led work compares products that offer cited answers. The verification pass here can sit on top of any of them.

The next step

Apply the same pattern, an evidence ledger plus a person who signs, to an internal investigation: build a SaaS onboarding investigation agent that drafts findings from account data for a customer team to review.

About these sources

All pages were accessed on 16 October 2026. This tutorial combines peer-reviewed and preprint research on citation quality, vendor documentation for search tools, independent survey research and security guidance, and AI Chronicle's own hypothetical design. Research figures describe systems tested in 2023 and are not current product rates. Vendor documentation establishes what a feature is described as doing, not how well it works. AI Chronicle has not built, run or tested the example agent. The scope box, tables, ledger and test cases are illustrative.

Related reading

• Build an ecommerce product-question agent
• Build an invoice-exception investigation agent