GitHub Copilot review: coding tasks and limitations

Editor's note: This is a documentation-led edition. We have not yet run our own hands-on tests of GitHub Copilot. What follows rests on official documentation, published pricing and independent studies we read in full, all accessed on 16 October 2026. A tested update will follow.

GitHub Copilot started out as autocomplete. It is now five products under one name: inline suggestions, chat, an agent that edits files in your editor, a cloud agent that turns an issue into a pull request, and a pull-request reviewer. Each runs under different rules and costs a different amount.

Two changes in 2026 matter as much as the features. On 24 April GitHub began training on individual users' interactions unless they opt out. On 1 June it replaced "premium requests" with metered AI credits.

So this review takes Copilot apart by coding task: what GitHub documents, what each part costs and where the limits sit, then the independent evidence. Our shortlist of AI coding assistants places Copilot among its rivals.

What GitHub Copilot is, and what each part is for

Sources: GitHub Docs.[1][2][3][4][5] Copilot works in VS Code, Visual Studio, JetBrains IDEs, Xcode, Neovim, Eclipse and other tools, and as Copilot CLI in the terminal.[6][1] It is not available for GitHub Enterprise Server.[7]

Plans and prices (as of 16 October 2026)

Sources: GitHub's plans page and docs.[6][7][8] Completions and next edit suggestions do not use credits; chat, agent mode, the cloud agent, code review and Copilot CLI do.[7][8]

From premium requests to credits. Until June, paid plans had a monthly quota of premium requests: 300 on Pro, 1,500 on Pro+, extras at $0.04.[9] Since 1 June 2026 every plan bills in GitHub AI Credits, priced from the tokens each model consumes.[10][11] Annual Pro and Pro+ subscribers stay on premium requests until their plan ends, and for them each code review now costs 13.[10][9]

"Today, a quick chat question and a multi-hour autonomous coding session can cost the user the same amount."

— GitHub, GitHub Copilot is moving to usage-based billing (April 2026).[10]

That is no longer true: a long agent session on a frontier model "will cost more".[8] For budgeting, note that only base credits are fixed; the flex allotment "is designed to adapt as the economics of AI evolve".[8] The old fallback to a cheaper model when the quota ran out has gone.[10] On Business and Enterprise, if admins disallow extra spending, "Copilot pauses until the next cycle."[6] Unused credits are forfeited at each monthly reset.[8]

Completions, chat and agent mode

Completions are the cheapest part of Copilot because they sit outside the credit system.[8] Chat has modes for questions, planning, multi-file edits and agent work.[2] GitHub warns that languages less common in public repositories "may produce fewer or less robust suggestions."[6]

In agent mode, Copilot "decides which files to change, makes the edits, and runs commands as needed, iterating until the task is done." You approve each command by default, but "Your administrator, or your own editor settings, may allow some commands to run automatically."[3] It runs in VS Code, Visual Studio, JetBrains IDEs, Xcode and Eclipse, can be switched off by admins, and uses credits on every prompt.[3][2] Each MCP server you add gives it another system to act on.[3]

The cloud agent: from issue to pull request

Assign a GitHub issue to Copilot and the cloud agent will "start working on the task, raise a pull request, then request a review from you when it's finished."[12] It edits files and runs tests in an ephemeral cloud environment, not on your machine.[4] It can also be started from Chat, the CLI, IDEs, Slack or Jira, or on a schedule.[12]

AI Chronicle

The guardrails are unusually well documented:[13]

• Only users with write access can trigger it; comments from others "are never presented to the agent."
• It pushes only to its own branch and cannot run git push itself.
• Its draft pull requests "must be reviewed and merged by a human", and the requester cannot approve them.
• GitHub Actions workflows wait for approval from someone with write access, unless you change that.
• It checks its code with CodeQL, the GitHub Advisory Database and secret scanning, and asks Copilot code review for "a second opinion".
• Hidden text such as HTML comments in an issue is filtered out, to blunt prompt injection.

The limits are as specific. A session covers one repository, one branch and one pull request, and stops after 59 minutes, which "cannot be extended or bypassed."[4] It works only on GitHub-hosted repositories.[4] Once assigned, it ignores later comments on the issue; corrections go on its pull request.[12] Session logs "do not replace your own review and testing."[4]

A firewall limits its internet access, with a package-registry allowlist on by default. It covers only processes started by the agent's shell tool, not MCP servers, and "Sophisticated attacks may bypass the firewall," GitHub warns.[14]

"the firewall provides protection for common scenarios, but should not be considered a comprehensive security solution."

— GitHub Docs, Customizing or disabling the firewall for GitHub Copilot (2026).[14]

Code review

Copilot code review comments on pull requests and suggests fixes on paid plans, on GitHub.com and in the main IDEs.[5] It uses GitHub Actions to gather repository context, so since 1 June 2026 each review costs Actions minutes as well as credits.[5][11] GitHub estimates $0.05 to $1 of credits per review on "Lite" effort and $0.25 to $5 on "Balanced", the default.[5] Model it before turning on automatic reviews for a busy repository.

"Model switching is not supported", some files such as package.json, logs and SVGs are skipped, and by default its reviews "do not count toward required approvals". A preview setting lets Copilot's approval satisfy a required-approval rule; weigh that carefully.[5]

Model choice

On 16 October 2026 GitHub listed more than 30 models from OpenAI, Anthropic, Google, xAI, Microsoft and Moonshot AI, adding that "Model availability is subject to change."[15] Paid plans can choose a model; Free and Student get automatic selection only.[7] Model choice can change data handling: with Claude Fable 5 and 5.1, "Anthropic retains data, including prompts and outputs, by default" for safety classifiers.[7] GitHub's hosting page describes no-training or zero-retention commitments from the providers that serve its models.[16]

Controls for companies: content exclusion and the public-code filter

Content exclusion lets Business and Enterprise admins stop Copilot reading named files.[17] But it is "currently not supported in Edit and Agent modes of Copilot Chat in Visual Studio Code and other editors," and Copilot may still use "semantic information from an excluded file" that the IDE supplies.[17] It is not a way to keep secrets from agents; keep secrets out of the repository.

The public-code filter hides suggestions that match public GitHub code, comparing each suggestion and about 150 characters of context with a public-code index.[18] Set to allow, Copilot shows "code references" with repositories and licences instead. GitHub says matches occur in under 1% of suggestions, and the index omits private and non-GitHub code.[1]

Data and training: the April 24, 2026 change

GitHub announced on 25 March 2026 that "From April 24 onward" it would train on interaction data from Free, Pro and Pro+ users unless they opt out.[19] GitHub's docs give the year, 2026, and now include Max; the plans FAQ still says "Starting on April 24" with no year.[18][6]

The data includes prompts and code snippets, code around your cursor, accepted outputs, file names and repository structure. Private repositories "at rest" are excluded, but private code Copilot processes while you work is not.[19] Data may go to Microsoft and other affiliates, not to third-party model providers, and earlier opt-outs carry over.[19] Turn off "Allow GitHub to use my data for AI model training" in your Copilot settings.[18]

Business and Enterprise data is not used for training. IDE chat and completion prompts are not retained; prompts on other surfaces are kept for 28 days.[6] A developer using a personal Pro account on client code is under individual terms, whatever their employer assumes.

IP indemnity: who is covered, and on what terms

GitHub offers IP indemnity only on Business and Enterprise; its plans table marks it "Not included" for individual plans.[6] For direct customers, GitHub's General Terms cover a paid product "unmodified as provided by GitHub and not combined with anything else", and since 5 March 2026 its Generative AI Services Terms extend that defence to outputs.[20][21][22]

One condition has shifted. GitHub's FAQ still ties indemnification to unmodified suggestions "when Copilot's filtering is enabled".[6] Microsoft's copyright-commitment page says the duplicate-detection filter "is no longer required for CCC coverage" of GitHub offerings from 3 April 2026.[23] Until the two agree, ask your account team which applies, and keep the filter on. Our explainer on AI training and copyright explains why output claims differ from training claims.

What independent research says

Peng and colleagues, 2023. In a randomised trial of 95 Upwork developers building an HTTP server in JavaScript in 2022, the 45 with Copilot finished 55.8% faster (95% confidence interval 21% to 89%).[24] One small task, no quality check, GitHub and Microsoft authors. GitHub's plans page claims users are "up to 55% more productive at writing code" without citing a source.[6]

Cui and colleagues. Field experiments at Microsoft, Accenture and a Fortune 100 company found 4,867 developers with an AI coding assistant completed 26.08% more tasks (standard error 10.3%), more so the less experienced.[25] The tool was 2022–23 autocomplete.

METR. In early 2025, 16 experienced developers took 19% longer with AI tools, mostly Cursor, while believing they were faster; METR now calls this out of date.[26] Its 2026 follow-up was "an unreliable signal".[27]

DORA 2025. A survey of nearly 5,000 professionals linked AI adoption to higher delivery throughput but lower stability; 30% had little or no trust in AI-generated code.[28]

None tested today's agent mode, cloud agent or code review. Our review of workplace AI studies compares them, and our guide to measuring AI productivity shows how to count review time.

Documented limits and risks

• Errors and vulnerabilities: chat and agents "can produce incorrect or suboptimal code, including code that contains security vulnerabilities."[4]
• Variable cost: agent sessions and reviews are metered by tokens; flex credits can change.[8][5]
• Training by default on individual plans, including private code in use.[19]
• Partial exclusion: none in agent or edit modes.[17]
• Firewall gaps: not "a comprehensive security solution".[14]
• Narrow cloud-agent scope: one repository, 59 minutes, GitHub-hosted code only.[4]
• Narrow indemnity: organisation plans, unmodified output, a disputed filter condition.[6][20][23]
• Rate limits can slow you at busy times, separately from credits.[29]

Who it suits, and who should skip it

It suits teams whose code and reviews already live on GitHub, since the cloud agent works inside branch protection, required approvals and Actions. It suits organisations that need Business or Enterprise terms: no training, admin policies, content exclusion and indemnity. For individuals, Pro is a low-cost start.

Skip it, or limit it, if your code is on GitLab, Bitbucket or GitHub Enterprise Server.[4][7] Skip individual plans for client code unless training is off and your contract allows it. If you need hard per-task cost ceilings, pilot with budgets first. If you want an agent-first editor rather than an extension, see our Cursor vs. GitHub Copilot comparison.

How to test it yourself

1. Set your controls first: training opt-out, public-code filter and, for organisations, content exclusion.
2. Pick three defined tasks: a failing test, a reproducible bug, a small feature with acceptance tests.
3. Try each surface separately on a throwaway branch: completions, agent mode, the cloud agent from an issue.
4. Log every command agent mode proposes, and how many you rejected.
5. Read the session log and full diff, then run the whole test suite yourself.
6. Run code review on Lite and Balanced and compare it with a human reviewer.
7. Record credits per task, plus Actions minutes for reviews.
8. Time the whole job, review included, against the same work without AI.

A worked example (hypothetical)

This scenario is invented to show how the documented controls fit together. No tool was run, and no outputs are reported.

Priya runs a five-person team on Copilot Business. Chores go to the cloud agent: a dependency upgrade becomes an issue with clear acceptance criteria, assigned to Copilot, with the firewall left on its default allowlist. Payments code stays local: agent mode with auto-run off, every command approved, and no secrets in the repository, because content exclusion does not cover agent mode. Review is layered: Copilot reviews every pull request, Copilot approvals stay off, and a human approves every merge. A monthly credit budget caps spending.

Her test for each change: could a teammate see what changed, why and at what cost, and roll it back in one step?

Questions readers commonly ask

What happened to premium requests?

GitHub AI Credits replaced them on 1 June 2026. Only annual Pro and Pro+ subscribers keep premium requests, until their plan expires.[10][9] The Free plan has 2,000 completions a month and limited chat and agent use, but no cloud agent.[7]

Does Copilot train on my code?

On Free, Pro, Pro+ and Max, GitHub may use your interactions, including code snippets, from 24 April 2026 unless you opt out. Business and Enterprise data is not used.[18][19][6]

Is Copilot's code safe to merge?

Not without review. GitHub says its output can contain vulnerabilities, and its cloud-agent rules require a human to approve and merge.[4][13] Our research on what agent platforms disclose about permissions compares vendors' controls.

Who is liable if a suggestion copies someone's code?

On Business and Enterprise, GitHub offers to defend claims over unmodified output; on individual plans, you carry the risk.[6][20] Check your contract for the current filter condition.

The next step

If you are deciding between Copilot and the best-known agent-first editor, read our Cursor vs. GitHub Copilot comparison next.

Sources and evidence note

Product descriptions, defaults, limits, prices and data terms come from GitHub's plans page, GitHub Docs, the GitHub Blog and Changelog, GitHub's customer terms and Microsoft's Customer Copyright Commitment page. They describe intended behaviour, not tested performance. Prices are US list prices as displayed on 16 October 2026. Independent evidence comes from Peng et al. (arXiv), Cui et al. (author-hosted version of the Management Science paper), METR's two posts and Google Cloud's DORA 2025 announcement, each read in full. The worked example is invented. All pages were accessed on 16 October 2026.

Related reading

• Cursor review: coding workflow and review effort
• Claude Code review: from task to reviewed change