340 of the Top 1,000 GitHub Repos Ship an AGENTS.md. I Linted Them.

I checked the 1,000 most-starred public repositories on GitHub. 340 of them have an AGENTS.md at the root. That’s a lot of adoption for a file format that barely existed two years ago, and it deserves more scrutiny than it’s getting, because every line in it gets loaded into every agent session.

So I wrote a linter, ran it over all of them and read the results. Of the 340, 321 could be analyzed; the rest were one-line pointers to another file or failed to clone.

Measure	Result
Median length	112 lines
90th percentile	370 lines
Median size	roughly 2,000 tokens
90th percentile size	roughly 6,600 tokens
Over Claude Code’s 200-line target	29.6%
Contains a repository overview or directory tree	40.8%
Contains an instruction nobody could verify (“follow best practices”)	17.1%
Has no test, lint or build command at all	14.6%
Has at least one confirmed stale path	5.9%

Token sizes use characters divided by four, which I’ve argued is a rough tool. It’s fine for the median of a prose file.

Before judging those numbers, it helps to know what the research says these files actually do.

The two studies, and why they seem to disagree

Lulla et al. ran agents on 124 pull requests across 10 repositories, with and without an AGENTS.md. With the file, median runtime dropped 28.64% and output tokens dropped 16.58%, with comparable task completion.

Gloaguen et al. at ETH Zurich and LogicStar.ai tested several coding agents on 300 SWE-bench Lite tasks and 138 instances from repositories that already had context files. Their full results are less flattering:

Context file	Effect on task success	Effect on cost
Developer-written	about +4% on average	up to +19%
LLM-generated	a small loss (0.5% and 2% across the two benchmarks)	about +20% to +23%

Those two results aren’t really in conflict. One measured how fast agents finished, the other whether they got the task right and what it cost overall. Agents with a context file do more: more testing, more exploration, more reasoning tokens. Sometimes that’s useful and sometimes it’s expensive diligence on a task that didn’t need it.

Three findings from the ETH paper should change how you write these files:

• Agents follow the instructions. When a file mentioned a specific tool, agents used it. These files are effective at steering behaviour, which is exactly why a bad one costs you.
• Overviews don’t work. Context files, “even developer-provided ones, are not effective at providing a repository overview.” They didn’t reduce the steps needed to reach the files a fix had to change.
• Generated files are the weak version. The authors suggest omitting LLM-generated files for now, with one honest exception: in a repository with its documentation removed, generated files helped.

Now look at my table again. 40.8% of these files contain a repository overview or directory tree. The AGENTS.md site itself lists “Project overview” among popular sections. The evidence says that section is the one to cut.

What I’d put in the file

The test I use: could the agent find this out by looking? If yes, leave it out. An agent can run ls. It can’t discover that your integration tests need a particular flag.

Keep:

• The exact commands to test, lint and build, in the form that works. 14.6% of these files don’t have any.
• Non-standard tooling. “Use uv, not pip” is the example that works in the research.
• Conventions a formatter can’t enforce, and the reason for each.
• The traps. “The users table is replicated, so migrations go through X” is worth its tokens.

Cut:

• Directory trees and architecture tours.
• Anything repeated from the README.
• Adjectives. “Write clean, maintainable code” gives the model nothing to act on.
• Anything that must always happen. Claude Code’s documentation is clear that instructions are context and not enforced configuration. Rules that can’t be broken belong in hooks and CI.

Keep it short, too. The same documentation targets under 200 lines per file and says that longer files “consume more context and reduce adherence.” At the 90th percentile, these files cost about 6,600 tokens on every session before the agent has read your request. That’s a context budget decision somebody made without noticing.

Why my linter was mostly wrong

Building the tool was more instructive than the results, so here’s the honest version.

My first pass reported dead paths in 49.2% of files. I nearly wrote that number down. Then I read a random sample, and it was garbage: bare extensions like .js counted as files, gitignored build output, placeholder paths like src/foo/bar.py, and “make sure” parsed as a make target with the argument sure.

After tightening the rules the figure dropped to 15.3%. I read 50 random findings, and only 17 were real. The surprise was the rest. These files are full of sentences like:

apps/legacy has been removed; do not restore it.

An agent instruction file legitimately names things that don’t exist. That’s a warning to the agent, and it’s a false alarm to a linter. Add generated files, private submodules, other repositories and optional paths (“if it exists”), and static checking gets hard fast.

After a third pass, which skips any line where the prose says a path is absent, 44 findings remained across 33 repositories. I read all 44 against the line that produced them:

• 20 were real stale references, in 19 repositories, or 5.9% of files. Things like a referenced workflow or test file that isn’t in the tree.
• 24 were still false alarms.

So the honest precision of this linter is 45%, after three rounds of work. It exits zero by default and only fails a build with --strict, because a tool that’s wrong half the time shouldn’t block your merge. It’s a list to read, and 5.9% is a floor: it only counts stale references my rules could see at all.

I’m not naming the repositories. The pattern is the interesting part, and each of them is one commit from fixing it.

What I’d do this week

1. Delete the directory tree.
2. Run your own instructions from a clean checkout. Every command in the file should work as written.
3. Grep your file for “best practices”, “clean code” and “be careful”.
4. If an LLM generated it, read every line and keep only what passes the could-it-find-this-out test.
5. When an agent makes the same mistake twice, add one line. That’s how the good ones seem to grow.

Limits

The most-starred repositories are not typical repositories. A root-level scan misses nested files. “Overview” detection is a heuristic on headings and tree characters, and the size target is one vendor’s guidance. The studies test specific agents and benchmarks, and agents have moved on since. I’m reading both as directional.

Linter, scan and audit counts: ajinb/agentsmd-lint. It’s standard-library Python, so python agentsmd_lint.py . is all it takes.