Anthropic ditched Git for its own version control system because Claude is writing too much code

Skye Jacobs

Posts: 2,278   +64
Staff
First look: Anthropic has replaced Git in its main code repository as AI-generated changes outpaced the infrastructure built to manage them. The new system, called "Scion," moves most source-control operations to servers, reducing the amount of data that agents and automated jobs must download.

Anthropic detailed the system in a recent engineering post, reporting that it averaged about 2,500 merged code changes a day in May. By September, that figure exceeded 30,000.

Anthropic says Scion uses just one two-hundredth as much CPU power and memory as the Git mirror infrastructure it replaced. Code checkouts for continuous integration (CI) and research jobs now take seconds instead of minutes. Anthropic says the change freed up 25% of the CPU capacity used by its CI infrastructure. A seven-person team built Scion and its supporting tools and completed the migration in five months.

Scion puts commits, merges, rebases and file-history queries on the server. The client tracks local edits, manages files and keeps a limited cache. It does not need to hold enough repository data to perform those operations independently.

That approach suits Anthropic's workload. Most clients are agents and automated jobs that stay online, so Git's offline support offers little benefit. Centralizing the work reduces local storage and synchronization, though it makes server performance and availability more important.

Jobs that need files can download selected directories through sparse checkouts or use virtualized checkouts, which retrieve files as they are read. Both approaches avoid fetching code a job will never use.

The size of Anthropic's repository helps explain the change. Its main-branch history approaches 100 GB, and a compressed working copy exceeds 25 GB. CI alone performs about 25 million checkouts each day, excluding developer environments, agent sandboxes and research jobs.

Downloading a full compressed copy for every CI checkout would generate more than 600 petabytes of daily traffic, according to the company. That estimate reflects what a naive implementation would require, not actual traffic.

Anthropic had already tried shallow clones, sparse checkouts, promisor remotes, snapshots in blob storage and internal mirrors. At one point, the company operated several hundred Git mirrors. The arrangement bought time but became harder to maintain as more changes entered the repository.

Scion records each change as a changeset in a write-ahead log. Each changeset lists files modified relative to its first parent and points to their compressed contents in blob storage. A Git commit instead points to a repository tree. Creating a changeset therefore takes resources proportional to the change, not the whole repository.

Higher merge volume has also forced Anthropic to rethink testing. The company says it could not effectively use a merge queue with a global repository lock at this scale. Hundreds or thousands of changes can merge while tests are still running.

Anthropic now relies more on automated testing after merges and before deployment. Conflicts still occur, and the company says Claude usually fixes them quickly. That puts more weight on catching problems after changes merge but before they are deployed.

Claude also helped build Scion. An early internal version of Claude Mythos Preview rewrote a Git-to-Scion ingestion pipeline. Anthropic says the replacement ran four times faster and used one-twentieth the memory. Engineers still set the system's constraints and rejected shortcuts, including attempts to scan commit logs when an agent should have queried the code review service.

Anthropic says that Scion would be difficult for other organizations to adopt because it requires custom review, search and CI tools, and that Git remains suitable for most repositories. But it sees room for existing platforms to adopt parts of the design, particularly server-side APIs that let agents work without repeatedly fetching a local copy of the code.

Permalink to story:

 
No testing, no review, no human can actually validate this amount of code. No surprise agent is not controllable, it is a slop code on slop code, and models are using this or other slop code to train itself. At some stage there will be strong reality check, and that will be painful.
 
No testing, no review, no human can actually validate this amount of code. No surprise agent is not controllable, it is a slop code on slop code, and models are using this or other slop code to train itself. At some stage there will be strong reality check, and that will be painful.
Impossible, we're constantly told that AAAAAAAAAAIIIIIIIIIIIIIIIIIII never makes mistakes, is Skynet and will end humanity any second now.

M$hite is the perfect personification of slop in, slop out. Replace all competent coders with 3x as many Actual Indians and Co-Pilot.
 
Well, you know the 10,000 monkeys with typewriters saying. Having 10,000 agents working 24/7 might in the end produce quality code.
 
Impossible, we're constantly told that AAAAAAAAAAIIIIIIIIIIIIIIIIIII never makes mistakes, is Skynet and will end humanity any second now.

M$hite is the perfect personification of slop in, slop out. Replace all competent coders with 3x as many Actual Indians and Co-Pilot.
IKR? These days when I see the news about a Microsoft patch Tuesday, the first thing that comes to mind is

"Oh, so now it's another week of news about what the latest patches broke"

And you might think it's an exaggeration, but it's not. First come the news about how it fixes something they broke previously:

Windows 11 KB5127648 fixes Group policy fails from buggy Admin Protection. Windows 11 KB5124010, KB5120998 have been causing Group Policy fails that Microsoft seems to have resolved with KB5127648.

Then within a few hours:

Windows 11 KB5127648 breaks WPA 802.1X PEAP Wi‑Fi auth

Then another few hours and:

Windows KB5127648 KB5127694 updates break Xbox sign in with 0x87DD0005 error

You get it? They break something with a patch, then push another patch to fix it, and the patch breaks other 2, new things. And these 3 news are from just the last 24 hours. And this is how it's been going for the past year or so.

It's beyond pathetic at this point. And we STILL have to listen all day to the mumbo-jumbo about how AI will solve all our problems, forever, heck it's sooo goood already they don't even dare to show it to us!
 
Back