Anthropic manager stops coding interviews because Claude beats most candidates

zohaibahd

Posts: 1,002   +21
Staff
A hot potato: These days, plenty of software engineers spend weeks grinding LeetCode problems just to get through a single round of interviews. One engineering manager at Anthropic has now stopped using that round altogether. Claude, he says, beats most candidates at those puzzles anyway.

That manager is Felix Rieseberg. He's led engineering on Claude.ai and the Claude Code desktop app, among other things. Earlier this week, he posted on X that the coding interview no longer tells him anything useful. So he pulled it from his team's hiring loops this month.

In a longer blog post, Rieseberg says watching candidates define a problem and land on a neat solution used to be a weak-but-real predictor of how they'd do on his teams. But once Claude got good at those exercises, a small React challenge started telling him about as little as a quiz on assembly.

Anthropic's performance team has run into the same problem with its take-home test. Team lead Tristan Hume wrote in January that Claude Opus 4 outscored most applicants within the time limit. So he made the test harder. Then Opus 4.5 came along and matched even the strongest candidates. Hume's third version now leans on odd, puzzle-game-style problems that Opus 4.5 couldn't crack.

As for Rieseberg, he hasn't settled on a replacement yet. For now, he's scanning resumes and portfolios for small prototypes that show taste, empathy, and principled thinking. He also asked other managers to share anything better they've come up with. Google principal engineer Jaana Dogan replied on X that she'd rather watch candidates work with coding agents because doing so tells her more about their coding skills.

Most of Rieseberg's essay is aimed at junior engineers. Its title, "Hunger beats tenure," pretty much sums up the pitch. He argues that newer engineers can now outpace veterans since building things has gotten so cheap. As proof, he points to two fairly new Anthropic engineers who built a working graphical interface for Claude Code over a single weekend. The first public version shipped about eight days later.

Most talk around AI and new grads has been a lot gloomier, though the hiring data is honestly still mixed. Meanwhile, ZipRecruiter labor economist Nicole Bachaud told The Register that tech employers are still the most likely to test for skills.

In the company's October survey of 1,500 employers, 62.5 percent of those in the information sector required formal skills tests as part of the initial application process. Around 70 percent have also moved those assessments to proctored or in-person sessions to keep AI out.

Anthropic's own candidate guidance still asks applicants to keep Claude out of live interviews and take-home tests unless told otherwise. Rieseberg dropped the coding round only for his own team, too. So anyone applying elsewhere at the company will likely still face one.

Permalink to story:

 
When our team hires (and we are looking for data engineers, data scientists, AI/MLOps engineers, etc), we give a take home data science modeling project where they are given publicly available data and must come up with a model that predicts something (they are given a full brief on details and about a week to do it). We expect them to use AI for coding, and we don't really pay as much attention to code quality (unlike in the past). What we are looking for is how they present their work. Can they explain the decisions that either they or the AI made, can they interpret the results successfully, can they answer hypotheticals (such as if the data changed how would the model perform), etc. You can very quickly tell who is merely claiming experience on their resume versus who can not only use AI but bring something to the table themselves.

The only downside is that merely having a take home projects weeds out a lot of applicants because it is quite the commitment from an applicant beyond your standard interview. But it works extremely well, and it also works regardless of which skill level you are hiring for because you can calibrate your expectations and questions accordingly.
 
My curiosity demands an answer to this.
Will AI be able to create fix and reliably
pass information on what it codes to the
managers who by that point in time will
not know a single thing about coding?

If so, why do we even need to understand
what we are coding? But then, it will probably
be a huge dissapointment to the people who spent
years and a lot of money on best programming schools.

I am not a pessimist, I do not want to think that this will
turn out very bad for our country. But at the same time
I want to know what the most realistic future of AI in coding is.


 
Back