Draft post about Leetcode

Author: john-kurkowski

src/content/posts/2025-04-03-still-measuring-devs-with-leetcode.mdx

@@ -0,0 +1,141 @@
+---
+layout: ../../components/layouts/post.astro
+title: 'Still Measuring Devs With Leetcode?'
+categories: [tech]
+tags: [engineering, programming]
+description: TODO
+---
+
+I'm disappointed to find tech interviews are still asking Leetcode questions.
+Companies haven't listened to old advice on competitive hiring like DHH's
+[Hiring programmers with a take-home test] or Matasano Security's [the Hiring
+Post][The Hiring Post].
+
+> Years from now, we'll look back at the 2015 developer interview as an
+> anachronism, akin to hiring an orchestra cellist with a personality test and a
+> quiz about music theory rather than a blind audition.
+
+Even new companies today with the chance of a fresh, competitive approach are
+still asking Leetcode questions, probably because the FAANG-equivalent companies
+of decades ago were asking Leetcode questions, and a few FAANGs still are. A
+100-employee startup primed me with an outline of their upcoming interview steps
+with a link to a sample [leetcode.com][Leetcode] question to expect, boasting,
+"Our interview process is similar to Meta's process."
+
+> The most dangerous phrase a \[manager\] can use is "We've always done it that
+> way."
+>
+> —Grace Hopper
+
+For the last 10 years of my tech career, I didn't have to interview for jobs:
+behold, the power of referral! Not interviewing meant no Leetcode questions. The
+last time I had to answer a Leetcode question, the name Leetcode didn't exist,
+although the style of questions was decades old.
+
+I used to be pretty good at Leetcode questions. In my college computer science
+degree, senior year, I went through 20 interview rounds with 12 different
+companies. Only the final company accepted me. I didn't get better at
+programming during that time, although that would have been a good use of my
+time. Rather, in that time, I just got better at interviewing, and a lot of it
+was practicing Leetcode-style questions, via [the first edition of Programming
+Interviews Exposed][Programming Interviews Exposed]. Having little professional
+experience, I accepted there was no other practice for companies to trust me.
+
+Within a few years in the industry, it was my turn to be the interviewer, which
+is when I got the perspective, the empathy for candidates. I wanted to shake
+free of the cult of Leetcode hazing. It was an inaccurate and unfair way to
+measure. I sought the advice of the posts at the beginning of this article. What
+if we measured actual developer skill: can the candidate do the job well, rather
+than can the candidate do Leetcode well? What if a candidate was a great fit for
+the job, but the specialized simulacrum and pressure cooker of live-coding
+Leetcode interviews excluded such a candidate?
+
+Don't get me wrong, improving the situation was easier said than done. Hiring
+developers is hard. My teams and I iterated for years on structuring our
+interviews—a structured interview is fairer but sadly not the friendliest
+candidate experience, to blankly read from a script—and developing relevant,
+realistic, self-contained assessment projects for candidates. For take-home
+tests, we didn't pay candidates for their time, which was tantamount to free
+labor and exclusionary in its own regard. We also lost a great candidate who was
+a prolific OSS contributor to our company's tech stack, because they
+diplomatically pointed to their public contributions as evidence of their skill
+and collaboration style, declined the take-home test as something they couldn't
+afford (what were all their free labor OSS contributions for, after all), and
+our team didn't have the capacity to develop and maintain a "fair" backup
+measurement for the special case candidate. While our take-home test was a
+promising hiring solution, we still could've done better.
+
+I took a career break from tech the last couple years. When I started the tech
+job hunt 12 months ago, wanting to try my old full stack web developer job in a
+new domain, I no longer had the power of a referral. I program every day. I know
+I can do the job. If an interviewer talks to me, I can demonstrate I know how to
+program, I'm emotionally intelligent, and that their team would want to work
+with someone friendly like me, right?
+
+Much to my chagrin, almost every company I've interviewed with this year still
+uses Leetcode-style interviews. If a company even gives the heads up of what to
+expect in an interview, don't necessary believe, "Our live code assessment uses
+real-world problems." In 9/10 of my interviews this year, the interview was
+still Leetcode in a trenchcoat.
+
+I was rusty at these live coding interviews, because in my 14 year career,
+Leetcode and what it teaches _never came up_ on the job. Software development is
+not typically done in a eye-movement surveilled, time-constrained screen-share
+with 1-2 others silently breathing down your neck, letting you squirm, maybe a
+modicum of hints that make sense to the interviewers because they've seen the
+exact problem before. And there's no time left for me to ask questions. The
+interview for mutual fit turns out not to be a 2-way street. It's dehumanizing
+for the experienced dev.
+
+When I'm rejected for a job, in my spirit of continual improvement, I ask for
+feedback why, and companies are always cagey. To cover their ass. So I'm not
+100% certain all the reasons I was rejected over the last 12 months. I might
+have a worse personality than I think. Or maybe a member of the hiring panel was
+underslept and cranky. Out of my control. All I know is, the interview process
+ending always _correlated_ after I failed to finish a Leetcode step of the
+interview, i.e. not finishing the coding puzzle prompt within 45 minutes.
+
+Roy Lee, who developed [a tool to cheat on these sort of
+interviews][Hard Fork: AI Action Plans], summarized what I was _supposed_ to do.
+
+> You're given 45 minutes and the task here is to _have_ seen the problem
+> before, solve the problem, and be able to regurgitate the memorized solution,
+> without acting like you have seen the problem before.
+
+In 2025's competitive tech hiring market, where laid-off tech workers outnumber
+available positions, some candidates spend months grinding leetcode.com to reach
+the top 1%, while others use cheating software like Roy Lee's. Meanwhile,
+experienced developers who could do the actual job well but haven't memorized
+these puzzles struggle to complete them under pressure. The unqualified
+candidate aces the test. The qualified candidate fails to finish in time. As the
+interviewer, which candidate would you advance?
+
+Next time I'm on the job hunt, I hope to cross-reference my applications with
+the [Hiring Without Whiteboards][Hiring Without Whiteboards] list. Although in
+my 12 months of the recent job hunt, I made hundreds of applications and
+received human responses from less than 10%. Given the odds, I felt I couldn't
+afford to be choosy to use this list.
+
+[Hiring programmers with a take-home test]:
+  https://signalvnoise.com/svn3/hiring-programmers-with-a-take-home-test/
+[The Hiring Post]: https://sockpuppet.org/blog/2015/03/06/the-hiring-post/
+[LeetCode]: https://leetcode.com
+[Programming Interviews Exposed]:
+  https://archive.org/details/programminginter0000mong
+[Hard Fork: AI Action Plans]:
+  https://www.nytimes.com/2025/03/21/podcasts/hardfork-ai-action-plans.html
+[Hiring Without Whiteboards]:
+  https://github.com/poteto/hiring-without-whiteboards
+
+## Unused
+
+While they assume just because a company is famous their interview process is
+solid and adaptable to a SMB, I do appreciate they were honest about the
+inclusion of Leetcode.
+
+Cheating aside, AI-assisted coding tools are really great at Leetcode problems.
+It's another hint, if it's easily solvable by computers, maybe it's not a great
+test of an engineer?
+
+- AI Cheating in interviews (LinkedIn email)
+- https://newsletter.pragmaticengineer.com/p/the-reality-of-tech-interviews