Hiring in the Age of AI

I’ve been in this industry long enough to watch hiring trends come and go. There was a time when interviews were all about clever puzzles. Later, it was all about whiteboards. Then came the era of algorithm questions, and after that, take-home assignments. The process keeps evolving, yet the underlying goal of finding people who can actually build things remains the same.

There’s always been a gap between what companies test for and what actually matters once you’re hired. Sometimes the gap is wide, sometimes it’s narrow, but it’s always there. The industry keeps inventing new ways to measure programmers, yet the things that make someone good at the job rarely fit neatly into a test.

Right now, the gap is wider than ever. LLMs have transformed how we work, but hiring hasn’t caught up in most places. Companies still interview as if nothing’s changed, asking the same old questions, looking for the same signals. It’s as if they’re trying to hire for 2019, a world that no longer exists.

Signals That Don’t Matter Anymore

Some signals that used to matter are now mostly noise.

Memorizing Algorithms

Being able to implement quicksort from memory doesn’t predict much when the person will have an LLM that can write sorting algorithms faster than they can think. These interviews reward the wrong mental model entirely. I once watched a candidate ace a whiteboard quicksort, then reach for it blindly in production against nearly sorted data and ship something slower than the built in sort.

They select for people who approach problems by retrieving pre-stored solutions rather than reasoning from first principles. In the LLM era, the valuable skill is knowing when quicksort is the wrong choice, not being able to implement it perfectly. The person who memorized the algorithm might not notice when the data has special properties that make a different approach better.

Language Trivia

Knowing the exact syntax for language features is less important when AI can fill in the gaps. Syntax memorization was always a proxy for something else: how much time someone had spent in the language ecosystem.

But now that expertise often points in the wrong direction. Someone who spent years mastering Ruby’s method_missing might struggle to adapt when the team switches to Go, while someone with great programming fundamentals can pick up any language with AI assistance. Deep specialization in one tool can actually become a liability when the landscape shifts quickly.

The real shift is that deep language knowledge matters less when LLMs can translate between languages almost fluently. The person who spent years mastering Python’s metaclasses isn’t necessarily more valuable than someone who understands systems thinking and can quickly adapt to any language. Domain knowledge now decays faster than it can be accumulated through traditional experience.

Coding Without Tools

The insistence on coding without internet access, without autocomplete, without any AI assistance creates an artificial constraint that selects for the wrong skills. Real work happens with tools. The constraint doesn’t make the test more pure. It makes it test for something completely different from the job.

Someone who’s excellent at whiteboard coding might struggle with the messy reality of debugging a distributed system, while someone who’s mediocre at whiteboard coding might be incredibly effective at orchestrating AI tools to solve complex problems.

System Design Numbers

Every system design trivia, like memorizing that Redis can handle 100k ops/sec or that Postgres has certain transaction isolation guarantees, matters less when current benchmarks are instantly available. But this goes beyond just looking up numbers.

The deeper issue is that these memorized facts often encode assumptions about how systems should be built. Someone who’s internalized that “databases are slow” might over-engineer around that constraint, not realizing that modern tooling has changed the performance characteristics entirely. The memorized wisdom becomes technical debt in their mental model.

I still remember a review where a senior engineer insisted on sharding Cassandra for a workload a single regional Postgres cluster handled at half the latency.

Signals That Matter Today

The signals we use to judge programmers are often proxies for deeper qualities.

Years ago, knowing obscure algorithms or being able to recite language trivia was a signal that someone cared enough to study. Now, with LLMs at everyone’s fingertips, those signals have faded. What matters isn’t whether you can recall the details of a sorting algorithm, but whether you know when to reach for one, and whether you can tell if the answer you get makes sense.

Here are the qualities that might be the most reliable signals for someone who can actually build things.

Good Judgment

If you look at what actually matters now, it’s judgment. Not just the ability to write code, but the ability to tell when code is good, and when it’s nonsense. LLMs will confidently hand you plausible-looking answers, and sometimes they’re right, but sometimes they’re not. The hard part is knowing which is which.

You can’t really test for judgment directly. If you ask someone “do you have good judgment?” they’ll say yes, and so would you. The only way to see it is to watch people work. Give them a real problem, let them use whatever tools they want, and see what happens. Do they blindly accept the first thing the LLM spits out, or do they notice when something’s off? Can they explain why a solution is wrong, or do they just shrug and try something else?

You can learn a lot from watching someone debug a piece of code that almost works but doesn’t quite.

Good judgment is learned by doing: you must build a lot of things that are slightly too hard, fail at them, and then notice why they broke. Each repair teaches which assumptions mattered and which were noise. Read widely and talk to people with different perspectives. Books and varied experience give you the vocabulary to recognize recurring patterns and the humility to distrust any single model.

Curiosity

We are experiencing a true paradigm shift in software engineering right now. Engineers are adapting to it in various ways. I’m observing two kinds:

One kind adopted LLMs wholeheartedly. They’ve integrated them into their daily workflow, from sketching out designs to debugging deployment issues. They experiment with different models, figure out what works and what doesn’t, and keep pushing the boundaries.

The other kind tried ChatGPT once in 2023, asked it to solve some impossible problem, got a bad answer, and concluded that LLMs are useless. They’ve been frozen in that moment ever since, missing what might be the biggest shift in how software gets built since the invention of high-level languages.

The gap between these two groups is widening every day.

Curiosity is the trait that separates people who survive a paradigm shift from those who merely survive it briefly. It is a habit of the mind: making small experiments, trying out new things, and following answers until you find a solution or clear signals what is yet not possible.

Systematic Thinking

Systematic thinking keeps an LLM from feeling like luck. You break problems into steps, write down what you know, list what you need, and decide how you will check the answer. That habit turns guesswork into work.

Engineers with this streak treat a prompt the way they treat a function. They define the goal, note the constraints, and sketch how to confirm the output. They run the loop, look at the result, and test it. When it fails, they change one variable at a time.

You can spot systematic thinking when someone takes an open-ended task and leaves breadcrumbs. They say the plan out loud, capture notes you can inspect later, and mark the assumptions they could not verify. They measure whenever they can, and they follow up when measurement is impossible.

Hiring for this means giving candidates a messy problem and letting them use their tools. Watch for a repeatable process. See whether they prove the answer before they ship it. That pattern scales even while the ground keeps shifting.

A Framework for Hiring

Given all this, what might an interview process look like? I’m betting on the following format:

Round 1: Initial screening (curiosity baseline)
Round 2: Build together (judgment in action)
Round 3: System review (systematic thinking + curiosity)
Round 4: Experience deep dive (learning velocity)

Each round leans on one of the core signals. Taken together, they trace how a person exercises judgment with modern tools, how they impose structure on messy systems, and how quickly they absorb new ideas.

Round 1: Initial screening (curiosity baseline)

A brief conversation to understand their background and motivations. No technical questions yet, just getting a sense of who they are, what they’ve built, and what kind of problems interest them. This is also where you explain the role and see if there’s mutual interest. Keep it short, maybe 30 minutes.

This part remains largely the same as before; no big changes needed here. The goal is still to have a human conversation and establish basic fit.

You can still learn something about curiosity: the strongest candidates turn the screening around and ask for context, constraints, and why the role exists. People who treat the call like a formality rarely switch gears later.

HR typically owns this round, so equip them with a rubric that maps to the same three signals. Have them ask how the candidate keeps skills current, what tools they reach for when a project shifts under them, and whether they experiment with AI on their own. Ask them to capture quotes and specific examples instead of gut feelings so the hiring manager reviews raw signal, not folklore.

Round 2: Build together (judgment in action)

Here the process has to change. Instead of whiteboard coding or algorithm puzzles, give them a real problem that’s relevant to your work. Let them use whatever tools they normally would: their IDE, Google, LLMs, documentation, whatever.

The key is to pick a problem that’s just hard enough to be interesting but not so hard that it takes hours. Maybe it’s debugging a piece of code that has a subtle bug, or implementing a small feature that requires understanding how different parts of a system fit together.

What you’re looking for isn’t whether they get the perfect answer. You’re watching how they think through the problem. Do they ask good questions? When they get stuck, do they know how to get unstuck? When they use an LLM, do they understand what it’s telling them? Can they explain their reasoning?

Judgment shows up fast here. You’ll see whether they blindly accept LLM suggestions or carefully evaluate them, whether they can spot when something looks wrong, and how they validate their solutions.

This is also a great opportunity to see how they collaborate. You as the interviewer should collaborate on the solution, not just silently judge. See how they communicate about code, how they handle feedback, and whether they’re someone you’d want to pair with on a difficult problem.

Keep this to about an hour. If they need more time, that’s probably a signal in itself.

Round 3: System review (systematic thinking + curiosity)

Give them a system design or codebase to review. This could be something they’ve built before, or a simplified version of something your team has built. Ask them to walk through it and explain how it works, what they’d change, and what questions they’d ask if they had to maintain it.

This round puts systematic thinking under the microscope. Can they map a new system quickly? Do they understand the tradeoffs? When they don’t know something, do they ask thoughtful questions or make assumptions?

Curiosity should be obvious too. Do they dive deeper when something seems interesting? Do they ask about edge cases, failure modes, or how the system evolved over time? Or do they just skim the surface and move on?

You also see how they think about maintenance and evolution. Building software is one thing, but most of our time is spent working with systems that already exist. Someone who can quickly understand existing code and reason about how to improve it is incredibly valuable.

The conversation should feel collaborative, not interrogative. You’re exploring the system together, not testing whether they know some specific fact. This approach tends to bring out people’s natural curiosity and gives you a much better sense of how they’d actually work on your team.

Keep this to about an hour. If they’re engaged and asking good questions, that’s usually a better signal than having all the “right” answers.

Round 4: Experience deep dive (learning velocity)

The last round finishes the loop by probing curiosity and learning velocity. Instead of asking hypothetical questions, dig into their actual experience. Pick something they’ve built that they’re proud of, and go deep.

How did they approach problems they’d never solved before? When they got stuck, what did they do? What would they do differently if they built it again? The best engineers have stories about projects that went sideways, mistakes that taught them something important, or moments when they had to learn an entire domain just to ship a feature.

What you’re really measuring is learning velocity, how quickly someone can go from not knowing something to being productive with it. In a world where the tools and best practices change every few years, this might be the most important trait of all.

You’ll also get a sense of their curiosity from these stories. Do they stop when something works, or do they keep digging to understand why? When they encounter new technologies, do they just cargo-cult the tutorials, or do they understand the principles underneath?

The conversation should feel more like two engineers sharing war stories than an interview. Ask about the hardest bug they’ve debugged, the most complex system they’ve had to understand, or the biggest technical mistake they’ve made. Their answers will tell you more about their potential than any coding exercise could.

Keep this to about 45 minutes. The goal isn’t to cover everything they’ve ever built, but to understand how they think about learning and problem-solving through the lens of real experience.

Closing the Loop

Most companies still screen for a world where the hardest part of programming was remembering syntax. Today the job is steering the firehose: noticing when the machine is wrong, stitching together incomplete answers, and staying curious enough to keep exploring.

Pilot this interview loop on the next three candidates you meet. Record what you learn in each round, then compare it with the notes from whatever process you run today. If you end up with clearer signals, keep the experiment and strip out the theater. Either way you stop hiring for a world that no longer exists.