The Problem
You just spent six months filling a senior role, and the candidate crushed your technical screen.
Three months in, they’re rebuilding a feature nobody asked for because they never checked what support tickets cost the business. The code is beautiful, but the solution is wrong.
ChatGPT now writes flawless explanations of SOLID principles. Candidates study LeetCode for a month to pass tests that predict nothing.
5 CTOs shared what they ask instead to reveal how engineers think when Stack Overflow doesn’t have the answer.
What Engineering Leaders Are Saying
Question 1: Explain This Like I’m in Fifth Grade
Reid L., Engineering Leader: “Have them identify the solid principles. Yeah, they could use AI in the background to get the answers. But to me it’s, OK, then great. Then go through and explain these to me like I’m an eighth grader. And if there’s longer pauses, then you know they’re actually typing into something in the background.”
The pause is the tell. Genuine understanding converts to simple language instantly. Reading ChatGPT’s output and translating it creates an 8-second gap.
Jason T., CTO: “I do a technical interview, but my technical interviews are still conversational. I ask things like, tell me how events work in React. Algorithms could be memorized. The coding tests all ask the same stuff. If you study for about a month, you can pass most of them. That doesn’t mean you’re a good engineer though.”
You’re testing if they can think through problems they’ve never seen and explain their reasoning while doing it.
Question 2: What Business Problem Did This Solve?
Reid L., Engineering Leader: “Can you convey to me the business value of what you’ve done? Like, great, you modernized the system. But what was the business value that came out of it? I like to see that people can connect the reasons behind why they’re doing the work. And they’re not just, you know, leave me alone. I’m sitting here with my headphones on cranking out code all day.”
Listen for the difference:
- Wrong answer: “I migrated to PostgreSQL and improved query performance by 40%.”
- Right answer: “Customer support was spending 3 hours daily reconciling duplicate orders. Now they spend 20 minutes, and we cut refund requests by half.”
Christopher M., CTO: “Most of the time when a technology or a project fails, it’s not that the technology couldn’t do the job. Most of the time it’s that we failed to understand what the client needed. It’s not always about the technology. It’s often about the people working on the technology.”
If they can’t connect technical decisions to business outcomes, they’ll optimize the wrong metrics brilliantly.
Question 3: Tell Me When You Were Wrong
Alan D., CEO: “We’re at the 99% level, we’re ready to sign. We just want to double check you know what you’re talking about, that you can listen. We’ve had people argue with us about technology. I learned this in school. That’s nice. I wrote that document 10 years before you went to university. When you’re arguing with someone trying to hire you, you’re telling them, I will never listen to you. I can’t pay for that.”
This detects unteachability before you pay for it. Someone who can’t admit being wrong in an interview will be impossible to coach during their sixth production incident.
Kaj P., CTO: “We have this hiring principle here, which we call the three Cs. Culture, capacity to master, and capabilities. Capacity to master is this ability to adapt and change and learn constantly. The capabilities you have today, will they suffice for the role? Understanding that the first two are probably the drivers for your success.”
Current capabilities depreciate. Learning capacity appreciates.
What to Watch For
- The 8-Second Pause: Time the gap between asking for a simple explanation and getting one. Three seconds is thinking. Eight seconds is typing into ChatGPT.
- Technical Metrics vs. Human Impact: “Reduced API latency by 200ms” is a feature. “Cut checkout abandonment from 12% to 4%” is a business outcome.
- Arguing During Interviews: If someone debates your technical approach while trying to get hired, they’re showing they won’t accept feedback once on payroll.
- Capacity Over Current Skills: Hire for learning ability over stack knowledge. Your stack changes. Learning capacity doesn’t.
Use This Right Now
Pick Reid’s eighth grader test for your next three interviews. Ask candidates to explain one technical concept from their resume as if teaching a fifth grader. Set a mental timer when you ask.
After three interviews, compare how candidates performed on this question versus your coding challenge. Ask yourself which signal better predicted the engineers on your team who consistently ship the right thing.
Add these three questions so you’re testing judgment alongside knowledge.
The 3 Interview Questions AI Can’t Answer, from Sonatafy’s Engineering Intelligence Hub. Insights drawn from over 160 CTO interviews on Software Leaders UNCENSORED. Practical tools for technical leaders navigating hiring and team building. Explore more at sonatafy.com/software-solution-directory/
