Here’s a good example mocking the technical interview.
And here’s one with numbers, proving no one knows what they’re doing anyway.
One step forward…
At Keepsafe, we’re always eager to improve. So when we read about how resumes don’t really provide a useful signal as to whether or not someone is a good developer, we decided to let candidates apply without resumes. We hired our Tech Lead this way, so we call that success!
But then there was the next step: interviewing. An interview process is supposed to determine whether the candidate has the skills required to do the job and will be able to operate in the company culture. In Keepsafe’s case, that means assessing iOS/Android app developers. (We’re hiring app devs!)
Inspired by questions from experienced hiring managers, we used to ask a set of questions on data structures, application architecture and implementing simple algorithms. We observed that many candidates didn’t answer these questions to our satisfaction — including plenty who had clearly worked on teams that successfully shipped quality mobile apps.
Were we testing for the wrong things?
And another…
We took a month to evaluate our process and asked ourselves the following questions:
- What were we struggling with?
- Where did we believe the existing way of interviewing was falling short?
- What things should we ask more about; technically and for culture fit?
- What did we believe a good technical interview should accomplish?
- What skills were we testing for?
- How could we refine the process to produce the best experience for the candidate and team?
- How would we evaluate candidates?
As we discussed, we quickly realized that we didn’t have a clear common understanding of these requirements. We talked about what we wanted to drill into during the technical interview. We considered the personality traits we sought in candidates. We determined what to filter out during the screening process. Many questions surfaced, and this reassessment allowed everyone to contribute to the solution.
What we’re looking for in app developers
We challenged ourselves to list out the skills, attitudes and knowledge we believe are necessary in different technical roles for success at Keepsafe. As we discussed, we quickly found out that we didn’t have a clear, common understanding of these requirements. First, we analyzed our team’s past and present composition and acknowledged the skill-set and behaviors that were productive and those that didn’t serve the team. Next, we narrowed down criteria for a successful team member at Keepsafe.
Here’s the list of abilities we’re looking for:
• Designs solutions taking use cases and requirement evolution into account
• Communicates clearly
• Reasons logically
• Thinks through edge cases
• Weighs impact over perfection
• Understands iOS/Android platform and limitations
• Learns quickly
• Open to feedback
• Team player
We compared how well suited our questions were to an assessment of these skills and — surprise — they didn’t match up very well. Our questions were designed with an abstract “able programmer” in mind. And while there are candidates that may solve our problem set, that didn’t mean they have the characteristics that make people thrive at Keepsafe.
What’s more, when we looked closely at the skills we believed important, it became clear that they are not easily assessed by solving a technical problem alone.
We decided to not only change the types of questions we ask, but to also define the kinds of answers to look for and how to interpret them. While the new process might look much like the old, how we attempt to evaluate candidates abilities is very different now.
Our interview process
Our phone screen is now centered on literacy. We want to know if you can write code. Can you solve a small, simple problem in a limited amount of time. For example, we may ask a candidate to write a function that reverses a string. We expect everyone to be able to solve this problem, but now we consider how fast and how much hesitation candidates have or questions the candidates raise.
When we invite candidates on-site, everything is centered around a small app project that the candidate works on independently.
A small project removes format limitations usually found in technical interviews. No whiteboards or other unrealistic environments used. This is about producing an app on the platform the candidate works on most (iOS or Android) and using the tools and environments they will have to employ on the job.
More importantly, it provides enough specific implementation, driven by the candidate, that allows us to assess the abilities listed above. What better way to assess clear communication and reasoning than talk about the project’s implementation? We also discuss trade-off decisions made specifically for this project and can glean whether a candidate can prioritize. During the process, we can see how feedback is handled during a discussion of the project. We like to move quickly and want to make sure the candidate also will advance in Keepsafe’s environment.
We’ve tested this new process for two months and have found that it truly addresses the issues we uncovered with our earlier interviews. Quizzing someone with random questions about their knowledge doesn’t get at the reason he/she wants to build cool stuff and join your company. Our revised interview process tries to create an environment more conducive to figuring this out. We’re glad to finally have an interview approach that reflects what we’re looking for and what we care about and also gives candidates a better interview experience.
