I recently shared on Twitter that I’m working on a book on machine learning interviews. It’s written with the candidates in mind, but hiring managers who saw the early drafts have told me that they find the book useful to learn how other companies are hiring and rethink their own process.

In November 2019, I gave a guest lecture at the Full Stack Deep Learning Bootcamp on the topic. Here is the link to the slides: “Machine learning interviews: Lessons from both sides”.

I also open-sourced the first draft of Chapter 9: Machine Learning Systems Design here.

A lot of people have asked me for more details about the book, so I decided to post the book’s introduction here. This is an early draft but it should give you an idea of what the book is about. To be notified when the book is out, you can follow me on Twitter or sign up for my mailing list.

If you’d like to share with me your interview experience or want me to take a look at your hiring pipeline, please shoot me an email (see footnote).

Chapter 1: Introduction

If you’ve picked up this book because you want a job in machine learning – whether as a machine learning engineer, a framework engineer, or a research scientist – but don’t know where to start, what to expect, or how to prepare for interviews, you’re not alone. Despite the aggressive growth in the number of machine learning related jobs and the number of people applying for these jobs, there is little public consensus on machine learning interviews.

According to LinkedIn and Indeed, machine learning jobs are among the fastest-growing and highest-paid jobs in the US, with the average base salary 30% higher than the already very high average base salary of a full-stack developer. Yet, because the use of machine learning in the industry is relatively new, companies are still figuring out what skills they want to see in candidates and how to evaluate those skills. This makes it hard for candidates to know what to expect during their interviews and to prepare effectively.

One of my students, after asking his recruiter about the scope of his upcoming interviews, was told: “Everything.” When I ask my friends or search online for preparation tips, the answers I get are along the lines of: “Just do a lot of coding challenges and read all the popular machine learning books.”

About This Book

Over the last two years, I’ve helped many friends and students prepare for their machine learning interviews at both big companies, including the good old FAANG (Facebook, Amazon, Apple, Netflix, Google), and startups that have raised from seed round to series D. I give them mock interviews and take notes of the questions they were asked in the real interviews as well as the processes they went through. I have also talked to founders and hiring managers at those companies about the challenges of hiring machine learning talents and where their pipelines succeed or fail at addressing those challenges.

At NVIDIA, I’m actively involved in the hiring process, having taken steps from cold emailing candidates whose work I love, screening resumes, doing exploratory and technical interviews, debating whether or not to hire a candidate, to trying to convince a candidate to choose us over competitive offers. I’ve also consulted several startups on their machine learning hiring pipelines.

I myself have got offers for machine learning roles at Google, NVIDIA, Snap, Netflix, and Primer. I have dropped out of the interviewing processes at several other big companies because the timing wasn’t right, the role wasn’t suitable, and/or the process was just a pain. I have also been rejected.

This book is the result of the collective wisdom of many people who have sat on the both sides of the table and who have spent a lot of time thinking about the hiring process. It was written with the candidates in mind, but hiring managers who saw the early drafts told me that they found the book helpful to learn how other companies are hiring, and to rethink their own process.

The book consists of four parts. The first part provides an overview of the machine learning interview process, what types of machine learning roles are available, what skills each role requires, what kinds of questions are often asked, and how to prepare for them. This part also explains the interviewers’ mindset and what kind of signals they look for.

The second part consists of 300 knowledge questions, each noted with its level of difficulty – interviews for more senior roles should expect harder questions – that cover important concepts and common misconceptions in machine learning.

The third part, my favorite, consists of 30 open-ended questions to test your ability to put together what you know to solve practical challenges. These questions test your problem solving skills as well as the extent of your experiences in implementing and deploying machine learning models. Some companies call them “machine learning systems design” questions. Almost all companies I’ve talked to ask at least a question of this type in their interview process, and they are the questions that candidates often find to be the hardest.

This book is not a replacement to machine learning textbooks nor a shortcut to game the interviews. Think of this book as a tool to consolidate your existing theoretical and practical knowledge in machine learning. The questions in this book can also help identify the areas of knowledge that you should focus on improving. Each topic is accompanied by resources that will help you strengthen your knowledge in that topic.

About The Questions

The questions in this book were selected out of thousands of questions. Most of them have been asked in actual interviews for machine learning roles at companies both big and small. To give you an even better sense of what an entire interview is like, the last part of the book features lists of questions for four full-length technical interviews at DeepMind, OpenAI, IBM, and a startup working on natural language processing.

You will find a few questions that are technically incorrect or ambiguous. This is on purpose. Sometimes, interviewers ask incorrect or ambiguous questions to see whether candidates will correct them, point out the edge cases, or ask for clarification. I hope that the answers provided make these questions clear. However, there is also a non-zero chance that the answer provided is simply wrong. If you believe you have found any errors in the answers, please let me know.

Even though most answers are sufficiently mathematically rigorous, I prioritize intuitive explanations over dense mathematical equations. I also prioritize knowing how to use something instead of just knowing the equations. For example, instead of asking a candidate to write out the exact algorithm for K-means clustering, I ask in what scenarios K-means doesn’t work. One doesn’t need to know machine learning to call sklearn.cluster.KMeans, but one needs to understand K-means to know when not to use it.

People often ask me: “Don’t you worry that candidates will just memorize the answers in this book and game the system?” First, I don’t encourage interviewers to ask the exact questions listed in this book, but I hope this book provides a framework for interviewers to distinguish good questions from bad ones. Second, there’s nothing wrong with memorizing something as long as that memorization is useful. The problem begins when memorization is either: impractical: candidates memorize something just so they can pass the interviews and never use that knowledge again. prioritized over understanding: candidates are unable to reuse and apply that knowledge in new situations.

For this book, I chose only the questions that I and many of my helpful colleagues deemed practical. For every concept, I ask the question: “Where in machine learning is it used?” If I can’t find a good answer after extensive research, the concept is discarded. For example, while I chose to include questions about inner product and outer product, I left out cross product. You can see the list of discarded questions in the list of “Bad questions” on our GitHub repository. Keep in mind that as the field expands, many concepts that people used to think of as not useful have proven otherwise. This book might leave out concepts that aren’t popular right now but might be all that AI researchers ever talk about in 2030.

This book has a small number of “what” questions – questions that ask candidates to explain particular concepts. They aren’t necessarily good interview questions, but are good questions to ask yourself while you’re preparing for interviews. Most of the questions in this book are about “why” and “how” things work, questions that test for understanding instead of rote memorization. The book doesn’t explain every concept, especially concepts that can easily be looked up online. If a piece of knowledge can be easily acquired, it isn’t worth testing for.

Please keep in mind that interviewing is a process, and questions aren’t asked in isolation. Your answer to each question is evaluated as part of your performance during the entire process. A candidate who claims to work with convolutional neural networks (CNNs) and fails to answer a question about convolutions is going to be evaluated differently from a candidate who doesn’t work with CNNs at all. Even though the book provides answers for all questions, interviewers often care more about the way you go about answering than the actual objective correctness of your answers.

Interviews are stressful, even more so when they are for your dream job. As someone who has been in your shoes, and might again be in your shoes in the future, I just want to tell you that it doesn’t have to be so bad. Each interview is a learning experience. An offer is great, but a rejection isn’t necessarily a bad thing, and is never the end of the world. I was pretty much rejected for every job when I began this process. Now I just get rejected at a less frequent rate. Keep on learning and improving. You’ve got this!