[E] Explain supervised, unsupervised, weakly supervised, semi-supervised, and active learning.
Empirical risk minimization.
[E] What’s the risk in empirical risk minimization?
[E] Why is it empirical?
[E] How do we minimize that risk?
[E] Occam's razor states that when the simple explanation and complex explanation both work equally well, the simple explanation is usually correct. How do we apply this principle in ML?
[E] What are the conditions that allowed deep learning to gain popularity in the last decade?
[M] If we have a wide NN and a deep NN with the same number of parameters, which one is more expressive and why?
[H] The Universal Approximation Theorem states that a neural network with 1 hidden layer can approximate any continuous function for inputs within a specific range. Then why can’t a simple neural network reach an arbitrarily small positive error?
[E] What are saddle points and local minima? Which are thought to cause more problems for training large NNs?
[E] What are the differences between parameters and hyperparameters?
[E] Why is hyperparameter tuning important?
[M] Explain algorithm for tuning hyperparameters.
Classification vs. regression.
[E] What makes a classification problem different from a regression problem?
[E] Can a classification problem be turned into a regression problem and vice versa?
Parametric vs. non-parametric methods.
[E] What’s the difference between parametric methods and non-parametric methods? Give an example of each method.
[H] When should we use one and when should we use the other?
[M] Why does ensembling independently trained models generally improve performance?
[M] Why does L1 regularization tend to lead to sparsity while L2 regularization pushes weights closer to 0?
[E] Why does an ML model’s performance degrade in production?
[M] What problems might we run into when deploying large machine learning models?
Your model performs really well on the test set but poorly in production.
[M] What are your hypotheses about the causes?
[H] How do you validate whether your hypotheses are correct?
[M] Imagine your hypotheses about the causes are correct. What would you do to address them?