7.1 Basics

1. [E] Explain supervised, unsupervised, weakly supervised, semi-supervised, and active learning.
2. Empirical risk minimization.
   1. [E] What’s the risk in empirical risk minimization?
   2. [E] Why is it empirical?
   3. [E] How do we minimize that risk?
3. [E] Occam's razor states that when the simple explanation and complex explanation both work equally well, the simple explanation is usually correct. How do we apply this principle in ML?
4. [E] What are the conditions that allowed deep learning to gain popularity in the last decade?
5. [M] If we have a wide NN and a deep NN with the same number of parameters, which one is more expressive and why?
6. [H] The Universal Approximation Theorem states that a neural network with 1 hidden layer can approximate any continuous function for inputs within a specific range. Then why can’t a simple neural network reach an arbitrarily small positive error?
7. [E] What are saddle points and local minima? Which are thought to cause more problems for training large NNs?
8. Hyperparameters.
   1. [E] What are the differences between parameters and hyperparameters?
   2. [E] Why is hyperparameter tuning important?
   3. [M] Explain algorithm for tuning hyperparameters.
9. Classification vs. regression.
   1. [E] What makes a classification problem different from a regression problem?
   2. [E] Can a classification problem be turned into a regression problem and vice versa?
10. Parametric vs. non-parametric methods.
    1. [E] What’s the difference between parametric methods and non-parametric methods? Give an example of each method.
    2. [H] When should we use one and when should we use the other?
11. [M] Why does ensembling independently trained models generally improve performance?
12. [M] Why does L1 regularization tend to lead to sparsity while L2 regularization pushes weights closer to 0?
13. [E] Why does an ML model’s performance degrade in production?
14. [M] What problems might we run into when deploying large machine learning models?
15. Your model performs really well on the test set but poorly in production.
    1. [M] What are your hypotheses about the causes?
    2. [H] How do you validate whether your hypotheses are correct?
    3. [M] Imagine your hypotheses about the causes are correct. What would you do to address them?