This is a legacy chapter which will eventually be incorporated into Chapter 17. This chapter will remain here as an appendix until we finish Chapter 17.
Feature engineering can incorporate important information about the data generation process into our model. However, adding features to the data also typically increases the variance of our model and can thus result in worse performance overall. Rather than throwing out features entirely, we can turn to a technique called regularization to reduce the variance of our model while still incorporating as much information about the data as possible.