# Section 10: Random Forests & Boosted Trees

Sections 9 and 10 are on tree-based methods. There are three main methods:
- Decision Trees (Section 9)
-  [Random Forests (Section 10)](https://nurfadhilah.tech/section-10-random-forests-and-boosted-trees) 
-  [Boosted Trees (Section 10)](https://nurfadhilah.tech/section-10-random-forests-and-boosted-trees) 

Each of these methods stems from the basic decision tree algorithm. Fundamentally, tree-based methods rely on the ability to split data based on information from features. Require a mathematical definition of information and the ability to measure it.

Classification and Regression Tree (CART) introduces many concepts:
- Cross validation of Trees
- Pruning Trees
- Surrogate Splits
- Variable Importance Scores
- Search for Linear Splits

**References:
**
- [An Introduction to Statistical Learning](https://www.statlearning.com/)  (Download free pdf)
- [Jose Portilla's 2021 Python for Machine Learning & Data Science Masterclass](https://www.udemy.com/course/python-for-machine-learning-data-science-masterclass/) 

![16_ml_10_2829Sep21.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1633104209403/aGAlntQpa.png)