Machine Learning – Decision Trees


I bet you thought that was going to say Decision Boundaries again 😀 – well… that is… if you’ve read the first 4 Machine Learning posts 😉

Nope, this time is Decision Trees, which are very similar to trees in programming – aka Binary Trees.

Beer!

No, do NOT go and get any beer… although im not exactly going to be able to stop you am i? 🙂

The best way to see a Decision Tree is with an example, and the best example is with Students and Beer! – no im not saying every student drinks, but seriously… the majority do… even if you’re like me who would rather have a nice cup of hot-chocolate; just go with it 😉

Lets say we have 2 kinds of people, Students and Teachers. We have 1 kind of drink; Beer. Now, a Student (go with it) only drinks beer where there isn’t an exam on. And a Teacher will only drink Beer when its the weekend. With this we can say:

  • IF student AND exam THEN beer
  • IF student AND not exam THEN no beer
  • IF teacher AND weekday THEN no beer
  • IF teacher AND weekend THEN beer

With this, we can easily construct a Decision Tree:

detresb

Here we can see, say the first example above.
If Person=student AND Exam=yes THEN Beer

 

Before anybody says anything, yes, I do realise the mistake ive made. If exam then beer should be If exam then no beer. i apologise 😦

 

Over-fitting

Yep, thats right. Time to revisit and old friend 😀

So, as described before, if we assume that we have all the data it can lead to over-fitting. Because of this it can mean that our techniques only work well on the Training Data. Yet, with a Decision Tree, over-fitting can lead to very large and obscure trees.

Advertisements

About Badgerati
Computer Scientist, Games Developer, and DevOps Engineer. Fantasy and Sci-fi book lover, also founder of Cadaeic Studios.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: