It is always a hassle for students and machine learning engineers to choose the best machine learning algorithm for their data model. As I mentioned in earlier posts, I prefer to follow Occam’s razor which states that “simplest algorithm is the best algorithm” and No free lunch theorem, which states that “there is not a single machine learning algorithm which works best for all problems”. Thus our aim is to find out the “simplest model which performs the best” . In order achieve simplest and best models, we need to understand all the machine learning algorithms and their purpose in depth.
In this series of tutorial article, I am going to explain all the ins and outs of the most famous machine learning algorithms. I will explain the mathematics, include python codes and show the results. I will also suggest what algorithms are useful when specific real time problems are given.
Before all, let us develop our basics of linear algebra, probability and statistics needed for machine learning.
Linear algebra is one of the pillars of machine learning. Before dealing further, let us highlight some of the reasons that you need linear algebra for:
Let us take an example: This is an Asian approach for arranged marriage. You want to get married and your parents have looked many boys/girls for you. You have to make a model to choose the one who best fits for you. What approaches will you take?
If we add them up, we get
Then weighted spouse model is
where spouse1Quals and weights are the row vectors.
Assume that you have to chose between 4 girls who have following values for qualities:
and the weighted spouse model can be expressed as :
Hence we choose the third spouse, who has the best score.
We can understand linear algebra only if we understand the geometric intuition behind it. Algebraically, Linear algebra solves a number of linear equations by expressing them in different matrix and vector forms. But geometrically, it provides a lot of insight on what is exactly happening when we are solving a problem. We can visualize the geometric meaning of vectors and matrices and the operations we perform in them. We can perform multiple operations in vectors as well as in matrix. Vector addition, subtraction, dot product, cross product, norm, angles between vectors, multiplying a vector by a scalar, vector calculus are some common vector operations, while Matrix addition, subtraction, multiplication, multiplying a matrix by a vector, multiplying a matrix by a scalar, trace of a matrix, determinant of a matrix, eigendecomposition of a matrix, singular value decomposition (SVD) of a matrix, matrix calculus are some common matrix operations.
Computationally, we represent vectors as one dimensional array, while matrices are the arrays in two dimensions. The arrays in n-dimensions are called tensors for n>2. Though it is geometrically hard to visualize multi-dimensional arrays, we can represent them well algebraically and perform number of operations on them such as addition, subtraction, multiplication and others.
There are many resources in the web for learning linear algebra. In the later blogs, I will definitely cover more about matrix operations such as eigen decomposition and singular value decomposition, with regards to dimensionality reduction algorithms such as principal component analysis (PCA) but for now
I strongly recommend you to learn from some of these video lectures to understand more about linear algebra. All of the instructors and videos are great and interesting. You just need some passion to watch them, understand and play around with problems.
1. Linear Algebra – Gilbert Strang (of the ubiquitous textbook): This is an undergraduate course from MIT by the great Gilbert Strang. I like his style of teaching and explaining things. It will provide you deeper insights on linear algebra and explains the mathematical techniques. Watch all if you have enough time and passion.
2. Linear Algebra – Foundations to Frontiers : These are short videos, well explained techniques and easy to understand.
3. Lecture series – Essence of Linear Algebra from 3Blue1Brown youtube channel : I find this one really interesting and enjoyable, because they have animated many things to make us able to imagine the geometric meaning. 3Blue1Brown has been one of my favorite channels since I discovered them in Youtube.