7. Summary of Data Mining Algorithms

Note

Know yourself and know your enemy, and you will never be defeated– idiom, from Sunzi’s Art of War

Although the tutorials presented here is not plan to focuse on the theoretical frameworks of Data Mining, it is still worth to understand how they are works and know what’s the assumption of those algorithm. This is an important steps to know ourselves.

7.1. Diagram of Data Mining Algorithms

An awesome Tour of Machine Learning Algorithms was published online by Jason Brownlee in 2013, it still is a good category diagram.

_images/mlalgs.png

Figure : Machine Learning Algorithms diagram from Jason Brownlee .

7.2. Categories of Data Mining Algorithms

  1. Dimensionality Reduction Algorithms
  • Principal Component Analysis (PCA)
  • Nonnegative Matrix Factorization (NMF)
  • Independent Component Analysis (ICA)
  • Linear Discriminant Analysis (LDA)
  1. Regression Algorithms
  • Ordinary Least Squares Regression (OLSR)
  • Linear Regression
  • Logistic Regression
  1. Regularization Algorithms
  • Ridge Regression
  • Least Absolute Shrinkage and Selection Operator (LASSO)
  • Elastic Net
  • Least-Angle Regression (LARS)
  1. Decision Tree Algorithms
  • Classification and Regression Tree (CART)
  • Conditional Decision Trees
  1. Bayesian Algorithms
  • Naive Bayes
  1. Clustering Algorithms
  • k-Means
  • k-Medians
  • Expectation Maximisation (EM)
  • Hierarchical Clustering
  1. Artificial Neural Network Algorithms
  • Perceptron
  • Back-Propagation
  • Hopfield Network
  • Radial Basis Function Network (RBFN)
  1. Deep Learning Algorithms
  • Deep Boltzmann Machine (DBM)
  • Deep Belief Networks (DBN)
  1. Ensemble Algorithms
  • Boosting
  • Bootstrapped Aggregation (Bagging)
  • AdaBoost
  • Gradient Boosting Machines (GBM)
  • Gradient Boosted Regression Trees (GBRT)
  • Random Forest