Open Learning

As a person who does a lot of autonomous learning, the Internet in these days offer a huge amount of possibilities to read/learn about any topic you might think of. There might be more the problem of filtering out useful/good content from the nearly infinite amount of sources. Inspired by a colleague I will try to give a record of whatever I read/saw and can recommend on specific topics. I will also try to add specific links that I have already studied in the past but may help any interested reader (or myself as lookup). Most stuff will be about machine learning in general and more specific about computer vision/image classification as my master thesis is related to these topics. But from time to time I might add also some more fun related topics.

Interview Questions

Everything - basic maths, ML, DL

• Questions on Linear Algebra, Numerical Optimization, Probability, Confidence Interval, Bias-Variance, Emperical Risk, VC Dimension, Model, Feature selection, Curse of Dimensionality, MLP, DL motivation, SVM, Bayesian, Regularization, ROC-AUC, Clustering, Dimensionality Reduction, Optimization, RNN, LSTM, Autoencoder, Representation Learning, Monte Carlo

  Overview ML questions

• SQL, ML, Company related, General Knowledge ML

  ML

• Top 10 FAQ on ML

  Python, Coding

• Python Coding, Time Complexity and Case Studies

  Preparation Guide

• How to answer questions on Machine Learning Basics, Machine Learning Coding, Applied Machine Learning Problems, Project-Based Machine Learning Questions

  DL, Case Study

• CNN, NLP, LSTM, GANs, MLP, Case Study Questions

  Resources

• Resource
• Resource
• Resource

Topics to Study

basic machine learning algorithms - summary

• Summary of supervised, semi-supervised, unsupervised and reinforcement
• All ML algorithms with basic explanation and code
• List of ML algorithms
• Advantages/Disadvantages/Uses of ML algorithms
• ML algorithms outline

  cheat sheet

• Pyspark

  cheat sheet

• Probability

  cheat sheet

• Python, Pandas, Matplotlib, Scikit Learn

Machine Learning

Statistics

• Type I and Type II error
• Understanding AUC ROC

  ML Algorithms

• Linear Regression
• Polynomial regression
• Linear Regression Assumptions
• Gradient Descent in Linear Regression
• [Video] Linear Regression
• Logistic Regression
• [Video] Logistic Regression
• Decision Tree
• [Video] Decision Tree
• SVM
• SVM loss function - hinge loss
• SVM loss function - hinge loss part 2
• Kernel trick in SVM
• [Video] SVM
• Naive Bayes
• [Video] Naive Bayes
• [Video] kNN
• kNN
• K-Means
• How to choose k in k-means?
• [Video] kMeans

  Ensemble Algorithms

• Random Forest
• [Video] Random Forest
• Limitations of Random Forest Regression
• GBM regression
• GBM classification
• XGBoost - what, why, benefits
• XGBoost - intuition, performance, hyperparameters
• XGBoost - part I
• XGBoost - part II
• LightGBM
• LightGBM parameters and implementation
• Bootstrapped Aggregation (Bagging)
• AdaBoost
• Weighted Average (Blending)
• Stacked Generalization (Stacking)
• Gradient Boosting Machines (GBM)
• Gradient Boosted Regression Trees (GBRT)
• CatBoost

  Dimensionality Reduction Algorithms

• Principal Component Analysis (PCA)
• Principal Component Analysis (PCA) part 2
• Principal Component Regression (PCR)
• t-SNE
• [Video] t-SNE
• Linear Discriminant Analysis (LDA)

  Feature selection algorithms

• Summary: Filter - Information Gain, Chi Square, Fisher score, correlation coefficient, varience threshold, Mean Absolute Difference, dispersion ratio Wrapper - forward, backward, exhaustive feature selection, recursive elimination Embedded - L1, Random Forest Importance
• chi square
• chi square part 2
• information gain
• information gain part 2
• fisher score

  Optimizers

• general guide

  Model Explainability

• SHAP explanation
• LIME
• SGD

  Regularization

• Regularization theory
• Regularization theory part 2
• Lasso vs Ridge theory
• [Video] Lasso vs Ridge
• [Video] Ridge
• [Video] Lasso

  Cross-Validation

• CV theory
• CV code

Deep learning

Book on DL

• NN and DL

  GAN

• Monitor GAN Training Progress and Identify Common Failure Modes
• How to Identify and Diagnose GAN Failure Modes
• Geometry Score

  Activation Functions

• Linear
• When to use sigmoid, softmax, relu
• How to use activaion functions in hidden and output layer

  Object Detection

• Object Detection using ResNet-50

  NLP

• NLP course Stanford
• [SO] Common questions on NLP implementation
• History of NLP
• Latent Semantic Analysis
• Word2Vec
• BERT word embeddings
• BERT research concepts
• Positional Encoding in Transformer Models
• Transformer positional encoding layer - keras code
• attention mechanisms
• attention mechanisms from scratch
• LSTM vs RNN
• Embeddings
• Word Embeddings, Continuous bag of words, skip grams, word2vec, negative sampling
• Evaluation of translation quality - - BLEU for machine translation and ROUGE for summarization.
• Evaluating Language Models by OpenAI, DeepMind, Google, Microsoft
• Evaluation metrics - entropy, perplexity, bits per character/word
• Contrastive Predictive Coding
• NLP data augmentation
• handling rare words
• basic NLP questions 1
• basic NLP questions 2

  RNN

• Forward pass, backward pass, gradients, loss, implementation
• Forward pass explained
• Backpropagation explained
• Sequene modelling interview questions
• teacher forcing
• NN vanishing gradient
• time series vanishing gradient

  RetinaNet

• The intuition behind RetinaNet - Focal Loss
• [Paper] Feature Pyramid Networks for Object detection
• [Paper] Focal loss for Dense object detection

  ResNet

• Concept

  Time Series

• Time Series with ML

Courses

• MIT 6.034 Artificial Intelligence by Patrick Winston (23 lectures + 7 Mega-Recitations)
• Coursera - Machine Learning by Stanford University - online class by Andrew Ng (highly recommended)
• Undergraduate machine learning at UBC 2012 - by Nando de Freitas (33 lectures)
• Deep Learning at Oxford 2015 - by Nando de Freitas (16 lectures)
• CS231n - CNNs for Visual Recognition - by Fei-Fei Li and mainly Andrej Karpathy (Overview)
• Neural networks class - Université de Sherbrook by Hugo Larochelle (92 mostly short videos)
• Deep Learning Talk MLSS 2014 - by Yoshua Bengio at MLSS 2014 (3 Parts)

Videos

• Visualizing Data Using t-SNE - GoogleTechTalk by Laurens van der Maaten presenting t-SNE
• Visualizing and Understanding DNNs by Matt Zeiler presenting deconv-nets for visualizing DNNs
• Fun demo of CNN from ‘93 - Yann LeCun presenting his CNN for handwritten digits.
• MarI/O - NEAT applied on Super Mario World by SethBling
• The Art of neural networks - Mike Tyka talking about the evolution of art and programming

Blogs/Sites

• Andrej Karpathys Blog - Blog with a lot of well written articles mainly about CV and CNNs
• CS231n - corresponding site to the CS231n course from Standford university. Each topic covered in an article.
• UFLDL Tutorial - Unsupervised Feature Learning and Deep Learning Tutorials by Standford University
• Understanding LSTMs - Some nice write up about LSTM-Nets by Christopher Olah

Programming

Videos

• Introduction to DL with Python - Presentation by Alec Radford giving an overview of Deep Learning with Theano

Blogs/Sites

• Undocumented Matlab - One of the best Matlab related Sites I know.
• CheckiO - Online game-based learning of Python
• Theano Tutorial - Introduction to Theano (Python libary)
• Theano Examples - some more applied, though advanced tutorials.

General Scientific stuff

Videos

-How to write a Scientific Paper - Talk by Simon Peyton Jones