References : https://github.com/dennybritz/reinforcement-learning http://www.wildml.com/2016/10/learning-reinforcement-learning/ https://gym.openai.com/docs/ https://gist.github.com/arcarchit/2b3363e2615df7ef5c8d4941d4dfa9e8

# Lagrange Multiplier With Equality Constraints

Stationary Point Definition of stationary point from wikipedia : In mathematics, particularly in calculus, a stationary point or critical point of a differentiable function of one variable is a point on the graph of the function where the function's derivative is zero. Informally, it is a point where the function "stops" increasing or decreasing… Continue reading Lagrange Multiplier With Equality Constraints

# Thompson Sampling

Thompson sampling is one approach for Multi Armed Bandits problem and about the Exploration-Exploitation dilemma faced in reinforcement learning. Challenge in solving such a problem is that we might end up fetching the same arm again and again. Bayesian approach helps us solving this dilemma by setting prior with somewhat high variance. Here is the… Continue reading Thompson Sampling

# Deep learning taking off

I recently started Andrew Ng's specialization on deep learning and found these two interesting points : One is about how performance of algorithm changes with the amount of data. Traditional algorithms have limits but Deep neural network has more advantages. Also for the small amount of data traditional algorithms may win over neural nets… Continue reading Deep learning taking off

# Lagrange Duality

There are three things : Original problem (Primal problem) Dual function (Function of lagrange multiplier) Dual problem Suppose p is the solution of primal problem and d the dual problem. If original problem is minimization, we are interested in lower bound (d) such that d<p. We want to find maximum value of d… Continue reading Lagrange Duality

# Parametric and Nonparametric tests

We rarely heard of nonparametric tests while reading standard statistical books. However there are some scenarios where they should be used instead of parametric tests. [1] has beautiful blog about it, I am putting just a summary from that. Different Tests Table below displays various tests, I have verified that all of these tests… Continue reading Parametric and Nonparametric tests

# Class 12 Geometry Notes

Motivation behind these notes is that geometry helps in providing intuitive derivation to machine learning models and optimization scenarios ! Line in 2D resembles plane in 3D, not the line in 3D. Concept of distance is essentially projection, It can be either sine (Cross product) or cosine (Dot product) References : http://www.ncert.nic.in/index.html

# Lagrange Multiplier and Constrained Optimization

Lagrange multipliers helps us to solve constrained optimization problem. An example would to maximize f(x, y) with the constraint of g(x, y) = 0. Geometrical intuition is that points on g where f either maximizes or minimizes would be will have a parallel gradient of f and g ∇ f(x, y) = λ ∇ g(x,… Continue reading Lagrange Multiplier and Constrained Optimization

# On multivariate Gaussian

Formulas Formula for multivariate gaussian distribution Formula of univariate gaussian distribution Notes: There is normality constant in both equations Σ being a positive definite ensure quadratic bowl is downwards σ2 also being positive ensure that parabola is downwards On Covariance Matrix Definition of covariance between two vectors: When we have more than two variable… Continue reading On multivariate Gaussian

# Quick Note on Probability Rules

p(X, Y) is joint distribution p(X/Y) is conditional distribution p(X) is marginal distribution (Y is marginalized out). You can not get conditional distribution from joint distribution with just by integration. There is no such relationship. There are just two rules for probability. Sum rule and product rules. And then there is Bayes theorem. … Continue reading Quick Note on Probability Rules