Stationary Point Definition of stationary point from wikipedia : In mathematics, particularly in calculus, a stationary point or critical point of a differentiable function of one variable is a point on the graph of the function where the function's derivative is zero. Informally, it is a point where the function "stops" increasing or decreasing… Continue reading Lagrange Multiplier With Equality Constraints

# Category: math

# Lagrange Duality

There are three things : Original problem (Primal problem) Dual function (Function of lagrange multiplier) Dual problem Suppose p is the solution of primal problem and d the dual problem. If original problem is minimization, we are interested in lower bound (d) such that d<p. We want to find maximum value of d… Continue reading Lagrange Duality

# Class 12 Geometry Notes

Motivation behind these notes is that geometry helps in providing intuitive derivation to machine learning models and optimization scenarios ! Line in 2D resembles plane in 3D, not the line in 3D. Concept of distance is essentially projection, It can be either sine (Cross product) or cosine (Dot product) References : http://www.ncert.nic.in/index.html

# Lagrange Multiplier and Constrained Optimization

Lagrange multipliers helps us to solve constrained optimization problem. An example would to maximize f(x, y) with the constraint of g(x, y) = 0. Geometrical intuition is that points on g where f either maximizes or minimizes would be will have a parallel gradient of f and g ∇ f(x, y) = λ ∇ g(x,… Continue reading Lagrange Multiplier and Constrained Optimization

# On multivariate Gaussian

Formulas Formula for multivariate gaussian distribution Formula of univariate gaussian distribution Notes: There is normality constant in both equations Σ being a positive definite ensure quadratic bowl is downwards σ2 also being positive ensure that parabola is downwards On Covariance Matrix Definition of covariance between two vectors: When we have more than two variable… Continue reading On multivariate Gaussian

# Quick Note on Probability Rules

p(X, Y) is joint distribution p(X/Y) is conditional distribution p(X) is marginal distribution (Y is marginalized out). You can not get conditional distribution from joint distribution with just by integration. There is no such relationship. There are just two rules for probability. Sum rule and product rules. And then there is Bayes theorem. … Continue reading Quick Note on Probability Rules