math

Class 12 Geometry Notes

Motivation behind these notes is that geometry helps in providing intuitive derivation to machine learning models and optimization scenarios !

Line in 2D resembles plane in 3D, not the line in 3D.

g1

g2

Concept of distance is essentially projection, It can be either sine (Cross product) or cosine (Dot product)

g5

g3

g4

 

References :

http://www.ncert.nic.in/index.html

 

Advertisements
math

On multivariate Gaussian

Formulas

Formula for multivariate gaussian distribution

g1

Formula of univariate gaussian distribution

g2

Notes:

  • There is normality constant in both equations
  • Σ being a positive definite ensure quadratic bowl is downwards
  • σ2 also being positive ensure that parabola is downwards

 

On Covariance Matrix

Definition of covariance between two vectors:

g3

When we have more than two variable we present them in matrix form. So covariance matrix will look like

g4

  • Formula of multivariate gaussian distribution demands Σ to be singular and symmetric positive semidefinite, which in terms means sigma will be symmetric positive semidefinite.
  • For some data above demands might not meet

 

Derivations

Following derivations  are available at [0]:

  • We can prove[0] that when covariance matrix is diagonal (i.e there is variables are independent) multivariate gaussian distribution is simply multiplication of single gaussian distribution of each variable.
  • It was derived that shape of isocontours (figure 1) is elliptical and axis length is proportional to individual variance of that variable
  • Above is true even when covariance matrix is not diagonal and for dimension n>2 (ellipsoids)

g5

Linear Transformation Interpretation

g6

This was proved in two steps [0]:

Step-1 : Factorizing covariance matrix

g7

Step-2 : Change of variables, which we apply to density function

g8

 

References

[0] http://cs229.stanford.edu/section/gaussians.pdf

 

 

 

math

Quick Note on Probability Rules

p(X, Y) is joint distribution

p(X/Y) is conditional distribution

p(X) is marginal distribution (Y is marginalized out).

 

You can not get conditional distribution from joint distribution with just by integration. There is no such relationship.

There are just two rules for probability. Sum rule and product rules. And then there is Bayes theorem.

p1p2

p3

 

We might want to look at a table like below and calculate joint and conditional distribution and marginalized out one of the variable. [1]

prob

 

Further reading : 

[0] :http://www.utm.utoronto.ca/~w3act/act245h5f/pr4.pdf

[1]:https://www.coursera.org/learn/probabilistic-graphical-models/lecture/slSLb/distributions