Knowledge base
optimum theories
Notes filed under 📐 Advanced Mathematics / optimum-theories.
Notes in this folder
📐 Advanced Mathematics / optimum-theories
1. Karush–Kuhn–Tucker (KKT) conditions
The KKT conditions turn a constrained optimization problem into stationarity, feasibility and complementary-slackness equations. Use them to test whether a candidate point can be optimal, and to derive closed-form solutions for constrained learning problems.
📐 Advanced Mathematics / optimum-theories
3. Variational Lower Bound (VLB)
The variational lower bound (VLB), also called the evidence lower bound (ELBO), is a tractable lower bound on a latent-variable model's log evidence. Use it when exact posterior inference or marginal-likelihood evaluation is intractable.
📐 Advanced Mathematics / optimum-theories
2. Kullback-Leibler divergence
The KL divergence evaluates the difference of two different distributions, It's often a powerful tool for make two distributions coincide in the training of deep-learning neural networks.