Knowledge base
random process
Notes filed under 📐 Advanced Mathematics / random-process.
Notes in this folder
📐 Advanced Mathematics / random-process
4. Markov-decision-process
A Markov decision process formalizes sequential decision making with states, actions, rewards and transition probabilities. Use it to set up value and policy iteration, and as the base model that reinforcement learning methods optimize.
📐 Advanced Mathematics / random-process
3. Markov Chain & Markov Sampling
A Markov chain describes memoryless state transitions, and Markov-chain Monte-Carlo turns such a chain into a sampler for a distribution known only up to a constant. Use Metropolis-Hastings or Gibbs sampling when direct sampling from the target is impossible.
📐 Advanced Mathematics / random-process
2. Monte-Carlo Sampling and Importance Sampling
Monte-Carlo estimates an expectation by averaging samples, while importance sampling reweights samples from a proposal distribution to cut the variance. Use them when the integral has no closed form, and as the sampling intuition behind MCMC and variational methods.
📐 Advanced Mathematics / random-process
1. Summary for basics of probability theory-Prior and Posterior Probability
Prior, likelihood and posterior describe how a belief about a hidden parameter is updated by observed data. Use this summary as the reference for Bayes rule before moving on to sampling, Markov chains and variational inference.