LDA, Latent Diriclet Allocation is the most basic Bayesian model.
This paper analyzes the derived method of LDA based on variational.
First, the definition of symbols
: The Number of Topics: The Number of Documents: The Number of Terms In Vocabulary: Index Topic: Index Document: Index Word: Denote a word
IN LDA:: Model Parameter: Model Parameter,: Hidden Variables.
Figure Model: Introducing Variational Parameter:: Dirichlet Parameter: Multinomial Parameter
We introduce Variational Distribution, A Fully Factorized Model
It should be noted that it is the subparatory distribution, we have hidden Given
Second, the general
We used Variational EM Algorithm: In E Step, we used the Variational Approximation to Posterior to optimize Variational Parameters to find the most reliable label distribution. In M Step, we upgrade Lower Bound with respect to the model parameters.
Specific algorithm: E-Step: For each document, Find Optimal Values of The Variational Parameters
M-Step: maximize the lower bound with respect to the model parameters and
Third, LOWER Bound
3.1 JenSens INEquality
There is a random variable, for Convex, there is; for Concave, there is;
3.2 Detecting Lower Bound
For Each Document Each Word
