Minimum description length (MDL)

Minimum description length (MDL), is that the algorithmic coding theory in computing, regards both model and data as codes. The code length is directly proportional to the generalization capability of the model, here the model provides the shortest description of the data should be chosen.

In the above equation, the L(data∣model) can be described as description length of the data given the model, measured in bits of information.

The above definition of minimum description length is can be applied to any well-defined models, even verbally defined, qualitative models.

The MDL principle are often given as example for model selection criterion, which is employed to work out the acceptable complexity of a model for a given dataset.

The principle of minimum description length

A Bayesian perspective on Occam’s razor

 Motivated by interpreting the definition of h(MAP) within the light of basic concepts from scientific theory.

which can be expressed in terms of maximizing the log2

or alternatively, minimizing the negative of this quantity

Rewrite Equation (1) to point the that hMAP is that the hypothesis h that minimizes the sum given by the outline length of the hypothesis plus the description length of the data given the hypothesis.

Where, CH and CD|h are the optimal encodings for H and for D given h

The Minimum Description Length (MDL) principle recommends choosing the hypothesis that minimizes the sum of those two description lengths of equ.

Minimum Description Length principle:

Here in the above equation, the codes C1 and C2 will represent the hypothesis and the data given the hypothesis

From the above analysis we can conclude that if we choose C1 to be the optimal encoding of hypotheses CH, and if we choose C2 to be the optimal encoding CD|h, then hMDL = hMAP

Leave a Comment

Your email address will not be published. Required fields are marked *