KL Divergence
MediumMachine LearningMath
Description
Given two probability distributions p and q of equal length, return the Kullback-Leibler divergence from q to p: the sum over outcomes of p * ln(p / q). Terms where p is 0 contribute 0. Round the result to 4 decimal places.
Examples
Input:
[0.5,0.5], [0.5,0.5]Output:
0Explanation:
The divergence accumulates how much more probable each outcome is under p than under q, weighted by p, and is zero only when the two match.
Input:
[1,0], [0.5,0.5]Output:
0.6931Explanation:
The divergence accumulates how much more probable each outcome is under p than under q, weighted by p, and is zero only when the two match.
Input:
[0.5,0.5], [0.9,0.1]Output:
0.5108Explanation:
The divergence accumulates how much more probable each outcome is under p than under q, weighted by p, and is zero only when the two match.
Constraints
- •
1 ≤ length ≤ 10³ - •
each distribution sums to 1 - •
q has no zeros where p is positive