Shannon Entropy Calculator

Shannon entropy measures the average uncertainty (information) in a distribution.

How the Math Works

Shannon entropy measures the uncertainty or randomness in a probability distribution. The formula H = -Σ pᵢ·log₂(pᵢ) calculates this by summing the contribution of each outcome: for each probability pᵢ, we multiply it by its base-2 logarithm (giving the information content), then negate and sum all contributions. When an event is certain (p=1), its contribution is zero; when equally likely (p=0.5), it contributes maximally. The result is expressed in bits, representing the average number of yes/no questions needed to identify an outcome.

Practical Applications

Use this calculator when analyzing data distributions in information theory, machine learning, or statistics. Enter your probability values (which must sum to 1) to measure how unpredictable your data is—useful for feature selection in ML, determining optimal code lengths in data compression, or evaluating the effectiveness of a classification model. Higher entropy indicates more uncertainty or information content, while lower entropy suggests more predictability in your dataset.

Day-to-Day Use

This concept appears in everyday decision-making and information systems. When you're uncertain about which route to take home and each has roughly equal probability of being fastest, that's high entropy—requiring more mental effort to decide. Conversely, if you know one route is consistently fastest, entropy is low. Search engines use entropy to rank results, data compression algorithms rely on it to shrink file sizes, and understanding entropy helps explain why some passwords are harder to guess than others—it's the mathematical foundation for measuring surprise and uncertainty in our daily information-rich world.

Worked example

Four equal probabilities (0.25 each) → 2 bits.

FAQ