# Entropy

Entropy is a heuristic that measures the model’s uncertainty about the class of the image. The higher is entropy – the more “confused” the model is. As a result, data samples with higher entropy are ranked higher and offered for annotation.&#x20;

Below is a mathematical formula for entropy calculation:

<figure><img src="https://wiki.cloudfactory.com/media/pages/docs/userdocs/active-learning/active-learning-in-hasty/entropy/2c0f35c05c-1684131886/entropy.webp" alt=""><figcaption><p><a href="https://jacobgil.github.io/deeplearning/activelearning">Source</a></p></figcaption></figure>

Imagine that our model predicts whether an object is a dog or a muffin. Let’s say it gave the following probabilities for instances A and B:

* Instance A: “dog” – 0.5, “muffin” – 0.5.\
  Entropy = H(\[0.5,0.5]) = **1.0**
* Instance B: “dog” – 1.0,  “muffin” – 0.0.\
  Entropy = H(\[1.0,0.0]) = **0.0**

As you see, the model was completely confused when the class probabilities were equal. In contrast, when one of the classes had a 100% probability, the entropy was equal to **0** since there was no uncertainty.

<figure><img src="https://wiki.cloudfactory.com/media/pages/docs/userdocs/active-learning/active-learning-in-hasty/entropy/5b105cdbbc-1684131886/muffins.webp" alt=""><figcaption><p>In case you are wondering how dogs might look similar to muffins. They really do!</p></figcaption></figure>

Entropy might be useful in datasets with very diverse images and classes.&#x20;

Learn more about the other heuristics:

* Variance
* Margin
