## terça-feira, 2 de fevereiro de 2010

### Formal Requirements for the Average Uncertainty

(An Introduction to Information Theory, Fazlollah M. Reza)

Shannon's approach, as well as several other authos', in suggesting a suitable H function has been to some extent directed toward an axiomatic description of such functions. The desired H function should habe the following basic properties:

1. Continuity.
That is, if the probabilities of the occurrence of events Ek are slightly changed, the measure of uncertainty associated with the system should vary accordingly in a continuous manner. (...) a slight change in the probability of the occurrence of an event should not provide us with a significantly large amount of information.

2. Symmetry.
The H function must be functionally symmetric in every pk. Indeed, the measure of uncertainty associated with a complete probability set [Ek, Ek'] must be exactly the same as the measure associated with the set [Ek',Ek]. Our measure must be invariant with respect to the order of the events.

3. Extremal Property.
When all the events are equally likely, the average uncertainty must have its largest value. In this case, it is most uncertain which event is going to occur. Conversely, once we know which specific event among a number of n equally likely events has occured, we have acquired the largest average amount of information relevant to the occurance of events of a universe consisting of n complete events.