The informational interpretation

The three possibilities can be phrased in terms of informational content:

1. The message is not informative. Your knowledge remains as it was. You knew that Florence is in Italy; 35° is not unusual in Florence in July.

2. You might not have known where Florence is and what the weather there is like in summer. Now you know that much at least: It is pretty warm there in July. The message is informative; it adds something to your knowledge.

3. You might have thought Florence is a place far up North in Norway. Now you know: This cannot be true. The third possibility is the most interesting case: the

2 N. Ilkka, Informaatio, Tieto ja Yhteiskunta (Information, Knowledge, and Society), Helsinki 1989, p.57.

35

message says: There is something wrong with your knowledge. You have to change your world view. This is the maximum information you can get from message. Such a message gives reason to become sceptic: Is it true at all?

Here we encounter something interesting: When we speak about “no information” (and that is minimum information) and maximum information – apparently there is something like an amount of information, a quantity that will vary in the cases between the extremes, at least in the sense of less and more information.

Apparently, information – in the sense “what is informative” – depends not so much on the message itself but mainly on the previous knowledge of the receiver of a message, on his expectations:

For an ignorant – a person who knows very little or nothing – everything is informative.

For average people like us what is informative depends on what we know, on our world view, on our expectation. For an all knowing person, an omniscient (if such a person exists) nothing is informative.

The ignorant is not very interesting, nor is the omniscient. We are interested in people like us, who have a world view, some knowledge about the world. We know where Florence is and have an idea, a certain expectation what the weather is like in summer in Florence.

But: The information the receiver gets from a message does not only depend on his knowledge but to a high degree on his competence to evaluate the information of the message:

And this competence depends on his knowledge and on his capability to process the message, to track the effects of a message in his knowledge.

In the silly example of weather in Florence in July again: Imagine the message is not “it is terribly hot here” but: Last night here fell snow!

The ignorant might conclude that Florence must be somewhere near the North Pole or deep down in South America. And he might think: “Aha, weather is much better here!” The best he will get from the message would be a wrong world view.

An average person, who has an idea where Florence is and what weather can be expected in July, would think: That is sensational. I would never have thought that. That is impossible!

And he would now imagine how traffic breaks down in the town and other consequences of snowfall.

Now think of a meteorologist. He has not only factual knowledge about weather in Florence, but he knows why it is hot in July in Florence. Snow in July is not only very unexpected, a sensation, but it contradicts his knowledge. He knows that weather in the seasons

36

has to do with the inclination of the earth axis and would consider, whether something might have happened to this inclination.

It is apparent that an average person gets more information from the message than an ignorant and that the expert gets more information than the average person. With a conventional opinion one would think that the information of a message has to do with expectations, with the probability of the message. This is and remains true. But this does not fully explain the difference between the meteorologist and the layman in weather science. For both, when asked, the probability of snow in July in Florence would be zero, it is impossible in the world as they know it.

But apparently the message has more significance for the meteorologist than for meteorological layman. We could conclude: The amount of information does not only depend on the probability of an event, on our expectations. The more somebody already knows and the better his ability is to reason and his logical ability (this is the capability to track the consequences of a message in a world view), the more information he can get from a message.

3. Quantification

This more or less information means apparently that different persons get different amounts of information from a message. And this brings us to the problem, how to determine the amount of information.

If the amount of information a person gets from a message is mainly not in the message itself but depends on his previous knowledge and on his logical competence, it seems futile to look for an amount of information that could be determined objectively. Knowledge differs from person to person.

But an objective amount of information is just, what Rudolf Carnap and Yehoshua Bar–

Hillel, two philosophers, where after in an article they published in 1952³. In this article they explored a measure of information. Their starting point was not real human knowledge and their logic was very simple: They took a very simple model consisting of three individuals and two properties. These made up their whole universe (our knowledge) to test a measure of quantification of information. If you want a more concrete situation: imagine an astronomer interested in three planets of a foreign star (the three individuals). And he is eager to get information whether there is water on them and whether there is life (the two properties). His

“universe” (in this scientific project) is closed to these states of the planets under review.

3 R. Carnap, Y. Bar–Hillel, An outline of the theory of Semantic information. Research Laboratory of Electronic, Massachusetts Institute of Technology, Report No. 247, 1952.

37

How many answers can he get? There are 64 possible answers: none, one, two or all three of the planets have water or life or both, thus from none of them has water (w) or life (l) – to all of them have water and life. This is a combinatorial calculation with the formula (2²)³ = 2⁶=64. Or, to be more elaborate: There are 4 possibilities for C (w+l, w–l, –w+l, –w–l). The same 4 possibilities exist for B, so we get 4 * 4, and again for A exist the same 4 possibilities, making 4*4*4 = 64 possibilities.

Carnap/Bar Hillel concluded: The maximum information one can get in such a universe is, when the two properties are determined for all three individuals (when our astronomer knows of all three planets whether there is or is not water or life). There remains a single possibility and 63 possibilities are excluded, and this is the maximum information our astronomer can sensibly hope for. The more possibilities are excluded – the more information you get. And this can be calculated and thus quantified in such a simple model. When the astronomer gets to know there is water on planet A, 32 possibilities are excluded. If additionally, he gets to know that there is life on planet B, then 48 possibilities are excluded. Generally: The more possibilities are excluded the bigger is the informational content of the message – or: The amount of information is equal to the amount of excluded possibilities.

Of course, the model Carnap and Bar Hillel have used is far from realistic. They assume a receiver who has complete knowledge and perfect logical skill. For real life situations it is absurdly small (or it may fit for exceptional situations astronomers might be in). In real life we have to do with an indefinite if not infinite number of individuals (where “individual” means not only persons but everything that qualifies for an item to make statements about) and a number alike of properties such items may have. One could begin to doubt, whether it is sensible at all to ponder over the amount of information of messages.

But, of course, we speak about information of messages and sentences of all kind. And we do this very sensibly. Indeed, we do not try to quantify such information, but we compare the informational content of messages, as we have done here, and we state that it is very well possible to speak about more or less information. How is this possible?

We know the answer already: Our situation is not the one, Carnap and Bar Hillel have taken as basis for their analysis. When evaluating the information of a message we do not start from scratch, with no previous knowledge, where everything is possible with the same grade of probability. This led into the problem of immense numbers.

On the contrary: We have a world view. And this world view is a very, very small cutout of these zillions of possibilities. This world view, our own “universe”, can be understood as decisions made among the immense number of possibilities. And these decisions are made

38

possible by our knowledge of facts, which we accept as well founded, or by our assured believes, as Niiniluoto says, and which make up our knowledge. We have established ideas how properties are distributed among things and persons. These are the basis of what we expect and what surprises us as new and informative. The universe has shrunk to something manageable, though it may be very large still.

A very important difference between the universe, Carnap and Bar–Hillel have taken as basis in their study and the real universe, we live in, is that their universe is complete, closed.

All possible states are known. This was a condition for quantification. Such exact quantification is not possible in an open system. Our universe, our world view is open. We are conscious thereof that we never have complete knowledge of the world. We have to accommodate our knowledge permanently learning new facts enlarging our knowledge or correcting our beliefs, changing our universe. This makes exact quantification of the informational content of any incoming message impossible.

What remains of Carnap and Bar–Hillel's project of quantification of information is the possibility of a rough estimation, an estimation of the informational value of an incoming message. But even such a rough estimation will often be sufficient to compare the informational content of different messages.

And what we still can accept is that a measure of information is the amount of excluded possibilities by a message. The more a message restricts what we have thought to be possible, the more it contradicts our previous knowledge, the more of our previous knowledge a message suggests to be wrong, the larger is its informational content. In an open system, of course, we cannot restrict information to excluded possibilities. We have to take into account messages that do not contradict our knowledge but just enlarge it. Here the question of quantification gets a new turn. Following the model of Carnap and Bar–Hillel one could now think of a list of possible additions to our knowledge, but we have seen that in the real world this leads to unmanageable numbers at least, if the idea is not mad from the outset. Again we will be limited to rough estimations of the amount of information of a message, at most.

39