Simplicity Theory

Simplicity, Complexity, Unexpectedness, Cognition, Probability, Information

by Jean-Louis Dessalles
(created 2008.12.31)

(updated 2010.02.19)

Simplicity Theory (ST) is a cognitive model based on the following observation: human individuals are highly sensitive to any discrepancy in complexity. In other words, their interest is aroused by any situation which appears "too simple" to them.

Context

Finding out what elicits human interest is a fundamental scientific question which, quite surprisingly, has not been given much scientific attention in past decades. This is unfortunate, given that:
- Our species is unique in what draws the attention of its members. For instance, only human beings are amazed at coincidences.
- Much in human social relationships depends on shared interests, unlike what is to be observed in other species.
- A considerable part of modern economies is devoted to eliciting interest through films, books, shows, media and games.

The scientific study of the determining factors of human interest may have been hindered by the tacit assumption that these factors are way too complex, multiple and fuzzy to be properly modelled. The results presented in these pages were obtained thanks to the converse assumption: our minds would be much simpler than commonly believed. Ironically, simplicity plays a major role in the theory presented here. Our minds seem to possess the amazing ability to monitor the complexity of some of their own processes. Here, complexity has to be taken in its technical acceptation (size of minimal description).

These pages are an invitation to challenge the model they present. All suggestions, critiques and contributions are welcome. Please send reactions to JL at@ dessalles.fr.

Significance for language and cognitive sciences

ST provides a formal and predictive model that impacts on the following domains:

- relevance in spontaneous language and in the news (what is interesting, and what is appropriate to say)

- high-level salience (what will attract my attention)

- subjective probability judgements (decision, regret, some so-called "biases" in probabilistic reasoning)

- cognitive aspects of emotions (amplification through unexpectedness)

Simplicity Theory (ST)

ST's main claim:

An event is unexpected
if it is simpler (=less complex)
to describe than to generate.

All terms are important here.

Event: Event means unique situation (not classes of objects nor sets or whatever). Perception present us with plenty of unique situations, often characterized by the four Ws (When, Where, What, Who).

Complex: Complexity refers to the size of the minimal unambiguous description (see below).

Simplicity: Simplicity is the amount of a complexity drop.

Describe: Since we are dealing with unique situations, the problem of a description is to produce that uniqueness. It can be achieved through a description of some characteristics of the situation that allow to isolate the situation from its class; it can also be achieved thanks to spatiotemporal determination.

Generate: Events are not any combination of disparate items. They were produced, generated, by the "world" (or some world, e.g. in fiction). Generation complexity is the minimal description of the parameters that have to be set for the "world" to generate the situation.

Unexpectedness is one of the two ingredients of interest (the other being emotional intensity). Unexpectedness, properly defined, predicts what will be considered interesting in human communication, e.g. in conversational narratives and in the news. It also predicts the kind of situations that will draw human attention.

Formally:

U = C_w – C

U: unexpectedness

C_w: generation complexity (= size of the minimal description of parameters values the "world" needs to generate the situation)

C: description complexity (= size of the minimal description that makes the situation unique)

Generation complexity C_w(s) is the complexity (minimal description) of all parameters that have to be set for the situation s to exist in the "world". For example, the complexity of generating the fact that you reach me by chance, knowing that you have to cross five four-road junctions to do so, is 5 x log₂ 3 = 7.9 bits.

We may consider C_w(s) as the length of a minimal programme for the W-machine to generate s.

Description complexity C(s) is the length of the shortest available description of s (that makes s unique). This notion corresponds to the usual definition introduced in the years 1960 by Andrei Kolmogorov, Gregory Chaitin and Ray Solomonoff. However, we must instantiate the machine used in that definition: C corresponds to the complexity of cognitive operations performed by an observer. Note that C thus depends on the cognitive model used (I recommend Michael's Leyton generative theory of shape as a possible component of such model). The other restriction consists in considering, not an ideally minimal description, but the shortest one among the descriptions currently available to the observer.

We may consider C(s) as the length of a minimal programme for the O-machine to generate s.

The O-machine is a computing machine that can rely on all cognitive abilities and knowledge of the observer.

Caveat: The main problem when beginning with ST is that one fails to consider unique situations. For instance, you may think that a window is a simple object, because it is easy to describe. Indeed! But that window over there is complex, precisely because it looks like any other windows. To make it unique, I need some significant amount of information in addition to the fact that it is a window.

Consider, for instance, the number 131072. You may see the difference between merely saying that it is a number and producing a way of distinguishing it from all other numbers. Kolmogorov complexity corresponds to the latter action. Describing a number supposes that one eventually can reconstitute all its digits (e.g. by saying that it's 2^17).

Simplicity theory has been used to explain several important phenomena concerning human interest in spontaneous communication and in news (see bibliography below). More should come. Below are a few didactic examples.

Predictions: Some phenomena explained by ST

Thanks to the above definition of unexpectedness, several quantitative predictions can be made about what human beings would regard as unexpected / improbable / interesting. Some of these predictions are illustrated in the following concrete examples. Most predictions can easily been checked by common sense, and experiments with human participants clearly confirm them (Dimulescu & Dessalles 2009). Note that no alternative theory seems available to predict these same facts.

The "rabid bat" effect (complex causal history)

Remarkable lottery drawings (interesting structures)

Inverted stamps (rarity)

The "Robert Wadlow" effect (record)

The fish story effect (atypicality)

Two running nuns (departure from norm)

The "next door" effect (proximity)

The "Eiffel Tower" effect (simple landmarks)

The Lincoln-Kennedy effect (coincidences)

The "You? Here?" effect (fortuitous encounters)

The "Pisa Tower" effect

Unexpectedness and Probability

One of ST's main claims is that unexpectedness can be used to define probability, thanks to the following formula:

p = 2^-^U

This formula (Dessalles 2006) can be regarded as a basic law in cognitive science. It reveals that human beings assess probability through complexity, not the reverse. Remarkably, complexity is used at all levels of cognition, from perception (Chater 1999; Chater & Vitányi 2003) up to narratives and probability assessments.

Note that the preceding formula is more adequate than Solomonoff’s classical definition of algorithmic probability, p = 2^–Cw, which takes only into account generation complexity C_w. See Remarkable lottery drawings to get convinced. To a human eye, absolute complexity does not matter: both a random pattern and a periodic one may be regarded as probable. None of the two woods below is surprising, as we suspect the more complex one to result from a complex process (e.g. wind dispersal of the seeds, C_w large) whereas the simpler one (on the right) is attributed to a simple human plan (C_w small). The regular pattern would be highly surprising if seen, say, on a long uninhabited island (C_w large again).

From: www.hockinghills.com/comfort/ From: iciouailleurs.free.fr/HautJura/hautjura.html

Moreover, is we follow Solomonoff's definition, all real life situations have virtually zero-probability to exist, as they needed countless parameters to be set up to have a chance to occur. This makes that definition of probability of little interest for practical purposes.

Complexity drop, as measured by U, is however crucial. For instance if I throw six coins on the floor and they end up perfectly aligned and regularly spaced, the situation is experienced as highly unexpected and thus very improbable. Generating the fact (regardless of the face) is complex: the x-y position of each coin (i.e. 12 real numbers for 6 coins, limited by some reasonable precision) must be determined independently. But describing the situation is simple (it only requires 4 numbers: two to determine a coin position, one for the direction and one for the spacing). Complexity drop amounts to 8 numbers, equivalent to 48 bits if one can discriminate 64 values for each number. This corresponds to an incredibly low probability: p=3.6 10^–¹⁵, which amounts to getting 48 tails in a row while flipping a coin. If I witness such a marvel, I’d rather imagine a trickery to diminish generation complexity (see Two running nuns).

Suggestion: compute additional complexity drop due to the fact of getting 6 identical faces.

Cognitive Information

ST’s formula p=2^–U looks like the converse of Claude Shannon’s definition of information. We might thus be tempted to equate unexpectedness U with information I:

I = U

This means that complexity and unexpectedness are cognitively fundamental, and that probability is, at best, a derived notion. We expect human individuals to be able to convert complexity drops into (low) probability, but not the converse (hence the impressive so-called cognitive ‘biases’ that Daniel Kahneman, Amotz Tversky and others have discovered).

The other fundamental dimension of information is however missing: emotional intensity E. One should rather write:

I = U + E

The term of unexpectedness U allows to make several non-trivial predictions about what constitutes valuable information. These predictions include logarithmic variations with distance (see The "next door" effect), the role of prominent places or individuals (see The "Eiffel Tower" effect), habituation effects, the importance of coincidences (see The Lincoln-Kennedy effect), recency effects, transitions, violations of norms (see Two running nuns), and records (see The "Robert Wadlow" effect). These theory-based predictions apply both to personalized information and to newsworthiness in the media.