The information conundrum

by
Peter Radford
My articles My siteMy booksMy videos
Follow on:

from Peter Radford

Kolmogorov, I think, hit the nail on the head when he said:

“At each given moment there is only a fine layer between the ‘trivial’ and the impossible.  Mathematical discoveries are made in this layer.”

He might just as well have said that life itself is discovered in this layer.  But let’s not get ahead of ourselves.

A few days ago I tried to point our way towards an acceptance of a certain humility.  I used a couple of quotes, one from Durer and one from Soros, to provide us with insights form well-regarded figures separated by a good long period.  Obviously the need to recognize our inherent fallibility has a long history.  The problem is that we keep forgetting and frequently end up acting as if we had some form of ultimate knowledge.  I used the word ‘utopia’ to stand in as a proxy for this illusion.  I was not talking about Utopia the book.

I have made it a long habit not to engage in public with correspondents who critique what I write because my purpose is simply to begin conversations and allow correspondents to create their own conversation.  Most often, and most interestingly to me, they go in very different directions than the one I had on  mind when I wrote.  Such is the richness of our modern ability to discuss in this remote, electronic, fashion.  We learn, or at least I do.  And that is the envigoration that motivates me to continue.

Once in a while, however, someone will say something that requires my public reaction.  My little Durer and Soros missive is one such occasion.

I have three points of contention to respond to.  I will go in the order with which they appeared.

First, I was accused of never had read “Utopia”.  This is a highly personal attack .  It is offensive.  But par for the course.  Quite how anyone could have this knowledge I don’t know.  In any case, as I indicated above, I was not referring to the book.  I was using the colloquial word ‘utopia’ as a placeholder for any body of knowledge that is based on the illusion of perfect knowledge.  Modern economics falls very closely to being such a vision of utopia.  It excludes reality, creates an alternative world, builds theories relating to that alternative world, and then presumes the lessons learned can be imported back to reality with relevance or practical application.  They cannot.  Most economists admit this, yet they continue teaching as if it were true.  The difficulty they face, even after their admission, is that the real world is highly unlikely to contain such pockets of perfection.  And, given its growing complexity, it is becoming ever less likely we would know whether it did.

Perhaps we could truncate this discussion by simply arguing that any body of knowledge that presumes perfection is actually an ideology.  It is a belief system best thought of as a placeholder awaiting substitution by an updated view based on the fallibility that Soros mentions.

Unfortunately in our modern world we are heavily encumbered by such ideologies.  Our poor understanding of the limits of reason and the inscrutability that uncertainty brings us are too often forgotten in our search for answers to questions.   We end up being too confident and often too unaware of the unintended consequences of our actions.

Second, I was then attacked because I asked the question “What new theory is there? Really new?”  This, I thought, was clearly a question pointed at mid-twentieth century economics.  Since about that time economics has been fairly static, contenting itself with embellishment and deep dives into the logic of allocation  and so on.  It has covered over and sanitized the major alternatives that threaten its hegemony and settled into a staid middle/old age.  That doesn’t mean that there isn’t activity.  There is.  But the creation of entirely new avenues of thought are few and far between.

In this second personal attack I was accused, yet again, of not having read a certain text.  In this case Shannon’s theory of mathematical communication.  Again, quite how this accusation can be made in the context of a few remarks I make about fallibility and utopia, I am not sure.

The same correspondent then returns later in the conversation to launch another missile.  This time it is not just me that hasn’t read or understood Shannon, it is the entire profession of economics.  Apparently the obsession that economists have with ‘market forces’ really aggravates this correspondent as he launches the ultimate criticism: economics is stuck in the age of Newton.

Well, there are worse places to be.  Pre-Newtonian fog and superstition being one.

But there are glimmers of sense in this accusation.  Which will, I assure you, bring me back to Kolmogorov.

Economics was born back in a simpler agriculturally dominated era.  Its roots still betray those origins.  It has never been able to reconstruct itself as a body of thought concerning information and its application to raw materials and energy.  It has focused, not on production, but on transactions and the optimal way of organizing them.  To do this its has had to invent for itself a pseudo-psychology in order to get how the demand for goods and services originates, and then how they are purchased.  It is only with the recent advent of behavioral economics that the older strictures about consumer attitudes has been challenged.  And even then the new ideas have not been allowed to retire a re-write of the core of economics.

Focusing on transactions implies that economists have spent a lot of time talking about the information needed to make markets work.  The conversation has been endless.  The limitations on thought brought about by the dependence on equilibrium as an organizing metaphor have severely reduced any interest in information beyond that needed to allow rational agents to arrive at such an equilibrium.  This means that, even though they use the word ‘information’ all the time, few economists actually think about information much.  The list of economists who have discussed information is a veritable who’s who of the discipline.  The problem is that they tend to truncate their enquiry by going straight to the word ‘price’, and because they are only interested in transactions, that is deemed sufficient.  The burden on prices is thus immense.

The key point here though is that information is semantic: a price is a piece of information that has meaning.  It is carrying a message of importance.

It is not what Shannon meant by information.

I am writing this missive on my laptop which contains a large array of what I would call information: letters, emails, articles, and so on.  Those things have meaning to me.  That meaning is what we call information in our vernacular usage of the word.  Shannon has a very different view.  If, for some reason, all those objects were mixed up and their contents smeared about, I would, in my vernacular sense, have less information.  But to Shannon the amount of information has increased.  This is because his notion of information is associated with the effort required to communicate the state of a system.  All that mixed up stuff on my addled computer needs a great deal more effort to communicate, so, according to Shannon, there is more information.  It is little wonder that Shannon ended up in a similar place to Boltzmann when the latter defined entropy in the early 1900s.  Both were dealing with probabilities or choices within populations.  For Boltzmann entropy was the number of equivalent micro states within a system; for Shannon it was the effort required to specify a message, which, in turn, is a function of the number of alternatives messages that could possibly be transmitted.

So here we hit a conundrum.

Information is crucial to understanding economies.  I think that is indisputable.  I regard it as the fundamental ingredient of all economic activity.  But information in an economy is not about communication exclusively.  We are not in the business of studying communication.  We are in the business of studying the order humans create and distribute in the form of the products and services they invent, need, or desire.  Information, in an economy, is an inherently creative phenomenon.  It is redolent with meaning.

Shannon himself warned us about confusing his usage of the word with the everyday version.  He wrote: “‘Information’ here, although related to the everyday meaning of the word, should not be confused with it”.  His version, the communication version, is closely associated with uncertainty, low probabilities, the difficult of transmitting a message, and ultimately with entropy.  In economics we are concerned with what appears to be the exact opposite: we are searching for the certainty that a product will take a particular shape or have particular properties;  we want to avoid surprise by making production and transaction predictable and easily replicable; we want to make production and transaction simple; and we want to instill order into our environment so we can extract the sustenance and pleasure we seek from whatever we create.

Which gets me back to Kolmogorov.

He, like Shannon was grappling with the question of how much information was contained within a given object.  He suggested three angles of analysis: the probabilistic, the combinatorial, and the algorithmic.  Shannon had covered the first two.  It was the third that most caught Kolmogorov’s attention.  It ought to be the one of most interest to economists also.

Why?

Because Kolmogorov goes down the road of trying to describe the information contained, not simply in messages, but in substantial things.  Instead of trying to define information in the context of multiple alternatives states or a population of entities — in the manner of Shannon and Boltzmann — he tried to identify information within a single entity.  His insight was this: the length of the algorithm needed to replicate the entity defined its information content.  The shorter the algorithm the simpler the object.

And this, I think, has enormous value to economics.

Kolmogorov introduced the word ‘complexity’ to describe what he was trying to measure.  The more information, the more complexity.  The two go hand in hand.  An object that can be described by a short algorithm has little complexity.

And here we arrive at a useful tool for analyzing economies and economic entities.  Products and services can be thought of by their algorithmic content.  Simpler products are easier to produce, require less information, and are easily understood.  And so on.

With the explosion of the division of labor, and the associated explosion of information since the industrial era began, our economies and the products and services within them, have risen in complexity.  This rise renders theories rooted within a simpler world irrelevant in that part of the new economy populated by complex entities/objects.

This, I believe is why firms exist.  Using complexity drives the need for organization and management.  Producing the algorithm is a feature of modern economies.  Economics and its obsession with transactions presumes that whatever it is being traded is simple: the activity prior to exchange is trivial compared with the transacting itself.  This attitude instilled within economics an indifference and a lack of comprehension towards production that cripples its ability to engage a modern complex economy.  It sufficed in an agricultural or early industrial setting.  It is obsolete in a modern highly interconnected and complex setting.

This emphasis on algorithmic information content opens up a host of fruitful avenues to explore.  It allows us to anchor economics within the general sphere of information science.  It allows us to create measures of economic complexity.  And it allows us to understand the co-evolution of the complexity of the economy and the institutions, like firms, needed to execute economic activity possible.

Information remains a conundrum in economics, but it isn’t because we haven’t thought about Shannon.