I ordinarily leave the heavy technical posts to writers far more capable than myself; I’m not close enough to the early-release cycle on the latest Frameworks/gismos/gadgets to be able to write the perfect tutorial on anything, even if that’s the way I was inclined.
Rather, my interest lies in understanding the actual import of the technologies we use. What capability does this give us? What can we do with this? How does this bit of tech enable us to solve previously intractable problems? And above all, how does this advance whatever Techno-Utopian agenda we might have packed away with a sigh in some forgotten attic 20 years ago that we can dust off now?
It’s this latter that has me intrigued at present. There was this idea… this throw-away comment that strangely stuck in my mind after a conversation with a seminar attendee in a program I gave many years ago that has never left me, and it goes something like this;
There is an element of “harmonics” in IT systems that has always fascinated me. We see it in mechanical systems, but mostly we see it in biological systems. We recall, dimly, that we once called ourselves “Software Engineers” in the same sense that “Mechanical Engineers” intended it. We built Big Things. And those Big Things had all the properties of large physical systems, like bridges, buildings and so on. But now, it is biological systems which hold the most promise for advancing computer science. Oddly though, our professional nomenclature has not really caught up.
It’s barely apparent now, but the infrastructure of large-scale computer systems is almost invisible. Unless of course you work in a data centre, and you have to wear earplugs and a jacket. These days, stuff just “works”. Computing power is part of the ambiance.
It wasn’t always so. I recall standing in the middle of a large open plan office in Sydney, when the Newcastle Earthquake struck, 250 kilometers away. It sounded and felt like a large truck had crashed into one of the basement pillars in the car park of the building 5 stories below. Within minutes, before the light fixtures had even stopped swaying, we knew, the mainframe computer our organisation used experienced a major system failure across all of its disks. The whole business was down for days.
That was a while ago now, but it taught me a big lesson. Today, it wouldn’t be quite the disaster it was then – oh sure, there would be local failures here and there, but cloud systems and solid-state-everything mean that there is redundancy and resilience in the network.
Yet we have still barely begun the process of securing our global digital infrastructure. And in terms of tapping the real power of all of this connectivity, well, we’ve done almost nothing compared to what is actually possible.
The global digital infrastructure, as I’ve touched on before has the potential at least of emergent properties we can barely dream of – the properties of Big Things. The properties of ecosystems. These emergent properties are not necessarily sentient, or even intelligent in the human sense, not in the grand scheme of things. But they carry latent within them the same sorts of power, resonance and harmonics of all large integrated systems.
For example, imagine a cloud cluster that spanned a continent that worked as one intelligent, self-regulating, self-healing system. Right now we have systems that tie data centres together, and these make use of remote locations to “spread the risk” of data loss to mitigate data centre disasters (which still happen surprisingly often). But we do not yet have a natural, intelligent system for spreading work and data around, based on the needs of the network itself as a harmonious whole.
What would be the characteristics of such a network? It would be introspective, in other words, it would know how it was habitually used. It would monitor and anticipate patterns in the data and processes that moved across it. And it would be able to optimise itself and its internal organisation according to these patterns, as it, and it’s users moved through time and space.
Say this network stored information about getting directions in urban centres; accesses to this type of information might peak in the mid-morning, then taper off, only to surge again in the evening. But come the early hours of the morning, demand falls to nothing. Sensing this, the network would tag certain types of data for caching between certain hours, according to a utility curve. When it was no longer needed, other data, with a higher utility curve would come along and push it out of the cache. Meanwhile, as the sun rose and moved west, a bubble of map data would travel with it, pushing out airport shuttle information from the night before, and old news articles, whatever it was that had dominated the attentions of the network in that place the night before.
What would be the effect of this? Cached data costs less in terms of energy and reduces wear on disk, and the closer the cache to the physical proximity of the consumer, the faster the performance from the user’s perspective. But stale cache data takes up space, and forces system owners to lay on more capacity than is really needed.
Humans can clear it out automatically, but this usually means dumping everything every few weeks. What is needed is for the network to understand itself and its own needs. If it could, the network would cost less to run, and the wasteful gap that exists between capacity needed to meet peak demand and actual demand would be saved everywhere. Fewer server instances would need to be spun up. Less disk would need to be allocated on standby. And less electricity would be needed to power it all and push those wasted electrons around.
The wider the distribution of such a network, the greater the emergent harmonics between the different areas of the network – imagining it, we can see waves of red waste heat dying down, and whole parts of the global infrastructure cooling and working more in tune with the variability of routine demand placed on it.
Harmonic resonances would then begin to appear, as the system began to learn, and was able to anticipate demand. Map data would appear instantly for most users, because most users use it when most other users use it, where they need it, in their precise geographic location. In the evenings, restaurant reviews appear in a flash, though not for all restaurants equally – pancake parlour and all-night breakfast places will not be needed until the evening is quite late.
Of course this amazing degree of intelligence is not free. Especially if we imagine it taking some kind of Giant Brain to coordinate and orchestrate everything. This is in fact precisely what we don’t need. What we need instead is something more on the lines of the intelligence of biological cells. Data can be tagged with meta-data structures like simple sugars that disk controllers and caches love, and which reward them with “energy” when they flood in at the right time, in the right place, and in the right amount. An internal energy economy, implemented with a chemical language that encoded energy values would enable the entire network and all its parts respond to demand, and develop rhythms of consumption and metabolism. We would experience this as breathtaking responsiveness and almost animal intelligence. But intelligence on the order of Man’s Best Friend, not some hateful intelligence empowered to monitor and track us. We would avoid the dangers of entrusting the network with human value judgements.
We haven’t even scratched the surface of what is possible with intelligent design. It will take a radical change in the way we think of “intelligence” however to realise the power of emergent behaviors in massively integrated systems.