APT research areas
Grand Challenges: Building a synthetic sensory-motor system
Leslie S. Smith, Department of Computing Science and Mathematics, University of Stirling. Email: lss@cs.stir.ac.uk
October 2002.
What is “a synthetic sensory-motor system”?
Living entities sense their environment, and move around in that environment in order to achieve various goals. The sensory part detects some form of variable flux (such as changing energy fields or electrical or material gradients) imp inging on sensors, and the motor part refers to moving part or all of the system around. Sensing the environment, and moving around in it are not separate: movement moves the sensors, and alters what they receive. Sensory systems are very varied: at one extreme, micro -organisms directly sense concentration gradients, and move within them, and at the other extreme, eye and head movement is a crucial aspect of human visual perception. What matters is that the movement and the sense organ are integrated in time: sensory-motor systems are real-time systems.
A synthetic sensory-motor system is like the biological sensory-motor systems, but built, rather than grown. Animals have developed senses based on pressure waves (hearing, bat sonar), light (vision), trace element gradients in air or water (smell, taste), for example. These senses are adapted to the needs of the animal, and its ecological niche: for example, we are sensitive to sound sources, rather than reflections, and to light reflections, rather than light sources. In a synthetic system, the needs of the animal become the aims of the system, and the ecological niche becomes the design environment. If we can understand the nature of the relationship between the sensorimotor system and the rest of the system, we can design entirely synthetic sensory-motor systems, allowing us to design systems to work in a very wide range of environments. Indeed, we would suggest that synthetic sensory-motor systems will be the underlying user interface technology of the 21st century.
Why build “a synthetic sensory-motor system” ?
We propose the development of a sensory-motor system that can work in a novel (and indeed, possibly unknown) environment. This system takes the variable flux (whether energy variations or materia l gradients) and allows action (or interpretation) based on these. I present two types of justifications for building such systems. The first is perhaps extreme, whereas the others are more workaday.
The grandest challenge of all…
The real grand challenge for life on Earth is, as it has always been, to conquer new ecological niches. And this challenge has been taken up successfully from the seas to the surface of the earth, to the skies. But nature’s evolutionary techniques imply a gradual colonisation of new ecologies: and there are limits to the ecological niches that traditionally evolving living entities can colonise. Life has reached these limits: there may be other colonisable places in the Universe, but the vast emptiness of space between here and there presents an unbridgable gap for traditional evolution.
Yet there is energy available from sunlight within some parts of space to support life, though the distances between the energy sources is very large. We can build electronic systems which can survive these low energy periods, and perhaps transport them to other stellar systems. But what is gained by simply transporting automata? Is this “the conquering of space”?
One possible solution is building truly enormous spaceships with complete ecologies, to travel space. Unfortunately, the time problem remains intractable: the science-fiction scenario of generations upon generations living on such ships does not seem attractive unless one is fleeing from some cataclysmic event on Earth. Rather than waiting for such events, let us consider an alternative. What if we could design/evolve some form of cyborg, an entity which
- had the longevity of electronics
- had the capability of evolutionary reproduction to permit colonisation of novel energy landscapes so that it could really move what we might consider as life to other planets?
H:\ Research\Proposals\grandchallenges\artificial_sensing.doc -1
Altogether this is a very tall order. But the outcome is very far-reaching. Realistically, and within the terms of this call, there are many “grand challenges” within this overall “grandest challenge”.
Apart from the challenge described here, the other grand challenges required are
- “Humanising” electronics. Really this means building (or perhaps growing) minds that can interface with the sensors and energy providers that are appropriate to novel environments. (Indeed, this begs the question of what a mind is, if we are considering building one, or alternatively begs many ethical questions if we are considering growing one.)
- Developing energy transducers for whatever energy sources are available, and energy stores for
periods when the energy is not available. These other challenges are not in the Computing field: they would be necessary for this whole system to come together.
I am proposing a designed system here. However, the designed element is really intended to jump over what seems otherwise to be an insurmountable gap through space to other perhaps more hospitable environments. A more holistic view might place our capability to design such a system as nature’s answer to the evolutionary leap problem: that is, to suggest that having conquered all the available spaces on Earth, nature “designed” human intelligence and ingenuity to allow life to be transported across the huge divide separating Earth from other habitable planets. It is in this sense that transporting life to other parts of the Universe is the grandest challenge of them all.
Sensory-motor systems for more autonomous systems.
Even excluding space, there are plenty of ecological niches which are very unfriendly to humans. For example, the deep sea is largely unexplored, yet we have oil-rigs whose legs rest on the seabed. Autonomous inspection systems could both explore these areas, and investigate problems with particular installations. Understanding the nature of sensorimotor systems and building appropriate ones would allow exploitation of this technology.
Ubiquitous computing clearly makes huge demands on the user interface. One way of attempting to solve this is to design machines which can learn which aspects of their environment to react to. This requires intelligent sensory systems, and we suggest that adaptive sensory systems are one way forward here. These systems can help to provide seamless integration between the ubiquitous computers and their natural and computing environment. Simply expecting to program this in is certain to lead to brittle systems.
Prosthetics for repairing and enhancing sensory systems.
Synthetic sensory-motor systems can provide one part of the solution to the problem of direct interfacing of assistive technology. (The other part is the actual transmission and receipt of nerve signals, without compromising the longevity of the neural tissue.) Basic systems already exist (most notably cochlear implants, but there is also research on retinal implants and direct stimulation of both brainstem and visual cortex), however, they are unidirectional (sensing system to neural tissue), and the neural stimulation they provide is relatively simple. This project would provide one of the technologies required to underpin bi-directional external sensory systems.
H:\ Research\Proposals\grandchallenges\artificial_sensing.doc -2
What do we propose building?
We propose to build integrated sensory-motor systems that work in real environments. These will receive a variety of different sensory inputs, and will be able to move part or all of themselves in order to alter this input.
We propose that these systems should learn from the signals impinging on their sensors what form their sensory system should take. The systems would be armed with a number of different types of sensor, some complex (such as artificial retinas, artificial cochleae, ultrasound detectors, infra-red sensors), and some perhaps simpler (pressure sensors), and that the systems learn to interpret their sensory environment by moving in their environment, thus changing the sensory stimuli. Although some aspects of the sensory -motor processing will be built in, most will be adapted by using the effect of movement on the sensing system. In this way the system will be able to make the most of the sensory information it receives.
How does this differ from plain robotics? The overall aim is that the sensory-motor technology be part of a larger overall system, part of the system’s environment (or user) interface. What makes this proposal new is the ability to detect and adapt to novel forms of energy flux. This proposal is the first step towards much more autonomous systems, systems that can realistically interpret an unpredictable environment. This is the enabling technology for much more effective ubiquitous usage of computing technology. Such capabilities are a prerequisite for building long-lived semi -autonomous systems, capable of providing robust ubiquitous computing where the design environment is not fixed, nor entirely specified, as well as for more fully autonomous systems, (including as self-repairing systems) which would be required for more challenging applications.
H:\ Research\Proposals\grandchallenges\artificial_sensing.doc -3