I'm pleased to thank the following institutions for generous financial support:

 

 

 

 

 

"eSMCs - Extending Sensorimotor Contingencies to Cognition"

The majority of current robot architectures is based on a “perception-then-action” control strategy. In this project, we will adopt a theoretical perspective that turns this classical view upside-down and emphasizes the constitutive role of action for perception. The key concept our project is based on is that of sensorimotor contingencies, that is, law-like relations between actions and associated changes in sensory input. We will advance this concept further and suggest that actions not only play a key role for perception, but also in developing more complex cognitive capabilities. We suggest that extended sensorimotor contingencies (eSMCs) may be exploited for the definition of object concepts and action plans and that their mastery can lead to goal-oriented behaviour. The project pursues the following objectives: We will employ this approach to establish computational models that are suitable as controllers for autonomous robots; we will implement these eSMCs-based models on robotic platforms with different sensor-actuator equipment; we will investigate learning and adaptivity of eSMCs in artificial systems, focussing on sensorimotor interactions, object recognition and action planning; we will investigate and validate the concept of eSMCs in natural cognitive systems, by carrying behavioural and neurophysiological studies on healthy human subjects; finally, we will test predictions derived from this concept in patients with movement dysfunctions, where ensuing changes in perceptual and cognitive processing will be tested. A set of benchmarks and task scenarios will be developed serving as demonstrators for the enhanced performance of artificial systems based on the eSMCs approach. Moreover, the usefulness of the approach for the development of applications in augmenting human behaviour will be demonstrated.

 

Below please find a list of the laboratories involved in this project:

This collaboration is supported by the EUROPEAN COMMISSION, 7th Framework Programme on Research, Technological Development (STREP funding).

 

 

 

"Does the study of simple visual stimuli assess the primitives of natural vision?"

In our natural daily experiences, we are constantly bombarded with a multitude of stimuli. Our sensory systems immediately and effortlessly make sense out of such complex scenes. However, the mechanisms underlying this remarkable performance are still largely unknown.
The Reverse Hierarchy Theory for visual processing (RHT), proposed by Hochstein & Ahissar suggests that the immediate first high-level scene interpretation - the gist of the scene - is based on a crude first pass of information processing. This high level representation, in turn, guides subsequent detailed analysis. Colloquially, we first recognize the forest, which helps in detailed analysis of the trees. A large body of experimental data based on well-controlled artificial visual stimuli supports this theory.
Using techniques of observing eye movements, measuring cortical activity and using advanced signal analysis tools, we shall test predictions of the RHT under natural conditions. By designing a unique set of visual stimuli, we shall decipher which aspects of visual stimuli are crucial for determining the first percept. Furthermore, we investigate whether high spatial frequencies are processed at a slower rate and do not contribute to this early percept. Alternatively , as suggested by RHT, less informative regions could be discarded irrespective of the (local) spatial frequency content. This would lead to a task driven, not stimulus driven, control of gaze movements right from the start of visual analysis. Comparing processing patterns and cortical activity under different conditions, we shall be able to decipher between task and stimulus related processing patterns and determine the dominant direction of processing during detailed analysis of sensory stimuli.
This approach stall be applied to assessing whether visual impairments of dyslexic individuals are task or stimulus related. Taken together this study will deepen our understanding of the factors that determine the immediate and intuitive percept of normal individuals and as well as of dylexics whose high-level visual processing is somewhat impaired.

Contact: Prof. Dr. Merav Ahissar, The Hebrew University (Dept. of Psychology)

This collaboration is supported by the Niedersächsisch-israelisches Gemeinschaftsvorhaben.

 

 

 

Aufmerksamkeitssteuerung in visueller Wahrnehmung
durch sprachliche Ausdrücke, Vorwissen und Affekte

Project summary:

Wir nutzen in dieser Untersuchung auditorische Signale, um den Kontext für die visuelle Verarbeitung zu definieren. Wir präsentieren komplette sprachliche Äußerungen, Satzkomplemente und Geräusche in einem „visual world“ Paradigma. Die Kongruenz der auditorischen und visuellen Stimuli und Merkmale der auditorischen Stimuli werden variiert. Auf diese Weise wird systematisch die explizit verfügbare Information, Aufmerksamkeit und der emotionale Status des Probanden beeinflusst.
Gleichzeitig messen wir Augenbewegungen und EEG (in Kooperation mit der Arbeitsgruppe Psychologie).
Der Einfluss der oben genannten unabhängigen Variablen auf die Korrelation von Fixationspunkten und einfachen visuellen Features, auf die Latenz und Lateralisierung der evozierten Potentiale und der induzierten Aktivität lassen direkte Schlüsse auf die Art und den Ort der Interaktion von bottom-up und top-down Signalen und auf kontextabhängige Adaption zu.
Das Projekt wird gemeinsam mit dem Arbeitsbereich Linguistik ausgeführt.

 

Below please find a list of the laboratories involved in this project:

This collaboration is supported by the University of Osnabrück and the MWK (Niedersächsisches Ministerium für Wissenschaft und Kultur).

 

 

 

SF - Synthetic Forager

Project summary:

The single overarching goal of the SF consortium is to identify the neuronal, cognitive and behavioural principles underlying optimal foraging in rodents and to implement these principles in a real-world foraging artefact or the Synthetic Forager (SF.01). SF.01 constitutes a novel biologically based cognitive technology for autonomous exploration and foraging in real-world man-made indoor and outdoor environments. SF exploits our growing understanding of exploration and foraging behaviour in rodents, advances current theories of the neuronal and behavioural organization of foraging and transfers this understanding towards the construction of novel real-world synthetic cognitive technologies.
The behaviour and neurophysiology of foraging will be studied in rodents behaving in automatically controlled multi-modal environments. The physical features of these environments can be fully controlled in real-time in relation to the behavioural and/or physiological state of the animal using an advanced experimental technology developed by the consortium. The overall integration of the perceptual, cognitive and behavioural control systems of SF will be accomplished using a well established robot based cognitive architecture, called Distributed Adaptive Control (DAC) further informed by the formal analysis of rodent foraging. The perceptual, cognitive and behavioural control systems of SF will be based on statistical analysis and detailed game theoretic models of the behavioural and neurophysiological data. The SF control systems are validated against the behavioural and physiological data. The SF phenotype comprises a high-mobility robotic platform equipped with visual, auditory, olfactory and tactile sensors. The SF will be evaluated in a number of stringent benchmarks ranging from robot equivalents of rodent foraging tasks to simulated de-mining.

Although the goal of the project is to demonstrate the SF technology for autonomous exploration and foraging, we expect that the approach and technologies developed in SF will have long-term implications to a number of other application areas including: cleaning robots, search and rescue systems, terrestrial and planetary exploration, delivery systems, autonomous transportation systems, military intelligence and battle field information control systems, environmental monitoring, internet information analysis and retrieval, information and communication networks and humanitarian de-mining.

 

Below please find a list of the laboratories involved in this project:

This collaboration is supported by the European Community / Federal Office for Education and Science (Bundesamt für Bildung und Wissenschaft).

 

Video lectures of the Synthetic Forager project are also available on YouTube.

 

 

 

"feelSpace"
Sensory enhancement – learning a new sense
Erweitertes Sensorium – Lernen eines neuen Sinnes

Project summary:

In den letzten Jahren ist die Beziehung der statistischen Eigenschaften natürlicher Stimuli zu der sensorischen Verarbeitung in den Mittelpunkt des Interesses gerückt. Viele Eigenschaften der Nervenzellen in frühen sensorischen Arealen können aus dem Blickwinkel der Signalverarbeitung als optimale Repräsentationen natürlicher Stimuli verstanden werden. Diese bilden sich während der Entwicklung durch aktivitätsabhängige Plastizität. Vor Kurzem haben enaktive Theorien der Wahrnehmung vorgeschlagen, dass dieses Schema nicht nur für die sensorische Verarbeitung, sondern auch für den bewussten Wahrnehmungsprozess als solchen gilt: Die Qualität der Wahrnehmung sensorischer Signale wird durch das erworbene Wissen über die systematischen Änderungen der afferenten Signale als Konsequenz eigener Handlungen und Interaktionen mit der Umwelt bestimmt. Dieses Konzept der sensorimotorischen Kontingenzen hat durch Studien sensorimotorischer Substitution experimentelle Unterstützung erfahren.

In einer Pilotstudie haben wir, um einige problematische Aspekte sensorimotorischer Substitution zu vermeiden, ein erweitertes Sensorium als neues experimentelles Paradigma untersucht. Wir konnten zeigen, dass es durch die Einführung neuer sensorimotorischer Kontingenzen möglich ist physiologische Reflexe und die Qualität der Wahrnehmung zu beeinflussen. Dies belegt die Rolle sensorimotorischer Kontingenzen für die bewusste Wahrnehmung.

In diesem Projekt verbessern wir wichtige Aspekte des neuen Paradigmas mit erweitertem Sensorium und führen quantitative Maße der Integration mit den anderen Sinnesmodalitäten und dem Verhalten ein. Wir untersuchen das Substrat der neu erworbenen sensorimotorischen Kontingenzen mit physiologischen Techniken. Erwachsene Versuchspersonen erhalten durch vibrotaktile Stimulation in der Taille Information über ihre Orientierung von einem Kompass. Die Integration dieser Information mit dem Verhalten wird mit einer Navigationsaufgabe in einer großen virtuellen Umgebung untersucht. Mit EEG und fMRI Messungen charakterisieren wir die Lernvorgänge und das physiologische Substrat der Wahrnehmungsänderungen. Die Änderungen der subjektiven Wahrnehmung werden dokumentiert und mit den psychophysischen und physiologischen Maßen korreliert. Zusammenfassend, dieses Projekt bietet die einzigartige Möglichkeit das neuronale Substrat sensorimotorischer Kontingenzen, ihr Einfluss auf das Verhalten und die bewusste Wahrnehmung miteinander in Beziehung zu setzen.

 

Below please find a list of the laboratories involved in this project:

This collaboration is supported by the DFG.

 

On 18th October the BBC reported on the NBP feelSpace project (takes a little scrolling!). Details about the functioning of the feelSpace-belt can also be watched on YouTube.

 

 

last update: 28th February 2011