#Personalizing Tech: Physiological Computing Paves the Way for Tailored Experiences

Physiological computing is a form of human-computer interaction that depends on measuring and responding to the physiological activity of the user in real-time which can transform how we engage with the AI systems. By enabling the AI to sense the implicit signals of the user we give the computer artificial emotional intelligence.

The normal way to interact with generative AI models is very asymmetrical: while the computer is able to output vast amounts of audiovisual information quickly, the input from the user is limited to the relatively low bandwidth provided by mouse and keyboard. Furthermore, while the user has access to the internal state of the computer system (e.g., memory consumption and processor utilization levels), the AI has no information on the cognitive and emotional state of the user. Physiological computing allows symmetry in terms of both information bandwidth (the added input modalities in the form of physiological signals) and the user state and context information derived from the physiological data.

Also the normal way of using AI systems such as ChatGPT is completely explicit: the AI reacts only to explicit commands given to it by the user. Combined with physiological computing the AI model can continuosly measure the user’s states and needs and respond accordingly, even without being explicitly instructed.

Emotionally Adaptive Gaming

My research career in physiological started back in 2006 when I was tasked to create emotionally adaptive games as part of a large EU project. My first creation was a version of Tetris that adapted to the user’s electrodermal activity, or simply skin conductance, a signal that is widely used in physiological computing and measures sweating related to emotional arousal, or what might be colloaquilly be called ‘sweaty palms’.

In the case of Tetris I connected the EDA signal to the speed the blocks were falling and thus created my first-ever biocybernetic loop: if I set the game to get easier when the player got more anxious nothing much happened, but when the game got MORE difficult with arousal, the system quickly entered in a spiral of person getting even more nervous which lead to game getting even harder which then again increasing difficulty and quicly ending the game.

pic eda.

Measuring the implicit feedback from the user has implications for several application areas. User interface designers can automatically evaluate their designs: mental workload of drivers (Solovey et al., 2014) can be measuredin real time to evaluate the cognitive demands of different interface designs. Implicit signals can also be used for automated annotation of data: the computer can automatically detect whether a document contains relevant information (Eugster et al., 2014; Barral et al., 2015; Ruotsalo et al., 1) and what kind of affective reaction the content generated in the user.

Go through the signals

Eda

ECG

…

Cover of special edition of Computer, edited by group of three pioneers of the topic: Prof Giulio Jacucci (my PhD supervisor) and leader of the Ubiquitous computing research group where I worked for 6 years, Erin Solovey Professor of Computer Science (my PhD thesis pre-examiner) and Stephen Farclough Professor of Psychophysiology(my opponent.)

Psychophysiology

jee