Brain Computer Interface in BMI

1. Brain Computer Interface in BMI Ioannis Papavasileiou Computer Science & Engineering Department The University of Connecticut 371 Fairfield Road, Unit 4155 Storrs, CT 06269-2155 papabasile@engr.uconn.edu

2. What is BCI? • BCI is: • System that allows direct communication pathway between human brain and computer • It consists of data acquisition devices, and appropriate algorithms • How is it used in BMI: • Clinical research • Disease-condition detection and treatment • Human computer interfaces for • Control • Emotions detection • Text input - communication

3. Research areas involved • Computer science • Data mining • Machine learning • Human computer interaction • Neuroscience • Cognitive science • Engineering

4. Key challenges • Technology-related: • Sensor quality – low SNR • Supervised learning – “curse of dimensionality” • System usability • Real-time constraints • Non-invasive EEG information transfer rate is approx. 1 order of magn. lower • People-related • People are not always familiar with technology • Preparation – training phases are not fun! • Concentration, attention consciousness levels • Task difficulty

5. BCI components • Data acquisition • Electroencephalography (EEG) • Electrical activity recording • Invasive or not • Functional Near Infrared Spectroscopy (fNIRS) • Recording of infrared light reflections of the brain • Functional magnetic resonance imaging (FMRI) • Detection of changes in blood flow • Data Analysis • Data mining & machine learning • Decision making • Output & Control • HCI

6. Typical BCI architecture

7. Electroencephalography (EEG) • What is it: • Recoding of the electrical activity of the brain • Types: • Invasive • Non-invasive • Properties: • High temporal resolution • Low spatial resolution • Scalp acts as filter!

8. International 10-20 standard • Electrodes located at the scalp at predefined positions • Number of electrodes can vary

9. The EEG waves • Alpha – occipitally • Beta – frontally and parietally • Theta – children, sleeping adults • Delta – infants, sleeping adults

10. fMRI • Functional magnetic resonance imaging • Fact: • Cerebral blood flow and neuronal activation coupled • Detection of blood flow changes • Use of magnetic fields • High spatial resolution • Low temporal resolution • Clinical use: • Assess risky brain surgery • Study brain functions • Normal • Diseased • Injured • Map functional areas of the brain

11. fNIRS • Functional Near Infrared Spectroscopy • Project near infrared light into the brain from the scalp • Measure changes in the reflection of the light due to • Oxygen levels associated with brain activity • Result absorption and scattering of the light photons • Used to build maps of brain activity • High spatial resolution • <1 cm • Lower temporal resolution • >2-5 seconds

12. BMI & clinical applications • Diagnose: • Epilepsy – seizures • Brain-death • Alzheimer’s disease • Physical or mental problems • Study of: • Problems with loss of consciousness • Schizophrenia (reduced Delta waves during sleep) • Find location of: • Tumor • Infection • bleeding Source: http://www.webmd.com/, http://www.nlm.nih.gov

13. Sleep disorders & mental tasks • Sleep disorders study • Insomnia • Hypersomnia • Circadian rhythm disorders • Parasomnia (disruptions in slow sleep waves) • Mental tasks monitoring • Mathematical operations • Counting • Etc.

14. Neurofeedback • Applications in • Autistic Spectrum Disorder (ASD) • Anxiety • Depression • Personality • Mood • Nervous system • Self control

15. Feedback EEG-BCI architecture

16. Typical data analysis process • Data acquisition and segmentation • Preprocessing • Removal of artifacts • Facial muscle activity • External sources, like power lines • Feature extraction • Typically sliding window • Time-frequency features • Latency introduced

17. Feature extraction • Model-based methods • Require selection of the model order • FFT (Fast Fourier Transform) – based methods • Apply a smoothing window • Features used: • Specific frequency band power • Band-pass filtering and squaring • Autoregressive spectral analysis • Many times a feature selection or projection is done to reduce the huge feature vectors

18. Data Classification • Typical classifiers used • Artificial Neural Networks (ANN) • Linear Discriminant analysis (LDA) • Support Vector Machines (SVM) • Bayesian classifier • Hidden Markov Models (HMM) • K-nearest neighbor (KNN) • Parameters for each classifier can affect the performance • # of hidden units in ANN • # of supporting vectors for SVMs • Etc.

19. Human computer interaction • BCIs are considered to be means of communication and control for their users • HCI community defines three types: • Active BCIs • Consciously controlled by the user • E.g. sensorimotor imagery (multi-valued control signal) • Reactive BCIs • Output derived from reaction to external stimulation • Like P300 spellers • Passive BCIs • Output is related to arbitrary brain activity • E.g. memory load, emotional state, surprise, etc. • Used in assistive technologies and rehabilitation therapies

20. BCI & Assistive Technologies • Communication systems • Basic yes/no • Character spellers • Virtual keyboards • Control • Movement imagination • Cursor • Wheelchairs • Artificial limbs & prosthesis • Automation in smart environments • Current BCI systems have at most 10-25 bits/minute maximum information transfer rates • It can be valuable for those with severe disabilities

21. P300 spellers • Most typical reactive BCI • 3-4 characters / min with 95% success

22. P300 wave • Event related potential (ERP) • Elicited in the process of decision making • Occurs when person reacts to stimulus • Characteristics: • Positive deflection in voltage • Latency 250 to 500 ms • Typically 300 ms • Close to the parietal lobe in the brain • Averaging over multiple records required

23. Other ERP uses • Lie detection • Increased legal permissibility • Compared to other methods • ERP abnormalities related to conditions such as: • Parkinson’s • Stroke • Head injuries • And others • Typical ERP paradigms • Event related synchronization (ERS) • Event related de-synchronization (ERD)

24. Other Control BCI paradigms • Lateralized readiness potential • Game control • 1~2 seconds latency • Negative shift in EEG develops before actual movement onset • Steady-state visually evoked potentials (SSVEPs) • Slow cortical potential (SCP) • Imaged movements affect mu-rhythms • They shift polarity (+ or -) of SCP • Sensorimotor cortex rhythms (SMR) • EMG

25. SCP & SMR vs P300 • Typically SCP and SMR BCIs require significant training to gain sufficient control • In contrast P300 BCIs require less as they record response to stimuli • However, they require some sort of stimuli like visual (monitor always in place) or audio • Also SCP BCIs have longer response times

26. Binary speller control • User imagines movement of cursor • Typically hand movement • The goal is to select a character

27. Wheel chair control • All the mentioned BCI paradigms have been applied to wheelchair control • Either using a monitor for feedback • Or active paradigms as sensorimotor imagery (SMR) • Similar approaches have been applied to robotics • Artificial limbs • etc

28. Environment control • BCIs used by disabled to improve quality of life • Operation of devices like • Lights • TV • Stereo sets • Motorized beds • Doors • Etc • Typically use of P300, SMR and EMG related BCIs

29. EMG-based human-robot interface example • Motion prediction based on hand position • EMG pattern classification as control command • Combination of both yields motion command to prosthetic hand

30. Emotions detection • Use of facial expressions to imply user emotions • ERD/ERS based BCIs • Emotional state can change the asymmetry of the frontal alpha • P300 - SSVEP • Emotional state can change the amplitude of the signal from 200ms after stimulus presentation

31. BCIs for recreation • Games • EPOC headset • Mindset • Virtual reality • Outputs of a BCI are Shown virtual environment • Creative Expression • Music • Generated form EEG signals • Visual art • Painting for artists who are locked in as a result of ALS – amyotrophic lateral sclerosis

32. Security and EEG • EEG has been used in user authentication • Every brain is different • Different characteristics of EEG waves are used in user authentication • Pros • User has nothing to remember • Harmless • Automatically applied • Cons • User has to wear an EEG headset • Accuracy is still not 100% • Still not used in practice

33. Thank you!