A simultaneously “recalled” ’real-time fMRI and EEG neurofeedback’ by Rostyslav Sklyar

YNIMG-10449; No. of pages: 11; 4C: 3, 4, 7, 8, 9 NeuroImage xxx (2013) xxx–xxx

Contents lists available at SciVerse ScienceDirect

NeuroImage journal homepage: www.elsevier.com/locate/ynimg

Review

Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback Vadim Zotev a, Raquel Phillips a, Han Yuan a, Masaya Misaki a, Jerzy Bodurka a, b,⁎ a b

Laureate Institute for Brain Research, Tulsa, OK, USA College of Engineering, The University of Oklahoma, Tulsa, OK, USA

a r t i c l e

i n f o

Article history: Accepted 30 April 2013 Available online xxxx Keywords: Neurofeedback Real-time fMRI EEG EEG–fMRI rtfMRI–EEG neurofeedback Emotion Amygdala Frontal EEG asymmetry

a b s t r a c t Neurofeedback is a promising approach for non-invasive modulation of human brain activity with applications for treatment of mental disorders and enhancement of brain performance. Neurofeedback techniques are commonly based on either electroencephalography (EEG) or real-time functional magnetic resonance imaging (rtfMRI). Advances in simultaneous EEG–fMRI have made it possible to combine the two approaches. Here we report the ﬁrst implementation of simultaneous multimodal rtfMRI and EEG neurofeedback (rtfMRI–EEG-nf). It is based on a novel system for real-time integration of simultaneous rtfMRI and EEG data streams. We applied the rtfMRI– EEG-nf to training of emotional self-regulation in healthy subjects performing a positive emotion induction task based on retrieval of happy autobiographical memories. The participants were able to simultaneously regulate their BOLD fMRI activation in the left amygdala and frontal EEG power asymmetry in the high-beta band using the rtfMRI−EEG-nf. Our proof-of-concept results demonstrate the feasibility of simultaneous self-regulation of both hemodynamic (rtfMRI) and electrophysiological (EEG) activities of the human brain. They suggest potential applications of rtfMRI–EEG-nf in the development of novel cognitive neuroscience research paradigms and enhanced cognitive therapeutic approaches for major neuropsychiatric disorders, particularly depression. © 2013 Elsevier Inc. All rights reserved.

Contents Introduction . . . . . . . . . . . . . . . . Methods . . . . . . . . . . . . . . . . . . Integration of simultaneous rtfMRI and EEG Experimental procedure . . . . . . . . . Data acquisition . . . . . . . . . . . . . Real-time data processing . . . . . . . . fMRI data analysis . . . . . . . . . . . . EEG data analysis . . . . . . . . . . . . EEG-informed fMRI analysis . . . . . . . Effects of EEG artifacts . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . Appendix A. Supplementary data . . . . . References . . . . . . . . . . . . . . . . .

. . . . . . data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Introduction

⁎ Corresponding author at: Laureate Institute for Brain Research, Tulsa, OK, USA. E-mail address: jbodurka@laureateinstitute.org (J. Bodurka).

Neurofeedback is a general methodological approach that uses various neuroimaging techniques to acquire real-time measures of brain activity and enable volitional self-regulation of brain function. The development of real-time functional magnetic resonance imaging

Please cite this article as: Zotev, V., et al., Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback, NeuroImage (2013), http://dx.doi.org/10.1016/j.neuroimage.2013.04.126

V. Zotev et al. / NeuroImage xxx (2013) xxx–xxx

(rtfMRI) (Cox et al., 1995), in which fMRI data processing and display keep up with MRI image acquisition, has made it possible to implement rtfMRI neurofeedback (e.g. deCharms, 2008; Sulzer et al., 2013; Weiskopf et al., 2004). rtfMRI neurofeedback (rtfMRI-nf) allows a subject inside an MRI scanner to watch and self-regulate blood-oxygenationlevel-dependent (BOLD) fMRI activity in target region(s) of his/her own brain in what is experienced as real time. Studies performed over the past decade demonstrated the feasibility of rtfMRI-nf-based selfregulation of various localized brain regions, including the dorsal anterior cingulate cortex (Weiskopf et al., 2003), rostral anterior cingulate cortex (deCharms et al., 2005), auditory cortex (Yoo et al., 2006), anterior insular cortex (Caria et al., 2007; Ruiz et al., 2013), inferior frontal gyrus (Rota et al., 2009), supplementary motor area (Subramanian et al., 2011), subgenual anterior cingulate cortex (Hamilton et al., 2011), amygdala (Zotev et al., 2011), orbitofrontal cortex (Hampson et al., 2012), primary motor cortex (Berman et al., 2012), and others. Implementations of rtfMRI-nf for regulation of extended networks of brain areas defined using either functional localizers (e.g. Johnston et al., 2010; Linden et al., 2012) or support vector classification (LaConte, 2011; Sitaram et al., 2011) have also been reported. In contrast to rtfMRI, which has temporal resolution equal to fMRI repetition time TR (order of a few seconds), electroencephalography (EEG) has millisecond temporal resolution and can record electrophysiological brain activity as it evolves in actual real time. EEG neurofeedback (EEG-nf) allows a subject to control certain characteristics of his/her own electrical brain activity as measured by EEG electrodes connected to the scalp. EEG-nf has a longer history and more reported applications to various patient populations than rtfMRI-nf. Some examples include: the sensorimotor rhythm (SMR) EEG-nf for treatment of epilepsy and seizure disorders (e.g. Sterman, 2000; Sterman and Friar, 1972); the SMR-theta and beta–theta EEG-nf for treatment of attention-deficit/ hyperactivity disorder (e.g. Gevensleben et al., 2009; Levesque et al., 2006; Lubar and Lubar, 1984); the alpha–theta EEG-nf for treatment of substance use disorders (e.g. Peniston and Kulkosky, 1989; Sokhadze et al., 2008); the alpha–theta EEG-nf for deep relaxation (e.g. Egner et al., 2002) and creative performance enhancement (e.g. Gruzelier, 2009); the upper-alpha EEG-nf for cognitive enhancement (Hanslmayr et al., 2005; Zoefel et al., 2011); the frontal asymmetry EEG-nf for emotion regulation (Allen et al., 2001); and the high-beta EEG-nf for treatment of major depressive disorder (MDD) (Paquette et al., 2009). The development and advances in simultaneous EEG–fMRI technique (e.g. Mulert and Lemieux, 2010), in which a subject wears an EEG cap inside an MRI scanner and EEG recordings are performed concurrently with fMRI data acquisition, have opened up new possibilities for neurofeedback research. Simultaneous EEG–fMRI provides the following important opportunities in the context of brain neuromodulation. First, electrophysiological correlates of rtfMRI-nf can be explored using EEG data recorded simultaneously with rtfMRI-nf training. Second, performance of EEG-nf can be validated based on fMRI data acquired simultaneously with EEG-nf training. Third, rtfMRI-nf can be dynamically modified using the simultaneously measured EEG activity. Finally, simultaneous multimodal rtfMRI–EEG neurofeedback can be provided to a subject to enable simultaneous self-regulation of both hemodynamic (rtfMRI) and electrophysiological (EEG) brain activities. Here we report the first implementation of simultaneous multimodal rtfMRI–EEG neurofeedback (rtfMRI–EEG-nf) and its proof-of-concept application in training of emotional self-regulation. Our implementation of rtfMRI–EEG-nf is based on a novel, first-of-its-kind real-time integration of rtfMRI and EEG data streams for the purpose of brain neuromodulation. During the experiment, healthy volunteers performed a positive emotion induction task by evoking happy autobiographical memories while simultaneously trying to regulate and raise two neurofeedback bars (rtfMRI-nf and EEG-nf) on the screen. The rtfMRI-nf was based on BOLD activation in a left amygdala region-of-interest (ROI), similar to our previous study of emotional self-regulation that used only

rtfMRI-nf (Zotev et al., 2011). The EEG-nf, provided simultaneously with the rtfMRI-nf, was based on frontal hemispheric (left–right) EEG power asymmetry in the high-beta (beta3, 21–30 Hz) EEG frequency band. Frontal EEG asymmetry is an important and widely used EEG characteristic of emotion and emotional reactivity (e.g. Davidson, 1992). It has been interpreted within the framework of the approach−withdrawal hypothesis (e.g. Davidson, 1992; Tomarken and Keener, 1998), which suggests that activation of the left frontal brain regions is associated with approach (i.e. higher responsivity to rewarding and positive stimuli), while activation of the right frontal regions is associated with withdrawal (i.e. tendency to avoid novel and potentially threatening stimuli). Brain activation is typically quantified by a reduction in alpha EEG power. The approach–withdrawal hypothesis applies to both emotional trait properties and emotional state changes in response to stimuli (e.g. Coan and Allen, 2004; Davidson et al., 1990; Sutton and Davidson, 1997). Numerous EEG studies have indicated that depression and anxiety are associated with reduced relative activation of the left frontal regions and increased relative activation of the right frontal regions (e.g. Thibodeau et al., 2006; Tomarken and Keener, 1998). Thus, frontal EEG power asymmetry is a natural target measure for EEG-nf aimed at training of emotional self-regulation, particularly in MDD patients. Two studies have previously employed EEG-nf paradigms involving frontal EEG asymmetry. Allen et al. (2001) used EEG-nf based on the frontal EEG asymmetry in the alpha band for a group of healthy participants. They observed systematic changes in the asymmetry as the training progressed and associated changes in self-reported emotional responses. Paquette et al. (2009) applied EEG-nf based on EEG power in the high-beta band measured at two frontal and two temporal sites and used it in combination with psychotherapy sessions for a group of MDD patients. They reported a significant reduction in MDD symptoms associated with a significant decrease in high-beta EEG activity within the right frontal and limbic regions. This work followed up on the results of an earlier study (Pizzagalli et al., 2002) that demonstrated that MDD patients exhibited significantly higher resting EEG activity in the right frontal brain regions than healthy controls specifically in the high-beta band. The psychoneurotherapy (Paquette et al., 2009) led to significant changes in the high-beta EEG power asymmetry between the corresponding brain regions on the left and on the right. In the present work, we implemented the EEG-nf based on the frontal EEG asymmetry in the high-beta band (21–30 Hz) rather than in the alpha band (8–13 Hz) because EEG–fMRI artifacts, caused by cardioballistic (CB) head motions as well as random head movements, are substantially reduced in this case. Also, electrophysiological activity in the high-beta band is relevant to depression, as mentioned above. The rtfMRI–EEG-nf was used in the present study for simultaneous upregulation of BOLD fMRI activation in the left amygdala ROI and frontal EEG power asymmetry in the high-beta band during the positive emotion induction task. Methods Integration of simultaneous rtfMRI and EEG data Our implementation of rtfMRI–EEG-nf is based on a novel, firstof-its-kind real-time system, integrating simultaneous rtfMRI and EEG data streams. The system is designed for operation with a General Electric Discovery MR750 whole-body 3 T MRI scanner and a 128-channel MR-compatible EEG system from Brain Products GmbH. It represents a further development of the custom real-time MRI system (Bodurka and Bandettini, 2008). A block diagram of the system is shown in Fig. 1. The system design utilizes real-time features of AFNI (Cox, 1996; Cox and Hyde, 1997) and real-time functionality of BrainVision RecView software. The AFNI real-time plugin is used to perform real-time volume registration of fMRI data. It is also used to compute mean values of fMRI

V. Zotev et al. / NeuroImage xxx (2013) xxx–xxx

Fig. 1. Block diagram of the real-time system for simultaneous rtfMRI–EEG neurofeedback. The diagram illustrates processing of simultaneous rtfMRI (blue arrows) and EEG (green arrows) data streams and their real-time integration into the rtfMRI–EEG neurofeedback data stream (red arrows).

signals for several user-defined ROIs and export them for each fMRI volume in real time via a TCP/IP socket (Fig. 1). The RecView software makes it possible to partially remove MR and CB artifacts from the EEG data in real time using a built-in automated implementation of the average artifact subtraction method (Allen et al., 1998, 2000). RecView was custom modified to enable export of the corrected EEG data in real time through a TCP/IP socket (Fig. 1). Control and communication programs (shown in pink in Fig. 1) were written in Python (RTeeg, eeg_client, Math modules) and Perl (RTmri, RTcontrol, mGUI). The RTmri program runs on the MRI scanner's Linux control computer. The other programs run on a dedicated Linux real-time workstation with a kernel customized for high-speed inter-process communications with message queues, synchronization with semaphores, and large data exchange via shared memory. The rtfMRI-nf signal is updated every TR and can be based on fMRI signal from a pre-selected ROI, such as the amygdala. It can also be computed using any combination of fMRI signals from multiple ROIs. Furthermore, our custom modification of the AFNI real-time plugin makes it possible to provide rtfMRI-nf based on real-time support vector machine (SVM) classification. The EEG-nf signal can be updated at a much faster rate (as often as every 100 ms). Real-time processing of the RecView-corrected data for EEG-nf is performed in Math modules (Fig. 1) utilizing NumPy functionality. It allows for: i) selection of an individual EEG channel or any combination of channels for analysis; ii) inspection of the EEG data and exclusion of data intervals with excessive artifacts; iii) real-time FFT spectrum analysis and computation of EEG power for any number of user-defined frequency bands; and iv) calculation of any metrics for EEG-nf, such as the frontal EEG power asymmetry. The multimodal neurofeedback graphical user interface (mGUI, Fig. 1) integrates rtfMRI and EEG real-time data streams, computes the neurofeedback signals, and converts these signals into graphical representations viewed by the subject inside the scanner. mGUI is a multithreaded application supporting images, graphics primitives (such as bars), and text to form a dynamic display based on the rtfMRI-nf and EEG-nf signal levels (see Fig. 2a below).

Performance of the rtfMRI–EEG-nf system was extensively tested for multiple MRI/EEG hardware configurations prior to the system's use in human subject experiments. The tests utilized a standard MRI phantom and a specialized test signal generator for EEG. The system demonstrated robust real-time operation with 8- and 16-channel MRI head coil arrays and 32- and 128-channel EEG configurations. Experimental procedure The study was conducted at the Laureate Institute for Brain Research. The research protocol was approved by the Western Institutional Review Board (IRB). Six healthy subjects (mean age 24 ± 9 years, four females) participated in the study. All the participants provided written informed consent as approved by the IRB. Each subject wore an EEG cap throughout the experiment. All the participants were neurofeedback-naïve. The experimental procedure was developed based on the results of our previous work, which demonstrated that healthy subjects could learn to upregulate their left amygdala activation using rtfMRI-nf during a positive emotion induction task based on retrieval of happy autobiographical memories (Zotev et al., 2011). The main contribution of the present study is a proof-of-concept demonstration of rtfMRI–EEG-nf. Accordingly, each subject was presented with a neurofeedback display screen showing two neurofeedback bars: the rtfMRI-nf bar on the right side of the screen and the EEG-nf bar on the left (Fig. 2a). The height of the rtfMRI-nf bar represented BOLD activation (with respect to a resting baseline) in the left amygdala (LA) ROI shown in Fig. 2d. The LA ROI was defined as a sphere of 7 mm radius centered at (−21, −5, −16) in the Talairach space (Talairach and Tournoux, 1988), as used in our previous work (Zotev et al., 2011). The rtfMRI-nf bar height was updated every 2 s. The height of the EEG-nf bar represented a change in the frontal EEG power asymmetry (with respect to a resting baseline) between EEG electrodes F3 (on the left) and F4 (on the right) as depicted in

International Scholarly Research Network ISRN Nanotechnology Volume 2012, Article ID 102783, 9 pages doi:10.5402/2012/102783

Review Article A CNTFET-Based Nanowired Induction Two-Way Transducers Rostyslav Sklyar Verchratskogo st. 15-1, Lviv 79010, Ukraine Correspondence should be addressed to Rostyslav Sklyar, sklyar@tsp.lviv.ua Received 15 December 2011; Accepted 28 February 2012 Academic Editors: C. A. Charitidis and J. Sha Copyright ÂŠ 2012 Rostyslav Sklyar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A complex of the induction magnetic field two-way nanotransducers of the different physical values for both the external and implantable interfaces in a wide range of arrays are summarized. Implementation of the nanowires allows reliable transducing of the biosignalsâ&#x20AC;&#x2122; partials and bringing of carbon nanotubes into circuits leading to examination of the superconducting transition. Novel sensors are based on the induction magnetic field principle, which causes their interaction with an ambient EM field. Mathematical description of both the signal and mediums defines space embracing of the relevant interfacing devices. As a result, a wide range of the nano-bio-transducers allow both delivering the variety of ionized biosignals and interface the bioEM signals with further stages of electronic systems. The space coverage and transducing values properties of the state-of-the-art magnetic interfaces are summarized, and directions for their future development are deduced.

1. Introduction: Biophysical Signals, Transducing, and Interface Applications A biosensor is a device that incorporates a biologically active layer as the recognition element and converts the physical parameters of the biological interaction into a measurable analytical signal [1]. Understanding of biosignalsâ&#x20AC;&#x2122; (BS) nature and properties of their mediums are a basis for effective design of magnetic interfaces (MIs). Rapid progress in the advancement of several key science areas including nanoscale interfaces has stimulated the development of electronic sensor technologies applicable to many diverse areas of human activity. For example, the conceptualization and production of electronic nose devices have resulted in the creation of a remarkable new sector of sensor technology resulting from the invention of numerous new types of olfactory-competent electronic sensors and sensor arrays [2]. The growing variety of biosensors can be grouped into two categories: implantable and external. In turn, the last one has two existing paradigms: wearable sensor and noncontact sensor. A wearable sensor had potential to be intrusive, and noncontact sensor methods may still be intrusiveness to a certain extent, while a noncontact sensor is limited in its capability of acquiring physiological signals [3]. Voltage potentials of the living organism and its organs are measured

by both implantable and external electric field probes of high sensitivity [4]. Information on organ activity is obtained by measuring biomagnetic signals. For such purposes a multichannel high-temperature superconducting quantum interference device (high Tc SQUID) system for magnetocardiography (MCG) and magnetoencephalography (MEG) of humans, with high magnetic field resolution, has been developed [5, 6]. The most current sensing devices give us the possibility to receive a full scale of both the internal and external control BS. The internal ones are picking up by polymeric microprobes, CMOS chips, and nanoneedles, while the external by electromyography and neuroprosthetic (electroencephalogram (EEG) and MEG) systems. Improving an informational capability of the interface is implemented by the application of the advanced superconducting transducer and electromagnetic (EM) transistor/memristor [7, 8]. These elements are arranged into the arrays of a different configurations and can cover the order of spaces from macro- to nanolevels. There are a number of methods and devices for transducing different BS into recordable or measurable information. The transfer of nerve impulses (NI) is the main data flow that carries sensory information to the brain and control signals from it and from the spinal cord to the limbs.

8 integrals respectively. The next two strings are explaining the bounds on the possible spreading of the said method. The aggregated interface qualities, which are given in the tables, have been shown in the graph (Figure 6). Upon its analysis, it becomes clear that the area which is bounded by the dashed lines presents the MIs. At the same time, there are some uncertain areas (marked on the figure) that are inaccessible to the designed transducing elements. Furthermore, the graph’s square is open to the right for perspective media of MIs.

5. Conclusions The reviewed variety of FETs shows the varying extent of readiness for them to be exploited in SuFETTr of electrical current signals (see Figure 7). The most appropriate for such an application are the ordinary solid-state SuFET modifications and novel CNT-based SuFETs. The organic SuFETs are not amply developed, but this work is being carried out in a number of directions. At the same time, the PCs, which are necessary for the external sensor with respect to the transducing medium (nerve fibre, flow of ions and DNA spiral), and corresponding low-ohmic wire traces for connecting PCs to the FET’s channel are sufficiently developed, even at nanodimensions. The preliminary calculations confirm the possibility of broadening the SuFETTr’s action from magnetic field to the biochemical medium of BSs. The main parameters of such BSs can be gained by applying the arrangement of the SuFETTr(s) to the whole measurement system. Two directions of SuFETTr function enable decoding of the BS by comparing the result of its action on some process or organ with an action on them of the simulated electrical or biochemical signal after their reverse transducing through the SuFETTr(s). Furthermore, this decoded signal will provide a basis for creating feedback and feedforward loops in the measuring system for more precise and complete influence on the biochemical process. The advance in the instrumentation techniques and technology of materials allows introducing of more accurate methods of interfacing and transducers for their execution (see Figure 8). At this level of progress, the head sensors of paramount sensitivity and simplicity in picking up function with minimal changes of the physical variables. The recent breakthrough in superconducting- and nanotechnologies caused the creation of induction transducers, which have better informational capability in some diagnostical purposes. These devices are based on the universal law of the EM induction on the one hand and different special effects of MF interaction with a medium on the other. Since the proposed variety of bio-nano-sensors are passive, they do not affect the functions of the organs and their interaction.

References [1] F. Lucarelli, G. Marrazza, A. P. F. Turner, and M. Mascini, “Carbon and gold electrodes as electrochemical transducers for DNA hybridisation sensors,” Biosensors and Bioelectronics, vol. 19, no. 6, pp. 515–530, 2004.

ISRN Nanotechnology [2] A. D. Wilson and M. Baietto, “Advances in electronic-nose technologies developed for biomedical applications,” Sensors, vol. 11, no. 1, pp. 1105–1176, 2011. [3] Y. Lin, “A natural contact sensor paradigm for nonintrusive and real-time sensing of biosignals in human-machine interactions,” IEEE Sensors Journal, vol. 11, no. 3, pp. 522–529, 2011. [4] K. T. Ng, T. E. Batchman, S. Pavlica, and D. L. Veasey, “Noise and sensitivity analysis for miniature E-field probes,” IEEE Transactions on Instrumentation and Measurement, vol. 38, no. 1, pp. 27–31, 1989. [5] H. Itozaki, S. Tanaka, T. Nagaishi, and H. Kado, “Multichannel high Tc SQUID,” IEICE Transactions on Electronics, vol. E77-C, no. 8, pp. 1185–1190, 1994. [6] O. V. Lounasma, J. Knuutila, and R. Salmelin, “Squid technology and brain research,” Physica B, vol. 197, no. 1-4, pp. 54–63, 1994. [7] R. Sklyar, “An EM transistor based brain-processor interface,” in Nanotechnology 2009: Life Sciences, Medicine, Diagnostics, Bio Materials and Composites, vol. 2, chapter 3: Nano Medicine, pp. 131–134, CRC Press, Houston, Tex, USA, 2009, http://www.nsti.org/procs/Nanotech2009v2/3/T82.602. [8] R. Sklyar, “Induction magnetic field biosensors: from the macro to nano dimensions,” in Proceedings of the 1st BioSensing Technology Conference, Bristol, UK, November 2009, paper P2.3.13 (2 pages) of the abstract book. [9] A. Kandori, D. Suzuki, K. Yokosawa et al., “A superconducting quantum interference device magnetometer with a roomtemperature pickup coil for measuring impedance magnetocardiograms,” Japanese Journal of Applied Physics I, vol. 41, no. 2 A, pp. 596–599, 2002. [10] R. Sklyar, “Superconducting induction magnetometer,” IEEE Sensors Journal, vol. 6, no. 2, pp. 357–364, 2006. [11] P. Fromherz, “Electrical interfacing of nerve cells and semiconductor chips,” ChemPhysChem, vol. 3, pp. 276–284, 2002. [12] Y. Cui, Q. Wei, H. Park, and C. M. Lieber, “Nanowire nanosensors for highly sensitive and selective detection of biological and chemical species,” Science, vol. 293, no. 5533, pp. 1289–1292, 2001. [13] R. Sklyar, “From nanosensors to the artificial nerves and neurons,” in Proceedings of the Nanotechnology in Industrial Applications (EuroNanoForum ’07), pp. 166–168, CCD, Düsseldorf, Germany, June 2007. [14] R. Sklyar, “Superconducting organic and CNT FETs as a Biochemical Transducer,” in Proceedings of the 14th International Symposium on Measurement and Control in Robotics (ISMCR ’04), IEEE, Houston, Tex, USA, September 2004. [15] R. Sklyar, “A SuFET based either implantable or non-invasive (Bio)transducer of nerve impulses,” in Measurement and Control in Robotics, M. A. Armada, P. Gonzales de Santos, and S. Tachi, Eds., pp. 121–126, Producción Gráfica Multimedia, Madrid, Spain, 2003. [16] J. Vrba and S. E. Robinson, “SQUID sensor array configurations for magnetoencephalography applications,” Superconductor Science and Technology, vol. 15, no. 9, pp. R51–R89, 2002. [17] V. Zalyubovskiy, A. Erzin, S. Astrakov, and H. Choo, “Energyefficient area coverage by sensors with two adjustable ranges,” Sensors, vol. 9, pp. 2446–2460, 2009. [18] X. Chen and O. Bai, “Towards multi-dimensional robotic control via noninvasive brain-computer interface,” in Proceedings of the ICME International Conference on Complex Medical Engineering (CME ’09), Tempe, Ariz, USA, April 2009. [19] R. Sklyar, “Sensors with a bioelectronic connection,” IEEE

A SuFET Based Either Implantable or Non-Invasive (Bio)Transducer of Nerve Impulses R. V. Sklyar, Space Sensing Instruments, Verchratskogo st. 15-1, Lviv 79010, Ukraine E-mail: r_sklyar@hotmail.com

Abstract The main goal is to develop methods and devices for living being- machine interaction in order to obtain input and output signals from brain and motor nerves to the external devices or organs and vice versa. For this reason an efficient and accurate method of transducing biosignals from sense organs to output voltage, or artificial control signals to motor nerves, in limbs is developed and explained in the paper. Interaction between living beings and automatic equipment for process or environmental control is also presented. The transducer circuits and the intelligent system are given analytical treatment.

1 Introduction. Biophysical signals, engineering and scientific applications Steady and rapid progress in the robotics field requires ever quicker and better human- machine interaction and the development of a new generation of interfaces for intelligent systems. Such advances give rise to markedly increased biophysical research on the one hand and the need for new bioelectronic devices on the other. As a result of such efforts the design of synthesized neuroelectronic devices is high on the agenda. Transduction and measurement of biosignals are key elements of bioelectronic and biomechanic systems design. There are two means involved in signal transduction: 1) biochemical- by hormones and enzymes; 2) biophysical - by nerve impulses (ionic currents). Let us consider the biophysical ones as useful for the said combined systems design above. There are two values voltage and electric current which characterize the pathway of transduction. 1.1 Methods of biosignals measurement: noninvasive and implantable, electro/magneticand biosensors Voltage potentials of the living organism and its organs are measured by both implantable and external electric field probes of high sensitivity [1]. Information on organ activity is obtained by measuring biomagnetic signals. For such purposes a multi-channel high

temperature superconducting interference (high Tc SQUID) system for magnetocardiography (MCG) and magnetoencephalography (MEG) of humans, with high magnetic field resolution has been developed [2, 3]. The known amperometric techniques of biosignals involve the Renview bight realising method [4], and the second method of "biosensors typically rely on an enzyme system which catalitically converts electrochemically non-active analytes into products which can be oxidized or reduced at a working electrode which is maintained at a specific potential with respect to a reference electrode" [5]. The same method is applicable also to potentiometic measurements "that can measure substrates, inhabitors or modulators of the enzyme". The Renview method requires extra stimulating of the isolated nerve fibre and the other method needs additional reagents and applied voltage. 1.2 Biosignals application to sensing techniques and control systems Many sensing organs of different physical values have been discovered. The most recent of these was the finding that "the antennae of jewel beetles can detect substances emitted in smoke from burning wood" [6]. Taking this into account, the exploitation of animals and even insects (schedulled for close attention in NASA's near future space explorations) as "living sensors" could be a potential reality in the near future [7]. In that case a secure and reliable biosignals pick-up method will be of paramount importance. On the other hand, such living objects could produce some control signals from their nervous systems directly. The first confirmation of the finding was achieved in recent experiments on fish, rats, monkeys and even humans [8,9]. The introduction of a bioelectronic mechanism for direct limb control by artificial nerve impulse previously received (implantable or non-invasive) from the nervous system or synthesized will be the next logical step [10,11].

2 The transducer arrangement The extensively developed SQUID systems do not suit the robotic and brain-machine applications because

they need cryogenic installation (or cryocoolers which introduce vibrational and magnetic noise problems) around the whole device and have a small dynamic range of detecting signals [12]. It is clear that the implantable variant of such magnetometers is completely impossible. That is why a method that conserves the best features of the superconducting device was developed for removing the pickup coil (PC) from cryogenic installation [13,14]; On the other hand, neuroelectronic systems for two-way interfacing of the neuronal and the electronic components by capacitive contacts and by field-effect transistors with an open gate was developed [15]. They observed a wide spectrum of transistor signals monitoring the activity of neurons from leeches, snails and rats. In order for the electronic component to function under cryogenic (superconducting) conditions, there are a number of methods to connect them in vivo through three types of contacts: biocompatible, flexible and with increased adhesion [16-18]. The main objective of the project is to combine the bioelectric nature of nerve impulses and synaptic currents between neighboring neurons with body-temperature PC and zero resistance input of the superconducting fieldeffect transistor (SuFET) device in order to obtain most advantageous organism (living being)- machine interface. The implantable and non-invasive variants of the device are defined by the type of contacting PCs and/or SuFET channel(s) with nerve impulses and/or synaptic currents (see Table). Table. The different arrangements of the transducer according to the detecting quantity. Type Implantable Non-invasive Object Nerve PC(s) (wrapped PC(s) (wrap. fibers around), SuFET around limb(s) channel(s) (introduand bodyced into the fiber) spinal cord) Brain SuFET channel(s) PC(s) activity (introduced between (helmet type) neighboring neurons) 2.1 Nerve fibre as a source of vortical magnetic field The method of transducing the vortical magnetic field from the nerve impulses by the PC wrapped around the nerve fibre was advanced long ago [19]. By introducing the said superconducting magnetometer with roomtemperature PC (SIM) [13,20] it is possible to create the implantable transducer (Fig. 1). A PC with inductance L, self-capacitance C0 and active resistance R is connected in parallel with the drain of a SuFET cryogenic device. The SuFET is used as a zero-resistance ammeter which converts drain currents (I0 >Ic ) into gate voltages [20]. The critical Josephson current Io should be determined using experimental data and Ic is defined by the e.m.f. Eo from the expression:

Nerve fibre

head sensor R

…

SuFET Draine Cin/2

gDS

× JJ

- +

Rgate

Gate

Cin/2

VGS

Source

Cryostat

Shield

JJ- indicates Josephson junction.

Figure 1. Equivalent circuit of a SuFET based transducer of vortical magnetic field H.

Ic =

E0 ωH = µ 0 S eq zc zc

(1)

where Seq=µeff πd2N/4 with µ0-the permeability of free space, µ0=4π·10-7 henry/meter; µeff-the effective relative permeability of a high-µ metal core; d-the average diameter of a coil; N-total turn number of solenoid; zcimpedance of PC. In view of the dependence between gate-to-source voltage VGS and drain current [20]. This relationship is called transfer function and is defined as:

VGS =

(

QG hω T (QG ) + 1 − 1 − (I c / I 0 ) 2 C in 2e

)

(2)

where QG- the average gate charge and ωT is closely related to the small signal transconductance of the SuFET at VDS =0. Its sensitivity is maximized at Ic=Io/√2; however, the point of maximum current sensitivity can be adjusted by adding a dc external magnetic field [20]. Taking into account Eq. 1 we have:

VGS =

jπµ 0 S eq hω T (QG ) f eI 0 z c

(3)

Taking into account the noise sources, the equivalent circuit of the SIM takes the form as shown in [13]. The main influence on SIM’s sensitivity are made by current and voltage noise sources in LF, MF and HF ranges and the general equation is simplified as the following:  H SIMl ≅ 4k (TR + TSuFET γ noiseVc / I 0 ) /  µ 0 S eqω , when ω << ω 0 ;  4kTK ed + I 22 (CVGS ) sin 2 2ΘL2 / 4  , H ≅ SIMm  µ 0 S eq  H SIM =  2 2 when R << ωL, ω LC0 ≅ 1, R / ω << K ed ;  4kTSuFET γ noiseVcω 2C02 / I 0 + H µ 0 S eq , SIMh ≅ L  I 22 (CVGS ) sin 2 2Θ / 4   when ω 2 LC0 >> 1. where k- the Boltzmann constant; T- absolute temperature of copper wire; γnoise is the ratio of the

kinetic energy of the JJ link to thermal energy and VC characteristic voltage; sin2θ distortion of the pure Josephson sinθ term which is caused by the fluxdependent gate voltage effect with θ=2φDS/ћ in terms of the drain to source flux; I2=ħωT2C/2e. 2.2 Nerve fibre and neighboring neurons as a transmission line The main informational flux from organs of the senses to motor nerves is transmitted through nerve fibres which consist of a myelin shield with axons as a core. Recent research results suggest that such an arrangement is similar to a transmission line [21]. Synaptic currents between first order neighboring neurons into in vivo or brain slice preparations have an order of 50 pA [22]. The nerve impulses passing through the fibre could be unambiguously defined by detecting the matching ionic current(s) or its superposition. Such a technique seems optimal because even precise voltage measurement could not give a current value according to the Ohm law. First of all, nerve fibre must be separated from a living organism for resistance of fibre measurement and, secondly, this resistance may vary in time. There are two feasible courses for impulse detection. The first one is measuring the magnetic flux around the fibres by PC [19]. The second one is to let the impulses pass through the electronic device with minimal impediments. Proceeding from the previously mentioned difficulties, including SuFET device into nerve fibre [15] seems to be the solution to the problem. Ionic currents in axons labeled ID and passing through the SuFET channel induce the output voltage Vg on its gate (Fig.2).

In any case the SuFET device should not introduce either losses of the nerve impulses or interferences or disturbances to them because of the superconducting nature of the transducer operational mode. Following signal processing and transmitting units which are completely separated from the living organism shouldn't interact with it due to the passive mode of transduction. Let us consider the connection between the input (biosignal ID) and output (voltage Vg ) variables of the transducer (Eq.3):

Vg =

jhωT (QG ) ID 2eI 0

Taking into account noise parametres ( en and in ) it is possible to calculate the sensitivity limit of the transducer in the wide frequency range. With the SuFET device, noise current in is very small and may be neglected [13]. The resulting sensitivity of the transducer could be defined roughly as:

En = kT / g DS ⋅ γ noise , g DS = I 0 / Vc At the same time, the frequency of the transducer ranges from DC to high frequencies (limited by current loss on spurious capacitance). In such a case their will be current through axons- myelin- tissue capacitance Ct. For the advanced device the upper frequency limit can be defined as follows:

ϖup =

1 + R f / RDC Rf = L f Ct RDCL f Ct

where Rf, Lf- resistance and inductance of the nerve fibre respectively; RDC- the convey contact junction resistance. Simulated performance data of the transducer in comparison with known preamplifiers of biosignals are myelin shown in Fig.3: Procesa) comparison of the measured and simulated sensitivity channel sing, nerve thresholds of different instrumentation amplifiers (IA) Vg impulse transtransmitvisualization for biomedical purposes, receiving ting and memory IA; unit ∗- a CMOS low- power low- noise monolithic mitting, unit units ionic ο- high performance, FET input IA; currents visualiza- voltage- sensitive differential input IA; gate tion, etc. SFET • - an SuFET based transducer (Vc= 100µV, I0= 100µA, SuFET T= 126 K, γn= 20) [20]; units ■ a low- noise CMOS preamplifier for SQUIDs (T= axons (Bio)Telemetry 4K); ◊ - electrochemical PH- sensitive transducer and external in vivo following external units amperometric biosensor [5]; Figure 2. SuNerve- the inclusion of a SuFET device b) evaluation of the transducers along the design axis into thebased nerve fibre Fig. 1. An SFET biotransducer and its signal behavior [23]. It is supposed that the ionic current will be less than 2.3 Future trends of transducer development the critical current of the superconducting channel. "For There are some channels of SuFET development nonzero drain voltages, the SuFET absorbs lowwhich are highly important for the further improvement frequency power and re- emits this power at extremely of transducer performance. First of all there is the high frequencies" [13]. The detection of such a signal integration of high-temperature superconductivity with gives information about input current ID. Moreover, electronic devices. Such a SuFET device is being investivoltage could be applied at the gate in order to adjust gated with continuing progress [24]. Moreover, recent critical current.

Figure 3. Sensitivity and design merits of the biotransducers achievements in high temperature superconductivity verify the promising nature of the trend [25]. It is clear that the elimination of the need for a refrigeration system could mean a cheaper product that could be massproduced. Another developmental trend is related to the adoption the transducer to living organism conditions via the introduction of organic superconductivity. A FET device based on these technologies has been studied in-depth [26]. Synthesized organic superconductors might be friendly to organisms and effective as a part of the electronic device. Ideally, high- temperature organic superconductor based SuFET device seems suitable for the above mentioned transducing technique. Increasing the suitability of the electronic devices was mentioned above. Implantation could be achieved by employing organic[27], diamond [28] and CNT [29] based (superconducting) electronic devices.

3 A biotransducer based sensing and control system The above arrangement seems quite suitable for use as a living object-machine interface or as an element of the intelligent system. The system is based on two technologies. On the one hand it is based on the the Supertransducer Human The nervous system motor nerves

SFET based transduser

Insect

Translator

a) Human (animal, insect) - machine interface

Organs of the senses

Supertransducer

Controlled drives

Visualization Artificial intelligence

b) A six-stages intelligent system

controlled drives

Figure 4. The (bio)transducer based intelligent system Fig. 3. The biotransducer based intelligent system

transducer and the automatic equipment that follows and on the other on the sensory system or motor nerves in limbs of living organisms. The advanced system procedure is shown on Fig. 4 (upper). As a result living beings control drives by previously translated biosignals. In the other variant, biosignals from organs of the senses or brain transduce directly into intelligent or robotic systems which, in such a way, pick up environmental information (Fig. 4 (lower). Both of the structures are subjects for further refinement of all the elements independently from one another.

4 Conclusions The invented biotransducer has the following fundamental improvements upon existing ones: a) the sign of the output voltage permits the determination of the direction of the input current passing through a single SuFET device; b) situating the reference electrode outside the living organism makes precise measurement possible; c) the capability to regulate the proportion of axons that are being investigated to the untouched ones- either the whole cross section of the nerve fibre or any part of it; d) the possibility to substitute the SuFET device or to adjust its ratings to comply with the conditions of the meas. process without repeatedly destroying nerve fibre; e) the transducer could be either implanted or noninvasive (like the MEG) with conversion in both directions; f) the combination of biocompatibility and tissue equivalence in both the diamond and protein-based (organic) FETs makes them naturally fit for implantation. In what areas can detected nerve impulses be applied? There are two basic applications: 1) process control and 2) the connection of artificial sense organs and limbs: a) artificial limbs function by picking up a biosignal off motor nerves and transducing it after translation to electromechanical drives. The multiplication of finalcontrol elements is possible after the preliminary stages;

b) lost or damaged organs of the senses could be substituted or complemented by similarly operating human, animal, etc. organs. Its output biosignals may be picked up by the transducer and injected into nerve fibres of the recipient after reverse changing; c) substitution of inoperative control or motor nerve centers by control biosignals simulation and transducing them to living organs as discussed above. All in all the complete robotic system (see Appendix) consists of a living organism in a feedback relationship with automation execution which interact with the aid of the proposed transducers in order, for example, to control some technological processes according to the state of the external environment. There are two operating channels. The first of these is between the sense organs and limbs. The second is between artificial sensors and drives. Between both channels a mutual flow of information exists by means of the explained external and implantable transducers. It is presumed that the nerve- machine interface will allow the close monitoring of flow data and the additional input of signals between exposure to physical environmental stimuli and the resultant action response.

5 References [1] Ng K.T. et al., “Noise and sensitivity analysis for miniature e- field probes”, IEEE Trans. Instrum. Meas., Vol. 30, No. 1, pp. 27-31, 1989. [2] Itozaki H. et al., “Multi-channel high Tc SQUID”, IEICE Trans. Electron., Vol. E77-C, No. 8, pp. 11851190, 1994. [3] Lounasmaa O.V. et al., “SQUID technology and braine research”, Physica B, No. 197, pp. 54-63, 1994. [4] Tasaki I., “Nervous transmission”, Charles C Thomas Publ., Springfield IL USA, 1953. [5] Sharma A. and Rogers K.R., “Biosensors”, Meas. Sci. Technol., No. 5, pp. 461-472, 1994. [6] Schütz S. et al., “Insect antenna as a smoke detector”, Nature, Vol. 398, No. 6725, pp. 298-299, 1999. [7] David L., “Beastly explorers”, New Scientist, Vol. 161, No. 2168, p. 32, 1999. [8] Miguel A. L., Nicolelis M.A.L. and Chapin J.K., “Controlling robots with the mind”, ScientificAmerican, October 2002. [9] DeMarse T.B., Wagenaar D.A. and Potter S.M., “The neurally- controlled artificial animal: a neuralcomputer interface between cultured neural networks and a robotic body”, Society for Neuroscience Abstracts 28: 347.1, 2002. [10] Aliaga J. et al., “Electronic neuron within a ganglion of a leech (Hirudo medicinalis)”, Phys. Rev. E, Vol. 67, 061915, 2003. [11] Miguel A. L., Nicolelis M.A.L., “Brain–machine interfaces to restore motor function and probe neural circuits”, Nature Reviews Neuroscience, Vol. 4, No. 5, pp. 417-422, 2003. [12] Prance R.J., Clark T.D. and Prance H., “Compact

room-temperature induction magnetometer with superconducting quantum interference device level field sensitivity”, Rev. Sci. Instrum., Vol.74, No. 8, pp. 3735-3739, 2003. [13] Sklyar R., “A cryogenic induction magnetometer”, XIII IMEKO World Congress, IMEKO XIIIACTA IMEKO'94, Vol. III, pp. 2377-2382, 1994. [14] Kandori A. et al., “A superconducting quantum interference device magnetometer with a roomtemperature pickup coil for measuring impedance magnetocardiograms”, Jpn. J. Appl. Phys., Vol. 41, Part 1, No. 2A, pp. 596-599, 2002. [15] Fromherz P., “Electrical interfacing of nerve cells and semiconductor chips”, CHEMPHYSCHEM, No. 3, pp. 276-284, 2002. [16] Belosludov R.V. et al., “Molecular enamel wires for electronic devices: theoretical study”, Jpn. J. Appl. Phys., Vol. 42, Part 1, No. 4B, pp. 2492-2494, 2003. [17] Gross M. et al., “Micromachining of flexible neural implants with low- ohmic wire traces using electroplating”, Sens. Act. A, Vol. 96, pp. 105-110, 2002. [18] Arzt E., Gorb S. and Spolenak R., “From micro to nano contacts in biological attachment devices”, PNAS, Vol. 100, No. 19, pp. 10603-10606, 2003. [19] Romani G.L., Williamson S.J. and Kaufman L., “Biomagnetic instrumentation”, Rev. Sci. Instrum., Vol. 53, No. 12, pp. 1815-1845, 1982. [20] Sklyar R., “Patent UA №21185”, Ukrainian State Patent Office, Bulletin №1, 2000. [21] Hanisch C., “Nervensache”, Bild der Wissenschaft, No.2, pp. 70-74, 1999. [22] Wyart C. et al., “Constrained synaptic connectivity in functional mammalian neuronal networks grown on patterned surfaces”, Journal of Neuroscience Methods, No. 117, pp. 123-131, 2002. [23] Mohri K., “Sensormagnetics”, IEEE Trans. J. on Magn. in Japan, Vol. 7, No. 8, pp. 654-665, 1992. [24] Suzuki Sh., Tobisaka H. and Oda Sh., “Electric properties of coplanar high-Tc superconducting field-effect devices”, Jpn. J. Appl. Phys., Vol.37, Part 1, No.2, pp. 492-495, 1998. [25] Moran O., Hott R. and Schneider R., “Current amplification in high-temperature superconductor current injection three-terminal devices”, J. Appl. Phys., Vol. 94, Iss. 10, pp. 6667-6672, 2003. [26] Schön J. H. et al., “Ambipolar pentacene fieldeffect transistors and inverters”, Science, Vol. 287, pp. 1022-1023, 2000. [27] Tanase C. et al., “Local charge carrier mobility in disordered organic field-effect transistors”, Organic Electronics, accepted Apr. 2003. [28] Garrido J. A. et al., “Fabrication of in-plane gate transistors on hydrogenated diamond surfaces”, Appl. Phys. Let., Vol. 82, No.6, pp. 988-990, 2003. [29] Nihey F. et al., “Carbon-nanotube FETs with very high intrinsic transconductance”, Jpn. J. Appl. Phys., Vol. 42, Part 2, pp. L1288-L1291, 2003.

SuFET

S u F E T

SuFET

Appendix