"An Imperceptible Supported" Magnetic Orientation

ARTICLE Received 2 Dec 2014 | Accepted 10 Dec 2014 | Published 21 Jan 2015

DOI: 10.1038/ncomms7080

OPEN

Imperceptible magnetoelectronics Michael Melzer1, Martin Kaltenbrunner2,3, Denys Makarov1, Dmitriy Karnaushenko1, Daniil Karnaushenko1, Tsuyoshi Sekitani2,3,4, Takao Someya2,3 & Oliver G. Schmidt1,5,6

2 3 4

Future electronic skin aims to mimic nature’s original both in functionality and appearance.

Although some of the multifaceted properties of human skin may remain exclusive to the biological system, electronics opens a unique path that leads beyond imitation and could equip us with unfamiliar senses. Here we demonstrate giant magnetoresistive sensor foils with high sensitivity, unmatched flexibility and mechanical endurance. They are o2 mm thick, extremely flexible (bending radii o3 mm), lightweight (E3 g m 2) and wearable as imperceptible magneto-sensitive skin that enables proximity detection, navigation and touchless control. On elastomeric supports, they can be stretched uniaxially or biaxially, reaching strains of 4270% and endure over 1,000 cycles without fatigue. These ultrathin magnetic field sensors readily conform to ubiquitous objects including human skin and offer a new sense for soft robotics, safety and healthcare monitoring, consumer electronics and electronic skin devices.

1 Institute

for Integrative Nanosciences, Institute for Solid State and Materials Research (IFW Dresden), Helmholtzstrasse 20, 01069 Dresden, Germany. and Electronic Engineering and Information Systems, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan. 3 Exploratory Research for Advanced Technology (ERATO), Japan Science and Technology Agency (JST), 2-11-16 Yayoi, Bunkyo-ku, Tokyo 113-0032, Japan. 4 The Institute of Scientific and Industrial Research (ISIR), Osaka University, Mihogaoka 8-1, Ibaraki, Osaka 567-0047, Japan. 5 Material Systems for Nanoelectronics, Chemnitz University of Technology, Reichenhainer Strasse 70, 09107 Chemnitz, Germany. 6 Center for Advancing Electronics Dresden, Dresden University of Technology, Helmholtzstrasse 10, 01062 Dresden, Germany. Correspondence and requests for materials should be addressed to D.M. (email: d.makarov@ifw-dresden.de).

2 Electrical

NATURE COMMUNICATIONS | 6:6080 | DOI: 10.1038/ncomms7080 | www.nature.com/naturecommunications

& 2015 Macmillan Publishers Limited. All rights reserved.

1

ARTICLE

E

5

6

7 8

NATURE COMMUNICATIONS | DOI: 10.1038/ncomms7080

lectronics of tomorrow will be imperceptible and will form a seamless link between soft, living beings and the digital world1. Inspired by natures antetype, electronic skin is an intriguing technological platform already able to perceive temperature changes2,3, mimic the sensation of touch4–6, monitor and display physiological conditions7,8, communicate wirelessly, and harvest and store energy for autonomous operation9–13. Concepts that enable self-healing14 will lead to durable, multifunctional artificial skin. Other functionalities, however, especially those that are unfamiliar to human beings, are not addressed so far.

Magnetoception is a sense, which allows bacteria, insects and even vertebrates such as birds and sharks, to detect magnetic fields for orientation and navigation15,16. Humans are however unable to perceive magnetic fields naturally, but electronic skin could soon help to bridge this gap. Soft, flexible and transient smart sensorics17 to monitor physiological conditions are at the forefront of multidisciplinary research efforts bridging materials science, electrical engineering and medicine.

Magnetosensorics is a versatile tool to assess mechanical movements also in vivo; foreseeable applications include realtime monitoring of artificial muscles, joints or valves of the heart to diagnose early stages of dysfunctions. Such advanced applications require very specific mechanical properties of the sensing elements, such as bending radii o10 mm, stretchabilities exceeding 100%, as well as a sensitivity for magnetic fields below 100 Oe, all of which are met by our ultrathin magnetic sensors, making them ideally suited for wearable, yet unobtrusive and imperceptible orientation and manipulation aids.

Here we go beyond imitating the features of human physiology and introduce e-skins with a magneto-sensory system, which equips the recipient with a ‘sixth sense’ able to perceive the presence of static or dynamic magnetic fields. We demonstrate an on-skin proximity detection systems for touchless human– machine interaction, motion and displacement sensorics applicable for soft robots18,19 or functional medical implants20,21, as well as magnetic functionalities for epidermal electronics7. In this work, we construct highly sensitive giant magnetoresistive (GMR) sensor elements on ultrathin, 1.4 mm polyethylene terephthalate (PET) foils. Weight and flexibility are key figures of merit for large area electronics or robotic skin7,22, as they critically influence the mechanical response and perception of the artificial sensory system. With just about 1.5 mm total thickness, imperceptible magneto-electronic foils are light (E3 g m 2) and unmatched in flexibility; they are operable with radii of curvature below 3 mm, yet are highly durable and can withstand severe crumpling without any performance degradation. These are prerequisites for intimate contact with soft, biological tissue or organs and complex, arbitrarily shaped three-dimensional free forms. Despite their imperceptible design, our GMR sensors exhibit high sensitivities of up to 0.25% per Oe, identical to their counterparts on rigid Si/SiO2 wafer substrates. Biological skin is soft and flexible but also stretchable, a feature that is most desirable for the artificial equivalent. Imperceptible electronic foils11,22,23 offer an elegant route to facilitate very high levels of strain without any sacrifices in device performance by a facile postfabrication transfer step onto a prestrained elastomer. We demonstrate magneto electronics that can reversibly attain tensile strains up to 270%, nearly a tenfold increase over previously reported concepts24. We prepare magneto-sensitive elements that are compliant to uniaxial and biaxial deformation, creating a universal potential for applications in stretchable electronic systems. Moreover, our devices are remarkably stable, withstanding 1,000 stretch cycles without fatigue.

2

Results Imperceptible magnetoresistive sensor skin. GMR multilayer elements (Co/Cu and Py/Cu multilayers; Py ¼ Ni81Fe19) are directly fabricated on 1.4 mm ultrathin PET foils (Supplementary Figs 1 and 2). The structure of an array of five sensor elements is schematically shown in Fig. 1a, details on the fabrication process are found in the Methods section. The substrate foil is commodity scale, commercially available PET (Mylar 1.4 CW02) that is fully compatible with lithography and lift-off processes, which allows for accurately patterned individual devices, yet large area low-cost manufacture. The ultrathin substrate is temporarily self-adhered to a reusable support (see Methods for details) for ease of handling. After device fabrication, the imperceptible sensor skin is readily peeled off the handling support without causing damage, as demonstrated in Fig. 1b with a free-standing ultrathin (1.5 mm total thickness) array of five lithographically structured Co/Cu GMR multilayer elements. The extreme light weight and compliant nature of the sensor elements is demonstrated in Fig. 1c, where an array of magnetic field sensors is floating on a fragile, spherical soap bubble. Our sensor foil is highly flexible; it can be completely scrunched up and rubbed between fingertips multiple times (Fig. 1d and Supplementary Movie 1) without signs of performance degradation. This is corroborated by GMR characteristics (Fig. 1e) recorded for the as-fabricated sensor and after crumpling as in Fig. 1d, showing that even severe mechanical stress has virtually no influence on the device characteristics. In spite of the 1.4-mm PET foils’ rough surface (29 nm root mean square as determined by atomic force microscopy, Supplementary Fig. 3), our ultrathin magnetic sensor foils perform equal to reference samples fabricated on smooth, rigid Si/ SiO2 waver substrates. Instead, we take advantage of the enhanced adhesion of thin metal films on rough polymer surfaces22, which leads to an improved mechanical resilience. The prepared elements reveal a high GMR ratio of 57.8% at room temperature, which is a typical value for Co/Cu multilayers25. Apart from a slightly increased saturation field, the GMR signal on PET is very similar to the reference data on Si/SiO2 and exhibits the same GMR magnitude. The electrical resistance of the flexible sensors (16.9 O) is comparable to their rigid counterparts (16.2 O). Motivated by these results, we continued to prepare Co/Cu and Py/Cu multilayers coupled in the second antiferromagnetic maximum on ultrathin PET foils. These GMR systems have more stringent requirements with respect to substrate quality and deposition parameters, but exhibit much higher sensitivities to small magnetic fields26,27. This is especially desirable for smart skin, biomedical and orientation applications28–30. The magneto-electric characterization of the second maximum GMR multilayers (Supplementary Fig. 4) reveals that even these highly sensitive elements prepared on ultrathin PET behave very similar to their rigid counterparts prepared on Si/SiO2 silicon wafers, which strengthens the potential of imperceptible magnetoelectronics for a wide field of GMR and spintronic systems.

Imperceptible magnetoelectronics is readily worn directly on the palm of a volunteer’s hand as demonstrated in Supplementary Movie 2. Here, a set of Co/Cu first maximum GMR sensors intimately conforms to the inner hand and unobtrusively follows the motions and deformations of the skin when the hand is moved. A sensor element is electrically contacted with thin copper wires (Fig. 1f) by using conductive silver paste and the resistance of the on-skin sensor is recorded while moving the fingers, opening and closing the hand (Fig. 1g), applying a magnetic field with a permanent magnet and alternating the distance to the magnet (Fig. 1h). The resistance plotted in Fig. 1i and Supplementary Movie 3 shows a small, noise-induced fluctuation during the motion of the hand (which amounts to o0.3%), whereas the field of the approaching permanent magnet

NATURE COMMUNICATIONS | 6:6080 | DOI: 10.1038/ncomms7080 | www.nature.com/naturecommunications

& 2015 Macmillan Publishers Limited. All rights reserved.

9

ARTICLE

NATURE COMMUNICATIONS | DOI: 10.1038/ncomms7080

induces a strong resistance drop of about 13% at it’s nearest position. Altering the distance between the permanent magnet and the on-skin sensors results in a corresponding change in magnetoresistance. Note that magnetic saturation of the Co/Cu multilayer element is not reached, as the full bandwidth of the sensor is not used here. Electronic skin on fingertips31,32 is especially attractive for precise input and as communications interface due to our finger’s fine motor skills. In Fig. 1j,k,l and Supplementary Movie 4, we demonstrate an on-skin magnetic proximity sensor where a single GMR element is attached to a fingertip and connected to a readout circuit (see Supplementary

a

Fig. 5 for details). The presence of any kind of magnetic field can thus be detected by simply pointing the finger towards it and its strength is visualized by an array of light-emitting diodes. Unlike optical sensors, no line-of-sight between the sensor and the magnetic field emitter is required. This allows transmitting ‘magnetic messages’ through all non-magnetic objects such as safety enclosures, displays or even walls. The encoding can be realized both statically via permanent magnets as well as dynamically simply with current-driven wirings. In all those demonstrations, the imperceptible GMR sensors require no encapsulation or capping layers.

b

c

d

GMR multilayer (Co/Cu or Py/Cu)

1.4-µm PET foil

e

g

f

60

h

GMR (%)

50 40 Imperceptible GMR sensor

30

Magnet

20 10 0

i

Resistance (Ω)

–4

–2 0 4 2 Magnetic field (kOe)

65 60 Resting hand (f)

Nearby magnet (h)

Moving hand (g)

55

Nearby magnet Alternating distance to magnet

50 0

10

20

30

40

50

Time (s)

j

k

l

Proximity indicator

GMR sensor on fingertip Magnet

Figure 1 | GMR multilayers on ultrathin PET. (a) Diagram of an imperceptible magnetic sensor foil. Cross-section of the GMR layer system (inset). (b) Free-standing array of five Co/Cu multilayer elements on 1.4-mm-thick PET foil. (c) Ultralight (3 g m 2) sensor array floating on a soap bubble and (d) crumpled between fingertips. Scale bars, 10 mm. (e) GMR characteristics of one Co/Cu first maximum element as prepared (green curve, R ¼ 16.9 O), after attaching to the palm and folding (blue curve, R ¼ 17.0 O), and after crumpling as shown in d (red curve, R ¼ 17.9 O). Comparison with a reference sample on rigid silicon (dashed grey curve, R ¼ 16.2 O). Corresponding measurements on the other elements in the array gave similar results. (f,g,h) Imperceptible GMR sensor array on a human palm with one element connected to a readout circuit during rest, moving the hand and in proximity to a permanent magnet. Scale bars, 20 mm. (i) Recorded resistance of the sensor element for f through h and Supplementary Movie 3. (j,k,l) On-skin magnetic proximity sensor using a Co/Cu second maximum GMR element on the fingertip connected to a linear array of light-emitting diodes (LEDs) (dashed red frame) as touchless interface. Scale bars, 10 mm (see Supplementary Movie 4).

NATURE COMMUNICATIONS | 6:6080 | DOI: 10.1038/ncomms7080 | www.nature.com/naturecommunications

& 2015 Macmillan Publishers Limited. All rights reserved.

3

10

ARTICLE

NATURE COMMUNICATIONS | DOI: 10.1038/ncomms7080

a first approximation for the peak strain in the wrinkled metal layer on the ultrathin PET film, an established model for the bilayer bending strain38 can be applied. This computes the strain etop in a rigid film (elastic module: Yf, thickness df) on a softer substrate (elastic module: Ys, thickness ds) on bending to a radius R via

df þ ds ð1 þ 2Z þ wZ2 Þ ; ð1Þ etop ¼ ð1 þ ZÞð1 þ wZÞ 2R

11

12

with Z ¼ df/ds and w ¼ Yf/Ys. The properties of the PET foil are Ys ¼ 3 GPa and ds ¼ 1,400 nm. We estimate the elastic module of the GMR layer (Co/Cu in the first maximum: df ¼ 111 nm) by its material composition (60-nm Cu: YCu ¼ 131 GPa; 51-nm Co: YCo ¼ 211 GPa) to be 171 GPa. For the measured radius of curvature (o3 mm), induced by wrinkling in the relaxed state (Fig. 2d), the calculation according to the equation above results in the strain of B5.6%. However, this value is above the fracture strain of the GMR layer. Different effects are considered to reduce the actual strain, for example, an increased strength observed in multilayer thin films39 or the shifting of the neutral mechanical plane towards the active element by the sandwiching VHB elastomer. Furthermore, for the case of compressive strain on the GMR layer underneath the buckles, the FIB cuts revealed additional microscopic wrinkling of the magnetic nanomembrane on the PET (Fig. 2d, right side). Beyond that, we observe a much lower curvature in the wrinkle valleys (Fig. 2c, right side), which would also reduce tensile strains. A maximum peak strain of 0.5% in the GMR layer would allow a bending radius of below 40 mm according to equation (1).

In conclusion, we develop imperceptible and highly sensitive magnetic field sensors with unique mechanical properties and demonstrated the remarkable potential of this technology for smart skins, consumer goods and medical implants. These readyto-use sensing elements extend the cognition of electronic skin systems to a medium, which, by no means, can be detected by human beings. They are ultra-lightweight, conform to arbitrary surfaces and seamlessly follow deformations or distortions without performance degradation. For the emerging field of stretchable magnetoelectronics, the GMR sensors presented here outperform all previously introduced elements in terms of stretchability, reliability and fabrication potential by a multiple.

These versatile features are imparted to the magnetoelectronic devices by their ultrathin and flexible, yet robust polymeric support. The sensor layout used in the present work was chosen for ease of precise characterization. As photolithographic patterning of the magnetoelectronic nanomembranes on the ultrathin plastic foils was successfully applied in the present work, the device structure can be adapted and scaled to meet the requirements for specific applications and design concepts.

Patterning defined meander structures, for instance, can adapt the electrical parameters of individual sensing elements to the requirements of a specific signal processing electronics. Preparing the GMR elements into a bridge configuration would allow for a differential signal and may compensate for temperature effects.

The utilization of spin valve layer stacks instead of GMR multilayers can result in even higher sensitivities to small magnetic fields. Future work will focus on optimizations to interface electrically and mechanically with other electronic components enabling, for example, wireless readout and remote sensing. The integration of magnetoelectronics with ultrathin functional elements such as solar cells11, light-emitting diodes23, transistors40, as well as temperature and tactile sensor arrays22, will enable autonomous and versatile smart systems with a multitude of sensing and actuation features. Detailed mechanical and electrical modelling will further guide sensor improvement.

We foresee our work to inspire a diverse number of devices that will benefit from a ‘sixth sense’ magnetoception.

Methods GMR multilayers on ultrathin PET. PET foil (1.4 mm thin; Mylar 1.4 CW02, Pütz GMBH þ Co. Folien KG) where temporally self-adhered to a reusable, 125-mmthick polymer support coated with a thin layer of poly(dimethylsiloxane). Van der Waals adhesion allows both for device handling during lab-scale fabrication processes and subsequent detachment and transfer of the final element to other substrates and surfaces. A positive ultraviolet lithography lift-off process was performed using AZ 5214E resist (AZ Electronic Materials) and an MA56 mask aligner (SÜSS Micro Tech), to define individual sensor stripes of 1 16 mm2 with a four-probe contact geometry. Giant magnetoresistive multilayers comprising of Co(1 nm)/[Co(1 nm)/Cu(1.2 nm)]50 (Co/Cu first maximum), Co(1 nm)/[Co(1 nm)/ Cu(2.2 nm)]50 (Co/Cu second maximum) or Py(1.5 nm)/[Py(1.5 nm)/Cu(2.3 nm)]30 (Py/Cu second maximum) were grown by magnetron sputter deposition at room temperature (Py denotes soft magnetic permalloy: Ni81Fe19). The sputter parameters were as follows: base pressure, 7.0 10 7 mbar, Ar sputter pressure, 7.5 10 4 mbar and deposition rate, 2 Å s 1. After the lift-off process, the ultrathin sensor foil could be readily peeled from the handling support without causing damage and attached on the skin or any arbitrarily shaped surface. Electrical contacts were realized using thin (100 mm) enameled copper wires and G 3303B conductive silver paste (Plano).

Stretchable GMR elements. VHB 4905 F (3 M) adhesive tape was used as the elastomeric support for the preparation of stretchable GMR sensors. Stripes of 5 50 mm2 were pre-stretched by 600% longitudinal (if not stated differently).

Individual elements with elongated contacts were cut from a prepared array of Py/Cu second maximum multilayer sensors and attached with the metallized side onto the sticky surface of the pre-stretched VHB so that the four contact pads reach sideways beyond the tape. After removing the polymeric handling support, the multilayer element was contacted with four copper wires and conductive silver paste. As the pre-strain is released, wrinkles occur on the ultrathin PET foil with the buried magnetic nanomembrane, which are aligned along the short axis of the VHB stripe. The viscoelastic behaviour of the VHB tape prevents a relaxation to the initial length from high prestrains. Consequently, a 600% prestretched tape allows the sensor to be stretched by 270%. In the stretching experiments, the length of the sensor element after a relaxation period of 1 day is defined as the reference length and corresponds to 0% strain.

GMR sensors on expanding diaphragm. A hole was drilled into the bottom of a 55-mm-diameter circular plastic tray to fix a fluidic tubing and a permanent magnet was glued into the tray at a central position. A piece (50 50 mm) of VHB 4905 F (3 M) was spanned over the circular edge of the tray with a strain of about 50% 50% and adhered around the outer edge to seal the inner camber housing the permanent magnet. The back end of the fluidic tubing was connected to a 50-ml syringe to pump water into the sealed chamber and inflate the VHB membrane. At a highly inflated state, a Co/Cu second maximum GMR sensor on ultrathin PET foil, contacted on the two outmost contact pads, was attached face down to the top of the thus created water dome. On deflating the diaphragm using the syringe, the sensor was biaxially compressed and wrinkled accordingly. In this setup, the sensor resistance gives a measure for the distance to the permanent magnet, and hence the inflation of the diaphragm controlled by the water syringe.

The GMR characterizations of the sensor on the diaphragm were performed with a similar setup without the permanent magnet.

In situ GMR stretching setup. We tested the uniaxial stretchability of the prepared GMR-sensing elements using an automated computer-controlled in situ stretching stage with four-point electrical contact clips, which can be operated between the pole shoes of an electromagnet for characterization. An outline of the setup is depicted in Supplementary Fig. 13. The sample was mounted between two stretching clamps and their separation was adapted until the slack was removed.

After applying the attached copper wires to the contact clips, the electrical resistance was measured while sweeping the field perpendicular to the sensor stripe with the electromagnet. For the stretchability tests, the strain was increased in steps of 10% with a rate of 100 mm s 1 (0.7% s 1) before measuring the next GMR curve.

For the cyclic loading experiment, the sensor was first stretched to 1.5 times its length and then the strain was repeatedly increased and decreased by 50% (with respect to its original length) with a rate of 150 mm s 1 (1.25% s 1) and 1-s delay at the reversal points. At each tenth cycle, a new GMR curve was recorded at the two reversal points, respectively, before the loading and unloading proceeded. For the GMR characterization on biaxial stretching, the expanding diaphragm setup was placed between the magnet pole shoes instead of the stretching stage. GMR curves were recorded using a two-point resistance measurement at different inflation states, which are defined by an additional volume of 10 ml water pumped into the chamber before the next measurement.

NATURE COMMUNICATIONS | 6:6080 | DOI: 10.1038/ncomms7080 | www.nature.com/naturecommunications

& 2015 Macmillan Publishers Limited. All rights reserved.

7

13

Induction Magnetic Field Sensor as an Organ of Robot Vision Rostyslav SKLYAR Space Sensing Instruments, Verchratskogo st. 15-1, Lviv 79010 Ukraine Tel/Fax: +380-322-762432/769613; r_sklyar@hotmail.com

2 4 3

Abstract. The robotic vision method of the autonomous system based on the orientation of the plane and according to the magnitude and gradient of the natural and industrial magnetic fields (MFs) has been developed. The head sensor(s) of an ambient temperature or superconducting further electronic circuit placed on the robot limb(s) without modifying them. Sensitivity of the robotic vision makes it possible to recognize the linear translation of 10-2 m and disposal in space of 10-3 m3.

I. Introduction. Magnetometers and their robotic applications

5

The measurement of MFs is an important task for the majority of autonomous missions. The distribution of permanent and the value of periodical MFs gives the data about placement of ferromagnetic objects and sources of EM radiation respectively. On the other hand, these signals will z

b

x

y

a

Fig.1 The w alking robotin M F environm ent. a) x-y plane defines the varying D C natural M F; b) around z axis spreads AC industrial M F interference.

2

6

7 11

Rostyslav SKLYAR

be a reference point and guiding line for a walking robot (Fig. 1).

Detection of some magnetic anomalies of the Earth MF and their variations is provided by fluxgate sensors [1]. When the spectrum of EM signals in the environment lies in a wide frequency range (10Hz-500kHz), the application of highly-sensitive induction sensors is necessary. The application of a low-Tc SQUID device is able to embrace both of the said frequency ranges with maximum sensitivity. It is made of non-magnetic material and the principle of operation is to detect two magnetic signals at different distances from the source and arrange for these to be in opposition around a local supercooled circuit. This ‘gradiometer’ approach eliminates most of the noise—caused by spurious MFs originating from, for example, electrical devices or natural (geomagnetic (GM) sources. A typical unshielded laboratory has a noise level in the 0—10 Hz frequency region of about 10-7 T and GM noise in the same frequency range is of the order of 10-10 T. Any small field changes the direction of the MF vector in the space and this produces a distortion in the waveform of the signal in the detection coils. The very expensive use of the SQUID magnetometer up until now has produced many advances in the understanding of MFs from weak sources. The advance of much cheaper room—temperature sensor technologies offers the prospect of much greater use of MF monitoring [2].

As a general rule, scientific instrumentation should impact mass constraints as little as possible; however, robot-mounted instruments in particular must be lightweight to survive deployment under high- stress conditions when used in connection with spacecraft or underwater. This weight constraint can limit both the size of the sensor and its placement on the moving part. Also, autonomous systems require the minimal consumption power. Both factors translate into severe constraints on the capability of the instrument to measure weak EMFs encountered in the environment.

In addition, the outer space and underwater conditions require endurance against variations in the wide range of temperature, humidity, and atmospheric pressure.

II. Induction (superconducting) sensor as the organ of vision Taking into account the said restrictions, it is undesirable to use in robot a fluxgate sensor due to its active type of action. Also is impossible exploiting SQUID device since it needs the liquid helium container.

SQUID systems being almost ideal for the detection of very small amounts of flux from small samples, but less well-suited to the detection of low flux densities (i.e. where very low MFs are encountered). The high coil induc-

!"#$%&! '(%)* +%&,&--./. )*#01 2 3.%+4(-*( .*)+*)*2+(*.

c

c

"limb

a

b

" limb

8

a

3

Fig. 2 Placement of PC on the limb. a)a winding of PC; b) a ferromagnetic core of PC as a part of the limb(s); c) a wire which connects PC with an electronic circuit in the body.

tance of the induction system, on the other hand, can couple to a large quantity of flux [3]. It appears that these difficulties frequently reduce, in principle, the performance of SQUID magnetometers to a level below that demonstrated in this article using an ambient-temperature induction system. This induction sensor is also compact, robust, operates at room temperature, exhibits a wide dynamic range, and may be easily integrated into differential or multiple sensor gradiometric configurations which are feasible in a multi-limb walking robot.

1. Placing of the PCs on the limbs

10

A PC of walking robot is connected in parallel with the drain of a SuFET cryogenic device or an ordinary OA which are placed in the body (Fig. 2). A PC realizing the oscillatory-forward movement along both AC industrial interferences and quasi-DC natural (Earth) environmental MFs. These fields are distributed on a surface and in space roughly according to Fig. 1. The movement in quasi-DC MF HDC with the defined speed ! and oscillating frequency " with a magnitude #$ gives e.m.f. from PC: (1) EPC=SN"µeffµ0HDCsin$%#$ , where S=µeff&d2N/4 with µ0 -the permeability of free space, µ0=4&%10-7 henry/meter; µeff -the effective relative permeability of a high-µ metal core; d-the average diameter of a PC; N-total turn number of solenoid; $- an angle between PC's magnetic axis and the vector of HDC. Further amplifying/processing circuit depends on the measuring conditions and can vary from the simplest of the ordinary induction sensor modifications to the superconducting one [4].

4

Rostyslav SKLYAR

2. The design of the MF transducer

13

Some combined device, that includes all the best features of the said MF sensors/transducers seems to be the preferable trend for further development as a vision organ [4]. The SuFET is implemented into a wideband induction sensor device in order to acquire the sensitivity threshold below 1fT/'Hz in the frequency range from small values of Hertz to tens of MHz (0.1Hz-107 Hz). The proposed magnetometer (SIM) circuit consists of both room-temperature or cooled (up to superconductive) pickup coil (PC) and a SuFET. Moreover, it gives the possibility of repudiating both windings and electronics of feedback loops that are used in the known magnetometers.

Magnetic induction BPC of AC MF with the frequency "limb of limbs' oscillations produce an e.m.f. in PC: (2) EPC=BPC"limbSN, where S- a cross-section of PC, N- its number of turns. All AC MFs with the frequencies ", high than "limb can be rejected by the passive HF filter [5].On the other hand, the value of EPC can be determined from the output voltage Uout of the specific kind of the induction transducer [4] with known its transfer function G according to the formula: (3) Uout=GEPC. As a result, an output signal receiving spontaneously, during twodimensional travel of a walking robot in a quasi-DC MF. Moreover, by picking up the signals from both horizontal and vertical parts of the limbs, the robot derives its' directional information from the axial course of the field lines and their inclination (defined as the angle between the direction of the field lines and the horizontal) in space. Executing the oscillations of PC with parameters (number of turns N=2%104 and a cross-section area S=&%10-4 m2) [4] in the earth MF B0=50 mkT [6] with the frequency 2 Hz and $=30(, #$=30( arouse e.m.f. with a magnitude 50 mV according to Eq. 1. This e.m.f. will be on average equal to 5 mkV for the possible variations of the said MF with magnitude 1)10 nT [6]. Hence these data it is possible to calculate the value of Uout for all five known of the induction transducer [4] with given parameters of their electrical circuit. Thus so then: a) for the basic transducer's variant with PC's resistance R=2.4 kOhm, inductance L=30 H and capacitance C=50 pF the values of Uout shown in the second column of the table; b) the voltage feedback resistance Rfb=27 kOhm is introduced into the circuit; c) the magnetic flux feedback turns Nfb=1000 is introduced into the device; d) for SuFET the constant partial of gate voltage equal to 0.1 V and "T, which closely relates to the small signal transconductance, are defined [4].

!"#$%&! '(%)* +%&,&--./. )*#01 2 3.%+4(-*( .*)+*)*2+(*.

5

Table. The dependence of signal's value Uout from the varying MF BPC. a b c d BPC!Uout 50 mkT 10 V 70 mV 1.4 V 3 mkV+0.1 V 5 nT 1 mV 7 mkV 0.14 mV 0.3 pV+0.1 V (0.1 mkT) III. Using an ambient DC and AC MFs for robot's walking

9

An output signal of the sensor will be involved into differencial (gradiometric) operation between the robot’s limbs. After envelope detection of the quantity Uout, can be presented by changing the corresponding quasiDC MF by the robot’s movement as defined in Fig. 3. In the similar manner, way an AC MF partial can be shown.

HDC, HAC, Vout

1-

1 2

oscillations of a PC modulated by variations of external natural MF;

2-

an envelope of sensor's output voltage UDC as the appropriate quasiDC MF along the walking way;

3-

changing of an AC industrial MF interference into the travel space;

4-

the integral output voltage UAC which determines changing of the interference's power by a distance.

Fig. 3 The variations of measured MF strength HDC, HAC.

4 3

t

The linear travel of the robot can be derived from the known distance between any two limbs and measured during the movement gradient of MF #H, which is presumed constant. Otherwise, the robot can be walking in the direction of minimal or maximal MF strength H according to the said gradient (Fig. 4). In both cases the precision of movement will be defined by the sensors’ sensitivity, which lies in a range of the orders from pT/'Hz to fT/'Hz .

1. Determination of the earth's MF gradient A portable single axis magnetic gradiometer, which is a relative instrument because it measures the spatial variation of the MF, has been described [7]. The finite distance between the magnetic sensors d for detecting the field difference is used to get an expression for the estimate of the exact magnetic gradient, adjusted by a function of the field distance, #Bz=Bz(z+(1/2)d)-Bz(z-(1/2)d), as follows:

6

Rostyslav SKLYAR

-B z z

#Bz *0 md 2 , +5 d 4&z 6

,

(4)

where *0=4&10-7 H/m, z is the distance from the dipole to the center of the gradiometer probe and m the magnetic moment of the dipole.

In the case to be treated in the following, the oscillations of the limbs are taking place according to the law closer to harmonical. Then the magnitude value of magnetic induction, which produced by oscillations of PC in the ambient DC MF Bz is: (5) BPC=Bz·sin$·#$ , where $ is an angle between PC's direction of detection and vector of B, #$ is a range (of a value) of this angle variation.

Having substituted Eqs. 1, 3, 5 into Eq. 4, we have the dependence between signals from PCs of any two limbs and the relevant magnetic gradient occurs:

-B z z ,

U out .(z + (1/2)) - (z - (1/2))/ * md 2 +5 0 6 G" limbSNsin$#$ 4&z

(6)

A first order radial gradiometer consists of two axially displaced magnetometer loops, wound in opposing sense from a common wire that is connected to the induction sensor. Two radial magnetometers with opposite polarity can be connected together to form planar gradiometers [8].

Such devices detect tangential gradient of the radial field, and two gradiometers are usually used at each site to detect two orthogonal planar gradients. The planar gradiometer behaviour is qualitatively different from that of the radial gradiometer.

2. Measuring of the MF signal (noise) in a triaxial arrangement A flux transformer detects more environmental noise if its baseline is long (or if it is a magnetometer). Thus long baseline gradiometers detect not only stronger signal from deep sources, but also larger environmental noise. The noise parameters for different baselines were measured and are shown graphically [8].

Gradiometers can also be configured to detect radial gradient of the tangential MF. Two orthogonal ‘tangential radial gradiometers’ and one ‘radial gradiometer’ can be combined to form a first-order gradiometer equivalent of the vector magnetometer. The vector of the industrial or household man-made AC MF (noise) " can be measured during the complete pass of the robot's walking. Placing of the PC's triplets (the three orthogonal components at each location) on the respective limbs dive the necessary data for the triaxial MF determination according to the geomet

!"#$%&! '(%)* +%&,&--./. )*#01 2 3.%+4(-*( .*)+*)*2+(*.

7

rical summation. In such case, frequency of the limb's oscillations much lower than ambient MF noise. That is why, they do not influence the measured components and, moreover, these oscillations can be additionally suppressed by a passive LF filter [9]. The value of MF induction along a single component will be calculated similarly to Eq. 1 and Eq. 2 by the formula: (7) BPC="SNUout/G Some example of the result of the calculations according Eq. 7 shown in Fig. 4. The dependence of volumetric error on the baseline for environmental noise is shown [8].

z

#HAC(n)

#HAC(m) x

#HDC(m)

#HDC(n)

y

grad(#HDC)

Fig. 4 Orientation of the walking robot in an environmental MF. a) distribution of the detected gradients of DC natural MF over the x-y plane; b) distribution of DC MF interference in the space around z axis as a difference between the limbs.

IV. Conclusions

12

The attempt to solve this problem of simplifying the robotic vision structure and reducing of such performance data as mass, volume and power consumption by the employing of the induction (also superconducting)

8

Rostyslav SKLYAR

sensor was done. This method allows us to use the oscillations of limbs during robot's walking for receiving the measured signal and make the vision more natural for this machine. The advanced sensor has an unlimited frequency range from DC to HF, and sensitivity to the ambient MF is closer to the theoretical possibility.

V. References [1] Cerman A., Ripka P., Ka!par P., “Precise Magnetic Sensors and Magnetometers for Military and Space Applications”, Sensors & Transducers Magazine, vol. 38, iss. 12, 2003, pp. 54-58; [2] Mapps D. J., “Remote Magnetic Sensing of People”, Sensors and Actuators A, vol. 106, 2003, pp. 321-325; [3] Prance R. J., Clark T. D. and Prance H., “Compact RoomTemperature Induction Magnetometer With Superconducting Quantum Interference Device Level Field Sensitivity”, Rev. Sci. Instrum., vol.74, No. 8, 2003, pp. 3735-3739; [4] Sklyar R., “SIM- A SuFET Based Induction Magnetometer”, Proceedings of 'Recent Advances of Space Technology: RAST-2003', Nov. 20-22, 2003, Istanbul, Turkey, pp.193-199; [5] Zambresky L. F., Watanabe T., “Equivalent Circuit of a Magnetic Sensor Coil and a Simple Filter for Rejection of 60 Hz Man- Made Noise”, J.Geomag. Geoelectr., vol. 32, 1980, pp. 325-331; [6] Ripka P., “Review of Fluxgate Sensors”, Sensors and Actuators A, vol. 33, 1992, pp. 129-141; [7] Merayo J. M. G., Petersen J. R., Nielsen et al., “A Portable Single Axis Magnetic Gradiometer”, Sens. Act. A, vol. 93, 2001, pp. 185-196; [8] Vrba J. and Robinson S E, “SQUID sensor array configurations for magnetoencephalography applications (Topical review)”, Supercond. Sci. Technol., vol. 15, 2002, pp. R51–R89; [9] Sklyar R., “Suppression of Low-Frequency Interferences in the Induction Sensor of Magnetic Field”, has been considering for publication in Measurement (M03/73).

1

Positioning, 2011, 2, 65-77 doi:10.4236/pos.2011.22007 Published Online May 2011 (http://www.SciRP.org/journal/pos)

Machine Perception Through Natural Intelligence Rostyslav Sklyar Verchratskogo st. 15-1, Lviv 79010 Ukraine Email: sklyar@tsp.lviv.ua Received December 30th , 2010; revised March 1st , 2011; accepted April 20th, 2011.

ABSTRACT sensing organs are exponentially better than any of analogous artificial ones. That is why using them in full scale is 5 The a perspective trend to the efficient (advanced) machine perception. On the other hand, limitations of sensing organs could be replaced by the perfect artificial ones with the subsequent training the nervous system on their output signals.

of physical quantities, also with their expansion into space arrays and external/implantable functioning in rela2 ducers tion to the nervous system is performed. The advances in nanotechnology are opening the way to achieving direct eleccontact of nanoelectronic structures with electrically and electrochemically active neurocellular structures. The 3 trical transmission of the sensors’ signals to a processing unit has been maintaining by an electromagnetic transis-

An attempt to lay down the foundations of biosensing by natural sensors and in addition to them by the artificial trans-

4

tor/memristor (externally) and superconducting transducer of ionic currents (implantable). The arrays of the advanced sensors give us information about the space and direction dynamics of the signals' spreading.The measuring method and necessary performance data of the sensor for the robot’s orientation in the ambient magnetic field with living being-machine interaction in order to obtain input and output signals from brain and motor nerves to the measurement system and vice versa are introduced. The range of applied sensors differs from an induction sensor to superconducting induction magnetometer. The analytical expressions for arrangements of the head sensors in differential and vector (3D) relative positions are deduced. Sensitivity of the perception method makes it possible to recognize the linear translation of 10!2 m and disposal in space of 10!3 m3. Interaction between living beings and robotic equipment is given analytical treatment.

Keywords: Magnetic Field, Induction Sensor, SuFET, Nerve Impulses, Interface, Gradiometer, Sensing Area

1. Introduction. Artificial Sensors with the Human Machine Interface Electronic Nose is a smart instrument that is designed to detect and discriminate among complex odours using an array of sensors. The array of sensors consists of a number of broadly tuned (non-specific) sensors that are treated with a variety of odour-sensitive biological or chemical materials [1].

This instrument provides a rapid, simple and noninvasive sampling technique, for the detection and identification of a range of volatile compounds. The key function of an electronic nose is to mimic human olfactory system. Typically an electronic nose consists of three elements: a sensor array which is exposed to the volatiles, conversion of the sensor signals to a readable format and software analysis of the data to produce characteristic outputs related to the odour encountered. The main parts of a typical biosensor are shown in Figure 1.

The artificial tactile sensor integrates a micro electro-mechanical system (MEMS) array having a number

of sensing elements (16 channels in about 20 mm2) similar to the innervation density of mechanoreceptors in the hand (about 1 unit/ mm2). The technological approach is based on a 3D MEMS core unit with a soft and compliant packaging. The microsensor can be integrated with a packaging architecture resulting in a robust and compliant tactile sensor for application in artificial hands, while sensitive enough to detect slip events, showing that silicon based tactile sensors can go beyond laboratory practice [2]. The tactile sensor array, depicted in Figure 2, had 16 channels as total tactile sensor outputs.

The measurement of magnetic fields (MFs) is an important task for the majority of autonomous missions.

The distribution of permanent and the value of periodical MFs give the data about placement of ferromagnetic objects and sources of EM radiation respectively. On the other hand, these signals will be a reference point and guiding line for a walking robot (Figure 3).

Detection of some magnetic anomalies of the Earth’s MF and their variations is provided by fluxgate sensors

Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

66

Machine Perception Through Natural Intelligence

Figure 1. (A) The main parts of a typical biosensor, (B) A sensor array, each polymer changes its size and therefore its resistance, by a different amount and making a pattern of the change. Copyright 2009 IFSA.

Figure 2 (a) 3D design of the tactile sensor array. (b) Top: the 2 × 2 MEMS array compared with human finger; bottom: a FIB image of the MEMS sensor. (c) Top view of the sensor array. Copyright 2008 IEEE.

Figure 3 The walking robot in MF environment. a) x-y plane defines the varying DC natural MF; b) around z axis spreads AC industrial MF interference.

[3]. When the spectrum of EM signals in the environment lies in a wide frequency range (10 Hz - 500 kHz),

the application of highly-sensitive induction sensors (ISs) is necessary. The application of a low-Tc SQUID device is able to embrace both of the said frequency ranges with maximum sensitivity. It is made of non-magnetic material and the principle of operation is to detect two magnetic signals at different distances from the source and arrange for them to be in opposition around a local supercooled circuit. This ‘gradiometer’ approach eliminates most of the noise—caused by spurious MFs originating from, for example, electrical devices or natural (geomagnetic sources. A typical unshielded laboratory has a noise level in the 0 - 10 Hz frequency region of about 10!7 T and geomagnetic noise in the same frequency range is of the order of 10!10 T. Any small field changes the direction of the MF vector in the space and this produces a distortion in the waveform of the signal in the detection coils. The very expensive use of the SQUID magnetometer up until now has produced many advances in the understanding of MFs from weak sources. The advance of much cheaper room—temperature sensor technologies offers the prospect of much greater use of MF monitoring [4]. As a general rule, scientific instrumentation should impact mass constraints as little as possible; however, robot-mounted instruments in particular must be lightweight to survive deployment under highstress conditions when used in connection with spacecraft or underwater. This weight constraint can limit both the size of the sensor and its placement on moving part.

Also, autonomous systems require minimal consumption power. Both factors translate into severe constraints on the capability of the instrument to measure weak EMFs encountered in the environment. In addition, outer space and underwater conditions require endurance against variations in the wide range of temperature, humidity, and atmospheric pressure.

The human machine interface using electromyjogram (EMG) signal to artificially control the limb movement.

Each muscle fiber has a potential and motor unit action potential generated by construction of muscle is studied and corresponding actuation is provided to robotics arm.

In this system a relatively more flexible and robust communication technique generates commands from EMG signals (Figure 4). Such an autonomous mobile humanoid robot will be able to assist in a workshop environment and interact with a human [5]. Three aspects to reach this target are considered. First, how the brain of the exoskeleton robot is constructed to be able to manipulate objects like humans do. Second exploration of appropriate binominals that can be used as an actuating quantity to move the Robotic arm; and finally how programming of manipulation tasks is realized.

Implantable neural probes are generally preferred to

Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

Machine Perception Through Natural Intelligence

11

have a minimum footprint as possible to minimize neural damage and to facilitate easy entry and movement through the brain tissue. Polymeric microprobes have received a great deal of attention owing to their simple fabrication process, flexibility and biocompatibility. Implantable neural probes for neuroscience and brain machine interfaces are generally preferred to have a minimum footprint as possible to minimize neural damage [6].

A conceptual representation of the multichip recording interface is shown in Figure 5. The device handles 16 neural recording channels as a proof of concept [7].

Alternatively, a superconducting field-effect transistor (SuFET) based neurotransducer (SuFETTr) with carbon nanotubes (CNT) or pickup coil (PC) kind of input circuit for the nerve and neuronic impulse has been designed [8]. A nanoSuFET with a high-temperature superconducting channel is introduced into the nerve fibre or brain tissue for transducing their signals in both directions. As a result, we have achieved SuFETTr that is suitable for ascertaining the variety of values [9]. Two directions of SuFETTr function enable decoding of these signals by comparing the result of its reaction on some process or organ with an action on them of the simulated electrical or biochemical signal after their reverse transducing through the SuFETTr(s). Moreover, a number of the relevant measuring devices, as an intermediate element between physical value and the nerve system, are designed on its basis [10].

2. Signal Processing in the Biointerfaces Magnetocardiography (MCG) is a noninvasive method of detecting the cardiac magnetic fields above the body surface using DC SQUIDs. MCG has the potential to detect arrhythmia and ischemia with a lower level of adverse effects than electrocardiograms. Also most SQUIDs are incorporated into whole-head systems for magnetoencephalography (MEG) - the detection of MFs produced by brain. A typical helmet contains about 300 sensors, including a number of reference sensors for noise cancellation, cooled to 4.2 K. [11] A current trend in such diagnostics are portable and wearable magnetometer systems which have, in the same time, high signal processing and signal-to-noise ratio parameter data characteristics. Simultaneous amplification of the piking up MCG & MEG neurosignals by an on-site magnetic flow has been implemented. Matrix of the sensing micro-nanoPCs are distributed under the heart or brain surface with temperature of the body [12].

Implantable neural probes are generally preferred to have a minimum footprint as possible to minimize neural damage and to facilitate easy entry and movement through the brain tissue. Polymeric microprobes have

67

received a great deal of attention owing to their simple fabrication process, flexibility and biocompatibility. Implantable neural probes for neuroscience and brain machine interfaces are generally preferred to have a minimum footprint as possible to minimize neural damage [13].

The concept of an implantable wireless platform, without any percutaneous skin-puncturing elements, demands the implementation of a heterogeneous, active microelectronics platform. At a minimum, the implanted “microsystem� requires in-situ integration of ultralow-power microelectronic ASICs with the cortical microelectrode neural probes, and must provide broadband telemetry and a means to deliver power wirelessly to the active implanted components [14].

A 16-channel version of a fully implantable microsystem has been implemented and tested on the benchtop and used in initial animal tests. This microsystem is a single-unit construct (Figure 6) where analog and digital chips are integrated on a flexible substrate together with a low threshold, infrared semiconductor diode laser to transmit the digitized neural signals through the skin.

Power and clocking are delivered to the system via inductive coupling, but the system can also be configured to be powered optically using a high efficiency photovoltaic energy converter.

By implementing CMOS technology for neural imaging, not only is imaging in multiple modalities possible, electrical stimulation and sensing can be integrated onto a single device, thereby resulting in a highly compact and miniaturized tool for the study of the brain (Figure 7) [15].

Improvement in the packaging process has resulted in

Figure 4. EMG has HF content (up to 250 Hz). Copyright 2008 IFSA.

Copyright Š 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

Machine Perception Through Natural Intelligence

6

Physiological Signals is sketched in Figure 9. The measurement system consists of a transducer to pick the human physiological signal, amplifier system, hardware band-pass filter, fifty hertz notch filter, DSC and a personal computer as the output display device.

An example of further processing of the sensing signals is analogous to the combination of heart rate and body position information recorded or monitored simultaneously can provide a valuable set of physiological and behavioral data related to the physical activity and cardiac activity. The added knowledge of the subject’s body position and activity intensity provides very valuable information and creates a history for monitoring, diagnostics, or health alerts. A discrete and minimally obtrusive wearable device allowing synchronous measurement of these physiological parameters can further offer enhanced mobility, for the very first time, to the collection and monitoring of these datasets for both research and practical applications [18].

The acceleration and bio-potential signals taken from the multi-functional sensor are first amplified through a custom signal conditioning circuit. The resulting signals are then inputted into the analog-digital converter (ADC) used to digitize the output signals. Figure 10 shows an overview diagram of the accelerometer data collection and processing system. This prototype device is able to detect different body positions such as standing, sitting, and lying on different sides of the body. It also processes motion sensor data by calculating the variance of the sample data, in order to determine movement intensity.

3. IS as the Organ of Sixth Feeling Taking into account the said restrictions, it is undesirable to use in the robot a fluxgate sensor due to its active type of action. Also is impossible to exploit a SQUID device since it needs a liquid helium container. SQUID systems being almost ideal for the detection of very small amounts of flux from small samples, but less well-suited to the detection of low flux densities (i.e. where very low MFs are encountered). The high coil inductance of the induction system, on the other hand, can couple to a large quantity of flux [19]. It appears that these difficulties frequently reduce, in principle, the performance of SQUID magnetometers to a level below that demonstrated in this article, using an ambient-temperature induction system. This IS is also compact, robust, operates

69

at room temperature, exhibits a wide dynamic range, and may be easily integrated into differential or multiple sensor gradiometric configurations which are feasible in a multi-limb walking robot.

A new method to assess a body inclination/azimuth using 3-D Earth’s MF sensor, even in dynamic conditions was proposed [19]. Indeed, a magnetometer is insensitive to acceleration while responding to a change of orientation in the 3-D space. However, in the general case, a 3-D magnetometer is not sufcient to track a 3-D orientation. Magnetometers are more commonly employed to either determine a compass heading information or to compensate a rate gyroscope integration drift.

The main point is that by restricting ourselves to particular movements with one or two degrees of freedom (DOFs), a magnetometer is sufcient to estimate 2 DOFs: the movement is constrained in one plane which can rotate around one xed (or not) axis. For example, during walking, we can be interested in rotations in the saggital plane (e.g., knee exion/extension) and azimuth direction of walk. Although limited to two DOFs, it is important to note that we can address a movement in 3-D space. Figure 11 presents the different normalized accelerations measured in t!hme sensor frame: the calibrated measured ! " acceleration v acc : dark "! solid line, the predicted gravitational component g B : !dotted line, and the computed kinematical component a B : light solid curve.

6

3.1. Placing of the PCs on the Limbs PC of a walking robot is connected in parallel with the drain of a SuFET cryogenic device or an ordinary OA which are placed in the body. A PC realizing the oscillatory!forward movement along both AC industrial interferences and quasi-DC natural (Earth) environmental MFs. These fields are distributed on a surface and in space roughly according to Figure 3. The movement in quasi-DC MF HDC with the defined speed v and oscillating frequency ! with a magnitude "" gives e.m.f. from PC:

EPC & SN #$eff $0 H DC sin % ' (%

(1)

where S=µeff # d2 N/4 with µ0- the permeability of free space, µ0=4#·10-7 henry/meter; µeff- the effective relative permeability of a high-µ metal core; d- the average diameter of a PC; N-total turn number of the solenoid; "-

Figure 9. Block diagram of DSP based real time embedded system for acquisition of bio-signals. Copyright 2009 IFSA. Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

10

70

Machine Perception Through Natural Intelligence

the frequency !limb of limbs' oscillations produce an e.m.f. in PC: EPC " BPC !lim b SN

Figure 10. Sensor data extraction setup. Copyright 2007 IEEE.

Figure 11. Separation of the gravitational and kinematical components in body frame. Accelerations during a sitto-stand transition: measured acceleration (solid line), predicted gravitational component (dotted line), and computed kinematical component (dashed line). xb-axis (top) denotes the craniocaudal axis, pointing upward; zb-axis (bottom) is the anteroposterior axis, pointing forward. Copyright 2007 IEEE.

7

an angle between PC’s magnetic axis and the vector of HDC. Further the amplifying/processing circuit depends on the measuring conditions and can vary from the simplest of the ordinary IS modifications to the superconducting one [21].

1) The design of the MF transducer Some combined device, that includes all the best features of the said MF sensors/transducers seems to be the prefered trend for further development as a perception organ [21]. The SuFET is implemented into a wide-band IS device in order to acquire the sensitivity threshold below 1fT/!Hz in the frequency range from small values of Hertz to tens of MHz (0.1 Hz!"!107 Hz). The proposed superconducting induction magnetometer (SIM) circuit consists of both a room-temperature or a cooled (up to superconductive) PC and a SuFET. Moreover, it gives the opportunity of repudiating both windings and electronics of feedback loops that are used in the known magnetometers. Magnetic induction BPC of AC MF with

7

(2)

where S- a cross-section of PC, N- its number of turns.

All AC MFs with the frequencies !, high than !limb can be rejected by the passive HF filter [22].On the other hand, the value of EPC can be determined from the output voltage Uout of the specific kind of IS [21] with its known transfer function G according to the formula: U out " GEPC

(3)

As a result, an output signal receiving spontaneously, during two-dimensional travel of a walking robot in a quasi-DC MF. Moreover, by picking up the signals from both horizontal and vertical parts of the limbs, the robot derives its’ directional information from the axial course of the field lines and their inclination (defined as the angle between the direction of the field lines and the horizontal) in space. Executing the oscillations of PC with parameters (number of turns N = 2!104 and a crosssection area S = "!10#4 m2) [13] in the earth MF B0 = 50 mkT [23] with the frequency 2 Hz and " = 30, $" = 30 arouse e.m.f. with a magnitude 50 mV according to Equation. 1. This e.m.f. will be on average equal to 5 mkV for the possible variations of the said MF with magnitude 1 - 10 nT. With these data it is possible to calculate the value of Uout for all the known five of ISs [13] with given parameters of their electrical circuit. So then (see Table 1 and Figure 4): a) for the basic transducer's variant with PC's resistance R = 2.4 kOhm, inductance L = 30 H and capacitance C = 50 pF the values of Uout shown in the second column of Table 1; b) the voltage feedback resistance Rfb = 27 kOhm is introduced into the circuit; c) the magnetic flux feedback turns Nfb = 1000 are introduced into the device; d) for SuFET the constant partial of gate voltage equal to 0.1 V and !T, which closely relates to the small signal transconductance, are defined [13].

3.2. Using an Ambient DC and AC MFs for Robot’s Perception An output signal of the sensor will be involved into the differential (gradiometric) operation between the robot’s limbs. After envelope detection of the quantity Uout, can be presented by changing the corresponding quasi-DC MF by the robot’s movement as defined in Figure 12. In Table 1.The dependence of signal’s value Uout from the varying MF BPC. BPC\Uout

a

b

c

d

50 mkT

10 V

70 mV

1.4 V

3 mcV + 0.1 V

5 nT

1 mV

7 mcV

0.14 mV

0.3 pV + 0.1 V

Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

9

71

Machine Perception Through Natural Intelligence

variation of the MF, has been described [24]. The finite distance d between the magnetic sensors for detecting the field difference is used to get an expression for the estimate of the exact magnetic gradient, adjusted by the function of the field distance, !Bz = Bz (z + (1/2)d) ! Bz(z ! (1/2)d), as follows: $Bz

9

Figure 12. The variations of measured MF strength HDC, HAC: 1-oscillations of a PC modulated by variations of external natural MF; 2-an envelope of sensor’s output voltage UDC as the appropriate quasi-DC MF along the walking way; 3-changing of an AC industrial MF interference into the travel space; 4-the integral output voltage UAC which determines changing of the interference's power by a distance.

a similar manner, way an AC MF partial can be shown.

The linear travel of the robot can be derived from the known distance between any two limbs and measured during the movement gradient of MF !H, which is presumed constant. Otherwise, the robot can be walking in the direction of minimal or maximal MF strength H according to the said gradient (Figure 13). In both cases the precision of movement will be defined by the sensors’ sensitivity, which lies in a range of the orders from pT/"Hz to fT/"Hz. 1) Determination of the earth’s MF gradient A portable single axis magnetic gradiometer, which is a relative instrument because it measures the spatial

Figure 13. 3D Orientation of the walking robot in an environmental MF: a) distribution of the detected gradients of DC natural MF over the x-y plane; b) distribution of DC MF interference in the space around z axis as a difference between the limbs.

z

%

' md 2 #Bz &5 0 6 d 4" z

(4)

where z is the distance from the dipole to the center of the gradiometer probe and m the magnetic moment of the dipole.

In the following case, the oscillations of the limbs are taking place according to the law closer to harmonical.

Then the magnitude value of magnetic induction, which produced by oscillations of PC in the ambient DC MF Bz is: BPC ) Bz * sin ( * #(

(5)

where ! is an angle between PC's direction of detection and vector of B, !! is a range of a value of this angle variation.

Having substituted Equations. 1, 3, 5 into Equation. 4, we have the dependence between signals from PCs of any two limbs and the relevant magnetic gradient occurs: $Bz

z

%

U out .0+ z & 1 2 , ! + z ! 1 2 , /1 ' md 2 &5 0 6 G-lim b SN sin ( * #( 4" z

(6)

A first order radial gradiometer consists of two axially displaced magnetometer loops, wound in an opposing way from a common wire that is connected to IS. Two radial magnetometers with opposite polarity can be connected together to form planar gradiometers [24]. Such devices detect the tangential gradient of the radial field, and two gradiometers are usually used at each site to detect two orthogonal planar gradients. The planar gradiometer behaviour is qualitatively different from that of the radial gradiometer.

2) Measuring of the MF signal in a triaxial arrangement A flux transformer detects more environmental noise if its baseline is long (or if it is a magnetometer). Thus long baseline gradiometers detect not only stronger signals from deep sources, but also larger environmental noise.

The noise parameters for different baselines were measured and are shown graphically [25]. Gradiometers can also be configured to detect the radial gradient of the tangential MF. Two orthogonal ‘tangential radial gradiometers’ and one ‘radial gradiometer’ can be combined to form a first-order gradiometer equivalent of the vector magnetometer. The vector of the industrial or household man-made AC MF (noise) # can be measured during the

Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

72

Machine Perception Through Natural Intelligence

complete pass of the robot’s walking. Placing of the PC’s triplets (the three orthogonal components at each location) on the respective limbs give the necessary data for the triaxial MF determination according to the geometrical summation. In such a case, the frequency of the limb’s oscillations is much lower than ambient MF noise. That is why, they do not influence the measured components and, moreover, these oscillations can be additionally suppressed by a passive LF filter [26]. The value of MF induction along a single component will be calculated similarly to Equation. 1 and Equation. 2 by the formula: ! SNU out BPC " (7) G The dependence of volumetric error on the baseline for environmental noise is shown [25].

3.3. Experimental Validation of the Method

8

In the Figure 6 are shown the oscillations of an electromotive force 2-3 of the described PC with respect to the basic voltage 1, which are conform to MF B0 with an onward movement speed of IS 1 m/s. The signal-wave envelope 4 is a result of the informational-measuring process and confirming about the space distribution of the MF variations, which are bases for the advanced perception-guide abilities. The levels of the IS’s output signals and boundaries of their changes in reality for all the described devices’ variants are shown in Figure 14.

In order to improve reliability of the method and increase the working frequency intensity signal filtration outside, a working frequency is introduced. Interference suppression is achieved by introducing n-unit RC-filters into the input stages of preamplifier intensification.

The equation for LF filtration is of the sixth order relative to RC: - 1 & 6 2 * 2 '(+ # RC!L $ ) 32 # RC!L $ ' 16 " 0 % , L .

tively “cut-out” from the rest of oscillations, both of EM and mechanical origin [26].

3.4. MF Feeling with a Living Being-Machine Interface The design provides a means to update operating-, monitoring parameters, operational thresholds, and sensor and RF link specific firmware modules “over-the-air”.

It is composed of two main components - a sensor- wireless hardware interface and system integration framework, which facilitates the defining of interaction between sensors/actuators based on process needs. The intelligence necessary to process the sensor signals, monitor the functions against defined operational templates, and enable swapping of sensor and RF link, resides on the microcontroller of the hardware interface [27].

The basic scenario is shown in Figure 16. As a result living beings control drives by previously translated biosignals. In the other variant, biosignals from organs of the senses or brain transduce directly into intelligent or robotic systems which, in such a way, pick up environmental information.

Let us consider the connection between the input (biosignal Inerve) and output (MF B) variables of the autonomous system in the wide frequency range "B:[28]

(8)

where !L and "L are the interference suppression factors.

The experimental curve, given in Figure 15 confirms the computed values of suppression a transfer function G for a two-unit RC-filter. It is also possible to apply a twin-T RC-filter, but only for the HF- fH, relative to the working one fwork [22]. The response curve in the frequency range that is of importance for orientation, would be virtually flat GF, when R and C are related to each other as follows: 1 " f work (9) 2/ RC As a result, the working MF frequency will be effec-

Figure 14. The magnitude changes of orientate on MF frequency during the walking path.

Figure 15. The simultaneous effect of both LF and HF filters on interferences in wide frequency range.

Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

8

12

Machine Perception Through Natural Intelligence

4. Multisensor Data Fusion for the Medium and Environment

computing, and processing technologies, it now seems likely that insects could be navigated as mobile information gatherers. Therefore, our specific methodology has broad implications not only for studying the isect – machine interface but also for the future use of navigated insects as environmental sentinels. When instrumented with equipment to gather information for environmental sensing, such insects potentially can assist man to monitor the ecosystems that we share with them.

Highly miniaturized CMOS electronics on micromachined probes has enabled complex BCI. Developments in microprobe technology have shifted the notion of vertebrate implantable neuromotor prosthetics from science ction to reality. This technology provides techniques and tools to understand and generate robust electronically controlled muscle movement [30]. The adoption of the payload by the body, as the developing tissue forms around it, not only ensures a secure attachment to the insect, but also enables a highly predictable electronic interface to the insect’s sensorial, neural, or muscular systems. We call this procedure Early Metamorphosis Insertion Technology (EMIT). EMIT procedure allows for reliable IMI and hybrid bioelectronic structures where CMOS devices and MEMS structures are coupled with insects natural sensors and actuators. These systems can lead to the solution of many engineering challenges by understanding and manipulating the communication and automatic control systems of insects (Figure 18).

The implanted probes can also be used to record electrophysiological signals during natural ight to study the locomotive behavior. Further feedback studies of insect muscle can be conducted to optimize the ight control. In this study, the recording capability was used solely to assess the success of bioelectrical coupling (Figure 19).

Recent developments in bioengineering, nanotechnology, and soft computing make it possible to create a new generation of intelligent sensing. There are developing opportunities for combining natural and artificial sensing abilities in the synthesized system. Backed up by the rapid strides of nanotechnology, nanosensor research is making a two-directional progress, firstly in evolving new sensors employing mesoscopic phenomena, and secondly in the performance enhancement of existing sensors. Nanosensors are nanotechnology-enabled sensors characterized by one of the following attributes: either the size of the sensor or its sensitivity is in the nanoscale, or the spatial interaction distance between the sensor and the object is in nanometers. These nanosensors have been broadly classified into physical and chemical categories, with the biosensors placed on borderlines of biological signals with the remaining classes [29].

With continued miniaturization of information sensing,

Figure 17. The sensor!actuator signals ("Inerve!"B) for the different distances z under the effect of MF with some frequencies f1 … f4.

8 Figure 16. The living being-machine interface in the biorobotic perception system.

Seq I nerve " !B B zc

12

(10)

Taking into account Equation. (6) is possible to calculate the relation between sensor out-in (Inerve) and actuator in-out (Uout) respectively when robot’s limbs are oscillating with the frequency !limb.

Seq I nerve " !B U out # z 1 2 ' z ' 1 2$ %0 md 2 zc +5 G!!"#$ SN sin &(& 4) z 6

(11)

Simulated qualitative performance data of the human robot interface in the ambient MF B are shown in Figure 17.

13

73

Copyright © 2011 SciRes.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!POS

!"#$%&!"#$!%&'&()*!+*),'$-.%!/#&0#!)*$!#&,#(&,#.$1!&-!.#*$$!)..)0#$1!)*.&0($%2 !

34(5)*!62!!78)0#&-$!9$*0$:.&;-! "#*;<,#!=).<*)(!>-.$((&,$-0$?

I !"#$%&!!"#$%&'()'*#'+,%-&!!"#$%&"'

"#$%&#'!()$*''+,*)-* J '2(#,'3#&)!"#$%%&'(%#%)#*$+#,)-"#%.&# /)0",$%1)"2#)/#31)2&"21"4#3+#"$%05$*#2&"2)52# $",#1"#$,,1%1)"#%)#%.&'#3+#%.&#$5%1/161$*# %5$"2,06&52#)/#(.+216$*#70$"%1%1&28#$*2)#-1%.# %.&15#&9($"21)"#1"%)#2($6&#$55$+2#$",# &9%&5"$*:1'(*$"%$3*&#/0"6%1)"1"4#1"#5&*$%1)"#%)# %.&#"&5;)02#2+2%&'#12#(&5/)5'&,<

K '2(#,'3#&)=.&#$,;$"6&2#1"#"$")%&6.")*)4+#

34(5)*!62!!7>-1<0.&;-!8),-$.&0!@&$(1!

3$-%;*!)%!)-!A*,)-!;+!6;B;.!C&%&;-?

8$(D$*!82E!F)(.$-B*<--$*!82E!8)4)*;G!H2E!$.!)(2!

7>':$*0$:.&B($!8),-$.;$($0.*;-&0%?

!"#$%&!.&/0#,%-&(!"1&',%#(2%'3/( 4'&5-*!'()'*)+,-'*!./!0.1.$!2+3+.)

!"#$%&!(45*&-*5$+1'*(6"1&',-'3'#,*-&%#5( .#/,01-/(,$'(7!8(5'&5-*5(!"#$%&'

'2(#,'3#&)!"#$%&'&()*$+),)&-$.#("&/$&0$("#$

'2(#,'3#&)()*)+,$,-,.*+/01.$2310$4152$*/$5151.$04*)+,62$

12(&-&.&2,$,3,(#.$'1,#/$&-$("#$&%)#-(1()&-$&0$ ("#$451-#$1-/$1**&%/)-6$(&$("#$.16-)(2/#$1-/$ 6%1/)#-($&0$("#$-1(2%15$1-/$)-/2,(%)15$.16-#()*$ 7#5/,$89:,;$"1,$'##-$/#+#5&4#/<

/+17104-$8/*9$10$:)0.*1/04-1*;$40<$4"",4+40.,#$=-*9/)79$ 2/5,$/:$*9,$5)-*1:4.,*,<$"+/",+*1,2$/:$9)540$2310$54;$ +,5410$,>.-)21?,$*/$*9,$81/-/71.4-$2;2*,5@$,-,.*+/01.2$ /",02$4$)01A),$"4*9$*94*$-,4<2$8,;/0<$151*4*1/0$40<$./)-<$ ,A)1"$)2$B1*9$)0:451-14+$2,02,2#

'2(#,'3#&)=#-,)()+)(3$&0$("#$%&'&()*$+),)&-$

'2(#,'3#&)C,+,$B,$<,5/02*+4*,$7140*$5470,*/+,212*1?,$

.1>#,$)($4&,,)'5#$(&$%#*&6-)?#$("#$5)-#1%$ 2,02/+$:/1-2$B1*9$9179$2,021*1?1*;@$)054*.9,<$D,>181-1*;$40<$ $5&#)(&"1"4#%.&#-$+#%)#$6.1&;1"4#,15&6%# BC B &*&6%516$*#6)"%$6%#)/#"$")&*&6%5)"16#2%506%05&2# (%1-,51()&-$&0$@A $.$1-/$/),4&,15$)-$,41*#$&0$@A 5,.9401.4-$,0<)+40.,# D $.D< -1%.#&*&6%516$**+#$",#&*&6%5)6.&'16$**+#$6%1;&# "&05)6&**0*$5#2%506%05&2<

L '2(#,'3#&)=.&#'&$2051"4#'&%.),#$",# "&6&22$5+#(&5/)5'$"6&#,$%$#)/#%.&#2&"2)5#/)5# %.&#5)3)%>2#)51&"%$%1)"#1"#%.&#$'31&"%# '$4"&%16#/1&*,#?@AB#-1%.#*1;1"4#3&1"4C'$6.1"&# 1"%&5$6%1)"#1"#)5,&5#%)#)3%$1"#1"(0%#$",#)0%(0%# 214"$*2#/5)'#35$1"#$",#')%)5#"&5;&2#%)#%.&# '&$205&'&"%#2+2%&'#$",#;16&#;&52$#$5&# 1"%5),06&,<

M '2(#,'3#&)=.&#2&"21"4#)54$"2#$5&# &9()"&"%1$**+#3&%%&5#%.$"#$"+#)/#$"$*)4)02# $5%1/161$*#)"&2<#=.$%#12#-.+#021"4#%.&'#1"#/0**# 26$*&#12#$#(&52(&6%1;&#%5&",#%)#%.&#&//161&"%# ?$,;$"6&,B#'$6.1"&#(&56&(%1)"<#D"#%.&#)%.&5# .$",8#*1'1%$%1)"2#)/#2&"21"4#)54$"2#6)0*,#3&#

'2(#,'3#&)!"#$"#1/$,#-,&%8,;$&0$1-$1.')#-($

'2(#,'3#&)E9,;$4+,$*91.3@$,>*+,5,-;$D,>18-,@$-179*B,179*$

(#.4#%1(2%#$&%$,24#%*&-/2*()-6$02%("#%$ #5#*(%&-)*$*)%*2)($451*#/$&-$("#$%&'&($5).'8,;$ E)("&2($.&/)03)-6$("#.<

40<$B,4+48-,$42$15",+.,"*18-,$5470,*/F2,021*1?,$2310$*94*$ ,048-,2$"+/>151*;$<,*,.*1/0@$04?174*1/0$40<$*/).9-,22$ ./0*+/-#$E9,2,$)-*+4*910$5470,*1.$G,-<$2,02/+2$+,4<1-;$ ./0:/+5$*/$)81A)1*/)2$/8H,.*2$10.-)<107$9)540$2310$40<$ /::,+$4$0,B$2,02,$:/+$2/:*$+/8/*1.2@$24:,*;$40<$9,4-*9.4+,$ 5/01*/+107@$./02)5,+$,-,.*+/01.2$40<$,-,.*+/01.$2310$ <,?1.,2#

E9,$5,42)+,5,0*$/:$I(2$12$40$15"/+*40*$*423$ :/+$*9,$54H/+1*;$/:$4)*/0/5/)2$51221/02#$E9,$ <12*+18)*1/0$/:$",+540,0*$40<$*9,$?4-),$/:$ ",+1/<1.4-$I(2$71?,2$*9,$<4*4$48/)*$"-4.,5,0*$ /:$:,++/5470,*1.$/8H,.*2$40<$2/)+.,2$/:$JI$ +4<14*1/0$+,2",.*1?,-;#$K0$*9,$/*9,+$940<@$*9,2,$

I470,*/.,"*1/0$12$4$2,02,@$B91.9$4--/B2$84.*,+14@$102,.*2$ 40<$,?,0$?,+*,8+4*,2$2).9$42$81+<2$40<$294+32@$*/$<,*,.*$ 5470,*1.$G,-<2$:/+$/+1,0*4*1/0$40<$04?174*1/0#$C)5402$4+,$ 9/B,?,+$)048-,$*/$",+.,1?,$5470,*1.$G,-<2$04*)+4--;@$8)*$ ,-,.*+/01.$2310$./)-<$2//0$9,-"$*/$8+1<7,$*912$74"#$!"#$L&

!"#$%&"'()*(+,"(#"!-"&+(%!+.-.&.%$(/0"1(2.+,(+,"( 13)1"43"0+(+!%.0.05(+,"(0"!6/31(1*1+"7(/0( +,".!(/3+#3+(1.50%$18

! 98(:;(%1(+,"(<!5%0(/-(;.=+,(>""$.05 ?,.1(.0'3&+./0(1"01/!(@:;A.1(%$1/(&/7#%&+B( !/)31+B(/#"!%+"1(%+(!//7(+"7#"!%+3!"B("=,.).+1( %(2.'"('*0%7.&(!%05"B(%0'(7%*()"("%1.$*( .0+"5!%+"'(.0+/('.--"!"0+.%$(/!(73$+.#$"(1"01/!( 5!%'./7"+!.&(&/0-.53!%+./01(2,.&,(%!"(-"%1.)$"( .0(%(73$+.C$.7)(2%$D.05(!/)/+8(E#8(FGH

" ;/7"(&/7).0"'('"6.&"B(+,%+(.0&$3'"1(%$$(+,"( )"1+(-"%+3!"1(/-(+,"(1%.'(7%50"+.&(-."$'( 1"01/!1I+!%01'3&"!1(1""71(+/()"(+,"(#!"-"!"'( +!"0'(-/!(-3!+,"!('"6"$/#7"0+(%1(%(#"!&"#+./0( /!5%08(J%50"+.&(.0'3&+./0(K(/-(LM(J>(2.+,(+,"( -!"43"0&*(N(/-($.7)1O(/1&.$$%+./01(#!/'3&"(%0( "878-8(E#8(PQH

# ?,"(1.50%$C2%6"("06"$/#"(.1(%(!"13$+(/-(+,"(

!"#$%&!'("&&')*'%'+*,*+*$-*'./"$0'%$1'#2"1"$#' &"$*',/+'%'(%&3"$#'+/)/0'45"#6'786'9.6'7:;< =$>'!?%&&'@*&1'-A%$#*!'0A*'1"+*-0"/$'/,'0A*'B5' C*-0/+'"$'0A*'!.%-*'%$1'0A"!'.+/12-*!'%' 1"!0/+0"/$'"$'0A*'(%C*,/+?'/,'0A*'!"#$%&'"$'0A* 1*0*-0"/$'-/"&!6'DA*'%1C%$-*'/,'?2-A'-A*%.*+' +//?E0*?.*+%02+*'!*$!/+'0*-A$/&/#"*!'/,,*+!'

0A*'.+/!.*-0'/,'?2-A'#+*%0*+'2!*'/,'B5' ?/$"0/+"$#'9;<6'9.6';<

B%#$*0/!*$!/+"-!'"!'%'C*+!%0"&*'0//&'0/'%!!*!!'?*-A%$"-%&' ?/C*?*$0!'666'%&&'/,'(A"-A'%+*'?*0')>'/2+'2&0+%0A"$' ?%#$*0"-'!*$!/+!F'?%3"$#'0A*?'"1*%&&>'!2"0*1',/+' (*%+%)&*F'>*0'2$/)0+2!"C*'%$1'"?.*+-*.0")&*'/+"*$0%0"/$' %$1'?%$".2&%0"/$'%"1!6'9.6';<

666'+/)/0:?/2$0*1'"$!0+2?*$0!'"$'.%+0"-2&%+'?2!0' )*'&"#A0(*"#A0'0/'!2+C"C*'1*.&/>?*$0'2$1*+' A"#A:!0+*!!'-/$1"0"/$!6'DA"!'(*"#A0'-/$!0+%"$0' -%$'&"?"0')/0A'0A*'!"G*'/,'0A*'!*$!/+'%$1'"0!' .&%-*?*$0'/$'0A*'?/C"$#'.%+06'=&!/F' %20/$/?/2!'!>!0*?!'+*H2"+*'0A*'?"$"?%&' -/$!2?.0"/$'./(*+6'9.6';<

I*+*'(*'#/')*>/$1'"?"0%0"$#'0A*',*%02+*!'/,'A2?%$' .A>!"/&/#>'%$1'"$0+/12-*'*:!3"$!'("0A'%'?%#$*0/:!*$!/+>' !>!0*?F'(A"-A'*H2".!'0A*'+*-"."*$0'("0A'%'J!"K0A'!*$!*L'%)&*' 0/'.*+-*"C*'0A*'.+*!*$-*'/,'!0%0"-'/+'1>$%?"-'?%#$*0"-' @*&1!6'9.6';<

DA"!'"$12-0"/$'!*$!/+'"!'%&!/'-/?.%-0F'+/)2!0F'

N*'1*?/$!0+%0*'%$'/$:!3"$'.+/K"?"0>'1*0*-0"/$'!>!0*?!' ,/+'0/2-A&*!!'A2?%$O?%-A"$*'"$0*+%-0"/$F'?/0"/$'%$1' 1"!.&%-*?*$0'!*$!/+"-!'%..&"-%)&*',/+'!/,0'+/)/0!'/+' ,2$-0"/$%&'?*1"-%&'"?.&%$0!F'%!'(*&&'%!'?%#$*0"-' ,2$-0"/$%&"0"*!',/+'*."1*+?%&'*&*-0+/$"-!6'P$'0A"!'(/+3F'(*' -/$!0+2-0'A"#A&>'!*$!"0"C*'#"%$0'?%#$*0/+*!"!0"C*'4QBR8' !*$!/+'*&*?*$0!6'9.6';<

/.*+%0*!'%0'+//?'0*?.*+%02+*F'*KA")"0!'%'("1*' .0-/!7%+./0%$C7"%13!.05(#!/&"11(%0'( &/0-.!7.05(%)/3+(+,"(1#%&"('.1+!.)3+./0(/-(+,"( 1>$%?"-'+%$#*F'%$1'?%>')*'*%!"&>'"$0*#+%0*1' J>(6%!.%+./01B(2,.&,(%!"()%1"1(-/!(+,"(%'6%0&"'( "$0/'1",,*+*$0"%&'/+'?2&0".&*'!*$!/+'#+%1"/?*0+"-' -/$@#2+%0"/$!'(A"-A'%+*',*%!")&*'"$'%'?2&0": #"!&"#+./0C53.'"(%).$.+."18 &"?)'(%&3"$#'+/)/06'9.6'M< 98R8(J>(>""$.05(2.+,(%(S.6.05(K".05CJ%&,.0"( :0+"!-%&"(E#8(PTH >.53!"(UF8(?,"($.6.05()".05C7%&,.0"(.0+"!-%&"(.0( +,"()./!/)/+.&(#"!&"#+./0(1*1+"78(E#8(P9H

$ L-+"!("06"$/#"('"+"&+./0(/-(+,"(43%0+.+*(V/3+B( =,0*+'*$C*&/.*'1*0*-0"/$'/,'0A*'H2%$0"0>'S/20F' &%0()"(#!"1"0+"'()*(&,%05.05(+,"( &/!!"1#/0'.05(43%1.CWM(J>()*(+,"(!/)/+X1( 7/6"7"0+(%1('"-.0"'(.0(>.53!"(UT8(:0(+,"( 1.7.$%!(7%00"!B(2%*(%0(LM(J>(#%!+.%$(&%0()"( 1,/208(E#8(PQCPUH

-%$')*'.+*!*$0*1')>'-A%$#"$#'0A*' -/++*!./$1"$#'H2%!":TU'B5')>'0A*'+/)/0L!' ?/C*?*$0'%!'1*@$*1'"$'5"#6'M6'P$'0A*'!"?"&%+' ?%$$*+F'(%>'%$'=U'B5'.%+0"%&'-%$')*'!A/($6' 9.6'V<

DA*'+*!"!0%$-*'.&/00*1'"$'5"#6'7"F'$/"!*:"$12-*1'W2-02%0"/$' 12+"$#'0A*'?/0"/$'/,'0A*'A%$1F'(A*+*%!'0A*'@*&1'/,'0A*' %..+/%-A"$#'.*+?%$*$0'?%#$*0'"$12-*!'%'!0+/$#' +*!"!0%$-*'1+/.6'=&0*+"$#'0A*'1"!0%$-*')*0(**$'0A*' .*+?%$*$0'?%#$*0'%$1'0A*'/$:!3"$'!*$!/+!'+*!2&0!'"$'%' -/++*!./$1"$#'-A%$#*'"$'?%#$*0/+*!"!0%$-*6'9.6';<

!"#$%&'#"%($)"*+,-"./0)$1$23"45/"(6%$))04(.78 E5/"/2%(;$23"%02">/"./0)$1/;">(45"6404$%0))7"=$0" 9(.:0.;"<(=/</24"0)(23">(45"!,"$2;'64.$0)" #/.<02/24"<032/46"06":/))"06";720<$%0))7"6$<#)7":$45" $24/.9/./2%/6"02;"?'06$8@,"204'.0)"*A0.45-" %'../248;.$=/2":$.$236D"H#D"IJ $24/.9/./2%/6"02;"?'06$8@,"204'.0)"*A0.45-" /2=$.(2</240)"BC6D"E5/6/"9$/);6"0./";$64.$>'4/;" /2=$.(2</240)"BC6D"E5/6/"F/);6"0./";$64.$>'4/;" (2"0"6'.90%/"02;"$2"6#0%/".('35)7"0%%(.;$23"4(" (2"0"6'.90%/"02;"$2"6#0%/".('35)7"0%%(.;$23"4(" C$3D"GD"H#D"IJ C$3D"FD"G#D"HIJ

! !"#$%&'#"%($)"*+,-"./0)$1$23"45/"(6%$))04(.78 " 9(.:0.;"<(=/</24"0)(23">(45"!,"$2;'64.$0)"

K(45"90%4(.6"4.026)04/"$24("6/=/./"%(264.0$246"(2" 45/"%0#0>$)$47"(9"45/"$264.'</24"4("</06'./" :/0&"ABC6"/2%('24/./;"$2"45/"/2=$.(2</24D"L2" GIJD""B(./(=/.K"0"2'<>/."(9"45/"./)/=024" </06'.$23";/=$%/6K"06"02"$24/.</;$04/"/)/</24" 0;;$4$(2M"45/"('4/."6#0%/"02;"'2;/.:04/." >/4://2"#576$%0)"=0)'/"02;"45/"2/.=/"6764/<K" %(2;$4$(26"./?'$./"/2;'.02%/"030$264"=0.$04$(26" $2"45/":$;/".023/"(9"4/<#/.04'./M"5'<$;$47M"02;" 0./";/6$32/;"(2"$46">06$6"GMNJD"G#D"HOJ 04<(6#5/.$%"#./66'./D"H#D"NJ

! !6"0"./6')4K":/"50=/"0%5$/=/;"L'CAEE."4504"$6" ! 6'$40>)/"9(."06%/.40$2$23"45/"=0.$/47"(9"=0)'/6"

L2"%(2%)'6$(2M":/";/=/)(#"$<#/.%/#4$>)/"02;"5$35)7" 6/26$4$=/"<032/4$%"F/);"6/26(.6":$45"'2$?'/"</%502$%0)" #.(#/.4$/6"02;";/<(264.04/;"45/"./<0.&0>)/"#(4/24$0)"(9" 45$6"4/%52()(37"9(."6<0.4"6&$26M"%(26'</."3((;6"02;" </;$%0)"$<#)0246D"H#D"OJ

! E5/">06$%"6%/20.$("$6"65(:2"$2"C$3'./"MHD"!6"0" E5/"044/<#4"4("6()=/"45$6"#.(>)/<"(9"6$<#)$97$23" E5/"'4$)$104$(2"(9"6#$2"=0)=/")07/."640%&6"$264/0;"(9"SBT" # ./6')4")$=$23">/$236"%(24.()";.$=/6">7"#./=$('6)7" 45/".(>(4$%"=$6$(2"64.'%4'./"02;"./;'%$23"(9"6'%5" <')4$)07/.6"%02"./6')4"$2"/=/2"5$35/."6/26$4$=$4$/6"4("6<0))" 4.026)04/;">$(6$320)6D"P2"45/"(45/."=0.$024K" >$(6$320)6"9.(<"(.3026"(9"45/"6/26/6"(.">.0$2" 4.026;'%/";$./%4)7"$24("$24/))$3/24"(.".(>(4$%" 6764/<6":5$%5K"$2"6'%5"0":07K"#$%&"'#" /2=$.(2</240)"$29(.<04$(2D"G#D"OQJ E0&$23"$24("0%%('24"A?'04$(2D"*H-"$6"#(66$>)/"4(" %0)%')04/"45/"./)04$(2">/4://2"6/26(."('48$2" *P2/.=/-"02;"0%4'04(."$28('4"*R('4-" ./6#/%4$=/)7":5/2".(>(4S6")$<>6"0./"(6%$))04$23" :$45"45/"9./?'/2%7"TD"G#D"OFJ

#/.9(.<02%/";040"06"<066M"=()'</"02;"#(:/." <032/4$%"F/);6D"C'4'./":(.&":$))"9(%'6"(2"(#4$<$104$(26"4(" %(26'<#4$(2">7"45/"/<#)(7$23"(9"45/"$2;'%4$(2" $24/.90%/"/)/%4.$%0))7"02;"</%502$%0))7":$45"(45/." *0)6("6'#/.%(2;'%4$23-"6/26(.":06";(2/D"E5$6" /)/%4.(2$%"%(<#(2/246"/20>)$23M"9(."/U0<#)/M":$./)/66" </45(;"0))(:6"'6"4("'6/"45/"(6%$))04$(26"(9")$<>6" ./0;('4"02;"./<(4/"6/26$23D"E5/"$24/3.04$(2"(9" ;'.$23".(>(4P6":0)&$23"9(."./%/$=$23"45/" <032/4(/)/%4.(2$%6":$45"')4.045$2"9'2%4$(20)"/)/</246" </06'./;"6$320)"02;"<0&/"45/"=$6$(2"<(./" 6'%5"06"6()0."%/))6M")$3548/<$44$23";$(;/6M"4.026$64(.6M"06" 204'.0)"9(."45$6"<0%5$2/D"E5/"0;=02%/;"6/26(." :/))"06"4/<#/.04'./"02;"40%4$)/"6/26(."0..076M":$))"/20>)/" 506"02"'2)$<$4/;"9./?'/2%7".023/"9.(<"@,"4(" 0'4(2(<('6"02;"=/.604$)/"6<0.4"6764/<6":$45"0"<')4$4';/" QCM"02;"6/26$4$=$47"4("45/"0<>$/24"BC"$6"%)(6/." (9"6/26$23"02;"0%4'04$(2"9/04'./6D"H#D"OJ 4("45/"45/(./4$%0)"#(66$>$)$47D"H#D"O8RJ

V(</"%(<>$2/;";/=$%/M"4504"$2%)';/6"0))"45/" [/"9(./6//"('.":(.&"4("$26#$./"0";$=/.6/"2'<>/."(9" >/64"9/04'./6"(9"45/"60$;"BC"6/26(.6W4.026;'%/.6" ;/=$%/6"4504":$))">/2/F4"9.(<"0"\6$U45"6/26/]" 6//<6"4(">/"45/"#./9/.0>)/"4./2;"9(."9'.45/." <032/4(%/#4$(2D"H#D"OJ #(66$>)/"4("%./04/"0"2/:"3/2/.04$(2"(9"$24/)8 ;/=/)(#</24"06"0"=$6$(2"(.302D"X#D"YZ )$3/24"6/26$23D"E5/./"0./";/=/)(#$23"(##(.4'2$8

! U/%/24";/=/)(#</246"$2">$(/23$2//.$23K" $ 202(4/%52()(37K"02;"6(94"%(<#'4$23"<0&/"$4"

4$/6"9(."%(<>$2$23"204'.0)"02;"0.4$9$%$0)"6/26$23" 0>$)$4$/6"$2"45/"67245/6$1/;"6764/<D"G#D"OFJ