Mobility Change-of-State Detection Using a Smartphone-based Approach G. Hache1, E.D. Lemaire1,2, N. Baddour1 1
Department of Mechanical Engineering University of Ottawa Ottawa, Canada 2 Institute for Rehabilitation Research and Development The Ottawa Hospital Rehabilitation Centre Ottawa, Canada
Abstract— Understanding the mobility of people with physical disabilities is important for rehabilitation decision making. This paper presents a smartphone-based approach to mobility monitoring. The BlackBerry-based system is clipped to the person’s belt. This approach uses an accelerometer signal to identify changes-of-state caused by starting/stopping and postural change. Our finding suggests that a smartphone integrated with an accelerometer could detect changes from static to dynamic movement (i.e., start to walking, standing still, slowing down), which compares favorably with previous studies using body-fixed accelerometers. This approach is part of the larger framework of Wearable Mobility Monitoring Systems (WMMS). Smartphone; accelerometer; mobility; wearable
I.
INTRODUCTION
Independent ambulation within the home and the community is an important rehabilitation goal for a person with physical impairments [1]. Monitoring a person’s mobility in these environments is thus important. A wearable mobility monitoring system is a promising tool for helping rehabilitation specialists determine mobility issues outside a hospital environment, evaluating the progress made during and after rehabilitation, and enhancing clinical decision-making about a rehabilitation program (i.e., assistive devices, exercises, treatment, etc.).
Mobile devices with integrated accelerometers are a great advantage, since accelerometers are one of the most commonly used sensors in mobility monitoring [6]. Many studies have explored and validated the use of accelerometers in movement and mobility analysis [7], [8], [9], [10], [11], [12]. However, in these studies the accelerometers were usually fixed to specific body locations. These approaches would not be applicable for mobile phones since mobile phones are usually worn and orientated in different ways. Recent studies have explored the use of mobile phones integrated with an accelerometer to create an orientationindependent system for activity recognition [13], [14]. Both classification algorithms required extensive training data to account for multiple possible orientations. In this paper, we explored a smartphone-based approach with the device worn on the pelvis, since this is a common location for wearing a phone (Fig. 1). We applied signal processing and analysis to extract accelerometer signal features and to detect a user’s change-of-state. This paper presents methods for detecting changes-of-state caused by the start/stop of walking and postural change. This paper also presents preliminary results collected from five able-bodied subjects. This approach was part of the development of a Wearable Mobility Monitoring System (WMMS).
An increasingly pertinent trend in wearable systems is to use mobile phones or smartphones as the system’s platform [2], [3], [4], [5]. With the constant increase in processing power, allowing for sophisticated real-time data processing, mobile phone and smartphones are a great choice as a central node for wearable mobility monitoring system. Having multiple sensors integrated in mobile phones can allow monitoring to happen at only one location on the body [2], which makes it easier to use and less obtrusive to the user. Additionally, mobile phones or smartphones take advantage of the user’s acquaintance with the mobile device [3]. Moreover, these mobile devices are often already integrated with sensors, such as accelerometers, cameras, and global positioning systems (GPS). This work was supported by Research In Motion , the Ontario Graduate Scholarships in Science and Technology program, and the Ontario Centers of Excellence.
Figure 1. Subject wearing the smartphone-based wearable system. Accelerometer’s axes are identified.
II.
METHODOLOGY
A triaxial accelerometer LIS344alh (STMicroelectronics, Geneva, Switzerland) was used with the BlackBerry Bold 9000 as the platform. The accelerometer was part of a sensor board that was attached to the BlackBerry holster. The sampling frequency was 50 Hz. Raw accelerometer data was first passed through a median filter (n=3) to remove spikes [10]. Then, a 0.25 Hz digital filter was used to separate the static and the dynamic components of the acceleration signal. A nonoverlapping window of 1.02 seconds was used to extract features from these two components. Standard deviation, a well supported measure for activity classification, was extracted from the acceleration signal [7], [8], [12], [15], [16]. Since most daily activities can be classified by changes in vertical axis acceleration, vertical acceleration (y-axis) was used to differentiate between static and dynamic states (Fig. 2). To differentiate between these two states, standard deviation of the y-axis acceleration was compared with two thresholds (static and dynamic). This was defined as the double threshold (DT) algorithm (Fig. 3). With a DT algorithm, if the initial state is static, the activity classification remains static until the signal crosses the dynamic threshold (0.120 g). Then, the state is set to dynamic and stays dynamic until the signal passes below the static threshold (0.075 g). Both thresholds were set based on preliminary observations. The next feature is the inclination angle, which has been used to help classify posture [7], [11], [12], [16] and identify postural transition [9]. The inclination angle was calculated for every window period using the two-axes method presented by Freescale Semiconductor [17]. The y-axis (vertical) and z-axis (forward) were used (See Fig. 1 for axes orientation). The averaged inclination angle was compared with a high and low threshold (200, 160 degrees) to verify if the person was in a standing position. If the person was not standing, the angle was compared with different high and low thresholds (320, 250 degrees) to verify if the person was lying on their back. If both conditions were false, the position was determined to be somewhere in between. Threshold values were based on the study by Culhane et al. [12] and our preliminary observations.
Figure 2. Standard deviation of y-axis acceleration during level ground walking (dynamic), followed by a short period of standing (static), and then back to walking.
Figure 3. Flowchart of the double threshold (DT) algorithm applied to the standard deviation of the y-axis acceleration.
Fig. 4 shows an example of the inclination angle signal during walking and lying. Another feature is the signal magnitude area (SMA) which has been found to be a viable activity and mobility measure [10], [11]. SMA normalized to the length of the signal (T) can be calculated using the following equation:
SMA =
T T 1⎛ T ⎜ ∫t =0 | a x |dt + ∫t =0 | a y |dt + ∫t =0 | a z |dt ⎞⎟ (1) ⎠ T⎝
where t is the time in seconds and ax , ay , and az are the acceleration of x-, y-, and z-axes respectively. During preliminary testing, peaks occurred in the SMA signal for sitting, rising from a chair, and lying down (Fig. 5). SMA also helped identify activity intensity changes that could indicate a change-of-state. Therefore, three thresholds were used and three states were determined: no peak with normal intensity, no peak with increased intensity, or a peak. A DT algorithm was used to determine increases in intensity and peak detection (Fig. 6). The “increased intensity” low and high thresholds
Figure 4. Inclination angle signal during level ground walking, followed by a short period of lying, and then back to walking.
were filmed to validate change-of-state detection and to determine the change-of-state timing. IV.
RESULTS
Changes-of-state produced by walking start/stop were well detected with an average sensitivity of 97.4 ± 5.3%. An average sensitivity of 97.8 ± 4.7% was found for changes-ofstate caused by postural changes. The average sensitivity for each type of change-of-state evaluated in this paper were also calculated and are given in Table I. TABLE I.
Figure 5. SMA of a person walking then sitting, standing up, walking, lying down on a bed, getting up from the bed, lying on the floor, and getting up again.
were 0.100 g and 0.190 g, respectively. Since an increase in intensity should happen when the person is moving, the algorithm verified that the person was in a dynamic state when detecting the “no peak with increase in intensity” state. The peak detection low and high thresholds were 0.100 g and 0.320 g, respectively. III.
EVALUATION
Mobility data were collected from a convenience sample of five able-bodied subjects (3 males, 2 females; age: 36.6 ± 6.4 years; height: 173.82 ± 13.17 cm; weight: 69.32 ± 16.09 kg). The subjects were asked to wear the system on their right hip with the device pointing forward. No additional instructions were given for positioning the instrumented holster. The subjects were then asked to follow a pre-defined sequence of tasks, but no instructions were given on how to perform the mobility tasks. Three trials per subject were done. For each trial, there were thirteen changes-of-state caused by the start/stop of walking and standing and nine caused by postural changes due to sitting and lying. During the tasks, the subject s
Figure 6. Flowchart of the SMA algorithm.
PERFORMANCE RESULTS FOR EACH CHANGE-OF-STATE CATEGORY.
Change-of-state
True positive
False Negative
Averaged Sensitivity ± Standard Deviation
From standing position start walking
116
3
97.4% ± 5.3%
From walking to stop and stand
71
2
97.3 % ± 6.0%
From standing to sitting
59
1
98.3 % ± 3.3%
From sitting to standing
28
2
93.3 % ± 9.4%
From standing to lying
30
0
100.0 % ± 0.0%
From lying to standing
15
0
100.0 % ± 0.0%
V.
DISCUSSION
Change-of-state produced by the start/stop of walking replicated results from previous studies. For instance, Lyons et al. [16] obtained an accuracy of 97% to detect static or dynamic states using the standard deviation of the vertical axis of a thigh accelerometer. Our approach used the standard deviation of the vertical acceleration at the waist and was able to detect if the subject started/stopped moving with a sensitivity of 97.4 ± 5.3%. This is a good result, considering that the device holster was worn on a belt and not fixed-still to the person’s body. This finding suggests that a phone integrated with an accelerometer could detect changes from static to dynamic movement (i.e., start to walking, standing still, slowing down). Changes-of-state due to postural change (i.e., stand-to-sit, sitting, lie-to-stand, etc.) were detected with a sensitivity of 97.8 ± 4.7%. These results compared favourably with previous studies, such as Karantonis et al. [11] where 94.2% accuracy was found for detecting tasks related to postural orientation. Using threshold methods, Culhane et al. [12] detected sitting at 92%, standing at 95%, and lying at 98%. However, their results were obtained from two accelerometers (one on the trunk and one on the thigh). Although our algorithm detected changes-ofstate due to postural change, our approach was not evaluated for its accuracy to classify the postures. From our observations,
our methods might not be precise enough to classify all postures. The way the smartphone-based wearable system was worn on the hip may have caused false positives during sitting ad lying due to the device holster’s free movement, the leg pushing on the device, the person’s belt location, and sitting angle. However, our evaluation protocol provided a real-time situation where the mobility tasks were performed consecutively and freely, instead of performing discrete mobility tasks in a controlled laboratory setting [11]. Furthermore, to better validate our smartphone-based approach, only one accelerometer was used and our protocol did not control the fixation and location of the wearable device. Wearing the wearable device on the right hip, attached to the belt was the only requirement given to the subject.
[8]
[9]
[10]
[11]
[12]
VI.
CONCLUSION
In this paper, methods to detect mobility change-of-state using a smartphone-based approach were presented. Results show that static/dynamic and postural changes could be detected with as high accuracy as previously reported. Detection of change-of-state caused by the start/stop of moving and postural change is part of the larger development of a Wearable Mobility Monitoring system (WMMS) that could monitor a person’s mobility state and take a photograph when the person’s change-of-state is detected.
[13]
[14]
ACKNOWLEDGMENT
[15]
The authors also would like to thank Keith Heggie for his technical work on the external sensor board and Cynthia Kendell for her assistance with data collection and analysis. The Ottawa Hospital Rehabilitation Centre is acknowledged for their facility support.
[16]
REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
S. E. Lord, K. McPherson, H. K. McNaughton, L. Rochester and M. Weatherall, "Community ambulation after stroke: How important and obtainable is it and what measures appear predictive?" Archives of Physical Medicine and Rehabilitation, vol. 85, pp. 234-239, 2004. J. Lester, T. Choudhury and G. Borriello, "A practical approach to recognizing physical activities," in 4th International Conference on Pervasive Computing, Dublin, 7 May 2006 through 10 May 2006, 2006, pp. 1-16. M. J. Moron, J. R. Luque, A. A. Botella, E. J. Cuberos, E. Casilari and A. Diaz-Estrella, "A smart phone-based personal area network for remote monitoring of biosignals," in Proceedings for the International Federation for Medical and Biological Engineering, 2007, pp. 116. P. Van De Ven, J. Nelson, A. Bourke and G. O. Laighin, "A wearable wireless platform for fall and mobility monitoring," in 1st International Conference on Pervasive Technologies Related to Assistive Environments, 2008. P. Roncagliolo, L. Arredondo and A. González, "Biomedical signal acquisition, processing and transmission using smartphone," Journal of Physics: Conference Series, vol. 90, 2007. W. Y. Wong, M. S. Wong and K. H. Lo, "Clinical applications of sensors for human posture and movement analysis: A review," Prosthetics and Orthotics International, vol. 31, pp. 62-75, 2007. P. H. Veltink, H. B. J. Bussmann, W. De Vries, W. L. J. Martens and R. C. Van Lummel, "Detection of static and dynamic activities using uniaxial accelerometers," IEEE Transactions on Rehabilitation Engineering, vol. 4, pp. 375-385, 1996.
[17]
C. V. C. Bouten, K. T. M. Koekkoek, M. Verduin, R. Kodde and J. D. Janssen, "A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity," IEEE Transactions on Biomedical Engineering, vol. 44, pp. 136-147, 1997. B. Najafi, K. Aminian, A. Paraschiv-Ionescu, F. Loew, C. J. Bu la and P. Robert, "Ambulatory system for human motion analysis using a kinematic sensor: Monitoring of daily physical activity in the elderly," IEEE Transactions on Biomedical Engineering, vol. 50, pp. 711-723, 2003. M. J. Mathie, A. C. F. Coster, N. H. Lovell and B. G. Celler, "Detection of daily physical activities using a triaxial accelerometer," Medical and Biological Engineering and Computing, vol. 41, pp. 296-301, 2003. D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell and B. G. Celler, "Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring," IEEE Transactions on Information Technology in Biomedicine, vol. 10, pp. 156-167, 2006. K. M. Culhane, G. M. Lyons, D. Hilton, P. A. Grace and D. Lyons, "Long-term mobility monitoring of older adults using accelerometers in a clinical environment," Clinical Rehabilitation, vol. 18, pp. 335-343, 2004. T. Brezmes, J. L. Gorricho and J. Cotrina, "Activity Recognition from Accelerometer Data on a Mobile Phone," in Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part II: Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living, 2009, pp. 799. J. Yang, "Toward physical activity diary: Motion recognition using simple acceleration features with mobile phones," in 1st ACM International Workshop on Interactive Multimedia for Consumer Electronics - IMCE'09, Co-Located with the 2009 ACM International Conference on Multimedia, MM'09, Beijing, 19 October 2009 through 24 October 2009, 2009, pp. 1-9. J. Baek, G. Lee, W. Park and B. -. Yun, "Accelerometer signal processing for user activity detection," in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2004, pp. 573-580. G. M. Lyons, K. M. Culhane, D. Hilton, P. A. Grace and D. Lyons, "A description of an accelerometer-based mobility monitoring technique," Medical Engineering and Physics, vol. 27, pp. 497-504, 2005. Freescale Semiconductor, "Application Note 3461, Rev 2: Tilt Sensing using Accelerometers Sensors," pp. 2-4,