SPIE Membership Get updates from SPIE Newsroom
  • Newsroom Home
  • Astronomy
  • Biomedical Optics & Medical Imaging
  • Defense & Security
  • Electronic Imaging & Signal Processing
  • Illumination & Displays
  • Lasers & Sources
  • Micro/Nano Lithography
  • Nanotechnology
  • Optical Design & Engineering
  • Optoelectronics & Communications
  • Remote Sensing
  • Sensing & Measurement
  • Solar & Alternative Energy
  • Sign up for Newsroom E-Alerts
  • Information for:
SPIE Photonics West 2019 | Call for Papers

2018 SPIE Optics + Photonics | Register Today



Print PageEmail PageView PDF

Electronic Imaging & Signal Processing

Toward a domestic system to assist people with Parkinson's

A new home tracking system based on a Microsoft Kinect sensor will help patients with the symptom of ‘freezing of gait.’
5 June 2013, SPIE Newsroom. DOI: 10.1117/2.1201305.004884

Freezing of gait (FoG) is a disabling symptom commonly occurring in the latter stages of Parkinson's disease. It is characterized by brief episodes of inability to step, or by extremely short steps that typically occur on gait initiation or on turning while walking.1 FoG heightens the risk of falls, which leads to loss of independence.2 Aggravated mobility and difficulties in activities of daily living have a direct effect on the quality of life of patients,3 demanding attention from a carer or causing affected people to move into specialized institutions. In the absence of any completely effective pharmacological treatment for FoG, technology-based solutions to alleviate the symptom and prolong people's ability to live independently are eagerly being sought.

Such technological solutions exploit ‘sensory cueing’. External stimuli, such as lines on the floor or rhythmic sounds, can focus the attention of a person experiencing an FoG episode and help him or her initiate gait.4 Researchers are currently developing wearable gait assistance systems, which comprise wearable devices with inertial sensors to detect FoG episodes and activate auditory rhythmic cueing over earphones.5, 6 However, it is difficult to detect FoG in a timely way under real-world conditions; the best detection result reported in laboratory trials to date is 83% sensitivity.7 Even higher detection accuracy is necessary for everyday domestic use. We have been developing a distributed home system based on Microsoft Kinect sensors that can use contextual information from the environment as an additional cue.

FoG depends on the person's walking situation. It often occurs on walking transitions: turns, setting off, arrival, and in open spaces.8 It can also occur when people approach narrow spaces, such as doors, and in crowded places. Recognizing such situations is a very powerful cue for correct detection. Three pieces of information are essential: the person's location, their surroundings, and a contextual model of the environmental influence on FoG. We decided to model the environmental influence on FoG, taking into account direct geometric relations,9 and we chose the Kinect camera to capture 3D data as the system sensor. We have been developing a system that uses this type of sensor to track multiple people inside a home.10

We use Kinect cameras only on the most problematic locations for FoG in people's homes; it is likely that camera coverage will be incomplete. The optimum mounting position for sensors is 2.2–2.4m above floor level, angled downward. We have developed a Kinect application that can track more than one person up to 6m from the camera. The application uses multiple particle filter trackers, one for each person in the scene.11 Our implemented algorithm can still give accurate results with a low, 320×240 image resolution, making it suitable for use with cheap, low-end computing nodes. Laboratory experiments have shown a consistent position estimation error <0.2m.

Formulating an appropriate context model for FoG is crucial for the success of our proposed approach. Behavioral data on affected people is essential for this. In the project's final stage, we will collect the necessary behavioral data through one-day recording sessions in people's homes. This requires our system to be portable and fast to set up on site. During setup, we enter details of the scene the camera will be observing, such as ground level and interest zones (see Figure 1).

Figure 1. Simulation scenario for tracking two individuals. One person, affected by freezing of gait (FoG), is standing and the other person, a carer, is sitting. The camera scene is described by its tracking boundaries (green) and interest zones (blue). pid: Person identification.

Our model is run independently for each camera in a distributed system, both for scene setup and people tracking. Data on the appearance of people being tracked is maintained in one central processing node and shared to every camera on request. For re-identification between cameras, we use an appearance model based on the person's color histogram and height along with the known average movement times between cameras. We are currently testing this re-identification approach in a laboratory setup and we hope to report the results soon. The system under operation is shown in a short video that is available online.12

Our indoor position tracking system is just the first step toward the final goal of FoG detection based on context. For daily use it is important that the system can automatically recognize the patient. Future work will focus on identifying the patient based on combined data from the camera and the wearable sensor. In addition, we will use the system to record behavioral data for context modeling in patients' homes.

This work was supported in part by the doctoral program fellowship offered by the Erasmus Mundus Joint Doctorate in Interactive and Cognitive Environments.

Boris Takač
Technical Research Centre for Dependency Care and Autonomous Living (CETpD)
Universitat Politècnica de Catalunya — BarcelonaTech (UPC)
Vilanova i la Geltru, Spain

Boris Takač is a doctoral candidate supported by the Erasmus Mundus Joint Doctorate in Interactive and Cognitive Environments program. His research interests include robotics, computer vision, data fusion, and design of intelligent systems.

Wei Chen, Matthias Rauterberg
Department of Industrial Design
Eindhoven University of Technology
Eindhoven, The Netherlands

Wei Chen is an assistant professor. Her research interests include sensor systems for ambient intelligent design, healthcare system design using wearable sensors, wireless body area networks, and smart environments.

Matthias Rauterberg is full professor for Interactive Systems Design. He has more than 400 publications in the areas of human computer interaction, entertainment and cultural computing, as well as interaction design.

1. J. G. Nutt, B. R. Bloem, N. Giladi, M. Hallett, F. B. Horak, A. Nieuwboer, Freezing of gait: moving forward on a mysterious clinical phenomenon, Lancet Neurol. 10(8), p. 734-744, 2011. doi:10.1016/S1474-4422(11)70143-0
2. B. R. Bloem, J. M. Hausdorff, J. E. Visser, N. Giladi, Falls and freezing of gait in Parkinson's disease: a review of two interconnected, episodic phenomena, Mov. Disorders 19(8), p. 871-884, 2004. doi:10.1002/mds.20115
3. O. Moore, C. Peretz, N. Giladi, Freezing of gait affects quality of life of peoples with Parkinson's disease beyond its relationships with mobility and gait, Mov. Disorders 22(15), p. 2192-2195, 2007. doi:10.1002/mds.21659
4. A. Nieuwboer, Cueing for freezing of gait in patients with Parkinson's disease: a rehabilitation perspective, Mov. Disorders 23 Suppl. 2, p. S475-481, 2008. doi:10.1002/mds.21978
5. Web page of EU REMPARK project. http://www.rempark.eu/
6. Web page of EU CUPID project. http://www.cupid-project.eu/
7. M. Bächlin, M. Plotnik, D. Roggen, I. Maidan, J. M. Hausdorff, N. Giladi, G. Tröster, Wearable assistant for Parkinson's disease patients with the freezing of gait symptom, IEEE Trans. Inf. Technol. Biomed. 14(2), p. 436-446, 2010. doi:10.1109/TITB.2009.2036165
8. J. D. Schaafsma, Y. Balash, T. Gurevich, A. L. Bartels, J. M. Hausdorff, N. Giladi, Characterization of freezing of gait subtypes and the response of each to levodopa in Parkinson's disease, Eur. J. Neurol. 10(4), p. 391-398, 2003. doi:10.1046/j.1468-1331.2003.00611.x
9. Q. J. Almeida, C. A. Lebold, Freezing of gait in Parkinson's disease: a perceptual cause for a motor impairment?, J. Neurol. Neurosurg. Psychiat. 81(5), p. 513-518, 2010. doi:10.1136/jnnp.2008.160580
10. B. Takač, A. Català, D. Rodríguez, W. Chen, M. Rauterberg, Ambient sensor system for freezing of gait detection by spatial context analysis, Ambient Assisted Living and Home Care 7657, p. 232-239, Springer, 2012.
11. R. Muñoz-Salinas, A Bayesian plan-view map based approach for multiple-person detection and tracking, Pattern Recog. 41(12), p. 3665-3676, 2008. doi:10.1016/j.patcog.2008.06.013
12. Video of real-time tracking of three people between two non-overlapping Kinect cameras. http://spie.org/documents/newsroom/videos/4884/spie.mov  Credit: Boris Takač, Eindhoven University of Technology.