Connecting the Globe through Motion Sensing Technology


Tomohiro Shibata
Associate Professor|Associate Professor
Graduate School of Information Science
Nara Institute of Science and Technology

Mr. Tomohiro Shibata earned a Ph.D. degree in 1996 from School of Engineering, The University of Tokyo. He has worked as a Postdoctral researcher with Japan Society for the Promotion of Science and as a senior researcher with Japan Science and Technology Agency. In 2002, he was an assistant professor at Graduate School of Information Science, Nara Institute of Science and Technology and now is an associate professor. The focus of his research is on integrative fields, which include: 1) Humanoid, Learning, Assistive robotics; 2) Cognitive, Computational, and Systems neuroscience; 3) Brain-computer interface; and 4) Modeling social activities.

Assisting with Mother

I started Agora Music Club, a nonprofit organization, in 2011 with Ms. Eriko Mizuno who has earned a Doctor of Philosophy degree and been involved in music therapy for Down syndrome over fifteen years in Tomio, Nara City, Nara Prefecture. At the organization, through use of Kinect*1, we conduct an analysis on the physically challenged person’s capability of playing music instruments based on scientific data and are striving to establish an evidenced-based music therapy.

Private sectors will not be interested in one such activity that has been conducted for physically challenged persons, because it is unlikely to bring to them a lucrative market segment. I don’t think that it is a good idea that Kinect, an affordable but high technology device, is used only for physically unimpaired ones who play games. My first idea was to start a nonprofit organization with some colleagues on a voluntary basis. Now I am tackling financial issues and problems by conducting feasibility study on a variety of business models to bring to us a profit. The fact of a matter was that I began to be involved in this field because of my mother. She has been a Parkinson disease patient for over fifteen years and I have gone through assisting with her for some of the years. Therefore, I realize the importance of elderly care and could have had the opportunities to join a variety of activities in the field of medical homecare and non-medical care.

*1: Kinect is a Microsoft’s motion sensing input device that runs through a natural user interface using gestures and spoken commands. Kinect initially was developed for video game console and Windows PCs; however, the technology gets widespread use for a variety of software applications around the globe due to low cost of development.

ICT/RT for Medical Homecare and Non-medical Care

The Japanese government has been putting its emphasis on elderly medical homecare from a political standpoint. However, an estimate based on the survey conducted by Ministry of Health, Labour and Welfare shows a shortage of 400,000 onsite workers next year, which may cause a big issue in the related facilities. There are several problems such as high turnover and absenteeism rates due to low income and heavy labor. We should focus on working conditions for care staffs in the first place, but it is difficult to improve it because of the constraint of elderly care insurance money. I have concluded that it is good to use information and communications technology (ICT) and robotics (RT) to come to grips with a situation that I have for better solution.

If we can create and update a database for information on the sites of medical homecare and non-medical care and send the summation of the information to medical institutes, high quality medical diagnosis will be conducted for patients without reducing diagnosis time. If we can summarize the information which has had occurred in geographically dispersed areas though use of social networking tools, we will be able to develop medical software applications, patient care devices, robots, etc. more effectively for each site. For example, we developed a web application and continuously are improving it in cooperation with Nara branch of Japan Parkinson Disease Association. We also developed a hardware device with which anyone easily can record body position information through use of Kinect and distributed the device to associated organizations for promotion in the field of medical homecare and non-medical care in cooperation with the onsite staffs. What’s more, my research team unveiled a new robot to assist in wearing in the late 2011. From now, we think we would like to focus on quantitative research on how much amount of procedure is needed to assist in wearing and on the summation of end user’s request for further technology development.

Brain Science for Market Entry

Through brain science and information and communications technology, our study is to clarify consumer’s buying behavior at retail stores. Furthermore, based on the study results, we also make inferences what they buy and make use of robots to change their behavior for products to enter into certain marketplaces. At retail stores, we experiment with communication robots from an experimental economics standpoint, and experiment on product selection in cooperation with participants who are equipped with near-infrared spectroscopy (NIRS) and eye movement tracking device. We also use Kinect in this case to recognize and record face direction and body position. By now, through communications with clients and use of face direction information, we have come to know by degrees that not only can we make inferences what they decide to buy, but we also can change a decision making that has been made for product selection. On principle, NIRS makes it possible to observe cerebral cortical activities outside laboratories. However, a variety of problems actually exist, so we continue to conduct research in the field of brain science to solve the problems on a step-by-step basis in order to make a contribution to business and social change.

Reducing Two Barriers to Study Abroad Programs: April Admission and Speak Japanese

I am a visiting educator of Indian Institutes of Technology Rajasthan (IIT Rajasthan) and Nara Institute of Science and Technology (NAIST) invites a few undergraduate students from IIT Rajasthan as intern each year. Last fall, the first student came to NAIST. This excellent student now contributes to improving a robot to assist in wearing through use of Kinect. Recently, NAIST began to provide a course at its Graduate School of Information Science for international students. Since the students can earn master degrees by taking classes in English, I believe that it is good for NAIST to secure students as well as for non-native Japanese speakers to study. In the meanwhile, it is impossible to grab the attention of excellent Indian students because of Indian education system if we, Japan, stick to April Admission when it comes to securing students. It was fortunate that there was no problem with respect to admission because NAIST has been adopting September Admission since its inception. If Japanese companies call on us to increase more excellent overseas students, it would be sure to do that if scholarships for September Admission were prepared. Rather, we are very much concerned about job opportunities for the students. When it comes to job search after earning master degrees at our university, there are a very few job opportunities within Japan for those who can not become fluent in Japanese. Despite that they came to study in Japan because they like Japan, there are not enough career development opportunities, encompassing practical training, after commencement in Japanese universities. They are joining classes in Japanese within the university; however, they are too far to reach professional levels to become competitive when it comes to job screening. As we think we would like Japanese companies to employ overseas great students for a variety of business activities in the organization, we call on the companies to make some changes and improvements in the organization, encompassing setting an English-speaking department, employing interpreter, etc. Don’t you think it also is a global innovation?