研究者詳細 - 中澤　篤志

論文 - 中澤　篤志

分割表示 >> ／全件表示 105 件中 1 - 105 件目

Cross-Action Cross-Subject Skeleton Action Recognition Via Simultaneous Action-Subject Learning With Two-Step Feature Removal 査読

Yu Mitsuzumi, Akisato Kimura, Go Irie, Atsushi Nakazawa

2024 IEEE International Conference on Image Processing (ICIP) 2182 - 2186 2024年10月

Phase Randomization: A data augmentation for domain adaptation in human action recognition 査読

Yu Mitsuzumi, Go Irie, Akisato Kimura, Atsushi Nakazawa

Pattern Recognition 146 110051 - 110051 2024年2月

Touching with Eye Contact and Vocal Greetings Increases the Sense of Security 査読

Miyuki Iwamoto, Atsushi Nakazawa

Human Activity and Behavior Analysis: Advances in Computer Vision and Sensors: Volume 1 1 104 - 119 2024年1月

掲載種別：論文集(書籍)内論文

Touching another person is the most direct way of communicating one’s presence to the other person and is also a means of communication. The act of touching can generate a sense of anxiety and a sense of security, and we hypothesized that there may be differences in the sense of anxiety and sense of security of the touched person depending on the person and the way of touching. Subjective evaluation and physiological index measurements were conducted. In the subjective evaluation, Russell’s circle model was used to measure arousal and discomfort caused by other stimuli such as prior vocalization and eye contact. We also measured skin conductance response (SCR) of skin electrical activity (electro dermal activity (EDA)) in response to trivial tension and arousal and investigated whether the stimulus combination affected SCR. In the subjective evaluation, the discomfort was reduced when the touch was preceded by a vocal greeting, and the arousal and discomfort were reduced when the touch was made in a position visible to the contacted person. In addition, the physiological index measurement showed that the SCR response was reduced when the participants were called before touch. This suggests that not only the act of touch but also the pre-touch behavior is an important indicator of the reduction of anxiety and a sense of security about touch.

DOI： 10.1201/9781003371540-7

Deep learning-based sperm motility and morphology estimation on stacked color-coded MotionFlow 査読

Sigit Adinugroho, Atsushi Nakazawa

Informatics in Medicine Unlocked 45 2024年1月

掲載種別：研究論文（学術雑誌）

Motility and morphology are crucial factors in determining male fertility. The current gold standard defined by the World Health Organization (WHO) mandates that semen analysis be performed by trained technicians. Despite strict standardization and technical guidelines set by the WHO, variability in semen analysis results remains prevalent owing to human subjectivity. Computer-Aided Sperm Analysis presents a further challenge because of its poor agreement with human analysis. This study aimed to enhance the accuracy of automated semen analysis by introducing a new method for expressing sperm motion and investigating advanced deep neural network architectures to estimate motility and morphology. Initially, we extracted motion information from the VISEM dataset using our novel motion representation technique called MotionFlow, along with shape information. Consequently, motility and morphology neural networks were constructed to exploit transfer learning in other fields to improve performance. These networks ingested motion and shape features and made separate predictions for motility and morphology. The evaluation process utilized a K-Fold cross-validation scheme to determine the mean absolute error (MAE) and maintain objectivity throughout the analysis. The proposed method achieved a greater level of performance than the current methods by attaining MAE of 6.842% and 4.148% for motility and morphology estimation, respectively. The improvement accomplished by this research may pave the way toward a fully automated human sperm quality assessment.

DOI： 10.1016/j.imu.2024.101459

認知症の人に対するケアコミュニケーション技法の研修を受講した看護学教員のアイコンタクトの特徴査読

OKAMOTO Eri, AOYAGI Hisami, TAKEUCHI Tomiko, NAKAZAWA Atsushi, HONDA Miwako

看護ケアサイエンス学会誌 22 ( 1 ) 2024年

Capturing Contact Surfaces by a Frustrated Total Internal Reflection System Using a Curved Plate for Comparison of the Beginning of Touching Motions by Humanitude Experts and Novices.

Akishige Yuguchi, Mayuki Toyoda, Sung-Gwi Cho, Atsushi Nakazawa, Jun Takamatsu, Koichiro Yoshino, Tsukasa Ogasawara

SMC 679 - 684 2024年

曲面板を用いた赤外線漏れ全反射方式でのユマニチュードの触れ始めの解析のための接触面情報の取得査読

湯口彰, 豊田真行, 趙崇貴, 吉野幸一郎, 高松淳, 中澤篤志, 小笠原司

日本ロボット学会誌 41 ( 10 ) 2023年10月

Behavioural changes in the interaction between child with autism spectrum disorder and mother through the Comprehensive Care Humanitude™ intervention 査読

Miyuki Iwamoto, Atsushi Nakazawa, Miwako Honda, Sakiko Yoshikawa, Toshihiro Kato, Yves Gineste

Proceedings of the Annual Meeting of the Cognitive Science Society 45 ( 45 ) 2023年7月

Behavioral and neural underpinnings of empathic characteristics in a Humanitude-care expert

Wataru Sato, Atsushi Nakazawa, Sakiko Yoshikawa, Takanori Kochiyama, Miwako Honda, Yves Gineste

Frontiers in Medicine 10 2023年5月

掲載種別：研究論文（学術雑誌）出版者・発行元：Frontiers Media SA

Background

Humanitude approaches have shown positive effects in elderly care. However, the behavioral and neural underpinnings of empathic characteristics in Humanitude-care experts remain unknown.

Methods

We investigated the empathic characteristics of a Humanitude-care expert (YG) and those of age-, sex-, and race-matched controls (n = 13). In a behavioral study, we measured subjective valence and arousal ratings and facial electromyography (EMG) of the corrugator supercilii and zygomatic major muscles while participants passively observed dynamic facial expressions associated with anger and happiness and their randomized mosaic patterns. In a functional magnetic resonance imaging (MRI) study, we measured brain activity while participants passively observed the same dynamic facial expressions and mosaics. In a structural MRI study, we acquired structural MRI data and analyzed gray matter volume.

Results

Our behavioral data showed that YG experienced higher subjective arousal and showed stronger facial EMG activity congruent with stimulus facial expressions compared with controls. The functional MRI data demonstrated that YG showed stronger activity in the ventral premotor cortex (PMv; covering the precentral gyrus and inferior frontal gyrus) and posterior middle temporal gyrus in the right hemisphere in response to dynamic facial expressions versus dynamic mosaics compared with controls. The structural MRI data revealed higher regional gray matter volume in the right PMv in YG than in controls.

Conclusion

These results suggest that Humanitude-care experts have behavioral and neural characteristics associated with empathic social interactions.

DOI： 10.3389/fmed.2023.1059203

Augmented reality-based affective training for improving care communication skill and empathy. 国際誌

Atsushi Nakazawa, Miyuki Iwamoto, Ryo Kurazume, Masato Nunoi, Masaki Kobayashi, Miwako Honda

PloS one 18 ( 7 ) e0288175 2023年

記述言語：英語掲載種別：研究論文（学術雑誌）

It is important for caregivers of people with dementia (PwD) to have good patient communication skills as it has been known to reduce the behavioral and psychological symptoms of dementia (BPSD) of PwD as well as caregiver burnout. However, acquiring such skills often requires one-on-one affective training, which can be costly. In this study, we propose affective training using augmented reality (AR) for supporting the acquisition of such skills. The system uses see-through AR glasses and a nursing training doll to train the user in both practical nursing skills and affective skills such as eye contact and patient communication. The experiment was conducted with 38 nursing students. The participants were assigned to either the Doll group, which only used a doll for training, or the AR group, which used both a doll and the AR system. The results showed that eye contact significantly increased and the face-to-face distance and angle decreased in the AR group, while the Doll group had no significant difference. In addition, the empathy score of the AR group significantly increased after the training. Upon analyzing the correlation between personality and changes of physical skills, we found a significant positive correlation between the improvement rate of eye contact and extraversion in the AR group. These results demonstrated that affective training using AR is effective for improving caregivers' physical skills and their empathy for their patients. We believe that this system will be beneficial not only for dementia caregivers but for anyone looking to improve their general communication skills.

DOI： 10.1371/journal.pone.0288175

ユマニチュードの触れ始めを再現するロボットハンド

豊田真行, 湯口彰重, 趙崇貴, 佐藤勇起, 高松淳, 中澤篤志, 和田隆広, 小笠原司

ロボティクス・メカトロニクス講演会講演概要集 2023 1A2-C18 2023年

Simulated communication skills training program effects using augmented reality with real‐time feedback: A randomized control study

Masaki Kobayashi, Miyuki Iwamoto, Saki Une, Ryo Kurazume, Atsushi Nakazawa, Miwako Honda

Alzheimer's & Dementia 18 ( S8 ) 2022年12月

掲載種別：研究論文（学術雑誌）出版者・発行元：Wiley

Abstract

Background

While communication with dementia patients is challenging, known educational methods to improve communication skill for medical professionals are lacking. Our study aimed to assess the efficacy of simulated communication skills training for nursing students using augmented reality (AR) with real‐time feedback.

Methods

This is a randomized control study. Twenty‐five nursing students enrolled and learned standardized multimodal comprehensive care communication skills through a self‐learning material, which includes pathophysiology and clinical manifestations of dementia. Subsequently, participants were randomly assigned to one of the two learning systems—AR training or conventional nursing mannequin training. Each group had the training intervention to change the clothes of mannequins for one hour. The mannequin of the AR group was a superimposed computer graphic of an elderly woman’s face which reacted to participants' communication. Further, the communication skills of gaze and voice were evaluated by artificial intelligence (AI) and feedbacked to the participants’ head‐mounted display in real‐time. The conventional training group had the self‐training with nursing mannequins. All participants performed basic nursing care, including changing clothes and bed bath, to simulated patients before and after the training intervention, which were video‐recorded by an eye‐tracking camera and a fixed camera, then the videos were analyzed the communication skills by AI. Additionally, participants' empathy to patients was evaluated by the Jefferson Scale of Empathy‐Health Professions Students Version (JSPE‐HSP). The primary outcome was the proportion of time spent in eye contact during the care to simulated patients. The secondary outcome was the empathy score of JSPE‐HSP.

Results

After the training intervention, the proportion of time spent in eye contact in the AR training group significantly increased than the conventional training group (eye contact 13.6% versus 4.4%, P<0.05). Moreover, the JSPE‐HSP score increased from pre‐training to post‐training in the AR training group, whereas it decreased in the conventional training group [Mean (SD): 9.1 (6.6) versus ‐1.3 (3.8), P<0.01].

Conclusions

The simulated communication skills training for nursing students using augmented reality with real‐time feedback was associated with increased interactive communication skills to simulated patients and the empathy to patients.

DOI： 10.1002/alz.062055

Editorial: Interaction in robot-assistive elderly care. 国際誌

Hidenobu Sumioka, Jim Torresen, Masahiro Shiomi, Liang-Kung Chen, Atsushi Nakazawa

Frontiers in robotics and AI 9 1020103 - 1020103 2022年

Non-goal-driven eye movement after visual search task

Ayumi Takemoto, Atsushi Nakazawa, Takatsune Kumada

Journal of Eye Movement Research 15 ( 2 ) 2022年

掲載種別：研究論文（学術雑誌）

We investigated the functions and mechanisms of non-goal-driven eye movements, which are defined as eye movements induced when looking at visual stimuli on a display without engaging in a specific task or looking at a display without any visual stimuli or tasks. In our experiment, participants were asked to perform a visual search task on a display, which was followed by a rest period in which stimuli remained on the display or all stimuli were erased. During the rest period, the participants were asked to only look at the displays without engaging in any visual or cognitive tasks. We mainly analyzed the gaze-shift patterns in both task and rest periods, in which eye movements were classified in accordance with the angles of saccade directions in two consecutive saccades. The results indicate a significant difference between goal-driven eye movements, which were observed in the task period, and nongoal-driven eye movements, which were observed in the rest period. Scanning gaze-shift patterns dominated the task period, and backward and corrective-saccade-like gaze-shift patterns dominated the rest period. The gaze-shift pattern was affected by the task-difficulty during the task period. From these findings, we propose a model describing the oculomotor system in terms of goal-driven and non-goal-driven eye movements. In this model, the engagement levels of top-down and bottom-up control change along with task difficulty and are affected by the gaze-shift patterns during a visual search task. Decoupling of top-down control from the oculomotor system during a rest period induces backward saccades, resulting in fixation around the central part of a display. Therefore, we suggest that non-goaldriven eye movements play a crucial role in maintaining the readiness of the oculomotor system for the next search task.

DOI： 10.16910/jemr.15.2.2

Drowsiness prevention using a social robot.

Koki Hara, Ayumi Takemoto, Atsushi Nakazawa

RO-MAN 603 - 609 2022年

Ethical Considerations in User Modeling and Personalization (ECUMAP): ACM UMAP 2022 Tutorial.

Jim Torresen, Atsushi Nakazawa

UMAP 351 - 353 2022年

Facial expression translations preserving speaking content.

Shiki Takeuchi, Atsushi Nakazawa

26th International Conference on Pattern Recognition(ICPR) 1215 - 1221 2022年

Development of educational system for paramedics specializing to dementia elderlies analyzed by artificial intelligence and its learning effects. 国際誌

Miwako Honda, Saki Une, Yves Gineste, Atsushi Nakazawa

Alzheimer's & dementia : the journal of the Alzheimer's Association 17 Suppl 8 e051576 2021年12月

記述言語：英語掲載種別：研究論文（学術雑誌）

BACKGROUND: While the number of emergency transfer of dementia elderlies to acute care hospitals has being increasing, paramedics are facing challenges to perform their duties promptly due to difficult communication with patients. OBJECTIVES: To develop the standardized educational system for paramedics specializing to dementia elderlies with the use of artificial intelligence. DESIGN: Single arm, pre-post study. Participants are paramedics in a metropolitan area of Japan. EXPOSURE: The two sessions of standardized multimodal communication training program which contains lecture and workshop were provided twice, 1 month apart. The training consists three parts. 1: Mechanism and clinical manifestations of dementia, 2: Communication skills: face-to-face interaction; mutual gaze, geometric characteristics and duration of eye contact, verbal communication with phonetic elements, and touch interactions. 3: Interaction strategies with dementia elderlies in emergency situation. All participants performed the rescue simulation training before the training and 1 month after the completion of the training, which were video-recorded and analyzed by artificial intelligence (AI). The video was taken by wearable camera with eye-trackers (1st person camera) and fixed camera (3rd person camera). The satisfaction survey of effectiveness on their work and the Jefferson Scale of Empathy for healthcare professionals were also taken. MAIN OUTCOMES: The primary outcome is the change of communication skills in physical position, distance, eye contact with patients, phonetic elements and touch. The system to analyze face-to-face communication; mutual gaze, geometric characteristics and duration of eye contact by AI was developed by our team. Other elements of communications were evaluated manually. Secondary outcome were satisfaction and the empathy to patients. RESULT: 31 paramedics enrolled the study. Prior to the training, 88% had experienced any difficult condition taking care of dementia patients. The video analysis showed significant change of the interactive communication in the direction of approach, eye contacts and duration of mutual gaze. 96.7% considered the training was effective for their daily work and 97.5% of participants recommended this training to a colleague. The score of empathy scale showed no significant change. CONCLUSION: The educational system for paramedics specializing to dementia with the use of artificial intelligence is effective for their work.

DOI： 10.1002/alz.051576

Image Emotion Recognition Using Visual and Semantic Features Reflecting Emotional and Similar Objects

Takahisa YAMAMOTO, Shiki TAKEUCHI, Atsushi NAKAZAWA

IEICE Transactions on Information and Systems E104.D ( 10 ) 1691 - 1701 2021年10月

Technical Challenges for Smooth Interaction With Seniors With Dementia: Lessons From Humanitude™ 国際誌

Hidenobu Sumioka, Masahiro Shiomi, Miwako Honda, Atsushi Nakazawa

Frontiers in Robotics and AI 8 650906 - 650906 2021年6月

記述言語：英語掲載種別：研究論文（学術雑誌）

Due to cognitive and socio-emotional decline and mental diseases, senior citizens, especially people with dementia (PwD), struggle to interact smoothly with their caregivers. Therefore, various care techniques have been proposed to develop good relationships with seniors. Among them, Humanitude is one promising technique that provides caregivers with useful interaction skills to improve their relationships with PwD, from four perspectives: face-to-face interaction, verbal communication, touch interaction, and helping care receivers stand up (physical interaction). Regardless of advances in elderly care techniques, since current social robots interact with seniors in the same manner as they do with younger adults, they lack several important functions. For example, Humanitude emphasizes the importance of interaction at a relatively intimate distance to facilitate communication with seniors. Unfortunately, few studies have developed an interaction model for clinical care communication. In this paper, we discuss the current challenges to develop a social robot that can smoothly interact with PwDs and overview the interaction skills used in Humanitude as well as the existing technologies.

DOI： 10.3389/frobt.2021.650906

Evaluating imitation and rule-based behaviors of eye contact and blinking using an android for conversation

Akishige Yuguchi, Tetsuya Sano, Gustavo Alfonso Garcia Ricardez, Jun Takamatsu, Atsushi Nakazawa, Tsukasa Ogasawara

ADVANCED ROBOTICS 35 ( 15 ) 907 - 918 2021年5月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：TAYLOR & FRANCIS LTD

In this paper, we investigate which approach to generate eye behaviors using an android robot makes what impressions on humans and clarify which are the important factors for attractive eye behaviors. Thus, we evaluate the human impression of eye behaviors displayed by an android robot while talking to a human by comparing the motions generated by the two approaches. In the first approach, we develop a method to imitate human eye behavior obtained from eye trackers. In the second approach, we control the eye direction, eye-contact duration, and eyeblinks based on the findings of human eye behavior in psychology and cognitive research. In the experiments, we asked male and female subjects to evaluate their impression by comparing the eye behaviors with an android that controls the eye-contact duration and eyeblinks by editing the imitation parameters or the rule-based behavior. In the experiments, we asked subjects to evaluate their impression of different eye behaviors displayed by an android. The eye behaviors were generated by modifying the imitation parameters or the rule-based behavior, which resulted in different eye-contact duration and eyeblink duration and timing. From the results, we found that (1) the imitation and rule-based behaviors showed no difference in terms of human-likeness, (2) the 3-second eye contact obtained better scores regardless of the imitation or rule-based eye behavior, (3) the subjects might regard the long eyeblinks as voluntary eyeblinks, with the intention to break eye contact, and (4) female subjects preferred short eyeblinks rather than long ones and considered that short eyeblinks might be one of the keys to making eye contact more suitable, in contrast to male subjects who preferred long eyeblinks.

DOI： 10.1080/01691864.2021.1928544

他者の視線方向・表情と参加者の体勢が介護ベッド上での対人認知に及ぼす影響-ヘッドマウントディスプレイを使用した検討-

布井雅人, 吉川左紀子, 中澤篤志

電子情報通信学会論文誌 A(Web) J104-A ( 2 ) 2021年

Visual Place Recognition from Eye Reflection

Yuki Ohshima, Kyosuke Maeda, Yusuke Edamoto, Atsushi Nakazawa

IEEE Access 9 57364 - 57371 2021年

掲載種別：研究論文（学術雑誌）

The cornea in the human eye reflects incoming environmental light, which means we can obtain information about the surrounding environment from the corneal reflection in facial images. In recent years, as the quality of consumer cameras increases, this has caused privacy concerns in terms of identifying the people around the subject or where the photo is taken. This paper investigates the security risk of eye corneal reflection images: specifically, visual place recognition from eye reflection images. First, we constructed two datasets containing pairs of scene and corneal reflection images. The first dataset is taken in a virtual environment. We showed pre-captured scene images in a 180-degree surrounding display system and took corneal reflections from subjects. The second dataset is taken in an outdoor environment. We developed several visual place recognition algorithms, including CNN-based image descriptors featuring a naive Siamese network and AFD-Net combined with entire image feature representations including VLAD and NetVLAD, and compared the results. We found that AFD-Net+VLAD performed the best and was able to accurately determine the scene in 73.08% of the top-five candidate scenes. These results demonstrate the potential to estimate the location at which a facial picture was taken, which simultaneously leads to a) positive applications such as the localization of a robot while conversing with persons and b) negative scenarios including the security risk of uploading facial images to the public.

DOI： 10.1109/ACCESS.2021.3071406

GAN-based Style Transformation to Improve Gesture-recognition Accuracy

Noeru Suzuki, Yuki Watanabe, Atsushi Nakazawa

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4 ( 4 ) 154 - 20 2020年12月

掲載種別：研究論文（学術雑誌）

Gesture recognition and human-Activity recognition from multi-channel sensory data are important tasks in wearable and ubiquitous computing. In these tasks, increasing both the number of recognizable activity classes and the recognition accuracy is essential. However, this is usually an ill-posed problem because individual differences in the same gesture class may affect the discrimination of different gesture classes. One promising solution is to use personal classifiers, but this requires personal gesture samples for re-Training the classifiers. We propose a method of solving this issue that obtains personal gesture classifiers using few user gesture samples, thus, achieving accurate gesture recognition for an increased number of gesture classes without requiring extensive user calibration the novelty of our method is introducing a generative adversarial network (GAN)-based style transformer to 'generate' a user's gesture data the method synthesizes the gesture examples of the target class of a target user by transforming of a) gesture data into another class of the same user (intra-user transformation) or b) gesture data of the same class of another user (inter-user transformation) the synthesized data are then used to train the personal gesture classifier. We conducted comprehensive experiments using 1) different classifiers including SVM and CNN, 2) intra-and inter-user transformations, 3) various data-missing patterns, and 4) two different types of sensory data. Results showed that the proposed method had an increased performance. Specially, the CNN-based classifiers increased in average accuracy from 0.747 to 0.822 in the CheekInput dataset and from 0.856 to 0.899 in the USC-HAD dataset. Moreover, the experimental results with various data-missing conditions revealed a relation between the number of missing gesture classes and the accuracy of the existing and proposed methods, and we were able to clarify several advantages of the proposed method these results indicate the potential of considerably reducing the number of required training samples of target users.

DOI： 10.1145/3432199

A Generative Self-Ensemble Approach to Simulated+Unsupervised Learning

Yu Mitsuzum, Go Irie, Akisato Kimura, Atsushi Nakazawa

Proceedings - International Conference on Image Processing, ICIP 2020-October 2151 - 2155 2020年10月

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

In this paper, we consider Simulated and Unsupervised (S+U) learning which is a problem of learning from labeled synthetic and unlabeled real images. After translating the synthetic images to real ones, existing S+U learning methods use only the labeled synthetic images for training a predictor (e.g., a regression function) and ignore the target real images, which may result in unsatisfactory prediction performance. Our approach utilizes both synthetic and real images to train the predictor. The main idea of ours is to involve a self-ensemble learning framework into S+U learning. More specifically, we require the prediction results for an unlabeled real image to be consistent between 'teacher' and 'student' predictors, even after some perturbations are added to the image. Furthermore, aiming at generating diverse perturbations along the underlying data manifold, we introduce one-to-many image translation between synthetic and real images. Evaluation experiments on an appearance-based gaze estimation task demonstrate that the proposed ideas can improve the prediction accuracy and our full method can outperform existing S+U learning methods.

DOI： 10.1109/ICIP40778.2020.9191100

その他リンク： https://dblp.uni-trier.de/db/conf/icip/icip2020.html#MitsuzumiIKN20

First-person Video Analysis for Evaluating Skill Level in the Humanitude Tender-Care Technique

Atsushi Nakazawa, Yu Mitsuzumi, Yuki Watanabe, Ryo Kurazume, Sakiko Yoshikawa, Miwako Honda

Journal of Intelligent and Robotic Systems: Theory and Applications 98 ( 1 ) 103 - 118 2020年4月

掲載種別：研究論文（学術雑誌）

In this paper, we describe a wearable first-person video (FPV) analysis system for evaluating the skill levels of caregivers. This is a part of our project that aims to quantize and analyze the tender-care technique known as Humanitude by using wearable sensing and AI technology devices. Using our system, caregivers can evaluate and elevate their care levels by themselves. From the FPVs of care sessions taken by wearable cameras worn by caregivers, we obtained the 3D facial distance, pose and eye-contact states between caregivers and receivers by using facial landmark detection and deep neural network (DNN)-based eye contact detection. We applied statistical analysis to these features and developed algorithms that provide scores for tender-care skill. In experiments, we first evaluated the performance of our DNN-based eye contact detection by using eye contact datasets prepared from YouTube videos and FPVs that assume conversational scenes. We then performed skill evaluations by using Humanitude training scenes involving three novice caregivers, two Humanitude experts and seven middle-level students. The results showed that our eye contact detection outperformed existing methods and that our skill evaluations can estimate the care skill levels.

DOI： 10.1007/s10846-019-01052-8

Real-time surgical needle detection using region-based convolutional neural networks

Atsushi Nakazawa, Kanako Harada, Mamoru Mitsuishi, Pierre Jannin

International Journal of Computer Assisted Radiology and Surgery 15 ( 1 ) 41 - 47 2020年1月

掲載種別：研究論文（学術雑誌）

Objective: Conventional surgical assistance and skill analysis for suturing mostly focus on the motions of the tools. As the quality of the suturing is determined by needle motions relative to the tissues, having knowledge of the needle motion would be useful for surgical assistance and skill analysis. As the first step toward demonstrating the usefulness of the knowledge of the needle motion, we developed a needle detection algorithm. Methods: Owing to the small needle size, attaching sensors to it is difficult. Therefore, we developed a real-time video-based needle detection algorithm using a region-based convolutional neural network. Results: Our method successfully detected the needle with an average precision of 89.2%. The needle was robustly detected even when the needle was heavily occluded by the tools and/or the blood vessels during microvascular anastomosis. However, there were some incorrect detections, including partial detection. Conclusion: To the best of our knowledge, this is the first time deep neural networks have been applied to real-time needle detection. In the future, we will develop a needle pose estimation algorithm using the predicted needle location toward computer-aided surgical assistance and surgical skill analysis.

DOI： 10.1007/s11548-019-02050-9

その他リンク： https://dblp.uni-trier.de/db/journals/cars/cars15.html#NakazawaHMJ20

Eye Contact Detection from Third Person Video

Yuki Ohshima, Atsushi Nakazawa

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12047 LNCS 667 - 677 2020年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Springer

Eye contact is fundamental for human communication and social interactions; therefore much effort has been made to develop automated eye-contact detection using image recognition techniques. However, existing methods use first-person-videos (FPV) that need participants to equip wearable cameras. In this work, we develop an novel eye contact detection algorithm taken from normal viewpoint (third person video) assuming the scenes of conversations or social interactions. Our system have high affordability since it does not require special hardware or recording setups, moreover, can use pre-recorded videos such as Youtube and home videos. In designing algorithm, we first develop DNN-based one-sided gaze estimation algorithms which output the states whether the one subject looks at another. Afterwards, eye contact is found at the frame when the pair of one-sided gaze happens. To verify the proposed algorithm, we generate third-person eye contact video dataset using publicly available videos from Youtube. As the result, proposed algorithms performed 0.775 in precision and 0.671 in recall, while the existing method performed 0.484 in precision and 0.061 in recall, respectively.

DOI： 10.1007/978-3-030-41299-9_52

その他リンク： https://dblp.uni-trier.de/db/conf/acpr/acpr2019-2.html#OhshimaN19

傾聴感を高めるためのアンドロイドの瞬目と頷きの模倣動作生成

湯口彰重, 高松淳, 中澤篤志, 小笠原司

ロボティクス・メカトロニクス講演会講演概要集 2020 1P2-E16 2020年

An examination of gaze during conversation for designing culture-based robot behavior

Louisa Hardjasa, Atsushi Nakazawa

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12194 LNCS 475 - 488 2020年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Springer

Gaze behavior, including eye contact and gaze direction, is an essential component of non-verbal communication, helping to facilitate human-to-human conversation in ways that have often been thought of as universal and innate. However, these have been shown to be influenced partially by cultural norms and background, and despite this, the majority of social robots do not have any cultural-based non-verbal behaviors and several lack any directional gaze capabilities at all. This study aims to observe how different gaze behaviors manifest during conversation as a function of culture as well as exposure to other cultures, by examining differences in behaviors such as duration of direct gaze, duration and direction of averted gaze, and average number of shifts in gaze, with the objective of establishing a baseline of Japanese gaze behavior to be implemented into a social robot. Japanese subjects were found to have much more averted gaze during a task that involves thinking as opposed to a task focused on communication. Subjects with significant experience living overseas were found to have different directional gaze patterns from subjects with little to no overseas experience, implying that non-verbal behavior patterns can change with exposure to other cultures.

DOI： 10.1007/978-3-030-49570-1_33

その他リンク： https://dblp.uni-trier.de/db/conf/hci/hci2020-14.html#HardjasaN20

Comparison of Gaze Skills Between Expert and Novice in Elderly Care

Miyuki Iwamoto, Atsushi Nakazawa

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12208 LNCS 91 - 100 2020年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Springer

It is known that a person, when ignored by others, causes various negative reactions and enhances aggressive behavior and Self-destructive behavior [1–3]. The same goes for elderly people with dementia, therefore, depending on the care may cause fear and confusion for elderly people with dementia [4]. At the same time, care for the elderly with dementia has a great mental and physical burden on caregivers. For this reason, the turnover rate has increased and it has become difficult to provide adequate care [5–8]. For these problems, as one of the dementia care is gaining attention Humanitude [4, 10]. Humanitude consists of four skills: “see”, “touch”, “speak”, and “stand” [9–11]. So, our research focus on “see”, one of the basic skills. A person’s gaze is generally directed to an object of interest or attention. The gaze is extremely useful information for estimating the mind of another person [3, 12, 13]. Therefore, we let a caregiver wear the first person camera, and four types of “seeing” behavior patterns during the oral care (care receiver → caregiver, care receiver ← caregiver, mutual gaze, none) measured. We compared the differences between Humanitude experts and novice. There was a large difference in the frequency and time of mutual gaze between expert and novice caregiver for care receiver. The act of matching the sight of eyes is an act of not ignoring the other person, indicating an interest in the care receiver, and it is considered that for the care receiver, the anxiety and fear during care are reduced.

DOI： 10.1007/978-3-030-50249-2_7

その他リンク： https://dblp.uni-trier.de/db/conf/hci/hci2020-28.html#IwamotoN20

Evaluating Imitation of Human Eye Contact and Blinking Behavior Using an Android for Human-like Communication

Tetsuya Sano, Akishige Yuguchi, Gustavo Alfonso Garcia Ricardez, Jun Takamatsu, Atsushi Nakazawa, Tsukasa Ogasawara

2019 28th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2019 1 - 6 2019年10月

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

The appearance of android robots is very similar to that of human beings. From their appearance, we expect that androids might provide us with high-level communication. The imitation of human behavior gives us the feeling of natural behavior even if we do not know what drives high-level communication. In this paper, we evaluate the imitation of human eye behavior by an android. We consider that the android imitates human eye behavior while explaining some research topic and a person acts as a listener. Then, we construct a method to imitate the eye behavior obtained from eye trackers. For the evaluation, we asked seventeen male subjects for their subjective evaluation and compared the imitation with an android that controlled eye-contact duration and eyeblinks by editing the imitation or programming rule-based behavior. From the results, we found out that 1) the rule-based behaviors kept human-likeness, 2) 3-second eye contact obtained better scores regardless of the imitation-based or rule-based eye behavior, and 3) the subjects might regard the longer eyeblinks as voluntary eyeblinks, with the intention to break eye contacts.

DOI： 10.1109/RO-MAN46459.2019.8956387

その他リンク： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2019.html#SanoYRTNO19

First-person camera system to evaluate tender dementia-care skill

Atsushi Nakazawa, Miwako Honda

Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019 4435 - 4442 2019年10月

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

In this paper, we describe a wearable first-person video (FPV) analysis system for evaluating the skill levels of tender dementia-care technique. Using this system, caregivers can evaluate and elevate their care levels by themselves using the systems' feedbacks. From the FPVs of care sessions taken by wearable cameras worn by caregivers, we obtained the 3D facial distance, pose and eye-contact states between caregivers and receivers by using facial landmark detection and deep neural network (DNN)-based eye contact detection. We applied statistical analysis to these features and developed algorithms that provide scores for tender-care skill. To find and confirm our idea, we conducted chronological study to observe the progression of tender care-skill learning using care learners. First, we took FPVs while care training scenes involving novice caregivers, tender-care experts and middle-level students, and found major behavioural differences among them. Second, we performed the same experiments for the participants before and after training sessions of the care. As the result, we found the same behavioural difference between 1) novices and experts and 2) novices before and after taking training sessions. These results indicate that our FPV-based behavior analysis can evaluate the skill progression of the tender dementia-care.

DOI： 10.1109/ICCVW.2019.00544

その他リンク： https://dblp.uni-trier.de/db/conf/iccvw/iccvw2019.html#NakazawaH19

HMD-based cover test system for the diagnosis of ocular misalignment

Noriyuki Uchida, Kayoko Takatuka, Hisaaki Yamaba, Atsushi Nakazawa, Masayuki Mukunoki, Naonobu Okazaki

Artificial Life and Robotics 24 ( 3 ) 332 - 337 2019年9月

掲載種別：研究論文（学術雑誌）

The diagnosis of ocular misalignment is difficult and needs examination by ophthalmologists and orthoptists. However, there are not enough qualified personnel to perform such diagnoses. The eye position check is in part systematized. With this check system, we can detect not only the symptoms but also the angle and the extent of strabismus. However, the types of strabismus that can be detected with this technique are limited to exotropia. The purpose of this study is to develop a simplified check system to screen at least the presence of strabismus apart from the type of strabismus or amount of ocular deviation. First, we digitalized the check process. Specifically, we digitized the elemental technology, i.e., the cover–uncover function, required for automation of the typical cover test for eye position check. Furthermore, we developed and implemented an abnormality determination process and evaluated the performance of the system through this experiment, the results of which indicated a higher detection capability than the conventional cover test performed by ophthalmologists and orthoptists.

DOI： 10.1007/s10015-018-0520-4

Towards overhead semantic part segmentation of workers in factory environments from depth images using a FCN

Masakazu Yoshimura, Murilo M. Marinho, Atsushi Nakazawa, Kanako Harada, Mamoru Mitsuishi, Takuya Maekawa, Yasuo Namioka

2019 IEEE International Conference on Cyborg and Bionic Systems, CBS 2019 204 - 209 2019年9月

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

In production lines, human workers assemble and/or inspect products in a predetermined process flow. Training new workers is a complex process with varying results. To track the workers motion in a way to better understand human skill, an overhead semantic part segmentation of workers is desired. For this purpose, in this work, we propose a fully-convolutional neural-network model paired with four proposed augmentation strategies. Artificial depth images were used as training data and the augmentation strategies were essential in aiding the network to generalize to the real images. The proposed method was evaluated in two tasks with different backgrounds: a part assembly task and a quality check task. We improved the F-measure by 12% in the part assembly task and 4% in the quality check task when compared to our previous work.

DOI： 10.1109/CBS46900.2019.9114417

その他リンク： https://dblp.uni-trier.de/db/conf/cbs/cbs2019.html#YoshimuraMNHMMN19

Eye Movement on Blank Displays: Effects of Difficulty in Preceding Visual Tasks and Mental State

Ayumi Takemoto, Atsushi Nakazawa, Takatsune Kumada

PERCEPTION 48 220 - 220 2019年9月

Robust Pupil Segmentation and Center Detection from Visible Light Images Using Convolutional Neural Network 査読

Kazunari Kitazumi, Atsushi Nakazawa

Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 862 - 868 2019年1月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

In this paper, we present a robust pupil detection method from visible light (VL) images using a convolutional neural network (CNN). In contrast to existing pupil detection algorithms, our method does not require infrared (IR) illuminations and cameras, and robustly works even for the images of dark-brown irises (black eyes) and/or images including strong corneal reflections such as outdoor scenes. Thus it has potential application scenarios. Our method first detects an eye region from an input image and applies CNN-based pupil segmentation which has a composition and decomposition structure. To learn the relationship between visible eye images and pupil segments, we construct two datasets and augmentation algorithms. One dataset is based on an existing eye image dataset (UBIRIS.v2), and the other consists of images taken by ourselves by using a corneal imaging camera that can take eye images in mobile environments. Applying color and corneal reflection augmentations to these image datasets and using them for learning the CNN, we built a robust pupil segmentation neural network. The performance is evaluated in three ways. First, we evaluate the segmentation accuracy. Second, we evaluate the pupil center detection accuracy using GI4E facial image sets. Third, we developed an eye gaze tracking (EGT) algorithm that uses the pupil detection and evaluated its accuracy. From the result, the proposed method detects pupil centers more accuracy than the state-of-the arts, and shows similar EGT accuracy to the commercial systems that use IR-active lighting setups.

DOI： 10.1109/SMC.2018.00154

その他リンク： https://dblp.uni-trier.de/db/conf/smc/smc2018.html#KitazumiN18

Eye Contact Detection Algorithms Using Deep Learning and Generative Adversarial Networks 査読

Yu Mitsuzumi, Atsushi Nakazawa

Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 3927 - 3931 2019年1月

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

Eye contact (mutual gaze) is a foundation of human communication and social interactions; therefore, it is studied in many fields such as psychology, social science, and medicine. Our group have been studied wearable vision-based eye contact detection techniques using a first person camera for the purpose of evaluating the gaze skills in the tender dementia care. In this work, we search for deep learning-based eye contact detection techniques from small number of labeled images. We implemented and tested two eye contact detection algorithms: naïve deep-learning-based algorithm and generative adversarial networks (GAN)-based semi supervised learning (SSL) algorithm. These methods are learned and verified by using Columbia Gaze Dataset, Facescrub and our original datasets. The results show the effectiveness and limitations of the deep-learning-based and GAN-based approaches. Interestingly, we found the bilateral difference of the accuracy of eye contact detection with respect to the facial pose with respect to the camera, which is expected to be caused by the learning datasets.

DOI： 10.1109/SMC.2018.00666

その他リンク： https://dblp.uni-trier.de/db/conf/smc/smc2018.html#MitsuzumiN18

Spatio-temporal eye contact detection combining CNN and LSTM

Yuki Watanabe, Atsushi Nakazawa, Yu Mitsuzumi, Toyoaki Nishida

PROCEEDINGS OF MVA 2019 16TH INTERNATIONAL CONFERENCE ON MACHINE VISION APPLICATIONS (MVA) 1 - 7 2019年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

Eye contact (mutual gaze) is fundamental for human communication and social interactions; therefore, it is studied in many fields. To support the study of eye contact, much effort has been made to develop automated eye-contact detection using image recognition techniques. In recent years, convolutional neural network (CNN) based eye-contact detection techniques are becoming popular due to their performance; however, they mainly use single frame for recognition. Eye contact is a human communication behavior, so temporal information, such as temporal eye images and facial poses, is important to increase the accuracy of eye-contact detection. We incorporate temporal information into eye-contact detection by using temporal neural network structures that combine CNNs and long short-term memory (LSTM). We tested several network combinations of CNNs and LSTM and found the best solution that uses the outputs of CNNs as well as the cell state vectors of LSTM in the fully connected layers. We prepared two types of eye contact video datasets. One dataset is based on online videos, and the other was taken by a first-person camera in assumed conversational scenarios. The results show that our method is better than the approaches that use single frames. Namely, our method performs 0.8781, while the existing method (DeepEC) performed 0.8319, in F-1-score.

DOI： 10.23919/MVA.2019.8757989

その他リンク： https://dblp.uni-trier.de/db/conf/mva/mva2019.html#WatanabeNMN19

General Improvement Method of Specular Component Separation Using High-Emphasis Filter and Similarity Function

Takahisa Yamamoto, Atsushi Nakazawa

ITE TRANSACTIONS ON MEDIA TECHNOLOGY AND APPLICATIONS 7 ( 2 ) 92 - 102 2019年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：INST IMAGE INFORMATION & TELEVISION ENGINEERS

Separating reflection components is a fundamental problem in computer vision and useful for many applications such as image quality. We propose a novel method that improves the accuracy of separating reflection components from a single image. Although several algorithms for separating reflection components have been proposed, our method can additionally improve the accuracy based on their results. First, we obtain diffuse and specular components by using an existing algorithm. Then, we apply a high-emphasis filter for each component. Since the responses of the high-emphasis filter where the separation fails become larger than the original values, we can detect erroneous pixels. Thus, we replace separation results of these erroneous pixels with results of other reference pixels from the image considering the similarity between the target and reference pixels. Experimental results show that our method can improve at most 13.61 dB in terms of the Peak Signal-to-Noise Ratio (PSNR).

DOI： 10.3169/mta.7.92

FASHION STYLE RECOGNITION USING COMPONENT-DEPENDENT CONVOLUTIONAL NEURAL NETWORKS

Takahisa Yamamoto, Atsushi Nakazawa

2019 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP) 2019-September 3397 - 3401 2019年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

The fashion style recognition is important in online marketing applications. Several algorithms have been proposed, but their accuracy is still unsatisfactory. In this paper, we share our proposed method for creating an improved fashion style recognition algorithm, component-dependent convolutional neural networks (CD-CNNs). Given that a lot of fashion styles largely depend on the features of specific body parts or human body postures, first, we obtain images of the body parts and postures by using semantic segmentation and pose estimation algorithms; then, we pre-train CD-CNNs. We perform the classification by the concatenated outputs of CD-CNNs and a support vector machine (SVM). Experimental results using the HipsterWars and FashionStyle14 datasets prove that our method is effective and can improve classification accuracy, namely 85.3% for HipsterWars and 77.7% for FashionStyle14, while those of existing methods were 80.9% for HipsterWars and 72.0% for FashionStyle14.

DOI： 10.1109/ICIP.2019.8803622

その他リンク： https://dblp.uni-trier.de/db/conf/icip/icip2019.html#YamamotoN19

優しい介護インタラクションの計算的・脳科学的解明査読

中澤篤志

ヒューマンインタフェースシンポジウム 2018 ( UBI-59 ) 2018年9月

Development of a skill evaluation system for the camera assistant using an infant-sized laparoscopic box trainer

Tetsuya Ishimaru, Kyoichi Deie, Tomoya Sakai, Hideyuki Satoh, Atsushi Nakazawa, Kanako Harada, Shinya Takazawa, Jun Fujishiro, Naohiko Sugita, Mamoru Mitsuishi, Tadashi Iwanaka

Journal of Laparoendoscopic and Advanced Surgical Techniques 28 ( 7 ) 906 - 911 2018年7月

掲載種別：研究論文（学術雑誌）

Aims: Our aims were to develop a training system for camera assistants (CA), and evaluate participants' performance as CA. Methods: A questionnaire on essential requirements to be a good CA was administered to experts in pediatric endoscopic surgery. An infant-sized box trainer with several markers and lines inside was developed. Participants performed marker capturing and line-tracing tasks using a 5-mm 30° scope. A postexperimental questionnaire on the developed system was administered. The task completion time was measured. Results: The 5-point evaluation scale was used for each item in the questionnaire survey of experts. The abilities to maintain a horizontal line (mean score: 4.5) and to center the target in a specified rectangle on the monitor (4.5) as well as having a full understanding of the operative procedure (4.3) were ranked as highly important. Fifty-two participants, including 5 surgical residents, were enrolled in the evaluation experiment. The completion time of capturing the markers was significantly longer in the resident group than in the nonresident group (244 versus 124 seconds, P = .04), but that of tracing the lines was not significantly different between the groups. The postexperimental questionnaire showed that the participants felt that the line-tracing tasks (3.7) were more difficult than marker-capturing tasks (2.9). Conclusions: Being proficient in manipulating a camera and having adequate knowledge of operative procedures are essential requirements to be a good CA. The ability was different between the resident and nonresident groups even in a simple task such as marker capturing.

DOI： 10.1089/lap.2017.0406

Computational Tender-Care Science: Computational and Cognitive Neuro-scientific Approaches for Understanding the Tender Care 査読

Atsushi Nakazawa, Kyoto University, Japan, Ryo Kurazume, Kyushu University, Japan, Miwako Honda, Tokyo Medical Center, Sato Wataru, Kyoto University, Japan, Shogo Ishikawa, Shizuoka University, Japan, Sakiko Yoshikawa

Workshop on Elderly Care Robotics – Technology and Ethics 2018 ( UBI-59 ) 2018年5月

Computational Tender-Care Science: Computational and Cognitive Neuroscientific Approaches for Understanding the Tender Care 査読

Nakazawa A, Kurazume R, Honda M, Sato W, Ishikawa S, Yoshikawa S, Ito M

IUI Workshop on Symbiotic Interaction and Harmonius Collaboration for Wisdom Computing 1 ( UBI-59 ) 1 - 9 2018年3月

深層学習を用いた可視光画像からの瞳孔検出と注視点推定への応用

北角一哲, 中澤篤志, 西田豊明

電子情報通信学会技術研究報告 117 ( 392 ) 93 - 99 2018年1月

Towards robust needle segmentation and tracking in pediatric endoscopic surgery.

Yujun Chen, Murilo M. Marinho, Yusuke Kurose, Atsushi Nakazawa, Kyoichi Deie, Kanako Harada, Mamoru Mitsuishi

Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling 105762 2018年

Point of Gaze Estimation Using Corneal Surface Reflection and Omnidirectional Camera Image. 査読

Taishi Ogawa, Atsushi Nakazawa, Toyoaki Nishida

IEICE Trans. Inf. Syst. 101-D ( 5 ) 1278 - 1287 2018年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Institute of Electronics, Information and Communication, Engineers, IEICE

We present a human point of gaze estimation system using corneal surface reflection and omnidirectional image taken by spherical panorama cameras, which becomes popular recent years. Our system enables to find where a user is looking at only from an eye image in a 360° surrounding scene image, thus, does not need gaze mapping from partial scene images to a whole scene image that are necessary in conventional eye gaze tracking system. We first generate multiple perspective scene images from an omnidirectional (equirectangular) image and perform registration between the corneal reflection and perspective images using a corneal reflection-scene image registration technique. We then compute the point of gaze using a corneal imaging technique leveraged by a 3D eye model, and project the point to an omnidirectional image. The 3D eye pose is estimate by using the particle-filter-based tracking algorithm. In experiments, we evaluated the accuracy of the 3D eye pose estimation, robustness of registration and accuracy of PoG estimations using two indoor and five outdoor scenes, and found that gaze mapping error was 5.546 [deg] on average.

DOI： 10.1587/transinf.2017MVP0020

その他リンク： https://dblp.uni-trier.de/db/journals/ieicet/ieicet101d.html#OgawaNN18

PRMU応用研究におけるオープンアイデア～ PRMU第二期グランドチャレンジ～招待

中澤篤志, 山崎俊彦, 松下康之, 安倍満, 舩冨卓哉, 木村昭悟, 内田誠一, 前田英作

電子情報通信学会技術研究報告 117 ( 362 ) 41 - 43 2017年12月

深層学習を用いた自己撮影画像の撮影位置検索

江川佳輝, 小川太士, 中澤篤志

電子情報通信学会技術研究報告 117 ( 392 ) 333 - 337 2017年12月

DEEP eye contact detector: Robust eye contact bid detection using convolutional neural network 査読

Mitsuzumi Y, Nakazawa A, Nishida T

British Machine Vision Conference (BMVC2017) 2017年9月

Noninvasive Corneal Image-Based Gaze Measurement System 査読国際共著

Eunji Chong, Christian Nitschke, Atsushi Nakazawa, Agata Rozga, James M Rehg

arXiv preprint arXiv:1708.00908 1 1 - 8 2017年8月

Point of gaze estimation using corneal surface reflection and omnidirectional camera image 査読

Taishi Ogawa, Atsushi Nakazawa, Toyoaki Nishida

Proceedings of the 15th IAPR International Conference on Machine Vision Applications, MVA 2017 227 - 230 2017年7月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Institute of Electrical and Electronics Engineers Inc.

We present a human point of gaze estimation system using corneal surface reflection and omni-directional image taken by a fish eye. Only capturing an eye image, our system entables to find where a user is looking in 360P surrounding scene image. We first generates multiple perspective scene images from an equi-rectangular image and perform registration between corneal reflection and perspective images. We then compute the point of gaze using a 3D eye model and project the point to an omni-directional image. We evaluated the robustness of registration and accuracy of PoG estimations using two indoor and five outdoor scenes, and found that gaze mapping error was 5.526[deg] on average. This result shows the potential to the marketing and outdoor training system.

DOI： 10.23919/MVA.2017.7986842

Corneal Imaging

Christian Nitschke, Atsushi Nakazawa

The Wiley Handbook of Human Computer Interaction Set 1 445 - 514 2017年6月

記述言語：英語掲載種別：論文集(書籍)内論文出版者・発行元：Wiley Blackwell

The cornea of the human eye acts as a mirror that reflects light from a person's environment. These corneal reflections can be extracted from an image of the eye by modeling the eye-camera geometry as a catadioptric imaging system. As a result, one obtains the visual information of the environment and the relation to the observer (view, gaze), which allows for applications in a number of fields, most notably eye tracking. The recovered illumination map can be further applied to various computational tasks. This chapter provides a comprehensive introduction and overview on corneal imaging and applications. It includes (1) an explanation of anatomic structures and geometric modeling of the eye, (2) eye tracking and pose estimation from eye images, (3) corneal reflection modeling and image processing, (4) applications, and (5) a discussion of implications and promising future directions. While mainly targeting readers in eye image processing and eye tracking, this chapter is intended for a diverse audience with different backgrounds, levels of experience, and theoretical and practical interest.

DOI： 10.1002/9781118976005.ch21

医療コミュニケーションにおける人工知能の可能性医師の「五感」をも定量化できる時代へ招待

中澤篤志

総合診療 27 ( 5 ) 621 - 623 2017年5月

Eye gaze tracking using corneal imaging and active illumination devices. 査読

Atsushi Nakazawa, Hiroaki Kato, Christian Nitschke, Toyoaki Nishida

Adv. Robotics 31 ( 8 ) 413 - 427 2017年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：TAYLOR & FRANCIS LTD

This paper shows a novel eye gaze tracking (EGT) technique using the corneal imaging technique. Compared to the existing pupil center and pupil reflection techniques, our approach directly finds PoG in the reflected scene image at the human corneal surface. As a result, it does not suffer from the parallax issue and does not require per-setup system calibrations. To achieve this system, we develop following techniques: First, we use the idea of the gaze-reflection point (GRP), where light from the PoG in the scene reflects at the corneal surface into an eye image. Second, illuminating the whole scene or particular objects using coded structured light enables robust and accurate matching at the GRP to obtain the PoG in a scene image. For this purpose, we show two implementations: a special high-power IR LED-array projector and active LED markers. Experimental evaluation shows that the proposed scheme achieves considerable accuracy and successfully supports depth-varying environments as well as practical applications including observation in a conversation scene. We believe the proposed EGT technique has considerably large potential to solve major issues of current EGT systems, expands the application fields of the EGT and increases the usability of interactive systems.

DOI： 10.1080/01691864.2016.1277552

その他リンク： https://dblp.uni-trier.de/db/journals/ar/ar31.html#NakazawaKNN17

Toward autonomous collision avoidance for robotic neurosurgery in deep and narrow spaces in the brain

Hiroaki Ueda, Ryoya Suzuki, Atsushi Nakazawa, Yusuke Kurose, Murilo M. Marinho, Naoyuki Shono, Hirofumi Nakatomi, Nobuhito Saito, Eiju Watanabe, Akio Morita, Kanako Harada, Naohiko Sugita, Mamoru Mitsuishi

3RD CIRP CONFERENCE ON BIOMANUFACTURING 65 110 - 114 2017年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：ELSEVIER SCIENCE BV

The present authors have been developing a master-slave neurosurgical robot and its intelligent control for tasks in the deep and narrow spaces of the brain. This paper proposes a robotic autonomous control method for avoiding possible collisions between the shaft of a surgical robotic instrument and the surrounding tissues. To this end, a new robotic simulator was developed and used to evaluate the proposed method. The results showed the proof of concept of the proposed autonomous collision avoidance, which has the potential to enhance the safety of robotic neurosurgery in deep and narrow spaces. (C) 2016 The Authors. Published by Elsevier B.V.

DOI： 10.1016/j.procir.2017.04.027

Registration of eye reflection and scene images using an aspherical eye model 査読

Atsushi Nakazawa, Christian Nitschke, Toyoaki Nishida

JOURNAL OF THE OPTICAL SOCIETY OF AMERICA A-OPTICS IMAGE SCIENCE AND VISION 33 ( 11 ) 2264 - 2276 2016年11月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：OPTICAL SOC AMER

This paper introduces an image registration algorithm between an eye reflection and a scene image. Although there are currently a large number of image registration algorithms, this task remains difficult due to nonlinear distortions at the eye surface and large amounts of noise, such as iris texture, eyelids, eyelashes, and their shadows. To overcome this issue, we developed an image registration method combining an aspherical eye model that simulates nonlinear distortions considering eye geometry and a two-step iterative registration strategy that obtains dense correspondence of the feature points to achieve accurate image registrations for the entire image region. We obtained a database of eye reflection and scene images featuring four subjects in indoor and outdoor scenes and compared the registration performance with different asphericity conditions. Results showed that the proposed approach can perform accurate registration with an average accuracy of 1.05 deg by using the aspherical cornea model. This work is relevant for eye image analysis in general, enabling novel applications and scenarios. (C) 2016 Optical Society of America

DOI： 10.1364/JOSAA.33.002264

Feedback methods for collision avoidance using virtual fixtures for robotic neurosurgery in deep and narrow spaces

Atsushi Nakazawa, Kodai Nanri, Kanako Harada, Shinichi Tanaka, Hiroshi Nukariya, Yusuke Kurose, Naoyuki Shono, Hirohumi Nakatomi, Akio Morita, Eiju Watanabe, Naohiko Sugita, Mamoru Mitsuishi

Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics 2016-July 247 - 252 2016年7月

掲載種別：研究論文（国際会議プロシーディングス）

Robotic assistance enables a surgeon to perform dexterous and precise manipulations. However, conducting robot assisted neurosurgery within the deep and narrow spaces of the brain presents the risk of unexpected collisions between the shafts of robotic instruments and their surroundings out of the microscopic view. Thus, we propose the provision of feedback using a truncated cone shaped virtual fixture generated by marking the edges of the top and bottom plane of a workspace in the deep and narrow spaces within the brain with the slave manipulator. The experimental results show that the virtual fixture generation method could precisely model the workspace. We also implemented force feedback, visual feedback, and motion scaling feedback in the microsurgical robotic system in order to inform the surgeon of the risk of collision. Performance of each feedback method and their combinations was evaluated in two experiments. The experimental results showed that the combination of the force and the visual feedback methods were the most beneficial for avoiding collisions.

DOI： 10.1109/BIOROB.2016.7523632

Synthetic Evidential Study as Augmented Collective Thought Process — Preliminary Report 査読

Toyoaki Nishida, Masakazu Abe, Takashi Ookaki, Divesh Lala, Sutasinee Thovuttikul, Hengjie Song, Yasser Mohammad, Christian Nitschke, Yoshimasa Ohmoto, Atsushi Nakazawa, Takaaki Shochi, Jean-Luc Rouas, Aurelie Bugeau, Fabien Lotte, Ming Zuheng, Geoffrey Letournel, Marine Guerry, Dominique Fourer

7th Asian Conference on Intelligent Informationo and Database systems (ACIIDS) 13 - 22 2015年3月

Corneal imaging revisited: An overview of corneal reflection analysis and applications 査読

Christian Nitschke, Atsushi Nakazawa, Haruo Takemura

IPSJ Transactions on Computer Vision and Applications 5 1 - 18 2013年7月

記述言語：英語

The cornea of the human eye acts as a mirror that reflects light from a person's environment. These corneal reflections can be extracted from an image of the eye by modeling the eye-camera geometry as a catadioptric imaging system. As a result, one obtains the visual information of the environment and the relation to the observer (view, gaze), which allows for application in a number of fields. The recovered illumination map can be further applied to various computational tasks. This paper provides a comprehensive introduction on corneal imaging, and aims to show the potential of the topic and encourage advancement. It makes a number of contributions, including (1) a combined view on previously unrelated fields, (2) an overview of recent developments, (3) a detailed explanation on anatomic structures, geometric eye and corneal reflection modeling including multiple eye images, (4) a summary of our work and contributions to the field, and (5) a discussion of implications and promising future directions. The idea behind this paper is a geometric framework to solve persisting technical problems and enable non-intrusive interfaces and smart sensors for traditional, ubiquitous and ambient environments. © 2013 Information Processing Society of Japan.

DOI： 10.2197/ipsjtcva.5.1

Corneal Imaging Revisited: An Overview of Corneal Reflection Analysis and Applications

Nitschke Christian, Nakazawa Atsushi, Takemura Haruo

Information and Media Technologies 8 ( 2 ) 389 - 406 2013年

記述言語：英語出版者・発行元：Information and Media Technologies Editorial Board

The cornea of the human eye acts as a mirror that reflects light from a person's environment. These corneal reflections can be extracted from an image of the eye by modeling the eye-camera geometry as a catadioptric imaging system. As a result, one obtains the visual information of the environment and the relation to the observer (view, gaze), which allows for application in a number of fields. The recovered illumination map can be further applied to various computational tasks. This paper provides a comprehensive introduction on corneal imaging, and aims to show the potential of the topic and encourage advancement. It makes a number of contributions, including (1) a combined view on previously unrelated fields, (2) an overview of recent developments, (3) a detailed explanation on anatomic structures, geometric eye and corneal reflection modeling including multiple eye images, (4) a summary of our work and contributions to the field, and (5) a discussion of implications and promising future directions. The idea behind this paper is a geometric framework to solve persisting technical problems and enable non-intrusive interfaces and smart sensors for traditional, ubiquitous and ambient environments.

DOI： 10.11185/imt.8.389

Motion Coherent Tracking Using Multi-label MRF Optimization 査読

David Tsai, Matthew Flagg, Atsushi Nakazawa, James M. Rehg

INTERNATIONAL JOURNAL OF COMPUTER VISION 100 ( 2 ) 190 - 202 2012年11月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：SPRINGER

We present a novel off-line algorithm for target segmentation and tracking in video. In our approach, video data is represented by a multi-label Markov Random Field model, and segmentation is accomplished by finding the minimum energy label assignment. We propose a novel energy formulation which incorporates both segmentation and motion estimation in a single framework. Our energy functions enforce motion coherence both within and across frames. We utilize state-of-the-art methods to efficiently optimize over a large number of discrete labels. In addition, we introduce a new ground-truth dataset, called Georgia Tech Segmentation and Tracking Dataset (GT-SegTrack), for the evaluation of segmentation accuracy in video tracking. We compare our method with several recent on-line tracking algorithms and provide quantitative and qualitative performance comparisons.

DOI： 10.1007/s11263-011-0512-5

Point of Gaze Estimation through Corneal Surface Reflection in an Active Illumination Environment 査読

Atsushi Nakazawa, Christian Nitschke

COMPUTER VISION - ECCV 2012, PT II 7573 ( PART 2 ) 159 - 172 2012年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：SPRINGER-VERLAG BERLIN

Eye gaze tracking (EGT) is a common problem with many applications in various fields. While recent methods have achieved improvements in accuracy and usability, current techniques still share several limitations. A major issue is the need for external calibration between the gaze camera system and the scene, which commonly restricts to static planar surfaces and leads to parallax errors. To overcome these issues, the paper proposes a novel scheme that uses the corneal imaging technique to directly analyze reflections from a scene illuminated with structured light. This comprises two major contributions: First, an analytic solution is developed for the forward projection problem to obtain the gaze reflection point (GRP), where light from the point of gaze (PoG) in the scene reflects at the corneal surface into an eye image. We also develop a method to compensate for the individual offset between the optical axis and true visual axis. Second, introducing active coded illumination enables robust and accurate matching at the GRP to obtain the PoG in a scene image, which is the first use of this technique in EGT and corneal reflection analysis. For this purpose, we designed a special high-power IR LED-array projector. Experimental evaluation with a prototype system shows that the proposed scheme achieves considerable accuracy and successfully supports depth-varying environments.

DOI： 10.1007/978-3-642-33709-3_12

Display-camera calibration using eye reflections and geometry constraints 査読

Christian Nitschke, Atsushi Nakazawa, Haruo Takemura

COMPUTER VISION AND IMAGE UNDERSTANDING 115 ( 6 ) 835 - 853 2011年6月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：ACADEMIC PRESS INC ELSEVIER SCIENCE

In this paper, we describe a novel method for calibrating display-camera setups from reflections in a user's eyes. Combining both devices creates a capable controlled illumination system that enables a range of interesting vision applications in non-professional environments, including object/face reconstruction and human computer interaction. One major issue barring such systems from average homes is the geometric calibration to obtain the pose of the display which requires special hardware and tedious user interaction. Our proposed approach eliminates this requirement by introducing the novel idea of analyzing screen reflections in the cornea of the human eye, a mirroring device that is always available. We employ a simple shape model to recover pose and reflection characteristics of the eye. Thorough experimental evaluation shows that the basic strategy results in a large error and discusses possible reasons. Based on the findings, a non-linear optimization strategy is developed that exploits geometry constraints within the system to considerably improve the initial estimate. It further allows to automatically resolve an inherent ambiguity that arises in image-based eye pose estimation. The strategy may also be integrated to improve spherical mirror calibration. We describe several comprehensive experimental studies which show that the proposed method performs stably with respect to varying subjects, display poses, eye positions, and gaze directions. The results are feasible and should be sufficient for many applications. In addition, the findings provide general insight on the application of eye reflections for geometric reconstruction. (C) 2011 Elsevier Inc. All rights reserved.

DOI： 10.1016/j.cviu.2011.02.008

サリエンシーディテクション

中澤篤志

映像情報メディア学会誌 : 映像情報メディア = The journal of the Institute of Image Information and Television Engineers 64 ( 12 ) 1830 - 1832 2010年12月

Motion capture systems 査読

Atsushi Nakazawa

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 63 ( 9 ) 1224 - 1227 2009年

Task recognition and person identification in cyclic dance sequences with Multi Factor Tensor analysis 査読

Manoj Perera, Takaaki Shiratori, Shunsuke Kudoh, Atsushi Nakazawa, Katsushi Ikeuchi

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E91D ( 5 ) 1531 - 1542 2008年5月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG

In this paper, we present a novel approach to recognize motion styles and identify people using the Multi Factor Tensor (MFT) model. We apply a musical information analysis method in segmenting the motion sequence relevant to the keyposes and the musical rhythm. We define a task model by considering the repeated motion segments, where the motion is decomposed into a person-invariant factor task and a person-dependent factor style. Given the motion data set, we formulate the MFT model, factorize it efficiently in different modes, and use it in recognizing the tasks and the identities of the persons performing the tasks. We capture the motion data of different people for a few cycles, segment it using the musical analysis approach, normalize the segments using a vectorization method, and realize our MFT model. In our experiments, Japanese traditional dance sequences performed by several people are used. Provided with an unknown motion segment which is to be probed and which was performed at a different time in the time space, we first normalize the motion segment and flatten our MFT model appropriately, then recognize the task and the identity of the person. We follow two approaches in conducting our experiments. In one experiment, we recognize the tasks and the styles by maximizing a function in the tensor subdomain, and in the next experiment, we use a function value in the tensorial subdomain with a threshold for recognition. Interestingly, unlike the first experiment, we are capable of recognizing tasks and human identities that were not known beforehand. We conducted various experiments to evaluate the potential of the recognition ability of our proposed approaches, and the results demonstrate the high accuracy of our model.

DOI： 10.1093/ietisy/e91-d.5.1531

Diskless Linux を用いた情報教育システムの開発とその評価査読

桝田秀夫, 小川剛史, 町田貴史, 中澤篤志, 清川清, 竹村治雄

情報処理学会論文誌 49 ( 3 ) 1239 - 1248 2008年3月

記述言語：日本語出版者・発行元：一般社団法人情報処理学会

教育用計算機システムでは，多数の利用者に対して各種のアプリケーションが動作する一様な計算機環境を，できるだけ低いTCO （Total Cost of Ownership）で実現する必要がある．運用コスト削減のためには，故障時に素早い復旧が可能であることだけでなく，そもそも故障しにくい構成をとることも重要である．本論文では，これまでの運用の中で最も故障の多かったハードディスクをクライアント上では使用しない，ディスクレス構成を用いたLinux ベースの情報教育システムの開発とその評価について述べる．また，予算措置の関係で導入時期や機種が異なる複数種類のパソコンを，一括して運用管理する必要性に対応するための構成上の工夫についても述べる．さらに，Microsoft Office で作られた文書の激増に対して，互換性のあるOpenOffice.org やStarSuite だけではなく，OS としてのMicrosoft Windows を運用せずに，Microsoft Office 自体をCrossOver Office というWINE ベースのエミュレータ上で稼働させることに挑戦している．これらの構成の設計・開発の経緯，ならびに導入後約2 年間の運用実績と得られた知見について報告する．This paper shows the configurations and evaluations of the educational computer system in our university that serves variety types of applications for many numbers of users, reducing Total Cost of Ownership (TCO). As for client operating systems, we developed newly designed diskless Linux. The OS and other applications are loaded from servers through networks; therefore the clients don't have hard drives. This scheme drastically reduces TCO because it is not necessary to replace hard drives which fail most frequently in these types of systems. Also, the software updates of the clients are achieved just by uploading the files in few servers. This also reduces the maintenance cost rather than updating the software stored in local hard drives of many clients. As for the client application software, we installed not only OpenOffice.org or StarSuite, but also Microsoft Office combined with Crossover Office emulator that is highly demanded by end users.

Iterative refinement of range images with anisotropic error distribution 査読

Ryusuke Sagawa, Takeshi Oishi, Atsushi Nakazawa, Ryo Kurazume, Katsushi Ikeuchi

Digitally Archiving Cultural Objects 193 - 205 2008年

記述言語：英語掲載種別：論文集(書籍)内論文出版者・発行元：Springer US

We propose a method which refines the range measurement of range finders by computing correspondences of vertices of multiple range images acquired from various viewpoints. Our method assumes that a range image acquired by a laser range finder has anisotropic error distribution which is parallel to the ray direction. Thus, we find corresponding points of range images along with the ray direction. We iteratively converge range images to minimize the distance of corresponding points. We describe the effectiveness of our method by the presenting the experimental results of artificial and real range data. Also we show that our method refines a 3D shape more accurately as opposed to that achieved by using the Gaussian filter. © 2008 Springer-Verlag US.

DOI： 10.1007/978-0-387-75807_11

A fast simultaneous alignment of multiple range images 査読

Takeshi Oishi, Atsushi Nakazawa, Ryo Kurazume, Katsushi Ikeuchi

Digitally Archiving Cultural Objects 89 - 107 2008年

記述言語：英語掲載種別：論文集(書籍)内論文出版者・発行元：Springer US

This chapter describes a fast, simultaneous alignment method for a large number of range images. Generally the most time-consuming task in aligning range images is searching corresponding points. The fastest searching method is the "Inverse Calibration" method. However, this method requires pre-computed lookup tables and precise sensor parameters. We propose a fast searching method using "index images", which work as look-up tables and are rapidly created without any sensor parameters by using graphics hardware. To accelerate the computation to estimate rigid transformations, we employed a linear error evaluation method. When the number of range images increases, the computation time for solving the linear equations becomes too long because of the large size of the coefficient matrix. On the other hand, the coefficient matrix has the characteristic of becoming sparser as the number of range images increases. Thus, we applied the Incomplete Cholesky Conjugate Gradient (ICCG) method to solve the equations and found that the ICCG greatly accelerates the matrix operation by pre-conditioning the coefficient matrix. Some experimental results in which a large number of range images are aligned demonstrate the effectiveness of our method. © 2008 Springer-Verlag US.

DOI： 10.1007/978-0-387-75807_6

Breed differentiation among Japanese native chickens by specific skull features determined by direct measurements and computer vision techniques 査読

Y. Ino, T. Oka, K. Nomura, T. Watanabe, S. Kawashima, T. Amano, Y. Hayashi, A. Okabe, Y. Uehara, T. Masuda, J. Takamatsu, A. Nakazawa, K. Ikeuchi, H. Endo, K. Fukuta, F. Akishinonomiya

BRITISH POULTRY SCIENCE 49 ( 3 ) 273 - 281 2008年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：TAYLOR & FRANCIS LTD

1. Inter-breed morphological comparisons were made among 11 breeds of Japanese native chickens (Gifujidori, Hinaidori, Shokoku, Totenko, Tomaru, Satsumadori, Shamo, Koshamo, Koeyoshi, Chabo and Nagoya), White Leghorn, broiler chickens (Chunky) and red junglefowl collected in the Philippines, based on results of direct measurements and analysis by computer vision techniques of the skull.
2. Analysis of direct measurements identified two groups of chicken: a small type that included the Chabo, Koshamo, red junglefowl, Gifujidori and Shokoku and a large type that included the remaining breeds studied. These groupings were made based on size determined both in the first (PC1) and second principal component (PC2). The greatest length of the cranium and condylobasal length greatly contributed to the morphological differences between these two groups.
3. Analysis by computer vision techniques, however, identified three groups: the Bantam group (which includes red junglefowl), Shokoku group and Shamo group. White Leghorn clustered within the Shokoku group while the broiler chicken belonged to the Shamo group. The region around the junction of the neural cranium and the visceral cranium contributed greatly to the morphological differences among breeds, both in the PC1 and PC2.

DOI： 10.1080/00071660802094727

Parallel alignment of a large number of range images 査読

Takeshi Oishi, Atsushi Nakazawa, Ryo Kurazume, Katsushi Ikeuchi, Ryusuke Sagawa

Digitally Archiving Cultural Objects 109 - 126 2008年

記述言語：英語掲載種別：論文集(書籍)内論文出版者・発行元：Springer US

This chapter describes a method for parallel alignment of multiple range images. There are problems of computational time and memory space in aligning a large number of range images simultaneously. We developed a parallel method to address the problems. Searching for corresponding points between two range images is time-consuming and requires considerable memory space when performed independently. However, this process can be preformed in parallel, with each corresponding pair of range images assigned to a node. Because the computation time is approximately proportional to the number of vertices, by assigning the pairs so that the number of vertices computed is equal on each node, the load on each node is effectively distributed. In order to reduce the amount of memory required on each node, a hypergraph that represents the correspondences of range images is created, and heuristic graph partitioning algorithms are applied to determine the optimal assignment of the pairs. Moreover, by rejecting redundant dependencies, it becomes possible to accelerate computation time and reduce the amount of memory required on each node. The method was tested on a 16-processor PC cluster, where it demonstrated high extendibility and improved performance. © 2008 Springer-Verlag US.

DOI： 10.1007/978-0-387-75807_7

The Great Buddha project: Digitally archiving, restoring, and analyzing cultural heritage objects 査読

Katsushi Ikeuchi, Takeshi Oishi, Jun Takamatsu, Ryusuke Sagawa, Atsushi Nakazawa, Ryo Kurazume, Ko Nishino, Mawo Kamakura, Yasuhide Okamoto

INTERNATIONAL JOURNAL OF COMPUTER VISION 75 ( 1 ) 189 - 208 2007年10月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：SPRINGER

This paper presents an overview of our research project on digital preservation of cultural heritage objects and digital restoration of the original appearance of these objects. As an example of these objects, this project focuses on the preservation and restoration of the Great Buddhas. These are relatively large objects existing outdoors and providing various technical challenges. Geometric models of the great Buddhas are digitally achieved through a pipeline, consisting of acquiring data, aligning multiple range images, and merging these images. We have developed two alignment algorithms: a rapid simultaneous algorithm, based on graphics hardware, for quick data checking on site, and a parallel alignment algorithm, based on a PC cluster, for precise adjustment at the university. We have also designed a parallel voxel-based merging algorithm for connecting all aligned range images. On the geometric models created, we aligned texture images acquired from color cameras. We also developed two texture mapping methods. In an attempt to restore the original appearance of historical objects, we have synthesized several buildings and statues using scanned data and a literature survey with advice from experts.

DOI： 10.1007/s11263-007-0039-y

Learning from observation paradigm: Leg task models for enabling a biped humanoid robot to imitate human dances 査読

Shin'ichiro Nakaoka, Atsushi Nakazawa, Fumio Kanehiro, Kenji Kaneko, Mitsuharu Morisawa, Hirohisa Hirukawa, Katsushi Ikeuchi

INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH 26 ( 8 ) 829 - 844 2007年8月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：SAGE PUBLICATIONS LTD

This paper proposes a framework that achieves the Learning from Observation paradigm for learning dance motions. The framework enables a humanoid robot to imitate dance motions captured from human demonstrations. This study especially focuses on leg motions to achieve a novel attempt in which a biped-ope robot imitates not only upper body motions but also leg motions including steps. Body differences between the robot and the original dancer make the problem difficult because the differences prevent the robot front straight forward-v following the original motions and they also change dynamic body balance. We propose leg task models, which play a key role in solving the problem. Low-level tasks in leg motion are modelled so that they clearly provide essential information required,for keeping dynamic stability and important motion characteristics. The models divide the problem of adapting motions into the problem of recognizing a sequence of the tasks and the problem of executing the task sequence. We have developed a method for recognizing the tasks from captured motion data and a method for generating the motions of the tasks that can be executed by existing robots including HRP-2. HRP-2 successfully performed the generated motions, which imitated a traditional folk dance performed by human dancers.

DOI： 10.1177/0278364907079430

Real-time space carving using graphics hardware 査読

Christian Nitschke, Atsushi Nakazawa, Haruo Takemura

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E90D ( 8 ) 1175 - 1184 2007年8月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG

Reconstruction of real-world scenes from a set of multiple images is a topic in computer vision and 3D computer graphics with many interesting applications. Attempts have been made to real-time reconstruction on PC cluster systems. While these provide enough performance, they are expensive and less flexible. Approaches that use a GPU hardware-acceleration on single workstations achieve real-time framerates for novel-view synthesis, but do not provide an explicit volumetric representation. This work shows our efforts in developing a GPU hardware-accelerated framework for providing a photo-consistent reconstruction of a dynamic 3D scene. High performance is achieved by employing a shape from silhouette technique in advance. Since the entire processing is done on a single PC, the framework can be applied in mobile environments, enabling a wide range of further applications. We explain our approach using programmable vertex and fragment processors and compare it to highly optimized CPU implementations. We show that the new approach can outperform the latter by more than one magnitude and give an outlook for interesting future enhancements.

DOI： 10.1093/ietisy/e90-d.8.1175

Human pose estimation from volume data and topological graph database 査読

Hidenori Tanaka, Atsushi Nakazawa, Haruo Takemura

COMPUTER VISION - ACCV 2007, PT I, PROCEEDINGS 4843 ( PART 1 ) 618 - + 2007年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：SPRINGER-VERLAG BERLIN

This paper proposes a novel volume-based motion capture method using a bottom-up analysis of volume data and an example topology database of the human body. By using a two-step graph matching algorithm with many example topological graphs corresponding to postures that a human body can take, the proposed method does not require any initial parameters or iterative convergence processes, and it can solve the changing topology problem of the human body. First, three-dimensional curved lines (skeleton) are extracted from the captured volume data using the thinning process. The skeleton is then converted into an attributed graph. By using a graph matching algorithm with a large amount of example data, we can identify the body parts from each curved line in the skeleton. The proposed method is evaluated using several video sequences of a single person and multiple people, and we can confirm the validity of our approach.

階層化3次元データを扱う対話的なコンテンツマネジメントシステム

木村洋介, 間下以大, 中澤篤志, 町田貴史, 清川清, 竹村治雄

ヒューマンインタフェースシンポジウム 2006年9月

記述言語：日本語掲載種別：研究論文（その他学術会議資料等）

本研究では、三次元データを扱うコンテンツマネジメントシステム(CMS)を提案する。システムに三次元データ等のコンテンツを登録し、それらを用いたウェブサイトの構築を可能とする。三次元データは単にHTMLファイルに組み込むだけでなく、他のコンテンツとの表示の連携を定義することが可能であり、ユーザによりわかりやすく三次元データを提示することが出来る。三次元データの描画にはQSplatをJavaアプレット化したシステムを用いる。これにより、端末の処理速度に合わせた解像度で、また端末のプラットフォームに依存せずに描画することが可能である。

モバイル端末向け注釈付加機能付き大規模三次元データ高速描画用Javaアプレット

木村洋介, 中澤篤志, 町田貴史, 清川清, 竹村治雄

情報処理学会マルチメディア，分散，協調とモバイルシンポジウム(DICOMO)論文集 2006年7月

記述言語：日本語掲載種別：研究論文（その他学術会議資料等）

近年，モバイル端末の普及に伴って，インターネットなどのネットワーク上でデータを閲覧する機会が増えている．しかし，モバイル環境でネットワーク上の大規模三次元データを閲覧する際には，通信速度及び描画能力が制限される，プラットフォームが統一されていないなどの問題がある．そこで，本研究では，大規模三次元データの高速描画方式であるQSplatをJavaアプレットを用いて実装することで，大規模な三次元データを端末の表示環境に合わせて高速に描画可能な，マルチプラットフォームシステムを提案する．これにより，システムの実行にあたってウェブブラウザを利用することができるため，専用アプリケーションが不要となる．また，本システムを他のシステムへ統合することも容易になる．さらに，ユーザの三次元データへの理解を補助するため，三次元データに対して階層的な意味構造を持つ注釈情報を付加し，画像と共に表示する機能を実装した．

脚タスクモデルを用いた2足歩行ヒューマノイドロボットによる人の舞踊動作の再現

中岡慎一郎, 中澤篤志, 金広文男, 金子健二, 森澤光晴, 比留川博久, 池内克史

日本ロボット学会誌 = Journal of Robotics Society of Japan 24 ( 3 ) 388 - 399 2006年4月

記述言語：日本語出版者・発行元：The Robotics Society of Japan

In this paper, we propose a method that enables a biped humanoid robot to reproduce human dance motions with its whole body. Our method is based on the paradigm ofLearning from Observation. In this study, a robot uses its own legs to support the body during a dance performance. We proposeleg task models, which can solve the problems caused by severe constraints in adapting human motions to the legs of a robot. First, elements of the leg task models are recognized from motion data captured from human performances. Then motion data of a robot is regenerated from the recognized elements so that the motion is stably executable on the robot. Our method was verified by experiments on a humanoid robotHRP-2using a traditional folk dance. HRP-2 successfully performed dance motions that were automatically reproduced from motion data captured from human dance performances.

DOI： 10.7210/jrsj.24.388

その他リンク： https://jlc.jst.go.jp/DN/JALC/00278397673?from=CiNii

モバイル端末用大規模三次元データ提示手法のJavaアプレットによる実装

木村洋介, 中澤篤志, 町田貴史, 清川清, 竹村治雄

電子情報通信学会総合大会講演論文集 2006年3月

移動ロボットによる災害地の三次元再構築のためのポテンシャル場を用いた経路計画

河合克哉, 中澤篤志, 清川清, 竹村治雄

電子情報通信学会技術研究報告 2006年2月

Hierarchical 3D data rendering system synchronizing with HTML 査読

Yousuke Kimura, Tomohiro Mashita, Atsushi Nakazawa, Takashi Machida, Kiyoshi Kiyokawa, Haruo Tamekura

ADVANCES IN ARTIFICIAL REALITY AND TELE-EXISTENCE, PROCEEDINGS 4282 1212 - + 2006年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：SPRINGER-VERLAG BERLIN

We propose a new rendering system for large-scale, 3D geometic data that can be used with web-based content management systems (CMS). To achieve this, we employed a geometry hierarchical encoding method "QSplat" and implemented this in a Java and JOGL (Java bindings of OpenGL) environment. Users can view large-scale geometric data using conventional HTML browsers with a non-powerful CPU and low-speed networks. Further, this system is independent of the platforms. We add new functionalities so that users can easily understand the geometric data: Annotations and HTML Synchronization. Users can see the geometric data with the associated annotations that describe the names or the detailed explanations of the particular portions. The HTML Synchronization enables users to smoothly and interactively switch our rendering system and HTML contents. The experimental results show that our system performs an interactive frame rate even for a large-scale data whereas other systems cannot render them.

DOI： 10.1007/11941354_126

Dancing-to-music character animation 査読

Takaaki Shiratori, Atsushi Nakazawa, Katsushi Ikeuchi

COMPUTER GRAPHICS FORUM 25 ( 3 ) 449 - 458 2006年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：WILEY-BLACKWELL

In computer graphics, considerable research has been conducted on realistic human motion synthesis. However most research does not consider human emotional aspects, which often strongly affect human motion. This paper presents a new approach for synthesizing dance performance matched to input music, based on the emotional aspects of dance performance. Our method consists of a motion analysis, a music analysis, and a motion synthesis based on the extracted features. In the analysis steps, motion and music feature vectors are acquired. Motion vectors are derived from motion rhythm and intensity, while music vectors are derived from musical rhythm, structure, and intensity. For synthesizing dance performance, we first find candidate motion segments whose rhythm features are matched to those of each music segment, and then we find the motion segment set whose intensity is similar to that of music segments. Additionally, our system supports having animators control the synthesis process by assigning desired motion segments to the specified music segments. The experimental results indicate that our method actually creates dance performance as if a character was listening and expressively dancing to the music.

DOI： 10.1111/j.1467-8659.2006.00964.x

災害地の3次元再構築のための移動ロボット制御システムの開発

河合克哉, 中澤篤志, 清川清, 竹村治雄

電子情報通信学会技術研究報告, MVE2005-29 105 ( 256 ) 13 - 18 2005年9月

記述言語：日本語掲載種別：研究論文（その他学術会議資料等）出版者・発行元：一般社団法人電子情報通信学会

災害時, 被害状況の確認や救助活動を行う専門家が現地に直行できない場合は, 遠隔地からその状況を把握することが必要である.災害地における危険な場所や狭い場所の環境情報を取得するための方法として移動ロボットがある.この際に, 画像に加えて3次元形状を計測することが有用であると考えられる.本研究では, 全方位レンジセンサと全方位カメラ及び姿勢センサを搭載した移動ロボットを用いて自己位置の推定と形状情報の取得を行うシステムを開発する.全方位レンジセンサはライン型のレーザスキャナと回転ステージによって実現する.自己位置の推定には車輪の回転量と姿勢センサによる計測値を用いる.また, 複数の位置で取得された形状データに対して, ICPアルゴリズムによる位置合わせを行い, 自己位置推定の補正値としてフィードバックする.試作したシステムを用いて実際に動作実験を行い, 自己位置推定精度の評価を行った.

移動ロボットを用いた3次元再構築のための計測・提示システムの開発

河合克哉, 齋藤研作, 町田貴史, 中澤篤志, 清川清, 竹村治雄

画像の認識・理解シンポジウム（MIRU）講演論文集 2005年7月

距離画像からの自然二次曲面表面群のパラメータ推定査読

中澤篤志, 清川清, 竹村治雄, 山本国士

画像の認識と理解シンポジウム 2005年7月

ハフ変換と期待値最大化法による距離画像からの自然二次曲面の抽出

山本国士, 中澤篤志, 清川清, 竹村治雄

電子情報通信学会技術報告PRMU2004-188 2005年2月

多関節 CG モデルと距離画像による上半身の姿勢推定

平尾公男, 中澤篤志, 清川清, 竹村治雄

電子情報通信学会技術研究報告 2005年1月

A Novel Osteometrical Method Using Computer Vision Techniques for Comparison of Morphological Differences

高松淳, 上原康彦, 増田智仁, 中澤篤志, 池内克史, 岡部篤行, 林良博, 井野靖子, 岡孝夫, 野村こう, 天野卓, 秋篠宮文仁

山階鳥類学雑誌 36 ( 2 ) 120 - 128 2005年

記述言語：英語出版者・発行元：Yamashina Institute for Ornitology

骨計測法は,骨格標本の形状の類似度から種間差を推定する系統分類学の手法として用いられてきた。しかしながら,従来の骨計測法は,実際には骨格の全形状ではなく,標本上の特徴的な2点の間の距離の測定にすぎず,結果として,骨格の大きさといった,わずかな特徴しか推定に利用できなかった。<br>そこで本論文は,レーザスキャナと様々なコンピュータビジョンの技術を組み合わせて精密な3次元形状計測技術を頼りに,鶏頭骨の2次元輪郭形状を用いた新たな骨計測法を提案する。これにより,本手法では,従来法と異なり,より多くの特徴を用いて種間差を推定することが可能となり,また貴重な標本を非接触,非破壊に繰り返し測定することができる。実際に,5品種24個体(岐阜地鶏5,シャモ5,薩摩鶏5,小シャモ4,白色レグホーン5)に対して分析を行った結果,提案手法の有用性が示された。

DOI： 10.3312/jyio.36.120

ユビキスタVR 学習システムおよびコンテンツの開発査読

中澤篤志, 梶田将司, 角所考

日本バーチャルリアリティ学会誌, 第10 巻2 号, pp. 98-103, 2005-6. 2005年

ハフ変換と期待値最大化法による距離画像からの二次曲面モデルのパラメータ推定

山本国士, 中澤篤志, 清川清, 竹村治雄

電子情報通信学会技術研究報告 104 ( 489 ) 27 - 32 2004年12月

記述言語：日本語掲載種別：研究論文（その他学術会議資料等）出版者・発行元：一般社団法人電子情報通信学会

本論文では,二次曲面モデル(球,円柱,円錐)が複数個混在する距離データから,それらの表面パラメータを同時に推定する手法を提案する.まず,二次曲面モデルのパラメータを,ハフ変換を使用して抽出する.次に,抽出したパラメータを初期値として,期待値最大化法を使用して正確なパラメータヘと収束させていく.また,本手法の有効性を検証するために,アルゴリズムを実装し,実際のデータに適用した結果について述べる.

Pose Estimation of Human Upper Body Using Multi-joint CG Model and Stereo Video Images 査読

Kimio Hirao, Atsushi Nakazawa, Kiyoshi Kiyokawa, Haruo Takemura

Proc. Int. Conf. on Artificial Reality and Telexistence (ICAT) 2004年11月

単眼カメラとモーションキャプチャデータによる両腕の姿勢推定

平尾公男, 中澤篤志, 清川清, 竹村治雄

ヒューマンインタフェースシンポジウム講演論文集 2004年10月

レーザスキャナと回転台を用いた遠隔地の 3 次元環境伝送システム

河合克哉, 中澤篤志, 清川清, 竹村治雄

日本バーチャルリアリティ学会大会論文集 51 - 52 2004年9月

多関節 CG モデル画像と人物画像の残差マッチングによる両腕の姿勢推定

平尾公男, 中澤篤志, 清川清, 竹村治雄

電子情報通信学会総合大会講演論文集, D-12-125, Mar. 2004. 2004年3月

実時間レンジファインダと回転台を用いた遠隔地環境の三次元伝送システムの提案

河合克哉, 清川清, 中澤篤志, 竹村治雄

電子情報通信学会総合大会講演論文集, D-11-78, Mar. 2004. 2004年3月

Leg motion primitives for a dancing humanoid robot 査読

S Nakaoka, A Nakazawa, K Yokoi, K Ikeuchi

2004 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, VOLS 1- 5, PROCEEDINGS 610 - 615 2004年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

The goal of the study described in this paper is to develop a total technology for archiving human dance motions. A key feature of this technology is a dance replay by a humanoid robot. Although human dance motions can be acquired by a motion capture system, a robot cannot exactly follow the captured data because of different body structure and physical properties between the human and the robot. In particular, leg motions are too constrained to be converted from the captured data because the legs must interact with the floor and keep dynamic balance within the mechanical constraints of current robots. To solve this problem, we have designed a symbolic description of leg motion primitives in a dance performance. Human dance actions are recognized as a sequence of primitives and the same actions of the robot can be regenerated from them. This framework is more reasonable than modifying the original motion to adapt the robot constraints. We have developed a system to generate feasible robot motions from a human performance, and realized a dance performance by the robot HRP-1S.

Creating Virtual Buddha Statues through Observation 査読

Katsushi Ikeuchi, Atsushi Nakazawa, Ko Nishino, Takeshi Oishi

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops 1 2003年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE Computer Society

This paper overviews our research on digital preservation of cultural assets and digital restoration of their original appearance. Geometric models are digitally achieved through a pipeline consisting of scanning, registering and merging multiple range images. We have developed a robust simultaneous registration method and an efficient and robust voxel-based integration method. On the geometric models created, we have to align texture images acquired from a color camera. We have developed two texture mapping methods. In an attempt to restore the original appearance of historical heritage objects, we have synthesized several buildings and statues using scanned data and literature survey with advice from experts.

DOI： 10.1109/CVPRW.2003.10001

Rhythmic motion analysis using motion capture and musical information 査読

T Shiratori, A Nakazawa, K Ikeuchi

PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON MULTISENSOR FUSION AND INTEGRATION FOR INTELLIGENT SYSTEMS 89 - 94 2003年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

The number of Japanese traditional dancers has been decreasing. Without performers, some dances will be disappeared, because they can not be recorded by conventional media, such as paper We have proposed an archiving method specifically for dancing pattern. Our method has four main stages: 1. Digitizing motions by motion capture systems, 2. Analyzing motions, 3. Synthesizing motions, 4. Reproducing the dance motions by CG and humanoid robots. In order to effectively record the moving patterns, motion primitives are extracted. Each motion primitive describes each basic motion. However most previous primitive extraction fails to provide the rhythm which results in unrhythmic synthesized motion. In this paper we propose a new motion analysis method, which integrates music rhythm into the motion primitives. Our experiment confirmed that our motion analysis yielded the motion primitives in accordance to the music rhythm.

Parallel alignment of a large number of range images 査読

T Oishi, R Sagawa, A Nakazawa, R Kurazume, K Ikeuchi

FOURTH INTERNATIONAL CONFERENCE ON 3-D DIGITAL IMAGING AND MODELING, PROCEEDINGS 2003-January 195 - 202 2003年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE COMPUTER SOC

This paper describes a method for parallel alignment of multiple range images. It is difficult to align a large number of range images simultaneously. Therefore, we developed the parallel method to improve the time and memory performances of the alignment process. Although a general simultaneous alignment algorithm searches correspondences for all pairs of all range images by rejecting redundant dependencies, our method makes it possible to accelerate computation time and reduce the amount of memory used Since the computation between two range images can be preformed independently, each correspondence pair of range images is assigned to each node. Because the computation time is proportional to the number of vertices assigned to each node, by assigning the pairs so that the number of vertices computed is equal on each node, the load on each node is effectively distributed. The heuristic algorithms for graph partitioning are applied to this problem in order to reduce the amount of memory used on each node. The method was tested on a 16 processor PC cluster, where it demonstrated the high extendibility and the performance improvement in time and memory.

DOI： 10.1109/IM.2003.1240250

Analysis and synthesis of human dance motions 査読

A Nakazawa, S Nakaoka, T Shiratori, K Ikeuchi

PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON MULTISENSOR FUSION AND INTEGRATION FOR INTELLIGENT SYSTEMS 83 - 88 2003年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

This paper presents the method for synthesizing stylistic human motions through visual observation. The human motion data is acquired from a motioncapture system. The whole motion sequence is divided into some motion elements and clusterd into some groups according to the correlation of end-effectors' trajectories. We call these segments as 'motion primitives'. Concatenating these motion primitives, we can generates new dance motions. We also think of a motion primitive consists of a basic motion and a motion style. The basic motion is common to all dancers, and the style represents their characteristics. We extracted these two components through the further analysis steps. The experiment results shows the validity of our approach.

Synthesize stylistic human motion from examples 査読

A Nakazawa, S Nakaoka, K Ikeuchi

2003 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, VOLS 1-3, PROCEEDINGS 3899 - 3904 2003年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

The human body motion synthesis is highly necessary for humanoid robots' motion planning and computer animations. In this paper, new method for generating human-like natural motions based on the motion database acquired by motion capture systems is described. On the analysis step, the acquired motions are divided into some motion segments, and then the characteristic poses and motions are archived as 'motion styles'. The motion style is a kind of the human skill, and it's unique to the motions' scenario, such as the different kinds of dances. On the synthesis step, users direct the key poses of human figures. The system generates the characteristic motions according to the user's directions and motion style database. The experiment result shows that this method can synthesize the realistic 'stylized' motions with this framework.

The Great Buddha Project: Modeling Cultural Heritage for VR Systems through Observation.

Katsushi Ikeuchi, Atsushi Nakazawa, Kazuhide Hasegawa, Takeshi Oishi

2003 IEEE / ACM International Symposium on Mixed and Augmented Reality (ISMAR 2003)(ISMAR) 7 2003年

Iterative Refinement of Range Images with Anisotropic Error Distribution 査読

Ryusuke Sagawa, Takeshi Oishi, Atsushi Nakazawa, Ryo Kurazume, Katsushi Ikeuchi

IEEE/RSJ International Conference on Intelligent Robots and Systems, Lausanne, Switzerland, September 30 - October 4, 2002 1 79 - 85 2002年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

We propose a method which refines the range measurement of range finders by computing correspondences of vertices of multiple range images acquired from various viewpoints. Our method assumes that a range image acquired by a laser range finder has anisotropic error distribution which is parallel to the ray direction. Thus, we find corresponding points of range images along with the ray direction. We iteratively converge range images to minimize the distance of corresponding points. We demonstrate the effectiveness of our method by presenting the experimental results of artificial and real range data. Also we show that our method refines a 3D shape more accurately as opposed to that achieved by using the Gaussian filter.

DOI： 10.1007/978-0-387-75807_11

その他リンク： http://dblp.uni-trier.de/db/conf/iros/iros2002.html#conf/iros/SagawaONKI02

　前のページ - 次のページ　

1