Faculty Profiles - HARA Sunao

Papers - HARA Sunao

Division display 21 - 40 of about 62 ／ All the affair displays >>

A signal processing perspective on human gait: Decoupling walking oscillations and gestures Reviewed International journal

Adrien Gregorj, Zeynep Yücel, Sunao Hara, Akito Monden, Masahiro Shiomi

Proceedings of the 4th International Conference on Interactive Collaborative Robotics 2019 (ICR 2019) 11659 75 - 85 2019.8

Language：English Publishing type：Research paper (international conference proceedings) Publisher：SPRINGER INTERNATIONAL PUBLISHING AG

This study focuses on gesture recognition in mobile interaction settings, i.e. when the interacting partners are walking. This kind of interaction requires a particular coordination, e.g. by staying in the field of view of the partner, avoiding obstacles without disrupting group composition and sustaining joint attention during motion. In literature, various studies have proven that gestures are in close relation in achieving such goals.Thus, a mobile robot moving in a group with human pedestrians, has to identify such gestures to sustain group coordination. However, decoupling of the inherent -walking- oscillations and gestures, is a big challenge for the robot. To that end, we employ video data recorded in uncontrolled settings and detect arm gestures performed by human-human pedestrian pairs by adopting a signal processing approach. Namely, we exploit the fact that there is an inherent oscillatory motion at the upper limbs arising from the gait, independent of the view angle or distance of the user to the camera. We identify arm gestures as disturbances on these oscillations. In doing that, we use a simple pitch detection method from speech processing and assume data involving a low frequency periodicity to be free of gestures. In testing, we employ a video data set recorded in uncontrolled settings and show that we achieve a detection rate of 0.80.

DOI： 10.1007/978-3-030-26118-4_8

Naturalness Improvement Algorithm for Reconstructed Glossectomy Patient's Speech Using Spectral Differential Modification in Voice Conversion Reviewed International journal

Hiroki Murakami, Sunao Hara, Masanobu Abe, Masaaki Sato, Shogo Minagi

Interspeech 2018, 19th Annual Conference of the International Speech Communication Association 2464 - 2468 2018.9

Language：English Publishing type：Research paper (international conference proceedings) Publisher：ISCA

In this paper, we propose an algorithm to improve the naturalness of the reconstructed glossectomy patient's speech that is generated by voice conversion to enhance the intelligibility of speech uttered by patients with a wide glossectomy. While existing VC algorithms make it possible to improve intelligibility and naturalness, the result is still not satisfying. To solve the continuing problems, we propose to directly modify the speech waveforms using a spectrum differential. The motivation is that glossectomy patients mainly have problems in their vocal tract, not in their vocal cords. The proposed algorithm requires no source parameter extractions for speech synthesis, so there are no errors in source parameter extractions and we are able to make the best use of the original source characteristics. In terms of spectrum conversion, we evaluate with both GMM and DNN. Subjective evaluations show that our algorithm can synthesize more natural speech than the vocoder-based method. Judging from observations of the spectrogram, power in high-frequency bands of fricatives and stops is reconstructed to be similar to that of natural speech.

DOI： 10.21437/Interspeech.2018-1239

Sound sensing using smartphones as a crowdsourcing approach Reviewed International journal

Sunao Hara, Asako Hatakeyama, Shota Kobayashi, Masanobu Abe

2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017 1328 - 1333 2017.12

Authorship：Lead author,　Corresponding author Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

Sounds are one of the most valuable information sources for human beings from the viewpoint of understanding the environment around them. We have been now investigating the method of detecting and visualizing crowded situations in the city in a sound-sensing manner. For this purpose, we have developed a sound collection system oriented to a crowdsourcing approach and carried out the sound-collection in two Japanese cities, Okayama and Kurashiki. In this paper, we present an overview of sound collections. Then, to show an effectiveness of analyzation by sensed sounds, we profile characteristics of the cities through the visualization results of the sound.

DOI： 10.1109/APSIPA.2017.8282238

An investigation to transplant emotional expressions in DNN-based TTS synthesis Reviewed International journal

Katsuki Inoue, Sunao Hara, Masanobu Abe, Nobukatsu Hojo, Yusuke Ijima

2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017 1253 - 1258 2017.12

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

In this paper, we investigate deep neural network (DNN) architectures to transplant emotional expressions to improve the expressiveness of DNN-based text-to-speech (TTS) synthesis. DNN is expected to have potential power in mapping between linguistic information and acoustic features. From multispeaker and/or multi-language perspectives, several types of DNN architecture have been proposed and have shown good performances. We tried to expand the idea to transplant emotion, constructing shared emotion-dependent mappings. The following three types of DNN architecture are examined; (1) the parallel model (PM) with an output layer consisting of both speaker-dependent layers and emotion-dependent layers, (2) the serial model (SM) with an output layer consisting of emotion-dependent layers preceded by speaker-dependent hidden layers, (3) the auxiliary input model (AIM) with an input layer consisting of emotion and speaker IDs as well as linguistics feature vectors. The DNNs were trained using neutral speech uttered by 24 speakers, and sad speech and joyful speech uttered by 3 speakers from those 24 speakers. In terms of unseen emotional synthesis, subjective evaluation tests showed that the PM performs much better than the SM and slightly better than the AIM. In addition, this test showed that the SM is the best of the three models when training data includes emotional speech uttered by the target speaker.

DOI： 10.1109/APSIPA.2017.8282231

New monitoring scheme for persons with dementia through monitoring-area adaptation according to stage of disease Reviewed International journal

Shigeki Kamada, Yuji Matsuo, Sunao Hara, Masanobu Abe

Proceedings of the 1st ACM SIGSPATIAL Workshop on Recommendations for Location-based Services and Social Networks, LocalRec@SIGSPATIAL 2017 1:1 - 1:7 2017.11

Prediction of subjective assessments for a noise map using deep neural networks Reviewed International journal

Shota Kobayashi, Masanobu Abe, Sunao Hara

Adjunct Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2017 113 - 116 2017.9

Language：English Publishing type：Research paper (international conference proceedings) Publisher：ACM

In this paper, we investigate a method of creating noise maps that take account of human senses. Physical measurements are not enough to design our living environment and we need to know subjective assessments. To predict subjective assessments from loudness values, we propose to use metadata related to where, who and what is recording. The proposed method is implemented using deep neural networks because these can naturally treat a variety of information types. First, we evaluated its performance in predicting five-point subjective loudness levels based on a combination of several features: location specific, participant-specific, and sound-specific features. The proposed method achieved a 16.3 point increase compared with the baseline method. Next, we evaluated its performance based on noise map visualization results. The proposed noise maps were generated from the predicted subjective loudness level. Considering the differences between the two visualizations, the proposed method made fewer errors than the baseline method.

DOI： 10.1145/3123024.3123091

Other Link： http://doi.acm.org/10.1145/3123024.3123091

Speaker Dependent Approach for Enhancing a Glossectomy Patient’s Speech via GMM-Based Voice Conversion

Kei Tanaka, Sunao Hara, Masanobu Abe, Masaaki Sato, Shogo Minagi

Interspeech 2017 2017.8

Speaker Dependent Approach for Enhancing a Glossectomy Patient's Speech via GMM-based Voice Conversion

Kei Tanaka, Sunao Hara, Masanobu Abe, Masaaki Sato, Shogo Minagi

18TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2017), VOLS 1-6 3384 - 3388 2017

Language：English Publishing type：Research paper (international conference proceedings) Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC

In this paper, using GMM-based voice conversion algorithm, we propose to generate speaker-dependent mapping functions to improve the intelligibility of speech uttered by patients with a wide glossectomy. The speaker-dependent approach enables to generate the mapping functions that reconstruct missing spectrum features of speech uttered by a patient without having influences of a speaker's factor. The proposed idea is simple, i.e., to collect speech uttered by a patient before and after the glossectomy, but in practice it is hard to ask patients to utter speech just for developing algorithms. To confirm the performance of the proposed approach, in this paper, in order to simulate glossectomy patients, we fabricated an intraoral appliance which covers lower dental arch and tongue surface to restrain tongue movements. In terms of the Mel-frequency cepstrum (MFC) distance, by applying the voice conversion, the distances were reduced by 25% and 42% for speaker dependent case and speaker-independent case, respectively. In terms of phoneme intelligibility, dictation tests revealed that speech reconstructed by speaker-dependent approach almost always showed better performance than the original speech uttered by simulated patients, while speaker-independent approach did not.

DOI： 10.21437/Interspeech.2017-841

Enhancing a glossectomy patient's speech via GMM-based voice conversion Reviewed International journal

Kei Tanaka, Sunao Hara, Masanobu Abe, Shogo Minagi

2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) 1 - 4 2016.12

LiBS: Lifelog browsing system to support sharing of memories Reviewed International journal

Atsuya Namba, Sunao Hara, Masanobu Abe

UbiComp 2016 Adjunct - Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 165 - 168 2016.9

Language：English Publishing type：Research paper (international conference proceedings) Publisher：Association for Computing Machinery, Inc

We propose a lifelog browsing system through which users can share memories of their experiences with other users. Most importantly, by using global positioning system data and time stamps, the system simultaneously displays a variety of log information in a time-synchronous manner. This function empowers users with not only an easy interpretation of other users' experiences but also nonverbal notifications. Shared information on this system includes photographs taken by users, Google street views, shops and restaurants on the map, daily weather, and other items relevant to users' interests. In evaluation experiments, users preferred the proposed system to conventional photograph albums and maps for explaining and sharing their experiences. Moreover, through displayed information, the listeners found out their interest items that had not been mentioned by the speakers.

DOI： 10.1145/2968219.2971401

Other Link： http://doi.acm.org/10.1145/2968219.2971401

Safety vs. Privacy: User preferences from the monitored and monitoring sides of a monitoring system Reviewed International journal

Shigeki Kamada, Sunao Hara, Masanobu Abe

UbiComp 2016 Adjunct - Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing 101 - 104 2016.9

Language：English Publishing type：Research paper (international conference proceedings) Publisher：Association for Computing Machinery, Inc

In this study, in order to develop a monitoring system that takes into account privacy issues, we investigated user preferences in terms of the monitoring and privacy protec-tion levels. The people on the monitoring side wanted the monitoring system to allow them to monitor in detail. Con-versely, it was observed for the people being monitored that the more detailed the monitoring, the greater the feelings of being surveilled intrusively. Evaluation experiments were performed using the location data of three people in differ-ent living areas. The results of the experiments show that it is possible to control the levels of monitoring and privacy protection without being affected by the shape of a living area by adjusting the quantization level of location informa-tion. Furthermore, it became clear that the granularity of location information satisfying the people on the monitored side and the monitoring side is different.

DOI： 10.1145/2968219.2971412

Other Link： http://doi.acm.org/10.1145/2968219.2971412

Sound collection systems using a crowdsourcing approach to construct sound map based on subjective evaluation Reviewed International journal

Sunao Hara, Shota Kobayashi, Masanobu Abe

IEEE ICME Workshop on Multimedia Mobile Cloud for Smart City Applications (MMCloudCity-2016) 1 - 6 2016

Authorship：Lead author Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

This paper presents a sound collection system that uses crowdsourcing to gather information for visualizing area characteristics. First, we developed a sound collection system to simultaneously collect physical sounds, their statistics, and subjective evaluations. We then conducted a sound collection experiment using the developed system on 14 participants. We collected 693,582 samples of equivalent Aweighted loudness levels and their locations, and 5,935 samples of sounds and their locations. The data also include subjective evaluations by the participants. In addition, we analyzed the changes in sound properties of some areas before and after the opening of a large-scale shopping mall in a city. Next, we implemented visualizations on the server system to attract users' interests. Finally, we published the system, which can receive sounds from any Android smartphone user. The sound data were continuously collected and achieved a specified result.

DOI： 10.1109/ICMEW.2016.7574694

A Spoken Dialog System with Redundant Response to Prevent User Misunderstanding Reviewed International journal

Masaki Yamaoka, Sunao Hara, Masanobu Abe

Proceedings of APSIPA Annual Summit and Conference 2015 229 - 232 2015.12

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

We propose a spoken dialog strategy for car navigation systems to facilitate safe driving. To drive safely, drivers need to concentrate on their driving; however, their concentration may be disrupted due to disagreement with their spoken dialog system. Therefore, we need to solve the problems of user misunderstandings as well as misunderstanding of spoken dialog systems. For this purpose, we introduced a driver workload level in spoken dialog management in order to prevent user misunderstandings. A key strategy of the dialog management is to make speech redundant if the driver's workload is too high in assuming that the user probably misunderstand the system utterance under such a condition. An experiment was conducted to compare performances of the proposed method and a conventional method using a user simulator. The simulator is developed under the assumption of two types of drivers: an experienced driver model and a novice driver model. Experimental results showed that the proposed strategies achieved better performance than the conventional one for task completion time, task completion rate, and user's positive speech rate. In particular, these performance differences are greater for novice users than for experienced users.

DOI： 10.1109/APSIPA.2015.7415511

Extracting Daily Patterns of Human Activity Using Non-Negative Matrix Factorization Reviewed International journal

Masanobu Abe, Akihiko Hirayama, Sunao Hara

Proceedings of IEEE International Conference on Consumer Electronics (IEEE-ICCE 2015) 36 - 39 2015.1

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

This paper presents an algorithm to mine basic patterns of human activities on a daily basis using non-negative matrix factorization (NMF). The greatest benefit of the algorithm is that it can elicit patterns from which meanings can be easily interpreted. To confirm its performance, the proposed algorithm was applied to PC logging data collected from three occupations in offices. Daily patterns of software usage were extracted for each occupation. Results show that each occupation uses specific software in its own time period, and uses several types of software in parallel in its own combinations. Experiment results also show that patterns of 144 dimension vectors were compressible to those of 11 dimension vectors without degradation in occupation classification performance. Therefore, the proposed algorithm compressed basic software usage patterns to about one-tenth of their original dimensions while preserving the original information. Moreover, the extracted basic patterns showed reasonable interpretation of daily working patterns in offices.

DOI： 10.1109/ICCE.2015.7066309

Sound collection and visualization system enabled participatory and opportunistic sensing approaches Reviewed International journal

Sunao Hara, Masanobu Abe, Noboru Sonehara

Proceedings of 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops) 390 - 395 2015

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

This paper presents a sound collection system to visualize environmental sounds that are collected using a crowd sourcing approach. An analysis of physical features is generally used to analyze sound properties; however, human beings not only analyze but also emotionally connect to sounds. If we want to visualize the sounds according to the characteristics of the listener, we need to collect not only the raw sound, but also the subjective feelings associated with them. For this purpose, we developed a sound collection system using a crowdsourcing approach to collect physical sounds, their statistics, and subjective evaluations simultaneously. We then conducted a sound collection experiment using the developed system on ten participants. We collected 6,257 samples of equivalent loudness levels and their locations, and 516 samples of sounds and their locations. Subjective evaluations by the participants are also included in the data. Next, we tried to visualize the sound on a map. The loudness levels are visualized as a color map and the sounds are visualized as icons which indicate the sound type. Finally, we conducted a discrimination experiment on the sound to implement a function of automatic conversion from sounds to appropriate icons. The classifier is trained on the basis of the GMM-UBM (Gaussian Mixture Model and Universal Background Model) method. Experimental results show that the F-measure is 0.52 and the AUC is 0.79.

DOI： 10.1109/PERCOMW.2015.7134069

Other Link： https://ousar.lib.okayama-u.ac.jp/ja/53271

Algorithm to Estimate a Living Area Based on Connectivity of Places with Home Reviewed International journal

Yuji Matsuo, Sunao Hara, Masanobu Abe

HCI International 2015 - Posters’ Extended Abstracts (Part II), CCIS 529 529 570 - 576 2015

Language：English Publishing type：Part of collection (book) Publisher：SPRINGER-VERLAG BERLIN

We propose an algorithm to estimate a person's living area using his/her collected Global Positioning System (GPS) data. The most important feature of the algorithm is the connectivity of places with a home, i.e., a living area must consist of a home, important places, and routes that connect them. This definition is logical because people usually go to a place from home, and there can be several routes to that place. Experimental results show that the proposed algorithm can estimate living area with a precision of 0.82 and recall of 0.86 compared with the grand truth established by users. It is also confirmed that the connectivity of places with a home is necessary to estimate a reasonable living area.

DOI： 10.1007/978-3-319-21383-5_95

Extraction of Key Segments from Day-Long Sound Data Reviewed International journal

Akinori Kasai, Sunao Hara, Masanobu Abe

HCI International 2015 - Posters’ Extended Abstracts (Part I), CCIS 528 528 620 - 626 2015

Language：English Publishing type：Part of collection (book) Publisher：SPRINGER-VERLAG BERLIN

We propose a method to extract particular sound segments from the sound recorded during the course of a day in order to provide sound segments that can be used to facilitate memory. To extract important parts of the sound data, the proposed method utilizes human behavior based on a multisensing approach. To evaluate the performance of the proposed method, we conducted experiments using sound, acceleration, and global positioning system data collected by five participants for approximately two weeks. The experimental results are summarized as follows: (1) various sounds can be extracted by dividing a day into scenes using the acceleration data; (2) sound recorded in unusual places is preferable to sound recorded in usual places; and (3) speech is preferable to nonspeech sound.

DOI： 10.1007/978-3-319-21380-4_105

Sub-Band Text-to-Speech Combining Sample-Based Spectrum with Statistically Generated Spectrum Reviewed International journal

Tadashi Inai, Sunao Hara, Masanobu Abe, Yusuke Ijima, Noboru Miyazaki, Hideyuki Mizuno

Proceedings of Interspeech 2015 264 - 268 2015

Language：English Publishing type：Research paper (international conference proceedings) Publisher：ISCA

As described in this paper, we propose a sub-band speech synthesis approach to develop a high quality Text-to-Speech (TTS) system: a sample-based spectrum is used in the high-frequency band and spectrum generated by HMM-based TTS is used in the low-frequency band. Herein, sample-based spectrum means spectrum selected from a phoneme database such that it is the most similar to spectrum generated by HMM-based speech synthesis. A key idea is to compensate over-smoothing caused by statistical procedures by introducing a sample-based spectrum, especially in the high-frequency band. Listening test results show that the proposed method has better performance than HMM-based speech synthesis in terms of clarity. It is at the same level as HMM-based speech synthesis in terms of smoothness. In addition, preference test results among the proposed method, HMM-based speech synthesis, and waveform speech synthesis using 80 min speech data reveal that the proposed method is the most liked.

Other Link： http://dblp.uni-trier.de/db/conf/interspeech/interspeech2015.html#conf/interspeech/InaiHAIMM15

Inhibitory Effects of an Orally Active Small Molecule Alpha4beta1/Alpha4beta7 Integrin Antagonist, TRK-170, on Spontaneous Colitis in HLA-B27 Transgenic Rats

Hiroe Hirokawa, Yoko Koga, Rie Sasaki, Sunao Hara, Hiroyuki Meguro, Mie Kainoh

GASTROENTEROLOGY 146 ( 5 ) S640 - S640 2014.5

A graph-based spoken dialog strategy utilizing multiple understanding hypotheses Reviewed

Norihide Kitaoka, Yuji Kinoshita, Sunao Hara, Chiyomi Miyajima, Kazuya Takeda

Information and Media Technologies 9 ( 1 ) 111 - 120 2014.3

Language：English Publishing type：Research paper (scientific journal) Publisher：Information and Media Technologies Editorial Board

We regarded a dialog strategy for information retrieval as a graph search problem and proposed several novel dialog strategies that can recover from misrecognition through a spoken dialog that traverses the graph. To recover from misrecognition without seeking confirmation, our system kept multiple understanding hypotheses at each turn and searched for a globally optimal hypothesis in the graph whose nodes express understanding states across user utterances in a whole dialog. In the search, we used a new criterion based on efficiency in information retrieval and consistency with understanding hypotheses, which is also used to select an appropriate system response. We showed that our system can make more efficient and natural dialogs than previous ones.

DOI： 10.11185/imt.9.111

　 PREV - NEXT 　

1
2
3
4