研究者詳細 - 原　直

論文 - 原　直

分割表示 62 件中 1 - 20 件目／全件表示 >>

Continual learning on audio scene classification using representative data and memory replay GANs 査読国際共著国際誌

Ibnu Daqiqil ID, Masanobu Abe, Sunao Hara

Bulletin of Electrical Engineering and Informatics 14 ( 1 ) 568 - 580 2025年2月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Institute of Advanced Engineering and Science

This paper proposes a methodology aimed at resolving catastropic forgetting problem by choosing a limited portion of the historical dataset to act as a representative memory. This method harness the capabilities of generative adversarial networks (GANs) to create samples that expand upon the representative memory. The main advantage of this method is that it not only prevents catastrophic forgetting but also improves backward transfer and has a relatively stable and small size. The experimental results show that combining real representative data with artificially generated data from GANs, yielded better outcomes and helped counteract the negative effects of catastrophic forgetting more effectively than solely relying on GAN-generated data. This mixed approach creates a richer training environment, aiding in the retention of previous knowledge. Additionally, when comparing different methods for selecting data as the proportion of GAN-generated data increases, the low probability and mean cluster methods performed the best. These methods exhibit resilience and consistency by selecting more informative samples, thus improving overall performance.

DOI： 10.11591/eei.v14i1.8127

researchmap

OtologMap: a case study on the construction of an environmental sound map recorded by smart devices at Okayama and Kurashiki 査読国際誌

Sunao Hara, Masanobu Abe

Noise Control Engineering Journal 1 - 12 2025年2月

Explicit Prosody Control to Realize Discourse Focus in End-to-End Text-to-Speech 査読国際誌

Takumi WADA, Sunao HARA, Masanobu ABE

IEEE International Workshop on Machine Learning for Signal Processing 2024年9月

Speech Synthesis Using Ambiguous Inputs From Wearable Keyboards 査読国際誌

Matsuri Iwasaki, Sunao Hara, Masanobu Abe

2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2023) 1172 - 1178 2023年11月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）

This paper proposes a new application in speech communication using text-to-speech (TTS), and the goal is to enable dysarthria, articulation disorder, or persons who have difficulty in speaking to communicate anywhere and anytime using speech to express their thoughts and feelings. To achieve this goal, an input method is required. Thus, we propose a new text-entry method based on three concepts. First, from an easy-to-carry perspective, we used a wearable keyboard that inputs digits from 0 to 9 in decimal notation according to 10-finger movements. Second, from a no-training perspective, users input sentences in a way of touch typing using the wearable keyboard. Following this method, we obtained a sequence of numbers corresponding to the sentence. Third, a neural machine translation (NMT) method is applied to estimate texts from the sequence of numbers. The NMT was trained using two datasets; one is a Japanese-English parallel corpus containing 2.8 million pairs of sentences, which were extracted from TV and movie subtitles, while the other is a Japanese text dataset containing 32 million sentences, which were extracted from a question-and-answer platform. Using the model, phonemes and accent symbols were estimated from a sequence of numbers. Thus, the result accuracy in symbol levels was 91.48% and 43.45% of all the sentences were completely estimated with no errors. To subjectively evaluate feasibility of the NMT model, a two-person word association game was conducted; one gave hints using synthesized speech that is generated from symbols estimated by NMT, while the other guessed answers. As a result, 67.95% of all the quizzes were correctly answered, and experiment results show that the proposed method has the potential for dysarthria to communicate with TTS using a wearable keyboard.

DOI： 10.1109/APSIPAASC58517.2023.10317228

Scopus

researchmap

Speech-Emotion Control for Text-to-Speech in Spoken Dialogue Systems Using Voice Conversion and x-vector Embedding 査読国際誌

Shunichi Kohara, Masanobu Abe, Sunao Hara

2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2023) 2280 - 2286 2023年11月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）

In this paper, we propose an algorithm to control both speaker individuality and emotional expressions in synthesized speech, where the most important feature is the controllability of intensity in emotional expressions. An aim of the proposed algorithm is to generate various responses including emotions in text-to-speech (TTS) for spoken dialogue systems (SDS), which results in making the system more human-like. An idea is to control emotion and its intensity in line with the user's utterances. For example, when a user happily talks to SDS, the agent of the SDS responses with happy voice. Generally, voice quality of a user and the agent are different. Therefore, the proposed algorithm consists of two steps: (1) voice conversion to change speaker individuality including emotional expressions and (2) TTS with x-vector acting as an embedding vector to mainly control speech quality related to the intensity of emotions. Evaluation experiments are carried out using a scenario of a spoken dialogue system, where a teacher system of TTS encourages or cheers up students according to students' utterances. The experiment results showed that TTS can successfully reproduce the emotion and its intensity that are extracted from students' utterances, while maintaining the teacher's speaker individuality.

DOI： 10.1109/APSIPAASC58517.2023.10317413

Scopus

researchmap

Sound map of urban areas recorded by smart devices: case study at Okayama and Kurashiki 招待国際誌

Sunao Hara, Masanobu Abe

Proceedings of the 52nd International Congress and Exposition on Noise Control Engineering (Inter-Noise 2023) 1 - 12 2023年8月

Predictions for sound events and soundscape impressions from environmental sound using deep neural networks 招待国際誌

Sunao Hara, Masanobu Abe

Proceedings of the 52nd International Congress and Exposition on Noise Control Engineering (Inter-Noise 2023) 1 - 12 2023年8月

Prediction method of Soundscape Impressions using Environmental Sounds and Aerial Photographs 査読国際誌

Yusuke Ono, Sunao Hara, Masanobu Abe

2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) 1222 - 1227 2022年11月

Incremental Audio Scene Classifier Using Rehearsal-Based Strategy 査読国際誌

Ibnu Daqiqil Id, Masanobu Abe, Sunao Hara

2022 IEEE 10th Global Conference on Consumer Electronics (GCCE) 69 - 623 2022年10月

Speech-Like Emotional Sound Generation Using WaveNet 査読

Kento Matsumoto, Sunao Hara, Masanobu Abe

IEICE Transactions on Information and Systems E105.D ( 9 ) 1581 - 1589 2022年9月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Institute of Electronics, Information and Communications Engineers (IEICE)

In this paper, we propose a new algorithm to generate Speech-like Emotional Sound (SES). Emotional expressions may be the most important factor in human communication, and speech is one of the most useful means of expressing emotions. Although speech generally conveys both emotional and linguistic information, we have undertaken the challenge of generating sounds that convey emotional information alone. We call the generated sounds "speech-like," because the sounds do not contain any linguistic information. SES can provide another way to generate emotional response in human-computer interaction systems. To generate "speech-like" sound, we propose employing WaveNet as a sound generator conditioned only by emotional IDs. This concept is quite different from the WaveNet Vocoder, which synthesizes speech using spectrum information as an auxiliary feature. The biggest advantage of our approach is that it reduces the amount of emotional speech data necessary for training by focusing on non-linguistic information. The proposed algorithm consists of two steps. In the first step, to generate a variety of spectrum patterns that resemble human speech as closely as possible, WaveNet is trained with auxiliary mel-spectrum parameters and Emotion ID using a large amount of neutral speech. In the second step, to generate emotional expressions, WaveNet is retrained with auxiliary Emotion ID only using a small amount of emotional speech. Experimental results reveal the following: (1) the two-step training is necessary to generate the SES with high quality, and (2) it is important that the training use a large neutral speech database and spectrum information in the first step to improve the emotional expression and naturalness of SES.

DOI： 10.1587/transinf.2021edp7236

Web of Science

researchmap

その他リンク： https://www.webofscience.com/wos/woscc/full-record/WOS:000852731400008

Concept drift adaptation for audio scene classification using high-level features 査読国際誌

Ibnu Daqiqil Id, Masanobu Abe, Sunao Hara

2022 IEEE International Conference on Consumer Electronics (ICCE) 2022年1月

Acoustic Scene Classifier Based on Gaussian Mixture Model in the Concept Drift Situation 査読国際誌

Ibnu Daqiqil Id, Masanobu Abe, Sunao Hara

Advances in Science, Technology and Engineering Systems Journal 6 ( 5 ) 167 - 176 2021年9月

Phonetic and Prosodic Information Estimation from Texts for Genuine Japanese End-to-End Text-to-Speech 査読国際誌

Naoto Kakegawa, Sunao Hara, Masanobu Abe, Yusuke Ijima

Interspeech 2021 126 - 130 2021年8月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：ISCA

The biggest obstacle to develop end-to-end Japanese text-to-speech (TTS) systems is to estimate phonetic and prosodic information (PPI) from Japanese texts. The following are the reasons: (1) the Kanji characters of the Japanese writing system have multiple corresponding pronunciations, (2) there is no separation mark between words, and (3) an accent nucleus must be assigned at appropriate positions. In this paper, we propose to solve the problems by neural machine translation (NMT) on the basis of encoder-decoder models, and compare NMT models of recurrent neural networks and the Transformer architecture. The proposed model handles texts on token (character) basis, although conventional systems handle them on word basis. To ensure the potential of the proposed approach, NMT models are trained using pairs of sentences and their PPIs that are generated by a conventional Japanese TTS system from 5 million sentences. Evaluation experiments were performed using PPIs that are manually annotated for 5,142 sentences. The experimental results showed that the Transformer architecture has the best performance, with 98.0% accuracy for phonetic information estimation and 95.0% accuracy for PPI estimation. Judging from the results, NMT models are promising toward end-to-end Japanese TTS.

DOI： 10.21437/interspeech.2021-914

Web of Science

researchmap

Model architectures to extrapolate emotional expressions in DNN-based text-to-speech 査読国際誌

Katsuki Inoue, Sunao Hara, Masanobu Abe, Nobukatsu Hojo, Yusuke Ijima

Speech Communication 126 35 - 43 2021年2月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Elsevier BV

This paper proposes architectures that facilitate the extrapolation of emotional expressions in deep neural network (DNN)-based text-to-speech (TTS). In this study, the meaning of “extrapolate emotional expressions” is to borrow emotional expressions from others, and the collection of emotional speech uttered by target speakers is unnecessary. Although a DNN has potential power to construct DNN-based TTS with emotional expressions and some DNN-based TTS systems have demonstrated satisfactory performances in the expression of the diversity of human speech, it is necessary and troublesome to collect emotional speech uttered by target speakers. To solve this issue, we propose architectures to separately train the speaker feature and the emotional feature and to synthesize speech with any combined quality of speakers and emotions. The architectures are parallel model (PM), serial model (SM), auxiliary input model (AIM), and hybrid models (PM&AIM and SM&AIM). These models are trained through emotional speech uttered by few speakers and neutral speech uttered by many speakers. Objective evaluations demonstrate that the performances in the open-emotion test provide insufficient information. They make a comparison with those in the closed-emotion test, but each speaker has their own manner of expressing emotion. However, subjective evaluation results indicate that the proposed models could convey emotional information to some extent. Notably, the PM can correctly convey sad and joyful emotions at a rate of 60%.

DOI： 10.1016/j.specom.2020.11.004

Web of Science

researchmap

その他リンク： https://arxiv.org/abs/2102.10345

Module Comparison of Transformer-TTS for Speaker Adaptation based on Fine-tuning 査読国際誌

Katsuki Inoue, Sunao Hara, Masanobu Abe

Proceedings of APSIPA Annual Summit and Conference (APSIPA-ASC 2020) 826 - 830 2020年12月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE/APSIPA

End-to-end text-to-speech (TTS) models have achieved remarkable results in recent times. However, the model requires a large amount of text and audio data for training. A speaker adaptation method based on fine-tuning has been proposed for constructing a TTS model using small scale data. Although these methods can replicate the target speaker s voice quality, synthesized speech includes the deletion and/or repetition of speech. The goal of speaker adaptation is to change the voice quality to match the target speaker ' s on the premise that adjusting the necessary modules will reduce the amount of data to be fine-tuned. In this paper, we clarify the role of each module in the Transformer-TTS process by not updating it. Specifically, we froze character embedding, encoder, layer predicting stop token, and loss function for estimating sentence ending. The experimental results showed the following: (1) fine-tuning the character embedding did not result in an improvement in the deletion and/or repetition of speech, (2) speech deletion increases if the encoder is not fine-tuned, (3) speech deletion was suppressed when the layer predicting stop token is not fine-tuned, and (4) there are frequent speech repetitions at sentence end when the loss function estimating sentence ending is omitted.

Web of Science

researchmap

Concept Drift Adaptation for Acoustic Scene Classifier Based on Gaussian Mixture Model 査読

Ibnu Daqiqil Id, Masanobu Abe, Sunao Hara

Proceedings of IEEE REGION 10 CONFERENCE (TENCON 2020) 450 - 455 2020年11月

Controlling the Strength of Emotions in Speech-Like Emotional Sound Generated by WaveNet 査読国際誌

Kento Matsumoto, Sunao Hara, Masanobu Abe

Proceedings of Interspeech 2020 3421 - 3425 2020年10月

Semi-Supervised Speaker Adaptation for End-to-End Speech Synthesis with Pretrained Models 査読国際共著国際誌

Katsuki Inoue, Sunao Hara, Masanobu Abe, Tomoki Hayashi, Ryuichi Yamamoto, Shinji Watanabe

Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2020) 7634 - 7638 2020年5月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

Recently, end-to-end text-to-speech (TTS) models have achieved a remarkable performance, however, requiring a large amount of paired text and speech data for training. On the other hand, we can easily collect unpaired dozen minutes of speech recordings for a target speaker without corresponding text data. To make use of such accessible data, the proposed method leverages the recent great success of state-of-the-art end-to-end automatic speech recognition (ASR) systems and obtains corresponding transcriptions from pretrained ASR models. Although these models could only provide text output instead of intermediate linguistic features like phonemes, end-to-end TTS can be well trained with such raw text data directly. Thus, the proposed method can greatly simplify a speaker adaptation pipeline by consistently employing end-to-end ASR/TTS ecosystems. The experimental results show that our proposed method achieved comparable performance to a paired data adaptation method in terms of subjective speaker similarity and objective cepstral distance measures.

DOI： 10.1109/icassp40776.2020.9053371

Web of Science

researchmap

DNN-based Voice Conversion with Auxiliary Phonemic Information to Improve Intelligibility of Glossectomy Patients' Speech 査読国際誌

Hiroki Murakami, Sunao Hara, Masanobu Abe

Proceedings of APSIPA Annual Summit and Conference 2019 138 - 142 2019年11月

Speech-like Emotional Sound Generator by WaveNet 査読国際誌

Kento Matsumoto, Sunao Hara, Masanobu Abe

Proceedings of APSIPA Annual Summit and Conference 2019 143 - 147 2019年11月

　前のページ - 次のページ　

<原 直>

論文 - 原 直

<原　直>

論文 - 原　直