Research Projects -
-
Consensus-Based Distributed Optimization Algorithms of Low Computational Cost and Their Applications to Machine Learning
Grant number：21H03510 2021.04 - 2025.03
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B) Grant-in-Aid for Scientific Research (B)
高橋 規一, 右田 剛史
Grant amount：\13520000 （ Direct expense: \10400000 、 Indirect expense：\3120000 ）
今年度は，当初の計画とは順番が異なるものの，課題２「深層学習への応用」と課題３「行列分解への応用」に注力し，いくつかの重要な成果を得た．まず，課題２では，複数のニューラルネットワークがパラメータ値に関する合意形成を行う際の通信量を大幅に低減することに焦点を当て，すべてのニューラルネットワークが分散的に全域木を求め，その全域木に沿って変数値を送受信して合意形成を行う方法を開発した．具体的には，合意形成アルゴリズムの設計，Webサーバー通信を用いたプロトコルの設計，Python言語による実装，複数台の計算機による実験を行い，従来手法よりも大幅に少ない通信量で完全合意が形成できることを示した．また，開発した合意形成手法を用いた分散学習アルゴリズムを実装し，10個以上のニューラルネットワークによる分散学習実験によって，完全合意を維持したまま効率的に学習を行うことが可能であることを確認した．次に，課題３では，非負値行列因子分解における階層的交互最小二乗法，主成分分析におけるべき乗法，lassoにおける座標降下法のそれぞれについて，複数台の計算機で分散的に実行するためのアルゴリズムを開発し，数値実験によって妥当性を確認した．中でも，非負値行列因子分解における階層的交互最小二乗法の分散アルゴリズムについては，大域収束性の証明のアイデアが生まれ，それに従って途中段階まで証明を行った．さらに，非負値行列因子分解の新たな計算アルゴリズムを複数開発し，それらの大域収束性を証明した．
-
Grant number：15K00035 2015.04 - 2018.03
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
Takahashi Norikazu
Grant amount：\4550000 （ Direct expense: \3500000 、 Indirect expense：\1050000 ）
We studied the problem of optimizing the network topology based on various measures such as the algebraic connectivity, the clustering coefficient, the betweenness centrality, the average shortest path length, and so on. We not only derived the algebraic connectivity maximizing (or locally maximizing) graphs, the clustering coefficient locally maximizing graphs, and the global clustering coefficient maximizing graphs through theoretical analysis, but also developed various algorithms for optimizing the network topology based on the betweenness centrality and the average shortest path length. We also developed some decentralized algorithms for computing the algebraic connectivity, and some fast methods for nonnegative matrix factorization to solve the problem of community detection.
-
Topology optimization of network systems based on graph theory and dynamical systems theory
Grant number：24560076 2012.04 - 2015.03
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
TAKAHASHI Norikazu
Grant amount：\5070000 （ Direct expense: \3900000 、 Indirect expense：\1170000 ）
We studied the problem of optimizing the network topology based on indices such as clustering coefficient, algebraic connectivity and average shortest path length. Not only some properties of the networks having optimal or locally optimal topologies were revealed by theoretical analysis, but also some algorithms that can generate networks with nearly optimal topologies were developed. We also studied some dynamics related problems such as the decentralized estimation of the algebraic connectivity, the convergence analysis of discrete-time recurrent neural networks, the analysis of the number of DC operating points in a certain nonlinear circuits, and the global convergence of iterative solution methods for nonnegative matrix factorization, and obtained many important results through both theoretical analysis and numerical experiments.
-
Grant number：23310104 2011.04 - 2015.03
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B) Grant-in-Aid for Scientific Research (B)
SHOZO Tokinaga, OONISHI Toshiro, ONO Hirotaka, TAKAHASHI Norikazu, NAKANISHI Makoto, MATSUNO Seigo, TAKAGI Noboru, IKEDA Yoshikazu
Grant amount：\19370000 （ Direct expense: \14900000 、 Indirect expense：\4470000 ）
This research attended to apply the estimation of structural change and event occurrence by using the complex systems and its application to risk control and option evaluation. Especially, we extend the theory of real option and Bayesian estimation for the analysis of complex system. In the final year of this research, we provided the result of research to other institutions. Namely, 1)Theory and practice for the investment and formation of relations among firms, 2) Development of software to analyze the complex system and installation to nother institutions, 3)Publications of result of researches in the journal of societies such as Institute of Electornics and Information Engineering in Japan.
-
Generation of extremely ill-conditioned matrices and illconditoned circuits
Grant number：23560472 2011 - 2013
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
NISHI TETSUO, TAKAHASHI Norikazu
Grant amount：\5330000 （ Direct expense: \4100000 、 Indirect expense：\1230000 ）
The quality of numerical algorithms can be evaluated by solving extremely ill-conditioned problems; For example, linear simultaneous equations having a coefficient matrix with extremely large condition number and nonlinear equations possessing infinitely many solutions. For this purpose we studied on the generation of extremely ill-conditioned matrices and on an upper bound of the condition number of a matrix and showed the possibility that the upper bound of the condition number derived by Guggenheimer, et al may approximately be achieved.
We also investigate a nonlinear equation derived originally from transistor circuits and found some interesting properties of solution curve equations derived from it. -
Grant number：21560068 2009 - 2011
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
TAKAHASHI Norikazu
Grant amount：\4160000 （ Direct expense: \3200000 、 Indirect expense：\960000 ）
We developed sequential partial optimization algorithms for fundamental continuous convex programming problems such as linear programs and convex quadratic programs, and evaluated their characteristics and efficiency through global convergence analysis and numerical experiments. We also carried out theoretical analysis and development of efficient algorithms for various optimization problems arising in the field of information and communication technology. To be more specific, we studied research issues such as efficient computation methods for nonnegative matrix factorization, optimal design methods for recurrent neural networks, maximizing clustering coefficient of complex networks, and optimization of the topology of multi-agent networks for fast consensus, and obtained many important results.
-
Grant number：20560374 2008 - 2010
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
NISHI Tetsuo, TAKAHASHI Norikazu, NAKAYA Yuusuke
Grant amount：\4420000 （ Direct expense: \3400000 、 Indirect expense：\1020000 ）
Main results are the following three items :
1)By using the accuracy-guaranteed algorithm developed in our laboratory we numerically proved that the specified two-transistor circuit proposed previously by other researchers has surely five solutions. As the result we see that the maximum number of solutions of two-transistor circuits is equal to or greater than five. 2) On the generation of an extremely ill-conditioned matrix 2.1) we clarified the relation between the singular values of the matrix and an upper bound of the condition number, 2.2) we proposed a new simple method to generate an extremely ill-conditioned matrix, 2.3) we proposed a method to generate a matrix with more desirable properties as the bench mark matrices for the accuracy-guaranteed algorithm. 3) We proposed several methods to estimate the upper bound of the errors of the solutions obtained by circuit analysis program such as SPICE. -
Grant number：19310099 2007 - 2010
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B) Grant-in-Aid for Scientific Research (B)
TOKINAGA Shozo, IWAMOTO Seiichi, NAKAI Wataru, TAKAHASHI Norikazu, MORIYASU Hiroshi, IKEDA Yoshikazu, TAN Kangrong, TAKAGI Noboru
Grant amount：\19500000 （ Direct expense: \15000000 、 Indirect expense：\4500000 ）
In this research, we published theoretical and practical results in many conferences and journals of academic societies, and we also subjective experiences reviewing from domestic and foreign researchers. We realized many publications of papers and books as well as practical software systems concerning about the Complex Systems and Risk Management which is the main topics of this research through valuable discussion with specialists of the underlying topics. At the same time, we provided efficient software to many manufactures managing supply chains, and also to financial institutions who are devoted to customer information management. Overview of the research is summarized as follows.
(1) We provided many seminar for the manufactures and financial institutions about the investment and supply chains.
(2) We aggregate the current result and problems still remained to be solved.
(3) We developed analytical software about the complex systems and risk management, and installed to another systems.
(4) The results of this research were widely published through the accepted papers in academic societies such as Operations Research Society of Japan, The Institute of Electronics, Information and Communication Engineering and The Japan Society of Information Processing.
(5) The result of this research was finally included in a book which was fully supported by the grant for publication of the Japan Society for the Promption of Science. -
Development of Learning Theory based on Information Measure
Grant number：19300051 2007 - 2010
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B) Grant-in-Aid for Scientific Research (B)
TAKEUCHI Junichi, TAKAHASHI Norikazu, JITSUMATSU Yutaka
Grant amount：\18590000 （ Direct expense: \14300000 、 Indirect expense：\4290000 ）
We studied machine learning, information theory, and other related topics from a unified viewpoint of minimum description length principle (MDL principle). In particular, we obtained the new sight on the relation between geometrical structure of tree models and stochastic complexity (SC) and that between communication channel capacity and SC. We also studied ensemble learning and kernel method, and obtained efficient learning method for them. Further, on the basis of the fundamental knowledge obtained in this research, we proposed new learning based methods for incident detection in network security, universal portfolio, super resolution etc., and showed their efficiency.
-
Research on the design of SAW filters for a duplexer based on network theory
Grant number：16560339 2004 - 2006
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
NISHI Tetsuo, OGATA Masato
Grant amount：\3500000 （ Direct expense: \3500000 ）
The aim of this research is to develop the design method of SAW filters, which is usually used in the duplexer of portable phones. A SAW element is a kind of electro-mechanical transducers utilizing the surface acoustic wave, can be manufactured in very small size, and has very high Q (about 800). The characteristics of SAW elements can be modelled by distributed-constant circuits. The use of SAW elements enables high Q bandpass filters with sharp cutoff characteristics for 800Mhz frequency band. The filter has to possess large attenuation at higher frequency band (or lower frequency band) of the passband due to the requirement for duplexers. We use the circuit-theoretic approach for the design.
1. Let T(s)=N(s)/D(s) be a transfer function of a SAW filter. We proposed a design method of filter characteristics T(s) with equiripple attenuation characteristics in the stop band and has the maximally flat passband characteristics.
2. We proposed a design method of SAW filters, (1) which has symmetrical structure, (2) which are composed not only of SAW elements but also lumped-constant inductances. Then the network is modelled as a mixed lumped-and distributed-network and therefore is difficult to analyze and design it.
The π-type or Y-type inductances are placed with a conventional SAW ladder filter. We investigated in detail the effects of inserted inductances for the attenuation frequency bands and showed that the Y-type is better for the analysis and synthesis. -
カオスに基づいたi.i.d.2値系列によるストリーム暗号システム
Grant number：16016269 2004 - 2005
日本学術振興会 科学研究費助成事業 特定領域研究 特定領域研究
香田 徹, 大濱 靖匡, 高橋 規一, 實松 豊
Grant amount：\11200000 （ Direct expense: \11200000 ）
従来,有限体理論に基づいた整数値の擬似乱数生成法が多用されている.一方,確率論ではコイン投げの数学的モデルであるi.i.d.(independent and identically distributed)2値系列が最も重要である.先に本研究代表者は,実数値系列に基づくi.i.d.2値系列生成法をカオス理論で明らかにし,i.i.d,2値系列が情報通信分野で必須の系列として有用であることを示した.本研究は,統計学・確率論基準の暗号強度評価法の確立を目的に遂行したものであり,その主要結果は以下の三点である.(1)i.i.d.2値系列を得るための十分条件を与え、それが可換な有理関数写像であるヤコビ楕円関数写像を含む広いクラスの1次元写像で生成される実数値系列の独立性判定にも適用可能であることを明らかにした.この結果は国際的に認知され,IEEEの会誌特集号の招待論文に纏められているので,IEEEの相当数の会員が知ることとなった.また,楕円暗号がWeierstrass型楕円関数とその微分間の代数的関係式である平面曲線で規定されることに鑑み,ヤコビ楕円関数とその微分,2階微分間を規定する(x,y,z)-空間曲線とその上の力学系及び各軸の1次元射影写像を定義した.その結果,三種類の射影写像は上記十分条件を満たすので,曲線上の実数値軌道ベクトルの2進展開は3次元i.i.d.2値ベクトル系列となることが明らかとなった.このことは,高精細なカラー画像用ストリーム暗号システムが容易に実現できることを示唆している.(2)ブロック暗号のDES(1ブロックからそれへの非線形写像をn段繰り返す.標準はn=16)の強度評価は暗号分野では古典的であるが基本テーマの一つである.偏りのある平文(偏りのあるi.i.d.2値系列も可)を入力するという前提だけの暗号文単独攻撃耐性基準のブロック暗号強度評価法を2種類与えた.これらは,標準DESの解読には,数十テラの英文Ascii平文と暗号文対を必要とする,従来の評価法とは全く異なる.具体的には,数メガの暗号文ビット間の入出力関係頻度表だけで4,6段DESでは全鍵が検出可能であるとの結果を得た.(3)米国の研究グループ提案のElgamal型公開鍵システムを模倣した可換多項式で生成された実数値カオス系列に基づくシステムの公開鍵は秘密鍵を露呈していることをカオス理論により示した.上記研究成果から明らかなように,カオス理論に基づく暗号強度評価法の研究は国策上今後も是非遂行されるべきである。
-
多層セルラニューラルネットワークの最適設計手法に関する研究
Grant number：15760268 2003 - 2004
日本学術振興会 科学研究費助成事業 若手研究(B) 若手研究(B)
高橋 規一
Grant amount：\1300000 （ Direct expense: \1300000 ）
本年度の成果は以下の通りである。
1.1次元2層セルラニューラルネットワーク(CMN)の安定パターンの特徴付けに関する前年度の成果をまとめ、国際会議2004 IEEE International Midwest Symposium on Circuits and Systemsにおいて発表した。
2.一般化固有値最小化に基づくCNN連想記憶設計法が実は線形計画問題に帰着されることを証明し、線形計画法に基づくCNN連想記憶設計法を提案した。提案手法は一般化固有値最小化に基づく方法に比べて設計時間が大幅に短く、従来法と同等の想起性能を示すことを数値実験によって明らかにした。また、大規模ランダムパターンを用いた実験により、提案手法は、外積法、射影学習則、固有構造法などの従来法よりも優れた想起性能を示すことを明らかにした。
3.1次元離散時間CNNの安定性を理論的に解析し、任意の初期条件に対してCNNが必ず平衡状態に収束するための必要十分条件を導出した。
4.サポートベクトルマシン(SVM)の学習法としてよく用いられる分割法の収束性を解析し、変数選択に関する非常に緩い条件の下で分割法の解が必ず最適解に収束することを証明した。また、非線形回帰のためのSVMの効率的学習法について検討し、分割法に基づく新しい学習アルゴリズムを提案した。提案アルゴリズムの有効性を数値実験によって示した。 -
Theoretical study on the number of solutions and the stability on transistor circuits
Grant number：13650414 2001 - 2003
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C) Grant-in-Aid for Scientific Research (C)
NISHI Tetsuo, OGATA Masato, TAKAHASHI Norikazu, NISHI Tetsuo
Grant amount：\3200000 （ Direct expense: \3200000 ）
This study aims to pursue two subjects on nonlinear active circuits including transistor circuits. One is to investigate the maximum number of solutions of transistor circuits under the prescribed topology. This problem was proposed by the Technical Committee on Nonlinear Circuits and Systems of the IEEE CAS Society several years ago.. The other is to investigate the stability conditions for general active circuits including transistor circuits. The main results of this research are summarized as follows:
1.We showed that the algebraic equations which have two variables and whose nonlinear terms are exponential functions of the variables have at most five solutions. This type of equations come from transistor circuits.. The above result show that the well-known conjecture does not hol in general.
2.We give some sufficient conditions for the denominator polynomial of active RC circuits including transistor circuits not to yield negative coefficients due to parasitic elements. This result then shows some conditions for stability of this class of circuits.
3.We give the necessary and sufficient conditions for a one-dimensional discrete-time cellular neural network, which is a class of active analog circuits.
4.We give the necessary and sufficient condition for the second-order differential equations to be globally stable. -
完全安定セルラーニューラルネットワークの最適設計に関する研究
Grant number：13750358 2001 - 2002
日本学術振興会 科学研究費助成事業 若手研究(B) 若手研究(B)
高橋 規一
Grant amount：\1400000 （ Direct expense: \1400000 ）
本年度の研究成果は次の通りである。
1.セルラーニューラルネットワーク(CNN)による連想記憶回路の構成法について検討し、ある種の最適化手法に基づくCNN設計を提案した。提案手法は、従来手法に比べて理論的裏付けがしっかりしているだけでなく、設計手続きが簡単であり、MATLAB等の数値計算ソフトを利用すればCNNのパラメータが簡単に求められるという利点もある。アルファベット26文字に提案手法と従来法を適用した結果、すべての場合において従来法よりも高い平均想起確率が達成されることが確認された。
2.遅延特性を有するCNNの完全安定性に関する新たな十分条件を導出した。この結果は、CNNの完全安定性に関してこれまでに得られているいくつかの結果の一般化になっている。また、2個のセルから構成されるCNNの完全安定性を解析し、ある仮定の下でそれが完全安定であるための必要十分条件を導出した。
3.直線上に配置された多数のセルから構成される1次元離散時間CNNが任意の初期条件に対して必ず平衡状態に収束するための必要十分条件を導出した。
4.DogaruとChuaによって提案されたユニバーサルCNNセルの解析および設計に関する研究を行った。具体的には、最急降下法に基づく効率的なユニバーサルCNNセルの設計法を提案し、その有効性を数値実験によって確認した。ユニバーサルCNNセルでは、多数の絶対値関数を用いることによって複雑な入出力関係が実現されるが、回路構造を簡単にするために絶対値関数の個数をできるだけ少なくする必要がある。数値実験の結果、提案手法においては、所望の入出力関係を実現するのに必要な絶対値関数の個数が各セルの結合数に比例することが明らかになった。 -
Discrimination theory of wavelet filters with learning ability
Grant number：11558039 1999 - 2002
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B) Grant-in-Aid for Scientific Research (B)
NIIJIMA Koichi, TAKANO Shigeru, KUZUME Koichi, OKADA Yoshihiro
Grant amount：\12900000 （ Direct expense: \12900000 ）
Wavelet filters with learning ability indicate lifting wavelet filters proposed by Sweldens who is a researcher at Lucent Technology in USA. The lifting wavelet filters are consist of biorthogonal analysis and synthesis filters. A signal is decomposed into lowpass and highpass components using the analysis filter, and the original signal can be reconstructed from the lowpass and highpass components using the synthesis filter. This means that the original signal is equivalent to the decomposed lowpass and highpass components. The lifting wavelet filters are constructed by adding lifting filters to biorthogonal wavelet filters. The lifting filter contains free parameters which can be determined adaptive to signals and images.
In our research, we proposed several learning methods of the free parameters adaptive to specific parts of signals and images, and established a discrimination theory for extracting pieces similar to the specific parts. We also presented an impulse noise reduction method based on our learning method. Furthermore, we proposed a fast simplification algorithm for generating 3D surfaces by using an idea of multiresolution analysis of wavelets. -
セルラーニューラルネットワークの完全安定性解析
Grant number：11750330 1999 - 2000
日本学術振興会 科学研究費助成事業 奨励研究(A) 奨励研究(A)
高橋 規一
Grant amount：\1100000 （ Direct expense: \1100000 ）
本年度の研究実績は以下の通りである.
1.遅延特性を有するセルラーニューラルネットワーク(Cellular Neural Network,以下CNN)が完全安定であるための新しい十分条件と,他の研究者によって予想された完全安定条件に対する反例をまとめ,IEEE Transactions on Circuits and Systems-Iに発表した.
2.CNNに対する二つの既存の完全安定条件が同一原理で導出できることを示すとともに,これらを統一的に扱うことのできる新しい十分条件を導出した.この結果は,国際会議2000 International Symposium on Nonlinear Theory and its Applicationsにおいて発表された.また,遅延特性を有するCNNに対しても同様の考察を行い,新しい十分条件を導出した.
3.与えられた平衡点集合を実現するCNNの設計問題は,連想記憶やパターン認識への応用において基本的かつ重要な問題である.これまでに,特異値分解を利用したCNN設計法が提案されているが,その方法では完全安定性が保証されないという問題があった.そこで本研究では,結合を対称にすることにより完全安定性を保証する新しい設計法を提案した.しかしながら,まだ問題点も多く,提案法に改良を加えていく必要がある.
4.2個のセルからなるCNNのダイナミクスを詳細に解析し,ある仮定の下で,CNNが完全安定であるための十分条件およびリミットサイクルが存在するための十分条件を新たに与えた.その過程で,2個のセルからなるCNNの完全安定性に関する従来の結果に誤りがあることを明らかにした.この成果は,平成13年度に国際会議等で発表したいと考えている. -
Discovery of structure from color images by entropy minimization learning of neural networks
Grant number：10480076 1998 - 2001
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B) Grant-in-Aid for Scientific Research (B)
NIJIMA Koichi, TAKANO Shigiru, TAKAHASHI Norikazu, OKADA Yoshihiro
Grant amount：\10300000 （ Direct expense: \10300000 ）
Neural networks play an important role in this research. In order to discover the structure that characterizes images, we need to study learning methods for neural networks and to analyze the behavior of the learned neural network. It is required to study the classification and recognition ability of the neural network. We must find rules from the learned neural network. Simplification of 3D images for inputting in the neural network is also an important research subject.
To resolve such problems, we studied various learning methods as well as the entropy minimization learning technique for neural networks and the behavior of cellular neural networks. And we applied the obtained results to image classification and rule extraction. We proposed a method for finding a hidden image from two similar images by combining the entropy minimization learning technique of the neural network with a wavelet theory, which is a main theme of this research. For discovering the structure of 3D models using a neural network, we studied simple surface generation algorithms for obtaining vertices which are input data of the neural network.