Information Theory SG Seminar

Master equations for general non-Markovian processes: the Hawkes process and beyond

February 5 (Wed) at 16:30 - 18:00, 2025

Kiyoshi Kanazawa (Associate Professor, Division of Physics and Astronomy, Graduate School of Science, Kyoto University)

The Markovian process is one of the most important classes of stochastic processes. The Markovian process is defined as a stochastic process whose time evolution is independent of the system's entire history and has been extensively studied using the master equation and Fokker-Planck equation approaches. In contrast, non-Markovian processes -- where time evolution depends on the full history of the system -- have not been systematically explored, except for a few special cases, such as semi-Markovian processes. In this talk, we present a recent master-equation approach to general non-Markovian jump processes [1-4]. Beginning with a general non-Markovian jump process, we derive the corresponding master equation through a Markovian-embedding approach. The Markovian embedding is a scheme to add a sufficient number of auxiliary variables to convert a non-Markovian model to a high-dimensional Markovian model. For the case of our model, the one-dimensional non-Markovian model is shown to be equivalent to a Markovian stochastic field theory, and we derive the field master equation correspondingly [4]. As an application, we examine the nonlinear Hawkes process, a history-dependent and self-exciting model frequently used in studying complex systems [1-3]. Additionally, we explore the stochastic thermodynamic framework for general jump processes [5] as another example.

Venue: Hybrid Format (3F #359 and Zoom), Seminar Room #359

Event Official Language: English

Introduction to the stochastic process and its application in physics

February 4 (Tue) - 5 (Wed), 2025

Kiyoshi Kanazawa (Associate Professor, Division of Physics and Astronomy, Graduate School of Science, Kyoto University)

The stochastic process is a popular tool for broad disciplines, such as physics, biophysics, chemistry, neuroscience, economics, and finance. In this lecture course, I will provide an elementary introduction to stochastic processes in physics without assuming rigorous background knowledge of probability theories. Most of the basic topics in stochastic processes will be covered in this lecture course, such as (1) the one-to-one correspondence between stochastic differential equations and master equations, (2) their standard forms, (3) Ito's lemma, and (4) the perturbation theories (the system-size expansion). I will also present its application to statistical physics, such as (5) kinetic theory and (6) a microscopic derivation of the Langevin equation from hard-sphere Hamiltonian dynamics in the dilute gas limit. My goal is to help the audience calculate most of the main calculations by their own hands by providing detailed explanations without abbreviations. This lecture is based on my Japanese notebook, available on my webpage (see the link below). Schedule: (Tue., Feb. 4) 13:00-14:30, 14:45-16:15, 16:30-18:00 (Wed., Feb. 5) 10:30-12:00, 13:00-14:30, 14:45-16:15

Venue: Hybrid Format (3F #359 and Zoom), Seminar Room #359

Event Official Language: English

An Investment Banker’s Journey to the World of Physicists -Seeking the Truth in Economics and Finance

February 26 (Mon) at 15:30 - 17:00, 2024

Irena Vodenska (Professor, Boston University, USA)

I encountered many exciting opportunities to learn, grow intellectually, and teach during my educational, professional, and scientific journey. Life brings chances, and it is up to us to take or leave them. I took my chances, one of the fascinating ones being to embark on a scientific interdisciplinary research collaboration with physicists. My background is in economics and finance, and doing research with physicists has been fascinating from many different points of view, especially in light of being free from any ONE discipline, free to explore research possibilities to answer finance and economic questions based on a boundless horizon of possible solutions. I worked as an investment banker after my first graduate degree before returning to academia to continue chartering new pathways to research. During my work as a hedge fund manager and a NASDAQ market maker, I had an opportunity to witness firsthand, on the trading floor, the US market collapse sparked by the demise of the Long Term Capital Management in 1998 and later the European market plunge during the tragic events of the terrorist attack on New York City on September 11, 2001, when I lived and worked on Manhattan. Most world problems today are complex to solve with one discipline, as multidisciplinary THINKING is needed to cover various aspects of scientific inquiry. Experience is essential, translating real-world knowledge into academia even more so. I was fortunate to be in a position to build the bridge between investment banking and academia. Learning about the pioneer of Econophysics, Boston University Professor H. Eugene Stanley, was like discovering a gold mine for me. After an exciting investment banking experience in the 1990s and early 2000s, I left my investment banking job in New York City to join Professor Stanley’s research laboratory, a time I will cherish and remember as formative, enlightening, and transformative for the rest of my life. One may ask why physicists work with economists on financial economics problems. The answer is simple: physicists are naturally curious, inquisitive, and open to new ideas. Moreover, physicists and economists share the same language, the language of mathematics. The value of the achievement in econophysics research is the results and the empirical outcome based on data obtained with solid models grounded in natural and social science theory. It is not trivial to produce interdisciplinary research, but recognizing its necessity is already prominently featured in many universities’ strategic plans, including Boston University. Let me lay out several studies and results to give you a glimpse into the research I will discuss today. We analyze economic time series and panel data to understand their relationships and investigate whether some economic data could be informative of the behavior of others. We use a novel approach comprised of Complex Hilbert Principal Component Analysis (CHPCA), Rotational Random Shuffling (RRS), and Helmholtz-Hodge (HH) potential to unearth statistically significant co-movements and identify noteworthy economic and geopolitical events that might influence such co-movement dynamics. I will present results from four cases studied collaboratively with my international research collaborators over the last decade since 2013.

Venue: Seminar Room #359 (Main Venue) / via Zoom

Event Official Language: English

Introduction to statistical decision theory and Stein’s paradox

June 21 (Wed) at 14:00 - 15:00, 2023

Takeru Matsuda (Unit Leader, Statistical Mathematics Collaboration Unit, RIKEN Center for Brain Science (CBS))

Statistical decision theory is a general framework for discussing optimality of statistical procedures such as estimation, testing and prediction. In 1956, Charles Stein found a counter-intuitive phenomenon in estimation of the mean parameter of a multivariate normal distribution. He showed that a ``shrinkage estimator” attains better estimation accuracy (smaller mean-squared error) than the maximum likelihood estimator when the dimension is greater than or equal to three. This phenomenon is related to several mathematical fields such as Markov processes and potential theory. The idea of shrinkage estimation has been employed in many statistical methods such as regularization, empirical Bayes and model selection. In this talk, I will introduce the statistical decision theory and illustrate Stein’s paradox.

Venue: Hybrid Format (3F #359 and Zoom), Main Research Building

Event Official Language: English

Physics-informed deep learning approach for modeling crustal deformation

January 23 (Mon) at 10:30 - 11:30, 2023

Naonori Ueda (Deputy Director, RIKEN Center for Advanced Intelligence Project (AIP))

The movement and deformation of the Earth’s crust and upper mantle provide critical insights into the evolution of earthquake processes and future earthquake potentials. Crustal deformation can be modeled by dislocation models that represent earthquake faults in the crust as defects in a continuum medium. In this study, we propose a physics-informed deep learning approach to model crustal deformation due to earthquakes. Neural networks can represent continuous displacement fields in arbitrary geometrical structures and mechanical properties of rocks by incorporating governing equations and boundary conditions into a loss function. The polar coordinate system is introduced to accurately model the displacement discontinuity on a fault as a boundary condition. We illustrate the validity and usefulness of this approach through example problems with strike-slip faults. This approach has a potential advantage over conventional approaches in that it could be straightforwardly extended to high dimensional, anelastic, nonlinear, and inverse problems.

Venue: via Zoom

Event Official Language: English

Geometric decomposition of entropy production in stochastic and chemical systems

December 16 (Fri) at 13:30 - 15:00, 2022

Kohei Yoshimura (Ph.D. Student, Department of Physics, Graduate School of Science, The University of Tokyo)

Entropy production is central to understanding nonequilibrium phenomena. It is known that decomposing entropy production enables us to separately treat distinct two aspects of dynamics, nonstationarity and breaking of detailed balance. In this seminar, I talk about our recent progress on geometric decomposition of entropy production in discrete stochastic systems and deterministic chemical systems. For the audience who may not be familiar with nonequilibrium thermodynamics and linear algebraic graph theory, which the latter enables us to treat the two kinds of systems at once, I would like to start with a very basic introduction. Then I explain why and how we decompose entropy production. Specifically, I mainly focus on the "Onsager-projective decomposition" we study in arXiv:2205.15227 rather than the information geometric decomposition provided in the following paper arXiv:2206.14599. Further, several physical consequences will be discussed, including generalization of Schnakenberg's decomposition stemming from cycles in a steady system, and its relation to gradient flow expressions of a master equation and a rate equation.

Venue: via Zoom

Event Official Language: English

Speed limits for macroscopic transitions

July 13 (Wed) at 13:30 - 15:00, 2022

Ryusuke Hamazaki (RIKEN Hakubi Team Leader, Nonequilibrium Quantum Statistical Mechanics RIKEN Hakubi Research Team, RIKEN Cluster for Pioneering Research (CPR))

Speed of state transitions in macroscopic systems is a crucial concept for foundations of nonequilibrium statistical mechanics as well as various applications in quantum technology represented by optimal quantum control. While extensive studies have made efforts to obtain rigorous constraints on dynamical processes since Mandelstam and Tamm, speed limits that provide tight bounds for macroscopic transitions have remained elusive. Here, by employing the local conservation law of probability, the fundamental principle in physics, we develop a general framework for deriving qualitatively tighter speed limits for macroscopic systems than many conventional ones. We show for the first time that the speed of the expectation value of an observable defined on an arbitrary graph, which can describe general many-body systems, is bounded by the “gradient” of the observable, in contrast with conventional speed limits depending on the entire range of the observable. This framework enables us to derive novel quantum speed limits for macroscopic unitary dynamics. Unlike previous bounds, the speed limit decreases when the expectation value of the transition Hamiltonian increases; this intuitively describes a new trade-off relation between time and the quantum phase difference. Our bound is dependent on instantaneous quantum states and thus can achieve the equality condition, which is conceptually distinct from the Lieb-Robinson bound. We also find that, beyond expectation values of macroscopic observables, the speed of macroscopic quantum coherence can be bounded from above by our general approach. The newly obtained bounds are verified in transport phenomena in particle systems and nonequilibrium dynamics in many-body spin systems. We also demonstrate that our strategy can be applied for finding new speed limits for macroscopic transitions in stochastic systems, including quantum ones, where the bounds are expressed by the entropy production rate. Our work elucidates novel speed limits on the basis of local conservation law, providing fundamental limits to various types of nonequilibrium quantum macroscopic phenomena.

Venue: Hybrid Format (Common Room 246-248 and Zoom)

Event Official Language: English

Simulation-based inference for multi-type cortical circuits

November 29 (Mon) at 13:30 - 15:00, 2021

Enrico Rinaldi (Research Fellow, Physics Department, University of Michigan, USA)

In many scientific fields, ranging from astrophysics to particle physics and neuroscience, simulators for dynamical systems generate a massive amount of data. One of the crucial tasks scientists are spending their precious time on is comparing observational data to the aforementioned simulations in order to infer physically relevant parameters and their uncertainties, based on the model embedded in the simulator. This poses a problem because the likelihood function for realistic simulations of complex physical systems is intractable. Simulation-based inference techniques attack this problem using machine learning tools and probabilistic programming. I will start with an overview of the problem and explain the general application of simulation-based inference methods. Then I will describe an application of the methods to a model of neurons in the visual cortex of mice."

Venue: via Zoom

Event Official Language: English

Hunting hypernuclei by machine learning in nuclear emulsions

November 8 (Mon) at 14:00 - 15:00, 2021

Takehiko Saito (Chief Scientist, High Energy Nuclear Physics Laboratory, RIKEN Cluster for Pioneering Research (CPR))

A hypernuclus is a subatomic systems with strange quark(s). They have been studied already for seven decades for understanding the fundamental baryonic interaction and nuclear matters inside the core of neutron stars. The hypertriton is the lightest hypernucleus with a neutron, a proton and a Lambda hyperon, and it is the benchmark in hypernuclear studies. However, recent experimental studies with heavy ion beams have revealed that the nature of the hypertriton is unclear, especially on its biding energy and lifetime. The most urgent issue is to measure its binding energy very precisely. Measurements with nuclear emulsion have provided the best precision for the hypernuclear binding energy, however, it requires a huge human load on visual image analyses. We have developed machine learning models to detect events associated with production and decay of hypertriton in nuclear emulsions data, and we have already discovered hypertriton events [1]. In the seminar, we’ll discuss the challenges and developments of our machine learning models as well as the outcomes and perspectives of our works.

Venue: Hybrid Format (Common Room 246-248 and Zoom)

Event Official Language: English

Boolean algebras and operator algebras

November 4 (Thu) at 15:00 - 16:30, 2021

Michiya Mori (Special Postdoctoral Researcher, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

The concept of Boolean algebra was introduced by George Boole in 1847. It plays a fundamental role in the theory of propositional logic. The theory of operator algebras was initiated by John von Neumann in around 1930. A keyword of the latter theory is "noncommutativity". In this talk, I will first explain basics of Boolean algebras and some ideas in operator algebra theory. Then I will talk about my recent attempt to give a new formulation of the concept of "noncommutative Boolean algebras" in an operator algebraic framework.

Venue: via Zoom

Event Official Language: English

Quantum annealing and its fundamental aspects/ Quantum annealing and its application to real world

August 4 (Wed) at 13:30 - 16:00, 2021

Masayuki Ohzeki (Professor, Graduate School of Information Sciences, Tohoku University / Professor, Institute of Innovative Research, Tokyo Institute of Technology / Founder, Leader, Sigma-i Co., Ltd.)

Talk A (13:30~14:30) Title: Quantum annealing and its fundamental aspects Abstract: We introduce a heuristic solver for combinatorial optimization problem, quantum annealing. The quantum annealing utilizes the quantum tunneling effect to search the ground state. In particular, the Ising model with the transverse field is employed for demonstration of the quantum annealing. Most of the combinatorial optimization problem can be described by the Ising model and they are solved by quantum annealing. A decade ago, the D-Wave systems Inc. succeeded in realizing the quantum annealing in their manufactured spin system. In this talk, the concept of quantum annealing and its implementation in the D-Wave quantum annealer are introduced. Talk B (14:40~15:40) Title: Quantum annealing and its application to real world Abstract: In this talk, we review the fundamental aspects of quantum annealing and show several applications to practical combinatorial optimization problems. In particular, in Japan, many researchers in industry are interested in practical applications of quantum annealing. We, Tohoku University, are performing various collaboration with many companies in Japan. The first example is to control automated guided vehicles in collaboration with DENSO. The second one is to list hotel recommendation on a web site with Recruit lifestyle. Other ones are also exhibited as far as possible. Let us discuss a future perspective of the quantum annealing in practical applications.

Venue: via Zoom

Event Official Language: English

Overview of Tensor Networks in Machine Learning

July 28 (Wed) at 13:30 - 14:50, 2021

Qibin Zhao (Team Leader, Tensor Learning Team, RIKEN Center for Advanced Intelligence Project (AIP))

Tensor Networks (TNs) are factorizations of high dimensional tensors into networks of many low-dimensional tensors, which have been studied in quantum physics, high-performance computing, and applied mathematics. In recent years, TNs have been increasingly investigated and applied to machine learning and signal processing, due to its significant advances in handling large-scale and high-dimensional problems, model compression in deep neural networks, and efficient computations for learning algorithms. This talk aims to present a broad overview of recent progress of TNs technology applied to machine learning from perspectives of basic principle and algorithms, novel approaches in unsupervised learning, tensor completion, multi-task, multi-model learning and various applications in DNN, CNN, RNN and etc. We also discuss the future research directions and new trend in this area.

Venue: via Zoom

Event Official Language: English

Introduction to the replica method

June 23 (Wed) at 13:30 - 15:40, 2021

Yoshiyuki Kabashima (Professor, Graduate School of Science, The University of Tokyo)

The replica method is a mathematical technique for evaluating the "quenched" average of logarithm (or a real number power) of the partition function with respect to predetermined random variables that condition the objective system. The technique has a long history, dating back at least to a book by Hardy et al in 1930s, but has become well known only since its application to the physics of spin glasses in 1970s. More recently, its application range is spreading rapidly to various fields in information science, including information theory, communication theory, signal processing, computational complexity theory, machine learning, etc. In this talk, we introduce the basic idea of the replica method and its mathematical fault illustrating a few examples. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English

Journal Club: Intrinsically Disordered Region (IDR)

May 19 (Wed) at 13:00 - 14:00, 2021

Kyosuke Adachi (Special Postdoctoral Researcher, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS) / Special Postdoctoral Researcher, Nonequilibrium Physics of Living Matter RIKEN Hakubi Research Team, RIKEN Center for Biosystems Dynamics Research (BDR))

A class of protein domain, which is called intrinsically disordered region (IDR), is known to take no rigid three dimensional structure. Recent studies have shown that IDRs can show biological functions through phase separation, and it is important to clarify what kind of amino acid sequence of IDR leads to phase separation and what kind of mutation results in malfunction. In this journal club, I will discuss these topics by reviewing recent papers. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English

Thermodynamic Uncertainty Relation Connects Physics, Information Science, and Biology

April 28 (Wed) at 13:30 - 16:00, 2021

Yoshihiko Hasegawa (Associate Professor, Department of Information and Communication Engineering, The University of Tokyo)

Higher precision demands more resources. Although this fact is widely accepted, it has only recently been theoretically proved. The thermodynamic uncertainty relation serves as a theoretical basis for this notion, and it states that current fluctuations are bounded from below by thermodynamic costs, such as entropy production and dynamical activity. In this seminar, I show a strong connection between the thermodynamic uncertainty relation and information theory by deriving it through information inequality known as a Cramér-Rao bound, which provides the error bound for any statistical estimator. Moreover, by using a quantum Cramér-Rao bound, I derive a quantum extension of thermodynamic uncertainty relation, which holds for general open quantum systems. The thermodynamic uncertainty relation predicts the fundamental limit of biomolecular processes, and thus it can be applied to infer the entropy production, corresponding to the consumption of adenosine triphosphate, of biological systems in the absence of detailed knowledge about them. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English

Journal Club: Trace inequalities and their applications

April 14 (Wed) at 14:30 - 15:30, 2021

Yukimi Goto (Special Postdoctoral Researcher, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

In this talk, I will explain trace inequalities and related topics. Mainly, I focus on results concerning quantum entropy. This talk is an elementary introduction to that subjects. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English

Journal Club: Reinforcement Learning

March 24 (Wed) at 13:00 - 14:00, 2021

Akinori Tanaka (Senior Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

Reinforcement Learning (RL) is a scheme of Machine Learning that is applicable "without training data." Instead, we prepare a "world" that agents (learners) can probe, and try to optimize their behavior. Historically, study of RL has deep connection to studies of psychology and neuroscience. In this journal club, I would like to give a lightning review of RL. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English

Journal Club: Large deviation statistics of Markovian quantum systems

February 17 (Wed) at 13:00 - 14:30, 2021

Ryusuke Hamazaki (Senior Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS) / RIKEN Hakubi Team Leader, Nonequilibrium Quantum Statistical Mechanics RIKEN Hakubi Research Team, RIKEN Cluster for Pioneering Research (CPR))

Large deviation is a mathematical framework to treat “rare events” in random processes [1]. In this journal club, I talk about recent development of large deviation analysis in open Markovian quantum systems [2,3]. I first introduce the notion of large deviation statistics using the simple independent and identically distributed random variables. I then review recent development of level 2.5 large deviation statistics for classical Markovian jump processes and its application to thermodynamic uncertainty relation [4]. Finally, I discuss how the classical results are extended to quantum regime. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English

Journal Club: Sampling the stable structures based on replica-permutation method

January 27 (Wed) at 13:00 - 14:30, 2021

Hiroshi Yokota (Postdoctoral Researcher, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

When we want to search the (meta)stable structures of the macromolecules such as protein, the combination of molecular dynamics simulation and replica exchange method (REM) is useful. In REM, sampling is performed by exchanging replicas (copies) of the system having different temperatures when this process is accepted based on Metropolis algorithm. In this method, the exchange can be rejected, which leads to the decrease in the sampling efficiency. To obtain more efficient sampling than that of REM, Itoh and Okumura proposed replica-permutation method (RPM) in which the replicas are permutated to perform sampling based on Suwa-Toudou algorithm. In this Journal club, I will introduce RPM and some examples of its application.

Event Official Language: English

Information theory in ecology: Markov chain, Venn diagram, Kronecker (and Cartesian graph) products, and Tsallis entropy

January 20 (Wed) at 13:00 - 14:00, 2021

Ryosuke Iritani (Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

This is more like an introductory talk on how I was motivated to work with information theory, and include unpublished data. Ecologists have been long interested in understanding diversity (divergence) of natural ecosystems. One possible way of accounting for diversity is to use a species' presence/absence table across spatial locations (species-location table), in which we record 1 if a focal species is present in a given site (otherwise 0). Recent interest lies in assessing how diversity (e.g., the number of species) changes with time: for instance, extinction and colonization of species may result in the modification of such tables with time. However, we are yet to have theoretical toolkits to model the dynamics of spcies-site tables. In this talk, I will introduce my model (in collaboration with R. Hamazaki, S. Tatsumi, and M Cadotte) of the dynamics of species-site tables based on Markovian stochastic processes. Specifically, our apporach allows us to analytically obtain the solution of the full stochastic dynamics by means of localizing the dynamics to a single site and then expanding it towards the global sites with Kronecker's prodcut (in linear algebra) or Cartesian product (in graph theory). Intuition obtains from illustrating the dynamics onto Venn diagram, where we draw several sets (corresponding to locations) and binary numbers (corresponding to presence-absence data) and consider random walks on Venn diagram acorss sets; also this Venn diagram based interpretation is mathematically underpinned by Cartesian product of graphs. Finally I will briefly talk about how we assess diversity of ecosystems using Tsallis entropy (or the generalized Shannon entropy).

Venue: via Zoom

Event Official Language: English