Information Theory Seminar

Information theory in ecology: Markov chain, Venn diagram, Kronecker (and Cartesian graph) products, and Tsallis entropy

January 20 (Wed) 13:00 - 14:00, 2021

Ryosuke Iritani (Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

This is more like an introductory talk on how I was motivated to work with information theory, and include unpublished data. Ecologists have been long interested in understanding diversity (divergence) of natural ecosystems. One possible way of accounting for diversity is to use a species' presence/absence table across spatial locations (species-location table), in which we record 1 if a focal species is present in a given site (otherwise 0). Recent interest lies in assessing how diversity (e.g., the number of species) changes with time: for instance, extinction and colonization of species may result in the modification of such tables with time. However, we are yet to have theoretical toolkits to model the dynamics of spcies-site tables. In this talk, I will introduce my model (in collaboration with R. Hamazaki, S. Tatsumi, and M Cadotte) of the dynamics of species-site tables based on Markovian stochastic processes. Specifically, our apporach allows us to analytically obtain the solution of the full stochastic dynamics by means of localizing the dynamics to a single site and then expanding it towards the global sites with Kronecker's prodcut (in linear algebra) or Cartesian product (in graph theory). Intuition obtains from illustrating the dynamics onto Venn diagram, where we draw several sets (corresponding to locations) and binary numbers (corresponding to presence-absence data) and consider random walks on Venn diagram acorss sets; also this Venn diagram based interpretation is mathematically underpinned by Cartesian product of graphs. Finally I will briefly talk about how we assess diversity of ecosystems using Tsallis entropy (or the generalized Shannon entropy).

Venue: via Zoom

Event Official Language: English

Accelerated equilibration in classical stochastic systems

January 13 (Wed) 13:00 - 14:00, 2021

Kyosuke Adachi (Special Postdoctoral Researcher, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS) / Special Postdoctoral Researcher, Nonequilibrium Physics of Living Matter RIKEN Hakubi Research Team, RIKEN Center for Biosystems Dynamics Research (BDR))

Shortcuts to adiabaticity (STA) [1] are processes that make a given quantum state evolve into a target state in a fast manner, which can be useful to avoid decoherence in quantum experiments. In this journal club, I will concisely review the concept of STA, and then focus on the recently proposed classical counterparts of STA, sometimes called engineered swift equilibration, in Brownian particle systems [2] and evolutionary systems [3].

Venue: via Zoom

Event Official Language: English

Review on the Lieb-Robinson bound

December 23 (Wed) 13:00 - 14:00, 2020

Yukimi Goto (Special Postdoctoral Researcher, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

The Lieb-Robinson bound is inequality on the group velocity of information propagation for quantum many-body systems. In this talk, I review this bound mathematically and explain some consequences of the bound.

Event Official Language: English

Quantum Wasserstein distance of order 1

December 16 (Wed) 13:00 - 14:30, 2020

Ryusuke Hamazaki (Senior Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS) / RIKEN Hakubi Team Leader, Nonequilibrium Quantum Statistical Mechanics RIKEN Hakubi Research Team, RIKEN Cluster for Pioneering Research (CPR))

The Wasserstein distance is an indicator for the closeness of two probability distributions and is applied to various fields ranging from information theory to neural networks [1]. It is particularly useful to treat the geometry of the underlying space, such as tensor-product structures. In this journal club, I talk about one of the recent proposals on quantum extension of the Wasserstein distance [2]. After reviewing basic properties of classical Wasserstein distance, e.g., its relation to concentration phenomena, I discuss how they might be generalized to quantum realm.

Venue: via Zoom

Event Official Language: English

Statistical model for meaning representation of language

December 16 (Wed) 10:30 - 12:00, 2020

Koichiro Yoshino (Team Leader, Robotics Project, RIKEN Cluster for Science, Technology and Innovation Hub (RCSTI))

One of the final goals of natural language processing is building a model to capture the semantic meaning of language elements. Language modeling is a recent research trend to build a statistical model to express the meaning of language. The language model is based on the distributional hypothesis. The distributional hypothesis indicates that the surrounding elements of the target element describe the meaning of the element. In other words, relative positions between sentence elements (morphologies, words, and sentences) are essential to know the element's meaning. Recent works on distributed representation mainly focus on relations between clear elements: characters, morphologies, words, and sentences. However, it is essential to use structural information of languages such as dependency and semantic roles for building a human-understandable statistical model of languages. In this talk, we describe the statistical language model's basis and then discuss our research direction to introduce the language structure.

Venue: via Zoom

Event Official Language: English

Journal Club of Information Theory SG II

December 8 (Tue) 13:00 - 14:00, 2020

Akinori Tanaka (Senior Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

The practical updating process of deep neural networks based on stochastic gradient descent is quite similar to stochastic dynamics described by Langevin equation. Under the Langevin system, we can "derive" 2nd law of thermodynamics, i.e. increasing the total entropy of the system. This fact suggests "2nd law of thermodynamics in deep learning." In this talk, I would like to explain this idea roughly, and there will be no concrete new result, but it may provide us new perspectives to study neural networks, I hope.

Venue: via Zoom

Event Official Language: English

Journal Club of Information Theory SG

December 1 (Tue) 13:00 - 14:00, 2020

Akinori Tanaka (Senior Research Scientist, RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS))

The practical updating process of deep neural networks based on stochastic gradient descent is quite similar to stochastic dynamics described by Langevin equation. Under the Langevin system, we can "derive" 2nd law of thermodynamics, i.e. increasing the total entropy of the system. This fact suggests "2nd law of thermodynamics in deep learning." In this talk, I would like to explain this idea roughly, and there will be no concrete new result, but it may provide us new perspectives to study neural networks, I hope. *Detailed information about the seminar refer to the email.

Venue: via Zoom

Event Official Language: English