On Dynamical Statistical Information Theory

Static statistical information theory is extended to dynamic processes and a dynamical statistical information theory is built up, whose subject is the evolution law of the information entropy and information of dynamical systems. Starting from the state variable evolution equation nonlinear evolution equations of dynamic information entropy density and dynamic information density are derived, that describes respectively the evolution law of information entropy and information. These two equations show that the time rate of change of information entropy density originates together from the drift, diffusion and production in coodinate space and state variable space; and that the time rate of change of information density is caused by the drift, diffusion and dissipation in coodinate space and state variable space. Expressions of drift information flow and diffusion information flow, and concise formulas of information dissipation rate and information entropy production rate are given. It is proved that the rate of the information dissipation (or increase) is equal to the rate of information entropy production (or decrease) in the dynamic system, and that information diffusion and information dissipation happen at the same time. Dynamic mutual information reflecting the dynamic character in the transmission process is presented, which in the limiting case when the proportion of channel length to signal transmission rate approaches zero reduces itself to the present static mutual information. All the above results are derived in a unified fasion from evolution equations of information and information entropy without the addition of any extra assumptions. As exampless of application of the above theoretical formulation, information and information entropy as well as their time rates for three dynamic topics are investigated, viz.: the drift-diffusion transmission of Brownian motion, the kinetics of production of thermal defects, and molecular motors; and the dynamic mutual information of the Gaussian channel are presented.