How To Find The Transition Matrix | The Science

 

 

The Science

How to find the transition matrix

transition matrices arise in the study of Markov chains, which are a special case of Markov processes. Determines their property is that the state of the process in the "future" depends on the current state (in this) and, thus, is not related to "the past".

How to find the transition matrix

Instruction how to find the transition matrix

Step 1:

It is necessary to consider the stochastic process (SP) X (t). Its probabilistic description is based on the consideration of the n-dimensional probability density of its sections W (x1, x2, ..., xn; t1, t2, ..., tn), which, based on the unit of conditional probability densities, can be written as W (x1, x2 , ..., xn; t1, t2, ..., tn) = W (x1, x2, ..., x (n-1); t1, t2, ..., t (n-1)) ∙ W (xn, tn | x1 , t1, x2, t2, ..., x (n-1), t (n-1)) cchitaya that t1lt; t2

Step 2:

Definition. JV, for which at any moment t1lt consecutive time; t2

Step 3:

Using the apparatus of the same conditional probability densities, it can be concluded, chtoW (x1, x2, ..., x (n-1), xn, tn; t1, t2, ..., t (n-1), tn) = W (x1, tn) ∙ W (x2, t2 | x1, t1) ... ∙ W (xn, tn | x (n-1), t (n-1)). Thus, all the states of the Markov process is completely determined by its initial state, and the densities of transition probabilities W (xn, tn | X (t (n-1)) = x (n-1))). For discrete sequences (possible states are discrete and time), where instead of transition probability densities present their probability and the transition matrix, the process is called - Markov chain.

Step 4:

Consider a homogeneous Markov chain (no, depending on the time). The transition matrix composed of p (ij) the transition of conditional probabilities (see. Figure 1). It is probable that in one step system, which had a condition equal to xi, to a state xj. Transition probabilities determines the formulation of the problem and its physical meaning. Substituting them in the matrix is ​​the answer to this problem.

Step 5:

Typical examples of the construction of transition matrices give the problem of stray particles. Example. Let the system has five states x1, x2, x3, x4, x5. The first and fifth are boundary. Suppose that on each step, the system can only go on to the neighboring state number, wherein when moving in the direction x5 with probability p, a in the direction x1 with probability q (p + q = 1). Upon reaching the boundaries of the system can go in with a chance of v x3, or stay in the same state with veroyatnostyu1-v. Decision . In order that the task has become crystal clear construct the state graph (see. Fig. 2).