So far we have considered one-dimensional and two-dimensional release processes. When the channel can take on two states—open or closed—we have seen that the associated probability density functions are governed by 2 × 2 systems of partial differential equations. When a drug is added to the Markov model, an extra state is introduced associated with either the open or the closed state and we obtain a model for the probability density functions phrased in terms of 3 × 3 systems of partial differential equations. In subsequent chapters, we will study situations involving many states and, to do so without drowning in cumbersome notation, we need mathematical formalism to present such models compactly. The compact form we use here is taken from Huertas and Smith [35]. We will introduce the more compact notation simply by providing a couple of examples. These will, hopefully, clarify how to formulate rather complex models in an expedient manner.

7.1 Two-Dimensional Calcium Release Revisited

Let us start by recalling that the two-dimensional process of calcium release illustrated in Fig. 5.2 on page 92 can be modeled as

$$\displaystyle\begin{array}{rcl} \bar{x}^{{\prime}}(t) =\bar{\gamma } (t)v_{ r}\left (\bar{y} -\bar{ x}\right ) + v_{d}\left (c_{0} -\bar{ x}\right ),& &{}\end{array}$$
(7.1)
$$\displaystyle\begin{array}{rcl} \bar{y}^{{\prime}}(t) =\bar{\gamma } (t)v_{ r}\left (\bar{x} -\bar{ y}\right ) + v_{s}\left (c_{1} -\bar{ y}\right ),& &{}\end{array}$$
(7.2)

where \(\bar{\gamma }=\bar{\gamma } (t)\) is a stochastic variable governed by a Markov model represented by a reaction scheme of the form

$$\displaystyle{C\mathop{\mathop{ \leftrightarrows }\limits^{ k_{oc}}}\limits_{k_{co}}O.}$$

We have seen (see, e.g., page 102) that the probability density functions of the open state (ρ o ) and the closed state (ρ c ) are governed by the system

$$\displaystyle\begin{array}{rcl} \frac{\partial \rho _{o}} {\partial t} + \frac{\partial } {\partial x}\left (a_{o}^{x}\rho _{ o}\right ) + \frac{\partial } {\partial y}\left (a_{o}^{y}\rho _{ o}\right )& = k_{co}\rho _{c} - k_{oc}\rho _{o},&{}\end{array}$$
(7.3)
$$\displaystyle\begin{array}{rcl} \frac{\partial \rho _{c}} {\partial t} + \frac{\partial } {\partial x}\left (a_{c}^{x}\rho _{ c}\right ) + \frac{\partial } {\partial y}\left (a_{c}^{y}\rho _{ c}\right )& = k_{oc}\rho _{o} - k_{co}\rho _{c},&{}\end{array}$$
(7.4)

where

$$\displaystyle\begin{array}{rcl} a_{o}^{x}& =& v_{ r}\left (y - x\right ) + v_{d}\left (c_{0} - x\right ), \\ a_{o}^{y}& =& v_{ r}\left (x - y\right ) + v_{s}\left (c_{1} - y\right ), \\ a_{c}^{x}& =& v_{ d}\left (c_{0} - x\right ), \\ a_{c}^{y}& =& v_{ s}\left (c_{1} - y\right ). {}\end{array}$$
(7.5)

To prepare ourselves for more complex systems, we number the states in this simple system with i = 1, 2, where i = 1 is for the open state and i = 2 is for the closed state. The system can now be written in the form

$$\displaystyle{\frac{\partial \rho _{i}} {\partial t} + \frac{\partial } {\partial x}\left (a_{i}^{x}\rho _{ i}\right ) + \frac{\partial } {\partial y}\left (a_{i}^{y}\rho _{ i}\right ) = \left (K\rho \right )_{i},}$$

where \(\left (K\rho \right )_{i}\) denotes the ith component of the matrix vector product K ρ. Here the vector ρ is given by

$$\displaystyle{\rho = \left (\begin{array}{c} \rho _{1}\\ \rho _{ 2} \end{array} \right ) = \left (\begin{array}{c} \rho _{o}\\ \rho _{c } \end{array} \right )}$$

and the matrix is given by

$$\displaystyle{K = \left (\begin{array}{cc} - k_{12} & k_{21} \\ k_{12} & - k_{21} \end{array} \right ) = \left (\begin{array}{cc} - k_{oc}& k_{co} \\ k_{oc} & - k_{co} \end{array} \right ).}$$

Furthermore, we introduce the functions

$$\displaystyle\begin{array}{rcl} a_{i}^{x}& =& \gamma _{ i}v_{r}\left (y - x\right ) + v_{d}\left (c_{0} - x\right ), {}\\ a_{i}^{y}& =& \gamma _{ i}v_{r}\left (x - y\right ) + v_{s}\left (c_{1} - y\right ), {}\\ \end{array}$$

where γ i is one for the open state (i.e., i = 1) and zero for the closed state (i.e., i = 2).

7.2 Four-State Model

It useful to illustrate this compact notation for a slightly more complex model based on four states. Suppose that the Markov model governing the stochastic variable \(\bar{\gamma }\) in model (7.1) and (7.2) is based on four states: two open states O 1 and O 2 and two closed states C 1 and C 2, as shown in Fig. 7.1.

Fig. 7.1
figure 1

Markov model including four possible states: two open states, O 1 and O 2, and two closed states, C 1 and C 2

Full size image

The probability density system associated with the model (7.1) and (7.2) when the Markov model is given by Fig. 7.1 can now be written in the form

$$\displaystyle\begin{array}{rcl} \frac{\partial \rho _{o_{1}}} {\partial t} + \frac{\partial } {\partial x}\left (a_{o}^{x}\rho _{ o_{1}}\right ) + \frac{\partial } {\partial y}\left (a_{o}^{y}\rho _{ o_{1}}\right )& =& k_{c_{1}o_{1}}\rho _{c_{1}} -\left (k_{o_{1}c_{1}} + k_{o_{1}o_{2}}\right )\rho _{o_{1}} + k_{o_{2}o_{1}}\rho _{o_{2}}, \\ \frac{\partial \rho _{o_{2}}} {\partial t} + \frac{\partial } {\partial x}\left (a_{o}^{x}\rho _{ o_{2}}\right ) + \frac{\partial } {\partial y}\left (a_{o}^{y}\rho _{ o_{2}}\right )& =& k_{c_{2}o_{2}}\rho _{c_{2}} -\left (k_{o_{2}c_{2}} + k_{o_{2}o_{1}}\right )\rho _{o_{2}} + k_{o_{1}o_{2}}\rho _{o_{1}}, \\ \frac{\partial \rho _{c_{1}}} {\partial t} + \frac{\partial } {\partial x}\left (a_{c}^{x}\rho _{ c_{1}}\right ) + \frac{\partial } {\partial y}\left (a_{c}^{y}\rho _{ c_{1}}\right )& =& k_{o_{1}c_{1}}\rho _{o_{1}} -\left (k_{c_{1}o_{1}} + k_{c_{1}c_{2}}\right )\rho _{c_{1}} + k_{c_{2}c_{1}}\rho _{c_{2}}, \\ \frac{\partial \rho _{c_{2}}} {\partial t} + \frac{\partial } {\partial x}\left (a_{c}^{x}\rho _{ c_{2}}\right ) + \frac{\partial } {\partial y}\left (a_{c}^{y}\rho _{ c_{2}}\right )& =& k_{c_{1}c_{2}}\rho _{c_{1}} -\left (k_{c_{2}c_{1}} + k_{c_{2}o_{2}}\right )\rho _{c_{2}} + k_{o_{2}c_{2}}\rho _{o_{2}},{}\end{array}$$
(7.6)

where

$$\displaystyle\begin{array}{rcl} a_{o}^{x}& =& v_{ r}\left (y - x\right ) + v_{d}\left (c_{0} - x\right ), \\ a_{o}^{y}& =& v_{ r}\left (x - y\right ) + v_{s}\left (c_{1} - y\right ), \\ a_{c}^{x}& =& v_{ d}\left (c_{0} - x\right ), \\ a_{c}^{y}& =& v_{ s}\left (c_{1} - y\right ). {}\end{array}$$
(7.7)

By defining the states O 1, O 2, C 1, and C 2 to be the states 1, 2, 3, and 4, respectively, we can write the system (7.6) in the more compact form

$$\displaystyle{ \frac{\partial \rho _{i}} {\partial t} + \frac{\partial } {\partial x}\left (a_{i}^{x}\rho _{ i}\right ) + \frac{\partial } {\partial y}\left (a_{i}^{y}\rho _{ i}\right ) = \left (K\rho \right )_{i} }$$
(7.8)

for i = 1, 2, 3, 4, where

$$\displaystyle\begin{array}{rcl} a_{i}^{x}& =& \gamma _{ i}v_{r}\left (y - x\right ) + v_{d}\left (c_{0} - x\right ), {}\\ a_{i}^{y}& =& \gamma _{ i}v_{r}\left (x - y\right ) + v_{s}\left (c_{1} - y\right ), {}\\ \end{array}$$

and ρ = (ρ 1, ρ 2, ρ 3, ρ 4)T. Here γ i is one for the open states (i.e., i = 1 and i = 2) and zero for the closed states (i.e., i = 3 and i = 4). Furthermore, the matrix is given by

$$\displaystyle{K = \left (\begin{array}{cccc} -\left (k_{o_{1}c_{1}} + k_{o_{1}o_{2}}\right )& k_{o_{2}o_{1}} & k_{c_{1}o_{1}} & 0 \\ k_{o_{1}o_{2}} & -\left (k_{o_{2}c_{2}} + k_{o_{2}o_{1}}\right )& 0 & k_{c_{2}o_{2}} \\ k_{o_{1}c_{1}} & 0 & -\left (k_{c_{1}o_{1}} + k_{c_{1}c_{2}}\right )& k_{c_{2}c_{1}} \\ 0 & k_{o_{2}c_{2}} & k_{c_{1}c_{2}} & -\left (k_{c_{2}c_{1}} + k_{c_{2}o_{2}}\right ) \end{array} \right ),}$$

which in compact notation is

$$\displaystyle{K = \left (\begin{array}{cccc} -\left (k_{13} + k_{12}\right )& k_{21} & k_{31} & 0 \\ k_{12} & -\left (k_{24} + k_{21}\right )& 0 & k_{42} \\ k_{13} & 0 & -\left (k_{31} + k_{34}\right )& k_{43} \\ 0 & k_{24} & k_{34} & -\left (k_{43} + k_{42}\right ) \end{array} \right ).}$$

7.3 Nine-State Model

We have seen how to formulate probability density systems for two-state and four-state Markov models. For even larger Markov models, it is useful to introduce two-dimensional numbering. This will be illustrated using the nine-state model given in Fig. 7.2.

Fig. 7.2
figure 2

Markov model including nine possible states

Full size image

 Here S ij , i, j = 1, 2, 3, denotes the states of the Markov model and K ij mn denotesFootnote 1 the reaction rate from the state S ij to the state S mn . The system governing the probability density functions of these states can be written in the form

$$\displaystyle{ \frac{\partial \rho _{ij}} {\partial t} + \frac{\partial } {\partial x}\left (a_{ij}^{x}\rho _{ ij}\right ) + \frac{\partial } {\partial y}\left (a_{ij}^{y}\rho _{ ij}\right ) = R_{ij}, }$$
(7.9)

where

$$\displaystyle\begin{array}{rcl} R_{ij}& =& K_{i,j+1}^{i,j}\rho _{ i,j+1} + K_{i+1,j}^{i,j}\rho _{ i+1,j} + K_{i,j-1}^{i,j}\rho _{ i,j-1} + K_{i-1,j}^{i,j}\rho _{ i-1,j} {}\\ & -& \left (K_{i,j}^{i,j+1} + K_{ i,j}^{i+1,j} + K_{ i,j}^{i,j-1} + K_{ i,j}^{i-1,j}\right )\rho _{ i,j}. {}\\ \end{array}$$

Here ρ ij denotes the probability density function of the state S ij and we use the convention that K ij mn = 0 for \(i,j,m,n\notin \left \{1,2,3\right \}.\) We also have

$$\displaystyle\begin{array}{rcl} a_{ij}^{x}& =& \gamma _{ ij}v_{r}\left (y - x\right ) + v_{d}\left (c_{0} - x\right ), {}\\ a_{ij}^{y}& =& \gamma _{ ij}v_{r}\left (x - y\right ) + v_{s}\left (c_{1} - y\right ), {}\\ \end{array}$$

where γ ij  = 1 when the state S ij represents an open state and γ ij  = 0 when S ij represents a closed state.