E. B. DYNKIN

NONHOMOGENEOUS STRICTLY MARKOV PROCESSES

(Presented by Academician A. N. Kolmogorov on 12 XII 1956)

For the case of processes homogeneous in time, the concept of a strictly Markov process was introduced by A. A. Yushkevich and the author in \((^1)\). There it was also shown that the known continuity conditions on the trajectories and transition probabilities of a Markov process imply the strict Markov property. Some special cases of these results were obtained independently of us by Hunt \((^2)\) and Reem \((^3)\). In the present note the general concept of a strictly Markov process is analyzed without the assumption that the process is homogeneous in time.

1. A Markov process is defined by specifying: 1) an interval \(I\) of the number line; 2) a set \(E\) (the phase space), and a certain \(\sigma\)-algebra \(\mathfrak B\) of subsets of \(E\); 3) a set \(\Omega\) (the set of elementary events); 4) a function \(x(t,\omega)\) \((t\in I,\ \omega\in\Omega)\) with values in \(E\); 5) a system of probability measures \(P_{s,x}\) \((s\in I,\ x\in E)\): the measure \(P_{s,x}\) is defined on the \(\sigma\)-algebra \(\mathfrak M^s\) generated by the \(\omega\)-sets \(\{x(t,\omega)\in\Gamma\}\) \((t\in I,\ t\ge s,\ \Gamma\in\mathfrak B)\), and satisfies the condition \(P_{s,x}\{x(s,\omega)=x\}=1\).

These elements define a Markov process if:

\((J_0)\) The function

\[ P(s,x;\,t,\Gamma)=P_{s,x}\{x_t\in\Gamma\}\qquad (s<t\in I,\ \Gamma\in\mathfrak B) \]

is \(\mathfrak B\)-measurable with respect to \(x\).

\((S_0)\) Whatever \(s<t<v\) from \(I\), \(x\in E\), and \(\Gamma\in\mathfrak B\) may be, for almost all \(\omega\in\Omega\),

\[ P_{s,x}\{x_v\in\Gamma\mid x_u,\ s\le u\le t\}=P(t,x_t;\,v,\Gamma) \]

(\(P_{s,x}\{\text{—}\mid x_u,\ s\le u\le t\}\) denotes conditional probability with respect to the \(\sigma\)-algebra \(\mathfrak M_{s,t}\) generated by the \(\omega\)-sets \(\{x(u,\omega)\in\Gamma\}\), where \(u\in[s,t]\), \(\Gamma\in\mathfrak B\)).

1. A function \(\tau(\omega)\) on some subset \(\Omega_\tau\) of the space \(\Omega\), taking values in the interval \(I\), will be called a random variable independent of the future and of the \(s\)-past if: 1) \(\tau(\omega)\ge s\) for all \(\omega\in\Omega_\tau\); 2) \(\{\tau(\omega)\le t\}\in\mathfrak M_{s,t}\) for all \(t\ge s\). We denote by \(\mathfrak M_{s,\tau}\) the totality of all \(A\subseteq\Omega_\tau\) such that \(A\cap\{\tau\le t\}\in\mathfrak M_{s,t}\) for every \(t\ge s\). The system \(\mathfrak M_{s,t}\) is a \(\sigma\)-algebra in the space \(\Omega_\tau\). Let \(A\in\mathfrak M^s\) and \(A\subseteq\Omega_\tau\). We shall write \(P_{s,x}\{\text{—}\mid x_u,\ s\le u\le\tau\}\) instead of \(P_{s,x}\{\text{—}\mid\mathfrak M_{s,\tau}\}\).

We give two definitions of strictly Markov processes. In the first definition we shall consider only variables \(\tau(\omega)\) defined on all of \(\Omega\). The notation \(\tau\le t\) means that \(\tau(\omega)\le t\) for all \(\omega\in\Omega\).

Definition 1. A Markov process is called strictly Markov in the first sense if it is measurable and satisfies the conditions:

\((J_1)\) The function \(P(s,x;\,t,\Gamma)\) is jointly measurable in \(s\) and \(x\) (with respect to the \(\sigma\)-algebra \(\mathfrak B_I\times\mathfrak B\), where \(\mathfrak B_I\) is the \(\sigma\)-algebra generated by the open subsets of the interval \(I\)).

\((S_1)\) Whatever \(s<t\in I\), \(x\in E\), \(\Gamma\in\mathfrak B\), and the random variable \(\tau\leq t\), independent of the future and the \(s\)-past, may be, for almost all \(\omega\in\Omega\) (in the sense of the measure \(P_{s,x}\)) the equality
\[ P_{s,x}\{x_t\in\Gamma\mid x_u,\ s\leq u\leq \tau\}=P(\tau,x;t,\Gamma) \]
holds.

Definition 2. A Markov process is called strongly Markov in the second sense if it is measurable and satisfies the following conditions:

\((J_2)\) The function \(P(s,x;t,\Gamma)\) is jointly measurable in \(s,x,t\) (with respect to \(\mathfrak B_I\times\mathfrak B\times\mathfrak B_I\)).

\((S_2)\) Whatever \(s\in I\), \(x\in E\), \(\Gamma\in\mathfrak B\), a random variable \(\tau\) independent of the future and the \(s\)-past, and an \(\mathfrak M_{s,\tau}\)-measurable function \(\eta(\omega)\geq\tau(\omega)\) (with values in \(I\)) may be, for almost all \(\omega\in\Omega\) (in the sense of \(P_{s,x}\)) the relation
\[ P_{s,x}\{x_\eta\in\Gamma\mid x_u,\ s\leq u\leq \tau\}=P(\tau,x_\tau;\eta,\Gamma) \]
holds.

Obviously, every process that is strongly Markov in the second sense is also strongly Markov in the first sense.

Theorem 1. Let \(x(t,\omega)\) be a process strongly Markov in the first sense. Let \(\tau\leq t\) be a random variable independent of the future and the \(s\)-past; let \(\xi(\omega)\) be a function measurable with respect to \(\mathfrak M^t\) and such that
\[ M_{s,x}\xi=\int_\Omega \xi(\omega)P_{s,x}(d\omega) \]
exists. Then, for almost all \(\omega\) (in the sense of \(P_{s,x}\)),
\[ M_{s,x}\{\xi\mid x_u,\ s\leq u\leq \tau\}=M_{\tau,x_\tau}\xi \]
(\(M_{s,x}\{\cdot\mid x_u,\ s\leq u\leq \tau\}\) denotes conditional expectation with respect to the \(\sigma\)-algebra \(\mathfrak M_{s,\tau}\)).

Theorem 2. Let \(x(t,\omega)\) be a process strongly Markov in the second sense; let \(f(x_1,\ldots,x_n,\ldots)\) be a \(\mathfrak B\times\cdots\times\mathfrak B\times\cdots\)-measurable function on the space \(E\times\cdots\times E\times\cdots\); let \(\tau\) be a random variable independent of the future and the \(s\)-past; and let \(\eta_1,\eta_2,\ldots,\eta_n,\ldots\geq\tau\) be a sequence of \(\mathfrak M_{s,\tau}\)-measurable \(\omega\)-functions such that \(M_{s,x}f(x_{\eta_1},\ldots,x_{\eta_n},\ldots)\) exists. Then, for almost all \(\omega\in\Omega_\tau\),
\[ M_{s,x}\{f(x_{\eta_1},\ldots,x_{\eta_n},\ldots)\mid x_u,\ s\leq u\leq\tau\} =F(\tau,x_\tau;\eta_1,\ldots,\eta_n,\ldots), \]
where
\[ F(u,y;\ v_1,\ldots,v_n,\ldots)=M_{u,y}f(x_{v_1},\ldots,x_{v_n},\ldots). \]

1. We shall now assume that \(E\) is a metric space and that \(\mathfrak B\) is the \(\sigma\)-algebra generated by the open subsets of \(E\). We shall say that the process \(x(t,\omega)\) is right-continuous if, for every \(\omega\in\Omega\), \(x(t,\omega)\) is a right-continuous function of \(t\).

Theorem 3. A right-continuous Markov process is strongly Markov in the first sense if and only if it is strongly Markov in the second sense.

Thus, for right-continuous processes one need not distinguish between strongly Markov processes in the first and in the second sense, and one may speak simply of strongly Markov processes.

Theorem 4. Let \(x(t,\omega)\) \((0\leq t<\infty,\ \omega\in\Omega)\) be a right-continuous Markov process satisfying condition \((J_2)\). In order that such a process be strongly Markov, it is sufficient that condition \((S_2)\) hold for \(\eta=\tau+h\), where \(h\) is an arbitrary nonnegative constant.

Introduce the following conditions:

\((F_1)\) Whatever continuous bounded function \(f(y)\) \((y\in E)\) may be, the function
\[ F(u,y)=\int_E P(u,y;t,dz)f(z) \]

satisfies the relation

\[ \lim_{\substack{y \to x\\ u \downarrow s}} F(u,y)=F(s,x) \]

at all points \(s\in I,\ x\in E\).

\((F_2)\) Whatever continuous bounded function \(f(y)\) \((y\in E)\) is chosen, the function

\[ \varphi(u,y)=\int_E P(u,y;\,u+h,dz)f(z) \]

satisfies the relation

\[ \lim_{\substack{y \to x\\ u \downarrow s}} \varphi(u,y)=\varphi(s,x) \]

at all points \(s\in I,\ x\in E\).

Theorem 5. If a Markov process is continuous from the right and satisfies conditions \((J_1)\)—\((F_1)\) or \((J_2)\)—\((F_2)\), then it is strictly Markov.

Moscow State University
named after M. V. Lomonosov

Received
11 XII 1956

CITED LITERATURE

¹ E. B. Dynkin, A. A. Yushkevich, Theory of Probability, 1, 149 (1956).
² J. A. Hunt, Trans. Am. Math. Soc., 81, 2, 294 (1956).
³ D. Ray, Trans. Am. Math. Soc., 82, 2 (1956).