MATHEMATICS

E. B. DYNKIN

ON SOME TRANSFORMATIONS OF MARKOV PROCESSES

(Presented by Academician A. N. Kolmogorov, 16 III 1960)

In the present paper one general class of transformations of Markov processes is introduced and studied; it contains, as special cases, a number of special transformations studied earlier (the formation of a subprocess \({}^{1}\), the transformation of a Wiener process leading to the appearance of drift (see, for example, \({}^{2}\), and others)). In constructing the aforementioned general class of transformations, the notion of a multiplicative functional of a process plays the basic role.*

We use the terminology and notation of the monograph \({}^{1}\).

1. Let \(X=(x_t,\zeta,\mathcal M_t^s,\mathbf P_{s,x})\) be a Markov process in a measurable space \((E,\mathfrak B)\) and on the time interval \([0,T)\). For an arbitrary finite measure \(\mu\) on the \(\sigma\)-algebra \(\mathfrak B\) we put

\[ \mathbf P_{s,\mu}(\mathfrak B)=\int_E \mathbf P_{s,x}(B)\mu(dx) \qquad (B\in \mathcal N^s) \]

(\(\mathcal N^s\) is the \(\sigma\)-algebra in the space of elementary events \(\Omega\), generated by the sets \(\{\omega:x_t(\omega)\in \Gamma\}\) \((t\geq 0),\ \Gamma\in\mathfrak B\)). Further, we put \(A\in \overline{\mathcal N}^s\), if for every finite measure \(\mu\) one can construct \(A_1,A_2\) from \(\mathcal N^s\) such that \(A_1\subseteq A\subseteq A_2\) and \(\mathbf P_{s,\mu}(A_1)=\mathbf P_{s,\mu}(A_2)\).

A nonnegative function \(\alpha_t^s(\omega)\) \((0\leq s\leq t<\zeta(\omega))\) is called a multiplicative functional of the Markov process \(X\), if the following conditions are fulfilled:

1 A. \(\alpha_t^s\) is \(\mathcal M_t^s\cap \overline{\mathcal N}^s\)-measurable.

1 B. \(\alpha_t^s(\omega)\alpha_u^t(\omega)=\alpha_u^s(\omega)\) \((0\leq s\leq t\leq u<\zeta(\omega))\).

We give some examples of multiplicative functionals.

a) If \(g(u,x)\) is any \(\mathfrak B_T^0\times \mathfrak B\)-measurable function (\(\mathfrak B_T^0\) denotes the \(\sigma\)-algebra of all Borel subsets of the interval \([0,T]\)), then \(\alpha_t^s=g(t,x_t)/g(s,x_s)\) is a multiplicative functional.

b) Let the function \(\tau_s(\omega)\) \((0\leq s<\zeta(\omega))\) be subject to the conditions:

\[ s\leq \tau_s(\omega)\leq \zeta(\omega);\qquad \{\tau_s>t\}\in \mathcal M_t^s\cap \overline{\mathcal N}^s;\qquad \{\tau_s>t\}\subseteq \{\tau_s=\tau_t\}\quad (0\leq s\leq t<T). \]

Then \(\alpha_t^s=\chi_{\tau_s>t}\) is a multiplicative functional (the symbol \(\chi_A\) denotes the characteristic function of the set \(A\)).

c) Let \(V(u,x)\) be a \(\mathfrak B_T^0\times \mathfrak B\)-measurable function, \(\mu\) a measure on \(\mathfrak B_T^0\), and let the integral

\[ \int_s^\zeta V(u,x_u)\mu(du) \]

converge or diverge to \(+\infty\) for all

* A brief visual description of this class of transformations (for homogeneous Markov processes) is contained in the survey article \({}^{3}\). Proofs of the results formulated in the present note will be published in the Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability.

\(0\le s<\zeta(\omega)\). Then

\[ \alpha_t^s=\exp\left[-\int_{(s,t]} V(u,x_u)\mu(du)\right]\qquad (0\le s\le t<\zeta(\omega)) \]

is a multiplicative functional.

d) Let \(X=(x_t,T,\mathcal M_t^s,\mathbf P_{s,x})\) be an \(n\)-dimensional Wiener process given on the time interval \([0,T]\). Let \(f(u,x)\) \((u\in[0,T], x\in E)\) be a function with values in the \(n\)-dimensional space \(E\), satisfying the conditions: for every \(\Gamma\in\mathcal B\), \(\{(u,x): f(u,x)\in\Gamma\}\in\mathcal B_T^0\times\mathcal B\); for any \(t\in[0,T)\),

\[ \sup_{0\le u\le t,\ x\in E} f^2(u,x)<\infty^*. \]

It is proved in \((^4)\) that one can choose the value of the stochastic integral so that the formula

\[ \alpha_t^s=\exp\left[-\int_s^t f(u,x_u)\,dx_u\right]\qquad (0\le s\le t<\zeta(\omega)) \]

defines a continuous\({}^{**}\) multiplicative functional.

1. Theorem. Let \(X=(x_t,\zeta,\mathcal M_t^s,\mathbf P_{s,x})\) be a normal\({}^{***}\) Markov process, given on the time interval \([0,T)\) and having elementary-event space \(\Omega\). Let \(\alpha_t^s\) be a multiplicative functional of the process \(X\). If \(\mathbf M_{s,x}\alpha_t^s=1\) for all \(0\le s\le t<T,\ x\in E\), then the formula

\[ \widetilde P(s,x;t,\Gamma)=\mathbf M_{s,x}\bigl[\chi_\Gamma(x_t)\alpha_t^s\bigr] \tag{1} \]

defines a certain transition function.

In order that there exist a Markov process with transition \(\widetilde X\)-function (1), for which each trajectory is obtained by terminating some trajectory of the process \(X\), it is sufficient that some nonnegative function \(\xi_t(\omega)\) \((0\le t<\zeta(\omega))\) satisfy the conditions:

2A. \(\alpha_t^s\xi_t\le \xi_s\) \((0\le s\le t<\zeta(\omega))\).

2B. \(\lim_{t\downarrow s}\alpha_t^s\xi_t=\alpha_s^s\xi_s\).

2C. \(\xi_t\) is \(\mathcal N_t\)-measurable.

2D. \(\mathbf M_{s,x}\xi_s=1\) for arbitrary \(0\le s<T\) and \(x\in E\).

The process \(\widetilde X\) can be constructed in the following way. Put

\[ \widetilde\Omega=\Omega\times[0,T],\qquad \widetilde{\mathcal M}^{\,s}=\mathcal M^s\times\mathcal B_T^0, \]

\[ \widetilde\zeta(\omega,u)=\min[\zeta(\omega),u],\qquad \widetilde x_t(\omega,u)=x_t(\omega)\quad\text{for }0\le t<\widetilde\zeta(\omega,u), \]

and denote by \(\widetilde{\mathcal M}_t^s\) the totality of all subsets of the space \(\widetilde\Omega\) having the form \(A\times(t,T]\), where \(A\in\mathcal M_t^s\). We have defined all elements of the process \(\widetilde X=(\widetilde x_t,\widetilde\zeta,\widetilde{\mathcal M}_t^s,\widetilde{\mathbf P}_{s,x})\), with the exception of the measures \(\widetilde{\mathbf P}_{s,x}\). The latter are defined as follows. Put \(\psi_t^s=\alpha_t^s\xi_t\) \((0\le s\le t<\zeta(\omega))\). Let \(\omega\in Q_s=\{\omega:\alpha_s^s(\omega)=1\}\). By virtue of conditions 2A and 2B, \(\psi_t^s\) is a nonincreasing right-continuous function of \(t\) for \(t\in[s,\zeta(\omega))\). Therefore there exists, and moreover a unique, measure \(\psi^s\) on the \(\sigma\)-algebra \(\mathcal B_T^0\), concentrated on the interval \((s,\zeta(\omega)]\), and such that \(\psi^s(t,\zeta]=\psi_t^s\). For \(\omega\in\bar Q_s\) denote by \(\psi^s\) the unit measure concentrated at the point \(s\).

* By \(f^2\) one should understand the scalar square of the vector \(f\).

** The functional \(\alpha_t^s\) is called continuous if, for any \(0\le s<\zeta(\omega)\), \(\alpha_t^s(\omega)\) is a continuous function of \(t\) on \([s,\zeta(\omega))\).

*** A Markov process is called normal if \(\mathbf P_{s,x}\{\zeta>s\}=1\) for all \(s\in[0,T),\ x\in E\).

The measure \(\widetilde{\mathbf P}_{s,x}\) is defined by the formula

\[ \widetilde{\mathbf P}_{s,x}(C)=\mathbf M_{s,x}\psi(C_\omega), \tag{2} \]

where \(C_\omega\) denotes the \(\omega\)-section of the set \(C\), i.e., the collection of numbers \(u\) such that \((\omega,u)\in C\).

The Markov process constructed by us,
\[ \widetilde X=(\widetilde x_t,\xi,\widetilde{\mathcal M}_t,\widetilde{\mathbf P}_{s,x}), \]
we propose to call the \((\alpha_t^s,\xi_t)\)-subprocess of the process \(X\).

Let us emphasize once more that all elements of \(\widetilde X\), except \(\widetilde{\mathbf P}_{s,x}\), do not depend on \(\alpha_t^s\) and \(\xi\), while the transition function \(\widetilde P(s,x;t,\Gamma)\) of the process \(\widetilde X\) does not depend on \(\xi_t\).

1. Let us consider the most important special classes of \((\alpha_t^s,\xi_t)\)-subprocesses. We call a \((\alpha_t^s,\xi_t)\)-subprocess an \(\alpha_t^s\)-subprocess if \(\xi_t(\omega)=1\) for all \(0\leq t<\zeta(\omega)\). In order for it to be possible to form an \(\alpha_t^s\)-subprocess, it is necessary and sufficient that the multiplicative functional \(\alpha_t^s\) satisfy the conditions

\[ \alpha_t^s(\omega)\leq 1 \quad (0\leq s\leq t<\zeta(\omega)),\qquad \lim_{t\downarrow s}\alpha_t^s(\omega)=\alpha_s^s(\omega). \]

(These conditions are satisfied, for example, for the functional 1б) and for the functional 1в) when \(V\geq 0\).)

If one identifies the subsets \(A\times[0,T]\) of the space \(\widetilde\Omega\) with the subsets \(A\) of the space \(\Omega\), then in the case of an \(\alpha_t^s\)-subprocess it turns out that the measure \(\widetilde{\mathbf P}_{s,x}\) is an extension of the measure \(\mathbf P_{s,x}\). Hence it easily follows that

\[ \alpha_t^s=\widetilde{\mathbf P}_{s,x}\{\widetilde\zeta>t\mid\mathcal M^s\} =\widetilde{\mathbf P}_{s,x}\{\widetilde\zeta>t\mid\widetilde{\mathcal M}_t^s\} \quad \text{almost surely }(\Omega_t,\widetilde{\mathbf P}_{s,x}).\ * \]

Therefore the intuitive picture of the formation of an \(\alpha_t^s\)-subprocess ** may be described as follows: the trajectories of the original process are cut off with a certain probability distribution, and \(\alpha_t^s(\omega)\) denotes the conditional probability that the trajectory \(x_u(\omega)\) will not be cut off during the time interval \([s,t]\) (provided all the phenomena connected with the process \(X\) during the time \([s,t]\) or during the time \([s,T]\) are known).

1. Let the multiplicative functional \(\alpha_t^s\) and the function \(\xi_t^s\) satisfy conditions 2В—2Г, as well as the conditions:

4А. \(\alpha_t^{s\xi_t}=\alpha_s^{s\xi_s}\) \((0\leq s\leq t<\zeta(\omega))\); \(\mathbf P_{s,x}\{\chi_s^s=1\}=1\) for all \(0\leq s<T,\ x\in E\).

Then, as is easy to see, conditions 2А—2Б are fulfilled, and one may form the \((\alpha_t^s,\xi_t)\)-subprocess of the process \(X\). For this subprocess
\[ \widetilde{\mathbf P}_{s,x}\{\widetilde\zeta=\zeta\}=1 \]
for all \(s\in[0,T),\ x\in E\).

According to Theorem 2.5 (¹), the mapping \(\gamma:\Omega\to\widetilde\Omega\), defined by the formula \(\gamma(\omega)=(\omega,\zeta(\omega))\), specifies a transformation of the elementary-event space of the process \(\Omega\). After this transformation we obtain the process
\[ X'=(x_t,\zeta,\mathcal M_t,\mathbf P'_{s,x}), \]
all elements of which, with the exception of \(\mathbf P'_{s,x}\), are the same as for the process \(X\), while the measures \(\mathbf P'_{s,x}\) can be obtained from the measures \(\mathbf P_{s,x}\) by the formula

\[ \mathbf P'_{s,x}(C)=\int_C \xi_s(\omega)\mathbf P_{s,x}(d\omega). \tag{3} \]

Suppose that the multiplicative functional \(\alpha_t^s\) is subject to the condition
\[ \mathbf P_{s,x}\{\alpha_s^s=1\}=1\quad (s\in[0,T),\ x\in E), \]
and that for each \(\omega\in\Omega_s\) there exist—

* We put \(\Omega_t=\{\omega:\zeta(\omega)>t\}\).

** \(\alpha_t^s\)-subprocesses of Markov processes are studied in detail in (¹), Ch. 3.

there exists the limit $\alpha_{\zeta-0}^s=\lim\limits_{t\uparrow \zeta}\alpha_t^s$ and $\mathbf M_{s,x}\alpha_{\zeta-0}^s=1$ for all $s\in[0,T)$, $x\in E$. It is easy to see that in this case the functions $\xi_t=\alpha_{\zeta-0}^t$ satisfy conditions 4A, 2B, 2Γ. Therefore one can transform the measures of the process by formula (3), putting $\xi_s=\alpha_{\zeta-0}^s$. What has been said applies, in particular, to the functional

\[ \alpha_t^s=\exp\left[-\frac12\int_s^t f^2(u,x_u)\,du-\int_s^t f(u,x_u)\,du\right], \]

where $f(u,x)$ is the function described in 1г), for which
\[ \sup_{0\le u<T,\ x\in E} f^2(u,x)<\infty \]
(see in more detail ($^4$)).

1. The third important class of $(\alpha_t^s,\xi_t)$-subprocesses is connected with functions $\eta_t(\omega)$ $(0\le t<\zeta(\omega))$ satisfying the conditions:

5A. $\eta_t(\omega)$ is $\mathcal N^t$-measurable.

5B. $0\le \eta_t(\omega)\le \eta_s(\omega)$ for $0\le s\le t<\zeta(\omega)$.

5В. $\eta_{t+0}(\omega)=\eta_t(\omega)$ $(0\le t<\zeta(\omega))$.

Put $f(t,x)=\mathbf M_{t,x}\eta_t$. Suppose that $0<f(t,x)<\infty$ for all $t\in[0,T)$, $x\in E$. It is easy to see that the pair $\alpha_t^s=f(t,x_t)/f(s,x_s)$, $\xi_t=\eta_t/f(t,x_t)$ satisfies requirements 2A—2Г. Therefore this pair corresponds to a certain $(\alpha_t^s,\xi_t)$-subprocess. Its transition function is given by the formula

\[ \widetilde P(s,x;t,\Gamma)=\frac{1}{f(t,x)}\int_\Gamma P(s,x;t,dy)\,\tilde f(t,y). \]

1. Suppose now that the process $X$ is homogeneous. A functional $\alpha_t^s$ of $X$ is called homogeneous if for any $h\ge0$, $0\le s\le t<T$ and $x\in E$,
   \[ \theta_h\varphi_t^s=\varphi_{t+h}^{s+h} \]
   (almost surely $\Omega_{t+h}$, $\mathbf P_{s+h,x}$). (The functionals 1a), 1b), 1г) are homogeneous if the corresponding functions $g,V,\tilde f$ do not depend on $u$.)

It is proved that if $\alpha_t^s$ is a homogeneous multiplicative functional of a homogeneous Markov process $X$, then every $(\alpha_t^s,\xi_t)$-subprocess is a homogeneous Markov process.

1. Let $X$ be a homogeneous Markov process. A function $\eta$ is called an excessive random variable for the process $X$ if $\eta$ is $\mathcal N^t$-measurable, $\theta_t\eta\le\eta$ for all $t\ge0$ and $\lim\limits_{t\downarrow0}\theta_t\eta=\eta$. If $\eta$ is an excessive random variable, then $\eta_t=\theta_t\eta$ satisfies conditions 5A—5В, and we can construct the $(\alpha_t^s,\xi_t)$-subprocess described in § 5. By virtue of § 6 this subprocess will be a homogeneous Markov process. Its transition function is given by the formula

\[ \widetilde P(t,x,\Gamma)=\frac{1}{f(x)}\int_\Gamma P(t,x,dy)f(y), \tag{4} \]

where $f(x)=\mathbf M_x\eta$. The function $f(x)$ is excessive in the sense of Hunt ($^5$). Formula (4) makes it possible, from each homogeneous transition function $P(t,x,\Gamma)$ and each function $f$ excessive with respect to it, to construct a new homogeneous transition function $\widetilde P(t,x,\Gamma)$. However, in the general case it is impossible to construct a process $\widetilde X$ with transition function $\widetilde P(t,x,\Gamma)$ for which each trajectory would be the beginning of some trajectory of the process $X$.

Moscow State University
named after M. V. Lomonosov

Received
11 III 1960

REFERENCES

1. E. B. Dynkin, Foundations of the Theory of Markov Processes, Moscow, 1959.
2. Yu. V. Prokhorov, Theory of Probability and Its Applications, 1, 2, 229 (1956).
3. E. B. Dynkin, UMN, 15, 2 (1960).
4. E. B. Dynkin, Theory of Probability and Its Applications (1960).
5. G. A. Hunt, Illinois J. Math., 1, 44 (1957).