UDC 519.95

CYBERNETICS AND CONTROL THEORY

B. S. KOCHKAREV

ON PARTIAL STABILITY OF PROBABILISTIC AUTOMATA

(Presented by Academician V. M. Glushkov, February 23, 1968)

One of the interesting problems in the theory of probabilistic automata (p.a.) is the problem of stability \((^{1})\). Is the behavior of a p.a. stable (invariant) under sufficiently small perturbations of the transition probabilities? In the case of actual p.a., the answer was obtained in the affirmative \((^{1})\). Subsequently, the stability theorem for p.a. of M. O. Rabin was extended to a more general class of p.a. \((^{2-6})\). It turned out that p.a. from this class with isolated cut points represent definite events. Thus, the capabilities of such p.a. for isolated cut points are no greater than those of actual p.a. In \((^{1})\) it was shown that it is in principle impossible to extend the stability theorem to p.a. representing indefinite events with isolated cut points. Moreover, even under a certain natural restriction on the method of perturbing transition probabilities, it was possible to construct an example of a p.a. \((^{1})\) for which the stability theorem with the adopted restriction does not hold. In connection with the foregoing, the question arises: if the behavior of a p.a. is not stable (not invariant) under sufficiently small perturbations of transition probabilities, then can some part of its behavior be stable? And if some part of the behavior is stable, then which part exactly? The present note is devoted to answering the questions posed.

In solving the problem posed, we do not restrict ourselves, on the one hand, to consideration of any particular class of p.a., and, on the other, we impose no restrictions whatever on the method of perturbing transition probabilities.

1. We shall use the concepts and notation given in \((^{5-6})\). For what follows we shall need a number of new definitions.

Definition 1. A cut point \(\lambda,\ 0 \le \lambda < 1\), will be called an isolated cut point of the p.a. \(A\) with respect to the event \(u \subseteq \Sigma^{*}\) if there exists \(\delta,\ \delta > 0\), such that for any word \(x \subseteq U\) the inequality \(|p(x)-\lambda| \ge \delta\) is satisfied.

Obviously, if \(\lambda\) is an isolated cut point of the p.a. \(A\) \((^{1})\), then it is an isolated cut point also with respect to any event \(U \subseteq \Sigma^{*}\). In general, if \(\lambda\) is an isolated cut point of the p.a. \(A\) with respect to an event \(U \subseteq \Sigma^{*}\), then it is an isolated cut point with respect to any event \(U'\), \(U' \subseteq U\).

Definition 2. We shall say that a p.a.
\(A = \langle \Sigma, S, \pi(s), \pi(s'/s,\sigma), F\rangle\) with a cut point \(\lambda\) isolated with respect to an event \(U\) is stable with respect to the event \(U\) if there exists \(\varepsilon,\ \varepsilon > 0\), such that for every automaton
\(A' = \langle \Sigma, S, \pi'(s), \pi'(s'/s,\sigma), F\rangle\) satisfying the conditions

\[ |\pi(s_i)-\pi'(s_i)| < \varepsilon,\qquad i=1,\ldots,n, \]

\[ |\pi(s_j/s_i,\sigma)-\pi'(s_j/s_i,\sigma)| < \varepsilon,\qquad i,j=1,\ldots,n,\ \sigma \in \Sigma, \]

\(\lambda\) is an isolated cut point with respect to the event \(U\) and for

for every $x \in U$

\[ p(x,A)>\lambda \Rightarrow p(x,A')>\lambda, \]
\[ p(x,A)<\lambda \Rightarrow p(x,A')<\lambda. \]

Obviously, if a p.a. $A$ with an isolated cut point $\lambda$ is stable with respect to the event $U \subseteq \Sigma^*$, then it is also stable with respect to any event $U' \subseteq U$.

1. In paper (5) a connection was established between the stability of a p.a. with an isolated cut point $\lambda$ and the ergodic property of the transition-probability matrices of the p.a. In this connection the following is valid.

Theorem 1. A p.a. $A$ with an isolated cut point $\lambda$, whose transition-probability matrices $A(\sigma)$, $\sigma \in \Sigma$, under the operation of matrix multiplication belong to the class of regular stochastic matrices $\mathcal{P}^{(n)}$, is stable.

Definition 3. A p.a. $A$ whose transition-probability matrices $A(\sigma)$, $\sigma \in \Sigma$, satisfy the condition of Theorem 1 will be called ergodic.

It is easy to establish that the following is true.

Theorem 2. If a p.a. $A$ is ergodic and $\lambda$, $0 \leq \lambda < 1$, is an isolated cut point with respect to the event $U \subseteq \Sigma^*$, then the p.a. $A$ is stable with respect to the event $U$.

Since the regularity of a stochastic matrix depends only on the mutual arrangement of the zero and nonzero elements of this matrix (7), in order to establish the ergodicity of a p.a. it is convenient to pass from consideration of the matrices $A(\sigma)$, $\sigma \in \Sigma$, of transition probabilities of the p.a. $A$ to consideration of the corresponding Boolean matrices (6).

If $P$ is a stochastic matrix and $\mathcal{L}$ is an arbitrary class of stochastic matrices, then by $B(P)$, $B(\mathcal{L})$ we shall denote, respectively, the Boolean matrix corresponding to the matrix $P$, and the class of Boolean matrices corresponding to matrices from $\mathcal{L}$. The correspondence $P \to B(P)$ is a homomorphism of the semigroup $\mathcal{P}^{(n)}$ of stochastic $(n \times n)$-matrices with the operation of multiplication onto the semigroup $B(\mathcal{P}^{(n)})$ of Boolean $(n \times n)$-matrices with the Boolean operation of multiplication (5, 6).

Let $A$ be some p.a., and let $A(\sigma)$, $\sigma \in \Sigma$, be its transition-probability matrices. Denote by $R \subseteq \Sigma^*$ an event such that for all words $x \in R$ and only for such $x$ the matrices $A(x)$ are regular stochastic matrices. Then the following is true.

Theorem 3. The event $R$ is a regular event.

Proof. Denote by $\{B(\sigma), \sigma \in \Sigma\}$ the system of Boolean matrices corresponding to the matrices $A(\sigma)$, $\sigma \in \Sigma$, of transition probabilities of the p.a. $A$. Some of the matrices $B(\sigma)$, $\sigma \in \Sigma$, may coincide. Nevertheless, we shall distinguish such matrices according to the input letters $\sigma$ occurring with them. Adjoin to the system $\{B(\sigma), \sigma \in \Sigma\}$ the identity matrix $E$***, which for convenience we denote by $B(\Lambda)$, and denote the resulting system by

\[ B^{(1)}=\{B(\sigma), \sigma \in \Sigma \cup \Lambda\}. \]

Let $B^{(k)}$, $k=1,2,\ldots$, be the system of matrices

\[ B^{(k)}=\{B(\sigma_1)B(\sigma_2)\ldots B(\sigma_k),\ \sigma_i \in \Sigma \cup \Lambda,\ i=1,\ldots,k\}. \]

Obviously,

\[ B^{(1)} \subseteq B^{(2)} \subseteq \ldots \subseteq B^{(k)} \subseteq \ldots \tag{1} \]

* We borrow the definition of a regular stochastic matrix from V. I. Romanovskii (7).

** For the definition of a regular event see (9–11).

*** The order of the matrix $E$ is equal to the number of states of the p.a. $A$.

Since the number of Boolean matrices of fixed order is finite, there will necessarily come a moment when \(B^{(l)}=B^{(l+1)}\). It is easy to establish that in this case \(B^{(l)}=B^{(l+i)}\), \(i=1,2,\ldots\). Let \(B^{(l)}\), where \(l\) is such that \(B^{(l)}=B^{(l+1)}\), be a system of Boolean matrices

\[ B^{(l)}=\{B_1,B_2,\ldots,B_m\}. \tag{2} \]

According to (1), the system (2) will contain all matrices \(B(\sigma)\), \(\sigma\in \Sigma\cup\Lambda\).

Consider the initial deterministic automaton \(B\) [7–9], whose states are the matrices of the system (2), whose input letters are the letters of the alphabet \(\Sigma\), and whose initial state is \(B(\Lambda)\). We define the transition function \(\delta(B_i,\sigma)\), \(B_i\in B^{(l)}\), \(\sigma\in\Sigma\), of the automaton \(B\) as follows: \(\delta(B_i,\sigma)=B_iB(\sigma)\). As the set of final (marked) states \(F\) of the automaton \(B\), we take the totality of all regular Boolean matrices of the system \(B^{(l)}\).* Obviously, the deterministic automaton \(B\) with the set of final states \(F\) represents the event \(R\), as was required to prove.

Analyzing the operation of the deterministic automaton \(B\) [9, 10], we can, using the operations of iteration, disjunction, and product of events, write a regular expression for the event \(R\).

Definition 4. A subset of matrices \(C\subseteq F\) will be called ergodic if the closure of the set of matrices \(C\) with respect to the operation of matrix multiplication belongs to the set of regular Boolean matrices \(B(\mathfrak P_i^{(n)})\).

Definition 5. A subset of matrices \(D\subseteq F\) will be called a maximal ergodic subset of the set \(F\) if there exists no other ergodic subset \(D'\) of the set \(F\) satisfying the condition \(D\subset D'\subseteq F\).

Obviously, every maximal ergodic subset \(D\subseteq F\) is a set closed with respect to the operation of matrix multiplication. In the case when the p.a. is ergodic, we shall obviously have one single maximal ergodic set, coinciding with the set \(B^{(l)}\).

Let \(D_1,D_2,\ldots,D_s\) be an arbitrary system of maximal ergodic subsets of the set \(F\) of the p.a. \(A\), and let \(M_{D_1},M_{D_2},\ldots\) (\(M_{D_s}\) are the events represented in the deterministic automaton \(B\) respectively by the sets of states \(D_1,D_2,\ldots,D_s\). Let further \(N_{D_1},N_{D_2},\ldots,N_{D_s}\) be arbitrary finite sets of words respectively from \(M_{D_1},M_{D_2},\ldots,M_{D_s}\).

For the event \((N_{D_i})^k\), \(i=1,\ldots,s\) let \(l_k(N_{D_i})\) denote \(\max\limits_{x\in (N_{D_i})^k} l(x)\)*.

Next denote by \(N_{D_i}(k)\), \(N'_{D_i}(k)\) the events defined respectively by the expressions

\[ N_{D_i}(k)=\Sigma^*[(N_{D_i})^k\cup (N_{D_i})^{k+1}\cup\ldots]\cup[\Lambda\cup(\Sigma)\cup(\Sigma)^2\cup\ldots\cup(\Sigma)^{l_k(N_{D_i})}], \]

\[ N'_{D_i}(k)=\Sigma^*[(N_{D_i})^k\cup (N_{D_i})^{k+1}\cup\ldots]\cup[\Lambda\cup(N_{D_i})\cup(N_{D_i})^2\cup\ldots\cup(N_{D_i})^{k-1}]. \]

Obviously, the relations

\[ N'_{D_i}(k_1)\subset N'_{D_i}(k_2)\quad \text{for } k_1>k_2, \]

\[ N'_{D_i}(k)\subset N_{D_i}(k)\quad \text{for any } k \tag{3} \]

hold.

* We call a Boolean matrix regular if some power of this matrix contains a column consisting entirely of ones.
* \((N_{D_i})^k\) denotes the \(k\)-th power of the event \(N_{D_i}\).
** \(l(x)\) denotes the length of the word \(x\).

It follows from relations (3) that if the section point \(\lambda\) of the probabilistic automaton \(A\) is isolated with respect to the event \(N'_{D_i}(k)\), then it is an isolated section point also with respect to any event \(N'_{D_i}(p)\), \(p > k\).

Theorem 4. Suppose that for some system of natural numbers \(k_i,\ i = 1,\ldots,s\), the section point \(\lambda\) of the probabilistic automaton \(A\) is an isolated section point with respect to the event \(\bigcup_{i=1}^{s} N_{D_i}(k_i)\). Then there exists a system of natural numbers \(l_i \geq k_i,\ i = 1,\ldots,s\), such that for any system of natural numbers \(p_i \geq l_i,\ i = 1,\ldots,s\), there is a section point \(\lambda(p_1,p_2,\ldots,p_s)\), isolated with respect to the event

\[ \bigcup_{i=1}^{s} N_{D_i}(p_i), \tag{4} \]

and moreover the probabilistic automaton \(A\) with the section point \(\lambda(p_1,p_2,\ldots,p_s)\), isolated with respect to the event (4), is stable with respect to the event (4).

Theorem 5. If \(\lambda\) is an isolated section point of the probabilistic automaton \(A\), then there exists a system of natural numbers \(k_i,\ i = 1,\ldots,s\), such that for any system of natural numbers \(p_i,\ p_i \geq k_i,\ i = 1,\ldots,s\), the probabilistic automaton \(A\) is stable with respect to the event

\[ \bigcup_{i=1}^{s} N_{D_i}(p_i). \]

Theorem 6. If there exists a system of natural numbers \(k_i,\ i = 1,\ldots,s\), such that the section point \(\lambda\) of the probabilistic automaton \(A\) is an isolated section point with respect to the event

\[ \bigcup_{i=1}^{s} N'_{D_i}(k_i), \]

then there exists a system of natural numbers \(l_i \geq k_i,\ i = 1,\ldots,s\), such that the probabilistic automaton \(A\) is stable with respect to the event

\[ \bigcup_{i=1}^{s} N'_{D_i}(l_i). \]

Kazan State University
named after V. I. Ulyanov-Lenin

Received
20 II 1968

CITED LITERATURE

1. M. O. Rabin, Inform. and Control, 6, No. 3, 230 (1963).
2. A. Paz, Inform. and Control, 9, No. 1, 26 (1966).
3. B. S. Kochkarev, Probabilistic Methods and Cybernetics, issue 5, 82 (1967).
4. A. A. Lorenz, Automation and Computer Engineering, No. 5, 57 (1967).
5. B. S. Kochkarev, Cybernetics, No. 2, 24 (1968).
6. B. S. Kochkarev, in: Theory of Automata, issue 4, 1968, p. 79.
7. V. I. Romanovskii, Discrete Markov Chains, Moscow, 1949.
8. M. O. Rabin, D. Scott, Int. Business Machines Corp. J. Res. Dev., 3, No. 2, 114 (1959).
9. V. M. Glushkov, UMN, 16, issue 5, 3 (1961).
10. V. M. Glushkov, Synthesis of Digital Automata, Moscow, 1962.