Abstract
Full Text
UDC 518.9
MATHEMATICS
I. V. EMELIN, M. A. KRASNOSEL’SKII, A. V. POKROVSKII
GAMES WITH INERTIA
(Presented by Academician L. V. Kantorovich on 6 I 1969)
In the present work an attempt is made to describe and study a new class of multistage matrix games—games with inertia. The main difference from an ordinary matrix game consists in the fact that at each step the players cannot use arbitrary mixed strategies: they must choose them from certain sets depending on which pure strategies were realized at several preceding steps of the game. Such situations arise, in particular, if the transition from one pure strategy to another requires large additional expenditures of time, resources, etc.
As in the theory of ordinary von Neumann games, the main questions are connected with the definition and clarification of conditions for the existence of the value of the game, with the construction of optimal strategies, with the investigation of the influence of inertia on the value of the game, etc. Let us note at once that, in defining an optimal strategy in long multistage games, each player seeks not only (and sometimes not so much) to guarantee the greatest possible average payoff over many games, or, equivalently, over the games of many players, but proceeds from the desire to ensure, with probability 1, the greatest possible payoff in the given game.
- Below we shall throughout study games with payoff matrix
\[ A=(a_{ij}) \qquad (i=1,\ldots,n;\ j=1,\ldots,n). \]
As usual \((^1)\), one step of the game consists in the first player \(u\) naming the number \(i\) of one of the rows of the matrix \(A\), and the second player \(v\) the number \(j\) of one of the columns; the number \(a_{ij}\) determines the payoff of the first player. By realizations of the multistage game played we shall mean sequences
\[ i_1,\ i_2,\ldots,\ i_s,\ldots;\qquad j_1,\ j_2,\ldots,\ j_s,\ldots \tag{1} \]
of the numbers of rows and columns named by the players.
In an ordinary matrix game the first player at each step may use an arbitrary mixed strategy
\[ p=\{p_1,\ p_2,\ldots,p_n\}. \tag{2} \]
The basic condition determining the inertia of depth \(\chi(u)\) of the player \(u\) is that, after \(k-1\) \((k>\chi(u))\) steps of the game have been realized, player \(u\) at the \(k\)-th step may use only such mixed strategies (2) whose components with indices \(i_{k-\chi(u)},\ldots,\ i_{k-1}\) satisfy the inequalities
\[ p_{i_{k-s}} \geq \alpha_{i_{k-s}}(i_{k-\chi(u)},\ldots,i_{k-1}) \qquad (s=1,\ldots,\chi(u)), \tag{3} \]
where the inertia numbers of player \(u\)
\[ \alpha_{\gamma_s}(\gamma_1,\gamma_2,\ldots,\gamma_{\chi(u)}) \qquad (s=1,\ldots,\chi(u)) \tag{4} \]
are nonnegative. Since the sum of the components in each strategy is equal to 1, for any set \(\gamma_1,\gamma_2,\ldots,\gamma_{\chi(u)}\) of row numbers the sum of the inertia numbers
does not exceed 1. Below it is assumed that the strict inequalities
\[ \sum_{s=1}^{\chi(u)} \alpha_{\gamma_s}(\gamma_1,\gamma_2,\ldots,\gamma_{\chi(u)})<1. \tag{5} \]
Analogously one defines the inertia of depth \(\chi(v)\) and the inertia numbers of player \(v\). Without loss of generality one may assume that \(\chi(u)=\chi(v)=\chi\).
- The rules of the game with inertia of depth \(\chi\) for player \(u\) consist in the following: he chooses, in some way (not playing a role in what follows), a set \(p^0\) of strategies for conducting the first \(\chi\) steps of the game and determines a system \(P\) of mixed strategies
\[ p^{\gamma_1,\ldots,\gamma_\chi}=\{p_1^{\gamma_1,\ldots,\gamma_\chi},\ldots,p_n^{\gamma_1,\ldots,\gamma_\chi}\}, \tag{6} \]
where the indices \(\gamma_i\) independently take any value \(1,\ldots,n\). The mixed strategies \(p^{\gamma_1,\ldots,\gamma_\chi}\) must satisfy the inertia conditions
\[ p_{\gamma_i}^{\gamma_1,\ldots,\gamma_\chi}\geqslant \alpha_{\gamma_i}(\gamma_1,\ldots,\gamma_\chi). \tag{7} \]
Player \(u\), after \(k-1\) (\(k>\chi\)) steps with realizations \(i_1,i_2,\ldots,i_{k-1}\), uses for the next step the mixed strategy \(p^{i_{k-\chi},\ldots,i_{k-1}}\).
Analogously one defines the initial strategy \(q^0\) (for the first \(\chi\) steps of the game) and the system \(Q\) of mixed strategies for the subsequent steps of the game for player \(v\).
- Let the average payoff of player \(u\) over \(s\) steps of the game with realizations (1) be denoted by \(a_s\). Suppose the game is played according to the strategies \(p^0, P\) and \(q^0, Q\). We then denote by \(M_s(p^0,P,q^0,Q)\) the mathematical expectation of the average payoff of player \(u\) over \(s\) steps of the game.
Theorem 1. For any \(p^0, P\) and \(q^0, Q\) there exists the limiting mathematical expectation of the average payoff of player \(u\):
\[ M(p^0,P,q^0,Q)=\lim_{s\to\infty} M_s(p^0,P,q^0,Q). \]
Let us note one difference between games with inertia and ordinary von Neumann games. If the strategies \(q^0,Q\) of player \(v\) are fixed, then the set of those strategies \(p^0,P\) of player \(u\) for which the limiting average payoff exceeds a certain number is, in general, nonconvex!
If there exists a number \(E\) such that
\[ E=\inf_{q^0,Q}\sup_{p^0,P} M(p^0,P,q^0,Q) =\sup_{p^0,P}\inf_{q^0,Q} M(p^0,P,q^0,Q), \]
then this number will be called the value of the game with inertia of depth \(\chi\).
Theorem 2. The value \(E\) of the game with inertia exists and coincides with the value of the ordinary von Neumann game with the same payoff matrix \(A\).
- Suppose a game with realizations (1) has been played. If there exists the limit \(a=\lim a_s\), then we shall call it the limiting average payoff. If the players \(u\) and \(v\) have chosen strategies, then for some realizations the limiting average payoff is undefined; for others it may take different values.
Theorem 3. For any fixed strategies \(p^0,P\) and \(q^0,Q\), the limiting average payoff exists with probability 1.
This simple theorem makes it possible to give a natural definition of optimal strategies for games with inertia.
We shall call a system \(P^*\) of mixed strategies (6) a stably optimal strategy of player \(u\) in the game with inertia if, for any of his initial strategies \(p^0\) and any strategy \(q^0,Q\) of player \(v\), with probability 1 the limiting average payoff is not less than the value \(E\) of the game. The stably optimal strategy \(Q^*\) of player \(v\) is defined analogously.
Theorem 4. There exist stably optimal strategies \(P^*\) and \(Q^*\).
If \(P^*, Q^*\) are stably optimal strategies, then for any initial strategies \(p^0, q^0\) the equality
\(M(p^0, P^*, q^0, Q^*) = E\) holds. But even the fact that, for some strategy \(p^0, P\) and any strategies \(q^0, Q\), the inequality
\(M(p^0, P, q^0, Q) \geq E\) is satisfied does not imply that \(P\) is a stably optimal strategy.
- Let us proceed to the construction of stably optimal strategies.
We begin with games with inertia of depth 1; such games will be called games with one-step inertia.
In the subsequent constructions we essentially use the optimal strategy
\[ p^*=\{p_1^*,p_2^*,\ldots,p_n^*\} \tag{8} \]
of player \(u\) in the Neumann game with payoff matrix \(A\). In a game with one-step inertia, player \(u\) has \(n\) inertia numbers; denote the largest of them by \(\alpha^*\); condition (5) means that \(\alpha^*<1\). Put
\[ P^*(\lambda)= \begin{pmatrix} 1-\lambda+\lambda p_1^* & \lambda p_1^* & \cdots & \lambda p_1^*\\ \lambda p_2^* & 1-\lambda+\lambda p_2^* & \cdots & \lambda p_2^*\\ \cdots & \cdots & \cdots & \cdots\\ \lambda p_n^* & \lambda p_n^* & \cdots & 1-\lambda+\lambda p_n^* \end{pmatrix} \tag{9} \]
where \(\lambda>0\) and \(\lambda\leq 1-\alpha^*\).
Theorem 5. The matrix strategies (9) are stably optimal strategies of player \(u\) in a game with one-step inertia.
- For games with multistep inertia, stably optimal strategies are described considerably more complicatedly.
We restrict ourselves to the case when, in the Neumann optimal strategy (8), all components are strictly positive. Let us merely note that the presence of zeros among the components of vector (8) simplifies the computations.
Denote by \(\Pi\) the set of collections \(\{\gamma_1,\ldots,\gamma_\chi\}\) \((\gamma_i=1,\ldots,n)\), ordered in some way. The number of the collection \(\{\gamma_1,\ldots,\gamma_\chi\}\) will be denoted by \(N(\gamma_1,\ldots,\gamma_\chi)\). From the system (6) of mixed strategies we define the matrix
\(\Phi=(\varphi_{NM})\) \((N,M=1,2,\ldots,n^\chi)\), whose elements
\(\varphi_{N(\delta_1,\ldots,\delta_\chi)\,N(\gamma_1,\ldots,\gamma_\chi)}\) are equal to zero if the collection \(\gamma_2,\ldots,\gamma_\chi\) does not coincide with the collection \(\delta_1,\ldots,\delta_{\chi-1}\), and
\[ \varphi_{N(\delta_1,\ldots,\delta_{\chi-1},\delta_\chi)\,N(\gamma_1,\delta_1,\ldots,\delta_{\chi-1})} = p_{\delta_\chi}^{\gamma_1,\delta_1,\ldots,\delta_{\chi-1}} \tag{10} \]
otherwise. We shall call the matrix \(\Phi\) a matrix strategy.
Let \(\varepsilon\) be a sufficiently small positive number. For each fixed \(l=1,\ldots,n\), define a system (6) of mixed strategies
\({}^{l}p^{\gamma_1,\ldots,\gamma_\chi}\) so that it satisfies the inertia conditions (7) and so that the mixed strategy
\({}^{l}p^{l,\ldots,l}\) has the form
\[ {}^{l}p_l^{\,l,\ldots,l}=1-\varepsilon,\qquad {}^{l}p_j^{\,l,\ldots,l}=\varepsilon/(n-1)\quad \text{for } j\ne l. \tag{11} \]
Each such system of mixed strategies will be regarded as a matrix strategy \(\Phi^l\) of order \(n^\chi\). It can be shown that every matrix strategy \(\Phi^l\) \((l=1,\ldots,n)\) has a unique eigenvector
\(\varphi^l=\{\varphi_1^l,\varphi_2^l,\ldots,\varphi_{n^\chi}^l\}\) with nonnegative components whose sum is 1, corresponding to the eigenvalue 1. Find all eigenvectors \(\varphi^l\) (this is the second step of the algorithm for finding stably optimal strategies).
Denote by \(\tau(\varphi^l)\) \((l=1,\ldots,n)\) the vectors in \(n\)-dimensional space whose components \(\tau(\varphi^l)_i\) are determined by the equality
\[ \tau(\varphi^l)_i= \sum_{\gamma_1=i} \varphi^l_{N(\gamma_1,\ldots,\gamma_\chi)} \qquad (i=1,\ldots,n). \]
For sufficiently small \(\varepsilon\), the component of the vector \(\tau(\varphi^l)\) with number \(l\) will be close to 1. Therefore the optimal Neyman strategy (8) (whose determination is the first step of the algorithm for finding stably optimal strategies) belongs to the convex hull of the points \(\tau(\varphi^1), \ldots, \tau(\varphi^n)\). In other words, one can find positive numbers \(c_1, \ldots, c_n\) such that
\[ p^* = c_1 \tau(\varphi^1) + \cdots + c_n \tau(\varphi^n), \qquad c_1 + \cdots + c_n = 1 \]
(finding the numbers \(c_i\) is the third step in the construction of stably optimal strategies).
We now define the system of strategies (6) by the equalities
\[ p_i^{\gamma_1,\ldots,\gamma_\varkappa} = \sum_{l=1}^{n} c_l^l p_i^{\gamma_1,\ldots,\gamma_\varkappa} \varphi_N^l{}_{(\gamma_1,\ldots,\gamma_\varkappa)} \Big/ \sum_{l=1}^{n} c_l \varphi_N^l{}_{(\gamma_1,\ldots,\gamma_\varkappa)} . \tag{12} \]
Theorem 6. The equalities (12) define a stably optimal strategy of the player in the game with inertia of depth \(\varkappa\).
It is clear from the construction itself that stably optimal strategies are not determined uniquely.
Voronezh State University
Received
30 XII 1968
REFERENCES
- J. McKinsey, Introduction to the Theory of Games, 1960.
- W. Feller, An Introduction to Probability Theory and Its Applications, 1, Moscow, 1967.