Reports of the Academy of Sciences of the USSR
1962, Volume 145, No. 3

MATHEMATICS

An. M. LEONTOVICH

ON THE EXISTENCE OF UNBOUNDED OSCILLATING TRAJECTORIES IN A CERTAIN BILLIARD PROBLEM

(Presented by Academician I. G. Petrovskii on 1 III 1962)

§ 1. The study of the behavior of dynamical systems over unbounded time leads to difficult mathematical questions. One of them is the question of the possible types of escape of trajectories to infinity, in particular, of the existence of unbounded oscillating trajectories (i.e., trajectories which at times go arbitrarily far away and at times return). Recently K. A. Sitnikov \((^1)\) proved the existence of such trajectories in the three-body problem. Since the time of Birkhoff’s works \((^2)\), problems concerning the motion of a material point (henceforth called a ball) on billiard tables of various shapes have been considered as the simplest models of problems of dynamics. In particular, the behavior of a ball on a table going off to infinity is a model of the motion of a particle in fields of certain configurations (“traps”).

In the present paper the existence of unbounded oscillating trajectories is proved under certain restrictions on the shape of the table. In the case where the table narrows without bound at infinity, this proof requires considerations new in comparison with \((^1)\).

We shall consider a table bounded by the straight line \(y=0\) and by a “bell-shaped” curve—the graph of a positive function \(y=f(x)\), continuous together with its first two derivatives for all \(x\) and satisfying the conditions:

\[ \begin{aligned} &1.\quad f(x)>0,\qquad f(0)=1,\qquad f(x)\to A\ge 0 \quad \text{as } |x|\to\infty.\\ &2.\quad f'(x)<0 \quad \text{for } x>0,\qquad f'(x)>0 \quad \text{for } x<0.\\ &3.\quad f''(x)>0 \quad \text{for } |x|>X_1. \end{aligned} \]

In what follows these conditions will always be assumed to be fulfilled.

An infinitely long oriented polygonal line along which the ball moves (being reflected from the boundaries of the table according to the law: the angle of incidence equals the angle of reflection) will be called a trajectory of the ball. The part of a trajectory after or before any fixed point of it will be called a semiorbit. A piece of a trajectory will be called its part between two successive intersections with the axis \(Oy\), as well as the part of a trajectory beginning at the last point of its intersection with the axis \(Oy\). If the path coincides with the axis \(Ox\) or with the segment \([0,1]\) of the axis \(Oy\), then the corresponding trajectory (semiorbit) will be called trivial.

A trajectory (semiorbit) is called unbounded if it cannot be contained in a strip \(x_1\le x\le x_2\) for any \(x_1\) and \(x_2\), \(-\infty<x_1<x_2<+\infty\). It is called oscillating if it intersects the axis \(Oy\) an infinite number of times.

Theorem 1. If \(A>0\), there exist nontrivial unbounded nonoscillating trajectories.

Theorem 2. If \(A=0\), every nontrivial semiorbit is oscillating.

Theorem 3. Let \(A=0\) and, in addition, let the following condition be fulfilled:

1. \(f''(x)\) tends monotonically to zero for \(|x|>X_2\), as \(x\to+\infty\) and \(x\to-\infty\).

Then there exist unbounded oscillating trajectories.

The result of the last theorem could be extended to the case \(A>0\) only under the following additional conditions on the function \(f(x)\):

1. \(f(x)=f(-x)\).

2. The equation \(f'(x)=0\) for \(x>0\) has a unique solution \(x_0\), and \(|f'(x_0)|\geqslant 1\).

Then the following is true.

Theorem 4. If \(A>0\) and conditions 5, 6 are satisfied, then there exist unbounded oscillating trajectories.

§ 2. Let a trajectory intersect the axis \(Oy\) at the point \(y=a\) at an angle \(\varphi\) with the positive direction of the axis \(Ox\), \(0\leqslant a\leqslant 1\), \(|\varphi|\leqslant \pi\) (here and below angles are measured counterclockwise). Denote this trajectory by \(\Gamma_{\varphi a}\), and its piece beginning at the point \(x=0,\ y=a\) with direction \(\varphi\) by \(\Delta_{\varphi a}\), and say that the trajectory \(\Gamma_{\varphi a}\) and the piece \(\Delta_{\varphi a}\) are “represented” by the point \((\varphi,a)\) of the rectangle \(\mathfrak A\): \(|\varphi|\leqslant \pi\), \(0\leqslant a\leqslant 1\). Every trajectory is “represented” by at least one (by virtue of condition 2) point of \(\mathfrak A\), and, if it is nontrivial, then by no more than a countable number of points. Every piece of a trajectory is “represented” by a unique point of the rectangle, if in it one identifies the points \((\pi,a)\) and \((-\pi,a)\); \((\varphi,0)\) and \((-\varphi,0)\); \((\varphi,1)\) and \((-\varphi,1)\). The rectangle \(\mathfrak A\) with identified sides will be called the phase space \(\mathfrak B\). Obviously, \(\mathfrak B\) is homeomorphic to a sphere.

The motion of the ball defines the following mapping \(T\) of the phase space into itself. Suppose that the trajectory has intersected the axis \(Oy\) successively at the points \(y=a\) and \(y=a_1\), at angles \(\varphi\) and \(\varphi_1\) with the axis \(Ox\). Then we set: \(T(\varphi,a)=(\varphi_1,a_1)\). If the trajectory intersected the axis \(Oy\) at the point \(y=a\) at an angle \(\varphi\) with the axis \(Ox\) for the last time, then the point \((\varphi,a)\) will be called outgoing. Denote the set of outgoing points by \(G^{\mathrm y}=G^{\mathrm y}_+\cup G^{\mathrm y}_-\), where \((\varphi,a)\in G^{\mathrm y}_+\) if \(|\varphi|<\pi/2\) (exit to the right), and \((\varphi,a)\in G^{\mathrm y}_-\) if \(|\varphi|>\pi/2\) (exit to the left). If \((\varphi,a)\in G^{\mathrm y}_+\), then set \(T(\varphi,a)=(\pi,0)\), and if \((\varphi,a)\in G^{\mathrm y}_-\), then \(T(\varphi,a)=(0,0)\). Thus the mapping \(T\) is defined for all points of \(\mathfrak B\), except points of the form \((\pm\pi/2,a)\). Obviously, it is discontinuous. Define also the analogous set of incoming points
\(G^{\mathrm p}=G^{\mathrm p}_+\cup G^{\mathrm p}_-\):
\(G^{\mathrm p}_+=T^{-1}\{(\pi,0)\}\), \(G^{\mathrm p}_-=T^{-1}\{(0,0)\}\). It is easy to see that, by virtue of condition 3, \(G^{\mathrm y}_+\), \(G^{\mathrm y}_-\), \(G^{\mathrm p}_+\), \(G^{\mathrm p}_-\) are closed connected sets, each of which is a domain bounded by a simple Jordan curve.

We shall call a point \((\varphi,a)\) and a piece of a trajectory \(\Delta_{\varphi a}\) semiperiodic if the ball on this piece moves from the axis \(Oy\) to the axis \(Oy\) along the very same path; in other words, if \(T(\varphi,a)=(\varphi+\pi,a)\) for \(\varphi\leqslant 0\) and \(T(\varphi,a)=(\varphi-\pi,a)\) for \(\varphi>0\). Denote the set of semiperiodic points by \(\mathfrak M\).

Denote by \(l(\varphi,a)\) the abscissa of the point of the piece \(\Delta_{\varphi a}\) farthest from the axis \(Oy\). We shall call the point \((\varphi,a)\) and the trajectory \(\Gamma_{\varphi a}\) strongly oscillating if
\[ \lim_{i\to\infty}\left|\,l\bigl(T^i(\varphi,a)\bigr)\,\right|=\infty. \]
Denote the set of such points by \(\mathfrak S\).

§ 3. In this section the case \(A>0\) is considered; then the following theorems are true.

Theorem \(1'\). Each of the sets \(G^{\mathrm y}\) and \(G^{\mathrm p}\) contains some neighborhoods of the points \((0,0)\) and \((\pi,0)\).

Theorem \(4'\). If the boundaries of the sets \(G^{\mathrm y}_+\) and \(G^{\mathrm p}_+\) intersect, then strongly oscillating trajectories exist.

The proof of Theorem \(1'\) is almost trivial. Obviously, Theorem 1 follows from it. We do not present the proof of Theorem \(4'\), since it is very close to the arguments of K. A. Sitnikov \({}^{(1)}\).

It is easy to see that in the case when \(f(x)=f(-x)\), if \(G^{\mathrm y}\ne G^{\mathrm p}\), then the boundaries of \(G^{\mathrm y}_+\) and \(G^{\mathrm p}_+\) intersect. Therefore the following is true:

Theorem 4″. If \(f(x)=f(-x)\) and \(G^{y}\ne G^{\mathrm{p}}\), then there exist strongly oscillating trajectories.

In the general case (perhaps always) \(G^{y}\) and \(G^{\mathrm{p}}\) do not coincide or, equivalently in view of condition 5, the set \(G^{y}\) is not symmetric with respect to the line \(\varphi=0\). One sufficient condition for this, as is easy to see, is condition 6. Consequently, Theorem 4 follows from Theorem 4″.

§ 4. In this section the case \(A=0\) is considered.

Let \(X\) be a number greater than \(X_1\) and \(X_2\) and such that, if \(|x|>X\), then
\[ 1.1\,\operatorname{arctg}|f'(x)|>|f'(x)|. \]
Denote by \(U_0:\ |\varphi|<\varphi^*,\ a<a^*\) such a rectangle that, if \((\varphi,a)\in U_0\), then all the points of reflection from the curve on the segment \(\Delta_{\varphi a}\) lie farther from the axis \(Oy\) than \(X\). It is easy to verify that, by virtue of condition 3, such a rectangle exists.

Consider the segment \(\Delta_{\varphi a}\). Denote by \(x^i\) the abscissa of the \(i\)-th reflection of the ball from the axis \(Ox\), and by \(\alpha^i\) the angle formed by the trajectory of the ball with the axis \(Ox\) at this point. The following lemmas are valid; because of lack of space we do not give their proofs.

Lemma 1. Let \((\varphi,a)\in U_0\). If \(f(x^{i-1})<0.5\,f(x^{i+1})\), then
\[ (\alpha^i-\alpha^j)(x^i-x^j)\ge 0.9\,f(x^i). \]

Lemma 2. Let \((\varphi,a)\in U_0\). If \(\alpha^i>2\alpha^j,\ \operatorname{tg}\alpha^i<1.1\,\alpha^i\), then \(f(x^i)<0.9\,f(x^j)\).

Lemma 3. Let \(f(x)\) satisfy condition 4. If \(x_1>x>X,\ f(x)>3f(x_1)\), then
\[ f'(x)/f'(x_1)>4/3. \]

Theorem 2 follows easily from Lemma 1.

Theorem 3 follows from the following assertion.

Theorem 3′. If condition 4 is fulfilled, then there exist strongly oscillating trajectories.

Let us briefly outline the proof of this theorem. For simplicity of exposition we restrict ourselves to the case when \(f(x)=f(-x)\). The phase space \(\mathfrak{B}\), as noted above, is homeomorphic to the sphere \(\Omega\). Let \(\psi:\mathfrak{B}\to\Omega\) be a smooth mapping such that the lines in \(\mathfrak{B}\) \(\varphi=0,\ \varphi=\pi,\ a=0,\ a=1\) go into circles of one and the same great circle (obviously, such a mapping exists). It maps the lines \(\varphi=\mathrm{const},\ a=\mathrm{const}\) into certain smooth lines on the sphere; thereby a coordinate system \((\varphi,a)\) is introduced on it.

We shall say that an arc \(FG\) of a curve on the sphere forms a positive (negative) spiral if, as the point \(P\) moves along the arc from \(F\) to \(G\), \(\angle FOP\) decreases (increases) monotonically. (Here \(O=(0,0)\); \(\angle FOP\) is the angle between the arcs \(FO\), \(OP\) of great circles.) The arc \(FG\) consists of \(k\) turns of a spiral if \(\angle FOG=+2\pi k\). The arc \(FG\) lies below the line \(a=a(\varphi)\) if the second coordinate of any point of the arc is less than \(a(\varphi)\).

Geometric lemma. Suppose that on the sphere \(\Omega\) there are given: 1) a continuous transformation \(S\); 2) a monotonically decreasing function \(\varphi=\gamma(a)\), \(\gamma(0)=0\); 3) a neighborhood \(V\) of the point \((0,0)\). Let
\[ \mathfrak{N}=\{(\varphi,a):\gamma(a)\le \varphi\le 0\}. \]
Consider the arcs \(FG\) and \(KL\) of the lines \(a=\mathrm{const}=\bar a\) and \(\varphi=\mathrm{const}=\bar\varphi\), where
\[ F=(\tfrac12\gamma(\bar a),\bar a),\quad G=(\gamma(\bar a),\bar a);\quad K=(\bar\varphi,a^1),\quad L=(\bar\varphi,a^2),\quad a^1>a^2>0; \]
\[ F,G,K,L\in V\cap\mathfrak{N}. \]
Assume that the following conditions are fulfilled:

A. \(S(0,0)=(0,0)\).

B. \(F'G'=S(FG)\) contains at least two turns of a positive spiral.

C. At least from one turn of this spiral the region \(\mathfrak{N}\) cuts out an arc lying below the line \(a=\bar a\) and representing the graph of a monotonically nondecreasing function \(a\) of \(\varphi\).

D. \(K'L'=S(KL)\) forms a positive spiral.

Then there exist points \((\varphi,a)\) such that \(S^i(\varphi,a)\to(0,0)\) as \(i\to\infty\).

The proof of this lemma is not difficult; we omit it.

Apply this lemma to our case. As \(S\) we take such a transforma-

condition: if \(T(\varphi,a)=(\varphi_1,a_1)\), then \(S(\varphi,a)=(\pi-\varphi_1,a_1)\) for \(\varphi_1\leqslant 0\) and \(S(\varphi,a)=(-\pi-\varphi_1,a_1)\) for \(\varphi_1>0\). As \(\gamma(a)\) we take the following function: from the point \(x=0,\ y=a\) draw such a tangent to the curve \(y=f(x)\) that it nowhere intersects the curve for \(x>0\); \(\gamma(a)\) denotes the angle between this tangent and the \(Ox\)-axis. It is not difficult to verify that on an arc of the line \(a=\mathrm{const}=\bar a,\ \bar a>0\), lying between the graphs of the functions \(\varphi=-\frac12\gamma(\bar a)\) and \(\varphi=\gamma(\bar a)\), there is only a finite number of semiperiodic points. Denote the number of these points by \(N(\bar a)\). From Lemmas 2 and 3 it follows quite simply that \(N(\bar a)\to\infty\) as \(\bar a\to 0\), i.e., there exists \(a^\vee\) such that, if \(\bar a<a^\vee\), then \(N(\bar a)\geqslant 2\). As \(V\) we take the intersection of the rectangle \(U_0\) and the rectangle \(|\varphi|\leqslant\gamma(a^\vee),\ a<a^\vee\).

We shall show that for \(S,\gamma,V\), introduced in this way, the conditions A–Г are valid. Let us verify that conditions Б, В are fulfilled. Let \(F=(\varphi^1,a)\) and \(G=(\varphi^2,a)\) be two neighboring semiperiodic points (i.e., for \(\varphi^1>\varphi>\varphi^2\) the point \((\varphi,a)\notin\mathfrak M\)), belonging to \(U_0\cap\mathfrak M\). Consider \(F'G'=S(FG)\). It is easy to see that the mapping \(\psi\) can be chosen so that \(F'G'\) contains one turn of a positive spiral, and the intersection of this spiral with the set \(\mathfrak M\) satisfies condition В. Since \(N(a)\to\infty\) as \(a\to 0\), it follows easily that in the neighborhood \(V\) conditions Б, В are fulfilled. Conditions A, Г are very simple to check. Hence it follows that there exist points \((\varphi,a)\) such that \(S^i(\varphi,a)\to(0,0)\) as \(i\to\infty\), from which Theorem \(3'\) follows quite easily. The general case, when condition 5 is not necessarily fulfilled, is treated in almost the same way (instead of \(S\) one must take \(T^2\)).

§ 5. Of interest is the question of the structure of the set \(\mathfrak H\) of strongly oscillating points. From the proof of Theorem \(3'\) it is seen that in some neighborhoods of the points \((0,0)\) and \((\pi,0)\) the intersection of \(\mathfrak H\) and the segment \(a=\mathrm{const}\) contains a Cantor set. The set \(\mathfrak H\) is arranged analogously in some neighborhood of the boundary \(G\) in the case \(A>0\).

Unfortunately, there remains an unresolved interesting question about the Lebesgue measure of the set of points “representing” unbounded trajectories. In some cases (perhaps always) it does not coincide with the measure of the entire phase space and, possibly, in the case \(A=0\) is necessarily equal to zero.

Let us make several further remarks on the motion of a ball on unbounded billiard tables of arbitrary form. The tables considered in this note possess the following property:

\(\beta)\) There exists a constant \(L>0\) such that, whatever the points \(P,Q\) of the table may be, there is a trajectory passing through the points \(P,Q\) of length less than \(L\rho\), where \(\rho\) is the shortest distance between the points \(P,Q\) on the table.

A table bounded by the curves \(y=2|x|\), \(y=2\sqrt{x^2+1}\) does not possess property \(\beta)\). For such tables, strongly oscillating trajectories, generally speaking, do not exist. Unbounded oscillating trajectories may perhaps exist.

Property \(\beta)\) is possessed not only by the tables considered in §§ 1–4, but also by many others, for example the table bounded by the curves \(y=x^2\) and \(y=x^2+1\). I believe that for such billiards the existence of strongly oscillating trajectories can be proved by approximately the same methods as in § 4.

In conclusion I express my deep gratitude to V. I. Arnol’d for posing the problem and for assistance in preparing the article for publication.

Moscow State University
named after M. V. Lomonosov

Received
27 II 1962

REFERENCES

1. K. A. Sitnikov, DAN, 133, No. 2, 303 (1960).
2. G. Birkhoff, Dynamical Systems, Moscow–Leningrad, 1941, Ch. VI.