Halting problem

History

In April 1936, Alonzo Church published his proof of the undecidability of a problem in the lambda calculus. Turing's proof was published later, in January 1937. Since then, many other undecidable problems have been described, including the halting problem which emerged in the 1950s.

Timeline

• 1900: David Hilbert poses his "23 questions" (now known as Hilbert's problems) at the Second International Congress of Mathematicians in Paris. "Of these, the second was that of proving the consistency of the 'Peano axioms' on which, as he had shown, the rigour of mathematics depended".
• 1920–1921: Emil Post explores the halting problem for tag systems, regarding it as a candidate for unsolvability. Its unsolvability was not established until much later, by Marvin Minsky.
• 1928: Hilbert recasts his 'Second Problem' at the Bologna International Congress. He posed three questions: i.e. #1: Was mathematics complete? #2: Was mathematics consistent? #3: Was mathematics decidable? The third question is known as the Entscheidungsproblem (Decision Problem).
• 1930: Kurt Gödel announces a proof as an answer to the first two of Hilbert's 1928 questions. "At first he [Hilbert] was only angry and frustrated, but then he began to try to deal constructively with the problem... Gödel himself felt—and expressed the thought in his paper—that his work did not contradict Hilbert's formalistic point of view".
• 1931: Gödel publishes "On Formally Undecidable Propositions of Principia Mathematica and Related Systems I".
• 19 April 1935: Alonzo Church publishes "An Unsolvable Problem of Elementary Number Theory", which proposes that the intuitive notion of an effectively calculable function can be formalized by the general recursive functions or equivalently by the lambda-definable functions. He proves that the halting problem for lambda calculus (i.e., whether a given lambda-expression has a normal form) is not effectively calculable.
• 1936: Church publishes the first proof that the Entscheidungsproblem is unsolvable, using a notion of calculation by recursive functions.
• 7 October 1936: Emil Post's paper "Finite Combinatory Processes. Formulation I" is received. Post adds to his "process" an instruction "(C) Stop". He called such a process "type 1 ... if the process it determines terminates for each specific problem."
• May 1936 – January 1937: Alan Turing's paper On Computable Numbers With an Application to the Entscheidungsproblem goes to press in May 1936 and reaches print in January 1937. Turing proves three problems undecidable: the "satisfaction" problem, the "printing" problem, and the Entscheidungsproblem. Turing's proof differs from Church's by introducing the notion of computation by machine. This is one of the "first examples of decision problems proved unsolvable".
• 1939: J. Barkley Rosser observes the essential equivalence of "effective method" defined by Gödel, Church, and Turing.
• 1943: In a paper, Stephen Kleene states that "In setting up a complete algorithmic theory, what we do is describe a procedure ... which procedure necessarily terminates and in such manner that from the outcome we can read a definite answer, 'Yes' or 'No,' to the question, 'Is the predicate value true?'."
• 1952: Kleene includes a discussion of the unsolvability of the halting problem for Turing machines and reformulates it in terms of machines that "eventually stop", i.e. halt: "...there is no algorithm for deciding whether any given machine, when started from any given situation, eventually stops."
• 1952: Martin Davis uses the term 'halting problem' in a series of lectures at the Control Systems Laboratory at the University of Illinois in 1952. It is likely that this is the first such use of the term.

Origin of the halting problem

Many papers and textbooks refer the definition and proof of undecidability of the halting problem to Turing's 1936 paper. However, this is not correct. Turing did not use the terms "halt" or "halting" in any of his published works, including his 1936 paper. A search of the academic literature from 1936 to 1958 showed that the first published material using the term "halting problem" was Rogers (1957). However, Rogers says he had a draft of Davis (1958) available to him, and Martin Davis states in the introduction that "the expert will perhaps find some novelty in the arrangement and treatment of topics", so the terminology must be attributed to Davis. Davis stated in a letter that he had been referring to the halting problem since 1952. The usage in Davis's book is as follows:

"[...] we wish to determine whether or not [a Turing machine] Z, if placed in a given initial state, will eventually halt. We call this problem the halting problem for Z. [...]

Theorem 2.2 There exists a Turing machine whose halting problem is recursively unsolvable.

A related problem is the printing problem for a simple Turing machine Z with respect to a symbol S_i".

A possible precursor to Davis's formulation is Kleene's 1952 statement, which differs only in wording:

there is no algorithm for deciding whether any given machine, when started from any given situation, eventually stops.

The halting problem is Turing equivalent to both Davis's printing problem ("does a Turing machine starting from a given state ever print a given symbol?") and to the printing problem considered in Turing's 1936 paper ("does a Turing machine starting from a blank tape ever print a given symbol?"). However, Turing equivalence is rather loose and does not mean that the two problems are the same. There are machines which print but do not halt, and halt but not print. The printing and halting problems address different issues and exhibit important conceptual and technical differences. Thus, Davis was simply being modest when he said:

It might also be mentioned that the unsolvability of essentially these problems was first obtained by Turing.

Formalization

In theoretical computer science, a decision problem is any problem that can be phrased as a yes–no question about a mathematical object. Formally, the halting problem is the decision problem:

Given the description of a program (P) and an input (x), does (P(x)) eventually halt?

The conventional representation of decision problems is the set of objects possessing the property in question. The halting set

K = {(i, x) | program i halts when run on input x}

represents the halting problem.

This set is recursively enumerable, which means there is a computable function that lists all of the pairs (i, x) it contains. However, the complement of this set is not recursively enumerable.

Undecidability

A decision problem is said to be decidable if there exists an algorithm that always halts with the correct answer, and an undecidable problem if no such algorithm exists. In his original proof Turing formalized the concept of algorithm by introducing Turing machines. However, the result is in no way specific to them; it applies equally to any other model of computation that is equivalent in its computational power to Turing machines, such as Markov algorithms, Lambda calculus, Post systems, register machines, or tag systems.

What is important is that the formalization allows a straightforward mapping of algorithms to some data type that the algorithm can operate upon. For example, if the formalism lets algorithms define functions over strings (such as Turing machines) then there should be a mapping of these algorithms to strings, and if the formalism lets algorithms define functions over natural numbers (such as computable functions) then there should be a mapping of algorithms to natural numbers. The mapping to strings is usually the most straightforward, but strings over an alphabet with n characters can also be mapped to numbers by interpreting them as numbers in an n-ary numeral system.

There are many undecidable problems; any set whose Turing degree equals that of the halting problem is such a formulation. Examples of such sets include, besides the halting problem:

• {i | program i eventually halts when run with input 0}
• {i | there is an input x such that program i eventually halts when run with input x}.

Proof concept

Christopher Strachey outlined a proof by contradiction that the halting problem is not solvable. The proof proceeds as follows: Suppose that there exists a total computable function halts(f) that returns true if the subroutine f halts (when run with no inputs) and returns false otherwise. Now consider the following subroutine:

<span id="mwAY8"></span><span class="k" id="mwAZA">def</span><span class="w" id="mwAZE"> </span><span class="nf" id="mwAZI">g</span><span class="p" id="mwAZM">()</span> <span class="o" id="mwAZQ">-></span> <span class="kc" id="mwAZU">None</span><span class="p" id="mwAZY">:</span> <span class="k" id="mwAZc">if</span> <span class="n" id="mwAZg">halts</span><span class="p" id="mwAZk">(</span><span class="n" id="mwAZo">g</span><span class="p" id="mwAZs">):</span> <span class="n" id="mwAZw">loop_forever</span><span class="p" id="mwAZ0">()</span>

halts(g) must either return true or false, because halts was assumed to be total. If halts(g) returns true, then g will call loop_forever and never halt, which is a contradiction. If halts(g) returns false, then g will halt, because it will not call loop_forever; this is also a contradiction. Overall, g does the opposite of what halts says g should do, so halts(g) can not return a truth value that is consistent with whether g halts. Therefore, the initial assumption that halts is a total computable function must be false.

Sketch of rigorous proof

The concept above shows the general method of the proof, but the computable function halts does not directly take a subroutine as an argument; instead it takes the source code of a program. Moreover, the definition of g is self-referential. A rigorous proof addresses these issues. The overall goal is to show that there is no total computable function that decides whether an arbitrary program i halts on arbitrary input x; that is, the following function h (for "halts") is not computable:

${\displaystyle h(i,x)=\{\begin{array}{ll}1& \text{if }\text{ program }i\text{ halts on input }x,\\ 0& \text{otherwise.}\end{array}}$

Here program i refers to the i th program in an enumeration of all the programs of a fixed Turing-complete model of computation.

Possible values for a total computable function f arranged in a 2D array. The orange cells are the diagonal. The values of f(i,i) and g(i) are shown at the bottom; U indicates that the function g is undefined for a particular input value.

The proof proceeds by directly establishing that no total computable function with two arguments can be the required function h. As in the sketch of the concept, given any total computable binary function f, the following partial function g is also computable by some program e:

${\displaystyle g(i)=\{\begin{array}{ll}0& \text{if }f(i,i)=0,\\ \text{undefined}& \text{otherwise.}\end{array}}$

The verification that g is computable relies on the following constructs (or their equivalents):

• computable subprograms (the program that computes f is a subprogram in program e),
• duplication of values (program e computes the inputs i,i for f from the input i for g),
• conditional branching (program e selects between two results depending on the value it computes for f(i,i)),
• not producing a defined result (for example, by looping forever),
• returning a value of 0.

The following pseudocode for e illustrates a straightforward way to compute g:

<span id="mwAlU"></span><span class="k" id="mwAlY">procedure</span><span class="w" id="mwAlc"> </span><span class="nf" id="mwAlg">e</span><span class="p" id="mwAlk">(</span><span class="n" id="mwAlo">i</span><span class="p" id="mwAls">)</span><span class="o" id="mwAlw">:</span> <span class="w" id="mwAl0"> </span><span class="k" id="mwAl4">if</span><span class="w" id="mwAl8"> </span><span class="n" id="mwAmA">f</span><span class="p" id="mwAmE">(</span><span class="n" id="mwAmI">i</span><span class="o" id="mwAmM">,</span><span class="w" id="mwAmQ"> </span><span class="n" id="mwAmU">i</span><span class="p" id="mwAmY">)</span><span class="w" id="mwAmc"> </span><span class="o" id="mwAmg">==</span><span class="w" id="mwAmk"> </span><span class="mi" id="mwAmo">0</span><span class="w" id="mwAms"> </span><span class="k" id="mwAmw">then</span> <span class="w" id="mwAm0"> </span><span class="n" id="mwAm4">return</span><span class="w" id="mwAm8"> </span><span class="mi" id="mwAnA">0</span> <span class="w" id="mwAnE"> </span><span class="k" id="mwAnI">else</span> <span class="w" id="mwAnM"> </span><span class="n" id="mwAnQ">loop</span><span class="w" id="mwAnU"> </span><span class="n" id="mwAnY">forever</span>

Because g is partial computable, there must be a program e that computes g, by the assumption that the model of computation is Turing-complete. This program is one of all the programs on which the halting function h is defined. The next step of the proof shows that h(e,e) will not have the same value as f(e,e).

It follows from the definition of g that exactly one of the following two cases must hold:

• f(e,e) = 0 and so g(e) = 0. In this case program e halts on input e, so h(e,e) = 1.
• f(e,e) ≠ 0 and so g(e) is undefined. In this case program e does not halt on input e, so h(e,e) = 0.

In either case, f cannot be the same function as h. Because f was an arbitrary total computable function with two arguments, all such functions must differ from h.

This proof is analogous to Cantor's diagonal argument. One may visualize a two-dimensional array with one column and one row for each natural number, as indicated in the table above. The value of f(i,j) is placed at column i, row j. Because f is assumed to be a total computable function, any element of the array can be calculated using f. The construction of the function g can be visualized using the main diagonal of this array. If the array has a 0 at position (i,i), then g(i) is 0. Otherwise, g(i) is undefined. The contradiction comes from the fact that there is some column e of the array corresponding to g itself. Now assume f was the halting function h. If g(e) is defined (g(e) = 0 in this case), then the program e halts on input e, thus f(e,e) = 1. But g(e) = 0 only when f(e,e) = 0, contradicting f(e,e) = 1. Similarly, if g(e) is not defined, then program e does not halt on input e, thus f(e,e) = 0, which leads to g(e) = 0 under g's construction. This contradicts the assumption of g(e) not being defined. In both cases contradiction arises. Therefore any arbitrary computable function f cannot be the halting function h.

Computability theory

In computability theory and computational complexity theory, a reduction is an algorithm for transforming one computational problem into another problem. A sufficiently efficient reduction from one problem to another may be used to show that the second problem is at least as difficult as the first. There are two main types of reductions used in computational complexity theory, the many-one reduction and the Turing reduction. A set is many-one reducible to the halting problem if and only if it is recursively enumerable. This says that with regards to many-one reducibility, the halting problem is in the most complicated class of recursively enumerable problems. Thus the halting problem is r.e. complete. Note that it is not the only r.e. complete problem. The Turing degree groups equivalent problems into classes and the degree of the halting problem is 0′ (the "jump" of class 0) representing the class of recursively enumerable, but not recursive, sets of numbers, or similarly ${\displaystyle {\mathrm{\Sigma }}_1^0}$ of the arithmetical hierarchy. Rice's theorem establishes that all non-trivial semantic properties of programs are undecidable and therefore in the halting problem's complexity class or a harder class.

Gregory Chaitin has defined a halting probability, represented by the symbol Ω, a real number that represents the probability that a randomly produced program halts. It is a normal and transcendental number which can be defined but cannot be completely computed. The task of computing the digits of this number has the same Turing degree as the halting problem. This means one can prove that there is no algorithm which produces the digits of Ω, although its first few digits can be calculated in simple cases.

Since the negative answer to the halting problem shows that there are problems that cannot be solved by a Turing machine, the Church–Turing thesis limits what can be accomplished by any machine that implements effective methods. However, not all machines conceivable to human imagination are subject to the Church–Turing thesis (e.g. oracle machines). It is an open question whether there can be actual deterministic physical processes that, in the long run, elude simulation by a Turing machine, and in particular whether any such hypothetical process could usefully be harnessed in the form of a calculating machine (a hypercomputer) that could solve the halting problem for a Turing machine amongst other things. It is also an open question whether any such unknown physical processes are involved in the working of the human brain, and whether humans can solve the halting problem.

Approximations

Turing's proof shows that there can be no mechanical, general method (i.e., a Turing machine or a program in some equivalent model of computation) to determine whether algorithms halt. However, each individual instance of the halting problem has a definitive answer, which may or may not be practically computable. Given a specific algorithm and input, one can often show that it halts or does not halt, and in fact computer scientists often do just that as part of a correctness proof. There are some heuristics that can be used in an automated fashion to attempt to construct a proof, which frequently succeed on typical programs. This field of research is known as automated termination analysis.

Some results have been established on the theoretical performance of halting problem heuristics, in particular the fraction of programs of a given size that may be correctly classified by a recursive algorithm. These results do not give precise numbers because the fractions are uncomputable and also highly dependent on the choice of program encoding used to determine "size". For example, consider classifying programs by their number of states and using a specific "Turing semi-infinite tape" model of computation that errors (without halting) if the program runs off the left side of the tape. Then ${\displaystyle \underset{n\to \mathrm{\infty }}{lim}P(x\text{halts is decidable}\mid x\text{has}n\text{states})=1}$, over programs ${\displaystyle x}$ chosen uniformly by number of states. But this result is in some sense "trivial" because these decidable programs are simply the ones that fall off the tape, and the heuristic is simply to predict not halting due to error. Thus a seemingly irrelevant detail, namely the treatment of programs with errors, can turn out to be the deciding factor in determining the fraction of programs.

To avoid these issues, several restricted notions of the "size" of a program have been developed. A dense Gödel numbering assigns numbers to programs such that each computable function occurs a positive fraction in each sequence of indices from 1 to n, i.e. a Gödelization φ is dense iff for all ${\displaystyle i}$, there exists a ${\displaystyle c>0}$ such that ${\displaystyle \underset{n\to \mathrm{\infty }}{lim inf}\mathrm{\#}\{j\in \mathbb{N}:0\le j<n,{\varphi }_i={\varphi }_j\}/n\ge c}$. For example, a numbering that assigns indices ${\displaystyle 2^n}$ to nontrivial programs and all other indices the error state is not dense, but there exists a dense Gödel numbering of syntactically correct Brainfuck programs. A dense Gödel numbering is called optimal if, for any other Gödel numbering ${\displaystyle \alpha }$, there is a 1-1 total recursive function ${\displaystyle f}$ and a constant ${\displaystyle c}$ such that for all ${\displaystyle i}$, ${\displaystyle {\alpha }_i={\varphi }_{f(i)}}$ and ${\displaystyle f(i)\le ci}$. This condition ensures that all programs have indices not much larger than their indices in any other Gödel numbering. Optimal Gödel numberings are constructed by numbering the inputs of a universal Turing machine. A third notion of size uses universal machines operating on binary strings and measures the length of the string needed to describe the input program. A universal machine U is a machine for which every other machine V there exists a total computable function h such that ${\displaystyle V(x)=U(h(x))}$. An optimal machine is a universal machine that achieves the Kolmogorov complexity invariance bound, i.e. for every machine V, there exists c such that for all outputs x, if a V-program of length n outputs x, then there exists a U-program of at most length ${\displaystyle n+c}$ outputting x.

We consider partial computable functions (algorithms) ${\displaystyle A}$. For each ${\displaystyle n}$ we consider the fraction ${\displaystyle {ϵ}_n(A)}$ of errors among all programs of size metric at most ${\displaystyle n}$, counting each program ${\displaystyle x}$ for which ${\displaystyle A}$ fails to terminate, produces a "don't know" answer, or produces a wrong answer, i.e. ${\displaystyle x}$ halts and ${\displaystyle A(x)}$ outputs DOES_NOT_HALT, or ${\displaystyle x}$ does not halt and ${\displaystyle A(x)}$ outputs HALTS. The behavior may be described as follows, for dense Gödelizations and optimal machines:

• For every algorithm ${\displaystyle A}$, ${\displaystyle \underset{n\to \mathrm{\infty }}{lim inf}{ϵ}_n(A)>0}$. In words, any algorithm has a positive minimum error rate, even as the size of the problem becomes extremely large.
• There exists ${\displaystyle ϵ>0}$ such that for every algorithm ${\displaystyle A}$, ${\displaystyle \underset{n\to \mathrm{\infty }}{lim sup}{ϵ}_n(A)\ge ϵ}$. In words, there is a positive error rate for which any algorithm will do worse than that error rate arbitrarily often, even as the size of the problem grows indefinitely.
• ${\displaystyle \underset{A}{inf}\underset{n\to \mathrm{\infty }}{lim inf}{ϵ}_n(A)=0}$. In words, there is a sequence of algorithms such that the error rate gets arbitrarily close to zero for a specific sequence of increasing sizes. However, this result allows sequences of algorithms that produce wrong answers.
• If we consider only "honest" algorithms that may be undefined but never produce wrong answers, then depending on the metric, ${\displaystyle \underset{A\text{honest}}{inf}\underset{n\to \mathrm{\infty }}{lim inf}{ϵ}_n(A)}$ may or may not be 0. In particular it is 0 for left-total universal machines, but for effectively optimal machines it is greater than 0.

The complex nature of these bounds is due to the oscillatory behavior of ${\displaystyle {ϵ}_n(A)}$. There are infrequently occurring new varieties of programs that come in arbitrarily large "blocks", and a constantly growing fraction of repeats. If the blocks of new varieties are fully included, the error rate is at least ${\displaystyle ϵ}$, but between blocks the fraction of correctly categorized repeats can be arbitrarily high. In particular a "tally" heuristic that simply remembers the first N inputs and recognizes their equivalents allows reaching an arbitrarily low error rate infinitely often.

Gödel's incompleteness theorems

The concepts raised by Gödel's incompleteness theorems are very similar to those raised by the halting problem, and the proofs are quite similar. In fact, a weaker form of the First Incompleteness Theorem is an easy consequence of the undecidability of the halting problem. This weaker form differs from the standard statement of the incompleteness theorem by asserting that an effective axiomatization of the natural numbers that is both complete and sound is impossible. The "sound" part is the weakening: it means that we require the axiomatic system in question to prove only true statements about natural numbers. Since soundness implies consistency, this weaker form can be seen as a corollary of the strong form. It is important to observe that the statement of the standard form of Gödel's First Incompleteness Theorem is completely unconcerned with the truth value of a statement, but only concerns the issue of whether it is possible to find it through a mathematical proof.

The weaker form of the theorem can be proved from the undecidability of the halting problem as follows. Assume that we have a sound (and hence consistent) and complete effective axiomatization of all true first-order logic statements about natural numbers. Then we can build an algorithm that enumerates all these statements. This means that there is an algorithm N(n) that, given a natural number n, computes a true first-order logic statement about natural numbers, and that for all true statements, there is at least one n such that N(n) yields that statement. Now suppose we want to decide if the algorithm with representation a halts on input i. We know that this statement can be expressed with a first-order logic statement, say H(a, i). Since the axiomatization is complete it follows that either there is an n such that N(n) = H(a, i) or there is an n′ such that N(n′) = ¬ H(a, i). So if we iterate over all n until we either find H(a, i) or its negation, we will always halt, and furthermore, the answer it gives us will be true (by soundness). This means that this gives us an algorithm to decide the halting problem. Since we know that there cannot be such an algorithm, it follows that the assumption that there is a sound and complete effective axiomatization of all true first-order logic statements about natural numbers must be false.

Generalization

Many variants of the halting problem can be found in computability textbooks. Typically, these problems are RE-complete and describe sets of complexity ${\displaystyle {\mathrm{\Sigma }}_1^0}$ in the arithmetical hierarchy, the same as the standard halting problem. The variants are thus undecidable, and the standard halting problem reduces to each variant and vice-versa. However, some variants have a higher degree of unsolvability and cannot be reduced to the standard halting problem. The next two examples are common.

Halting on all inputs

The universal halting problem, also known (in recursion theory) as totality, is the problem of determining whether a given computer program will halt for every input (the name totality comes from the equivalent question of whether the computed function is total). This problem is not only undecidable, as the halting problem is, but highly undecidable. In terms of the arithmetical hierarchy, it is ${\displaystyle {\mathrm{\Pi }}_2^0}$-complete.

This means, in particular, that it cannot be decided even with an oracle for the halting problem.

Recognizing partial solutions

There are many programs that, for some inputs, return a correct answer to the halting problem, while for other inputs they do not return an answer at all. However the problem "given program p, is it a partial halting solver" (in the sense described) is at least as hard as the halting problem. To see this, assume that there is an algorithm PHSR ("partial halting solver recognizer") to do that. Then it can be used to solve the halting problem, as follows: To test whether input program x halts on y, construct a program p that on input (x,y) reports true and diverges on all other inputs. Then test p with PHSR.

The above argument is a reduction of the halting problem to PHS recognition, and in the same manner, harder problems such as halting on all inputs can also be reduced, implying that PHS recognition is not only undecidable, but higher in the arithmetical hierarchy, specifically ${\displaystyle {\mathrm{\Pi }}_2^0}$-complete.

Lossy computation

A lossy Turing machine is a Turing machine in which part of the tape may non-deterministically disappear. The halting problem is decidable for a lossy Turing machine but non-primitive recursive.

Oracle machines

A machine with an oracle for the halting problem can determine whether particular Turing machines will halt on particular inputs, but they cannot determine, in general, whether machines equivalent to themselves will halt.