Formal language

Words over an alphabet

An alphabet, in the context of formal languages, can be any set; its elements are called letters. An alphabet may contain an infinite number of elements; however, most definitions in formal language theory specify alphabets with a finite number of elements, and many results apply only to them. It often makes sense to use an alphabet in the usual sense of the word, or more generally any finite character encoding such as ASCII or Unicode.

A word over an alphabet can be any finite sequence (i.e., string) of letters. The set of all words over an alphabet Σ is usually denoted by Σ^* (using the Kleene star). The length of a word is the number of letters it is composed of. For any alphabet, there is only one word of length 0, the empty word, which is often denoted by e, ε, λ or even Λ. By concatenation one can combine two words to form a new word, whose length is the sum of the lengths of the original words. The result of concatenating a word with the empty word is the original word.

In some applications, especially in logic, the alphabet is also known as the vocabulary and words are known as formulas or sentences; this breaks the letter/word metaphor and replaces it by a word/sentence metaphor.

Definition

Given a non-empty set ${\displaystyle \mathrm{\Sigma }}$, a formal language ${\displaystyle L}$ over ${\displaystyle \mathrm{\Sigma }}$ is a subset of ${\displaystyle {\mathrm{\Sigma }}^{\ast }}$, where ${\displaystyle {\mathrm{\Sigma }}^{\ast }}$ is the set of all possible finite-length words over ${\displaystyle \mathrm{\Sigma }}$. We call the set ${\displaystyle \mathrm{\Sigma }}$ the alphabet of ${\displaystyle L}$. On the other hand, given a formal language ${\displaystyle L}$ over ${\displaystyle \mathrm{\Sigma }}$, a word ${\displaystyle w\in {\mathrm{\Sigma }}^{\ast }}$ is well-formed if ${\displaystyle w\in L}$. Similarly, an expression ${\displaystyle E\subseteq {\mathrm{\Sigma }}^{\ast }}$ is well-formed if ${\displaystyle E\subseteq L}$. Sometimes, a formal language ${\displaystyle L}$ over ${\displaystyle \mathrm{\Sigma }}$ has a set of clear rules and constraints for the creation of all possible well-formed words from ${\displaystyle {\mathrm{\Sigma }}^{\ast }}$.

In computer science and mathematics, which do not usually deal with natural languages, the adjective "formal" is often omitted as redundant. On the other hand, we can just say "a formal language ${\displaystyle L}$" when its alphabet ${\displaystyle \mathrm{\Sigma }}$ is clear in the context.

While formal language theory usually concerns itself with formal languages that are described by some syntactic rules, the actual definition of the concept "formal language" is only as above: a (possibly infinite) set of finite-length strings composed from a given alphabet, no more and no less. In practice, there are many languages that can be described by rules, such as regular languages or context-free languages. The notion of a formal grammar may be closer to the intuitive concept of a "language", one described by syntactic rules. By an abuse of the definition, a particular formal language is often thought of as being accompanied with a formal grammar that describes it.

Examples

The following rules describe a formal language L over the alphabet Σ = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, +, =}:

• Every nonempty string that does not contain "+" or "=" and does not start with "0" is in L.
• The string "0" is in L.
• A string containing "=" is in L if and only if there is exactly one "=", and it separates two valid strings of L.
• A string containing "+" but not "=" is in L if and only if every "+" in the string separates two valid strings of L.
• No string is in L other than those implied by the previous rules.

Under these rules, the string "23+4=555" is in L, but the string "=234=+" is not. This formal language expresses natural numbers, well-formed additions, and well-formed addition equalities, but it expresses only what they look like (their syntax), not what they mean (semantics). For instance, nowhere in these rules is there any indication that "0" means the number zero, "+" means addition, "23+4=555" is false, etc.

Constructions

For finite languages, one can explicitly enumerate all well-formed words. For example, we can describe a language L as just L = {a, b, ab, cba}. The degenerate case of this construction is the empty language, which contains no words at all (L = ∅).

However, even over a finite (non-empty) alphabet such as Σ = {a, b} there are an infinite number of finite-length words that can potentially be expressed: "a", "abb", "ababba", "aaababbbbaab", .... Therefore, formal languages are typically infinite, and describing an infinite formal language is not as simple as writing L = {a, b, ab, cba}. Here are some examples of formal languages:

• L = Σ^*, the set of all words over Σ;
• L = {a}^* = {aⁿ}, where n ranges over the natural numbers and "aⁿ" means "a" repeated n times (this is the set of words consisting only of the symbol "a");
• the set of syntactically correct programs in a given programming language (the syntax of which is usually defined by a context-free grammar);
• the set of inputs upon which a certain Turing machine halts; or
• the set of maximal strings of alphanumeric ASCII characters on this line, i.e.,
  the set {the, set, of, maximal, strings, alphanumeric, ASCII, characters, on, this, line, i, e}.

Language-specification formalisms

Formal languages are used as tools in multiple disciplines. However, formal language theory rarely concerns itself with particular languages (except as examples), but is mainly concerned with the study of various types of formalisms to describe languages. For instance, a language can be given as

• those strings generated by some formal grammar;
• those strings described or matched by a particular regular expression;
• those strings accepted by some automaton, such as a Turing machine or finite-state automaton;
• those strings for which some decision procedure (an algorithm that asks a sequence of related YES/NO questions) produces the answer YES.

Typical questions asked about such formalisms include:

• What is their expressive power? (Can formalism X describe every language that formalism Y can describe? Can it describe other languages?)
• What is their recognizability? (How difficult is it to decide whether a given word belongs to a language described by formalism X?)
• What is their comparability? (How difficult is it to decide whether two languages, one described in formalism X and one in formalism Y, or in X again, are actually the same language?).

Surprisingly often, the answer to these decision problems is "it cannot be done at all", or "it is extremely expensive" (with a characterization of how expensive). Therefore, formal language theory is a major application area of computability theory and complexity theory. Formal languages may be classified in the Chomsky hierarchy based on the expressive power of their generative grammar as well as the complexity of their recognizing automaton. Context-free grammars and regular grammars provide a good compromise between expressivity and ease of parsing, and are widely used in practical applications.

Metasyntax

A metasyntax is a syntax used to define the syntax of a programming language or formal language. It describes the allowable structure and composition of phrases and sentences of a metalanguage, which is used to describe either a natural language or a computer programming language. Some of the widely used formal metalanguages for computer languages are Backus–Naur form (BNF), extended Backus–Naur form (EBNF), Wirth syntax notation (WSN), and augmented Backus–Naur form (ABNF).

Metalanguages have their own metasyntax each composed of terminal symbols, nonterminal symbols, and metasymbols. A terminal symbol, such as a word or a token, is a stand-alone structure in a language being defined. A nonterminal symbol represents a syntactic category, which defines one or more valid phrasal or sentence structure consisted of an n-element subset. Metasymbols provide syntactic information for denotational purposes in a given metasyntax. Terminals, nonterminals, and metasymbols do not apply across all metalanguages.

Typically, the metalanguage for token-level languages (formally called "regular languages") does not have nonterminals because nesting is not an issue in these regular languages. English, as a metalanguage for describing certain languages, does not contain metasymbols since all explanation could be done using English expression. There are only certain formal metalanguages used for describing recursive languages (formally called context-free languages) that have terminals, nonterminals, and metasymbols in their metasyntax.

Operations on languages

Certain operations on languages are common. This includes the standard set operations, such as union, intersection, and complement. Another class of operation is the element-wise application of string operations.

Examples: suppose ${\displaystyle L_1}$ and ${\displaystyle L_2}$ are languages over some common alphabet ${\displaystyle \mathrm{\Sigma }}$.

• The concatenation ${\displaystyle L_1\cdot L_2}$ consists of all strings of the form ${\displaystyle vw}$ where ${\displaystyle v}$ is a string from ${\displaystyle L_1}$ and ${\displaystyle w}$ is a string from ${\displaystyle L_2}$.
• The intersection ${\displaystyle L_1\cap L_2}$ of ${\displaystyle L_1}$ and ${\displaystyle L_2}$ consists of all strings that are contained in both languages
• The complement ${\displaystyle \mathrm{\neg }{L}_1}$ of ${\displaystyle L_1}$ with respect to ${\displaystyle \mathrm{\Sigma }}$ consists of all strings over ${\displaystyle \mathrm{\Sigma }}$ that are not in ${\displaystyle L_1}$.
• The Kleene star: the language consisting of all words that are concatenations of zero or more words in the original language;
• Reversal:
  • Let ε be the empty word, then ${\displaystyle {\epsilon }^R=\epsilon }$, and
  • for each non-empty word ${\displaystyle w={\sigma }_1\cdots {\sigma }_n}$ (where ${\displaystyle {\sigma }_1,\dots ,{\sigma }_n}$are elements of some alphabet), let ${\displaystyle w^R={\sigma }_n\cdots {\sigma }_1}$,
  • then for a formal language ${\displaystyle L}$, ${\displaystyle L^R=\{w^R\mid w\in L\}}$.
• String homomorphism

Such string operations are used to investigate closure properties of classes of languages. A class of languages is closed under a particular operation when the operation, applied to languages in the class, always produces a language in the same class again. For instance, the context-free languages are known to be closed under union, concatenation, and intersection with regular languages, but not closed under intersection or complement. The theory of trios and abstract families of languages studies the most common closure properties of language families in their own right.

Applications

Programming languages

A compiler usually has two distinct components. A lexical analyzer, sometimes generated by a tool like lex, identifies the tokens of the programming language grammar, e.g. identifiers or keywords, numeric and string literals, punctuation and operator symbols, which are themselves specified by a simpler formal language, usually by means of regular expressions. At the most basic conceptual level, a parser, sometimes generated by a parser generator like yacc, attempts to decide if the source program is syntactically valid, that is if it is well formed with respect to the programming language grammar for which the compiler was built.

Of course, compilers do more than just parse the source code – they usually translate it into some executable format. Because of this, a parser usually outputs more than a yes/no answer, typically an abstract syntax tree. This is used by subsequent stages of the compiler to eventually generate an executable containing machine code that runs directly on the hardware, or some intermediate code that requires a virtual machine to execute.

Formal theories, systems, and proofs

In mathematical logic, a formal theory is a set of sentences expressed in a formal language.

A formal system (also called a logical calculus, or a logical system) consists of a formal language together with a deductive apparatus (also called a deductive system). The deductive apparatus may consist of a set of transformation rules, which may be interpreted as valid rules of inference, or a set of axioms, or have both. A formal system is used to derive one expression from one or more other expressions. Although a formal language can be identified with its formulas, a formal system cannot be likewise identified by its theorems. Two formal systems ${\displaystyle \mathcal{F}\mathcal{S}}$ and ${\displaystyle \mathcal{F}{\mathcal{S}}^{\prime }}$ may have all the same theorems and yet differ in some significant proof-theoretic way (a formula A may be a syntactic consequence of a formula B in one but not another for instance).

A formal proof or derivation is a finite sequence of well-formed formulas (which may be interpreted as sentences, or propositions) each of which is an axiom or follows from the preceding formulas in the sequence by a rule of inference. The last sentence in the sequence is a theorem of a formal system. Formal proofs are useful because their theorems can be interpreted as true propositions.

Interpretations and models

Formal languages are entirely syntactic in nature, but may be given semantics that give meaning to the elements of the language. For instance, in mathematical logic, the set of possible formulas of a particular logic is a formal language, and an interpretation assigns a meaning to each of the formulas—usually, a truth value.

The study of interpretations of formal languages is called formal semantics. In mathematical logic, this is often done in terms of model theory. In model theory, the terms that occur in a formula are interpreted as objects within mathematical structures, and fixed compositional interpretation rules determine how the truth value of the formula can be derived from the interpretation of its terms; a model for a formula is an interpretation of terms such that the formula becomes true.