Automata & Formal Languages · Primer

Machines that read

A language is just a set of strings. The deep question of computation is: what kind of machine do you need to decide whether a string belongs? The answer comes in a ladder — finite memory, then a stack, then an unbounded tape — and each rung recognizes languages the one below cannot.

A scroll-through tour of the Chomsky hierarchy. Six ideas, each with a small machine running live. Then four instruments to take the controls.
01

Languages and the machines that accept them

Fix an alphabet — say {0,1}. A string is a finite sequence of symbols; a language is any set of strings you care about. A machine reads a string left to right, one symbol at a time, and at the end gives a verdict: accept if the string is in the language, reject if not. Everything that follows is about how much memory that reading takes.

reading a string · accepts if it has an even number of 1s·
02

Finite automata: a handful of states

The simplest machine has a fixed, finite set of states and a transition for each symbol. No counting, no scratch paper — all it remembers is which state it's in. Here, one bit of memory (is the count of 1s even or odd?) suffices. Languages recognizable this way are called regular.

DFA · double ring = accepting state · token follows each symbol·
03

Nondeterminism: being in many states at once

Allow the machine to guess — to follow several transitions in parallel — and you get a nondeterministic finite automaton. It accepts if any path ends in an accepting state. It's no more powerful than a DFA (you can always convert one to the other), but it's often far smaller and easier to design.

NFA · accepts strings ending in "01" · lit = currently-active states·
04

Regular expressions: patterns are machines

The patterns you write to search text — regular expressions — describe exactly the regular languages, no more and no less. Concatenation, choice (|), and repetition (*) can each be wired into an automaton, and every automaton can be written as a pattern. Same expressive power, two notations.

pattern (a|b)*abb · testing strings against it·
05

The stack: pushdown automata

Give the machine one piece of unbounded memory with a catch — you may only touch the top — and you get a pushdown automaton. The stack can count. Now it can match nested structure: balanced parentheses, aⁿbⁿ, arithmetic expressions. These are the context-free languages, the backbone of programming-language grammars.

PDA · push on "(", pop on ")" · accept if balanced·
06

The tape: Turing machines

Replace the stack with an unbounded tape the head can read, write, and move across in both directions, and you reach the ceiling: the Turing machine. It can compute anything any computer can. Here one increments a binary number — sweeping right to left, flipping bits and carrying. With this much memory, "accept" and "compute" become the same idea.

Turing machine · incrementing a binary number (+1)·
the hierarchy, bottom to top:
  regular — finite automata, regular expressions
  context-free — pushdown automata, grammars
  decidable / r.e. — Turing machines
each rung strictly contains the one below.