TeX

When the first paper volume of Knuth's The Art of Computer Programming was published in 1968,^[5] it was typeset using hot metal typesetting on a Monotype machine. This method, dating back to the 19th century, produced a "classic style" appreciated by Knuth.^[6] When the second edition was published, in 1976, the whole book had to be typeset again because the Monotype technology had been largely replaced by phototypesetting, and the original fonts were no longer available. When Knuth received the galley proofs of the new book on 30 March 1977, he found them inferior. Disappointed, Knuth set out to design his own typesetting system.

Knuth saw for the first time the output of a high-quality digital typesetting system, and became interested in digital typography. On 13 May 1977, he wrote a memo to himself describing the basic features of TeX.^[7] He planned to finish it on his sabbatical in 1978 (though ultimately the language's syntax was not frozen until 1989^{[citation needed]}). Guy Steele happened to be at Stanford during the summer of 1978, when Knuth was developing his first version of TeX. When Steele returned to the Massachusetts Institute of Technology that autumn, he rewrote TeX's input/output (I/O) to run under the Incompatible Timesharing System (ITS) operating system. The first version of TeX, called TeX78, was written in the SAIL programming language to run on a PDP-10 under Stanford's WAITS operating system.^[8]

For later versions of TeX, Knuth invented the concept of literate programming, a way of producing compilable source code and cross-linked documentation typeset in TeX from the same original file. The language used is called WEB and produces programs in DEC PDP-10 Pascal.

TeX82, a new version of TeX rewritten from scratch, was published in 1982. Among other changes, the original hyphenation algorithm was replaced by a new algorithm written by Frank Liang. TeX82 also uses fixed-point arithmetic instead of floating-point, to ensure reproducibility of the results across different computer hardware,^[9] and includes a real, Turing-complete programming language, following intense lobbying by Guy Steele.^[10] In 1989, Donald Knuth released new versions of TeX and Metafont.^[11] Despite his desire to keep the program stable, Knuth realized that 128 different characters for the text input were not enough to accommodate foreign languages; the main change in version 3.0 of TeX is thus the ability to work with 8-bit inputs, allowing 256 different characters in the text input. TeX3.0 was released on March 15, 1990.^[12]

Since version 3, TeX has used an idiosyncratic version numbering system, where updates have been indicated by adding an extra digit at the end of the decimal, so that the version number asymptotically approaches π. This is a reflection of the fact that TeX is now very stable, and only minor updates are anticipated. The current version of TeX is 3.141592653; it was last updated in 2021.^[13] The design was frozen after version 3.0, and no new feature or fundamental change will be added, so all newer versions will contain only bug fixes.^[14] Even though Donald Knuth himself has suggested a few areas in which TeX could have been improved, he indicated that he firmly believes that having an unchanged system that will produce the same output now and in the future is more important than introducing new features. For this reason, he has stated that the "absolutely final change (to be made after my death)" will be to change the version number to π, at which point all remaining bugs will become features.^[15] Likewise, versions of Metafont after 2.0 asymptotically approach e (currently at 2.7182818), and a similar change will be applied after Knuth's death.^[14]

The name TeX is intended by its developer to be pronounced /tɛx/, with the final consonant of loch.^[16] The letters of the name are meant to represent the capital Greek letters tau, epsilon, and chi, as TeX is an abbreviation of τέχνη (ΤΕΧΝΗ technē), Greek for both "art" and "craft", which is also the root word of technical. English speakers often pronounce it /tɛk/, like the first syllable of technical. Knuth instructs that it be typeset with the "E" below the baseline and reduced spacing between the letters. This is done, as Knuth mentions in his TeXbook, to distinguish TeX from other system names such as TEX, the Text EXecutive processor (developed by Honeywell Information Systems).^[17]

TeX commands commonly start with a backslash and are grouped with curly braces. Almost all of TeX's syntactic properties can be changed on the fly, which makes TeX input hard to parse by anything but TeX itself. TeX is a macro- and token-based language: many commands, including most user-defined ones, are expanded on the fly until only unexpandable tokens remain, which are then executed. Expansion itself is practically free from side effects. Tail recursion of macros takes no memory, and if-then-else constructs are available. This makes TeX a Turing-complete language even at the expansion level.^[18] The system can be divided into four levels: in the first, characters are read from the input file and assigned a category code (sometimes called "catcode", for short). Combinations of a backslash (actually, any character of category zero) followed by letters (characters of category 11) or a single other character are replaced by a control-sequence token. In this sense, this stage is like lexical analysis, although it does not form numbers from digits. In the next stage, expandable control sequences (such as conditionals or defined macros) are replaced by their replacement text. The input for the third stage is then a stream of characters (including the ones with special meaning) and unexpandable control sequences (typically assignments and visual commands). Here, the characters get assembled into a paragraph, and TeX's paragraph breaking algorithm works by optimizing breakpoints over the whole paragraph. The fourth stage breaks the vertical list of lines and other material into pages.

The TeX system has precise knowledge of the sizes of all characters and symbols, and using this information, it computes the optimal arrangement of letters per line and lines per page. It then produces a DVI file ("DeVice Independent") containing the final locations of all characters. This DVI file can then be printed directly given an appropriate printer driver, or it can be converted to other formats. Nowadays, pdfTeX is often used, which bypasses DVI generation altogether.^[19] The base TeX system understands about 300 commands, called primitives.^[20] These low-level commands are rarely used directly by users, and most functionality is provided by format files (predumped memory images of TeX after large macro collections have been loaded). Knuth's original default format, which adds about 600 commands, is Plain TeX.^[21] The most widely used format is LaTeX, originally developed by Leslie Lamport, which incorporates document styles for books, letters, slides, etc., and adds support for referencing and automatic numbering of sections and equations. Another widely used format, AMS-TeX, is produced by the American Mathematical Society and provides many more user-friendly commands, which can be altered by journals to fit with their house style. Most of the features of AMS-TeX can be used in LaTeX by using the "AMS packages" (e.g., amsmath, amssymb) and the "AMS document classes" (e.g., amsart, amsbook). This is then referred to as AMS-LaTeX.^[22] Other formats include ConTeXt, used primarily for desktop publishing and written mostly by Hans Hagen at Pragma.

The TeX software incorporates several aspects that were not available, or were of lower quality, in other typesetting programs at the time when TeX was released. Some of the innovations are based on interesting algorithms, and have led to several theses for Knuth's students. While some of these discoveries have now been incorporated into other typesetting programs, others, such as the rules for mathematical spacing, are still unique.^{[as of?]}

Since the primary goal of the TeX language is high-quality typesetting for publishers of books, Knuth gave a lot of attention to the spacing rules for mathematical formulae.^[23][24] He took three bodies of work that he considered to be standards of excellence for mathematical typography: the books typeset by the Addison-Wesley Publishing house (the publisher of The Art of Computer Programming) under the supervision of Hans Wolf; editions of the mathematical journal Acta Mathematica dating from around 1910; and a copy of Indagationes Mathematicae, a Dutch mathematics journal. Knuth looked closely at these printed papers to sort out and look for a set of rules for spacing.^[25][26] While TeX provides some basic rules and the tools needed to specify proper spacing, the exact parameters depend on the font used to typeset the formula. For example, the spacing for Knuth's Computer Modern fonts has been precisely fine-tuned over the years and is now set; but when other fonts, such as AMS Euler, were used by Knuth for the first time, new spacing parameters had to be defined.^[27]

The typesetting of math in TeX is not without criticism, particularly with respect to technical details of the font metrics, which were designed in an era when significant attention was paid to storage requirements. This resulted in some "hacks" overloading some fields, which in turn required other "hacks". On an aesthetics level, the rendering of radicals has also been criticized.^[28] The OpenType math font specification largely borrows from TeX, but has some new features/enhancements.^[29][30][31]

In comparison with manual typesetting, the problem of justification is easy to solve with a digital system such as TeX, which, provided that good points for line breaking have been defined, can automatically spread the spaces between words to fill in the line. The problem is thus to find the set of breakpoints that will give the most visually pleasing result. Many line-breaking algorithms use a first-fit approach, where the breakpoints for each line are determined one after the other, and no breakpoint is changed after it has been chosen.^[32] Such a system is not able to define a breakpoint depending on the effect that it will have on the following lines. In comparison, the total-fit line-breaking algorithm used by TeX and developed by Donald Knuth and Michael Plass ^[33] considers all the possible breakpoints in a paragraph, and finds the combination of line breaks that will produce the most globally pleasing arrangement.

Formally, the algorithm defines a value called badness associated with each possible line break; the badness is increased if the spaces on the line must stretch or shrink too much to make the line the correct width. Penalties are added if a breakpoint is particularly undesirable: for example, if a word must be hyphenated, if two lines in a row are hyphenated, or if a very loose line is immediately followed by a very tight line. The algorithm will then find the breakpoints that will minimize the sum of squares of the badness (including penalties) of the resulting lines. If the paragraph contains ${\displaystyle n}$ possible breakpoints, the number of situations that must be evaluated naively is ${\displaystyle 2^{n}}$. However, by using the method of dynamic programming, the complexity of the algorithm can be brought down to ${\displaystyle O(n^{2})}$ (see Big O notation). Further simplifications (for example, not testing extremely unlikely breakpoints such as a hyphenation in the first word of a paragraph, or very overfull lines) lead to an efficient algorithm whose running time is ${\displaystyle O(nw)}$, where ${\displaystyle w}$ is the width of a line. A similar algorithm is used to determine the best way to break paragraphs across two pages, in order to avoid widows or orphans (lines that appear alone on a page while the rest of the paragraph is on the following or preceding page). However, in general, a thesis by Michael Plass shows how the page-breaking problem can be NP-complete because of the added complication of placing figures.^[34] TeX's line-breaking algorithm has been adopted by several other programs, such as Adobe InDesign (a desktop publishing application)^[35] and the GNU fmt Unix command line utility.^[36]

If no suitable line break can be found for a line, the system will try to hyphenate a word. The original version of TeX used a hyphenation algorithm based on a set of rules for the removal of prefixes and suffixes of words, and for deciding if it should insert a break between the two consonants in a pattern of the form vowel–consonant–consonant–vowel (which is possible most of the time).^[37] TeX82 introduced a new hyphenation algorithm, designed by Frank Liang in 1983, to assign priorities to breakpoints in letter groups. A list of hyphenation patterns is first generated automatically from a corpus of hyphenated words (a list of 50,000 words). If TeX must find the acceptable hyphenation positions in the word encyclopedia, for example, it will consider all the subwords of the extended word .encyclopedia., where . is a special marker to indicate the beginning or end of the word. The list of subwords includes all the subwords of length 1 (., e, n, c, y, etc.), of length 2 (.e, en, nc, etc.), etc., up to the subword of length 14, which is the word itself, including the markers. TeX will then look into its list of hyphenation patterns, and find subwords for which it has calculated the desirability of hyphenation at each position. In the case of our word, 11 such patterns can be matched, namely ₁c₄l₄, ₁cy, ₁d₄i₃a, ₄edi, e₃dia, ₂i₁a, ope₅d, ₂p₂ed, ₃pedi, pedia₄, y₁c. For each position in the word, TeX will calculate the maximum value obtained among all matching patterns, yielding en₁cy₁c₄l₄o₃p₄e₅d₄i₃a₄. Finally, the acceptable positions are those indicated by an odd number, yielding the acceptable hyphenations en-cy-clo-pe-di-a. This system based on subwords allows the definition of very general patterns (such as ₂i₁a), with low indicative numbers (either odd or even), which can then be superseded by more specific patterns (such as ₁d₄i₃a) if necessary. These patterns find about 90% of the hyphens in the original dictionary; more importantly, they do not insert any spurious hyphen. In addition, a list of exceptions (words for which the patterns do not predict the correct hyphenation) are included with the Plain TeX format; additional ones can be specified by the user.^{[38][page needed][39]}

Metafont, not strictly part of TeX, is a font description system which allows the designer to describe characters algorithmically. It uses Bézier curves in a fairly standard way to generate the actual characters to be displayed, but Knuth devotes substantial attention to the rasterizing problem on bitmapped displays. Another thesis, by John Hobby, further explores this problem of digitizing "brush trajectories". This term derives from the fact that Metafont describes characters as having been drawn by abstract brushes (and erasers). It is commonly believed that TeX is based on bitmap fonts but, in fact, these programs "know" nothing about the fonts that they are using other than their dimensions. It is the responsibility of the device driver to appropriately handle fonts of other types, including PostScript Type 1 and TrueType. Computer Modern (commonly known as "the TeX font") is freely available in Type 1 format, as are the AMS math fonts. Users of TeX systems that output directly to PDF, such as pdfTeX, XeTeX, or LuaTeX, generally never use Metafont output at all.

TeX documents are written and programmed using an unusual macro language. Broadly speaking, the running of this macro language involves expansion and execution stages which do not interact directly. Expansion includes both literal expansion of macro definitions as well as conditional branching, and execution involves such tasks as setting variables/registers and the actual typesetting process of adding glyphs to boxes.

The definition of a macro not only includes a list of commands but also the syntax of the call. It differs with most widely used lexical preprocessors like M4, in that the body of a macro gets tokenized at definition time.

The TeX macro language has been used to write larger document production systems, most notably including LaTeX and ConTeXt.

A sample Hello world program in plain TeX is:

Hello, World
\bye          % marks the end of the file; not shown in the final output

This might be in a file myfile.tex, as .tex is a common file extension for plain TeX files. By default, everything that follows a percent sign on a line is a comment, ignored by TeX. Running TeX on this file (for example, by typing tex myfile.tex in a command-line interpreter, or by calling it from a graphical user interface) will create an output file called myfile.dvi, representing the content of the page in a device independent format (DVI). A DVI file could then be either viewed on screen or converted to a suitable format for any of the various printers for which a device driver existed (printer support was generally not an operating system feature at the time that TeX was created). Knuth has said that there is nothing inherent in TeX that requires DVI as the output format, and later versions of TeX, notably pdfTeX, XeTeX, and LuaTeX, all support output directly to PDF.

TeX provides a different text syntax specifically for mathematical formulas. For example, the quadratic formula (which is the solution of the quadratic equation) appears as:

Source code	Renders as
The quadratic formula is $-b \pm \sqrt{b^2 - 4ac} \over 2a$ \bye	${\displaystyle {\hbox{The quadratic formula is }}\textstyle {-b\pm {\sqrt {b^{2}-4ac}} \over 2a}}$

The formula is printed in a way a person would write by hand, or typeset the equation. In a document, entering mathematics mode is done by starting with a $ symbol, then entering a formula in TeX syntax, and closing again with another of the same symbol. Knuth explained in jest that he chose the dollar sign to indicate the beginning and end of mathematical mode in plain TeX because typesetting mathematics was traditionally supposed to be expensive.^[40] Display mathematics (mathematics presented centred on a new line) is similar but uses $$ instead of a single $ symbol. For example, the above with the quadratic formula in display math:

Source code	Renders as
The quadratic formula is $$-b \pm \sqrt{b^2 - 4ac} \over 2a$$ \bye	${\displaystyle {\begin{matrix}{\text{The quadratic formula is}}\\\displaystyle {\frac {-b\pm {\sqrt {b^{2}-4ac}}}{2a}}\end{matrix}}}$

In several technical fields such as computer science, mathematics, engineering and physics, TeX has become a de facto standard. Many thousands of books have been published using TeX, including books published by Addison-Wesley, Cambridge University Press, Elsevier, Oxford University Press, and Springer. Numerous journals in these fields are produced using TeX or LaTeX, allowing authors to submit their raw manuscript written in TeX.^[41] While many publications in other fields, including dictionaries and legal publications, have been produced using TeX, it has not been as successful as in the more technical fields, as TeX was primarily designed to typeset mathematics.

When he designed TeX, Donald Knuth did not believe that a single typesetting system would fit everyone's needs; instead, he designed many hooks inside the program so that it would be possible to write extensions, and released the source code, hoping that the publishers would design versions tailoring to their own needs. While such extensions have been created (including some by Knuth himself),^[42] most people have extended TeX only using macros and it has remained a system associated with technical typesetting.^[43][44][26]

Packages like mhchem, XyMTeX, chemfig, and chemmacros for typesetting chemical equations, structures, and reactions in LaTeX.^[45]

It is possible to use TeX for automatic generation of sophisticated layout for XML data. The differences in syntax between the two description languages can be overcome with the help of TeXML. In the context of XML publication, TeX can thus be considered an alternative to XSL-FO. TeX allowed scientific papers in mathematical disciplines to be reduced to relatively small files that could be rendered client-side, allowing fully typeset scientific papers to be exchanged over the early Internet and emerging World Wide Web, even when sending large files was difficult. This paved the way for the creation of repositories of scientific papers such as arXiv, through which papers could be 'published' without an intermediary publisher.^[46]

The original source code for the current TeX software is written in WEB, a mixture of documentation written in TeX and a Pascal subset in order to ensure readability and portability. For example, TeX does all of its dynamic allocation itself from fixed-size arrays and uses only fixed-point arithmetic for its internal calculations. As a result, TeX has been ported to almost all operating systems, usually by using the web2c program to convert the source code into C instead of directly compiling the Pascal code. Knuth has kept a very detailed log of all the bugs he has corrected and changes he has made in the program since 1982; as of 2021^[update], the list contains 440 entries, not including the version modification that should be done after his death as the final change in TeX.^[47][48] Knuth offers monetary awards to people who find and report a bug in TeX. The award per bug started at US$2.56 (one "hexadecimal dollar"^[49]) and doubled every year until it was frozen at its current value of $327.68. Knuth has lost relatively little money as there have been very few bugs claimed. In addition, recipients have been known to frame their check as proof that they found a bug in TeX rather than cashing it.^[50][51]

Due to scammers finding scanned copies of his checks on the internet and using them to try to drain his bank account, Knuth no longer sends out real checks, but those who submit bug reports can get credit at The Bank of San Serriffe instead.^[52]

TeX is usually provided in the form of an easy-to-install bundle of TeX itself along with Metafont and all the necessary fonts, documents formats, and utilities needed to use the typesetting system. On UNIX-compatible systems, including Linux and Apple macOS, TeX is distributed as part of the larger TeX Live distribution. (Prior to TeX Live, the teTeX distribution was the de facto standard on UNIX-compatible systems.) On Microsoft Windows, there is the MiKTeX distribution (enhanced by proTeXt) and the Microsoft Windows version of TeX Live.

Several document processing systems are based on TeX, notably jadeTeX, which uses TeX as a backend for printing from James Clark's DSSSL Engine, the Arbortext publishing system, and Texinfo, the GNU documentation processing system. TeX has been the official typesetting package for the GNU operating system since 1984.

Numerous extensions and companion programs for TeX exist, among them BibTeX for bibliographies (distributed with LaTeX); pdfTeX, a TeX-compatible engine which can directly produce PDF output (as well as continuing to support the original DVI output); XeTeX, a TeX-compatible engine that supports Unicode and OpenType; and LuaTeX, a Unicode-aware extension to TeX that includes a Lua runtime with extensive hooks into the underlying TeX routines and algorithms. Most TeX extensions are available for free from CTAN, the Comprehensive TeX Archive Network.

Donald Knuth has indicated several times^{[53][54][55][26]} that the source code of TeX has been placed into the "public domain", and he strongly encourages modifications or experimentations with this source code. However, since Knuth highly values the reproducibility of the output of all versions of TeX, any changed version must not be called TeX, or anything confusingly similar. To enforce this rule, any implementation of the system must pass a test suite called the TRIP test^[56] before being allowed to be called TeX. The question of licence is somewhat confused by the statements included at the beginning of the TeX source code,^[57] which indicate that "all rights are reserved. Copying of this file is authorized only if ... you make absolutely no changes to your copy". This restriction should be interpreted as a prohibition to change the source code as long as the file is called tex.web. The copyright note at the beginning of tex.web (and mf.web) was changed in 2021 to explicitly state this. This interpretation is confirmed later in the source code when the TRIP test is mentioned ("If this program is changed, the resulting system should not be called 'TeX'").^[58] The American Mathematical Society tried in the early 1980s to claim a trademark for TeX. This was rejected because at the time "TEX" (all caps) was registered by Honeywell for the "Text EXecutive" text processing system.^{[citation needed]}

Since the source code of TeX is essentially in the public domain, other programmers are allowed (and explicitly encouraged) to improve the system, but are required to use another name to distribute the modified TeX, meaning that the source code can still evolve. For example, the Omega project was developed after 1991, primarily to enhance TeX's multilingual typesetting abilities.^[59] Knuth created "unofficial" modified versions, such as TeX-XeT, which allows a user to mix texts written in left-to-right and right-to-left writing systems in the same document.^[42]

Notable entities in the TeX community include the TeX Users Group (TUG), which currently publishes TUGboat and formerly published The PracTeX Journal, covering a wide range of topics in digital typography relevant to TeX. The Deutschsprachige Anwendervereinigung TeX (DANTE) is a large user group in Germany. The TeX Users Group was founded in 1980 for educational and scientific purposes, provides an organization for those who have an interest in typography and font design, and are users of the TeX typesetting system invented by Knuth. The TeX Users Group represents the interests of TeX users worldwide. The TeX Users Group publishes the journal TUGboat three times per year;^[60] DANTE publishes Die TeXnische Komödie [de] four times per year. Other user groups include DK-TUG in Denmark, GUTenberg [fr] in France, GuIT in Italy, NTG in the Netherlands and UK-TUG in the United Kingdom; the user groups jointly maintain a complete list.^[61]

There are a variety of editors designed to work with TeX:

• The TeXmacs text editor is a WYSIWYG-WYSIWYM scientific text editor, inspired by both TeX and Emacs. It uses Knuth's fonts and can generate TeX output.
• Overleaf is a partial-WYSIWYG, online editor that provides a cloud-based solution to TeX along with additional features in real-time collaborative editing.
• LyX is a WYSIWYM document processor which runs on a variety of platforms including:
  • Linux,
  • Microsoft Windows (newer versions require Windows 2000 or later)
  • Apple macOS (using a non-native Qt front-end).
• TeXShop (for macOS), TeXworks (for Linux, macOS and Windows) and WinShell (for Windows) are similar tools and provide an integrated development environment (IDE) for working with LaTeX or TeX. For KDE/Qt, Kile provides such an IDE.
• Texmaker is the Pure Qt equivalent of Kile, with a user interface that is nearly the same as Kile's.
• TeXstudio is an open-source fork (2009) of Texmaker that offers a different approach to configurability and features. Free downloadable binaries are provided for Windows, Linux, macOS, OS/2, and FreeBSD.
• GNU Emacs has various built-in and third-party packages with support for TeX, the major one being AUCTeX.
• Visual Studio Code. A notable extension is LaTeX Workshop
• For Vim, possible plugins include Vim-LaTeX Suite,^[62] Automatic TeX,^[63] and TeX-9.^[64]
• For Apache OpenOffice and LibreOffice, iMath and TexMaths extensions can provide mathematical TeX typesetting.^[65][66]
• For MediaWiki, the Math extension provides mathematical TeX typesetting, but the code needs to be surrounded by <math> tag.

• List of TeX extensions

• List of TeX typesetting software and tools
• Comparison of document markup languages
• Formula editor – Computer program used to typeset mathematical works or formulae
• List of document markup languages
• MathJax – Cross-browser JavaScript library that displays mathematical equations in web browsers
• MathTime – Typeface for TeX
• New Typesetting System
• PGF/TikZ – Graphics languages
• PSTricks
• xdvi – Open Source Computer Program