The UNIX family — BSD, System V and the POSIX standard

Objective

Understand how UNIX split into several rival branches, why a common standard (POSIX) became necessary, and why most of what you learn on Linux also works on macOS or FreeBSD.

Key points

• At the University of Berkeley, UNIX became BSD: this is where TCP/IP networking, the vi editor and the csh shell came from.
• After the break-up of AT&T (1984), the company sold its official version: System V.
• Each manufacturer created its own paid UNIX (Solaris, AIX, HP-UX…), incompatible with the others: these were the "UNIX wars".
• The POSIX standard (1988) defines what a "UNIX-like" system must provide; Linux follows it closely.
• A lawsuit (1992-1994) blocked BSD at the very moment Linux was born, which pushed many developers towards Linux.

1. Berkeley receives UNIX (1974-1977)

As you saw in lesson 0.1, AT&T distributed UNIX to universities with its source code. The University of California, Berkeley installed it in 1974. Ken Thompson even spent a sabbatical year there (1975-1976) as a visiting professor.

A student, Bill Joy, started improving the system. In 1978, he released a first collection of programs: the Berkeley Software Distribution, or BSD. At first, it was a simple add-on to AT&T's UNIX; little by little, it became a complete system.

What BSD brought, and what you will use on Linux:

The arrival of TCP/IP was a turning point: the US military agency DARPA, which funded the future Internet, chose BSD as its reference system in the early 1980s. UNIX and the Internet grew up together.

2. AT&T enters the market: System V (1983)

In 1982, an agreement with the US justice system provided for the break-up of AT&T, effective on 1 January 1984. In exchange, the company finally gained the right to sell computing products.

AT&T then marketed its official version of UNIX: System III (1982), then System V (1983), with its successive "releases" (Release 2, 3, 4). SVR4 (System V Release 4, late 1980s), developed with Sun, brought together a large part of the ideas of BSD and System V.

Several Linux habits come from System V:

ls /etc/init.d/       # the "System V style" startup scripts (RHEL 6, old Debian)
ps -ef                # the System V syntax of ps (with a dash)
ps aux                # the BSD syntax of ps (without a dash) — Linux accepts both!

The fact that ps accepts two styles of options is a living reminder of this double origin.

3. The commercial UNIX systems (1980s-1990s)

Every major manufacturer took a UNIX licence and created its own version, adapted to its machines (and sold with them):

The case of Sun is interesting: the company was co-founded in 1982 by… Bill Joy, the author of BSD. Sun workstations were the reference machines of universities and of the Internet in the 1980s and 1990s.

4. The "UNIX wars" (1988-1993)

Each of these UNIX systems added its own commands, options and tools. A program written for AIX had to be modified to run on HP-UX. An administrator trained on Solaris was lost on IRIX.

At the end of the 1980s, two camps fought each other:

          AT&T + Sun                       IBM, DEC, HP and others
   (System V Release 4, 1988-1989)       (Open Software Foundation, 1988)
       "UNIX International"                       "OSF/1"
                   \                               /
                    \___ incompatibilities, lawsuits, ___/
                          lost customers

While the manufacturers were fighting, a competitor was quietly advancing on PCs: Microsoft Windows NT (1993). And, without anyone suspecting it yet, a Finnish student started writing a kernel on his PC in 1991…

5. POSIX: the common rule (1988)

To get out of the chaos, the standards body IEEE published the POSIX standard (IEEE 1003.1) in 1988. The name, suggested by Richard Stallman (whom you will meet again in lesson 0.3), roughly means Portable Operating System Interface, with an "X" for UNIX.

POSIX describes what a UNIX-like system must provide, without saying how to program it:

Later, the organisation The Open Group (owner of the "UNIX" trademark since the 1990s) published the Single UNIX Specification. Only certified systems that pay for the licence are allowed to officially call themselves "UNIX". This is the case for macOS (certified since version 10.5, in 2007), AIX and Solaris.

Linux is not UNIX-certified (certification is expensive and brings nothing to a free project), but it follows POSIX very closely. That is why Linux is called a "UNIX-like" system.

getconf _POSIX_VERSION    # which version of POSIX? e.g. 200809 = 2008 edition

Practical advice for your whole career: when you write a script that must run everywhere, write it in POSIX sh rather than with tricks specific to bash. On Debian and Ubuntu, /bin/sh is in fact dash, a minimal POSIX shell: a "bash" script run with sh may fail there.

6. The lawsuit that slowed BSD down (1992-1994)

At the end of the 1980s, the Berkeley developers set out to replace all the original AT&T code in BSD, so that it could be distributed freely. In 1991, Net/2 was released, almost entirely free. Projects were derived from it: 386BSD (1992), for PCs, and the company BSDi, which sold a commercial version.

In 1992, USL (Unix System Laboratories, AT&T's UNIX subsidiary) sued BSDi, then the University of California: BSD supposedly still contained AT&T code and trade secrets. For two years, nobody knew whether BSD could continue to be distributed. Companies hesitated to use it.

In 1993, Novell bought USL. The lawsuit ended with a settlement in early 1994: a few files were removed or modified, and Berkeley released 4.4BSD-Lite (1994), free of any AT&T code.

But during those two years of uncertainty, another system, free and with no legal risk at all, advanced very quickly: Linux. Linus Torvalds said it himself: if a free BSD for PCs had been available at that time, he might never have written his kernel.

7. The descendants of BSD today

Net/2 (1991) and 386BSD (1992), then 4.4BSD-Lite (1994)
 ├── FreeBSD (1993) ── servers, Netflix (video streaming), base of the system of recent PlayStations
 ├── NetBSD  (1993) ── "runs on anything", very portable
 │     └── OpenBSD (1995) ── security first; created OpenSSH (1999)
 └── NeXTSTEP / Darwin ── base of macOS and iOS (with the Mach microkernel)

FreeBSD and NetBSD were born in 1993 from 386BSD; after the 1994 settlement, they were rebuilt on 4.4BSD-Lite.

So you may already be using BSD without knowing it: macOS and iOS descend from it, and OpenSSH, which you will use on every Linux server, is developed by the OpenBSD team.

8. The lesson timeline

1974  Berkeley receives UNIX
1978  First BSD distribution (Bill Joy)
1982  Sun founded; AT&T break-up agreement; System III
1983  System V (AT&T); 4.2BSD brings TCP/IP
1984  AT&T broken up: UNIX becomes a commercial product
1986  AIX (IBM)
1988  POSIX (IEEE 1003.1); OSF created; start of the "UNIX wars"
1991  Net/2; Linus Torvalds starts Linux
1992  USL lawsuit against BSDi; Solaris 2 (System V R4 base)
1993  FreeBSD, NetBSD; Novell buys USL
1994  Settlement: 4.4BSD-Lite, free of AT&T code

Common pitfalls

• Believing that a command seen on macOS will work the same way on Linux. Both follow POSIX, but the options often differ: sed -i is written sed -i '' on macOS (BSD tools), sed -i on Linux (GNU tools).
• Confusing "UNIX" and "UNIX-like". "UNIX" is a trademark reserved for certified systems. Linux and FreeBSD are "UNIX-like".
• Writing #!/bin/sh at the top of a script full of bash syntax. On Debian and Ubuntu, sh is dash: arrays or [[ … ]] fail there. Write #!/bin/bash if you use bash.
• Thinking that /etc/init.d/ scripts are still used. They come from System V; since RHEL 7 and Debian 8, systemd starts the services.

Lab

This time, you are on Debian 12. You will check the system's POSIX heritage for yourself:

getconf _POSIX_VERSION    # the version of POSIX followed
ls -l /bin/sh             # which shell does /bin/sh point to?
ps -ef | head -n 3        # System V syntax
ps aux | head -n 3        # BSD syntax: the same result, or nearly

The tasks: save the POSIX version, find out which shell is hiding behind /bin/sh, and write down the two main branches of UNIX. Then click Check.

Quiz

1. What are the two main branches of UNIX, and who created each of them?
2. Name two major contributions of BSD that you will use on Linux.
3. What is POSIX, and why is it useful to an administrator?
4. Is Linux allowed to officially call itself "UNIX"? Why?
5. Why did the USL lawsuit against BSDi (1992-1994) benefit Linux?

Reference sources: M. K. McKusick (Open Sources, 1999), POSIX and The Open Group standards documents, Debian Reference (free licence). Original MasterLinux text.