FreeBSD Operating System
Greg Lehey


1. Introduction

FreeBSD features; Licensing conditions; Alittle history; Other free UNIX-like operating systems; FreeBSD system documentation; Other documentation on FreeBSD; The FreeBSD community; Mailing lists; The Berkeley daemon.

Introduction

FreeBSD is a free operating system derived from AT&T's UNIX operating system1). It runs on the following platforms:

This course describes the released versions of FreeBSD for Intel and Alpha processors. Current support for SPARC 64 processors is changing too fast for it to be practical to give details specific to this processor, but nearly everything in this course also applies to SPARC 64.

How to use this course

This course is intended for a number of different audiences. It attempts to present the material without too many forward references. It contains the following parts:

  1. The first part, Chapters 1 to 6, tells you howtoinstall FreeBSD and what to do if things go wrong.
  2. Chapters 7to15introduce you to life with FreeBSD, including setting up optional features and system administration.
  3. Chapters 16 to 27 introduce you to FreeBSD's rich network support.
  4. Finally, Chapters 28 to 33 look at system administration topics that build on all the preceding material.

In more detail, we'll discuss the following subjects:

FreeBSD features

FreeBSD is derived from Berkeley UNIX,the flavour of UNIX developed by the Computer Systems Research Group at the University of California at Berkeley and previously released as the Berkeley Software Distribution (BSD) of UNIX.

UNIX is a registered trademark of the Open Group, so legally; FreeBSD may not be called UNIX. The technical issues are different, of course; makeup your own mind as to how much difference this makes.

Like commercial UNIX, FreeBSD provides you with many advanced features, including:

FreeBSD is based on the 4.4BSD UNIX released by the Computer Systems Research Group (CSRG) at the University of California at Berkeley. The FreeBSD Project has spent many thousands of hours fine-tuning the system for maximum performance and reliability. FreeBSD's features, performance and reliability compare very favorably with those of commercial operating systems.

Since the source code is available, you can easily customize it for special applications or projects, in ways not generally possible with operating systems from commercial vendors. You can easily start out small with an inexpensive 386 class PC and upgrade as your needs grow. Here are a few of the applications in which people currently use FreeBSD:

Licensing conditions

As the name suggests, FreeBSD is free. You don't have to pay for the code, you can use it on as many computers as you want, and you can give away copies to your friends. There are some restrictions, however. Here's the BSD license as used for all new FreeBSD code:

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

This software is provided by the FreeBSD project "as is and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the FreeBSD project or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in anyway out of the use of this software, even if advised of the possibility of such damage.

The last paragraph is traditionally written in ALL CAPS, for reasons which dont seem to have anything to do with the meaning. Older versions of the license also contained additional clauses relating to advertising.

A little history

FreeBSD is a labour of love: big commercial companies produce operating systems and charge lots of money for them; the FreeBSD project produces a professional-quality operating system and give sit away. That's not the only difference.

In 1981, when IBM introduced their Personal Computer, the microprocessor industry was still in its infancy. They entrusted Microsoft to supply the operating system. Microsoft already had their own version of UNIX, called XENIX, but the PC had a minimum of 16 kB and no disk. UNIX was not an appropriate match for this hardware. Microsoft went looking for something simpler. The "operating system "theychose was correspondingly primitive: 86/DOS, a clone of Digital Researchs successful CP/M operating system, written by Tim Paterson of Seattle Computer Products and originally called QDOS (Quick and Dirty Operating System). At the time, it seemed just the thing: it ran fine without a hard disk (in fact, the original PC didn't have a hard disk, not even as an option), and it didn't use up too much memory. The only thing that they really had to do was to change the name. IBM called its version PC-DOS, while Microsoft marketed its version under the name MS-DOS.

By this time, a little further down the US West Coast, the Computer Systems Research Group (CSRG) of the University of California at Berkeley had just modified AT&T's UNIX operating system to run on the new DEC VAX 11/780 machine, which sported virtual memory, and had turned their attention to implementing some new protocols for the ARPANET: the so-called Internet Protocols.The version of UNIX that they had developed was now sufficiently different from AT&T's system that it had been dubbed Berkeley UNIX.

As time went on, both MS-DOS and UNIX evolved. Before long, MS-DOS was modified to handle hard disks - not well, but it handled them, and for the PC users, it was so much better than what they had before that they ignored the inefficiencies. After all, the PC gave you your own hard disk on your desk, and you didnt have to share it with all the other people in the department. Microsoft even tried to emulate the UNIX directory structure, but succeeded only in implementing the concept of nested directories. At Berkeley, they were developing a higher performance disk subsystem, the Fast File System, now known as the UNIX File System.

By the late 80s, it was evident that Microsoft no longer intended to substantially enhance MS-DOS. New processors with support for multitasking and virtual memory had replaced the old Intel 8088 processor of the IBM PC, but they still ran MS-DOS by emulating the 8088 processor, which was now completely obsolete. The 640 kB memory limit of the original PC, which once appeared bigger than anybody would ever need, became a serious problem. In addition, people wanted to do more than one thing at a time with their computers.

A solution to both problems was obvious: move to the 32 bit address mode of the new Intel 80386 processor and introduce real multitasking, which operating systems on larger machines had had for decades. Of course, these larger machines were only physically larger. The average PC of 1990 had more memory, more disk and more processing power than just about any of the large computers of the 70s. Nevertheless, Microsoft didn't solve these problems for its "Windows" platform until much later, and the solutions still leave a lot to be desired.

UNIX, on the other hand, was a relatively mature operating system at the time when the PC was introduced. As a result, Microsoft-based environments have had little influence on the development of UNIX. UNIX development was determined by other factors: changes in legal regulations in the USA between 1977 and 1984 enabled AT&T first to license UNIX to other vendors, noticeably Microsoft, who announced XENIX in 1981, and then to market its own version of UNIX. AT&T developed System III in 1982, and System V in 1983. The differences between XENIX and System V were initially small, but they grew: by the mid-80s, there were four different versions of UNIX: the Research Version, used almost only inside AT&T, which from the eighth edition on derived from 4.1cBSD, the Berkeley Software Distribution (BSD) from Berkeley, the commercial System V from AT&T, and XENIX, which no longer interested Microsoft, and was marketed by the company that had developed it, the Santa Cruz Operation, or SCO.

One casualty of UNIXs maturity was the CSRG in Berkeley. UNIX was too mature to be considered an object of research, and the writing was on the wall: the CSRG would close down. Some people decided to port Berkeley UNIX to the PCafter all, SCO had ported its version of UNIX to the PC years earlier. In the Berkeley tradition, however, they wanted to give it away. The industrys reaction was not friendly. In 1992, AT&T's subsidiary USL (UNIX Systems Laboratories)!! led a lawsuit against Berkeley Software Design, Inc. (BSDI), the manufacturer of the BSD/386 operating system, later called BSD/OS, a system very similar to FreeBSD. They alleged distribution of AT&T source code in violation of licence agreements. They subsequently extended the case to the University of California at Berkeley. The suit was settled out of court, and the exact conditions were not all disclosed. The only one that became public was that BSDI would migrate their source base to the newer 4.4BSD-Lite sources, a thing that they were preparing to do in any case. Although not involved in the litigation, it was suggested to FreeBSD that they should also move to 4.4 BSD-Lite, which was done with the release of FreeBSD release 2.0 in late 1994.

Now, in the early 21st century, FreeBSD is the best known of the BSD operating systems, one that many consider to follow in the tradition of the CSRG. I can think of no greater honour for the development team. It was developed on a shoestring budget, yet it manages to outperform commercial operating systems by an order of magnitude.

The end of the UNIX wars

In the course of the FreeBSD project, a number of things have changed about UNIX. Sun Microsystems moved from a BSD base to a System V base in the late 80s, a move that convinced many people that BSD was dead and that System V was the future. Things turned out differently: in 1992, AT&T sold USL to Novell, Inc., who had introduced a product based on System V.4 called UnixWare. Although UnixWare has much better specifications than SCO's old System V3 UNIX, it was never success, and Novell finally sold their UNIX operation to SCO. SCO itself was then bought out by Caldera (which recently changed its name back to SCO), while the ownership of the UNIX trade mark has passed to the Open Group. System V UNIX is essentially dead: current commercial versions of UNIX have evolved so far since System V that they can't be considered the same system. By contrast, BSD is alive and healthy, and lives on in FreeBSD, NetBSD, OpenBSD and Apple's Mac OS X.

The importance of the AT&T code in the earlier versions of FreeBSD was certainly overemphasized in the lawsuit. All of the disputed code was over 10 years old at the time, and none of it was of great importance. In January 2002, Caldera released all "ancient" versions of UNIX under a BSD license. These specifically included all versions of UNIX from which BSD was derived: the first to seventh editions of Research UNIX and 32V, the predecessor to 3BSD. As a result, all versions of BSD, including those over which the lawsuit was conducted, are now freely available.

Other free UNIX-like operating systems

FreeBSD isn't the only free UNIX-like operating system availableit's not even the best-known one. The best-known free UNIX-like operating system is undoubtedly Linux, but there are also a number of other BSD-derived operating systems. We'll look at them first:

You could get the impression that there are lots of different, incompatible BSD versions. In fact, from a user viewpoint they're all very similar to each other, much more than the individual distributions of Linux, which we'll look at next.

FreeBSD and Linux

In 1991, Linux Torvalds, then a student in Helsinki, Finland, decided he wanted to run UNIX on his home computer. Atthat time the BSD sources were not freely available, and so Linus wrote his own version of UNIX, which he called Linux.

Linux is a superb example of how a few dedicated; clever people can produce an operating system that is better than well-known commercial systems developed by a large number of trained software engineers. It is better even than a number of commercial UNIX systems.

Obviously, I prefer FreeBSD over Linux, or I wouldn't be writing this course , but the differences between FreeBSD and Linux are more a matter of philosophy rather than of concept. Here are a few contrasts:

1.1. Differences between FreeBSD and Linux
FreeBSD is a direct descendent of the original UNIX, though it contains no residual AT& T codeLinux is a clone and never contained any AT&T code
FreeBSD is a complete operating system, maintained by a central group of software developers under the Concurrent Versions System which maintains a complete history of the project development. There is only one distribution of FreeBSDLinux is a kernel, personally maintained by Linus Torvalds and a few trusted companions. The non-kernel programs supplied with Linux are part of a distribution, of which there are several. Distributions are not completely compatible with each other.
The FreeBSD development style emphasizes accountability and documentation of changesThe Linux kernel is maintained by a small number of people who keep track of all changes. Unofficial patches abound.
The kernel supplied with a specific release of FreeBSD is clearly defined.Linux distributions often have subtly different kernels. The differences are not always documented
FreeBSD aims to be a stable production environmentMany versions of Linux are still "bleeding edge" development environments. This is changing rapidly, however
As a result of the centralized development style, FreeBSD is straightforward and easy to installThe ease of installation of Linux depends on the distribution. If you switch from one distribution of Linux to another, youll have to learn a new set of installation tools.
FreeBSD is still relatively unknown, since its distribution was initially restricted due to the AT&T lawsuitsLinux did not have any lawsuits to contend with, so for some time it was thought to be the only free UNIX-type system available
As a result of the lack of knowledge of FreeBSD, relatively little commercial software is available for itA growing amount of commercial software is becoming available for Linux.
As a result of the smaller user base, FreeBSD is less likely to have drivers for brand-new boards than LinuxJust about any new board will soon have a driver for Linux
Because of the lack of commercial applications and drivers for FreeBSD, Free-BSD runs most Linux programs, whether commercial or notLinux appears not to need to be able to run FreeBSD programs
FreeBSD is licensed under the BSD licensesee page 6. There are very few restrictions on its use.Linux is licensed under the GNU General Public License. Further details are athttp://www.gnu.org/licenses/gpl.html.comparison with the BSD license, it imposes significant restrictions on what you can do with the source code.
FreeBSD Linux has aficionados who are prepared to pared to fame anybody who dares suggest fame anybody who dares suggest that it's that it's not better than LinuxIn summary, Linux is also a very good operating system. For many, it's better than FreeBSD

FreeBSD system documentation

FreeBSD comes with a considerable quantity of documentation which we'll look at in the following few pages:

Reading online documentation

You'll find a number of HTML documents in the directory /usr/share/doc/en/books:

In addition to the directory /usr/share/doc/en/books, there's also a directory /usr/share/doc/en/articles with a number of shorter items of documentation.

Note the component en in the pathnames above. That stands for English.A number of these books are also installed in other languages: change en to de for a German version, to es for Spanish, to fr for French, to ja for Japanese, to ru for Russian, or to zh for Chinese. Translation efforts are continuing, so you may find documentation in other languages as well.

If you're running X, you can use a browser like mozilla to read the documents. If you dont have Running yet, use lynx. Both of these programs are included in the CD-ROM distribution. To install them, use sysinstall, which is described on page 92.

lynx is not a complete substitute for complete web browsers such as mozilla:since it is text-only, it is not capable of displaying the large majority of web pages correctly. It's good enough for reading most of the FreeBSD online documentation, however.

In each case, you start the browser with the name of the document, for example:

$ lynx /usr/share/doc/en/books/handbook/index.html
$ mozilla /usr/share/doc/en/books/handbook/index.html &

Enter the & after the invocation of mozilla to free up the window in which you invoke it: mozilla opens its own window.

If you havent installed the documentation, you can still access it from the Live File system CD-ROM. Assuming the CD-ROM is mounted on /cdrom, choose the file /cdrom/usr/share/doc/en/books/handbook/index.html.

Alternatively, you can print out the handbook. This is a little more difficult, and of course you'll lose the hypertext references, but you may prefer it in this form. To format the handbook for printing, you'll need a PostScript printer or ghostscript. See page 271 for more details of how to print PostScript.

The printable version of the documentation doesn't usually come with the CD-ROM distribution. You can pick it up with ftp (see page 433) from ftp://ftp.FreeBSD.ORG/pub/FreeBSD/doc/,which has the same directory structure as described above. For example, you would download the handbook in PostScript form from ftp://ftp.FreeBSD.ORG/pub/FreeBSD/doc/en/books/handbook/book.ps.bz2.

The online manual

The most comprehensive documentation on FreeBSD is the online manual, usually referred to as the man pages. Nearly every program, file, library function, device or interface on the system comes with a short reference manual explaining the basic operation and various arguments. If you were to print it out, it would run to well over 8,000 pages.

When online, you view the man pages with the command man. For example, to learn more about the command ls, type:

$ man ls
LS(1)   FreeBSD Reference Manual   LS(1)
NAME
   ls - list directory contents
SYNOPSIS
   ls [-ACFLRTacdfiloqrstu1][file ... ]
DESCRIPTION
   For each operand that names a file of a type other than directory, ls
   displays its name as well as any requested, associated information. For
   each operand that names a file of type directory, ls displays the names.
(etc)

In this particular example, with the exception of the first line, the text in constant width bold is not input, it's the way it appears on the screen.

The online manual is divided up into sections numbered:

  1. User commands
  2. System calls and error numbers
  3. Functions in the C libraries
  4. Device drivers
  5. File formats
  6. Games and other diversions
  7. Miscellaneous information
  8. System maintenance and operation commands
  9. Kernel interface documentation

In some cases, the same topic may appear in more than one section of the online manual. For example, there is a user command chmod and a system call chmod(). In this case, you can tell the man command which you want by specifying the section number:

$ man 1 chmod 

This command displays the manual page for the user command chmod. References to a particular section of the online manual are traditionally placed in parentheses in written documentation. For example, chmod(l) refers to the user command chmod, and chmod(2) means the systcode call.

This is fine if you know the name of the command and forgot how to use it, but what if you can't recall the command name? You can use man to search for keywords in the command descriptions by using the -k option, or by starting the program apropos:

$ man -k mail $
apropos mail

Both of these commands do the same thing: they show the names of the man pages that have the keyword mail in their descriptions.

Alternatively, you may browse through the /usr/bin directory, which contains most of the system executables. You'll see lots of file names, but you don't have any idea what they do. To find out, enter one of the lines:

$ cd /usr/bin; man -f * 
$ cd /usr/bin; whatis *

Both of these commands do the same thing: they print out a one-line summary of the purpose of the program:

$ cd /usr/bin; man -f *
a2p(1)        - Awk to Perl translator
addftinfo(l)  - add information to troff font files for use with groff
apply(l)      - apply a command to a set of arguments
apropos(1)    - search the whatis database
...etc
Printing man pages

If you prefer to have man pages in print, rather than on the screen, you can do this in two different ways:

GNU info

The Free Software Foundation has its own online hypertext browser called info. Many FSF programs come with either no man page at all, or with an excuse for a man page (gcc, for example). To read the online documentation, you need to browse the info files with the info program, or from Emacs with the info mode. To start info,simply type:

$ info

In Emacs, enter CTRL-h i or ALT-x info. Whichever way you start info, you can get brief introduction by typing h, and a quick command reference by typing ?.

Other documentation on FreeBSD

FreeBSD users have access to probably more top-quality documentation than just about any other operating system. Remember that word UNIX is trademarked. Sure, the lawyers tell us that we can't refer to FreeBSD as UNIX, because UNIX belongs to the Open Group. That doesn't make the slightest difference to the fact that nearly every book on UNIX applies more directly to FreeBSD than any other flavour of UNIX. Why?

Commercial UNIX vendors have a problem, and FreeBSD doesn't help them: why should people buy their products when you can get it free from the FreeBSD Project (or, for that matter, from other free UNIX-like operating systems such as NetBSD, OpenBSD and Linux)? One obvious reason would be "value-added features." So they add features or fix weak points in the system, put a copyright on the changes, and help lock their customers in to their particular implementation. As long as the changes are really useful, this is legitimate, but it does make the operating system less compatible with "standard UNIX," and the books about standard UNIX are less applicable.

In addition, many books are written by people with an academic background. In the UNIX world, this means that they are more likely than the average user to have been exposed to BSD. Many general UNIX books handle primarily BSD, possibly with an additional chapter on the commercial System V version.

In Appendix A, Bibliography, youll find a list of books that I find worthwhile. I'd like to single out some that I find particularly good, and that I frequently use myself:

There are also many active Internet groups that deal with FreeBSD. Read about them in the online handbook.

The FreeBSD community

FreeBSD was developed by a world-wide group of developers. It could not have happened without the Internet. Many of the key players have never even met each other in person; the main means of communication is via the Internet. If you have any kind of Internet connection, you can participate as well. If you don't have an Internet connection, it's about time you got one. The connection doesn't have to be complete: if you can receive email, you can participate. On the other hand, FreeBSD includes all the software you need for a complete Internet connection, not the very limited subset that most PC-based "Internei" packages offer you.

Mailing lists

As it says in the copyright, FreeBSD is supplied as-is, without any support liability. If you're on the Internet, you're not alone, however. Liability is one thing, but there are plenty of people prepared to help you, most for free, some for fee. A good place to start is with the mailing lists. There are a number of mailing lists that you can join. Some of the more interesting ones are:

You can find a complete list of FreeBSD mailing lists on the web site, currently at http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/eresources.html. This address is part of the online handbook and may change when the handbook is modified; follow the link Mailing Lists from http://www.FreeBSD.org/ if it is no longer valid, or if you can't be bothered typing in the URI.

The mailing lists are run by mailman (in the Ports Collection). Join them via the web interface mentioned above. You will receive a mail message from mailman asking you to confirm your subscription by replying to the message. You don't need to put anything in the reply: the reply address is used once only, and you're the only person who will ever see it, so the system knows that it's you by the fact that you replied at all. You also have the option of confirming via a web interface with a specially generated URI. Similar considerations apply in this case.

FreeBSD mailing lists can have a very high volume of traffic. The FreeBSD-questions mailing list, for example, has thousands of subscribers, and many of them are themselves mailing lists. It receives over a hundred messages every day. That's about a million messages a day in total for just one mailing list, so when you sign up for a mailing list, be sure to read the charter. You can find the URI from the mailman confirmation message. It's also a good idea to "lurk" (listen, but not say anything) on the mailing list a while before posting anything: each list has its own traditions.

When submitting a question to FreeBSD-questions, consider the following points:

  1. Remember that nobody gets paid for answering a FreeBSD question. They do it of their own free will. You can influence this free will positively by submitting a well-formulated question supplying as much relevant information as possible. You can influence this free will negatively by submitting an incomplete, illegible, or rude question. It's perfectly possible to send a message to FreeBSD-questions and not get an answer even if you follow these rules. It's much more possible to not get an answer if you don't.
  2. Not everybody who answers FreeBSD questions reads every message: they look at the subject line and decide whether it interests them. Clearly, it's in your interest to specify a subject. "FreeBSD problem" or "Help" aren't enough. If you provide no subject at all, many people wont bother reading it. If your subject isn'tspecific enough, the people who can answer it may not read it.
  3. When sending a new message, well, send a new message. Don't just reply to some other message, erase the old content and change the subject line. That leaves an In-Reply-To: header which many mail readers use to thread messages, so your message shows up as a reply to some other message. People often delete messages a whole thread at a time, so apart from irritating people, you also run a chance of having the message deleted unread.
  4. Format your message so that it is legible, and PLEASE DON'T SHOUT!!!!!. It's really painful to try to read a message written full of typos or without any line breaks. Alot of badly formatted messages come from bad mailers or badly configured mailers. The following mailers are known to send out badly formatted messages without you finding out about them:

    Eudora

    Exmh

    Microsoft Exchange

    Microsoft Internet Mail

    Microsoft Outlook

    Netscape

    As you can see, the mailers in the Microsoft world are frequent offenders. If at all possible, use a UNIX mailer. If you must use a mailer under Microsoft environments, make sure it is set up correctly. Try not to use MIME: a lot of people use mailers which don't get on very well with MIME.

    For further information on this subject, check out http://www.lemis.com/email.html.

  5. Make sure your time and time zone are set correctly. This may seem a little silly, since your message still gets there, but many of the people you are trying to reach get several hundred messages a day.Theyfrequently sort the incoming messages by subject and by date, and if your message doesn't come before the first answer, they may assume they missed it and not bother to look.
  6. Dont include unrelated questions in the same message. Firstly, along message tends to scare people off, and secondly, it's more difficult to get all the people who can answer all the questions to read the message.
  7. Specify as much information as possible. This is a difficult area: the information you need to submit depends on the problem. Heres start:
    • If you get error messages, don't say "I get error messages", say (for example) "I get the error message No route to host".
    • If your system panics, don'tsay "My system panicked", say (for example) "my system panicked with the message free vnode isn't".
    • If you have difficulty installing FreeBSD, please tell us what hardware you have, particularly if you have something unusual.
    • If, for example, you have difficulty getting PPP to run, describe the configuration. Which version of PPP do you use? What kind of authentication do you have? Do you have a static or dynamic IP address? What kind of messages do you get in the log file? See Chapter 20, Configuring PPP, for more details in this particular case.
  8. If you don't get an answer immediately, or if you don't even see your own message appear on the list immediately, don't resend the message. Wait at least 24 hours. The FreeBSD mailer offloads messages to a number of subordinate mailers around the world. Usually the messages come through in a matter of seconds, but sometimes it can take several hours for the mail to get through.
  9. If you do all this, and you still don't get an answer, there could be other reasons. For example, the problem is so complicated that nobody knows the answer, or the person who does know the answer was offine. If you don't get an answer after, say, a week, it might help to re-send the message. If you don't get an answer to your second message, though, you're probably not going to get one from this forum. Resending the same message again and again will only make you unpopular.
How to follow up to a question

Often you will want to send in additional information to a question you have already sent. The best way to do this is to reply to your original message. This has three advantages:

  1. You include the original message text, so people will know what you're talking about. Don't forget to trim unnecessary text, though.
  2. The text in the subject line stays the same (you did remember to put one in, didn't you?). Many mailers will sort messages by subject. This helps group messages together.
  3. The message reference numbers in the header will refer to the previous message. Some mailers, such as mutt, can thread messages, showing the exact relationships between the messages.

There are more suggestions, in particular for answering questions, at http://www.lemis.com/questions.html. See also Chapter 26, Electronic mail: clients for more information about sending mail messages. You may also like to check out the FreeBSD web site at http://www.FreeBSD.org/ and the support page at http://www.FreeBSD.org/support.html.

In addition, a number of companies offer support for FreeBSD. See the web page http://www.FreeBSD.org/commercial/consulting_bycat.html for some possibilities.

Unsubscribing from the mailing lists

There's a lot of traffic on the mailing lists, particularly on FreeBSD-questions. You may find you can't take it and want to get out again. Again, you unsubscribe from the list either via the web or via a special mail address, not by sending mail to the the list. Each message you get from the mailing lists finishes with the following text:

freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"

Don't be one of those people who send the unsubscribe request to the mailing list instead.

User groups

But how about meeting FreeBSD users face to face? There are a number of user groups around the world. If you live in a big city, chances are that there's one near you. Check http://www.FreeBSD.org/support.html#user for a list. If you don't find one, consider taking the initiative and starting one.

In addition, USENIX holds an annual conference, the BSDCon, which deals with technical aspects of the BSD operating systems. It's also a great opportunity to get to know other users from around the world. If you're in Europe, there is also a BSDCon Europe, which at the time of writing was not run by USENIX. See http://www.eurobsdcon.org for more details.

Reporting bugs

If you find something wrong with FreeBSD, we want to know about it, so that we can fix it. To report a bug, use the send-pr program to send it as a mail message.

There used to be a web form at http://www.FreeBSD.org/send-pr.html, but it has been closed down due to abuse.

The Berkeley daemon

The little daemon at the right symbolizes BSD. It is included with kind permission of Marshall Kirk McKusick, one of the leading members of the former Computer Sciences Research Group at the University of California at Berkeley, and owner of the daemon's copyright. Kirk also wrote the foreword to this course .



. 1.1.

The daemon has occasionally given rise to a certain amount of confusion. In fact, it's a joking reference to processes that run in the backgroundsee Chapter 8, Taking control, page 150, for a description. The outside world occasionally sees things differently, as the following story indicates:

Newsgroups:   alt.humor.best-of-usenet
Subject:   [comp.org.usenix] A Great Daemon Story
From:   Rob Kolstad <kolstad@bsdi.com>
Newsgroups:   comp.org.usenix
Subject:   A Great Daemon Story

Linda Branagan is an expert on daemons. She has a T-shirt that sports the daemon in tennis shoes that appears on the cover of the 4.3BSD manuals and The Design and Implementation of the 4.3BSD UNIX Operating System by S. Leffler, M. McKusick, M. Karels, J. Quarterman, Addison Wesley Publishing Company, Reading, MA 1989.

She tells the following story about wearing the 4.3BSD daemon T-shirt:

Last week I walked into a local "home style cookin' restaurant/watering hole" in Texas to pick up a take-out order. I spoke briefly to the waitress behind the counter, who told me my order would be done in a few minutes.

So, while I was busy gazing at the farm implements hanging on the walls, I was approached by two "natives." These guys might just be the original Texas rednecks.

"Pardon us, ma'am. Mind if we ask you a question?"

Well, people keep telling me that Texans are real friendly, so I nodded.

"Are you a Satanist?"

Well, at least they didn't ask me if I liked to party.

"Uh, no, I can't say that I am."

"Gee, ma'am. Are you sure about that?" they asked.

I put on my biggest, brightest Dallas Cowboys cheerleader smile and said, "No, I'm positive. The closest I've ever come to Satanism is watching Geraldo."

"Hmmm. Interesting. See, we was just wondering why It is you have the lord of darkness on your chest there."

I was this close to slapping one of them and causing a scenethen I stopped and noticed the shirt I happened to be wearing that day. Sure enough, it had a picture of a small, devilish-looking creature that has for some time now been associated with a certain operating system. In this particular representation, the creature was wearing sneakers.

They continued: "See, ma'am, we don'texactly appreciate it when people showoff pictures of the devil. Especially when he'slookin' so friendly."

These idiots sounded terrifyingly serious.

Me: "Oh, well, see, this isn't really the devil, it's just, well, it's sort of a mascot.

Native: "And what kind of football team has the devil as a mascot?"

Me: "Oh, it's not a team. It's an operatinguh, a kind of computer."

I figured that an ATM machine was about as much technology as these guys could handle, and I knew that if I so much as uttered the word "UNIX" I would only make things worse.

Native: "Where does this satanical computer come from?"

Me: "California. And there's nothing satanical about it really."

Somewhere along the line here, the waitress noticed my predicamentbut these guys probably outweighed her by 600 pounds, so all she did was look at me sympathetically and run off into the kitchen.

Native: "Ma'am, I think you're lying. And we'd appreciate it if you'd leave the premises now."

Fortunately, the waitress returned that very instant with my order, and they agreed that it would be okay for me to actually pay for my food before I left. While I was at the cash register, they am used themselves by talking to each other.

Native #1: "Do you think the police know about these devil computers?"

Native #2: "If they come from California, then the FBI oughta know about 'em."

Theye scorted me to the door. I tried one last time: "You're really blowing this all out of proportion. Alot of people use this 'kind of computers.' Universities, researchers, businesses. They're actually very useful."

Big, big, big mistake. I should have guessed at what came next.

Native:"Does the government use these devil computers?"

Me: "Yes."

Another big boo-boo.

Native:"And does the government pay for 'em? With our tax dollars?"

Idecided that it was time to jump ship.

Me: "No. Nope. Not at all. Your tax dollars never entered the picture at all. I promise. No sir, not a penny. Our good Christian congressmen would never let something like that happen. Nope. Never. Bye."

Texas. What acountry.

The daemon tradition goes back quite a way. Asrecently as 1996, after the publication of the first edition of this book, the following message went through the FreeBSD-chat mailing list:

To:   "Jonathan M. Bresler" <jmb@freefall.freebsd.org>
Cc:   obrien@antares.aero.org (Mike O'Brien),
    joerg_wunsch@uriah.heep.sax.de,
    chat@FreeBSD.org, juphoff@tarsier.cv.nrao.edu 
Date:   Tue, 07 May 1996 16:27:20 -0700
Sender:   owner-chat@FreeBSD.org
    >details and gifs PLEASE!
   If you insist. :-)
   Sherman, set the Wayback Machine for around 1976 or so (see Peter Salus' A 
Quarter Century of UNIX for details), when the first really national UNIX 
meeting was held in Urbana, Illinois. This would be after the "forty people 
in a Brooklyn classroom" meeting held by Mel Ferentz (yeah I was at that too) 
and the more-or-less simultaneous West Coast meeting(s) hosted by SRI, but before 
the UNIX Users Group was really incorporated as a going concern.
   I knew Ken Thompson and Dennis Ritchie would be there. I was living in Chicago 
at the time, and so was comic artist Phil Foglio, whose star was just beginning 
to rise. At that time I was a bonded locksmith. Phil's roommate had unexpectedly 
split town, and he was the only one who knew the combination to the wall safe in 
their apartment. This is the only apartment I've ever seen that had a wall safe, 
but it sure did have one, and Phil had some stuff locked in there. I didn't hold 
out much hope, since safes are far beyond where I was (and am) in my locks mithing 
sphere of competence, but I figured "no guts no glory" and told him I'd 
give it a whack. In return, I told him, he could do some T-shirt art for me. He readily agreed.
   Wonder of wonders, this safe was vulnerable to the same algorithm that Master 
locks used to be susceptible to. I opened it in about 15 minutes of manipulation. 
It was my greatest moment as a locksmith and Phil was overjoyed. I went down to my 
lab and shot some Polaroid snaps of the PDP-11 system I was running UNIX on at the 
time, and gave it to Phil with some descriptions of the visual puns I wanted: pipes, 
demons with forks running along the pipes, a "bit bucket" named /dev/null, all that.
   What Phil came up with is the artwork that graced the first decade's worth of 
"UNIX T-shirts," which were made by a Ma and Paoperation in a Chicago 
suburb. They turned out transfer art using a 3M color copier in their basement. 
Hence, the PDP-11 is reversed (the tape drives are backwards) but since Phil left off 
the front panel, this was hard to tell. His trademark signature was photo-reversed, 
but was recopied by the T-shirt people and "re-forwardized," which is why 
it looks a little funny compared to his real signature.
   Dozens and dozens of these shirts were produced. Bell Labs alone accounted for 
an order of something like 200 for a big picnic. However, only four (4) REAL 
originals were produced: these have a distinctive red collar and sleeve cuff. 
One went to Ken, one to Dennis, one to me, and one to my then-wife. I now possess 
the latter two shirts. Ken and Dennis were presented with their shirts at the Urbana conference.
People ordered these shirts direct from the Chicago couple. Many years later, 
when I was living in LA, I got a call from Armando Stettner, then at DEC, 
asking about that now-famous artwork. I told him I hadn't talked to the Illinois 
T-shirt makers in years. At his request I called them up. They'd folded the operation 
years ago and were within days of discarding all the old artwork. I requested its return, 
and duly received It back in the mail. It looked strange, seeing it again in its 
original form, a mirror image of the shirts with which I and everyone else were now familiar.
I sent the artwork to Armando, who wanted to give it to the Ultrix marketing people. 
   They came out with the Ultrix poster that showed a nice shiny Ultrix machine 
contrasted with the chewing-gum-and-string PDP-11 UNIX people were familiar with.
They still have the artwork, so far as I know.
   I no longer recall the exact contents of the letter I sent along with the artwork. 
I did say that as far as I knew, Phil had no residual rights to the art, 
since it was a 'work made for hire', though nothing was in writing 
(and note this was decades before the new copyright law). I do not now recall 
if I explicitly assigned all rights to DEC. What is certain is that John Lassiter's daemon, 
whether knowingly borrowed from the original, or created by parallel evolution, 
postdates the first horde of UNIX daemons by at least a decade and probably more. 
And if Lassiter's daemon looks a lot like a Phil Foglio creation, there's are a son.
   I have never scanned in Phil's artwork; I've hardly ever scanned in anything, 
so I have no GIFs to show. But I have some very very old UNIX T-shirts in 
startlingly good condition. Better condition than I am at any rate: 
I no longer fit into either of them.

Mike O'Brien
Creaky antique

Note the date of this message: it appeared since the first edition of this book. Since then, the daemon image has been scanned in, and you can find a version at http://www.mckusick.com/beastie/shirts/usenix.html

2. Before you install

Using old hardware; PC Hardware; How the system detects hardware; Configuring ISA cards; PCMCIA, PC Card and Card Bus; Universal Serial Bus; Disks; Disk data layout; Making the file systems; Disk size limitations; Display hardware; The hardware; Compaq/Digital Alpha machines; The CD-ROM distribution.

FreeBSD runs on just about any modern PC, Alpha or 64 bit SPARC machine. You can skip this chapter and the next and move to chapter 3, and you'll have a very good chance of success. Nevertheless, it makes things easier to know the contents of this chapter before you start. If you do run into trouble, it will give you the background information you need to solve the trouble quickly and simply.

FreeBSD also runs on most Intel-based laptops; in general the considerations above apply for laptops as well. In the course of the book we'll see examples of where laptops require special treatment.

Most of the information here applies primarily to Intel platforms. We'll look at the Compaq Alpha architecture on page 42. The first release of FreeBSD to support the SPARC 64 architecture is 5.0, and support is still a little patchy. At the time of going to press, it's not worth describing, since it will change rapidly. The instructions on the CD-ROM distribution are currently the best source of information on running FreeBSD on SPARC 64.

Using old hardware

FreeBSD runs on all relatively recent machines. In addition, a lot of older hardware that is available for a nominal sum, or even for free, runs FreeBSD quite happily, though you may need to take more care in the installation.

FreeBSD does not support all PC hardware: the PC has been on the market for over 20 years, and it has changed a lot in that time. In particular:

If you're planning to install FreeBSD on an old machine, consider the following to be an absolute minimum:

You don't absolutely need a keyboard and display board: many FreeBSD machines run server tasks with neither keyboard nor display. Even then, though, you may find it convenient to put a display board in the machine to help in case you run into trouble.

When I say absolute minimum, I mean it. You can't do very much with such a minimal system, but for some purposes it might be adequate. You can improve the performance of such a minimal system significantly by adding memory. Before you go to the trouble to even try such a minimal installation, consider the cost of another 16 MB of memory. And you can pick up better machines than this second-hand for $50. Is the hassle worth it?

To get full benefits from a desktop or laptop FreeBSD system (but not from a machine used primarily as a server), you should be running the X Window system. This uses more memory. Consider 32 MB a usable minimum here, though thanks to FreeBSD's virtual memory system, this is not such a hard limit as it is with some other systems.

The speed of a virtual memory-based system such as FreeBSD depends at least as much on memory performance as on processor performance. If you have, say, a 486DX-33 and 16 MB of memory, upgrading memory to 32 MB will probably buy you more performance than upgrading the motherboard to a Pentium 100 and keeping the 16 MB memory. This applies for a usual mix of programs, in particular, programs that don't perform number crunching.

Any SPARC 64 machine runs FreeBSD acceptably, as the machines are relatively new. If you're running Intel or Alpha, consider the following the minimum for getting useful work done with FreeBSD and X:

Your mileage may vary. During the review phase of an earlier edition of this book, one of the reviewers stated that he was very happy with his machine, which has a 486-33 processor, 16MB main memory, and 1 MB memory on his display board. He said that it ran a lot faster than his Pentium 100 at work, which ran Microsoft. The moral: if your hardware doesn't measure up to the recommended specifi cation, don't be discouraged. Try it out anyway.

Beyond this minimum, FreeBSD supports a large number of other hardware components.

Device drivers

The FreeBSD kernel is the only part of the system that can access the hardware. It includes device drivers, which control the function of peripheral devices such as disks, displays and network boards. When you install new hardware, you need a driver for it.

There are two ways to get a driver into the kernel: you can build a kernel that includes the driver code, or you can load a driver module (Kernel Loadable Module or kld) into the kernel at run time. Not all drivers are available as klds. If you need one of these drivers, and it's not included in the standard kernel, you have to build a new kernel. We look at building kernels in Chapter 33.

The kernel configuration supplied with FreeBSD distributions is called GENERIC after the name of the configuration file that describes it. It contains support for most common devices, though support for some older hardware is missing, usually because it conflicts with more modern drivers. For a full list of currently supported hardware, read the web page http://www.FreeBSD.org/releases/ and select the link Hardware Notes for the release you're interested in. This file is also available on installed FreeBSD systems as /usr/share/doc/en_US.ISO_8859-1/books/faq/hardware.html. It is also available in other languages; see the subdirectories of /usr/share/doc.

PC Hardware

This section looks at the information you need to understand to install FreeBSD on the i386 architecture. In particular, in the next section we'll look at how FreeBSD detects hardware, and what to do if your hardware doesn't correspond to the system's expectations. On page 32 we'll see how FreeBSD and other PC operating systems handle disk space, and how to set up your disk for FreeBSD.

Some of this information also applies to the Alpha and SPARC 64 architectures. We'll look at the differences for the Alpha architecture on page 42. Currently the SPARC 64 implementation is changing too fast to describe it in a meaningful manner.

Since the original PC, a number of hardware standards have come, and some have gone:

How the system detects hardware

When the system starts, each driver in the kernel examines the system to find any hardware that it might be able to control. This examination is called probing. Depending on the driver and the nature of the hardware it supports, the probe may be clever enough to set up the hardware itself, or to recognize its hardware no matter how it has been set up, or it may expect the hardware to be set up in a specific manner in order to find it. In general, you can expect PCI drivers to be able to set up the card to work correctly. In the case of ISA or EISA cards, you may not be as lucky.

Configuring ISA cards

ISA cards are rapidly becoming obsolete, but sometimes they're still useful:

Most ISA cards require some configuration. There are four main parameters that you may need to set for PC controller boards:

  1. The port address is the address of the first of possibly several control registers that the driver uses to communicate with the board. It is normally specified in hexadecimal, for example 0x320
    If you come from a Microsoft background, you might be more used to the notation 320H. The notation 0x320 comes from the C programming language. Youll see a lot of it in UNIX

    Each board needs its own address or range of addresses. The ISA architecture has a sharply limited address range, and one of the most frequent causes of problems when installing a board is that the port addresses overlap with those of another board

    Beware of boards with a large number of registers. Typical port addresses end in (hexadecimal) 0. Don't rely on being able to take any unoccupied address ending in 0, though: some boards, such as Novell NE2000 compatible Ethernet boards, occupy up to 32 registersor example, from 0x320 to 0x33f. Note also that a number of addresses, such as the serial and parallel ports, often end in 8.

  2. Boards use an Interrupt Request, also referred to as IRQ, to get the attention of the driver when a specific event happens. For example, when a serial interface reads a character it generates an interrupt to tell the driver to collect the character. Interrupt requests can sometimes be shared, depending on the driver and the hardware. There are even fewer interrupt requests than port addresses: a total of 15, of which a number are reserved by the motherboard. You can usually expect to be able to use IRQs 3, 4, 5, 7, 9, 10, 11 and 12. IRQ 2 is special: due to the design of the original IBM PC/AT, it is the same thing as IRQ 9. FreeBSD refers to this interrupt as IRQ 9

    As if the available interrupts weren't already restricted enough, ISA and PCI boards use the same set of interrupt lines. PCI cards can share interrupt lines between multiple boards, and in fact the PCI standard only supports four interrupts, called INTA, INTB, INTC and INTD. In the PC architecture they map to four of the 15 ISA interrupts. PCI cards are self-configuring, so all you need to do is to ensure that PCI and ISA interrupts don't conflict. You normally set this up in a BIOS setup menu.

  3. Some high-speed devices perform Direct Memory Access, also known as DMA, to transfer data to or from memory without CPU intervention. To transfer data, they assert a DMA Request (DRQ) and wait for the bus to reply with a DMA Acknowledge (DACK). The combination of DRQ and DACK is sometimes called a DMA Channel. The ISA architecture supplies 7 DMA channels, numbered 0 to 3 (8 bit) and 5 to 7 (16 bit). The floppy driver uses DMA channel 2. DMA channels may not be shared.
  4. Finally, controllers may have on-board memory, sometimes referred to as I/O memory or IOmem. It is usually located at addresses between 0xa0000 and 0xeffff.

If the driver only looks at specific board configurations, you can set the board to match what the driver expects, typically by setting jumpers or using a vendor-supplied diagnostic program to set on-board configuration memory, or you can build a kernel to match the board settings.

PCMCIA, PC Card and CardBus

Laptops don't have enough space for normal PCI expansion slots, though many use a smaller PCI card format. It's more common to see PC Card or CardBus cards, though. PC Card was originally called PCMCIA, which stands for Personal Computer Memory Card International Association: the first purpose of the bus was to expand memory. Nowadays memory expansion is handled by other means, and PC Card cards are usually peripherals such as network cards, modems or disks. It's true that you can insert compact flash memory for digital cameras into a PC Card adapter and access it from FreeBSD, but even in this case, the card looks like a disk, not a memory card.

The original PC Card standard already has one foot in the grave: its a 16 bit bus that doesn't work well with modern laptops. The replacement standard has a 32 bit wide bus and is called CardBus.The cards look almost identical, and most modern laptops support both standards. In this book I'll use use the term PC Card to include CardBus unless otherwise stated. FreeBSD Release 5 includes completely new PC Card code. It now supports both 16 bit PC Card and 32 bit CardBus cards.

PC Card offers one concept that conventional cards don't: the cards are hot swappable. You can insert them and remove them in a running system. This poses a number of potential problems, some of which are only partially solved.

PC Card and CardBus cards

PC Card and CardBus both use the same form factor cards: they are 54 mm wide and at least 85 mm long, though some cards, noticeably wireless networking cards, are up to 120 mm long and project beyond the casing of the laptop. The wireless cards contain an antenna in the part of the card that projects from the machine.

PC Card cards can have one of three standard thicknesses:

The GENERIC FreeBSD kernel contains support for PC Card, so you don't need to build a new kernel.

Universal Serial Bus

The Universal Serial Bus (USB) is a new way of connecting external peripherals, typically those that used to be connected by serial or parallel ports. It's much faster than the old components: the old serial interface had a maximum speed of 115,200 bps, and the maximum you can expect to transfer over the parallel port is about 1 MB/s. By comparison, current USB implementations transfer data at up to 12 Mb/s, and a version with 480 Mb/s is in development.

As the name states, USB is a bus: you can connect multiple devices to a bus. Currently the most common devices are mid-speed devices such as printers and scanners, but you can connect just about anything, including keyboards, mice, Ethernet cards and mass storage devices.

Disks

A number of different disks have been used on PCs:

Disk data layout

Before you install FreeBSD, you need to decide how you want to use the disk space available to you. If desired, FreeBSD can coexist with other operating systems on the Intel platform. In this section, we'll look at the way data is laid out on disk, and what we need to do to create FreeBSD file systems on disk.

PC BIOS and disks

The basics of disk drives are relatively straightforward: data is stored on one or more rotating disks with a magnetic coating similar in function to the coating on an audio tape. Unlike a tape, however, disk heads do not touch the surface: the rotating disk produces an air pressure against the head, which keeps it floating very close to the surface. The disk has (usually) one read/write head for each surface to transfer data to and from the system. People frequently talk about the number of heads, not the number of surfaces, though strictly speaking this is incorrect: if there are two heads per surface (to speed up access), you're still interested in the number of surfaces, not the number of heads.

While transferring data, the heads are stationary, so data is written on disks in a number of concentric circular tracks. Logically, each track is divided into a number of sectors, which nowadays almost invariably contain 512 bytes. A single positioning mechanism moves the heads from one track to another, so at any one time all the tracks under the current head position can be accessed without repositioning. This group of tracks is called a cylinder.

Since the diameter of the track differs from one track to the other, so does the storage capacity per track. Nevertheless, for the sake of simplicity, older drives, such as ST-506 (MFM and RLL) drives, had a fixed number of sectors per track. To perform a data transfer, you needed to tell the drive which cylinder, head and sector to address. This mode of addressing is thus called CHS addressing.

Modern disks have a varying number of sectors per track on different parts of the disk to optimize the storage space, and for the same reason they normally store data on the disk in much larger units than sectors. Externally, they translate the data into units of sectors, and they also optionally maintain the illusion of "tracks" and "heads," though the values have nothing to do with the internal organization of the disk. Nevertheless, BIOS setup routines still give you the option of specifying information about disk drives in terms of the numbers of cylinders, heads and sectors, and some insist on it. In reality, modern disk drives address sectors sequentially, so-called Logical Block Addressing or LBA. CHS addressing has an additional problem: various standards have limited the size of disks to 504 MB or 8 GB. We'll look at that in more detail on page 39.

SCSI drives are a different matter: the system BIOS normally doesn't know anything about them. They are always addressed in LBA mode. It's up to the host adapter to interrogate the drive and find out how much space is on it. Typically, the host adapter has a BIOS that interrogates the drive and finds its dimensions. The values it determines may not be correct: the PC BIOS 1 GB address limit (see page 39) might bite you. Check your host adapter documentation for details.

Disk partitioning

The PC BIOS divides the space on a disk into up to four partitions, headed by a partition table. For Microsoft systems, each partition may be either a primary partition that contains a file system (a "drive" in Microsoft terminology), or an extended partition that contains multiple file systems (or "logical partitions").

FreeBSD does not use the PC BIOS partition table directly. It maintains its own partitioning scheme with its own partition table. On the PC platform, it places this partition table in a single PC BIOS partition, rather in the same way that a PC BIOS extended partition contains multiple "logical partitions." It refers to PC BIOS partitions as "slices."

This double usage of the word partition is really confusing. In this book, I follow BSD usage, but I continue to refer to the PC BIOS partition table by that name.

Partitioning offers the flexibility that other operating systems need, so it has been adopted by all operating systems that run on the PC platform. Figure 2-1 shows a disk with all four slices allocated. The Partition Table is the most important data structure. It contains information about the size, location and type of the slices (PC partitions). The PC BIOS allows one of these slices to be designated as active: at system startup time, its bootstrap record is used to start the system.

The partition table of a boot disk also contains a Master Boot Record (MBR), which is responsible for finding the correct slice and booting it. The MBR and the partition table take up the first sector on disk, and many people consider them to be the same thing. You only need an MBR on disks from which you boot the system.

 Partition table


. 2.1. Partition table

PC usage designates at least one slice as the primary partition, the C: drive. Another slice may be designated as an extended partition that contains the other "drives" (all together in one slice).

UNIX systems have their own form of partitioning which predates the PC and is not compatible with the PC method. As a result, all versions of UNIX that can coexist with Microsoft implement their own partitioning within a single slice (PC BIOS partition). This is conceptually similar to an extended partition. FreeBSD systems define up to eight partitions per slice. They can be used for the following purposes:

Block and character devices

Traditional UNIX treats disk devices in two different ways. As we have seen, you can think of a disk as a large number of sequential blocks of data. Looking at it like this doesn't give you a file systemit's more like treating it as a tape. UNIX calls this kind of access raw access. You'll also hear the term character device.

Normally, of course, you want files on your disk: you don't care where they are, you just want to be able to open them and manipulate them. In addition, for performance reasons the system keeps recently accessed data in a buffer cache. This involves a whole lot more work than raw devices. These devices are called block devices.

By contrast with UNIX, Linux originally did not have character disk devices. Starting with Release 4.0, FreeBSD has taken the opposite approach: there are now no user-accessible block devices anymore. There are a number of reasons for this:

The removal of block devices caused significant changes to device naming. In older releases of FreeBSD, the device name was the name of the block device, and the raw (character) device had the letter r at the beginning of the name, as shown in the example above.

Let's look more carefully at how BSD names its partitions:

Figure 2-2 shows a typical layout on a system with a single SCSI disk, shared between Microsoft and FreeBSD. You'll note that partition /dev/da0s3c is missing from the FreeBSD slice, since it isn't a real partition. Like the PC BIOS partition table, the disk label contains information necessary for FreeBSD to manage the FreeBSD slice, such as the location and the lengths of the individual partitions. The bootstrap is used to load the kernel into memory. Well look at the boot process in more detail in Chapter 29 .

Partition table with FreeBSD file system


. 2.2. Partition table with FreeBSD file system

Table 2-1 gives you an overview of the devices that FreeBSD defines for this disk.

2.1. Disk partition terminology
Slice nameUsage
/dev/da0s1First slice (PC BIOS C: partition)
/dev/da0s2Second slice (PC BIOS extended partition)
/dev/da0s3Third slice (PC BIOS partition), FreeBSD
/dev/da0s5First drive in extended PC BIOS partition (D:)
/dev/da0s6Second drive in extended PC BIOS partition (E: )
/dev/da0s3aThird slice (PC BIOS partition), partition a (root file system)
/dev/da0s3bThird slice (PC BIOS partition), partition b (swap space)
/dev/da0s3cThird slice (PC BIOS partition), entire partition
/dev/da0s3eThird slice (PC BIOS partition), partition e (/usr file system)
/dev/da0aCompatibility partition, root file system, same as /dev/da0s1a
/dev/da0bCompatibility partition, swap partition, same as /dev/da0s1b
/dev/da0cWhole BSD slice, same as /dev/da0s1c
/dev/da0eCompatibility partition, usr file system, same as /dev/da0s1e

Making the file systems

Armed with this knowledge, we can now proceed to make some decisions about how to install our systems. First, we need to answer some questions:

If you already have another system installed on the disk, it is best to use that system's tools for manipulating the partition table. FreeBSD does not normally have difficulty with partition tables created by other systems, so you can be reasonably sure that the other system will understand what it has left. If the other system is Microsoft, and you have a slice that you don't need, use the MS-DOS FDISK program to free up enough space to install FreeBSD. If you don't have a slice to delete, you can use the FIPS program to create onesee Chapter 5 , Installing FreeBSD, page 52.

If for some reason you can't use MS-DOS FDISK, for example because you're installing FreeBSD by itself, FreeBSD also supplies a program called fdisk that manipulates the partition table. Normally you invoke it indirectly via the sysinstall programsee page 63.

Disk size limitations

Disk storage capacity has grown by several orders of magnitude since FreeBSD was first released. As it did so, a number of limits became apparent:

None of these problems affect FreeBSD directly. The FreeBSD bootstrap no longer uses the system BIOS, so it is not bound by the restrictions of the BIOS and the controller. If you use another operating system's loader, however, you could have problems. If you have the choice, use LBA addressing. Unfortunately, you can't do so if the disk already contains software that uses CHS addressing.

Other things to consider are:

Display hardware

For years, UNIX users have worked with a single 80x25 character mode display. Many people consider this extremely old-fashioned, but in fact the flexibility of the UNIX system made this quite a good way to work. Still, there's no doubt of the advantage of a system that offers the possibility of performing multiple operations at once, and this is one of the particular advantages of UNIX. But you normally need a terminal to interact with each task. The best way to do this is with the X Window System. You might also want to use a desktop, a set of programs that offer commonly used functionality.

In many other environments, the GUI and the graphical display are the same thing, and in some systems, notably Microsoft, there is no clear distinction between the operating system and the GUI. In UNIX, there are at least four levels of abstraction:

Why do it this way? Because it gives you more choice. There are dozens of window managers available, and also several desktops. You're not locked in to a single product. This has its down side, though: you must make the choice, and so setting up X requires a little more thought than installing Microsoft.

The hardware

X runs on almost any hardware. That doesn't mean that all hardware is equal, of course. Here are some considerations:

The keyboard

X uses the keyboard a lot more than Microsoft. Make sure you get a good one.

The mouse

X prefers a three-button mouse, though it has provisions for up to five buttons. It can support newer mice with rollers and side buttons, but most software does not use them. Some mice, such as the Logitech wireless mouse, require undocumented sequences to enable some buttons (the thumb button in the case of Logitech). X does not support this button.

Get the best mouse you can. Prefer a short, light switch. It must have at least three buttons. Accept no substitutes. Look for one with an easy-to-use middle button. Frequently mice with both a middle button and a roller make it difficult to use the middle button: it's either misplaced, too heavy in action, or requires pressing on the roller (and thus possibly turning it). All of these prove to be a nuisance over time.

Older mice connected via the serial port or a special card (&bus mouse&). Nowadays most mice are so-called PS/2 mice, and USB mice are becoming more popular.

The display board and monitor

X enables you to do a lot more in parallel than other windowing environments. As a result, screen real estate is at a premium. Use as big a monitor as you can afford, and as high a resolution as your monitor can handle. You should be able to display a resolution of 1600x1200 on a 21" monitor, 1280x1024 on a 17" monitor, and 1024x768 on a 14" monitor. Premium quality 21" monitors can display 2048x1536. If thats not enough, we'll look at multiple monitor configurations on page 523.

Laptop hardware

If you have a laptop, you don't get any choice. The display has a native resolution which you can't change. Most laptops display lower resolutions by interpolation, but the result looks much worse than the native resolution. LCD screens look crisper than CRT monitors, so you can choose higher resolutions modern laptops have display resolutions of up to 1600x1200.

If you're going to use your laptop for presentations with overhead projectors, make sure you find one that can display both on the internal screen and also on the external output at the same time, while maintaining a display resolution of 1024x768: not many over head projectors can display at a higher resolution.

Compaq/Digital Alpha machines

FreeBSD also supports computers based on the Compaq (previously Digital) AXP processor, commonly called Alpha. Much of the information above also applies to the Alpha; notable exceptions are:

In this section we'll look at some additional topics that only apply to the Alpha.

FreeBSD requires the SRM console firmware, which is used by Tru64 (formerly known as Digital UNIX). It does not work with the ARC firmware (sometimes called AlphaBIOS) used with Microsoft NT. The SRM firmware runs the machine in 64 bit mode, which is required to run FreeBSD, while the ARC firmware sets 32 bit mode. If your system is currently running Tru64, you should be able to use the existing SRM console.

The SRM console commands differ from one version to another. The commands supported by your version are described in the hardware manual that was shipped with your system. The console help command lists all supported console commands. If your system

has been set to boot automatically, you must type Ctrl-C to interrupt the boot process and get to the SRM console prompt (>>>). If the system is not set to boot automatically, it displays the SRM console prompt after performing system checks.

All SRM console versions support the set and show commands, which operate on environment variables that are stored in non-volatile memory. The show command lists all environment variables, including those that are read-only.

Alpha's SRM is picky about which hardware it supports. For example, it recognizes NCR SCSI boards, but it doesn't recognize Adaptec boards. There are reports of some Alphas not booting with particular video boards. The GENERIC kernel configuration (/usr/src/sys/alpha/conf/GENERIC) shows what the kernel supports, but that doesn't mean that the SRM supports all the devices. In addition, the SRM support varies from one machine to the next, so there's a danger that what's described here won't work for you.

Other differences for Alpha include:

The CD-ROM distribution

The easiest way to install FreeBSD is from CD-ROM. You can buy them at a discount with the order form at the back of the book, or you can download an ISO image from ftp://ftp.FreeBSD.org and create your own CD-ROM. There are a number of CD-ROMs in a FreeBSD distribution, but the only essential one is the first one, the Installation CD-ROM. It contains everything you need to install the system itself. The other CD-ROMs contain mainly installable packages. Individual releases may contain other data, such as a copy of the source code repository. We'll take a more detailed look at the installation CD-ROM here.

Installation CD-ROM

The Installation CD-ROM contains everything you need to install FreeBSD on your system. It supplies two categories of installable software:

packages/Latest contains the latest versions of the packages.

packages/All contains a large subset of the Ports Collection. To make it easier for you to find your way around them, symbolic links to appropriate packages have been placed in the directories archivers, astro, audio, benchmarks, biology, cad, chinese, comms, converters, databases, deskutils, devel, editors, emulators, french, ftp, games, german, graphics, hebrew, irc, japanese, java, korean, lang, mail, math, mbone, misc, net, news, palm, picobsd, plan9, print, russian, science, security, shells, sysutils, templates, textproc, ukrainian, vietnamese, www, x11, x11-clocks, x11-fm, x11-fonts, x11-servers, x11-toolkits and x11-wm. Don't get the impression that these are different packagesthey are really pointers to the packages in All. You will find a list of the currently available packages in the file packages/INDEX.

We'll look at the Ports Collection in more detail in Chapter 9.

2.2. lists typical files in the main directory of the installation CD-ROM
FileContents
ERRATA.TXTAlist of last-minute changes. Read this file. It can save you a lot of headaches.
HARDWARE.TXTAlist of supported hardware.
INSTALL.TXTInformation about installing FreeBSD.
README.TXTThe traditional first file to read. It describes how to use the other files.
RELNOTES.TXTRelease notes.
baseInstallation directory: the base distribution of the system. This is the only required directory for installation. See Chapter 5, Installing FreeBSD, for more detail.
bootFiles related to booting, including the installation kernel.
catpagesPre-formatted man pages. See page 13 for more detail.
cdrom.infMachine-readable file describing the CD-ROM contents for the benefit of sysinstall.
compat1xDirectory containing libraries to maintain compatibility with Release 1.X of FreeBSD
compat20Directory containing libraries to maintain compatibility with Release 2.0 of FreeBSD.
compat21Directory containing libraries to maintain compatibility with Release 2.1 of FreeBSD.
compat22Directory containing libraries to maintain compatibility with Release 2.2 of FreeBSD.
compat3xDirectory containing libraries to maintain compatibility with Release 3 of FreeBSD.
compat4xDirectory containing libraries to maintain compatibility with Release 4 of FreeBSD.
cryptoInstallation directory: cryptographic software.
dictInstallation directory: dictionaries.
docInstallation directory: documentation.
docbook.cssStyle sheet for documentation.
filename.txtA list of all the files on this CD-ROM.
floppiesA directory containing installation floppy disk images.
gamesInstallation directory: games.
infoInstallation directory: GNU info documents.
kernelThe boot kernel.
manpagesA directory containing the man pages for installation.
packagesA directory containing installable versions of the Ports Collection. See page 168.
portsThe sources for the Ports Collection. See Chapter 9 , The Ports Collection, page 167.
proflibsA directory containing profiled libraries, useful for identifying performance problems when programming.
srcA directory containing the system source files.
toolsA directory containing tools to prepare for installation from another operating system

The .TXT files are also supplied in HTML format with a .HTM suffix.

The contents of the CD-ROM will almost certainly change from one release to another. Read README.TXT for details of the changes.

Live File System CD-ROM

Although the installation CD-ROM contains everything you need to install FreeBSD, the format isn't what you'd like to handle every day. The distribution may include a Live File System CD-ROM, which solves this problem: it contains substantially the same data stored in file system format in much the same way as you would install it on a hard disk. You can access the files directly from this CD-ROM.

CVS Repository CD-ROM

One of the disks may also contain the &CVS Repository.&The repository is the master source tree of all source code, including all update information. We'll look at it in more detail in Chapter 31 , Keeping up to date,page 581.

The Ports Collection CD-ROM

An important part of FreeBSD is the Ports Collection, which comprises many housand popular programs. The Ports Collection automates the process of porting software to FreeBSD. A combination of various programming tools already available in the base FreeBSD installation allows you to simply type make to install a given package. The ports mechanism does the rest, so you need only enough disk space to build the ports you want. We'll look at the Ports Collection in more detail in Chapter 9 . The files are spread over a number of CD-ROMs:

3. Quick installation

Making things easy foryourself FreeBSD on a disk with free space; FreeBSD shared with Microsoft; Configuring XFree86.

In Chapters 4 to 6 we'll go into a lot of detail about how to install the system. Maybe this is too much detail for you. If you're an experienced UNIX user, you should be able to get by with significantly less reading. This chapter presents checklists for some of the more usual kinds of installation. Each refers you to the corresponding detailed descriptions in Chapters 4 through 6.

On the following pages we'll look at the simplest installation, where FreeBSD is the only system on the disk. Starting on page 49 we'll look at sharing the disk with Microsoft, and on page 50 we'll look at how to install XFree86. You may find it convenient to photo copy these pages and to mark them up as you go along.

Making things easy for yourself

It is probably easier to install FreeBSD than any other PC operating system, including Microsoft products. Well, most of the time, any way. Some people spend days trying to install FreeBSD, and finally give up. That happens with Microsoft's products as well, but unfortunately it happens more often with FreeBSD.

Now you're probably saying, "That doesn't make sense. First you say it's easier to install, then you say it's more likely to fail. What's the real story?"

As you might expect, the real story is quite involved. In Chapter 2, Before you install, I went into some of the background. Before you start, let's look at what you can do to make the installation as easy as possible:

FreeBSD on a disk with free space

This procedure applies if you can install FreeBSD without first having to make space on disk. Perform the following steps:

FreeBSD shared with Microsoft

If you have a disk with Microsoft installed on only part of the disk, and you don't want to change the partition layout, you can proceed as in the instructions above. This is pretty unusual, though: normally Microsoft takes the whole disk, and it's difficult to persuade it otherwise. To install FreeBSD on a disk that currently contains a single Microsoft partition taking up the entire disk, go through the following steps:

Configuring XFree86

4. Shared OS installation

Separate disks; Sharing adisk; Sharing with Linux or another BSD; Repartitioning with FIPS.

In many cases, you won't want to install FreeBSD on the system by itself: you may need to use other operating systems as well. In this chapter, well look at what you need to do to prepare for such an installation. If you're only running FreeBSD on the machine, you don't need to read this chapter, and you can move onto Chapter 5, Installing FreeBSD.

Before you start the installation, read this chapter carefully. It's easy to make a mistake, and one of the most frequent results of mistakes is the total loss of all data on the hard disk.

Currently, only the ia32 (Intel) port of FreeBSD is capable of sharing with other operating systems. We'll concentrate on how to share your system with Microsoft, because that's both the most difficult and the most common, but most of this chapter applies to other operating systems as well. You may want to refer to the discussion of Microsoft and FreeBSD disk layouts on page 34.

Separate disks

The first question is: do you need to share a disk between FreeBSD and the other operating system? It's much easier if you don't have to. In this section, we'll look at what you need to do.

Many operating systems will only boot from the first disk identified by the BIOS, usually called the C: disk in deference to Microsoft. FreeBSD doesn't have this problem, so the easiest thing is to install FreeBSD on the entire second disk. BIOS restrictions usually make it difficult to boot from any but the first two disks.

In this case, you don't really need to do anything special, although it's always a good idea to back up your data first. Install FreeBSD on the second disk, and choose the Boot Manager option in the partition editor (page 64). This will then give you the choice of booting from the first or second disk. Note that you should not change the order of disks after such an installation; if you do, the system will not be able to find its file systems after boot.

Sharing a disk

If you intend to share a disk between FreeBSD and another operating system, the first question is: is there enough space on the disk for FreeBSD? How much you need depends on what you want to do with FreeBSD, of course, but for the sake of example we'll take 120 MB as an absolute minimum. In the following section, we'll consider what to do if you need to change your partitions. If you already have enough space for a FreeBSD partition (for example, if you have just installed Microsoft specifically for sharing with FreeBSD, and thus have not filled up the disk), continue reading on page 66.

Sharing with Linux or another BSD

Sharing with other free operating systems is relatively simple. You still need to have space for FreeBSD, of course, and unlike Microsoft, there are no tools for shrinking Linux or BSD file systems: you'll have to remove them or recreate them. You can find some information about sharing with Linux in the mini-Howto at http://wwwlinux.org/docs/ldp/howto/mini/Linux+FreeBSD.html.

NetBSD and Open BSD file systems and slices are very similar to their FreeBSD counterparts. They're not identical, however, and you may find that one of the systems recognizes the partition of another system and complains about it because it's not quite right. For example, NetBSD has a d partition that can go outside the boundary of the slice. FreeBSD does not allow this, so you get a harmless error message.

Repartitioning with FIPS

Typically, if you've been running Microsoft on your machine, it will occupy the entire disk. If you need all this space, of course, there's no way to install another operating system as well. Frequently, though, you'll find that you have enough free space in the partition. Unfortunately, thats not where you want it: you want the space in a new partition. There are a number of ways to do so:

In the rest of the section, we'll look at how to shrink a partition with FIPS. If you do it with PartitionMagic, the details are different, but the principles are the same. In particular:

Before repartitioning your disk, make a backup. You can shoot yourself in the foot with this method, and the result will almost invariably be loss of data.

If you've been running Microsoft on your system for any length of time, the data in the partition will be spread all around the partition. If you just truncate the partition, you'll lose a lot of data, so you first need to move all the data to the beginning of the partition. Do this with the Microsoft defragmentation utility. Before proceeding, consider a few gotchas:

For further information, read the FIPS documentation in /cdrom/tools/fips.doc. In particular, note these limitations:

Repartitioningan example

In this section, we'll go through the mechanics of repartitioning a disk. We'll start with a disk containing a single, complete Microsoft system.

First, run the Microsoft error check utility on the partition you want to split. Make sure no "dead" clusters remain on the disk.

Next, prepare a bootable floppy. When you start FIPS, you will be given the opportunity to write backup copies of your root and boot sector to a file on drive A:. These will be called ROOTBOOT.00x, where x represents a digit from 0 to 9. If anything goes wrong while using FIPS, you can restore the original configuration by booting from the floppy and running RESTORRB.

If you use FIPS more than once (this is normally not necessary, but it might happen), your floppy will contain more than one ROOTBOOT file. RESTORRB lets you choose which configuration file to restore. The file RESTORRB.000 contains your original configuration. Try not to confuse the versions.

Before starting FIPS you must defragment your disk to ensure that the space to be used for the new partition is free. If you're using programs like IMAGE or MIRROR, note that they store a hidden system file with a pointer to your mirror files in the last sector of the hard disk. You must delete this file before using FIPS. It will be recreated the next time you run MIRROR. To delete it, in the root directory enter:

C\:&gt; attrib -r -s -h image.idx  for IMAGE
C\:&gt; attrib -r -s -h mirorsav.fil  for MIRROR

Then delete the file.

If FIPS does not offer as much disk space for creation of the new partition as you expect, this may mean that:

If you are running early versions of MS-DOS (before 5.0), or another operating system, such as OS/2, or you are using programs like Stacker, SuperStor, or Doublespace, read the FIPS documentation for other possible problems.

Running FIPS

After defragmenting your Microsoft partition, you can run FIPS:

C:\&gt; D:             changetoCD-ROM
D:\&gt; cd \tools      Make sure you're in the tools directory
D:\tools\&gt; fips     and start the FIPS program
... a lot of copyright information omitted 
Press any key              do what the computer says
Which Drive (1=0x80/2=0x81)?

The message Which Drive may seem confusing. It refers to BIOS internal numbering. Don't worry about it: if you want to partition the first physical drive inthe system, (C:), enter 1, otherwise enter 2. Like the BIOS, FIPS handles only two hard disks.

If you start FIPS under Windows, it will complain and tell you to boot from a floppy disk. It won't stop you from continuing, but it is a Bad Idea to do so.

Next, FIPS reads the root sector of the hard disk and displays the partition table:

.
StartEnd
Part.bootableHeadCyl.SectorSystemHeadCyl.SectorStart SectorNumber of SectorsMB
1yes1010ch239204763634008305719571
2no00000h000000
3no00000h000000
4no00000h000000

This shows that only the first partition is occupied, that it is bootable, and that it occupies the whole disk (19571 MB, from Cylinder 0, Head 1, Sector 1 to Cylinder 2047, Head 238, Sector 63). It also claims that this makes 40083057 sectors. It doesn't: the cylinder number has been truncated, and FIPS complains about a partition table inconsistency, which it fixes. After this, we have:

.
StartEnd
Part.bootableHeadCyl.SectorSystemHeadCyl.SectorStart SectorNumber of SectorsMB
1yes1010ch239265063634008305719571
2no00000h000000
3no00000h000000
4no00000h000000

Don't worry about the "bootable" flag herewe'll deal with that in the FreeBSD installation. First, FIPS does some error checking and then reads and displays the boot sector of the partition:

Checking boot sector ... OK
Press any Key                  do what it says
Bytes per sector: 512
Sectors per cluster: 32
Reserved sectors: 32
Number of FATs: 2
Number of rootdirectory entries: 0
Number of sectors (short): 0
Media descriptor byte: f8h
Sectors per FAT: 9784
Sectors per track: 63
Drive heads: 240
Hidden sectors: 63
Number of sectors (long): 40083057
Physical drive number: 80h
Signature: 29h

After further checking, FIPS asks you if you want to make a backup floppy. Enter your formatted floppy in drive A: and makethe backup. Next, you see:

Enter start cylinder for new partition (35 - 2650):
Use the cursor keys to choose the cylinder, <enter> to continue
Old partition  Cylinder  New Partition
  258.4 MB        35       19313.4 MB

Use the Cursor Left and Cursor Right keys to adjust the cylinder number at which the new partition starts. You can also use the keys Cursor Up and Cursor Down to change in steps of ten cylinders. FIPS updates the bottom line of the display to show the new values selected. Initially, FIPS chooses the smallest possible Microsoft partition, so initially you can only increase the size of the old partition (with the Cursor Right key). When you're happy with the sizes, press Enter to move on to the next step.

Be very sure you're happy before you continue. If you make the first partition too small, there is no way to make it larger again. On the other hand, if you make it too large, you can split it again and then use fdisk or MS-DOS FDISK to remove the superfluous partitions.

In this example, we choose equal-sized partitions:



. 4.1.

To ensure that the partition is recognized, reboot immediately. Make sure to disable all programs that write to your disk in CONFIG.SYS and AUTOEXEC.BAT before rebooting. It might be easier to to rename the files or to boot from floppy. Be particularly careful to disable programs like MIRROR and IMAGE, which might get confused if the partitioning is not to their liking. After rebooting, use CHKDSK or Norton Disk Doctor to make sure the first partition is OK. If you don't find any errors, you may now reboot with your normal CONFIG.SYS and AUTOEXEC.BAT. Start some programs and make sure you can still read your data.

After that, you have two valid Microsoft partitions on your disk. We'll look at what to do with them in the next chapter. The specific differences from a dedicated install are on page 66, but you'll need to start from the beginning of the chapter to do the install.

5. Installing FreeBSD

Booting from CD-ROM; Installing on the Intel i386 architecture Booting to sysinstall Setting installation options; Partitioning the disk Defining file systems Selecting distributions Selecting the installation medium Performing the installation; Installing on an Alpha system; Upgrading an old version of FreeBSD How to uninstall FreeBSD If things go wrong Alternative installation methods.

In the previous chapters, we've looked at preparing to install FreeBSD. In this chapter, we'll finally do it. If you run into trouble, I'll refer you back to the page of Chapter 2 which discusses this topic. If you want to install FreeBSD on the same disk as Microsoft or another operating system, you should have already read Chapter 4 , Shared OS installation.

The following discussion relates primarily to installation on the i386 architecture. See page 78 for differences when installing on the AXP ("Alpha") processor.

Installing on the Intel i386 architecture

To install FreeBSD you need the software in a form that the installation software understands. You may also need a boot diskette. Nowadays you will almost invariably install from CD-ROM, so we'll assume that medium. On page 85, we'll look at some alternatives: installation from floppy disk or via the network.

The first step in installing FreeBSD is to start a minimal version of the operating system. The simplest way is to boot directly from the installation CD-ROM. If your system doesn't support this kind of boot, boot from floppy. See page 85 for more details.

The description in this chapter is based on a real-life installation on a real machine. When you install FreeBSD on your machine, a number of things will be different, depending on the hardware you're running, the way you're installing the software and the release of FreeBSD you're installing. Nevertheless, you should be able to recognize what is going on.

Booting from CD-ROM is mainly a matter of setting up your system BIOS and possibly your SCSI BIOS. Typically, you perform one of the following procedures:

These settings are probably not what you want to use for normal operation. If you leave the settings like this, and there is a bootable CD-ROM in your CD-ROM drive, it always boots from that CD-ROM rather than from the hard disk. After installation, change the parameters back again to boot from hard disk before CD-ROM. See your system documentation for further details.

Booting to sysinstall

The boot process itself is very similar to the normal boot process described on page 528. After it completes, though, you are put into the sysinstall main menu.

 Main installation menu


. 5.1. Main installation menu

Figure 5-1 shows the main sysinstall menu. sysinstall includes online help at all stages. Simply press F1 and you will get appropriate help. Also, if you haven't been here before, the Doc menu gives you a large part of the appropriate information from the handbook.

Kinds of installation

To get started, select one of Standard, Express or Custom.The names imply that the Standard installation is the best way to go, the Express installation is for people in a hurry, and Custom installation is for when you want to specify exactly what is to be done.

In fact, the names are somewhat misleading. There isn't really that much difference between the three forms of installation. They all perform the same steps:

We looked at disk partitions and file systems on page 34. We'll look at the other points when we get to them. So what's the difference between the kinds of installation?

The big problem with Standard and Express installations is that they don't let you back up: if you pass a specific step and discover you want to change something, you have to abort the installation and start again. With the Custom installation, you can simply go back and change it. As a result, I recommend the Custom installation. In the following discussion, you won't see too much difference: the menus are the same for all three installation forms.

 Custom Installation options


. 5.2. Custom Installation options

Setting installation options

The first item on the menu is to set installation options. There's probably not too much you'll want to change. About the only thing of interest might be the editor ec, which is a compromise between a simple editor for beginners and more complicated editors like vi. If you're planning to edit anything during the installation, for example the file /etc/exports , which we'll look at on page 566, you may prefer to set an editor with which you are familiar. Select the fields by moving the cursor to the line and pressing the space bar.

Installation options


. 5.3. Installation options

Partitioning the disk

The first installation step is to set up space for FreeBSD on the disk. We looked at the technical background in Chapter 2, on page 39. In this section only, well use the term partition to refer to a slice or BIOS partition, because that's the usual terminology.

Even if your disk is correctly partitioned, select the Partition menu: the installation routines need to enter this screen in order to read the partition information from the disk. If you like what you see, you can leave again immediately with q (quit), but you must first enter this menu. If you have more than one disk connected to your machine, you will next be asked to choose the drives that you want to use for FreeBSD.

#isDisk selection menu

This screen shows entries for each drive that sysinstall has detected; in this example, the system has one ATA (IDE) drive, /dev/ad0, and one SCSI drive, Dao. You only get this screen if you have at least two drives connected to your machine; otherwise sysinstall automatically goes to the next screen.

If you intend to use more than one disk for FreeBSD, you have the choice of setting up all disks now, or setting the others up after the system is up and running. We'll look at the latter option in Chapter 11 , on page 199.

To select the disk on which you want to install FreeBSD, move the cursor to the appropriate line and press the space bar. The screen you get will probably look like Figure 5-5. Table 5-1 explains the meanings of the columns in this display. The first partition contains the Master Boot Record, which is exactly one sector long, and the bootstrap, which can be up to 15 sectors long. The partitioning tools use the complete first track: in this case, the geometry information from BIOS says that it has 63 sectors per track.

In this case, the Microsoft file system uses up the whole disk except for the last track, 1008 sectors (504 kB) at the end of the disk. Clearly there's not much left to share. We have the option of removing the Microsoft partition, which we'll look at here, or we can shorten it with FIPS.We looked at FIPS in Chapter 4, page 52, and we'll look at what to do with the resultant layout on page 66.

 Partition editor menu


. 5.4. Partition editor menu

Don't forget that if you remove a partition, you lose all the data in it. If the partition contains anything you want to keep, make sure you have a readable backup.

You remove the partition with the d command. After this, your display looks like:



. 5.5.

The next step is to allocate a FreeBSD partition. There are two ways to do this: if you want to have more than one partition on the drive (for example, if you share the disk with another operating system), you use the c (create) command. We'll look at that on page 66. In this case, though, you want to use the entire disk for FreeBSD, so you choose the a option. The resultant display is effectively the same as in Figure 5-5 : the only difference is that the Desc field now shows freebsd instead of fat.

That's all you need to do here: leave /disk by pressing the q key.

Don't use the W (Write Changes) command here. It's intended for use only once the system is up and running.

5.1. fdisk information
ColumnDescription
OffsetThe number of the first sector in the partition
SizeThe length of the partition in sectors
EndThe number of the last sector in the partition
NameWhere present, this is the device name that FreeBSD assigns to the partition. In this example, only the second entry has a name.
PtypeThe partition type. Partition type 6 is the Master Boot Record, which is exactly one track long (note that the header says that this drive has 63 sectors per track). Type 2 is a regular partition.
DescAtextual description of the kind of partition. fat stands for File Allocation Table, a central part of the Microsoft disk space allocation strategy
SubtypeThe partition subtype. This corresponds to the descriptive text
FlagsCan be one or more of the following characters
  • = The partition is correctly aligned.
  • > The partition finishes after cylinder 1024, which used to cause problems for Microsoft.
  • A This is the active (bootable) partition.
  • B The partition employs BAD144 bad-spot handling.
  • C This is a FreeBSD compatibility partition.
  • R This partition contains a root file system

On a PC, the next screen asks what kind of boot selector (in other words, MBR) you want. You don't get this on an Alpha.

Boot selector menu


. 5.6. Boot selector menu

If you plan to have only one operating system on this disk, select Standard .If you are sharing with another operating system, you should choose BootMgr instead. We'll look at this in more detail in the section on booting the system on page 529. Exit by pressing the tab key until the OK tab is highlighted, then press Enter.

5.2. MBR choices
ChoiceDescription
BootMgrInstall the FreeBSD boot manager in the MBR. This will enable you choose which partition to boot every time you start the system.
StandardUse a standard MBR. Youwill be able to boot only from the active partition
NoneDon't change the MBR. This is useful if you already have another boot manager installed. If no MBR is installed, though, you won't be able to boot from this disk.
Shared partition

If you are installing on a disk shared with another operating system, things are a little different. The section continues the example started in Chapter 4 . When you enter the partition editor, you will see something like:

Shared partitions


. 5.7. Shared partitions

This display shows the two Microsoft partitions, adOsl and ad0s2,which is what you see after using FIPS; if you have just installed Microsoft on one partition, the partition ad0s2 will not be present. If it is, you first need to remove it. Be very careful to remove the correct partition. It's always the second of the two partitions, in this case ad0s2.

Remove the partition by moving the highlight to the second partition and pressing d. After this, the display looks like:



. 5.8.

The next step is to allocate a FreeBSD partition with the c command. The menu asks for the size of the partition, and suggests a value of 35899920 sectors, the size of the unused area at the end. You can edit this value if you wish, but in this case it's what you want, so just press ENTER. You get another window asking you for the partition type, and suggesting type 165, the FreeBSD partition table. When you accept that, you get:



. 5.9.

The new partition now has a partition type 8 and subtype 165 (0xa5), which identifies it as a FreeBSD partition.

After this, select a boot method as described on page 66 and exit the menu with the q command. There are two operating systems on the disk, so select the BootMgr option.

Defining file systems

The next step is to tell the installation program what to put in your FreeBSD partition. First, we'll look at the simple case of installing FreeBSD by itself. On page 75 we'll look at what differences there are when installing alongside another operating system on the same disk.

When you select Label, you get the screen shown in Figure 5-8 .

Label editor menu


. 5.10. Label editor menu

What partitions?

In this example, you have 20GB of space to divide up. How should you do it? You don't have to worry about this issue, since sysinstall can do it for you, but we'll see below why this might not be the best choice. In this section we'll consider how UNIX file systems have changed over the years, and we'll look at the issues in file system layout nowadays.

When UNIX was young, disks were tiny. At the time of the third edition of UNIX, in 1972, the root file system was on a Digital RF-11, a fixed head disk with 512 kB. The system was growing, and it was no longer possible to keep the entire system on this disk, so a second file system became essential. It was mounted on a Digital RK03 with 2 MB of storage. To quote from a paper published in the Communications of the ACM in July 1974:

In our installation, for example, the root directory resides on the fixed-head disk, and the large disk drive,which contains user's files, is mounted by the system initialization program...

As time went on, UNIX got bigger, but so did the disks. By the early 80s, disks were large enough to put / and /usr on the same disk, and it would have been possible to merge / and /usr, but they didn't, mainly because of reliability concerns. Since that time, an additional file system, /var, has come into common use for frequently changed data, and just recently sysinstall has been changed to create a /tmp file system by default. This is what sysinstall does if you ask it to partition automatically:

Default file system sizes


. 5.11. Default file system sizes

It's relatively simple to estimate the size of the root file system, and sysinstall's value of 128 MB is reasonable. But what about /var and /tmp? Is 256 MB too much or too little? In fact, both file systems put together would be lost in the 18.7 GB of /usr file system. Why are things still this way? Let's look at the advantages and disadvantages:

In the early days of UNIX, system crashes were relatively common, and the damage they did to the file systems was relatively serious. Times have changed, and nowadays file system damage is relatively seldom, particularly on file systems that have little activity. On the other hand, disk drive shave grown beyond most peoples' wildest expectations. The first edition of this book, only six years ago, showed how to install on a 200 MB drive. The smallest disk drives in current production are 20 GB in size, more than will fit on many tapes.

As a result of these considerations, I have changed my recommendations. In earlier editions of this book, I recommended putting a small root file system and a /usr file system on the first (or only) disk on the system. /var was to be a symbolic link to /usr/var.

This is still a valid layout, but it has a couple of problems:

As a result, I now recommend:

This layout allows for easy backup of the file systems, and it also allows for easy upgrading to a new system version: you just need to replace the root file system. It's not a perfect fit for all applications, though. Ultimately you need to make your own decisions.

How much swap space?

Apart from files, you should also have at least one swap partition on your disk. It's very difficult to predict how much swap space you need. The automatic option gave you 522 MB, slightly more than twice the size of physical memory. Maybe you can get by with 64 MB. Maybe you'll need 2 GB. How do you decide?

It's almost impossible to know in advance what your system will require. Here are some considerations:

A couple of examples might make this clearer:

  1. Some years ago I used to run X, StarOffice, Netscape and a whole lot of other memory-hungry applications on an old 486 with 16 MB. Sure, it was really slow, especially when changing from one application to another, but it worked. There was not much memory, so it used a lot of swap.

    To view the current swap usage, use pstat. Here's a typical view of this machine's swap space:

    $ pstat -s
    Device      1024-blocks    Used   Avail  Capacity  Type
    /dev/da0s1   122880       65148   57668   53%      Interleaved
    
  2. At the time of writing I run much more stuff on an AMD Athlon with 512 MB of memory. It has lots of swap space, but what I see is:
    $  pstat s
    Device      1024-blocks  Used    Avail  Capacity     Type
    /dev/ad0s1b  1048576    14644   1033932    1%     Interleaved
    

It's not so important that the Athlon is using less swap: it's using less than 3% of its memory in swap, whereas the 486 used 4 times its memory. In a previous edition of this book, I had the example of a Pentium with 96 MB of memory, which used 43 MB of swap. Look at it from a different point of view, and it makes more sense: swap makes up for the lack of real memory, so the 486 was using a total of 80 MB of memory, the Pentium was using 140 MB, and the Athlon is using 526 MB. In other words, there is a tendency to be able to say "the more main memory you have, the less swap you need."

If, however, you look at it from the point of view of acceptable performance, you will hear things like "you need at least one-third of your virtual memory in real memory." That makes sense from a performance point of view, assuming all processes are relatively active. And, of course, it's another way of saying "take twice as much swap as real memory."

In summary: be generous in allocating swap space. If you have the choice, use more. If you really can't make up your mind, take 512 MB of swap space or 1 MB more than the maximum memory size you are likely to install.

For the file systems, the column Mount now shows the mount points, and the Newfs column contains the letters UFS1 for UNIX File System, Version 1, and the letter Y, indicating that you need to create a new file system before you can use it. At this point, you have two choices: decide for yourself what you want, or let the disk label editor do it for you. Let's look at both ways:

Creating the file systems

With these considerations in mind, we'll divide up the disk in the following manner:

To create a file system, you press c. You get a prompt window asking for the size of the file system, and offering the entire space. Enter the size of the root file system

Specifying partition size


. 5.12. Specifying partition size

When you press ENTER, you see another prompt asking for the kind of partition. Select A File System:

 Selecting partition type


. 5.13. Selecting partition type

When you press ENTER,you see another prompt asking for the mount point for the file system. Enter / for the root file system, after which the display looks like:

Allocated root file system


. 5.14. Allocated root file system

It's not immediately obvious at this point that soft updates are not enabled for this file system. Press s to enable them, after which the entry in the Newfs column changes from UFS1 to UFSl+S. See page 191 for reasons why you want to use soft updates.

Next, repeat the operation for the swap partition and the /home file system, entering the appropriate values each time. Don't change the value offered for the length of /home: just use all the remaining space. At the end, you have:

Completed partition allocation


. 5.15. Completed partition allocation

You don't need to enable soft updates for /home; that happens automatically. That's all you need to do. Exit the menu by pressing q.

Where you are now

At this point in the installation, you have told sysinstall the overall layout of the disk or disks you intend to use for FreeBSD, and whether or how you intend to share them with other operating systems. The next step is to specify how you want to use the FreeBSD partitions. First, though, we'll consider some alternative scenarios.

Second time through

If you have already started an installation and aborted it for some reason after creating the file systems, things will look a little different when you get to the label editor. I twill find the partitions, but it won't know the name of the mount points, so the text under Mount will be <none>. Under Newfs, you will find an asterisk (*) instead of the text UFS1 Y. The label editor has found the partitions, but it doesn't know where to mount the file systems. Before you can use them, you must tell the label editor the types and mount points of the UFS partitions. To do this:

File systems on shared disks

If you have another operating system on the disk, you'll notice a couple of differences. In particular, the label editor menu of Figure 5-8 (on page 68) will not be empty: instead, you'll see something like this:



. 5.16.

Be careful at this point. The file system shown in the list is the active Microsoft partition, not a FreeBSD file system. The important piece of information here is the fact that we have 17529 MB of free space on the disk. We'll create the file systems in that free space in the same way we saw on page 72.

Selecting distribution

The next step is to decide what to install. Figure 5-14 shows you the menu you get when you enter Distributions. A complete installation of FreeBSD uses about 1 GB of space, so there's little reason to choose anything else. Position the cursor on the line All, as shown, and press the space bar.

Why press the space bar when so far you have been pressing ENTER? Because in this particular menu, ENTER will return you to the upper level menu or simply continue to the media selection menu, depending on the type of installation you're doing. It's one of the strangenesses of sysinstall.

Next, sysinstall asks you if you want to install the Ports Collection. We'll look at the Ports Collection in Chapter 9 . You don't have to install it now, and it takes much more time than you would expect from the amount of space that it takes: the Ports Collection consists of over 150,000 very small files, and copying them to disk can take as long as the rest of the installation put together. On the other hand, it's a lot easier to do now, so if you have the time, you should install them.

Whatever you answer to this question, you are returned to the distribution menu of Figure 5-14. Select Exit, and you're done selecting your distributions.

Earlier versions of sysinstall asked you questions about XFree86 at this point. Nowadays you do that after completing the installation.

Distribution selection menu


. 5.17. Distribution selection menu

Where you are now

Now sysinstall knows the layout of the disk or disks you intend to use for FreeBSD, and what to put on them. Next, you specify where to get the data from.

Selecting the installation medium

The next thing you need to specify is where you will get the data from. Where you go now depends on your installation medium. Figure 5-15 shows the Media menu. If you're installing from anything except an ftp server or NFS, you just need to select your medium and then commit the installation, which we look at on page 77. If you're installing from media other than CD-ROM, see page 85.

At this point, sysinstall knows everything it needs to install the software. It's just waiting for you to tell it to go ahead.

Installation medium menu


. 5.18. Installation medium menu

Performing the installation

So far, every thing you have done has had no effect on the disk drives. If you change your mind, you can just abort the installation, and the data on your disks will be unchanged. That changes completely in the next step, which you call committing the installation. Now is the big moment. You've set up your partitions, decided what you want to install and from where. Now you do it.

If you are installing with the Custom installation, you need to select Commit explicitly. The Standard installation asks you if you want to proceed:

Last Chance!   Are you SURE you want continue the installation?

If you're running this on an existing system, we STRONGLY
encourage you to make proper backups before proceeding.
We take no responsibility for lost disk contents!

When you answer yes, sysinstall does what we've been preparing for:

After the file systems are mounted, and before installing the software, sysinstall starts processes on two other virtual terminals1). On /dev/ttyvl you get log output showing you what's going on behind the scenes. You can switch to it with ALT-F2. Right at the beginning you'll see a whole lot of error messages as sysinstall tries to initialize every device it can think of. Don't worry about them, they're normal. To get back to the install screen, press ALT-F1.

In addition, after sysinstall mounts the root file system, it starts an interactive shell on /dev/ttyv3. You can use it if something goes wrong, or simply to watch what's going on while you're installing. You switch to it with ALT-F4.

After installing all the files, sysinstall asks:

Visit the general configuration menu for a chance to set
any last options?

You really have the choice here. You can answer Yes and continue, or you can reboot: the system is now run able. In all probability, though, you will have additional installation work to do, so it's worth continuing. We'll look at that in the following chapter.

Installing on an Alpha system

Installing FreeBSD on an Alpha (officially Compaq AXP) has a few minor differences due to the hardware itself. In principle, you perform the same steps to install FreeBSD on the Alpha architecture that you perform for the Intel architecture. See page 42 for some differences.

The easiest type of installation is from CD-ROM. If you have a supported CD-ROM drive and a FreeBSD installation CD for Alpha, you can start the installation by building a set of FreeBSD boot floppies from the files fbppies/kern.fp andfbppies/mfsroot.ftp as described for the Intel architecture on page 85. Use the CD-ROM marked "Alpha installation." From the SRM console prompt, insert the kern.flp floppy and type the following command to start the installation:

>>>boot dvaO

Insert the mfsroot.flp floppy when prompted and you will end up at the first screen of the install program. You can then continue as for the Intel architecture on page 59.

To install over the Net, fetch the floppy images from the ftp site, boot as above, then proceed as for the Intel architecture.

Once the install procedure has finished, you will be able to start FreeBSD/Alpha by typing something like this to the SRM prompt:

>>>boot dkcO

This instructs the firmware to boot the specified disk. To find the SRM names of disks in your machine, use the show device command:

>>>show device
dka0.0.0.4.0       DKA0   TOSHIBA CD-ROM XM-57  3476
dkc0.0.0.1009.0    DKC0               RZ1BB-BS  0658
dkc100.1.0.1009.0  DKC100     SEAGATE ST34501W  0015
dva0.0.0.0.1       DVA0
ewa0.0.0.3.0       EWA0      00-00-F8-75-6D-01
pkc0.7.0.1009.0    PKC0          SCSI Bus ID 7  5.27
pqa0.0.0.4.0       PQA0               PCI EIDE

This example comes from a Digital Personal Workstation 433au and shows three disks attached to the machine. The first is a CD-ROM called dka0 and the other two are disks and are called dkc0 and dkc100 respectively.

You can specify which kernel file to load and what boot options to use with the -file and -flags options to boot:

>>>boot -file kernel.old -flags s

To makeFreeBSD/Alpha boot automatically,use these commands:

>>>set boot_osflags a
>>>set bootdef_dev dkcO
>>>set auto_action BOOT

Upgrading an old version of FreeBSD

Paradoxically, upgrading an old version of FreeBSD is more complicated than installing from scratch. The reason is that you almost certainly want to keep your old configuration. There's enough material in this topic to fill a chapter, so that's what I've done: see Chapter 31, for more details on how to upgrade a system.

How to uninstall FreeBSD

What, you want to remove FreeBSD? Why would you want to do that?

Seriously, if you decide you want to completely remove FreeBSD from the system, this is no longer a FreeBSD issue, it's an issue of whatever system you use to replace it. For example, on page 63 we saw how to remove a Microsoft partition and replace it with FreeBSD; no Microsoft software was needed to remove it. In the same way, you don't need any help from FreeBSD if you want to replace it with a different operating system.

If things go wrong

In this section, we'll look at the most common installation problems. Many of these are things that once used to happen and haven't been seen for some time: sysinstall has improved considerably, and modern hardware is much more reliable and easy to configure. You can find additional information on this topic in the section Known Hardware Problems in the file INSTALL.TXT on the first CD-ROM.

Problems with sysinstall

sysinstall is intended to be easy to use, but it is not very tolerant of errors. You may well find that you enter something by mistake and cant get back to where you want to be. In case of doubt, if you haven't yet committed to the install, you can always just reboot.

Problems with CD-ROM installation

If you select to install from CD-ROM, you may get the message:

No CD-ROM device found

This might even happen if you have booted from CD-ROM! The most common reasons for this problem are:

Can't boot

One of the most terrifying things after installing FreeBSD is if you find that the machine just won't boot. This is particularly bad if you have important data on the disk (either another operating system, or data from a previous installation of FreeBSD).

At this point, seasoned hackers tend to shrug their shoulders and point out that you still have the backup you made before you did do the installation. If you tell them you didn't do a backup, they tend to shrug again and move on to something else.

Still, all is probably not lost. The most frequent causes of boot failure are an incorrect boot installation or geometry problems. In addition, it's possible that the system might hang and never complete the boot process. All of these problems are much less common than they used to be, and a lot of the information about how to address them is a few years old, as they haven't been seen since.

Incorrect boot installation

It's possible to forget to install the bootstrap, or even to wipe it the existing bootstrap. That sounds like a big problem, but in fact it's easy enough to recover from. Refer to the description of the boot process on page 529, and boot from floppy disk or CD-ROM. Interrupt the boot process with the space bar. You might see:

BTX loader 1.00   BTX version is 1.01
BIOS drive A: is diskO
BIOS drive C: is diskl
BIOS drive D: is disk1
BIOS 639kB/130048kB available memory

FreeBSD/i386 bootstrap loader, Revision 0.8
(grog@freebie.example.com, Thu Jun 13 13:06:03 CST 2002)
Loading /boot/defaults/loader.conf

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [kernel] in 6 seconds...    press space bar here
ok unload                           unload the current kernel
ok set currdev=disk1s1a             and set the location of the newone
ok load /boot/kernel/kernel         load the kernel
ok boot                             then start it

This boots from the drive /dev/ad0s1a, assuming that you are using IDE drives. The correspondence between the name /dev/ad0s1a and disklsla goes via the information at the top of the example: BTX only knows the BIOS names, so you'd normally be looking for the first partition on drive C. After booting, install the correct bootstrap with bsdlabel -B or boot0cfg, and you should be able to boot from hard disk again.

Geometry problems

Things might continue a bit further: you elect to install booteasy, and when you boot, you get the Boot Manager prompt, but it just prints F? at the boot menu and won't accept any input. In this case, you may have set the hard disk geometry incorrectly in the partition editor when you installed FreeBSD. Go back into the partition editor and specify the correct geometry for your hard disk. You may need to reinstall FreeBSD from the beginning if this happens.

It used to be relatively common that sysinstall couldn't calculate the correct geometry for a disk, and that as a result you could install a system, but it wouldn't boot. Since those days, sysinstall has become a lot smarter, but it's still barely possible that you'll run into this problem.

If you cant figure out the correct geometry for your machine, and even if you don't want to run Microsoft on your machine, try installing a small Microsoft partition at the beginning of the disk and install FreeBSD after that. The install program sees the Microsoft partition and tries to infer the correct geometry from it, which usually works. After the partition editor has accepted the geometry, you can remove the Microsoft partition again. If you are sharing your machine with Microsoft, make sure that the Microsoft partition is before the FreeBSD partition.

Alternatively, if you don't want to share your disk with any other operating system, select the option to use the entire disk (a in the partition editor). You're less likely to have problems with this option.

System hangs during boot

A number of problems may lead to the system hanging during the boot process. All the known problems have been eliminated, but there's always the chance that something new will crop up. In general, the problems are related to hardware probes, and the most important indication is the point at which the boot failed. It's worth repeating the boot with the verbose fag: again, refer to the description of the boot process on page 529. Interrupt the boot process with the space bar and enter:

Hit [Enter] to boot immediately,  or any other key for command prompt.
Booting [kernel] in 6 seconds...     press space bar here
ok set boot_verbose                  set a verbose boot
ok boot                              then continue

This fag gives you additional information that might help diagnose the problem. See Chapter 29 for more details of what the output means.

If you're using ISA cards, you may need to reconfigure the card to match the kernel, or change the file /boot/device.hints to match the card settings. See the example on page 609. Older versions of FreeBSD used to have a program called UserConfig to perform this function, but it is no longer supported.

System boots, but doesn't run correctly

If you get the system installed to the point where you can start it, but it doesn't run quite the way you want, don't reinstall. In most cases, reinstallation won't help. Instead, try to find the cause of the problemwith the aid of the FreeBSD-questions mailing list if necessaryand fix the problem.

Root file system fills up

You might find that the installation completes successfully, and you get your system up and running, but almost before you know it, the root file system fills up. This is relatively unlikely if you follow my recommendation to have one file system for /, /usr and /var, but if you follow the default recommendations, it's a possibility. It could be, of course, that you just haven't made it big enoughFreeBSD root file systems have got bigger over the years. In the first edition of this book I recommended 32 MB "to be on the safe side." Nowadays the default is 128 MB.

On the other hand, maybe you already have an128 MB root file system, and it still fills up. In this case, check where you have put your /tmp and /var file systems. There's a good chance that they're on the root file system, and that's why it's filling up.

Panic

Sometimes the system gets into so much trouble that it can't continue. It should notice this situation and stop more or less gracefully. You might see a message like:

panic: free vnode isn't

Syncing disks 14 13 9 5 5 5 5 5 5 5 giving up

dumping to dev 20001 offset 0
dump 16 32 48 64 80 96 112 128 succeeded
Automatic reboot in 15 seconds - press a key on the console to abort
Reboooting...

Just because the system has panicked doesn't mean that you should panic too. It's a sorry fact of life that software contains bugs. Many commercial systems just crash when they hit a bug, and you never know why, or they print a message like General protection fault, which doesn't tell you very much either. When a UNIX system panics, it usually gives you more detailed information-in this example, the reason is free v node isn't. You may not be any the wiser for a message like this (it tells you that the file system handling has got confused about the current state of storage on a disk), but other people might. In particular, if you do get a panic and you ask for help on FreeBSD-questions, please don't just say "My system panicked, what do I do?" The first answerif you get one will be "What was the panic string??" The second will be "Where's the dump?"

After panicking, the system tries to write file system buffers back to disk so that they don't get lost. This is not always possible, as we see on the second line of this example. It started off with 14 buffers to write, but it only managed to write 9 of them, possibly because it was confused about the state of the disk. This can mean that you will have difficulties after rebooting, but it might also mean that the system was wrong in its assumptions about the number of buffers needed to be written.

In addition to telling you the cause of the panic, FreeBSD will optionally copy the current contents of memory to the swap file for post-mortem analysis. This is called dumping the system, and is shown on the next two lines. To enable dumping, you need to specify where the dump should be written. In /etc/defaults/rc.conf, you will find:

dumpdev="NO"         # Device name to crashdump to (if enabled)

To enable dumping, put something like this in /boot/loader.conf:

dumpdev="/dev/ad0s1b"

This enables the dumps to be taken even if a panic occurs before the system reads the /etc/rc.conf file. Make sure that the name of the dumpdev corresponds to a swap partition with at least as much space as your total memory. You can use pstat to check this:

# pstat s
Device      1024-blocks    Used   Avail  Capacity  Type
/dev/ad0s1b       51200   50108    1028     98%    interleaved
/dev/da0s1b       66036   51356   14616     78%    interleaved
/dev/da2s1b      204800   51220  153516     25%    interleaved
Total            321844  152684  169160     47%

As long as this machine doesn't have more than about 192 MB of memory, it will be possible to take a dump on /dev/da2s1b.

In addition, ensure that you have a directory called /var/crash. After rebooting, the system first checks the integrity of the file systems, then it checks for the presence of a dump. If it finds one, it copies the dump and the current kernel to /var/crash.

It's always worth enabling dumping, assuming your swap space is at least as large as your memory. You can analyze the dumps with gdbsee page 623 for more details.

To get the best results from a dump analysis, you need a debug kernel.This kernel is identical to a normal kernel, but it includes a lot of information that can be used for dump analysis. See page 614 for details of how to build a debug kernel. You never know when you might run into a problem, so I highly recommend that you use a debug kernel at all times. It doesn't have any effect on the performance of the system.

Fixing a broken installation

A really massive crash may damage your system to such an extent that you need to reinstall the whole system. For example, if you overwrite your hard disk from start to finish, you don't have any other choice. In many cases, though, the damage is repairable. Sometimes, though, you can't start the system to fix the problems. In this case, you have two possibilities:

In either case, the hard disks aren't mounted; you might want to do repair work on them before any other access.

Use this option only if you have a good understanding of the system installation process. Depending on the damage, you may or may not be successful. If you have a recent backup of your system, it might be faster to perform a complete installation than to try to fix what's left, and after a reinstallation you can be more confident that the system is correctly installed.

Alternative installation methods

The description at the beginning of this chapter applied to the most common installation method, from CD-ROM. In the following sections we'll look at the relatively minor differences needed to install from other media. The choices you have are, in order of decreasing attractiveness:

Preparing boot floppies

If your machine is no longer the youngest, you may be able to read the CD-ROM drive, but not boot from it. In this case, you'll need to boot from floppy. If you are using 1.44 MB floppies, you will need two or three of them, the Kernel Disk and the MFS Root Disk and possibly the DriversDisk to boot the installation programs. If you are using 2.88 MB floppies or a LS-120 disk, you can copy the single Boot Disk, which is 2.88 MB long, instead of the kernel and MFS root disks. The images of these floppies are on the CD-ROM distribution in the files floppies/kern.fp, floppies/mfsroot.fp, floppies/driv-ers.flp and foppies/boot.fp respectively. If you have your CD-ROM mounted on a Microsoft system, they may be called FLOPPIESKERN.FLP, FLOPPIESMFS-ROOT.FLP, FLOPPIES\DRIVERS.FLP and FLOPPIES\BOOT.FLP respectively. The bootstrap does: not recover bad blocks, so the floppy must be 100% readable.

The way you get the boot disk image onto a real floppy depends on the operating system you use. If you are using any flavour of UNIX, just perform something like:

# dd if=/cdrom/floppies/k:erri.flp of=/dev/fd0 bs=36b
change the floppy
# dd if=/cdrom/floppies/mfsroot.flp of=/dev/fd0 bs=36b
change the floppy
# dd if=/cdrom/floppies/drivers.flp of=/dev/fd0 bs=36b

This assumes that your software is on CD-ROM, and that it is mounted on the directory /cdrom. It also assumes that your floppy drive is called /devfd 0. This is the FreeBSD name as of Release 5.0, and it's also the name that Linux uses. Older FreeBSD and other BSD systems refer to it as /dev/fd0c.

The dd implementation of some versions of UNIX, particularly older System V variants, may complain about the option bs=36b. If this happens, just leave it out. It might take up to 10 minutes to write the floppy, but it will work, and it will make you appreciate FreeBSD all the more.

If you have to create the boot floppy from Microsoft, use the program FDIMAGE.EXE, which is in the tools directory of the first CD-ROM.

Booting from floppy

In almost all cases where you don't boot from CD-ROM, you'll boot from floppy, no matter what medium you are installing from. If you are installing from CD-ROM, put the CD-ROM in the drive before booting. The installation may fail if you boot before inserting the CD-ROM.

Boot the system in the normal manner from the first floppy (the one containing the kern.flp image). After loading the kernel, the system will print the message:

Please insert MFS root floppy and press enter

After you replace the floppy and press enter, the boot procedure carries on as before.

If you're using the 2.88 MB image on a 2.88 MB floppy or an LS-120 drive, you have every thing you need on the one disk, so you don't get the prompt to change the disk. Depending on your hardware, you may later get a prompt to install additional drivers from the driver floppy.

Installing via ftp

The fun way to install FreeBSD is via the Internet, but it's not always the best choice. There's a lot of data to transfer, and unless you have a really high-speed, non-overloaded connection to the server, it could take forever. On the other hand, of course, if you have the software on another machine on the same LAN, and the system on which you want to install FreeBSD doesn't have a CD-ROM drive, these conditions are fulfilled, and this could be for you. Before you decide, though, read about the alternative of NFS installation below: if you don't have an ftp server with the files already installed, it's a ot easier to set up an NFS installation.

There are two ftp installation modes you can use:

Whichever mode of installation and whichever remote machine you choose, you need to have access to the remote machine. The easiest and most common way to ensure access is to use anonymous ftp. If you're installing from another FreeBSD machine, read how to install anonymous ftp on page 450. This information is also generally correct for other UNIX systems.

Setting up the ftp server

Put the FreeBSD distribution in the public ftp directory of the ftp server. On BSD systems, this will be the home directory of user ftp, which in FreeBSD defaults to /var/spool/ftp The name of the directory is the name of the release, which in this example we'll assume to be 5.0-RELEASE. You can put this directory in a subdirectory of /var/spool/ftp, for example /var/spool/ftp/FreeBSD/5.0-RELEASE, but the only optional part in this example is the parent directory FreeBSD.

This directory has a slightly different structure from the CD-ROM distribution. To set it up, assuming you have your distribution CD-ROM mounted on /cdrom, and that you are installing in the directory /var/spool/ftp/FreeBSD/5.0-RELEASE, perform the following steps:

# cd /var/spool/ftp/FreeBSD/5.0-RELEASE
# mkdir floppies
# cd floppies
# cp /cdrom/floppies/* .    don't omit the . at the end
# cd /cdrom                 the distribution directory on CD-ROM
# tar cf - . | (cd /var/spool/ftp/FreeBSD/5.0-EELEASE; tar xvf -)

This copies all the directories of /cdrom into /var/spool/ftp/FreeBSD/5.0-RELEASE. For a minimal installation, you need only the directory base. To just install base rather than all of the distribution, change the last line of the example above to:

# mkdir base
# cp /cdrom/base/* base
Installing via ftp

On page 77 we saw the media select menu. Figure 5-16 shows the menu you get when you select FTP or FTP Passive.To see the remainder of the sites, use the PageDown key. Let's assume you want to install from presto, a system on the local network. presto isn't on this list, of course, so you select URL. Another menu appears, asking for an ftp pathname in the URL form ftp://hostname/pathname. hostname is the name of the system, in this case presto.example.org, and pathname is the path relative to the anonymous ftp directory, which on FreeBSD systems is usually /var/spool/ftp.The install program knows its version number, and it attaches it to the name you supply.

You can change the version number from the options menu, for example to install a snapshot of a newer release of FreeBSD.

In this case, we're installing Release 5.0 of FreeBSD, and it's in the directory /var/spool/ftp/pub/FreeBSD/5.0-RELEASE. sysinstall knows the 5.0-RELEASE, so you enter only ftp://presto.example.org/pub/FreeBSD. The next menu asks you to configure your network. This is the same menu that you would normally fill out at the end of the installationsee page 98 for details.

This information is used to set up the machine after installation, so it pays to fill out this information correctly. After entering this information, continue with Commit (on page 77).

Selecting ftp server


. 5.19. Selecting ftp server

Installing via NFS

If you're installing from a CD-ROM drive on another system in the local network, you might find an installation via ftp too complicated for your liking. Installation is a lot easier if the other system supports NFS. Before you start, make sure you have the CD-ROM mounted on the remote machine, and that the remote machine is exporting the file system (in System V terminology, exporting is called sharing). When prompted for the name of the directory, specify the name of the directory on which the CD-ROM is mounted. For example, if the CD-ROM is mounted on directory /cdrom on the system presto.example.org, enter presto.example.org:/cdrom. That's all there is to it!

Older versions of FreeBSD stored the distribution on a subdirectory dists. Newer versions store it in the root directory of the CD-ROM.

Next, you give this information to sysinstall, as shown in Figure 5-17 . After entering this information, sysinstall asks you to configure an interface. This is the same procedure that you would otherwise do after installationsee page 98. After performing this configuration, you continue with Commit (on page 77).

Installing from a Microsoft partition

On the Intel architecture you can also install from a primary Microsoft partition on the first disk. To prepare for installation from a Microsoft partition, copy the files from the distribution into a directory called C:\FREEBSD. For example, to do a minimal installation of FreeBSD from Microsoft using files copied from a CD-ROM, copy the directories floppies and base to the Microsoft directories C:\FREEBSD\FLOPPIES and C:\FREEBSD\BIN respectively. You need the directory FLOPPIES because that's where sysinstall looks for the boot.flp, the first image in every installation.

Specifying NFS file system


. 5.20. Specifying NFS file system

The only required directory is base. You can include as many other directories as you want, but be sure to maintain the directory structure. In other words, if you also wanted to install XF86336 and manpages, you would copy them to C:\FREEBSD\XF86336 and C:\FREEBSD\MANPAGES.

Creating floppies for a floppy installation

Installation from floppy disk is definitely the worst choice you have. You will need nearly 50 floppies for the minimum installation, and about 250 for the complete installation. The chance of one of them being bad is high. Most problems on a floppy install can be traced to bad media, or differences in alignment between the media and the drive in which they are used, so:

Before starting, format all floppies in the drive you intend to use, even if they are preformatted.

The first two floppies you'll need are the Kernel floppy and the MFS Root floppy, which were described earlier.

In addition, you need at minimum as many floppies as it takes to hold all files in the base directory, which contains the binary distribution. Read the file LAYOUT.TXT paying special attention to the "Distribution format" section, which describes which files you need.

If you're creating the floppies on a FreeBSD machine, you can put ufs file systems on the floppies instead:

# fdformat -f 1440 fd0.1440
# bsdlabel -w fd0.1440 floppy3
# newfs -t 2 -u 18 -l 1 -i 65536 /dev/fd0

Next, copy the files to the floppies. The distribution files are split into chunks that will fit exactly on a conventional 1.44MB floppy. Copy one file to each floppy. Make very sure to put the file base.inf on the first floppy; it is needed to find out how many floppies to read.

The installation itself is straightforward enough: follow the instructions starting on page 63, select Floppy in the installation medium menu on page 76, then follow the prompts.

6. Post-installation configuration

Installing additional software; Adding users; Time zone; Network services; Startup preferences; Configuring the mouse; Configuring X; Rebooting the new system.

In the last chapter we looked at the installation of the basic system, up to the point where it could be rebooted. It's barely possible that this could be enough. Almost certainly, though, you'll need to perform a number of further configuration steps before the system is useful. In this chapter we roughly follow the final configuration menu, but there are a few exceptions. The most important things to do are:

In this chapter, well concentrate on getting the system up and running as quickly as possible. Later on in the book we'll go into more detail about these topics.

At the end of the previous chapter, we had a menu asking whether we wanted to visit the "last options" menu. If you answer YES, you get the configuration menu shown in Figure 6-1 . If you have rebooted the machine, log in as root and start sysinstall. Then select Configure, which gets you into the same menu.

Configuration menu


. 6.1. Configuration menu

As the markers under the word Networking indicate, this menu is larger than the window in which it is displayed. We'll look at some of the additional entries below. Only some of these entries are of interest in a normal install; we'll ignore the rest.

There may be some reasons to deviate from the sequence in this chapter. For example, if your CD-ROM is mounted on a different system, you may need to set up networking before installing additional software.

Installing additional software

The first item of interest is Packages. These are some of the ports in the Ports Collection, which we'll look at in more detail in Chapter 9 .

The Ports Collection contains a large quantity of software that you may want to install. In fact, there's so much that just making up your mind what to install can be a complicated process: there are over 8,000 ports in the collection. Which ones are worth using? I recommend the following list:

Why do l recommend these particular ports? Simple: because I like them, and I use most of them myself. That doesn't mean they're the only choice, though. Others prefer the Gnome window manager to kde, or the pine or elm MUAs to mutt, or the vim editor to Emacs. This is the stuff of holy wars. See http://catb.org/~esr/jargon/html/Ufholy-wars.htmlfor more details.

Instant workstation

The ports mentioned in the previous section are included in the misc/instant-workstation port, which installs typical software and configurations for a workstation and allows you to be productive right away. At a later point you may find that you prefer other software, in which case you can install it.

It's possible that the CD set you get will not include instant-workstation. That's not such a problem. Due to space restrictions, some CD distributions include instant-workstation-lite instead. If that's not there either, just install the individual ports from this list. You can also do this if you don't like the list of ports.

Changing the default shell for root

After installation, you may want to change the default shell for existing users to bash. If you have installed instant-workstation, you should copy the file /usr/lo-cal/share/dot.bashrc to root's home directory and call it .bashrc and .bash_profile. First, start

presto# cp /usr/local/share/dot.bashrc .bashrc
presto# ln .bashrc .bash_profile
presto# bash
=== root@presto (/dev/ttyp2) ~ 1 -&gt; chsh

The last command starts an editor with the following content:

#Changing user database information for root.
Login: root
Password:
Uid [#]: 0
Gid [# or name]: 0
Change [month day year]:
Expire [month day year]:
Class:
Home directory: /root
Shell: /bin/csh
Full Name: Charlie &
Office Location:
Office Phone:
Home Phone:
Other information:

Change the Shell line to:

Shell: /usr/local/bin/bash

Note that the bash shell is in the directory /usr/local/bin; this is because it is not part of the base system. The standard shells are in the directory /bin.

Adding users

A freshly installed FreeBSD system has a number of users, nearly all for system components. The only login user is root, and you shouldn't log in as root. Instead you should add at least one account for yourself. If you're transferring a master.passwd file from another system, you don't need to do anything now. Otherwise select this item and then the menu item User, and fill out the resulting menu like this:

Adding a user


. 6.2. Adding a user

You should not need to enter the fields UID and Home directory: sysinstall does this for you. It's important to ensure that you are in group wheel so that you can use the su command to become root, and you need to be in group operator to use the shutdown command.

Don't bother to add more users at this stage; you can do it later. Well look at user management in Chapter 8 , on page 112.

Setting the root password

Next, select Root Password. Well talk about passwords more on page 144. Select this item to set the password in the normal manner.

Time zone

Next, select the entry time zone. The first entry asks you if the machine CMOS clock (i.e. the hardware clock) is set to UTC (sometimes incorrectly called GMT, which is a British time zone). If you plan to run only FreeBSD or other UNIX-like operating systems on this machine, you should set the clock to UTC. If you intend to run other software that doesn't understand time zones, such as many Microsoft systems, you have to set the time to local time, which can cause problems with daylight savings time.

Time zone select menu: USA


. 6.3. Time zone select menu: USA

The next menu asks you to select a "region," which roughly corresponds with a continent. Assuming you are living in Austin, TX in the United States of America, you would select America -- North and South and then (after scrolling down) United States of America. The next menu then looks like this: Select Central Time and select Yes when the system asks you whether the abbreviation CST sounds reasonable.

This particular step is relatively cumbersome. You may find it easier to look in the directory /usr/share/zoneinfo after installation. There you find:

# cd /usr/share/zoneinfo/
# ls
Africa       Australia   Etc       MET       WET
America      CET         Europe    MST       posixrules
Antarctica   CST6CDT     Factory   MST7MDT   zone tab
Arctic       EET         GMT       PST8EDT
Asia         EST         HST       Pacific 
Atlantic     EST5EDT     Indian    SystemV

If you want to set the time zone to, say, Singapore, you could enter:

# cd Asia/
# ls
Aden       Chungking  Jerusalem     Novosibirsk  Tehran
Almaty     Colombo    Kabul         Omsk         Thimbu
Amman      Dacca      Kamchatka     Phnom_Penh   Tokyo
Anadyr     Damascus   Karachi       Pyongyang    Ujung_Pandang
Aqtau      Dili       Kashgar       Qatar        Ulaanbaatar
Aqtobe     Dubai      Katmandu      Rangoon      Ulan -Bator
Ashkhabad  Dushanbe   Krasnoyarsk   Riyadh       Urumqi
Baghdad    Gaza       Kuala_Lumpur  Saigon       Vientiane
Bahrain    Harbin     Kuching       Samarkand    Vladivostok
Baku       Hong_Kong  Kuwait        Seoul        Yakutsk
Bangkok    Hovd       Macao         Shanghai     Yekaterinburg
Beirut     Irkutsk    Magadan       Singapore    Yerevan
Bishkek    Istanbul   Manila        Taipei
Brunei     Jakarta    Muscat        Tashkent
Calcutta   Jayapura   Nicosia       Tbilisi
# cp Singapore /etc/localtime

Note that the files in /usr/share/zoneinfo/Asia (and the other directories) represent specific towns, and these may not correspond with the town in which you are located. Choose one in the same country and time zone.

You can do this at any time on a running system.

Network services

 Network services menu


. 6.4. Network services menu

The next step is to configure your networking equipment. Figure 6-4 shows the Network Services Menu. There are a number of ways to get to this menu:

The first step should always be to set up the network interfaces, so this is where you find yourself if you are performing a Standard or Express installation.

Setting up network interface

Figure 6-5 shows the network setup menu. On a standard 80x25 display it requires scrolling to see the entire menu. If you installed via FTP or NFS, you will already have set up your network interfaces, and sysinstall won't ask the questions again. The only real network board on this list is xl0, the Ethernet board. The others are standard hardware that can also be used as network interfaces. Don't try to set up PPP here; there's more to PPP configuration than sysinstall can handle. We'll look at PPP configuration in Chapter 20 .

Network setup menu


. 6.5. Network setup menu

In our case, we choose the Ethernet board. The next menu asks us to set the internet parameters. Figure 6-6 shows the network configuration menu after filling in the values. Specify the fully qualified local host name; when you tab to the Domain: field, the domain is filled in automatically. The names and addresses correspond to the example network that we look at in Chapter 16, on page 294. We have chosen to call this machine presto, and the domain is example.org. In other words, the full name of the machine is presto.example.org. It's IP address is 223.147.37.2. In his configuration, all access to the outside world goes via gw.example.org, which has the IP address 223.147.37.5. The name server is located on the same host, presto.example.org. The name server isn't running when this information is needed, so we specify all addresses in numeric form.

What happens if you don't have a domain name? If you're connecting to the global Internet, you should go out and get onesee page 318. But in the meantime, don't fake it. Just leave the fields empty. If you're not connecting to the Internet, of course, it doesn't make much difference what name you choose.

Network configuration menu


. 6.6. Network configuration menu

As is usual for a class C network, the net mask is 255.255.255.0. You don't need to fill in this informationif you leave this field without filling it in, sysinstall inserts it for you. Normally, as in this case, you wouldn't need any additional options to ifconfig.

Other network options

It's up to you to decide what other network options you would like to use. None of the following are essential, and none need to be done right now, but you may possibly find some of the following interesting:

You don't need to specify any of the remaining configuration options during configuration. See the online handbook for further details.

Startup preferences

The next step of interest is the Startup submenu, which allows you to choose settings that take effect whenever you start the machine. See Chapter 29 for details of the startup files.

Startup configuration menu The most important ones are:


. 6.7. Startup configuration menu The most important ones are:

Configuring the mouse

FreeBSD detects PS/2 mice at boot time only, so the mouse must be plugged in when you boot. If not, you will not be able to use it. To configure, select Mouse from the configuration menu. The menu in Figure 6-8 appears.

Mouse menu


. 6.8. Mouse menu

With a modern PS/2 mouse, you don't need to do any configuration at all. You just enable the mouse daemon or moused. Select the menu item Enable: you have the chance to move the mouse and note that the cursor follows. The keys don't work in this menu: select Yes and exit the menu. That's all you need to do.

If you're running a serial mouse, choose the item Select mouse port and set it to correspond with the port you have; if you have an unusual protocol, you may also need to set it with the Type menu. For even more exotic connections, read the man page for moused and set the appropriate parameters.

Configuring X

You should have installed X along with the rest of the systemsee page 75. If you haven't, install the package x11/XFree86. In this section, we'll look at what you need to do to get X up and running.

X configuration has changed a lot in the course of time, and it's still changing. The current method of configuring X uses a program called x/86c/g, which is still under development, and it shows a few strangenesss. Quite possibly the version you get will not behave identically with the following description. The differences should be relatively clear, however.

xf86cfg main menu


. 6.9. xf86cfg main menu

The configuration is stored in a file called XF86Config, thhough the directory has changed several times in the last few years. It used to be in /etc/X11/XF86Config or

/etc/XF86Config, but the current preferred place is /usr/X11R6/lib/X11/XF86Config.The server looks for the configuration file in multiple places, so if you're upgrading from an earlier version, make sure you remove any old configuration files. We'll look at the contents of the file in detail in Chapter 28 . In this section, we'll just look at how to generate a usable configuration.

From the configuration menu, select XFree86 and then xf86cfg. There is a brief delay while xf86cfg creates an initial configuration file, then you see the main menu of Figure 6-9. This application runs without knowing what the hardware is, so the rendering is pretty basic. The window on the left shows the layout of the hardware, and the window on the right is available in case your mouse isn't working. Select the individual components with the mouse or the numeric keypad. For example, to configure the mouse, select the image at top left:

xf86cfg mouse menu


. 6.10. xf86cfg mouse menu

In all likelihood that won't be necessary. The configuration file that xf86cfg has already created may be sufficient, so you could just exit and save the file. You'll probably want to change some things, though. In the following, we'll go through the more likely changes you may want to make.

Configuring the keyboard

You can select a number of options for the keyboard, including alternative key layouts. You probably won't need to change anything here.

xf86cfg keyboard menu


. 6.11. xf86cfg keyboard menu

Describing the monitor

Probably the most important thing you need to change are the definitions for the monitor and the display card. Some modern monitors and most AGP display cards supply the information, but older devices do not. In this example we'll configure a Hitachi CM813U monitor, which does not identify itself to x/86cfg. Select the monitor image at the top right of the window, then Configure Monitor(s). You see:



. 6.12.

xf86cfg doesn't know anything about the monitor, so it assumes that it can only display standard VGA resolutions at 640x480. The important parameters to change are the horizontal and vertical frequencies. You can select one of the listed possibilities, but unless you don't know your's monitor specifications, you should set exactly the frequencies it can do. In this case, the monitor supports horizontal frequencies from 31 kHz to 115 kHz and vertical frequencies from 50 Hz to 160 Hz, so that's what we enter. At the same time, we change the identifier to indicate the name of the monitor:

 xf86cfg monitor menu


. 6.13. xf86cfg monitor menu

Select OK to return to the previous menu.

Configuring the display card

xf86cfg recognizes most modern display cards, including probably all AGP cards, so you probably don't need to do anything additional to configure the display card. If you find that the resultant configuration file doesn't know about your card, you'll have to select the card symbol at the top of the screen. Even if the card has been recognized, you get this display:



. 6.14.

The only indication you have that xf86cfg has recognized the card (here a Matrox G200) is that it has selected mga for the driver name. If you need to change it, scroll down the list until you find the card:

xf86cfg card select menu


. 6.15. xf86cfg card select menu

Selecting display resolution

The display resolution is defined by Mode Lines, which we'll look at in detail on page 513. The names relate to the resolution they offer. By default, xf86cfg only gives you 640x480, so you'll certainly want to add more. First, select the field at the top left of the screen:

xf86cfg configuration selection


. 6.16. xf86cfg configuration selection

From this menu, select Configure ModeLine. You see:

xf86cfg mode line menu


. 6.17. xf86cfg mode line menu

If you pass the cursor over the image of the screen, you'll see this warning:

xf86cfg mode line warning


. 6.18. xf86cfg mode line warning

Take it seriously. Well look at this issue again in Chapter 28 on page 510. For an initial setup, you shouldn't use this interface. Instead, select Add standard VESA mode at the top. We get another menu:

xf86cfg VESA mode lines


. 6.19. xf86cfg VESA mode lines

Select the resolutions you want with the highest frequency that your hardware can handle. In this case, you might select 1024x768 @ 85 Hz, because it's still well within the range of the monitor. Answer yes to the question of whether you want to add it. You can select as many resolutions as you want, but the ModeLine window does not show them.

You can also use the ModeLine window to tune the display, but it's easier with another program, xvidtune.We'll look at those details in Chapter 28 .

Finally, select Quit at the bottom right of the display. You get this window:

xf86cfg quit


. 6.20. xf86cfg quit

When you answer Yes, you get a similar question asking whether you want to save the keyboard definition. Once you've done that, you're finished.

Desktop configuration

Next, select Desktop from the Configuration menu. You get this menu:

Desktop select menu


. 6.21. Desktop select menu

Which one do you install? You have the choice. If you know what you want, use it. There are many more window managers than shown here, so if you don't see what you're looking for, check the category x11-wm in the Ports Collection. The select menu gives you the most popular ones: Gnome, Afterstep, Enlightenment, KDE, Window maker and fwm2. In this book, we'll consider the KDE desktop and the fvwm2 window manager. KDE is comfortable, but it requires a lot of resources. Gnome is similar in size to KDE. By contrast, fvwm2 is much faster, but it requires a fair amount of configuration. We'll look at KDE and fvwm2 in Chapter 7 .

Additional X configuration

At this point, we're nearly done. A few things remain to be done:

Configuring xdm

To enable xdm, edit the file /etc/t/ys. By default it contains the following lines:

ttyv8  "/usr/X11R6/bin/xdm  -nodaemon"  xterm  off secure

Using an editor, change the text off to on:

ttyv8  "/usr/X11R6/bin/xdm  -nodaemon"  xterm  on secure

If you do this from a running system, send a HUP signal to init to cause it to re-read the configuration file and start xdm:

# kill -1 1

This causes an xdm screen to appear on /dev/ttyv8. You can switch to it with Alt-F9.

User X configuration

If you're starting X manually with startx, create a file .xinitrc in your home directory. This file contains commands that are executed when X starts. Select the line that corresponds to your window manager or desktop from the following list, and put it in .xinitrc:

 
Startkde    for kde
exec gnome-session   for Gnome
fvwm2        for fvwm2

If you're using xdm, you put the same content in the file .xsession in your home directory.

Rebooting the new system

When you get this for, you should have a functional system. If you're still installing from CD-ROM, you reboot by exiting sysinstall. If you have already rebooted, you exit sysinstall and reboot with:

# shutdown -r now

Don't just press the reset button or turn the power off. That can cause data loss. We'll look at this issue in more detail on page 541.

7. The tools of the trade

Users and groups; Gaining access; The KDE desktop; The fvwm2 window manager; Changing the X display; Getting a shell; Files and file names; Differences from Microsoft; The Emacs editor; Stopping the system.

So now you have installed FreeBSD, and it successfully boots from the hard disk. If you're new to FreeBSD, your first encounter with it can be rather puzzling. You probably didn't expect to see the same things you know from other platforms, but you might not have expected what you see either:

   FreeBSD (freebie.example.org) (ttyvO)
login:

If you have installed xdm, you'll at least get a graphical display, but it still asks you to log in and provide a password. Where do you go from here?

There isn't space in this book to explain everything there is about working with FreeBSD, but in the following few chapters I'd like to make the transition easier for people who have prior experience with Microsoft platforms or with other favours of UNIX. You can find a lot more information about these topics in UNIX for the Impatient, by Paul W. Abrahams and Bruce R. Larson, UNIX Power Tools, by Jerry Peek, Tim O'Reilly, and Mike Loukides, and UNIX System Administration Handbook, by Evi Nemeth, Garth Snyder, Scott Seebass, and Trent R. Hein. The third edition of this book also covers FreeBSD Release 3.2. See Appendix A, Bibliography, for more information.

If you've come from Microsoft, you will notice a large number of differences between UNIX and Microsoft, but in fact the two systems have more in common than meets the eye. Indeed, back in the mid-80s, one of the stated goals of MS-DOS 2.0 was to make it more UNIX-like. You be the judge of how successful that attempt was, but if you know The MS-DOS command-line interface, you'll notice some similarities in the following sections.

In this chapter, we'll look at FreeBSD from the perspective of somebody with computer experience, but with no UNIX background. If you do have a UNIX background, you may still find it interesting.

If you're coming from a Microsoft platform, you'll be used to doing just about everything with a graphical interface. In this book I recommend that you use X and possibly a desktop, but the way you use it is still very different. FreeBSD, like other UNIX-like systems, places much greater emphasis on the use of text. This may seem primitive, but in fact the opposite is true. It's easier to point and click than to type, but you can express yourself much more accurately and often more quickly with a text interface.

As a result, the two most important tools you will use with FreeBSD are the shell and the editor. Use the shell to issue direct commands to the system, and the editor to prepare texts. We'll look at these issues in more detail in this chapter. In Chapter 8, Taking control, we'll look at other aspects of the system. First, though, we need to get access to the system.

Users and groups

Probably the biggest difference between most PC operating systems and FreeBSD also takes the longest to get used to: FreeBSD is a multi-user, multi-tasking system. This means that many people can use the system at once, and each can do several things at the same time. You may think "Why would I want to do that?" Once you've got used to this idea, though, you'll never want to do without it again. If you use the X Window System, you'll find that all windows can be active at the same timeyou don't have to select them. You can monitor some activity in the background in another window while writing a letter, testing a program, or playing a game.

Before you can access a FreeBSD system, you must be registered as a user. The registration defines a number of parameters:

By the time you get here, you should have defined a user name, as recommended on page 94. If you haven't, you'll have to login as root and create one as described there.

Gaining access

Once you have a user name, you can log in to the system. Already you have a choice: FreeBSD offers both virtual terminals and the X WindowSystem. The former displays plain text on the monitor, whereas the latter uses the system's graphics capabilities. Once running, you can switch from one to the other, but you have the choice of which interface you use first. If you don't do anything, you get a virtual terminal. If you run xdm, you get X.

It's still relatively uncommon to use xdm, and in many instances you may not want X at all, for example if you're running the system as a server. As a result, we'll look at the "conventional" login first.

If you're logging in on a virtual terminal, you'll see something like this:

login:    grog
Password:              password doesn't show on the screen
Last login: Fri Apr 11 16:30:04 from canberra
Copyright (c)  1980, 1983, 1986,  1988, 1990,  1991, 1993, 1994
    The Regents of the University of California. All rights reserved.

FreeBSD 5.0-RELEASE (FREEBIE) #0: Tue Dec 31 19:08:24 CST 2002

Welcome to FreeBSD!

You have mail.
erase ^H, kill ^U, intr ^C, status ^T
Niklaus Wirth has lamented that, whereas Europeans pronounce his name
correctly (Ni-klows Virt), Americans invariably mangle it into
(Nick-les Worth).  Which is to say that Europeans call him by name, but
Americans call him by value.
 === grog@freebie (/dev/ttyv0) ~ 1 ->

There's a lot of stuff here. It's worth looking at it in more detail:

The prompt illustrates a number of things about the UNIX environment. By default, sh and friends prompt with a $, and csh and friends prompt with a %. You can change it to just about anything you want with the UNIX shells. You don't have to like my particular version, but it's worth understanding what it's trying to say.

The first part, ===, is just to make it easier to find in a large list on an X display. An xterm window on a high resolution X display can contain up to 120 lines, and searching for command prompts can be non-trivial.

Next, grog@freebie is my user ID and the name of system on which I am working, in the RFC 2822 format used for mail IDs. Multiple systems and multiple users can all be present on a single X display. This way, I can figure out which user I am and what system I am running on.

/dev/ttyv0 is the name of the terminal device. This can sometimes be useful.

~ is the name of the home directory. Most shells, but not all of them, support this symbolism.

1 is the prompt number. Each time you enter a command, it is associated with this number, and the prompt number is incremented. One way to re-execute the command is to enter !!1 (two exclamation marks and the number of the command). We'll look at more comfortable ones on page 131.

To start X from a virtual terminal shell, use the startx command:

$ startx

If you use xdm, you bypass the virtual terminals and go straight into X. Enter your user name and password to the login prompt or the xdm login screen, and press Enter. If you use the xdm login, you'll go straight into X.

KDE display




. 7.1. KDE display

Either way, assuming that you've installed and configured kde, you'll get a display similar to that in Figure 7-1 . This example includes four windows that are not present on startup. On startup the central part of the screen is empty. We'll look at the windows further below.

The KDE desktop

KDE is a complicated system, and good documentation is available at http://www.kde.0rg/documentation/. Once you have KDE running, you can access the same information via the help icon on the panel at the bottom (the life ring icon). The following description gives a brief introduction.

The KDE display contains a number of distinct areas. At the top is an optional menu, at the bottom an almost optional panel, and the middle of the screen is reserved for windows.

The Desktop Menu

The Desktop Menu is at the very top of the screen. It provides functionality that is not specific to a particular application. Select the individual categories with the mouse. For example, the New menu looks like this:

KDE desktop menu


. 7.2. KDE desktop menu

As the menu indicates, you can use these menus to create new files.

The Panel

At the bottom of the screen is the panel, which consists of a number of fields. The left-hand section is used for starting applications.



. 7.3.

The stylized letter K at the extreme left is the Application Starter. When you select it, a long vertical menu appears at the left of the screen and allows you to start programs ("applications") or access just about any other function.

Next comes an icon called "showdesktop." This is a convenient way to iconify all the windows currently on the desktop.

The remaining icons on this part of the panel represent various applications.

The next section of the panel contains some control buttons and information about the current desktop layout:



. 7.4.

The section at the left shows the current contents of four screens, numbered 1 to 4. Screen 1 is the currently displayed screen; you can select one of the others by moving the cursor in the corresponding direction, or by selecting the field with the mouse.

To the right of that are icons for the currently active windows. The size expands and contracts depending on the number of different kinds of window active. If you select one of these icons with the left mouse button, it will iconify or deiconify ("minimize" or "maximize") the window. If you have multiple xterms active, you will only have one icon. In this case, if you select the icon, you will get another pop-up selection menu to allow you to choose the specific window.

The right part of the panel contains a further three fields:



. 7.5.

Probably the most useful part of this section of the panel is not very obvious: the right-pointing arrow allows you to remove the panel if you find it's in the way. The entire panel is replaced by a single left-pointing arrow at the extreme right of the display.

Using the mouse

By default, kde only uses the left and the right mouse buttons. In general, the left button is used to select a particular button, and the right button is used for an auxiliary menu.

Manipulating windows

You'll notice that each window has a frame around it with a number of features. In X terminology, they're called decorations. Specifically:

The fvwm2 window manager

If you come from a conventional PC background, you shouldn't have much difficulty with KDE. It's a relatively complete, integrated environment. But it isn't really UNIX. If you come from a UNIX environment, you may find it too all-encompassing. You may also find that there are significant delays when you start new applications.

UNIX has a very different approach to windows. There is no desktop; just a window manager. It takes up less disk space, less processor time, and less screen real estate. By default, XFree86 comes with the twm window manager, but that's really a little primitive. With modern machines, there's no reason to choose such a basic window manager. You may, however, find that fvwm2 is more your style than KDE.

Starting fvwm2

Like KDE, you install fvwm2 from the Ports Collection. It's not designed to work completely correctly out of the box, though it does work. As with KDE, the first thing you need to do is to create a .xsession or .xinitrc file, depending on whether you're running xdm. It must contain at least the line:

fvwm2

Start X the same way you did for KDE. This time you see, after starting the same applications as before:





. 7.6.

This picture shows both similarities with and differences from KDE. The similarities include:

Still, there are a number of differences as well:

The menus above show one of the problems: look at those system names on the left submenu (dopey, snoopy and friends). They don't exist on our sample network and the chance of them existing on your network is pretty low as well. They're hard-coded in the sample configuration file, /usr/X11R6/etc/system.fvwm2rc. To use fvwm2 effectively; you'll have to modify the configuration file. The best thing to do is to make a copy of /usr/X11R6/etc/system.fvwm2rc in your own directory, as ~/.fVwm2/.fvwm2rc. Then you can have lots of fun tweaking the file to do exactly what you want it to do. Clearly, KDE is easier to set up.

Changing the X display

When you set up your XF86Config file, you may have specified more than one resolution. For example, on page 107 we selected the additional resolution 1024x768 pixels. When you start X, it automatically selects the first resolution, in this case 640x480. You can change to the previous resolution (the one to the left in the list) by pressing the Ctrl-Alt-Keypad - key, and to the following resolution (the one to the right in the list) with Ctrl-Alt-Keypad +. Keypad + and Keypad - refer to the + and - symbols on the numeric keypad at the right of the keyboard; you can't use the + and - symbols on the main keyboard for this purpose. The lists wrap around: in our example, if your current resolution is 640x480, and you press Ctrl-Alt-Keypad -, the display changes to 1024x768. It's a very good idea to keep the default resolution at 640x480 until you have debugged your XF86Config parameters: 640x480 almost always works, so if your display is messed up, you can just switch back to a known good display with a single keystroke.

Selecting pixel depth

You can configure most display boards to display a number of different pixel depths (a different number of bits per pixel, which translates to a different number of colours). When you start X, however, it defaults to 8 bits per pixel (256 colours), which is a very poor rendition. To start it with a different number, specify the number of planes. For example, to start with 32 bits per pixel (4,294,967,296 colours), enter:

$ Startx ---bpp 32

With older display boards, which had relatively limited display memory, there was a tradeoff between maximum resolution and maximum pixel depth. Modern display cards no longer have this limitation.

Getting a shell

As we saw at the beginning of the chapter, your main tools are the shell and the editor, and that's what we saw on the sample screens. But when you start X, they're not there: you need to start them.

In KDE, you have two ways to start a terminal window:

Obviously the first is the intended approach, and it's easier. Nevertheless, I recommend using xterm at least until you're sure you want to stick with kde: there are some subtle differences, and konsole is intended to work with kde only. If you do stick with KDE, you should change the configuration of the konsole button to start xterm instead; that's relatively straightforward.

In fvwm2, you start an xterm from the left mouse menu, as shown above.

Shell basics

The most basic thing you can do with the shell is to start a program. Consider program names to be commands: like you might ask somebody to "wash the dishes" or "mow the lawn," you can tell the shell to "remove those files":

$ rm filel file2 file3

This starts a program called rm (remove), and gives it a list of three file names to remove.

If you're removing a whole lot of files, this could take a while. Consider removing the entire directory hierarchy /usr/obj, which is created when building a new version of the system (see page 595). This directory hierarchy contains about 15,000 files and directories, and it'll take a while to remove it. You can do this with the following command:

# rm -rf /usr/obj &

In this example, we have a couple of options led in by a hyphen (-) and also the character & at the end of the line.

Options

In the previous example, we saw a couple of options. By convention, they come between the command name and other parameters, and they're identified because they start with a hyphen character (-). There's a lot of variation, though, is depending on the individual program.

This is an alternative file naming convention.

Shell parameters

When you invoke a program with the shell, it first parses the input line before passing it to the program: it turns the line into a number of parameters (called arguments in the C programming language). Normally the parameters are separated by white space, either a space or a tab character. For example, consider the previous example:

$ rm file1 file2 file3

the program receives four arguments, numbered 0 to 3:

7.1. Program arguments
ArgumentValue
0rm
1File1
2File2
3File3

What happens if you want to pass a name with a space? For example, you might want to look for the text "Mail rejected" in a log file. UNIX has a standard program for looking for text, called grep. The syntax is:

grep expression files

Argument 1 is the expression; all additional arguments are the names of files to search. We could write:

$ grep Mail rejected /var/log/maillog

But that would try to look for the text Mail in the files rejected (probably causing an error message that the file did not exist) and /var/log/maillog (where just about every line contains the text Mail). That's not what we want. Instead, we do pretty much what I wrote above:

$ grep "Mail rejected" /var/log/maillog

In other words, if we put quote characters "" around a group of words, the shell will interpret them as a single parameter. The first parameter that is passed to grep is Mail rejected, not "Mail rejected".

This behaviour of the shell is a very good reason not to use file names with spaces in them. It's perfectly legitimate to embed spaces into UNIX files names, but it's a pain to use. If you want to create a file name that contains several words, for example All files updated since last week, consider changing the spaces to underscores: All_files_updated_since_last_week.

It's even more interesting to see what happens when you pass a globbing character to a program, for example:

$ cc -o foo *.c

This invocation compiles all C source files (*.c) and creates a program foo. If you do this with Microsoft, the C compiler gets four parameters, and it has to find the C source files itself. In UNIX, the shell expands the text *.c and replaces it with the names of the source files. If there are thirty source files in the directory, it will pass a total of 33 parameters to the compiler.

Fields that can contain spaces

The solution to the "Mail rejected" problem isn't ideal, but it works well enough as long as you don't have to handle fields with blanks in them too often. In many cases, though, particularly in configuration files, fields with blanks are relatively common. As a result, a number of system configuration files use a colon (:) as a delimiter. This looks very confusing at first, but it turns out not to be as bad as the alternatives. We'll see some examples in the PATH environment variable on page 130, in the password file on page 144, and in the login class file on page 571.

Files and file names

Both UNIX and Microsoft environments store disk data in files, which in turn are placed in directories .A file may be a directory: that is, it may contain other files. The differences between UNIX and Microsoft start with file names. Traditional Microsoft file names are rigid: a file name consists of eight characters, possibly followed by a period and another three characters (the so-called file name extension). There are significant restrictions on which characters may be used to form a file name, and upper and lower case letters have the same meaning (internally, Microsoft converts the names to UPPER CASE). Directory members are selected with a backslash (\), which conflicts with other meanings in the C programming languagesee page 138 for more details.

FreeBSD has a very fexible method of naming files. File names can contain any character except /, and they can be up to 255 characters long. They are case-sensitive: the names FOO, Foo and foo are three different names. This may seem silly at first, but any alternative means that the names must be associated with a specific character set. How do you upshift the German name ? What if the same characters appear in a Russian name? Do they still shift the same? The exception is because the / character represents directories. For example, the name /home/fred/longtext-with-a-long-name represent:

First character is a /, representing the root file system.

home is the name of a directory in the root file system.

fred is the name of a directory in /home.

The name suggests that longtext-with-a-long-name is probably a file, not a directory, though you can't tell from the name.

As a result, you can't use / in a file name. In addition, binary 0s (the ASCII NUL character) can confuse a lot of programs. It's almost impossible to get a binary 0 into a file name anyway: that character is used to represent the end of a string in the C programming language, and it's difficult to input it from the keyboard.

Case sensitivity no longer seems as strange as it once did: web browsers have made UNIX file names more popular with Uniform Resource Indicators or URIs, which are derived from UNIX names.

File names and extensions

The Microsoft naming convention (name, period and extension) seems similar to that of UNIX. UNIX also uses extensions to represent specific kinds of files. The difference is that these extensions (and their lengths) are implemented by convention, not by the file system. In Microsoft, the period between the name and the extension is a typographical feature that only exists at the display level: it's not part of the name. In UNIX, the period is part of the name, and names like foo.bar.bazzot are perfectly valid file names. The system doesn't assign any particular meaning to file name extensions; instead, it looks for magic numbers, specific values in specific places in the file.

Relative paths

Every directory contains two directory entries, . and .. (One and two periods). These are relative directory entries: . is an alternative way to refer to the current directory, and .. refers to the parent directory. For example, in /home/fred, . refers to /home/fred, and .. refers to /home. The root directory doesn't have a parent directory, so in this directory only, .. refers to the same directory. We'll see a number of cases where this is useful1).

Globbing characters

Most systems have a method of representing groups of file names and other names, usually by using special characters for representing an abstraction. The most common in UNIX are the characters *,? and the square brackets []. UNIX calls these characters globbing characters. The Microsoft usage comes from UNIX, but the underlying file name representation makes for big differences. Table 7-2 gives some examples.

7.2. Globbing examples
NameMicrosoft meaningUNIX meaning
CONFIG.*All files with the name CONFIG, no matter what their extension.All files whose name starts with CONFIG., no matter what the rest is. Note that the name contains a period.
CONFIG.BA?All files with the name CONFIG and an extension that starts with BA, no matter what the last character.All files that start with CONFI.BA and have one more character in their name.
*Depending on the Microsoft version, all files without an extension, or all files.All files.
*.*All files with an extension.All files that have a period in the middle of their name.
foo[127]In older versions, invalid. In newer versions with long file name support, the file with the name foo[127].The three files foo1, foo2 and foo7.

Input and output

Most programs either read input data or write output data. To make it easier, the shell usually starts programs with at least three open files:

With an interactive shell (one that works on a terminal screen, like we're seeing here), all three files are the same device, in this case the terminal you're working on.

Why two output files? Well, you may be collecting something important, like a backup of all the files on your system. If something goes wrong, you want to know about it, but you don't want to mess up the backup with the message.

Redirecting input and output

But of course, even if you're running an interactive shell, you don't want to back up your system to the screen. You need to change stdout to be a file. Many programs can do this themselves; for example, you might make a backup of your home directory like this:

$ tar -cf /var/tmp/backup-

This creates (option c) a file (option f) called /var/tmp/backup, and includes all the files in your home directory (~). Any error messages still appear on the terminal, as stderr hasn't been changed.

This syntax is specific to tar.The shell provides a more general syntax for redirecting input and output streams. For example, if you want to create a list of the files in your current directory, you might enter:

$ ls -l
drwxr-xr-x  2 root  wheel  512   Dec  20  14:36  CVS
-rw-r--r--    1 root  wheel  7928  Oct  23  12:01  Makefile
-rw-r--r--    5 root  wheel  209   Jul  26  07:11  amd.map
-rw-r--r--    5 root  wheel  1163  Jan  31  2002  apmd.conf
-rw-r--r--    5 root  wheel  271   Jan  31  2002  auth.conf
-rw-r--r--    1 root  wheel  741   Feb  19  2001  crontab
-rw-r--r--    5 root  wheel  108   Jan  31  2002  csh.cshrc
-rw-r--r--    5 root  wheel  482   Jan  31  2002  csh.login
(etc)

You can redirect this output to a file with the command:

$ ls -l > /var/tmp/etclist

This puts the list in the file /var/tmp/etclist. The symbol > tells the shell to redirect stdout to the file whose name follows. Similarly, you could use the < to redirect stdin to that file, for example when using grep to look for specific texts in the file:

$ grep csh < /var/tmp/etclist
  -rw-r--r--     5 root   wheel  108 Jan 31  2002 csh.cshrc
  -rw-r--r--     5 root   wheel  482 Jan 31  2002 csh.login
  -rw-r--r--     5 grog   lemis  110 Jan 31  2002 csh.logout

In fact, though, there's a better way to do that: what we're doing here is feeding the output of a program into the input of another program. That happens so often that there's a special method of doing it, called pipes:

| grep csh
  -rw-r--r--     5 root  wheel  108 Jan 31  2002 csh.cshrc
  -rw-r--r--     5 root  wheel  482 Jan 31  2002 csh.login
  -rw-r--r--     5 grog  lemis  110 Jan 31  2002 csh.logout

The | symbol causes the shell to start two programs. The first has a special file, a pipe, as the output, and the second has the same pipe as input. Nothing gets written to disk, and the result is much faster.

A typical use of pipes is to handle quantities of output data in excess of a screenful. You can pipe to the less2)program, which enables you to page backward and forward:

$ ls -l | less

Another use is to sort arbitrary data:

$ ps aux | sort -n +1

This command takes the output of the ps command and sorts it by the numerical (-n) value of its second column (+1). The first column is numbered 0.

Environment variables

The UNIX programming model includes a concept called environment variables. This rather unusual sounding name is simply a handy method of passing relatively long-lived information of a general nature from one program to another. It's easier to demonstrate the use than to describe. Table 7-3 takes a look at some typical environment variables. To set environment variables from Bourne-style shells, enter:

 $ export TERM=xterm

This sets the value of the TERM variable to xterm. The word export tells the shell to pass this information to any program it starts. Once it's exported, it stays exported. If the variable isn't exported, only the shell can use it.

Alternatively, if you want to set the variable only once when running a program, and then forget it, you can set it at the beginning of a command line:

$ TERM=xterm-color mutt

This starts the mutt mail reader (see page 474) with xterm's colour features enabled.

For csh and tcsh, set environment variables with:

% setenv TERM xterm

To start a process with these variables, enter:

% env xterm-color mutt
7.3. Common environment variables
NamePurpose
BLOCKSIZEThe size of blocks that programs like df count. The default is 512 bytes, but its often more convenient to use 1024 or even 1048576 (1 MB).
DISPLAYWhen running X, the name of the X server. For a local system, this is typically unix:0. For remote systems, its in the form system-name: server-number.screen-number. For the system bumble.example.org, you would probably write bumble.example.org:0.
EDITORThe name of your favourite editor. Various programs that start editors look at this variable to know which editor to start.
HOMEThe name of your home directory.
LANGThe locale that you use. This should be the name of a directory in /usr/share/locale.
MAILSome programs use this variable to find your incoming mail file.
MANPATHA list of path names, separated by colons (:), that specifies where the man program should look for man pages. A typical string might be /usr/share/man:/usr/local/man, and specifies that there are many pages in each of the directories /usr/share/man and /usr/local/man.
NTAPEThe name of the non-rewinding tape device. See page 252 for more details.
PATHA list of path names, separated by colons (:), that specifies where the shell should look for executable programs if you specify just the program name.
PS1In Bourne-style shells, this is the prompt string. Its usually set to $, but can be changed. See page 114 for a discussion of a possible prompt for bash.
PS2In Bourne-style shells, this is the prompt string for continuation lines. Its usually set to >.
SHELLThe name of the shell. Some programs use this for starting a shell.
TAPEThe name of the rewinding tape device. See page 252 for more details.
TERMThe type of terminal emulation you are using. This is very important: there is no other way for an application to know what the terminal is, and if you set it to the wrong value, full-screen programs will behave incorrectly.
TZTime zone. This is the name of a file in /usr/share/zoneinfo that describes the local time zone. See the section on timekeeping on page 155 for more details.

Note particularly the PATH variable. One of the most popular questions in the FreeBSD-questions mailing list is "I have compiled a program, and I can see it in my directory, but when I try to run it, I get the message "command not found." This is usually because PATH does not include the current directory.

It's good practice not to have your current directory or your home directory in the PATH: if you do, you can be subject to security compromises. For example, somebody could install a program called ps in the directory /var/tmp. Despite the name, the program might do something else, for example remove all files in your home directory. If you change directory to /var/tmp and run ps, you will remove all files in your home directory. Obviously much more subtle compromises are possible.

Instead, run the program like this:

$ ./program

You should set your PATH variable to point to the most common executable directories. Add something like this to your .profile file (for Bourne-style shells):

PATH=/usr/bin:/usr/local/bin:/usr/sbin:/bin:/sbin:/usr/X11R6/bin
export PATH

This variable is of great importance: one of the leading problems that beginners have is to have an incorrect PATH variable.

Printing out environment variables

So you can't start a program, and you're wondering whether your PATH environment variable is set correctly. You can find out with the echo command:

$ echo $PATH
/bin:/usr/bin

The $ at the beginning of $PATH tells the shell to substitute the value of the environment variable for its name. Without this, the shell has no way of knowing that it's an environment variable, so it passes the text PATH to echo, which just prints it out.

If you want to print out all the environment variables, use the printenv command:

$ printenv | sort
BLOCKSIZE=1048576
CLASSPATH=/usr/local/java/lib:/usr/local/java/lib/classes.zip:/hcme/grcg/netscape/
CVSROOT=/home/ncvs
DISPLAY=freebie:0
EDITOR=emacs
HOME=/home/grog
PAGER=less
PATH=.:/usr/bin:/usr/sbin:/bin:/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin
XAUTHORITY=/home/grog/.Xauthority

This example sorts the variables to make it easier to find them. In all probability, you'll find many more variables.

Command line editing

Typing is a pain. If you're anything like me, you're continually making mistakes, and you may spend more time correcting typing errors than doing the typing in the first place. It's particularly frustrating when you enter something like:

$ groff -rex=7.5 -r$$ -rL -rW -rN2 -mpic tmac.M unixerf.nm
troff: fatal error: can't open 'unixerf.mm': No such file or directory

This command should create the PostScript version of this chapter, but unfortunately I messed up the name of the chapter: it should have been unixref.mm, and I typed unixerf.mm.

Yes, I know this looks terrible. In fact, UNIX has ways to ensure
 you almost never need to write commands like this. The command I really use to format this chapter
is "make unixref".

It would be particularly frustrating if I had to type the whole command in again. UNIX offers a number of ways to make life easier. The most obvious one is so obvious that you tend to take it for granted: the Backspace key erases the last character you entered. Well, most of the time. What if you're running on a machine without a Backspace key? You won't have that problem with a PC, of course, but a lot of workstations have a DEL key instead of a Backspace key. UNIX lets you specify what key to use to erase the last character entered. By default, the erase character really is DEL, but the shell startup changes it and prints out a message saying what it has done:

 erase ^H, kill ^U, intr ^C, status ^T

in the example on page 113. ^H (Ctrl-H) is an alternative representation for Backspace.

The three other functions kill, intr, and status perform similar editing functions. Kill erases the whole line, and intr stops a running program.

More correctly, intr sends a signal called SIGINT to the process. This normally causes a program to stop.

You'll notice that it is set to Ctrl-C, so its function is very similar to that of the MS-DOS Break key. status is an oddball function: it doesn't change the input, it just displays a statistics message. bash doesn't in fact use it: it has a better use for Ctrl-T.

In fact, these control characters are just a few of alarge number of control characters that you can set. Table 7-4 gives a nover view of the more common control characters. For a complete list, see the man page stty(1).

7.4. Terminal control characters
NameDefaultFunction
CR\rGo to beginning of line. Normally, this also terminates input (in other words, it returns the complete line to the program, which then acts on the input).
NL\nEnd line. Normally, this also terminates input.
INTRCtrl-CGenerate a SIGINT signal. This normally causes the process to terminate.
QUITCtrl-|Generate a SIGQUIT signal. This normally causes the process to terminate and core dump, to save a copy of its memory to disk for later analysis.
ERASEDELErase last character. FreeBSD sets this to Backspace on login, but under some unusual circumstances you might find it still set to DEL.
KILLCtrl-UErase current input line.
EOFCtrl-DReturn end-of-file indication. Most programs stop when they receive an EOF.
STOPCtrl-SStop output. Use this to examine text that is scrolling faster than you can read.
STARTCtrl-QResume output after stop.
SUSPCtrl-ZSuspend process. This key generates a SIGTSTP signal when typed. This normally causes a program to be suspended. To restart, use the fg command
DSUSPCtrl-YDelayed suspend. Generate a SIGTSTP signal when the character is read. Otherwise, this is the same as SUSP.
REPRINTCtrl-RRedisplay all characters in the input queue (in other words, characters that have been input but not yet read by any process). The term "print" recalls the days of harcopy terminals. Many shells disable this function.
DISCARDCtrl-ODiscard all terminal output until another DISCARD character arrives, more input is typed or the program clears the condition.

To set these characters, use the stty program. For example, if you're used to erasing the complete input line with Ctrl-X, and specifying an end-of-file condition with Ctrl-Z, you could enter:

$ stty susp \377 kill "X eof "Z

You need to set SUSP to something else first, because by default it is Ctrl-Z, so the system wouldn't know which function to perform if you press "Z.

The combination \377 represents the character octal 377 (this notation comes from the C programming language, and its origin is lost in the mists of time, back in the days when UNIX ran on PDP-11s). This character is the "null" character that turns off the corresponding function. System V uses the character \0 for the same purpose.

In this particular case, ^X really does mean the character ^ followed by the letter X, and not Ctrl-X, the single character created by holding down the Control character and pressing X at the same time.

Command history and other editing functions

Nowadays, most shells supply a command history function and additional functionality for editing it. We'll take a brief look at these features herefor more details see the man pages for your shell.

Shell command line editing has been through a number of evolutionary phases. The original Bourne shell supplied no command line editing at all, though the version supplied with FreeBSD gives you many of the editing features of more modern shells. Still, it's unlikely that you'll want to use the Bourne shell as your shell: bash, ksh, and zsh are all compatible with the Bourne shell, but they also supply better command line editing.

The next phase of command line editing was introduced with the C shell, csh.By modern standards, it's also rather pitiful. It's described in the csh man page if you really want to know. About the only part that is still useful is the ability to repeat a previous command with the !! construct. Modern shells supply command line editing that resembles the editors vi or Emacs. In bash, sh, ksh, and zsh you can make the choice by entering:

$ set -o emacs   for Emacs-style editing
$ set -o vi      for vi-style editing

In tcsh, the corresponding commands are:

% bind emacs
% bind vi

Normally you put one of these commands in your startup file.

In Emacs mode, you enter the commands simply by typing them in. In vi mode, you have to press ESC first. Table 7-5 shows an overview of the more typical Emacs-style commands in bash. Many other shells supply similar editing support.

As the name suggests, the Emacs editor understands the same editing characters. It also understands many more commands than are shown here. In addition, many X-based commands, including web browsers, understand some of these characters.

7.5. Emacs editing characters
KeyFunction
Ctrl-AMove to the beginning of the line.
LeftArrowMove to previous character on line.
Ctrl-BMove to previous character on line (alternative).
Ctrl-DDelete the character under the cursor. Be careful with this character: its also the shells end-of-file character, so if you enter it on an empty line, it stops your shell and logs you out.
Ctrl-EMove to the end of the line.
RightArrowMove to next character on line.
Ctrl-FMove to next character on line (alternative).
Ctrl-KErase the rest of the line. The contents are saved to a ring buffer of erased text and can be restored, possibly elsewhere, with Ctrl-Y.
Ctrl-LErase screen contents (shell) or redraw window (Emacs).
DownArrowMove to next input line.
Ctrl-NMove to next input line (alternative).
UpArrowMove to previous input line.
Ctrl-PMove to previous input line (alternative).
Ctrl-RIncremental search backward for text.
Ctrl-SIncremental search forward for text.
Ctrl-TTranspose the character under the cursor with the character before the cursor.
Ctrl-YInsert previously erased with Ctrl-K or Alt-D.
Ctrl-_Undo the last command.
Alt-CCapitalize the following word.
Alt-DDelete the following word.
Alt-FMove forward one word
Alt-LConvert the following word to lower case.
Alt-TTranspose the word before the cursor with the one after it.
Alt-UConvert the following word to upper case.
Ctrl-X Ctrl-SSave file (Emacs only).
Ctrl-X Ctrl-CExit the Emacs editor.

You'll note a number of alternatives to the cursor keys. There are two reasons for them: firstly, the shell and Emacs must work on systems without arrow keys on the keyboard. The second reason is not immediately obvious: if you're a touch-typer, it's easier to type Ctrl-P than take your hands away from the main keyboard and look for the arrow key. The arrows are good for beginners, but if you get used to the control keys, you'll never miss the arrow keys.

File name completion

As we have seen, UNIX file names can be much longer than traditional Microsoft names, and it becomes a problem to type them correctly. To address this problem, newer shells provide file name completion. In Emacs mode, you typically type in part of the name, then press the Tab key. The shell checks which file names begin with the characters you typed. If there is only one, it puts in the missing characters for you. If there are none, it beeps (rings the "terminal bell"). If there are more than one, it puts in as many letters as are common to all the file names, and then beeps. For example, if I have a directory docco in my home directory, I might enter:

=== grog@freebie (/dev/ttyp4) ~ 14 -> cd docco/
=== grog@freebie (/dev/ttyp4) "/docco 15 -> ls freebsd.faq   freebsd.fbc freeware
=== grog@freebie (/dev/ttyp4) "/docco 16 -> emacs freebeepbsd.fbeepaq

Remember that my input is in constant width bold font, and the shell's output is in constant width font. On the first line, I entered the characters cd doc followed by a Tab character, and the shell completed with the text co/. On the last line, I entered the characters emacs f and a Tab. In this case, the shell determined that there was more than one file name that started like this, so it added the letters ree and rang the bell. I entered the letter b and pressed Tab again, and the shell added the letters sd.f and beeped again. Finally, Iadded the letters aq to complete the file name freebsd.faq.

Command line completion in vi mode is similar: instead of pressing Tab, you press ESC twice.

Shell startup files

As we saw above, there are a lot of ways to customize your shell. It would be inconvenient to have to set them every time, so all shells provide a means to set them automatically when you login. Nearly every shell has its own startup file. Table 7-6 gives an overview.

7.6. Shell startup files
Shellstartup file
bash.profile, then .bashrc
csh.login on login, always .cshrc
sh.profile
tcsh.login on login, always .tcshc, .cshrc if .tcshrc not found

These files are shell scriptsin other words, straight shell commands. listing 7-1 shows a typical .bashrc file to set the environment variables we discussed.

umask 022
export BLOCKSIZE=1024     #for df
export CVSROCT=/src/ncvs
export EDITCR=/opt/bin/emacs
export MANPATH=/usr/share/man:/usr/local/man
export MCZILLA_HOME=/usr/local/netscape
export PAGER=less
export PATH=/usr/bin:/usr/local/bin:/usr/sbin:/bin:/sbin:/usr/X11R6/bin
PS1="=== \u@\h rtty1) \w \# -> "
PS2="\u@\h \w \! ++ "
export SHELL=/usr/local/bin/bash
export TAPE=/dev/nsa0     #note non-rewinding as standard 
if [ "$TERM" = "" ]; then
  export TERM=xterm
fi
if [ "$DISPLAY" = "" ]; then
  export DISPLAY=:0
fi
/usr/games/fortune       # print a fortune cookie
7.1. Minimal .bashrc file (html, txt)

It would be tedious for every user to put settings in their private initialization files, so the shells also read a system-wide default file. For the Bourne shell family, it is /etc/profile, while the C shell family has three files: /etc/csh.login to be executed on login, /etc/csh.cshrc to be executed when a newshell is started after you login, and /etc/csh.logout to be executed when you stop a shell. The start files are executed before the corresponding individual files.

In addition, login classes (page 571) offer another method of setting environment variables at a global level.

Changing your shell

The FreeBSD installation gives root a C shell, csh.This is the traditional BSD shell, but it has a number of disadvantages: command line editing is very primitive, and the script language is significantly different from that of the Bourne shell, which is the de facto standard for shell scripts: if you stay with the C shell, you may still need to understand the Bourne shell. The latest version of the Bourne shell sh also includes some command line editing. See page 133 for details of how to enable it.

If you want to stay with a csh-like shell, you can get better command line editing with tcsh, which is also in the base system. You can get both better command line editing and Bourne shell syntax with bash, in the Ports Collection.

If you have root access, you can use vipw to change your shell, but there's a more general way: use chsh (ChangeShell). Simply run the program. It starts your favourite editor (as defined by the EDITOR environment variable). Here's an example before:

#Changing user database information for velte.
Shell: /bin/csh
Full Name: Jack Velte
Location:
Office Phone:
Home Phone:

You can change anything after the colons. For example, you might change this to:

#Changing user database information for velte.
Shell: /usr/local/bin/bash
Full Name: Jack Velte
Location: On the road
Office Phone: +1-408-555-1999
Home Phone:

chsh checks and updates the password files when you save the modifications and exit the editor. The next time you login, you get the new shell. chsh tries to ensure you don't make any mistakesor example, it won't let you enter the name of a shell that isn't mentioned in the file /etc/shellsbut it's a very good idea to check the shell before logging out. You can try this with su, which you normally use to become super user:

bumble# su velte Password:
su-2.00$       note the newprompt

You might hear objections to using bash as a root shell. The argument goes something like this: bash is installed in /usr/local/bin, so it's not available if you boot into single-user mode, where only the root file system is available. Even if you copy it to, say, /bin, you can't run it in single-user mode because it needs libraries in /usr/lib.

In fact, this isn't a problem. If you install the system the way I recommend in Chapter 5, /usr is on the root file system. Even if it isn't, though, you don't have to use bash in single-user mode. When you boot to single-user mode, you get a prompt asking you which shell to start, and suggesting /bin/sh.

Differences from Microsoft

If you're coming from a Microsoft background, there are a few got chas that you might trip over.

Slashes: backward and forward

/ (slash) and \ (backslash) are confusing. As we've seen, UNIX uses / to delimit directories. The backslash \ is called an escape character. It has, several purposes:

Tab characters

We've seen that the shell treats "white space," either spaces or tab characters, as the same. Unfortunately, some other programs do not. make, sendmail and syslogd make a distinction between the two kinds of characters, and they all require tabs (not spaces) in certain places. This is a real nuisance, because hardly any editor makes a distinction between them.

Carriage control characters

In the olden days, the standard computer terminal was a Teletype, a kind of computer-controlled electric type writer. When the carriage, which contained the print head, got to the end of a line, it required two mechanical operations to move to the beginning of the next line: the carriage return control character told it to move the carriage back to the beginning of the line, and the line feed character told it turn the platen to the next line.

Generations of computer systems emulated this behaviour by putting both characters at the end of each text line. This makes it more difficult to recognize the end of line, it uses up more storage space, and normally it doesn't buy you much. The implementors of UNIX decided instead to use a single character, which it calls the newline character. For some reason, they chose the line feed to represent new line, though the character generated by Enter is a carriage return. As we saw a bove, the C programming language represents it as \n.

This causes problems transferring data between FreeBSD and Microsoft, and also when printings to printers that still expect both characters. We'll look at the file transfer issues on page 260 and the printer issues on page 267.

The Emacs editor

A part from the shell, your second most important tool is the editor, a program that creates and changes texts. Another divergence of concept between UNIX and Microsoft environments is that UNIX gives you a choice of editors in just about anything you do. Microsoft products frequently try to redefine the whole environment, so if you change mailers, you may also have to change the editor you use to write mail. This has a profound effect on the way you work. In particular, the Microsoft way makes it uninteresting to write a really good editor, because you can't use it all the time.

The standard BSD editor is vi, about which people speak with a mixture of admiration, awe and horror. vi is one of the oldest parts of BSD. It is a very powerful editor, but nobody would say that it is easy to learn. There are two reasons to use vi:

If, on the other hand, you don't know vi, and you only work on systems whose software you can control, you probably shouldn't use vi. Emacs is much easier to learn, and it is more powerful than vi.

Emacs main menu


. 7.8. Emacs main menu

When running under X, Emacs displays its own window (v/' uses an xterm under these circumstances). As are sult, if you start Emacs from an xterm, you should use the & character to start it in the background:

$ emacs &

Figure 7-8 shows the resulting display. As you can see, the first thing that Emacs offers you is a tutorial. You should take it. You'll also notice the menu bars at the top. Although they look primitive compared to graphics toolbars, they offer all the functionality of graphics-oriented menus. In addition, they will tell you the keystrokes that you can use to invoke the same functions. Figure 7-9 gives an example of the Files menu.

There is a lot of documentation for Emacs, much of it on line. The complete Emacs handbook is available via the info mode of Emacs, which is described in the tutorial. If that's not enough, read Learning GNU Emacs, byDebra Cameron, Bill Rosenblatt and Eric Raymond.

Emacs files menu


. 7.9. Emacs files menu

Stopping the system

To stop X, press the key combination Ctrl-Alt-Backspace, which is deliberately chosen to resemble the key combination Ctrl-Alt-Delete used to reboot the machine. Ctrl-Alt-Backspace stops X and returns you to the virtual terminal in which you started it. If you run from xd/w, it reds plays a login screen.

To stop the system, use the shutdown program. To do so, you need to be a member of group operator.

By default, KDE uses the halt program. Only root can use this program, so you should reconfigure KDE to use shutdown. After this, you can shut down from KDE with the keystroke combination Ctrl-Alt-PageDown.

8. aking control

Users and groups; The super user; Processes; Daemons; Stopping processes; Timekeeping; Log files; Multiple processor support; PC Card devices; Emulating other systems; Emulating Linux; Emulating SCO UNIX; Emulating Microsoft Windows;

In Chapter 7 we saw the basics of working with FreeBSD. In this part of the book, we'll look at some more system-specific issues. This chapter discusses the following topics:

Users and groups

We've already looked at users in Chapter 7. In this chapter, well take a deeper look.

In traditional UNIX, information about users was kept in the file /etc/passwd. As the name suggests, it included the passwords, which were stored in encrypted form. Any user could read this file, but the encryption was strong enough that it wasn't practical to decrypt the passwords. Nowadays processors are much faster, and it's too easy to crack a password. As a result, FreeBSD keeps the real information in a file called /etc/mas-ter.passwd, and for performance reasons it also makes it available in database form in /etc/pwd.db and /etc/spwd.db. None of these file are user-readable. /etc/passwd remains for compatibility reasons: some third-party programs access it directly to get information about the environment in which they are running.

Choosing a user name

So what user name do you choose? User names are usually related to your real name and can be up to eight characters long. Like file names, they're case-sensitive. By convention, they are in all lower case, even when they represent real names. Typical ways to form a user name are:

I choose the last possibility, as we will see in the following discussion.

Adding users

We've already seen how to use sysinstall to create a user. It's not the only way. There are at least two other methods. One is the program adduser:

# adduser
Use option "-verbose" if you want see more warnings & questions or try to repair bugs.
Enter username [a-z0-9]:  yana
Enter full name []:  Yana Lehey
Enter shell bash csh date no sh [bash]:        accept the default
Uid [1000]:                                    accept the default
Enter login class:  default []:                accept the default
Login group yana [yana]:  home
Login group is "home". Invite  yana into other groups:  no
[no]:  wheel                                   to be able to use su
Enter password []:                             no echo
Enter password again []:                       no echo

Name:      yana
Password:  ****
Fullname:  Yana Lehey
Uid:       1000
Gid:       1001 (home)
Class:
Groups:    home wheel
HOME:      /home/yana
Shell:     /bin/bash
OK? (y/n)   [y]:                              accept the default
Added user "yana"
Add another user? (y/n)   [y]: n

An alternative way of adding or removing users is with the vipw program. This is a more typical UNIX-hackish approach: vipw starts your favorite editor and allows you to edit the contents of the file /etc/master.passwd. After you have finished; it checks the contents and rebuilds the password database. Figure 8-1 shows an example.

 Figure8-1: vipw display




. 8.1. Figure8-1: vipw display

You might be wondering why would you ever want to do things this way, and you might find it funny that most experienced UNIX administrators prefer it. The reason is that you get more of an overview than with a peephole approach that graphical environments give you, but of course you need to understand the format better. It's less confusing once you know that each line represents a single user, that the lines are divided into fields (which may be empty), and that each field is separated from the next by a colon (:). Table 8-1 describes the fields you see on the line on which the cursor is positioned. You can read more about the format of /etc/master.passwd in the man page passwd(5).

8.1. /etc/master.passwd format
FieldMeaning
yvonneUser name.
(gibberish)Encrypted password. When adding a new user, leave this field empty and add it later with the passwd program.
1005User number.
1001Group number.
(empty)Login class, which describes a number of parameters for the user. Well look at it in Chapter 29, on page 571. This field is not included in /etc/passwd
0Password change time. If non-0, it is the time in seconds after which the password must be changed. This field is not included in /etc/passwd.
0Account expiration time. If non-0, it is the time in seconds after which the user expires. This field is not included in /etc/passwd.
Yvonne LeheyThe so-called gecos field, which describes the user. This field is used by a number of programs, in particular mail readers, to extract the real name of the user.
/home/yvonneThe name of the home directory.
/bin/bashThe shell to be started when the user logs in.

The super user

FreeBSD has a number of privileged users for various administration functions. Some are just present to be the owners of particular files, while others, such as daemon and uucp, exist to run particular programs. One user stands above all others, however: root may do just about anything. The kernel gives root special privileges, and you need to become root to perform a number of functions, including adding other users. Make sure root has a password if there is any chance that other people can access your system (this is a must if you have any kind of dialup access). Apart from that, root is a user like any other, but to quote the man page su(1):

By default (unless the prompt is reset by a startup file) the super user prompt is set to # to remind one of its awesome power.

Becoming super user

Frequently when you're logged in normally, you want to do something that requires you to be root. You can log out and log in again as root, of course, but there's an easier way:

$ su        become super user
Password:   as usual, it doesn't echo
#           root prompt

To use su , you must be a member of the group wheel. Normally you do this when you add the user, but otherwise just put the name of the user at the end of the line in /etc/group:

wheel:*:0:root,grog    add the text in boldface
BSD treats su somewhat differently from System V. First, you need to be a member of the group wheel, and secondly BSD gives you more of the super user environment than System V. See the man page for further information.

Having a single root password is a security risk on a system where multiple people know the password. If one of them leaves the project, you need to change the password. An alternative is the sudo port (/usr/ports/security/sudo). It provides fine-grained access to root privileges, all based on the user's own password. Nobody needs to know the root password. If a user leaves, you just remove his account, and that cancels his access.

Adding or changing passwords

If your system has any connection with the outside world, it's a good idea to change your password from time to time. Do this with the passwd program. The input doesn't look very interesting:

$ passwd
Changing local password for yana.
Old password:                        doesn't echo
New password:                        doesn't echo
Retype new password:                 doesn't echo
passwd: rebuilding the database...
passwd: done

You have to enter the old password to make sure that some passer-by doesn't change it for you while you're away from your monitor, and you have to enter the new password twice to make sure that you don't mistype and lock yourself out of your account. If this does happen anyway, you can log in as root and change the password: root doesn't have to enter the old password, and it can change anybody's password. For example:

# passwd yana
Changing local password for yana.
New password:                     doesn't echo
Retype new password:              doesn't echo
passwd: rebuilding the database...
passwd: done

In this case, you specify the name of the user for whom you change the password.

If you are changing the root password, be careful: it's easy enough to lock your self out of the system if you mess things up, which could happen if, for example, you mistyped the password twice in the same way (don't laugh, it happens). If you're running X, open another window and use su to become root If you're running in character mode, select another virtual terminal and log in as root there. Only when you're sure you can still access root should you log out.

If you do manage to lose the root password, all may not be lost. Reboot the machine to single-user mode (see page 540), and enter:

# mount -u /           mount root file system read/write
# mount /usr           mount /usrfile system (if separate)
# passwd root          change the password for root
Enter new password:
Enter password again:
# ^D                   enter ctrl-D to continue with startup

If you have a separate /usr file system (the normal case), you need to mount it as well, since the passwd program is in the directory /usr/bin. Note that you should explicitly state the name root: in single-user mode, the system doesn't have the concept of user IDs.

Processes

As we have seen, UNIX is a multi-user, multi-tasking operating system. In particular, you can run a specific program more than once. We use the term process to refer to a particular instance of a running program. Each process is given a process ID more frequently referred to as PID a number between 0 and 99999 that uniquely identifies it. There are many things that you might like to know about the processes that are currently running, such as:

Your primary tool for investigating process behavior is the ps (process status) command. It has a large number of command options, and it can tell you a whole lot of things that you will only understand when you have investigated how the kernel works, but it can be very useful for a number of things. Here are some typical uses:

What processes do I have running?

After starting a large number of processes in a number of windows under X, you probably can't remember what is still running? Maybe processes that you thought had stopped are still running. To display a brief summary of the processes you have running, use the ps command with no options:

$ ps
PID  TT  STAT    TIME  COMMAND
187  p0  Is+  0:01.02  -bash (bash)
188  Pi  Ss   0:00.62  -bash (bash)
453  Pi  R+   0:00.03  ps

This display shows the following information:

What processes are running?

There are many more processes in the system than the list above shows. To show them all, use the a option to ps. To show daemons as well (see the next section for a definition of daemon ), use the x option. To show much more detail, use the u or l options. For example:

$ ps waux
USER    PID  %CPU  %MEM   VSZ   RSS  TT  STAT  STARTED        TIME  COMMAND
root     12  95.7   0.0     0    12  ??  RL     1Jan70  1406:43.85  (idle: cpu0)
root     11  95.1   0.0     0    12  ??  RL     1Jan70  1406:44.64  (idle: cpu1)
root      1   0.0   0.0   708    84  ??  ILs    1Jan70     0:09.10  /sbin/init
root     12   0.0   0.0     0    12  ??  WL     1Jan70    15:04.95  (swi1: net)
root     13   0.0   0.0     0    12  ??  WL     1Jan70    21:30.29  (swi6: tty:sio clock)
root     15   0.0   0.0     0    12  ??  DL     1Jan70     2:17.27  (random)
root     18   0.0   0.0     0    12  ??  WL     1Jan70     0:00.00  (swi3: cambio)
root     20   0.0   0.0     0    12  ??  WL     1Jan70     0:00.00  (irq11: ahc0 uhci0++)
root     21   0.0   0.0     0    12  ??  WL     1Jan70    39:00.32  (irq5: rl0)
root     22   0.0   0.0     0    12  ??  WL     1Jan70     7:12.92  (irq14: ata0)
root     23   0.0   0.0     0    12  ??  WL     1Jan70     0:47.99  (irq15: ata1)
root     24   0.0   0.0     0    12  ??  DL     1Jan70     0:00.08  (usb0)
root     25   0.0   0.0     0    12  ??  DL     1Jan70     0:00.00  (usbtask)
root     26   0.0   0.0     0    12  ??  DL     1Jan70     0:00.07  (usb1)
root     27   0.0   0.0     0    12  ??  DL     1Jan70     0:00.08  (usb2)
root    340   0.0   0.1  1124   280  ??  S     18Dec02    16:41.11  nfsd: server (nfsd)
root    375   0.0   0.0  1192    12  ??  Ss    18Dec02     0:01.70  /usr/sbin/lpd
daemon  408   0.0   0.0  1136   152  ??  Ss    18Dec02     0:11.41  /usr/sbin/rwhod
root    420   0.0   0.1  2648   308  ??  Ss    18Dec02     0:04.20  /usr/sbin/sshd
root    491   0.0   0.1  2432   368  ??  Ss    18Dec02     0:38.61  /usr/local/sbin/httpd
root    551   0.0   0.0  1336    12  ??  Ss    18Dec02     0:02.71  /usr/sbin/inetd -wW
root    562   0.0   0.0  1252   216  ??  Is    18Dec02     0:15.50  /usr/sbin/cron
root    572   0.0   0.0  1180     8  v2  IWs+   -          0:00.00  /usr/libexec/getty Pc
www     582   0.0   0.0  2432     8  ??  IW     -          0:00.00  /usr/local/sbin/httpd
grog    608   0.0   0.1  1316   720  v0  I     18Dec02     0:00.04  -bash (bash)
root   2600   0.0   0.0  1180     8  v1  IWs+   -          0:00.00  /usr/libexec/getty Pc
root  33069   0.0   0.3  5352  1716  ??  Ss    29Dec02     0:01.30  xterm -name xterm
grog  33081   0.0   0.1  1328   752  p8  Is+   29Dec02     0:00.09  /usr/local/bin/bash

This list is just an excerpt. Even on a freshly booted system, the real list of processes will be much larger, about 50 processes.

We've seen a number of these fields already. The others are:

In addition, a surprising number of processes don't have a controlling terminal. They are daemons and we'll look at them in the next section.

Daemons

A significant part of the work in a FreeBSD system is performed by daemons. A daemon is not just the BSD mascot described on page 21it's also a process that goes around in the background and does routine work such as sending mail (sendmail), handling incoming Internet connections (inetd), or starting jobs at particular times (cron).

To quote the Oxford English Dictionary: Demon Also daemon. ME [In form and in sense I, a. L. dcemon (med. L. demon)...] 1a. In ancient Greek mythology (): A supernatural being of a nature intermediate between that of gods and men, an inferior divinity, spirit, genius (including the souls of deceased persons, esp deified heros). Often written dcemon for distinction.

You can recognize daemons in a ps waux listing by the fact that they don't have a controlling terminalinstead you see the characters ?? Each daemon has a man page that describes what it does.

Normally, daemons are started when the system is booted and run until the system is stopped. If you stop one by accident, you can usually restart them. One exception is init, which is responsible for starting other processes. If you kill it, you effectively kill the system. Unlike traditional UNIX systems, FreeBSD does not allow init to be killed.

cron

One of the more useful daemons is cron named after Father Time.cron performs functions at specific times. For example, the system runs the script /etc/periodic/daily every day at 2:00 am, the script /etc/periodic/weekly every Saturday at 3:30 am, and the script /etc/periodic/monthly on the first day of every month at 5:30 am.

To tell cron to perform a function at a particular time, you need a file called a crontab. The system keeps the realcrontab where you can't get at it, but you can keep a copy. It's a good idea to call it crontab as well.

Let's look at the format of the default systemcrontab, located in /etc/crontab:

# /etc/crontab - root's crontab for FreeBSD
#
# $Id: crontab, v 1.10 1995/05/27 01:55:21 ache Exp $
# From: Id: crontab, v 1.6 1993/05/31 02:03:57 cgd Exp
#
SHELL=/bin/sh
PATH=/etc: /bin: /sbin: /usr/bin: /usr/sbin
HOME=/var/log
#
# minute  hour  mday  month  wday  who   command
#
*/5       *     *     *      *     root  /usr/libexec/atrun
#
# rotate log files every hour, if necessary
#0        *     *     *      *     root  /usr/bin/newsyslog
#
#do daily/weekly/monthly maintenance
0         2     *     *      *     root  /etc/daily 2>&1
30        3     *     *      6     root  /etc/weekly 2>&1
30        5     1     *      *     root  /etc/monthly 2>&1
#
#time zone change adjustment for wall cmos clock,
#See adjkerntz (8) for details.
1, 31     0-4   *     *      *     root  /sbin/adjkerntz -a

As usual, lines starting with # are comments. The others have seven fields. The first five fields specify the minute, the hour, the day of the month, the month, and the day of the week on which an action should be performed. The character * means "every." Thus, 0 2*** (for /etc/daily) means "0 minutes, 2 o'clock (on the 24 hour clock), every day of the month, every month, every weekday."

Field number six is special: it only exists in /etc/crontab, not in private crontabs. It specifies the user for whom the operation should be performed. When you write your own crontab file, don't use this field.

The remaining fields define the operation to be performed cron doesn't read your shell initialization files. In particular, this can mean that it won't find programs you expect it to find. It's a good idea to put in explicit PATH definitions, or specify an absolute pathname for the program, as is done in this example. Cron mails the output to you, so you should check root's mail from time to time.

To install or list a crontab, use the crontab program:

$ crontab crontab              install a crontab
$ crontab l                   list the contents of an installed crontab
#DO NOT EDIT THIS FILE - edit the master and reinstall.
#(crontab installed on Wed Jan   115:15:10 1997)
#(Cron version -- $Id: crontab.c,v 1.7 1996/12/17 00:55:12 pst Exp $)
00***/home/grog/Scripts/rotate-log

Processes in FreeBSD Release 5

Some of the processes in the example above are specific to FreeBSD Release 5:

top

Another tool for investigating system performance is top, which shows a number of performance criteria, including the status of the processes are using the most resources. Start it with the number of processes you want displayed. Example 8-1 gives an example.

$ top -S 10
last pid:    3992; load averages:    0.59, 0.17, 0.06 up 0+23:54:49 17:25:13
87 processes:    3running, 73 sleeping, 8 waiting, 3 lock
CPU states: 10.2% user, 0.0% nice, 18.8% system, 1.7% interrupt, 69.4% idle
Mem: 43M Active, 36M Inact, 31M Wired, 7460K Cache, 22M Buf, 2996K Free
Swap: 512M Total, 512M Free

 PID  USER  PRI  NICE    SIZE    RES  STATE   C   TIME    WCPU     CPU  COMMAND
  12  root  -16     0      0K    12K  RUN     0  23.7H  55.32%  55.32%  idle: cpu0
  11  root  -16     0      0K    12K  CPU1    1  23.7H  54.49%  54.49%  idle: cpu1
2854  grog   97     0   4940K  3932K  *Giant  1   0:04   3.88%   3.86%  xterm
  20  root  -64  -183      0K    12K  WAIT    1   0:08   0.83%   0.83%  irq14: ata0
2925  root   96     0    712K   608K  select  1   0:01   0.15%   0.15%  make
3193  grog   96     0   2220K  1304K  CPU0    0   0:01   0.15%   0.15%  top
3783  root   96     0    520K   416K  select  1   0:00   0.10%   0.05%  make
 167  root   96     0  13876K  2112K  select  0   1:02   0.00%   0.00%  xcpustate
  25  root  -68  -187      0K    12K  WAIT    0   0:28   0.00%   0.00%  irq9: xl0
 110  root   96     0   1528K   956K  select  1   0:26   0.00%   0.00%  ntpd

Figure8-2: top display

By default, the display is updated every two seconds and contains a lot of information about the system state:

Stopping processes

Sometimes you may find that you want to stop a currently running process. There are a number of ways to do this, but the easiest are:

FreeBSD also has a script called killall. As the name implies, it kills a group of processes, by name. If you find that you have, say, a whole lot of runaway sendmail processes, and you might save the day by writing:

# killall sendmail

As we'll see else where, you can also use killall to send a signal to a single process when you know that only one is present. For example, to cause inetd to re-read its configuration file, you could write:

# killall -1 inetd

Timekeeping

FreeBSD is a networking system, so keeping the correct time is more important than on a standalone system. Apart from the obvious problem of keeping the same time as other local systems, it's also important to keep time with systems in other time zones.

Internally, FreeBSD keeps the time as the number of seconds since the epoch, the beginning of recorded history: 00:00:00 UTC, 1 January 1970. UTC is the international base time zone, and means Universal Coordinated Time, despite the initials. It corresponds very closely, but not exactly, to Greenwich Mean Time (GMT), the local time in England in the winter. It would be inconvenient to keep all dates in UTC, so the system understands the concept of time zones. For example, in Walnut Creek, CA, the time zone in the winter is called PST (Pacific Standard Time), and in the summer it is PDT (Pacific Daylight Time) FreeBSD comes with a set of time zone description files in the directory hierarchy /usr/share/zoneinfo. We've already seen on page 95 that when you install the system, it stores information about the local time zone in the file /etc/localtime. If you move time zones, you should change the time zone, not the time, either by running the tzsetup program, or simply by copying the file. For example, if you travel with a laptop from Adelaide, South Australia, to San Francisco CA, you would do:

# cp /usr/share/zoneinfo/America/Los_Angeles /etc/localtime

When you get home again, you would do:

# cp /usr/share/zoneinfo/Australia/Adelaide /etc/localtime

At no time do you need to change the date or time directly.

Why Los_Angeles and not San_Francisco? The developers of the time zone package chose the largest city in the time zone. You need to have a certain understanding of the time zones to choose the correct one.

The TZ environment variable

An alternate means of describing the time zone is to set the environment variable TZ, which we looked at on page 128. You might use this form if you're connected to a remote system in a different time zone, or maybe just to find the time at some other place. For example, in Adelaide, SA I might find:

$ date
Sun Apr 14 13:31:15 CST 2002
$ TZ=America/Los_Angeles date
Sat Apr 13 21:01:15 PDT 2002

Set the TZ variable to the name of the time zone info file in the /usr/share/zoneinfo hierarchy. For example, the value of TZ for Berlin, Germany is Europe/Berlin in FreeBSD.

This is not the same as the usage of the TZ variable in UNIX System V. System V doesn't have the time zone definition files in /usr/share/zoneinfo, so the TZ variable tells it information about the time zone. If you were using System V in Berlin, you would set your TZ variable to MEZ1MSZ2, indicating time zone names and offsets from UTC.

Keeping the correct time

If you're connected to the Internet on a reasonably regular basis, there are a number of programs which can help you synchronize your time via the ntp (Network Time Protocol) service.

A number of systems around the world supply time information via the ntp service. Look at http://www.eecis.udel.edu/~mills/ntp/servers.html to find one near you.

Your choice of program depends on the nature of your connection to the Internet. If you're connected full time, you'll probably prefer ntpd, which keeps the system synchronized. Otherwise you can use ntpdate, which you can run as you feel like it.

ntpd

ntpd performs periodic queries to keep the system synchronized with a time server. There are many ways to run itsee the man page ntpd(8). In most cases, you can set up one system on the network to connect to an external time reference, and the other systems on the same Ethernet can get the time information from the first system.

To get the time from an external source and broadcast it to the other systems on the network, create a file /etc/ntp.conf with a content like this:

server     227.21.37.18        this address is invalid; check what 's near you
driftfile  /etc/ntp.drift
broadcast  223.147.37.255

The first line defines the server. The value in this example is invalid , so don't try to use it. It's important to get one near you: network delays can significantly impair the accuracy of the results. ntpd uses the file /etc/ntp.drift to record information about the (in) accuracy of the local system's clock. You only need the final line if you have other systems on the network which wait for a broadcast message. It specifies the broadcast address for the network and also tells ntpd to broadcast on this address.

After setting up this file, you just need to start ntpd:

# ntpd

To ensure that ntpd gets started every time you reboot make sure that you have the following lines in /etc/rc.conf:

ntpd_enable="YES"         # Run ntpd Network Time Protocol (or NO).

The comment on the first line is misleading: the value of ntpd_enable must be YES. You don't need any figs. You put exactly the same text in the /etc/rc.conf on the other machines, and simply omit the file /etc/ntp.conf. This causes ntpd on these machines to monitor broadcast messages.

In previous versions of FreeBSD, ntpd was called xntpd, so you may find things like xntpd_enable in your /etc/rc.conf. If you do, you'll have to change the name.

ntpdate

If you connect to the Internet infrequently, ntpd may become discouraged and not keep good time. In this case, it's better to use ntpdate. Simply run it when you want to set the time:

# ntpdate server

You can't use both ntpdate and ntpd at the same time: they both use the same port. Ntpd takes quite some time to synchronize, and if the time is wildly out, it won't even try, so it's often a good idea to run ntpdate on startup and then start ntpd manually.

Log files

Various components of FreeBSD report problems or items of interest as they happen. For example, there can always be problems with mail delivery, so a mail server should keep some kind of record of what it has been doing. If hardware problems occur, the kernel should report them. If somebody tries to break into the machine, the components affected should report the fact.

FreeBSD has a generalized system for logging such events. The syslogd daemon takes messages from multiple sources and writes them to multiple destinations, usually log files in the directory /var/log. You can change this behavior by modifying the file /etc/syslog.conf. See syslog.conf(5) for further details. In addition to syslogd, other programs write directly to files in this directory. The following files are of interest:

A number of the more important log files are kept through several cycles. As the example above shows, cron runs the newsyslog command every hour new syslog checks the size of the files, and if they are larger than a certain size, it renames the old ones by giving them a numerical extension one higher than the current one, then renames the base file with an extension .0 and compresses it. The result looks like this:

-rw-r--r--  1  root  wheel  31773  Jan   5  13:01 messages
-rw-r--r--  1  root  wheel   8014  Jan   2  01:00 messages.0.bz2
-rw-r--r--  1  root  wheel  10087  Dec  15  14:00 messages.1.bz2
-rw-r--r--  1  root  wheel   9940  Dec   3  17:00 messages.2.bz2
-rw-r--r--  1  root  wheel   9886  Nov  16  11:00 messages.3.bz2
-rw-r--r--  1  root  wheel   9106  Nov   5  18:00 messages.4.bz2
-rw-r--r--  1  root  wheel   9545  Oct  15  17:00 messages.5.bz2

newsyslog has a configuration file /etc/newsyslog.conf, which we discuss on page 572.

Multiple processor support

FreeBSD Release 5 can support most current Intel and AMD multiprocessor motherboards with the ia32 architecture. It also supports some Alpha, SPARC64 and Intel ia64 motherboards. Documentation on SMP support is currently rather scanty, but you can find some information at http://www.freebsd.org/~fsmp/SMP/SMP.html.

The GENERIC kernel does not support SMP, so you must build a new kernel before you can use more than one processor. The configuration file /usr/src/sys/i386/conf/GENERIC contain the following commented-out entries:

# To make an SMP kernel, the next two are needed
#options  SMP          #Symmetric Multiprocessor Kernel
#options  APIC_IO      #Symmetric (APIC) I/O

For other platforms, you don't need APIC_IO. See Chapter 33 for information on how to build a new kernel.

PC Card devices

As we have already seen, PC Card devices are special because they can be hot-plugged. They are also intended to be recognized automatically. Starting with Release 5, FreeBSD recognizes card insertion and removal in the kernel and invokes the appropriate driver to handle the event. When you insert a card you will see something like this on the system console:

ata2 at port 0x140-0x14f irq 11 function 0 config 1 on pccard0
ad4: 7MB <LEXAR ATA FLASH>  [251/2/32]  at ata2-master BIOSPIO

This is a compact fash memory card, which the system sees as an ATA disk. The kernel has created the necessary structures, but it can't know how to mount the device, for example. We'll look at what we can do about this in the next section.

devd: The device daemon

The device daemon, devd, provides a way to run user land programs when certain kernel events happen. It is intended to handle userland configuration of PC Card devices such as Ethernet cards, which it can do automatically. We'll look at this automatic usage on page 304.

devd reads the kernel event information from the device /dev/devctl and processes it according to rules specified in the configuration file /etc/devd.conf, which is installed with the system. If you want to use it for other devices, you must modify /etc/devd.conf. This file contains a number of sections, referred to as statements in the man page:

So what does devd do when we insert the compact flash card? By default, nothing. The ATA driver recognizes and configures the card. It would be nice to get devd to mount it as well. That's relatively simple:

After this, the file system will be automatically mounted when you insert the card.

Removing PC Card devices

The next thing we'd like to do is to unmount the file system when you remove the flash card. Unfortunately, that isn't possible. Unmounting can involve data transfer, so you have to do it before you remove the card. If you forget, and remove the card without unmounting, the system may panic next time you try to access the card.

After unmounting, you can remove the card. On the console you'll see something like:

ad4: removed from configuration
ad4: no status, reselecting device
ad4: timeout sending command=e7 s=ff e=04
ad4: flushing cache on detach failed
ata2: detached

Alternate PC Card code

The PC Card implementation described here, called NEWCARD, is new in FreeBSD Release 5. At the time of writing, the older implementation, called OZDCARD, is still included in the system. It's possible that you might have an older card that is supported by OLDCARD but not by NEWCARD. In that case, you will need to build a kernel with OLDCARD support. Check the NOTES files in /usr/src/sys/conf and /usr/src/sys/arch/conf, where arch is the architecture of your system, and the man pages pccard and pccard.conf.

Configuring PC Card devices at startup

A number of entries in /etc/rc.conf relate to the use of PC Card devices, but nearly all of them are for OLDCARD. You only need one for NEWCARD:

devd_enable="YES"

This starts devd at system startup.

Emulating other systems

A large number of operating systems run on Intel hardware, and there is a lot of software that is available for these other operating systems, but not for FreeBSD.

Emulators and simulators

There are a number of ways to execute software written for a different platform. The most popular are:

Simulation is a process where a program executes the functions that are normally performed by the native instruction set of another machine. They simulate the low-level instructions of the target machine, so simulators don't have to run on the same kind of machine as the code that they execute. A good example is the port emulators/p11, which simulates a PDP-11 minicomputer, the machine for which most early versions of UNIX were written.

FreeBSD can emulate many other systems to a point where applications written for these systems will run under FreeBSD. Most of the emulators are in the Ports Collection in the directory /usr/ports/emulators.

In a number of cases, the emulation support is in an experimental stage. Here's an overview:

Emulating Linux

Linux is a UNIX-like operating system that in many ways is very similar to FreeBSD. We discussed it on page 10. Although it looks very UNIX-like, many of the internal kernel interfaces are different from those of FreeBSD or other UNIX-based systems. The Linux compatibility package handles these differences, and most Linux software will run on FreeBSD. Most of the exceptions use specific drivers that don't run on FreeBSD, though there is a considerable effort to minimize even this category.

To install the Linux emulator, you must:

Running the Linux emulator

Normally you load the Linux emulator when you boot the system. Put the following line in your /etc/rc.conf:

linux_enable="YES"

If you don't want to do this for some reason, you can start it from the command line:

# kldload linux

You don't interact directly with the emulator module: it's just there to supply kernel functionality, so you get a new prompt immediately when you start it.

linux is a kld, so it doesn't show up in a ps listing. To check whether it is loaded, use

kldstat :

$ kldstat
Id  Refs     Address  Size    Name
 1     5  0xc0100000  1d08b0  kernel
 2     2  0xc120d000  a000    ibcs2.ko
 3     1  0xc121b000  3000    ibcs2_coff.ko
 5     1  0xc1771000  e000    linux.ko

This listing shows that the SCO UNIX emulation (ibcs2) has also been loaded.

The Linux emulator and many Linux programs are located in the directory hierarchy /usr/compat/linux. You won't normally need to access them directly, but if you get a Linux program that includes libraries destined for /lib, you will need to manually place them in /usr/compat/linux/lib. Be very careful not to replace any files in the /usr/lib hierarchy with Linux libraries; this would make it impossible to run FreeBSD programs that depend on them, and it's frequently very difficult to recover from such problems. Note that FreeBSD does not have a directory /lib, so the danger is relatively minor.

Linux procfs

Linux systems have a file system called procfs, or Process File System, which contains information used by many programs. FreeBSD also has a procfs, but it is completely different. To be able to run Linux programs which refer to procfs, place the following entry in your /etc/fstab file:

Linproc  /compat/linux/proc  linprocfs  rw  0  0

Problems executing Linux binaries

One of the problems with the ELF format used by older Linux binaries is that they may contain no information to identify them as Linux binaries. They might equally well be BSD/OS or UnixWare binaries. That's normally not a problem, unless there are library conflicts: the system can't decide which shared library to use. If you have this kind of binary, you must brand the executable using the program brandelf. For example, to brand the Star Office program swriter3, you would enter:

# brandelf -t Linux /usr/local/StarOffice-3.1/linux-x86/bin/swriter3

This example deliberately shows a very old version of Star Office: it's not clear that there are any modern binaries that cause such problems.

Emulating SCO UNIX

SCO UNIX, also known as SCO OpenDesktop and SCO Open Server, is based on UNIX System V3.2. This particular version of UNIX was current in the late 1980s. It uses an obsolete binary format called COFF (Common Object File Format).

Like Linux support, SCO support for FreeBSD is supplied as a loadable kernel module. It's not called sco, though: a number of older System V.3.2 systems, including Interactive UNIX, also support the ibcs2 1)standard. As a result, the kld is called ibcs2.

Run ibcs2 support like Linux support: start it manually, or modify /etc/rc. conf to start it automatically at boot up:

ibcs2_enable="YES"  # Ibcs2  (SCO) emulation loaded at startup (or NO).

Alternatively, load the kld:

# kldload ibcs2

One problem with SCO emulation is the SCO shared libraries. These are required to execute many SCO executables, and they're not supplied with the emulator. They are supplied with SCO's operating systems. Check the SCO license to determine whether you are allowed to use them on FreeBSD. You may also be eligible for a free SCO licensesee the SCO web site for further details.

Emulating Microsoft Windows

The wine project has been working for some time to provide an emulation of Microsoft's Windows range of execution environments. It's changing continually, so there's little point describing it here. You can find up-to-date information at http://www.winehq.com/about/, and you can install it from the port emulators/wine . Be prepared for a fair amount of work.

Accessing Microsoft files

Often you're not as interested in running Microsoft applications as decoding their proprietary formats. For example, you might get a mail message with an attachment described only as

[-- Attachment #2: FreeBSD.doc
[-- Type: application/octet-stream, Encoding: x-unknown, Size: 15K
[-- application/octet-stream is unsupported (use 'v' to view this part)

This attachment has an unspecific MIME type2), but you might guess that it is Microsoft Word format because the file name ends in .doc. That doesnt make it any more legible. To read it, you need something that understands the format. A good choice is OpenOffice.org, a clone of Microsofts Office product. Install from the Ports Collection (/usr/ports/editors/openoffice).

OpenOffice.org is not a good example of the UNIX way. It breaks a number of conventions, and in general it's a lot more difficult to use than normal FreeBSD tools. Its only real advantage is that you can process Microsoft document formats.

9. The Ports Collection

How to install a package; Building a port; Package documentation; Getting binary-only software; Maintaining ports; Upgrading ports; Controlling installed ports; Submitting a new port;

The Internet is full of free software that is normally distributed in source form. That can be a problem in it self: the way from the source archive that you get free from the Internet to the finished, installed, running program on your machinenormally called porting, can be a long and frustrating one. See my book Porting UNIX Software, for more details of the porting process.

To get a software package up and running on your system, you need to go through most of these steps:

  1. Get the source files on your machine. They are usually contained in an archive, a file containing a number of other files. Archives used for the ports collection are generally gzipped tar, files, packaged with tar, and compressed with gzip, but other formats are also possible. What ever the format, you'll typically use ftp, to get them to your machine.
  2. Unpack the archive source tree, in this case using gunzip, and tar,
  3. Configure the package. Most packages include shell scripts to do this. Configuration performs a threefold adaptation of the package:
    1. It adapts it to the system hardware.
    2. It adapts it to the software environment you're running (in this case, FreeBSD).
    3. It adapts it to your personal preferences.
  4. Build the package. For most packages, this involves compiling the source files and creating executable. The main tool for this purpose is make, which uses a set of rules, traditionally stored in a file called Makefile, to decide how to build the package. There is nearly always a Makefile, in the sources, but the Ports Collection includes a second one that controls the build at a higher level.
  5. Install the package. This involves mainly copying the executable, configuration files and documentation created by a build to the correct place in the directory hierarchy.
  6. Configure the installed software. This is similar in concept to package configuration, except that it occurs in the run-time environment. The package configuration may perform all the necessary configuration for you.

These are a lot of steps, and you'll often find they're laid through a minefield: one false move and everything blows up. To make porting and installing software easier, the FreeBSD team created a framework called the Ports Collection, which makes it trivial to perform these steps. It also provides a method of packaging and installing the resultant ported software, called packages, The CD-ROM edition of FreeBSD includes a large number of pre-built packages that can be installed directly.

In this chapter, we'll consider the following points as they relate to the FreeBSD ports collection:

How to install a package

In FreeBSD parlance, a package is simply a special archive that contains those files (usually executable binary files) that are installed when you build and install a port. Effectively it's a snapshot of the port build process that we saw above, taken after step 4 has completed. Compared to the full-blown port, packages are much faster to installit's usually a matter of seconds. On the other hand, they don't give you the choice of configuration that the complete port does. The distribution CD-ROMs contain a directory packages, with a large number of pre-compiled software packages. Alternatively, you can find FreeBSD packages on many servers on the Internetcheck the online handbook for some places to look.

To help maintain an overview, both ports and packages are divided into categories. They are stored in directories named after the category. See the file /usr/ports/INDEX, for a list. For example, emacs, under editors, is currently in the file packages/editors/emacs-21.2.tgz, though this name will change with updated versions of emacs. For, the latest version of the packages only, you'll find another copy without the extension in packages/Latest/emacs.tgz. To install it, you enter:

pkg_add /cdrom/packages/Latest/emacs.tgz

Alternatively, you can install packages from the sysinstall, final configuration menu shown in Figure 6-1.

Building a port

The more general way to install third-party software is with a port, The FreeBSD project uses the term port to describe the additional files needed to adapt a package to build under FreeBSD. It does not include the source code itself, though the CD-ROM distribution includes many code archives in the directory /ports/distfiles, spread over several of the CD-ROMs.

Before you get started with the ports, you need to install the port information on your system. Normally this will be in /usr/ports. This directory tree is frequently called the Ports Tree. There are a number of ways to install them.

Installing ports during system installation

The simplest way to install the Ports Collection is when you install the system. When you choose the components to install, sysinstall, offers to install the Ports Collection for you as well.

Installing ports from the first CD-ROM

The file ports/ports.tgz, on the first CD-ROM is a tar, archive containing all the ports. If you didn't install it during system installation, use the following method to install the complete collection (about 200 MB). Make sure your CD-ROM is mounted (in this example on /cdrom), and enter:

#  cd /usr
#  tar xzvf /cdrom/ports/ports.tgz

If you only want to extract a single package, say inn, which is in the categorynews, enter :

#  cd /usr
#  tar xzvf /cdrom/ports/ports.tgz ports/news/inn

It takes a surprisingly long time to install the ports; although there isn't much data in the archive, there are about 250,000 files in it, and creating that many files takes a lot of disk I/O.

Installing ports from the live file system CD-ROM

Alternatively, the files are also on the live file system CD-ROM. This is not much of an advantage for installation, but you may find it convenient to browse through the source trees in the directory ports, on the CD-ROM. Let's assume you have found a directory /cdram/ports/graphics/hp scan, on the CD-ROM, and it is your current working directory. You can move the data across with the following:

#  cd /cdram/ports/graphics
#  mkdir -p /usr/ports/graphics
#  tar cf - . | (cd /usr/ports/graphics; tar xvf -)

Getting new ports

What happens when a new version of a port comes out? For example, you've been using Emacs Version 20 forever, and now Version 21.2 becomes available? It's brand new, so it's obviously not on your CD-ROM.

One way to get the port is via ftp. This used to be quite convenient: you could download a tar ball directly and extract it locally. That is unfortunately no longer possible: currently you must download files a directory at a time. If you're following the Ports Collection at all closely, you should consider using cvsup, which can keep your sources up to date automatically. See Chapter 31, page 585, for more details.

All ports are kept in subdirectories of the URL ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/. This directory has the following contents:

drwxr-xr-x    6  1006  1006     512  Jun   8  13:18  alpha
drwxr-xr-x  209  1006  1006  401408  May  28  14:08  dist files
drwxr-xr-x    6  1006  1006    1536  May  28  17:53  i386
drwxr-xr-x    3  1006  1006     512  Apr   6  13:45  ia64
drwxr-xr-x   83  1006  1006    3072  May  20  15:35  local-dist files
lrwxrwxrwx    1  root  wheel     13  Jun   1   2001  packages -> i386/packages
lrwxrwxrwx    1  root  wheel     24  Jun   1   2001  ports -../FreeBSD-current/ports
lrwxrwxrwx    1  root  wheel      5  Jun   1   2001  ports-current -> ports
lrwxrwxrwx    1  root  wheel      5  Jun   1   2001  ports-stable -> ports
drwxr-xr-x    4  1006  1006     512  Apr   9  10:37  sparc64

The directories alpha, i386, ia64, and sparc64, contain packages (not ports) for the corresponding architecture. dist files contains a large number of the original sources for the third-party packages; it's intended as a ''last resort'' location if you can't find them at other locations.

The directory local-dist files, is used by people working on the Ports Collection; you don't normally need anything from these directories. The important directories for you are ports, ports-current, and ports-stable. Currently these are really all the same directory, but things may not remain like that.

Getting back to your emacs, port: you would find it in the directory /pub/Free-BSD/ports/ports/editors/ . Note the final / in that directory name: if you leave it out, ftp prints an error message and exits. Here's what might happen:

$ ftp ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/ports/editors/
Connected to ftp.beastie.tdk.net.
220 ftp.beastie.tdk.net FTP server (Version 6.00LS) ready.
331 Guest login ok, send your email address as password.
230- The FreeBSD mirror at Tele Danmark Internet.
...much blurb omitted
250 CWD command successful.
250 CWD command successful.
ftp> ls
229 Entering Extended Passive Mode (|||55649|)
150 Opening ASCII mode data connection for '/bin/ls'.
total 704

drwxr-xr-x  3  1006  1006  512  May  20  10:07  emacs
drwxr-xr-x  4  1006  1006  512  May  20  10:08  emacs20
drwxr-xr-x  4  1006  1006  512  May  20  10:08  emacs20-dl
drwxr-xr-x  4  1006  1006  512  May  20  10:08  emacs20-mule-devel
drwxr-xr-x  3  1006  1006  512  May  20  10:08  emacs21
drwxr-xr-x  2  1006  1006  512  May  20  10:08  eshell-emacs20

This shows that your files will be in the directory emacs21. You can get them with the ftp mget, command:

ftp> mget emacs21
mget emacs21/files [anpqy?]? a           answer a for all files
Prompting off for duration of mget.
ftp: local: emacs21/files: No such file or directory
ftp: local: emacs21/Makefile: No such file or directory
(etc)

This happens because you need to create the destination directory manually. Try again:

ftp> !mkdir emacs21                      create the local directory
ftp> mget emacs21
mget emacs21/files [anpqy?]? a
Prompting off for duration of mget.
229 Entering Extended Passive Mode (|||57074|)
550 emacs21/files: not a plain file.
229 Entering Extended Passive Mode (|||57085|)
150 Opening BINARY mode data connection for 'emacs21/Makefile'   (2185 bytes).
100%  |*************************************|   2185   2.34 MB/s   00:00 ETA
226 Transfer complete.
(etc)

You get one of these for each file transferred. But note the error message: not a plain file. emacs21/files, is a directory, so we need to get it separately:

ftp> !mkdir emacs21/files
ftp> mget emacs21/files
mget emacs21/files/patch-lib-src:Makefile.in [anpqy?]? a
Prompting off for duration of mget.
229 Entering Extended Passive Mode (|||57258|)
150 Opening BINARY mode data connection for 'emacs21/files/patch-lib-src: Make file. in '(908 bytes).
100%  |*************************************|      908  1.64 MB/s       00:00 ETA
226 Transfer complete.
(etc)

Note that the ftp command specifies the URL of the directory. It must have a trailing /, otherwise ftp will complain. This form is supported by FreeBSD ftp but many other ftp clients will require you to do it in two steps:

# ftp ftp.EreeBSD.org
Connected to ftp.beastie.tdk.net.
(etc)
ftp> cd /pub/FreeBSD/ports/ports/editors
250 CWD command successful.

What's in that port?

One problem with the Ports Collection is the sheer number. It can be difficult just to find out what they're supposed to do. If you build all the ports, you'll be busy for weeks, and there's no way you could read all the documentation in one lifetime. Where can you get an overview? Here are some suggestions. In each case, you should have the directory /usr/ports, as your current working directory.

Getting the source archive

You'll see from the above example that there are not many files in the port. Most of the files required to build the software are in the original source code archive (the "tar ball"), but that's not part of the port.

There are a number of places from which you can get the sources. If you have a CD-ROM set, many of them are scattered over the CD-ROMs, in the directory /cdrom/ports/distfiles, on each CD-ROM. The Ports Collection Make files look for them in this directory (another good reason to mount your CD-ROM on /cdrom), and also in /usr/ports/distfiles.

If you don't have the source tar ball, that's not a problem. Part of the function of the Ports Collection is to go out on the Net and get them for you. This is completely automatic: you just type make, and the build process gets the source archive for you and builds it. Of course, you must be connected to the Internet for this to work.

If you mount your CD-ROM elsewhere (maybe because you have more than one CD-ROM drive, and so you have to mount the CD-ROM on, say, /cd4), the Make files will not find the distribution files and will try to load the files from the Internet. One way to solve this problem is to create a symbolic link from /cd4/ports/distfiles, to /usr/ports/distfiles. The trouble with this approach is that you will then no longer be able to load new distribution files into /usr/ports/distfiles, because it will be on CD-ROM. Instead, do:

#  cd /cd4/ports/distfiles
#  mkdir -p /usr/ports/distfiles       make sure you have a distfiles directory
#  for i in *; do
>  ln -s /cd4/ports/distfiles/$i /usr/ports/distfiles/$i
>  done

If you're using csh, or tcsh, enter:

# cd /cd4/ports/distfiles
# mkdir -p /usr/ports/distfiles       make sure you have a distfiles directory
# foreach i (*)
?  ln -s /cd4/ports/distfiles/$i /usr/ports/distfiles/$i
? end

This creates a symbolic link to each distribution file, but if the file for a specific port isn't there, the Ports Collection can fetch it and store it in the directory.

Building the port

Once you have the skeleton files for the port, the rest is simple. Just enter:

#  cd /usr/ports/editors/emacs21
#  make
#  make install
====> To enable menubar fontset support, define WITH_MENUBAR_FONTSET
>> emacs-21.2.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
>> Attempting to fetch from ftp://ftp.gnu.org/gnu/emacs/.
===>   Extracting for emacs-21.2_1
>> Checksum OK for emacs-21.2.tar.gz.
===>     emacs-21.2_1 depends on executable: gmake - found
===>     emacs-21.2_1 depends on executable: autoconf213 - not found
===>      Verifying install for autoconf213 in /usr/ports/devel/autoconf213
===>   Extracting for autoconf213-2.13.000227_2
>> Checksum OK for autoconf-000227.tar.bz2.
===>     autoconf213-2.13.000227_2 depends on executable: gm4 - not found
===>      Verifying install for gm4 in /usr/ports/devel/m4
===>   Extracting for m4-1.4_1
>> Checksum OK for m4-1.4.tar.gz.
===>   Patching for m4-1.4_1
===>   Applying FreeBSD patches for m4-1.4_1
===>   Configuring for m4-1.4_1
creating cache ./config.cache
checking for mawk... no
(etc)

It's a good idea to perform the make, step first: make install, does not always build the package.

Port dependencies

Sometimes, it's not enough to build a single port. Many ports depend on other ports. If you have the complete, up-to-date ports tree installed on your systcode, the Ports Collection will take care of this for you: it will check if the other port is installed, and if it isn't, it will install it for you. For example, tkdesk, depends on tk, tk depends on tcl. If you don't have any of them installed, and you try to build tkdesk, it will recursively install tk and tcl for you.

Package documentation

Once you have installed your port, you'll want to use it. In almost every case, that requires documentation. Most packages have documentation, but unfortunately it's not always obvious where it is. In some cases, the port doesn't install all the documentation.

More generally, there are the following possibilities:

Getting binary-only software

A lot of software doesn't need to be ported. For example, if you want Netscape , you can just download it from ftp://ftp.netscape.com. But how, do you install it? Netscape's installation procedures are getting better, but they still leave something to be desired.

The answer's simple: take the port! Although Netscape comes only in binary form, the port handles getting the correct version and installing it for you. Another advantage to using a port instead of installing the package manually is that the port installs the software as a FreeBSD package, which makes it much easier to remove the software later.

This method can be used to install some other software as well, for example Star Office. The moral is simple: always check the Ports Collection before getting a software package from the Net.

Maintaining ports

Once you install a port, you might consider that to be the end of the story. That's seldom the case. For example:

Upgrading ports

From time to time, new versions of software will appear. There are a number of approaches to upgrading:

Using portupgrade

Portupgrade iswhat else?a port. Install it in the usual manner:

# cd /usr/ports/sysutils/portupgrade
# make install

Before you can perform the upgrade, you should first back up /var/db/pkg, then build a ports database with pkgdb. A typical build might look like this:

# cd /var/db
# tar czvf db.pkg.tar.gz pkg/
# pkgdb -F
[Updating the pkgdb <format:bdb1_btree> in /var/db/pkg ... - 181 packages
found (-5 +92)  (...)...............done]
Checking the origin of AbiWord-1.0.3
Checking the origin of ImageMagick-5.5.1.1
Checking the origin of ORBit-0.5.17
...
Checking the origin of xv-3.10a_3
Checking the origin of zip-2.3_1
Checking for origin duplicates
Checking AbiWord-1.0.3
Checking ImageMagick-5.5.1.1
Stale dependency: ImageMagick-5.5.1.1 -> ghostscript-gnu-7.05_3:
ghostscript-gnu-6.52_4 (score:64%) ?  ([y]es/[n]o/[a]ll)   [no] y
Fixed. (-> ghostscript-gnu-6.52_4)
Checking ORBit-0.5.17 Checking XFree86-4.2.0_1,1
Checking bonobo-1.0.21_1
Stale dependency: bonobo-1.0.21_1 -> ghostscript-gnu-7.05_3:
ghostscript-gnu-6.52_4 ? ([y]es/[n]o/[a]ll)   [yes]        Enter pressed
Fixed. (-> ghostscript-gnu-6.52_4)
Checking cdrtools-1.11.a28
...
Checking xv-3.10a_3
Checking zip-2.3_1
Regenerating +REQUIRED_BY files
Checking for cyclic dependencies

In this example, the port ghostscript-gnu-7.05 3 had been replaced by the earlier version ghostscript-gnu-6.52_4, since ghostscript Release 7 has some annoying bugs. The dialogue shows how pkgdb, recognized the discrepancy, and how it recovered from it.

Now you can start the upgrade. To upgrade a specific port, simply specify its base name, without the version number. This example uses the -v, option to show additional information:

# portupgrade -v bison
---> Upgrade of devel/bison started at: Mon, 04 Nov 2002 13:20:52 +1030
---> Upgrading 'bison-1.35_1' to 'bison-1.75' (devel/bison)
---> Build of devel/bison started at: Mon, 04 Nov 2002 13:20:52 +1030
... normal port build output
===> Registering installation for bison-1.75
make clean issued by portupgrade
===> Cleaning for libiconv-1.8_2
===> Cleaning for gettext-0.11.5_1
...
---> Removing the temporary backup files
---> Installation of devel/bison ended at: Mon, 04 Nov 2002 13:23:00 +1030 (consume
d00:00:06)
---> Removing the obsoleted dependencies
---> Cleaning out obsolete shared libraries
---> Upgrade of devel/bison ended at: Mon, 04 Nov 2002 13:23:01 +1030 (consumed 00:
02:08)
---> Reporting the results ( + :succeeded / -:ignored / *:skipped / !:failed)
+devel/bison (bison-1.35_1)

If the port is already up to date, you'll see something like this:

# portupgrade -v perl-5.8.0_3
** No need to upgrade 'perl-5.8.0_3'   (>= perl-5.8.0_3).  (specify -f to force)
---> Reporting the results ( + :succeeded / -:ignored / *:skipped / !:failed)
-lang/perl5.8 (perl-5.8.0_3)

To upgrade all ports, use the command:

# portupgrade -a

Controlling installed ports

We've already seen the program pkg_add, when installing pre-compiled packages. There are a number of other pkg_ programs that can help you maintain installed ports, whether they have been installed by pkg_add, or by make install from the Ports Collection:

Keeping track of updates

The best way to find out about updates is to subscribe to the FreeBSD-ports mailing list. That way, you will get notification every time something changes. If you're tracking the ports tree with CVSup, you also get the updates to the ports tree automatically. Otherwise you will have to download the port. In either case, to update your installed port, just repeat the build.

Submitting a new port

The Ports Collection is constantly growing. Hardly a day goes by without a new port being added to the list. Maybe you want to submit the next one? If you have something interesting that isn't already in the Ports Collection, you can find instructions on how to prepare the port in the FreeBSD Porter's Handbook. The latest version is available on the FreeBSD web site, but you'll also find it on your system as /usr/share/doc/en/porters-handbook/index.html.

10. File systems and devices

File permissions; Mandatory Access; Control Links; Directory hierarchy; File system types; Mounting file systems; FreeBSD devices ; Virtual terminals;

One of the most revolutionary concepts of the UNIX operating system was its file system the way in which it stores data. Although most other operating systems have copied it since then, including Microsoft's platforms, none have come close to the elegance with which it is implemented. Many aspects of the file system are not immediately obvious, some of them not even to seasoned UNIX users.

We've already looked at file naming conventions on page 125. In the next section, we'll look at the file system access, structure and hierarchy, and on page 195 we'll look at how the file system treats hardware devices as files.

File permissions

A UNIX system may potentially be used by many people, so UNIX includes a method of protecting data from access by unauthorized persons. Every file has three items of information associated with it that describe who can access it in what manner:

For example, you might have a program that accesses private data, and you want to be sure that only you can execute it. You do this by setting the permissions so that only the owner can execute it. Or you might have a text document in development, and you want to be sure that you are the only person who can change it. On the other hand, the people who work with you have a need to be able to refer to the document. You set the permissions so that only the owner can write it, that the owner and group can read it, and, because it's not ready for publication yet, you don't allow anybody else to access it.

Traditionally, the permissions are represented by three groups of rwx: r stands for read permission, w stands for write permission, and x stands for execute permission. The three groups represent the permissions for the owner, the group and others respectively. If the permission is not granted, it is represented by a hyphen (-). Thus, the permissions for the program I discussed above would be r-x------ (I can read and execute the program, and nobody else can do anything with it). The permissions for the draft document would be rw-r----- (I can read and write, the group can read, and others can't access it).

Typical FreeBSD file access permissions are rwxr-xr-x for programs and rw-r--r-- for other systcode files. In some cases, however, you'll find that other permissions are required. For example, the file ~/.rhosts, which is used by some network programs for user validation, may contain the user's password in legible form. To help ensure that other people don't read it, the network programs refuse to read it unless its permissions are rw-------. The vast majority of system problems in UNIX can be traced to in correct permissions, so you should pay particular attention to them.

Apart from these access permissions, executable can also have two bits set to specify the access permissions of the process when it is run. If the setuid (set user ID) bit is set, the process always runs as if it had been started by its owner. If the setgid (set group ID) bit is set, it runs as if it had been started by its group. This is frequently used to start system programs that need to access resources that the user may not access directly. We'll see an example of this with the ps command on page 185. ls represents the setuid bit by setting the third letter of the permissions string to s instead of x; similarly, It represents the setgid bit by setting the sixth letter of the permissions string to s instead of x.

In addition to this access information, the permissions contain a character that describes what kind of file it represents. The first letter may be a - (hyphen), which designates a regular file, the letter d for directory, or the letters b or c for a device node. We'll look at device nodes in Chapter 11. There are also a number of other letters that are less used. See the man page ls(1) for a full list.

To list files and show the permissions, use the ls command with the -l option:

$ ls -l
total 2429
-rw-rw-r--  1 grog    wheel    28204 Jan  4 14:17 %backup%?
drwxrwxr-x  3 grog    wheel      512 Oct 11 15:26 2.1.0-951005-SNAP
drwx------  4 grog    wheel      512 Nov 25 17:23 Mail
-rw-rw-r--  1 grog    wheel      149 Dec  4 14:18 Makefile
-rw-rw-r--  1 grog    wheel      108 Dec  4 12:36 Makefile.bak
-rw-rw-r--  1 grog    wheel      108 Dec  4 12:36 Makefile?
-rw-rw-r--  1 grog    wheel        0 Dec  4 12:36 depend
-rw-rw-r--  1 daemon  wheel  1474560 Dec 14 17:03 deppert.floppy
-rwxr-xr-x  1 grog    wheel      100 Dec 19 15:24 doio
-rwxrwxr-x  1 grog    wheel      204 Dec 19 15:25 doiovm
-rwxrwxr-x  1 grog    wheel      204 Dec 19 15:16 doiovm~
-rwxr-xr-x  1 grog    wheel      115 Dec 26 08:42 dovm
-rwxr-xr-x  1 grog    wheel      114 Dec 19 15:30 dovm~
drwxr-xr-x  2 grog    wheel      512 Oct 16  1994 emacs
drwxrwxrwx  2 grog    wheel      512 Jan  3 14:07 letters

This format shows the following information:

A couple of the permissions are of interest. The directories all have the x (execute) permission bit set. This enables accessing (i.e. opening) files in the directorythat's the way the term execute is defined for a directory. If l reset the execute permission, I can still list the names of the files, but I can't access them.

I am the only person who can access the directory Mail This is the normal permission for a mail directory.

Changing file permissions and owners

Often enough, you may want to change file permissions or owners. UNIX supplies three programs to do this:

10.1. chmod permission codes
SpecificationEffect
go-wDeny write permission to group and others
=rw,+XSet the read and write permissions to the usual defaults, but retain any execute permissions that are currently set
+XMake a directory or file searchable/executable by everyone if it is already searchable/executable by anyone
u=rwx,go=rxMake a file readable/executable by everyone and writable by the owner only
go=Clear all mode bits for group and others
g=u-wSet the group bits equal to the user bits, but clear the group write bit
Permissions for new files

None of this tells us what the permissions for new files are going to be. The wrong choice could be disastrous. For example, if files were automatically created with the permissions rwxrwxrwx anybody could access thcode in any way. On the other hand, creating thcode with r-------- could result in a lot of work setting thcode to what you really want them to be. UNIX solves this problem with a thing called umask (User mask) This is a default non-permission: it specifies which permission bits not to allow.

As if this weren't confusing enough, it's specified in the octal number system, in which the valid digits are 0 to 7. Each octal digit represents 3 bits. By contrast, the more common hexadecimal system uses 16 digits, 0 to 9 and a to f .The original versions of UNIX ran on machines that used the octal number system, and since the permissions come in threes, it made sense to leave the umask value in octal.

An example: by default, you want to create files that anybody can read, but only you can write. You set the mask to 022. This corresponds to the binary bit pattern 000010010

The leading 0 is needed to specify that the number is in octal, not to make up three digits. If you want to set the permissions so that by default nobody can read, you'd set it to 0222. Some shells automatically assume that the number is octal, so you may be able to omit the 0, but it's not good practice.

The permissions are allowed where the corresponding bit is 0:

rwxrwxrwx     Possible permissions 
000010010     umask
rwxr-xr-x     resultant permissions

By default, files are created without the x bits, whereas directories are created with the allowed x bits, so with this umask a file would be created with the permissions rw-r--r--

umask is a shell command. To set it, just enter:

$ umask 022

It's preferable to set this in your shell initialization filesee page 135 for further details.

Beware of creating a too restrictive umask. For example, you will get into a lot of trouble with a umask like 377, which creates files that you can only read, and that nobody else can access at all. If you disallow the x (executable) bit, you will not be able to access directories you create, and you won't be able to run programs you compile.

Making a program executable

File permissions enable one problem that occurs so often that it's worth drawing attention to it. Many operating systems require that an executable program have a special naming convention, such as COMMAND.COM or FOO.BAT which in MS-DOS denotes a specific kind of binary executable and a script file, respectively. In UNIX, executable programs don't need a special suffix, but they must have the x bit set. Sometimes this bit gets reset (turned off), for example if you copy it across the Net with ftp. The result looks like this:

$ ps
bash: ps: Permission denied $ ls -l /bin/ps
-r--r--r--  1 bin  kmem  163840 May 6 06:02 /bin/ps
$ su                             you need to be super user to set ps permission
Password:                        password doesn't echo
# chmod +x /bin/ps               make it executable
# ps                             now it works
PID  TT     TIME  COMMAND
226  p2  0:00.56  su (bash)
239  p2  0:00.02  ps
146  v1  0:00.06  /usr/libexec/getty Pc ttyv1
147  v2  0:00.05  /usr/libexec/getty Pc ttyv2
# ^D                             exit su
$ ps
ps: /dev/mem: Permission denied  hey! its stopped working

Huh? It only worked under su and stopped working when I became a mere mortal again? What's going on here?

There's a second problem with programs like ps: some versions need to be able to access special files, in this case /dev/mem, a special file that addresses the system memory. To do this, we need to set the setgid bit, s, which requires becoming super user again:

$ su                  you need to be super user to set ps permission
Password:             password doesn't echo
# chmod g+s /bin/ps   set the setgid bit
# ls -l /bin/ps       see what it looks like
-r-xr-sr-x  1 bin  kmem  163840 May   6 06:02 /bin/ps
# ^D                  exit su
$ ps                  now it still works
PID  TT  STAT     TIME  COMMAND
226  p2  S     0:00.56  su (bash)
239  p2  R+    0:00.02  ps
146  v1  Is+   0:00.06  /usr/libexec/getty Pc ttyv1
147  v2  Is+   0:00.05  /usr/libexec/getty Pc ttyv2

In this example, the permissions in the final result really are the correct permissions for ps. It's impossible to go through the permissions for every standard program. If you suspect that you have the permissions set in correctly, use the permissions of the files on the Live File system CD-ROM as a guideline.

setuid and setgid programs can be a security issue. What happens if the program called ps is really something else, a Trojan Horse? We set the permissions to allow it to break into the system. As a result, FreeBSD has found an alternative method for ps to do its work, and it no longer needs to be set setgid.

Mandatory Access Control

For some purposes, traditional UNIX permissions are insufficient. Release 5.0 of FreeBSD introduces Mandatory Access Control, or MAC, which permits loadable kernel modules to augment the system security policy. MAC is intended as a toolkit for developing local and vendor security extensions, and it includes a number of sample policy modules, including Multi-Level Security (MLS) with compartments, and a number of augmented UNIX security models including a file system firewall. At the time of writing it is still considered experimental software, so this book doesn't discuss it further. See the man pages for more details.

Links

In UNIX, files are defined by inodes structures on disk that you can't access directly. They contain the meta data, all the information about the file, such as owner, permissions and timestamps. What they don't contain are the things you think of as making up a file: they don't have any data, and they don't have names. Instead, the inode contains information about where the data blocks are located on the disk. It doesn't know anything about the name: that's the job of the directories.

A directory is simply a special kind of file that contains a list of names and inode numbers: in other words, they assign a name to an Inode, and thus to a file. More than one name can point to the same inode, so files can have more than one name. This connection between a name and an inode is called a link sometimes confusingly hard link. The inode numbers relate to the file system, so files must be in the same file system as the directory that refers to them.

Directory entries are independent of each other: each points to the Inode, so they're completely equivalent. The inode contains a link count that keeps track of how many directory entries point to it: when you remove the last entry, the system deletes the file data and metadata.

Alternatively, symbolic links sometimes called soft links, are not restricted to the same file system (not even to the same system!), and they refer to another file name, not to the file itself. The difference is most evident if you delete a file: if the file has been hard linked, the other names still exist and you can access the file by them. If you delete a file name that has a symbolic link pointing to it, the file goes away and the symbolic link can't find it anymore.

It's not easy to decide which kind of link to usesee UNIX Power Tools (O'Reilly) for more details.

Directory hierarchy

Although Microsoft platforms have a hierarchical directory structure, there is little standardization of the directory names: it's difficult to know where a particular program or data file might be. UNIX systems have a standard directory hierarchy, though every vendor loves to change it just a little bit to ensure that they're not absolutely compatible. In the course of its evolution, UNIX has changed its directory hierarchy several times. It's still better than the situation in the Microsoft world. The most recent, and probably most far-reaching changes, occurred over ten years ago with System V.4 and 4.4BSD, both of which made almost identical changes.

Nearly every version of UNIX prefers to have at least two file systems, / (the root file system) and /usr even if they only have a single disk. This arrangement is considered more reliable than a single file system: it's possible for a file system to crash so badly that it can't be mounted anymore, and you need to read in a tape backup, or use programs like fsck or fsdb to piece them together. We have already discussed this issue on page 68, where I recommend having /usr on the same file system as /.

Standard directories

The physical layout of the file systems does not affect the names or contents of the directories, which are standardized. Table 10-2 gives an over view of the standard FreeBSD directories; see the man page hier(7) for more details.

10.2. FreeBSD directory hierarchy
Directory nameUsage
/Root file system. Contains a couple of system directories and mount points for other file systems. It should not contain anything else.
/binExecutable programs of general use needed at system startup time. The name was originally an abbreviation for binary,but many of the files in here are shell scripts.
/bootFiles used when booting the system, including the kernel and its associated klds.
/cdromAmount point for CD-ROM drives.
/compatA link to /usr/compat: see below.
/devDirectory of device nodes. The name is an abbreviation for devices. From FreeBSD 5.0 onward, this is normally a mount point for the device file system, devfs. We'll look at the contents of this directory in more detail on page 195.
/etcConfiguration files used at system startup. Unlike System V, /etc does not contain kernel build files, which are not needed at system startup. Unlike earlier UNIX versions, it also does not contain executablethey have been moved to /sbin.
/homeBy convention, put user files here. Despite the name, /usr is for system files.
/mntA mount point for floppies and other temporary file systems.
/procThe process file system. This directory contains pseudo-files that refer to the virtual memory of currently active processes.
/rootThe home directory of the user root. In traditional UNIX file systems, root's home directory was /, but this is messy.
/sbinSystem executable needed at system startup time. These are typically system administration files that used to be stored in /etc.
/sysIf present, this is usually a symbolic link to /usr/src/sys, the kernel sources. This is a tradition derived from 4.3BSD.
/tmpA place for temporary files. This directory is an anachronism: normally it is on the root file system, though it is possible to mount it as a separate file system or make it a symbolic link to /var/tmp. It is maintained mainly for programs that expect to find it.
/usrThe "second file system." See the discussion above.
/usr/XllR6The X WindowSystem.
/usr/XllR6/binExecutable X11 programs.
/usr/XllR6/includeHeader files for X11 programming.
/usr/XllR6/libLibrary files for X11.
/usr/XllR6/manMan pages for X11.
/usr/binStandard executable programs that are not needed at system start. Most standard programs you use are stored here.
/usr/compatA directory containing code for emulated systems, such as Linux.
/usr/gamesGames.
/usr/includeHeader files for programmers.
/usr/libLibrary files. FreeBSD does not have a directory /lib.
/usr/libexecExecutable files that are not started directly by the user, for example the phases of the C compiler (which are started by /usr/bin/gcc)or the getty program, which is started by init.
/usr/libdataMiscellaneous files used by system utilities.
/usr/localAdditional programs that are not part of the operating system. It parallels the /usr directory in having subdirectories bin, include, lib, man, sbin, and share. This is where you can put programs that you get from other sources.
/usr/objObject files created when building the system. See Chapter 33 .
/usr/portsThe Ports Collection.
/usr/sbinSystem administration programs that are not needed at system startup.
/usr/shareMiscellaneous read-only files, mainly informative. Subdirectories include doc, the FreeBSD documentation, games, info, the GNU info documentation, locale, internationization information, and man, the man pages.
/usr/srcSystem source files.
/varA file system for data that changes frequently, such as mail, news, and log files. If /var is not a separate file system, you should create a directory on another file system and symlink /var to it.
/var/logDirectory with system log files
/var/mailIncoming mail for users on this system
/var/spoolTransient data, such as outgoing mail, print data and anonymous ftp.
/var/tmpTemporary files.

File system types

FreeBSD supports a number of file system types. The most important are:

10.3. File system support
File systemmount option
CD-ROMcd9660
DVDudf
Linux ext2ext2fs
Microsoft MS-DOSmsdosfs
Microsoft NTntfs
Novell Netwarenwfs
Microsoft CIFSsmbfs

Soft updates

Soft updates change the way the file system performs I/O. They enable metadata to be written less frequently. This can give rise to dramatic performance improvements under certain circumstances, such as file deletion. Specify soft updates with the U option when creating the file system. For example:

# newfs -U /dev/da1s2h

If you forget this fag, you can enable them later with tunefs :

# tunefs -n enable /dev/da1s2h

You can't perform this operation on a mounted file system.

Snapshots

One of the problems with backing up file systems is that you don't get a consistent view of the file system: while you copy a file, other programs may be modifying it, so what you get on the tape is not an accurate view of the file at any time. Snapshots are a method to create a unified view of a file system. They maintain a relatively small file in the file system itself containing information on what has changed since the snapshot was taken. When you access the snapshot, you get this data rather than the current data for the parts of the disk which have changed, so you get a view of the file system as it was at the time of the snapshot.

Creating snapshots

You create snapshots with the mount command and the -o snapshot option. For example, you could enter

# mount -u -o snapshot /var/snapshot/snapl /var

This command creates a snapshot of the /var file system called /var/snapshot/snapl. Snapshot files have some interesting properties:

Probably the most useful thing you can do with a snapshot is to take a backup of it. We'll look at backups on page 253.

At the time of writing, snapshots are still under development. It's possible that you might still have trouble with them, in particular with deadlocks that can only be cleared by rebooting.

It takes about 30 seconds to create a snapshot of an 8 GB file system. During the last five seconds, file system activity is suspended. If there's a lot of soft update activity going on in the file system (for example, when deleting a lot of files), this suspension time can become much longer, up to several minutes. To remove the same snapshot takes about two minutes, but it doesn't suspend file system activity at all.

Mounting file systems

Microsoft platforms identify partitions by letters that are assigned at boot time. There is no obvious relation between the partitions, and you have little control over the way the system assigns them. By contrast, all UNIX partitions have a specific relation to the root file system, which is called simply /.This flexibility has one problem: you have the choice of where in the overall file system structure you put your individual file systems. You specify the location with the mount command. For example, you would typically mount a CD-ROM in the directory /cdrom, but if you have three CD-ROM drives attached to your SCSI controller, you might prefer to mount them in the directories /cd0, /cd1, and /cd2. To mount a file system, you need to specify the device to be mounted, where it is to be mounted, and the type of file system (unless it is ufs). The mount point (the directory where it is to be mounted) must already exist. To mount your second CD-ROM on /cd1, you enter:

# mkdir /cdl                    only if it doesn't exist
# mount -t cd9660 -o ro /dev/cdla /cdl

When the system boots, it calls the startup script /etc/rc, which among other things automatically mounts the file systems. All you need to do is to supply the information: what is to be mounted, and where? This is in the file /etc/fstab. If you come from a System V environment, you'll notice significant difference in formatsee the man page fstab(5), for the full story. A typical /etc/fstab might look like:

/dev/ad0s1a           /                   ufs        rw         1 1  root file system
/dev/ad0s1b           none                swap       sw         0 0  swap
/dev/ad0s1e           /usr                ufs        rw         2 2  /usr file system
/dev/dalsle           /src                ufs        rw         2 2  additional file system
/dev/da2s1            /linux              ext2fs     rw         2 2  Linux file system
/dev/adlsl            /C:                 msdos      rw         2 2  Microsoft file system
proc                  /proc               procfs     rw         0 0  proc pseudo-file system
linproc               /compat/linux/proc  linprocfs  rw         0 0  
/dev/cd0a             /cdrom              cd9660     ro         0 0  CD-ROM
presto:/              /presto/root        nfs        rw         0 0  NFS file systems on other systems
presto:/usr           /presto/usr         nfs        rw         0 0  
presto:/home          /presto/home        nfs        rw         0 0  
presto:/S             /S                  nfs        rw         0 0  
//guestasamba/public  /smb                smbfs      rw,noauto  0 0  SMB file system

The format of the file is reasonably straight forward:

Mounting files as file systems

So far, our files have all been on devices, also called special files. Sometimes, though, you may want to access the contents of a file as a file system:

In each case, the solution is the same: you mount the files as a vnode device with the md driver.

The md driver creates a number of different kinds of pseudo-device. See the man page md(4).. We use the vnode device, a special file that refers to file system files. Support for md is included in the GENERIC kernel, but if you've built a kernel without the md drive, you can load it as a kld. If you're not sure, try loading the kld anyway.

In the following example, we associate a vnode device with the ISO image iso-image using the program mdconfig:

# kldload md                          load the kld module if necessary
kldload: can't load md: File exists   already loaded or in the kernel
# mdconfig -a -t vnode -f iso -image  and configure the device
md0                                   this is the name assigned in directory /dev
# mount -t cd9660 /dev/md0 /mnt       then mount it

After this, you can access the image at /mnt as a normal file system. You specify -t cd9660 in this case because the file system on the image is a CD9660 file system. You don't specify this if you're mounting a UFS file system, for example a snapshot image.

Older versions of FreeBSD used the vn driver, which used different syntax. Linux uses loop mounts, which FreeBSD doesn't support.

Unmounting file systems

When you mount a file system, the system assumes it is going to stay there, and in the interests of efficiency it delays writing data back to the file system. This is also the reason why you can't just turn the power off when you shut down the system. If you want to stop using a file system, you must tell the system about it so that it can flush any remaining data. You do this with the umount command. Note the spelling of this commandthere's no n in the command name.

You need to do this even with read-only media such as CD-ROMs: the system assumes it can access the data from a mounted file system, and it gets quite unhappy if it can't. Where possible, it locks removable media so that you can't remove them from the device until you unmount them.

Using umount is straightforward: just tell it what to unmount, either the device name or the directory name. For example, to unmount the CD-ROM we mounted in the example above, you could enter one of these commands:

# umount /dev/cdla
# umount /cdl

Before unmounting a file system, umount checks that nobody is using it. If somebody is using it, it refuses to unmount it with a message like umount: /cdl: Device busy. This message often occurs because you have changed your directory to a directory on the file system you want to remove. For example (which also shows the usefulness of having directory names in the prompt):

=== root@freebie (/dev/ttyp2)  /cdl 16 -> umount /cdl
umount: /cd1: Device busy
=== root@freebie (/dev/ttyp2)  /cd1 17 -> cd
=== root@freebie (/dev/ttyp2)  ~ 18 -> umount /cdl
=== root@freebie (/dev/ttyp2)  ~ 19 ->

After unmounting a vnode file system, don't forget to unconfigure the file:

# umount /mnt
# mdconfig -d -u 0

The parameter 0 refers to md device 0, in other words /dev/md0.

FreeBSD devices

UNIX refers to devices in the same manner as it refers to normal files. By contrast to normal ("regular") files, they are called special files. They're not really files at all: they're information about device support in the kernel, and the term device node is more accurate. Conventionally, they are stored in the directory /dev. Some devices don't have device nodes, for example Ethernet interfaces: they are treated differently by the ifconfig program.

Traditional UNIX systems distinguish two types of device, blockdevices and character devices. FreeBSD no longer has block devices; we discussed the reasons for this on page 36.

In traditional UNIX systems, including FreeBSD up to Release 4, it was necessary to create device nodes manually. This caused a number of problems when they didn't match what was in the system. Release 5 of FreeBSD has solved this problem with the device file system, also known as devfs. devfs is a pseudo-file system that dynamically creates device nodes for exactly those devices that are in the kernel, which makes it unnecessary to manually create devices.

Overview of FreeBSD devices

Every UNIX system has its own peculiarities when it comes to device names and usage. Even if you're used to UNIX, you'll find the following table useful.

10.4. FreeBSD device names
acd0First ata (IDE) CD-ROM drive.
ad0First ata (IDE or similar) disk drive. See Chapter 2, page 38, for a complete list of disk drive names.
bpf0Berkeley packet filter.
cd0First SCSI CD-ROM drive.
ch0SCSI CD-ROM changer (juke box)
consoleSystem console, the device that receives console messages. Initially it is /dev/ttyv0, but it can be changed.
cuaa0First serial port in callout mode.
cuaia0First serial port in callout mode, initial state. Note the letter i for initial.
cuala0First serial port in callout mode, lock state. Note the letter l for lock.
da0First SCSI disk drive. See Chapter 2, page 38, for a complete list of disk drive names.
esa0First SCSI tape drive, eject on close mode.
fdFile descriptor pseudo-devices: a directory containing pseudo-devices that, when opened, return a duplicate of the file descriptor with the same number. For example, if you open /dev/fd/0, you get another handle on your stdin stream (file descriptor 0).
fd0The first floppy disk drive, accessed as a file system.
kmemKernel virtual memory pseudo-device.
lpt0First parallel printer.
memPhysical virtual memory pseudo-device.
nsa0First SCSI tape drive, no-rewind mode.
nullThe "bit bucket." Send data to this device if you never want to see it again.
psm0PS/2 mouse.
ptyp0First master pseudo-terminal. Master pseudo-terminals are named /dev/ptyp0 through /dev/ptypv, /dev/ptyq0 through /dev/ptyqv, /dev/ptyr0 through /dev/ptyrv, /dev/ptys0 through /dev/ptysv, /dev/ptyP0 through /dev/ptyPv, /dev/ptyQ0 through /dev/ptyQv, /dev/ptyR0 through /dev/ptyRv and /dev/ptyS0 through /dev/ptySv.
randomRandom number generator.
sa0First SCSI tape drive, rewind on close mode.
sysmouseSystem mouse, controlled by moused. Well look at this again on page 519.
ttyCurrent controlling terminal.
ttyd0First serial port in callin mode.
ttyid0First serial port in callin mode, initial state.
ttyld0First serial port in callin mode, lock state.
ttyp0First slave pseudo-terminal. Slave pseudo-terminals are named /dev/ttyp0 through /dev/ttypv, /dev/ttyq0 through /dev/ttyqv, /dev/ttyr0 through /dev/ttyrv, /dev/ttys0 through /dev/ttysv, /dev/ttyP0 through /dev/ttyPv, /dev/ttyQ0 through /dev/ttyQv, /dev/ttyR0 through /dev/ttyRv and /dev/ttyS0 through /dev/ttySv. Some processes, such as xterm, only look at /dev/ttyp0 through /dev/ttysv.
ttyv0First virtual tty. This is the display with which the system starts. Up to 10 virtual ttys can be activated by adding the appropriate getty information in the file /etc/ttys. See Chapter 19 , page 338, for further details.
ugen0First generic USB device.
ukbd0First USB keyboard.
ulpt0First USB printer.
umass0First USB mass storage device.
ums0First USB mouse.
uscanner0First USB scanner.
vinumDirectory for Vinum device nodes. See Chapter 12 , for further details.
zeroDummy device that always returns the value (binary) 0 when read.

You'll note a number of different modes associated with the serial ports. We'll look at them again in Chapter 19 .

Virtual terminals

As we have seen, UNIX is a multitasking operating system, but a PC generally only has one screen. FreeBSD solves this problem with virtual terminals. When in text mode, you can change between up to 16 different screens with the combination of the Alt key and a function key. The devices are named /dev/ttyv0 through /dev/ttyv15 and correspond to the keystrokes Alt-Fl through Alt-F16. By default, three virtual terminals are active: /dev/ttyv0 through /dev/ttyv2. The system console is the virtual terminal /dev/ttyv0 and that's what you see when you boot the machine. To activate additional virtual terminals, edit the file /etc/ttys. There you find:

ttyvO  "/usr/libexec/getty Pc"  cons25  on secure
ttyvl  "/usr/libexec/getty Pc"  cons25  on secure
ttyv2  "/usr/libexec/getty Pc"  cons25  on secure
ttyv3  "/usr/libexec/getty Pc"  cons25  off secure

The key words on and off refer to the state of the terminal: to enable one, set its state to on. To enable extra virtual terminals, add a line with the corresponding terminal name, in the range /dev/ttyv4 to /dev/ttyv15. After you have edited /etc/ttys, you need to tell the system to re-read it in order to start the terminals. Do this as root with this command:

# kill -1 1

Process 1 is init see page 528 for more details.

Pseudo-terminals

In addition to virtual terminals, FreeBSD offers an additional class of terminals called pseudo-terminals. They come in pairs: a master device also called a pry (pronounced pity) is used only by processes that use the interface, and has a name like /dev/ptyp0. The slave device looks like a terminal, and has a name like /dev/ttyp0. Any process can open it without any special knowledge of the interface. These terminals are used for network connections such as xterm, telnet and rlogin. You don't need a getty for pseudo-terminals. Since FreeBSD Release 5.0, pseudo-terminals are created as required.

11. Disks

Adding a hard disk; Using sysinstall; Doing it the hard way; Creating file systems; Moving file systems; Recovering from disk data errors;

One of the most important parts of running any computer system is handling data on disk. We have already looked at UNIX file handling in Chapter 10. In this chapter, we'll look at two ways to add another disk to your system, and what you should put on them. In addition.

Adding a hard disk

When you installed FreeBSD, you created file systems on at least one hard disk. At a later point, you may want to install additional drives. There are two ways to do this: with sysinstall and with the traditional UNIX command-line utilities.

There was a time when it was dangerous to use sysinstall after the system had been installed: there was a significant chance of shooting yourself in the foot. There's always a chance of doing something wrong when initializing disks, but sysinstall has become a lot better, and now it's the tool of choice. It's good to know the alternatives, though. In this section we'll look at sysinstall, and on page 209 we'll see how to do it manually if sysinstall won't cooperate.

We've been through all the details of disk layout and slices and partitions in Chapter 2, so I won't repeat them here. Basically, to add a new disk to the system, you need to:

These are the same operations that we performed in Chapter 5.

Disk hard ware installation

Before you can do anything with the disk, you have to install it in the system. To do this, you must normally shut down the system and turn the power off, though high-end SCSI enclosures allow hot-swapping, changing disks in a running system. If the disk is IDE, and you already have an IDE disk on the controller, you need to set the second disk as "slave" drive. And you may have to set the first disk as "master" drive: if you only have one drive, you don't set any jumpers, but if you have two drives, some disks require you to set jumpers on both disks. If you don't do this, the system will appear to hang during the power-on self test, and will finally report some kind of disk error.

Adding a SCSI disk is more complicated. You can connect up to 15 SCSI devices to a host adapter, depending on the interface. Many systems restrict the number to 7 for compatibility with older SCSI interfaces. Typically, your first SCSI disk will have the SCSI ID 0, and the host adapter will have the SCSI ID 7. Traditionally, the IDs 4, 5, and 6 are reserved for tape and CD-ROM drives, and the IDs 0 to 3 are reserved for disks, though FreeBSD doesn't impose any restrictions on what goes where.

What ever kind of disk you're adding, look at the boot messages, which you can retrieve with the dmesg command. For example, if you're planning to add a SCSI device, you might see:

sym0: <875> port 0xc400-0xc4ff mem 0xec002000-0xec002fff,0xec003000-0xec0030ff irq 10
at device 9.0 on pci0
sym0: Symbios NVRAM, ID 7, Fast-20, SE, NO parity
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: SCAN FOR LUNS disabled for targets 0.
sym1: <875> port 0xc800-0xc8ff mem 0xec001000-0xec001fff,0xec000000-0xec0000ff irq 9
at device 13.0 on pci0
sym1: No NVRAM, ID 7, Fast-20, SE, parity checking
further down...
Waiting 3 seconds for SCSI devices to settle sa0
at sym0 bus 0 target 3 lun 0
sa0: <EXABYTE EXB-8505SMBANSH2 0793> Removable Sequential Access SCSI-2 device
sa0: 5.000MB/s transfers (5.000MHz, offset 11) sa1 at sym0 bus 0 target 4 lun 0
sa1: <ARCHIVE Python 28849-XXX 4.CM> Removable Sequential Access SCSI-2 device
sa1: 5.000MB/s transfers (5.000MHz, offset 15) sa2 at sym0 bus 0 target 5 lun 0
sa2: <TANDBERG TDC 3800 -03:> Removable Sequential Access SCSI-CCS device
sa2: 3.300MB/s transfers
pass4 at sym0 bus 0 target 4 lun 1
pass4: <ARCHIVE Python 28849-XXX 4.CM> Removable Changer SCSI-2 device
pass4: 5.000MB/s transfers (5.000MHz, offset 15) cd0 at sym0 bus 0 target 6 lun 0
cd0: <NRC MBR-7 110> Removable CD-ROM SCSI-2 device
cd0: 3.300MB/s transfers
cd0: cd present [322265 x 2048 byte records]
da0 at sym1 bus 0 target 3 lun 0
da0: <SEAGATE ST15230W SUN4.2G 0738> Fixed Direct Access SCSI-2 device
da0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da0: 4095MB (8386733 512 byte sectors: 255H 63s/t 522C)

This output shows two Sym bios SCSI host adapters /dev/(sym0 and /dev/syml), three tape drives/dev/(sa0, /dev/sal and /dev/sa2), a CD-ROM drive /dev/(cd0), a tape changer /dev/(pass4), and also a disk drive /dev/da0 on ID 3, which is called a target in these messages. The disk is connected to the second host adapter, and the other devices are connected to the first host adapter.

Installing an external SCSI device

External SCSI devices have two cable connectors: one goes towards the host adapter, and the other towards the next device. The order of the devices in the chain does not have to have anything to do with the SCSI ID. This method is called daisy chaining. At the end of the chain, the spare connector may be plugged with a terminator, a set of resistors designed to keep noise off the bus. Some devices have internal terminators, however. When installing an external device, you will have to do one of the following:

You can add external SCSI devices to a running system if they're hot-pluggable. It might even work if they're not hot-pluggable, but it's not strictly the correct thing to do, and there's the risk that you might damage something, possibly irreparably. After connecting the devices, powering them up and waiting for them to come ready, run camcontrol rescan. For example, if you added a second disk drive to the second host adapter in the example above, you might see:

# camcontrol rescan 1
dal at syml bus 0 target 0 lun 0
dal: <SEAGATE ST15230W SUN4.2G 0738> Fixed Direct Access SCSI-2 device
dal: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da1: 4095MB (8386733 512 byte sectors: 255H 63s/t 522C)
Re-scan of bus 1 was successful

There's a problem with this approach: note that /dev/dal has ID 0, and the already present /dev/da0 has ID 3. If you now reboot the system, they will come up with the device names the other way round. We'll look at this issue in more detail in the next section.

Installing an internal SCSI device

Installing an internal SCSI device is much the same as installing an external device. Instead of daisy chains, you have a tat band cable with a number of connectors. Find one that suits you, and plug it into the device. Again, you need to think about termination:

In this chapter, we'll look at two ways of installing a drive in an existing SCSI chain. We could be in for a surprise: the device ID we get for the new drive depends on what is currently on the chain. For example, consider our example above, where we have a chain with a single drive on it:

da0 at sym1 bus 0 target 3 lun 0
da0: <SEAGATE ST15230W SUN4.2G 0738> Fixed Direct Access SCSI-2 device
da0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da0: 4095MB (8386733 512 byte sectors: 255H 63S/T 522C)

This drive on target (ID) 2. If we put our new drive on target 0 and reboot, we see:

da0 at sym1 bus 0 target 0 lun 0
da0: <SEAGATE ST15230W SUN4.2G 0738> Fixed Direct Access SCSI-2 device
da0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da0: 4095MB (8386733 512 byte sectors: 255H 63S/T 522C)
da1 at sym1 bus 0 target 3 lun 0
da1: <SEAGATE ST15230W SUN4.2G 0738> Fixed Direct Access SCSI-2 device
da1: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da1: 4095MB (8386733 512 byte sectors: 255H 63S/T 522C)

At first glance, this looks reasonable, but that's only because both disks are of the same type. If you look at the target numbers, you'll notice that the new disk is /dev/da0,not /dev/dal. The target ID of the new disk is lower than the target ID of the old disk, so the system recognizes the new disk as /dev/da0, and our previous /dev/da0 has become /dev/dal .

This change of disk ID can be a problem. One of the first things you do with a new disk is to create new disk labels and file systems. Both offer excellent opportunities to shoot yourself in the foot if you choose the wrong disk: the result would almost certainly be the complete loss of data on that disk. Even apart from such catastrophes, you'll have to edit /etc/fstab before you can mount any file systems that are on the disk. The alternatives are to wire down the device names, or to change the SCSI IDs. In FreeBSD 5.0, you wire down device names and busses by adding entries to the boot configuration file /boot/device.hints. We'll look at that on page 575.

Formatting the disk

Formatting is the process of rewriting every sector on the disk with a specific data pattern, one that the electronics find most difficult to reproduce: if they can read this pattern, they can read anything. Microsoft calls this a low-level format.1) Obviously it destroys any existing data, so

If you have anything you want to keep, back it up before formatting. Most modern disks don't need formatting unless they're damaged. In particular, formatting will not help if you're having configuration problems, if you can't get PPP to work or you're running out of disk space. Well, it will solve the disk space problem, but not in the manner you probably desire.

If you do need to format a SCSI disk, use camcontrol. camcontrol is a control program for SCSI devices, and it includes a lot of useful functions that you can read about in the man page. To format a disk, use the following syntax:

# camcontrol format dal

Remember that formatting a disk destroys all data on the disk. Before using the command, make sure that you need to do so: there are relatively few cases that call for formatting a disk. About the only reasons are if you want to change the physical sector size of the disk, or if you are getting "medium format corrupted" errors from the disk in response to read and write requests.

FreeBSD can format only floppies and SCSI disks. In general it is no longer possible to reformat ATA (IDE) disks, though some manufacturers have programs that can recover from some data problems. In most cases, though, in sufficient to write zeros to the entire disk:

# dd if=/dev/zero of=/dev/ad1 bs=128k

If this doesn't work, you may find formatting programs on the manufacturer's web site. You'll probably need to run them under a Microsoft platform.

Using sysinstall

If you can, use sysinstall to partition your disk. Looking at the dmesg output for our new disk, we see:

dal at syml bus 0 target 0 lun 0
dal: <SEAGATE ST15230W SUN4.2G 0738> Fixed Direct Access SCSI-2 device
dal: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da1: 4095MB (8386733 512 byte sectors: 255H 63s/t 522C)

You see the standard installation screen (see Chapter 5, page 60). Select Index, then Partition, and you see the following screen:

Disk selection menu


. 11.1. Disk selection menu

In this case, we want to partition /dev/dai, so we position the cursor on da1 (as shown) and press Enter. We see the disk partition menu, which shows that the disk currently contains three partitions:

We want a FreeBSD partition, not a Microsoft partition. At this point, we have a number of choices:

Disk partition menu


. 11.2. Disk partition menu

To use the whole disk, we first delete the current partition: we press the cursor down key until it highlights the FreeBSD partition. Then we press d, and the three partitions are joined into one, marked unused.

The next step is to create a new partition using the entire disk. If we press f, we get the following message:



We don't get this message if we use the a command: it just automatically assumes Yes. In this case we've decided to use the whole disk, so we move the cursor right to No and press Enter. That gives us a boot manager selection screen:



This isn't a boot disk, so we don't need any boot record, and it doesn't make any difference what we select. It's tidier, though, to select None as indicated. Then we press q to exit the partition editor, get back to the function index, and select Label. We see:



The important information on this rather empty looking menu is the information at the top about the free space available. We want to create two partitions: first, a swap partition of 512 MB, and then a file system taking up the rest of the disk. We press C, and are shown a submenu offering us all 8386733 blocks on the disk. We erase that and enter 512m, which represents 512 MB. Then we press Enter, and another submenu appears, asking us what kind of slice it is. We move the cursor down to select A swap partition :



Next, we press c again to create a new partition. This time, we accept the offer of the rest of the space on the disk, 7338157 sectors, we select A file system , and we are presented with yet another menu asking for the name of the file system. We enter the name, in this case /S:



After pressing Enter, we see:



Finally, we press W to tell the disk label editor to perform the function. We get an additional warning screen:



We're doing this online, so that's OK. We select Yes, and sysinstall creates the file system and mounts both it and the swap partition. This can take quite a while. Don't try to do anything with the drive until it's finished.

Doing it the hard way

Unfortunately, sometimes you may not be able to use the sysinstall method. You may not have access to sysinstall, or you may want to use options that sysinstall doesn't offer. That leaves us with the old way to add disks. The only difference is that this time we need to use different tools. In the following sections, we'll look at what we have to do to install this same 4 GB Seagate drive manually. This time we'll change the partitioning to contain the following partitions:

We've called this file system /newhome to use it as an example of moving file systems to new disks.

Creating a partition table

The first step is to create a PC BIOS style partition table on the disk. As in Microsoft, the partitioning program is called fdisk. In the following discussion, you'll find a pocket calculator indispensable.

If the disk is not brand new, it will have existing data of some kind on it. Depending on the nature of that data, fdisk could get sufficiently confused to not work correctly. If you don't format the disk, it's a good idea to overwrite the beginning of the disk with dd:

# dd if=/dev/zero of=/dev/da1 count=100
100+0 records in 100+0 records out
51200 bytes transferred in 1 sees (51200 bytes/sec)

We'll as sign 1 GB for Microsoft and use the remaining approximately 3 GB for FreeBSD. Our resulting partition table should look like:

Partition table on second FreeBSD disk


. 11.3. Partition table on second FreeBSD disk

The Master Boot Record and the Partition Table take up the first sector of the disk, but many of the allocations are track oriented, so the entire first track of the disk is not available for allocation. The rest, up to the end of the last entire cylinder, can be divided between the partitions. It's easy to make a mistake in specifying the parameters, and fdisk performs as good as no checking. You can easily create a partition table that has absolutely no relationship with reality, so it's a good idea to calculate them in advance. For each partition, we need to know three things:

In addition, we need to decide which partition is the active partition, the partition from which we want to boot. In this case, it doesn't make any difference, because we won't be booting from the disk, but it's always a good idea to set it anyway.

We specify the partitions we don't want by giving them a type, start sector and end sector of 0. Our disk has 8386733 sectors, numbered 0 to 8386732. Partitions should start and end on a cylinder boundary, and we want the Microsoft partition to be about 1 GB. 1GB is 1024 MB, and 1 MB is 2048 sectors of 512 bytes each, so theoretically we want 1024 x 2048, or 2197152 sectors. Because of the requirement that partitions begin and end on a "cylinder" boundary, we need to find the closest number of "cylinders " to this vaflue. First we need to find out how big a "cylinder "is. We can do this by running /fdisk without any options:

# fdisk dal
******* Working on device /dev/dal *******
parameters extracted from in-core disklabel are:
cylinders=13726 heads=13 sectors/track=47 (611 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1 
parameters to be used for BIOS calculations are:
cylinders=13726 heads=13 sectors/track=47 (611 blks/cyl)

fdisk: invalid fdisk partition table found
Media sector size is 512
Warning: BIOS sector numbering starts with sector 1 
Information from DOS bootblock is:
The data for partition 1 is:
<UNUSED>
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
   start 47, size 8386539 (4094 Meg), flag 80 (active)
       beg: cyl 0/ head 1/ sector 1;
       end: cyl 413/ head 12/ sector 47

You'll notice that /fdisk has decided that there is a FreeBSD partition in partition 4. That happens even if the disk is brand new. In fact, this is a less desirable feature of /fdisk: it "suggests" this partition, it's not really there, which can be really confusing. This printout does, however, tell us that /fdisk thinks there are 611 sectors per cylinder, sowe divide 2197152 by 611 and get 3423.327 cylinders. We round down to 3423 cylinders, which provestobe2091453 sectors. This is the length we give to the first partition.

We use the remaining space for the FreeBSD partition. How much? Well, dmesg tells us that there are 8386733 sectors, but if you look at the geometry that /fdisk outputs, there are 13726 cylinders with 13 heads (tracks) per cylinder and 47 sectors per track. 13726 x 13 x 47 is 8386586. This rounding down is the explanation for the missing data at the end of the disk that we saw on page 204. The best way to calculate the size of the FreeBSD partition is to take the number of cylinders and multiply by the number of tracks per cylinder. The FreeBSD partition starts behind the Microsoft partition, so it goes from cylinder 3423 to cylinder 13725 inclusive or 10303 cylinders. At 611 sectors per cylinder, we have a total of 6295133 sectors in the partition. Our resulting information is:

11.1. sample fdisk parameters
Partition numberPartitionStartSize
1612091453
216520914536295133
300
400

Next we run fdisk in earnest by specifying the -i option. During this time, you may see messages on the console:

da1: invalid primary partition table: no magic

The message no magic doesn't mean that fdisk is out of purple smoke. It refers to the fact that it didn't find the so-called magic number, which identifies the partition table. We don't have a partition table yet, so this message isn't surprising. It's also completely harmless.

fdisk prompts interactively when you specify the -i flag:

# fdisk -i dal
******* Working on device /dev/da1 *******
parameters extracted from in-core disklabel are:
cylinders=13726 heads=13 sectors/track=47 (611 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=13726 heads=13 sectors/track=47 (611 blks/cyl)

Do you want to change our idea of what BIOS thinks ? [n] Enter pressed Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblack is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
   start 0, size 8386733 (4095 Meg), flag 80 (active)
       beg: cyl 0/ head 0/ sector 1; 
       end: cyl 522/ head 12/ sector 47
Do you want to change it?  [n] y
Supply a decimal value for "sysid (165=FreeBSD)"  [0] 6
Supply a decimal value for "start"  [0]    Enter pressed Supply a decimal value for "size"  [0] 2091453
Explicitly specify beg/end address ?  [n]    Enter pressed
sysid 6,(Primary 'big' DOS (> 32MB))
   start 0, size 2091453 (1021 Meg), flag 0 
      beg: cyl 0/ head 0/ sector 1;
      end: cyl 350/ head 12/ sector 47
Are we happy with this entry?  [n] y
The data for partition 2 is:
<UNUSED>
Do you want to change it?  [n] y
Supply a decimal value for "sysid (165=FreeBSD)"  [0] 165 Supply a decimal value for "start"  [0] 2091453 
Supply a decimal value for "size"  [0] 6295133 
Explicitly specify beg/end address ?  [n]    Enter pressed
sysid 165,(FreeBSD/NetBSD/386BSD)
start 2091453, size 6295133  (3073 Meg), flag 0
   beg: cyl 351/ head 0/ sector 1;
   end: cyl 413/ head 12/ sector 47
Are we happy with this entry?  [n] y
The data for partition 3 is:
<UNUSED>
Do you want to change it?  [n] Enter pressed
The data for partition 4 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
    start 47, size 8386539 (4094 Meg), flag 80 (active)
      beg: cyl 0/ head 1/ sector 1;
      end: cyl 413/ head 12/ sector 47 
Do you want to change it?  [n] y
The static data for the DOS partition 4 has been reinitialized to:
sysid 165,(FreeBSD/NetBSD/386BSD)
start 47, size 8386539 (4094 Meg), flag 80 (active)
    beg: cyl 0/ head 1/ sector 1;
    end: cyl 413/ head 12/ sector 47
Supply a decimal value for "sysid (165=FreeBSD)"  [165] 0 Supply a decimal value for "start"  [47] 0 
Supply a decimal value for "size"  [8386539] 0 
Explicitly specify beg/end address ?  [n] Enter pressed 
<UNUSED>
Are we happy with this entry?  [n] y 
Do you want to change the active partition?  [n] y 
Supply a decimal value for "active partition"  [1] 2 
Are you happy with this choice [n] y

We haven't changed the partition table yet.   This is your last chance, parameters extracted from in-core disklabel are: 
cylinders=13726 heads=13 sectors/track=47 (611 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are:
cylinders=13726 heads=13 sectors/track=47 (611 blks/cyl)

Information from DOS bootblack is:
1: sysid 6,(Primary 'big' DOS (> 32MB))
    start 0, size 2091453 (1021 Meg), flag 0
      beg: cyl 0/ head 0/ sector 1;
      end: cyl 350/ head 12/ sector 47 
2: sysid 165,(FreeBSD/NetBSD/386BSD)
    start 2091453, size 6295133  (3073 Meg), flag 80 (active)
      beg: cyl 351/ head 0/ sector 1;
      end: cyl 413/ head 12/ sector 47 
3: <UNUSED> 
4: <UNUSED>
Should we write new partition table?  [n] y

You'll notice a couple of things here:

Labeling the disk

Once we have a valid PC BIOS partition table, we need to create the file systems. We won't look at the Microsoft partition in anymore detail, but we still need to do some more work on our FreeBSD slice (slice or PC BIOS partition 2). It'll make life easier here to remember a couple of things:

The first thing we need is the disk (slice) label, which supplies general information about the slice:

The only information we need to input is the kind, size and locations of the partitions. In this case, we have decided to create a file system on partition h (/dev/da1s2h) and swap space on partition b (/dev/dalslb). The swap space will be 512 MB, and the file system will take up the rest of the slice. This is mainly tradition: traditionally data disks use the h partition and not the a partition, so we'll stick to that tradition, though there's nothing to stop you from using the a partition if you prefer. In addition, we need to define the c partition, which represents the whole slice. In summary, the FreeBSD slice we want to create looks like:

11.2. TABLE FreeBSD slice on second disk
/dev/da1s2b: FreeBSD swap, 512 MB
/dev/da1s2h: /newhome file system, 2.5 GB

bsdlabel

The program that writes the disk label used to be called disklabel As FreeBSD migrated to multiple platforms, this proved to be too generic: many hardware platforms have their own disk label formats. For example, FreeBSD on SPARC64 uses the Sun standard labels. On platforms which use the old BSD labels, such as the PC, the name was changed to bsdlabel. On SPARC64 it is called sunlabel. On each platform, the appropriate file is linked to the name disklabel, but some of the options have changed. In addition, the output format now normally ignores a number of historical relics. It's not as warty as fdisk, but it can still give you a run for your money. You can usually ignore most of the complexity, though. You can normally create a disk label with the single command:

# bsdlabel -w /dev/da1s2 auto

This creates the label with a single partition c. You can look at the label with bsdlabel without options:

#bsdlabel /dev/da1s2
#/dev/da0s2:
8partitions:
#       size  offset  fstype  [fsize bsize bps/cpg]
 c:  6295133       0  unused     0     0          # "raw" part, don't edit

At this point, the only partition you have is the "whole disk" partition c. You still need to create partitions b and h and specify their location and size. Do this with bsdlabel -e, which starts an editor with the output you see above. Simply add additional partitions:

8partitions:
#       size   offset  fstype   [fsize bsize bps/cpg]
 c:  6295133        0  unused      0     0  # "raw" part, don't edit
 b:  1048576        0  swap        0     0
 h:  5246557  1048576  unused      0     0

You don't need to maintain any particular order, and you don't need to specify that partition h will be a file system. In the next step , newfs does that for you automatically.

Problems running biddable

Using the old disklabel program used to be like walking through a minefield. Things have got a lot better, but it's possible that some problems are still hiding. Here are some of the problems that have been encountered in the past, along with some suggestions about what to do if you experience them:

Creating file systems

Once we have a valid label, we need to create the file systems. In this case, there's only one file system, on /dev/da1s2/z. Mercifully, this is easier:

# newfs -U /dev/da1s2h
/dev/vinum/da1s2h: 2561.8MB (5246556 sectors) block size 16384, fragment size 2048 
    using 14 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
    with soft updates
super-block backups (for fsck -b #)at:
160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328,
3763680, 4140032, 4516384, 4892736

The -U flag tells newfs to enable soft updates, which we looked at on page 191.

Mounting the file systems

Finally the job is done. Well, almost. You still need to mount the file system, and to tell the system that it has more swap. But that's not much of a problem:

# mkdir /newhome             make sure we have a directory to mount on
# mount /dev/da1s2h/newhome  and mount it
# swapon /dev/da1s2b
# df                         show free capacity and mounted file systems
Filesystem    1024-blocks     Used    Avail  Capacity  Mounted on
/dev/ad0s1a         19966    17426      944     95%    /
/dev/ad0s1e       1162062   955758   113340     89%    /usr
procfs                  4        4        0    100%    /proc
presto:/            15823     6734     8297     45%    /presto/root
presto:/usr        912271   824927    41730     95%    /presto/usr
presto:/home      1905583  1193721   521303     70%    /presto/home
presto:/S         4065286  3339635   563039     86%    /S
/dev/da1s2h       2540316        2  2337090      0%    /newhome
# pstat -s                   show  swap usage
Device          1K-blocks     Used    Avail  Capacity  Type
/dev/ad0s4b        524160        0   524160      0%    Interleaved
/dev/da1s2b        524160        0   524160      0%    Interleaved
Total             1048320        0  1048320      0%

This looks fine, but when you reboot the system, /newhome and the additional swap will be gone. To ensure that they get mounted after booting, you need to add the following lines to /etc/fstab:

/dev/da1s2b  none      swap  sw  0  0
/dev/da1s2h  /newhome  ufs   rw  0  0

Moving file systems

Very frequently, you add a new disk to a system because existing disks have run out of space. Let's consider the disk we have just added and assume that currently the files in /home are physically located on the /usr file system, and that /home is a symbolic link to /usr/home. We want to move them to the new file system and then rename it to /home. Here's what to do:

Recovering from disk data errors

Modern hard disks are a miracle in evolution. Today you can buy a 200 GB hard disk for under $200, and it will fit in your shirt pocket. Thirty years ago, a typical disk drive was the size of a washing machine and stored 20 MB. You would need 10,000 of them to store 200 GB.

At the same time, reliability has gone up, but disks are still relatively unreliable devices. You can achieve maximum reliability by keeping them cool, but sooner or later you are going to run into some kind of problem. One kind is due to surface irregularities: the disk can't read a specific part of the surface.

Modern disks make provisions for recovering from such errors by allocating an alternate sector for the data. IDE drives do this automatically, but with SCSI drives you have the option of enabling or disabling reallocation. Usually reallocation is enabled when you buy the disk, but occasionally it is not. When installing a new disk, you should check that the parameters ARRE (Auto Read Reallocation Enable) and AWRE (Auto Write Reallocation Enable) are turned on. For example, to check and set the values for disk /dev/da1 , you would enter:

#  camcontrol modepage dal -m 1 -e

This command will start up your favorite editor (either the one specified in the EDITOR environment variable, or vi by default) with the following data:

AWRE (Auto Write Reallocation Enbld): 0
ARRE (Auto Read Reallocation Enbld): 0
TB (Transfer Block): 1
EER (Enable Early Recovery): 0
PER (Post Error): 1
DTE (Disable Transfer on Error): 0
DCR (Disable Correction): 0
Read Retry Count: 41
Write Retry Count: 24

The values for AWRE and ARRE should both be 1. If they aren't, as in this case, where AWRE is 0, change the data with the editor, write it back, and exit. camcontrol writes the data back to the disk and enables the option.

Note the last two lines in this example. They give the number of actual retries that this drive has performed. You can reset these values too if you want; they will be updated if the drive performs any additional retries.

12. The Vinum Volume Manager

Vinum objects; Creating Vinum drives; Starting Vinum; Configuring Vinum; Vinum configuration database; Installing FreeBSD on Vinum; Recovering from drive failures; Migrating Vinum to a new machine; Things you shouldn't do with Vinum.

Vinum is a Volume Manager, a virtual disk driver that addresses these three issues:

From a user viewpoint, Vinum looks almost exactly the same as a disk, but in addition to the disks there is a maintenance program.

Vinum objects

Vinum implements a four-level hierarchy of objects:

Plexes can include multiple subdisks spread overall drives in the Vinum configuration, so the size of an individual drive does not limit the size of a plex, and thus of a volume.

Mapping disk space to plexes

The way the data is shared across the drives has a strong influence on performance. It's convenient to think of the disk storage as a large number of data sectors that are addressable by number, rather like the pages in a book. The most obvious method is to divide the virtual disk into groups of consecutive sectors the size of the individual physical disks and store them in this manner, rather like the way a large encyclopaedia is divided into a number of volumes. This method is called concatenation, and sometimes JBOD (Just a Bunch Of Disks). It works well when the access to the virtual disk is spread evenly about its address space. When access is concentrated on a smaller area, the improvement is less marked. Figure 12-1 illustrates the sequence in which storage units are allocated in a concatenated organization.

Concatenated organization


. 12.1. Concatenated organization

An alternative mapping is to divide the address space into smaller, equal-sized components, called stripes, and store them sequentially on different devices. For example, the first stripe of 292 kB may be stored on the first disk, the next stripe on the next disk and so on. After filling the last disk, the process repeats until the disks are full. This mapping is called striping or RAID-01), though the latter term is somewhat misleading: it provides no redundancy. Striping requires somewhat more effort to locate the data, and it can cause additional I/O load where a transfer is spread over multiple disks, but it can also provide a more constant load across the disks. Figure 12-2 illustrates the sequence in which storage units are allocated in a striped organization.

Striped organization


. 12.2. Striped organization

Data integrity

Vinum offers two forms of redundant data storage aimed at surviving hardware failure: mirroring, also known as RAID level1, and parity, also known as RAID levels 2 to 5.

Mirroring maintains two or more copies of the data on different physical hardware. Any write to the volume writes to both locations; a read can be satisfied from either, so if one drive fails, the data is still available on the other drive. It has two problems:

The most interesting of the parity solutions is RAID level5, usually called RAID-5. The disk layout is similar to striped organization, except that one block in each stripe contains the parity of the remaining blocks. The location of the parity block changes from one stripe to the next to balance the load on the drives. If anyone drive fails, the driver can reconstruct the data with the help of the parity information. If one drive fails, the array continues to operate in degraded mode: a read from one of the remaining accessible drives continues normally, but a read request from the failed drive is satisfied by recalculating the contents from all the remaining drives. Writes simply ignore the dead drive. When the drive is replaced, Vinum recalculates the contents and writes them back to the new drive.

In the following figure, the numbers in the data blocks indicate the relative block numbers.

RAID-5 organization


. 12.3. RAID-5 organization

Compared to mirroring, RAID-5 has the advantage of requiring significantly less storage space. Read access is similar to that of striped organizations, but write access is significantly slower, approximately 25% of the read performance.

Vinum also offers RAID-4, a simpler variant of RAID-5 which stores all the parity blocks on one disk. This makes the parity disk a bottleneck when writing. RAID-4 offers no advantages overRAID-5, so it's effectively useless.

Which plex organization?

Each plex organization has its unique advantages:

The following table summarizes the advantages and disadvantages of each plex organization.

12.1. Vinum plex organizations
Plex typeMinimum subdisksCan add subdisksMust be equal sizeApplication
concatenated1yesnoLarge data storage with maximum placement flexibility and moderate performance.
striped2noyesHigh performance in combination with highly concurrent access.
RAID-53noyesHighly reliable storage, primarily read access.

Creating Vinum drives

Before you can do anything with Vinum, you need to reserve disk space for it. Vinum drive objects are in fact a special kind of disk partition, of type vinum. We've seen how to create disk partitions on page 215. If in that example we had wanted to create a Vinum volume instead of a UFS partition, we would have created it like this:

8 partitions:
#        size   offset  fstype  [fsize bsize bps/cpg]
  c:  6295133        0  unused       0     0         # (Cyl. 0 - 10302)
  b:  1048576        0  swap         0     0         # (Cyl. 0 - 10302)
  h:  5246557  1048576  vinum        0     0         # (Cyl. 0 - 10302)

Starting Vinum

Vinum comes with the base system as a kld. It gets loaded automatically when you run the vinum command. It's possible to build a special kernel that includes Vinum, but this is not recommended: in this case, you will not be able to stop Vinum.

FreeBSD Release 5 includes a new method of starting Vinum. Put the following lines in

/boot/loader.conf:

vinum_load="YES"
vinum.autostart="YES"

The first line instructs the loader to load the Vinum kld, and the second tells it to start Vinum during the device probes. Vinum still supports the older method of setting the variable start_vinum in /etc/rc.conf, but this method may go away soon.

Configuring Vinum

Vinum maintains a configuration database that describes the objects known to an individual system. You create the configuration database from one or more configuration files with the aid of the vinum utility program. Vinum stores a copy of its configuration database on each Vinum drive. This database is updated on each state change, so that a restart accurately restores the state of each Vinum object.

The configuration file

The configuration file describes individual Vinum objects. To define a simple volume, you might create a file called, say, configl, containing the following definitions:

drive a device /dev/da1s2h
volume myvol
plex org concat
sd length 512m drive a

This file describes four Vinum objects:

After processing this file, vinum(8) produces the following output:

vinum -> create config1
1 drives:
D a              State: up  /dev/da1s2h  A: 3582/4094 MB (87%)

1 volumes:
V myvol          State: up  Plexes:    1 Size:        512 MB

1 plexes:
P myvol.p0     C State: up  Subdisks:  1 Size:        512 MB
1 subdisks:
S myvol.p0.s0    State: up  D: a         Size:        512 MB

This output shows the brief listing format of vinum. It is represented graphically in Figure 12-4 .

A simple Vinum volume


. 12.4. A simple Vinum volume

This figure, and the ones that follow, represent a volume, which contains the plexes, which in turn contain the subdisks. In this trivial example, the volume contains one plex, and the plex contains one subdisk.

Creating a file system

You create a file system on this volume in the same way as you would for a conventional disk:

# newfs -U /dev/vinum/myvol
/dev/vinum/myvol: 512.0MB (1048576 sectors) block size 16384, fragment size 2048
  using 4 cylinder groups of 128.02MB, 8193 blks, 16512 inodes.
super-block backups (for fsck -b #) at:
  32, 262208, 524384, 786560

This particular volume has no specific advantage over a conventional disk partition. It contains a single plex, so it is not redundant. The plex contains a single subdisk, so there is no difference in storage allocation from a conventional disk partition. The following sections illustrate various more interesting configuration methods.

Increased resilience: mirroring

The resilience of a volume can be increased either by mirroring or by using RAID-5 plexes. When laying out a mirrored volume, it is important to ensure that the subdisks of each plex are on different drives, so that a drive failure will not take down both plexes. The following configuration mirrors a volume:

drive b device /dev/da2s2h
volume mirror
  plex org concat
    sd length 512m drive a
  plex org concat
    sd length 512m drive b

In this example, it was not necessary to specify a definition of drive a again, because Vinum keeps track of all objects in its configuration database. After processing this definition, the configuration looks like:

2 drives:
D a             State: up            /dev/da1s2h  A: 3070/4094 MB (74%)
D b             State: up            /dev/da2s2h  A: 3582/4094 MB (87%)

2 volumes:
V myvol         State: up            Plexes:    1 Size:        512 MB
V mirror        State: up            Plexes:    2 Size:        512 MB

3 plexes:
P myvol.p0    C State: up            Subdisks:  1 Size:        512 MB
P mirror.p0   C State: up            Subdisks:  1 Size:        512 MB
P mirror.p1   C State: initializing  Subdisks:  1 Size:        512 MB

3 subdisks:
S myvol.p0.s0   State: up            D: a         Size:        512 MB
S mirror.p0.s0  State: up            D: a         Size:        512 MB
S mirror.p1.s0  State: empty         D: b           Size:      512 MB

. 12.5shows the structure graphically.

In this example, each plex contains the full 512 MB of address space. As in the previous example, each plex contains only a single subdisk.

Note the state of mirror.p1 and mirror.p1.s0: initializing and empty respectively. There's a problem when you create two identical plexes: to ensure that they're identical, you need to copy the entire contents of one plex to the other. This process is called reviving, and you perform it with the start command:

vinum -> start mirror.pl
vinum[278]: reviving mirror.p1.s0
Reviving mirror.p1.s0 in the background
vinum -> vinum[278]: mirror.p1.s0 is up

A mirrored Vinum volume


. 12.5. A mirrored Vinum volume

During the start process, you can look at the status to see how far the revive has progressed:

vinum -> list mirror.pl.s0
Smirror.pl.s0   State: R43%      D: bSize:   512 MB

Reviving a large volume can take a very long time. When you first create a volume, the contents are not defined. Does it really matter if the contents of each plex are different? If you will only ever read what you have first written, you don't need to worry too much. In this case, you can use the setupstate keyword in the configuration file. We'll see an example of this below.

Adding plexes to an existing volume

At some time after creating a volume, you may decide to add additional plexes. For example, you may want to add a plex to the volume myvol we saw above, putting its subdisk on drive b. The configuration file for this extension would look like:

plex name myvol.p1 org concat volume myvol
  sd size 1g drive b

To see what has happened, use the recursive listing option -r for the list command:

vinum -> l-r myvol
V myvol          State: up            Plexes:    2 Size:  1024 MB
P myvol.p0     C State: up            Subdisks:  1 Size:   512 MB
P myvol.p1     C State: initializing  Subdisks:  1 Size:  1024 MB
S myvol.p0.s0    State: up            D: a         Size:   512 MB
S myvol.p1.s0    State: empty         D: b         Size:  1024 MB

The command l is a synonym for list, and the -r option means recursive: it displays all subordinate objects. In this example, plex myvol.p1 is 1 GB in size, although myvol.p0 is only 512 MB in size. This discrepancy is allowed, though it isn't very useful by itself: only the first half of the volume is protected against failures. As we'll see in the next section, though, this is a useful stepping stone to extending the size of a file system.

Note that you can't use the setupstate keyword here. Vinum can't know whether the existing volume contains valid data or not, so you must use the start command to synchronize the plexes.

Adding subdisks to existing plexes

After adding a second plex to myvol, it had one plex with 512 MB and another with 1024 MB. It makes sense to have the same size plexes, so the first thing we should do is add a second subdisk to the plex myvol.p0.

If you add subdisks to striped, RAID-4 or RAID-5 plexes, you will change the mapping of the data to the disks, which effectively destroys the contents. As a result, you must use the -f option. When you add subdisks to concatenated plexes, the data in the existing subdisks remains unchanged. In our case, the plex is concatenated, so we create and add the subdisk like this:

sd name myvol.p0.s1 plex myvol.p0 size 512m drive c

After adding this subdisk, the volume looks like this:

An extended Vinum volume


. 12.6. An extended Vinum volume

It doesn't look too happy, however:

vinum -> l -r myvol
V myvol          State: up            Plexes:    2 Size:  1024 MB
P myvol.p0     C State: corrupt       Subdisks:  2 Size:  1024 MB
P myvol.p1     C State: initializing  Subdisks:  1 Size:  1024 MB
S myvol.p0.s0    State: up            D: a         Size:   512 MB
S myvol.p0.s1    State: empty         D: c         Size:   512 MB
S myvol.p1.s0    State: stale         D: b         Size:  1024 MB

In fact, it's in as good a shape as it ever has been. The first half of myvol still contains the file system that we put on it, and it's as accessible as ever. The trouble here is that there is nothing in the other two subdisks, which are shown shaded in the figure. Vinum can't know that that is acceptable, but we do. In this case, we use some maintenance commands to set the correct object states:

vinum -> setstate up myvol.p0.s1 myvol.p0
vinum -> l -r myvol
V myvol          State: up      Plexes:    2 Size:  1024 MB
P myvol.p0     C State: up      Subdisks:  2 Size:  1024 MB 
P myvol.p1     C State: faulty  Subdisks:  1 Size:  1024 MB
S myvol.p0.s0    State: up      D:           Size:   512 MB
S myvol.p0.s1    State: up      D:           Size:   512 MB
S myvol.p1.s0    State: stale   D:           Size:  1024 MB
vinum -> saveconfig

The command setstate changes the state of individual objects without updating those of related objects. For example, you can use it to change the state of a plex to up even if all the subdisks are down. If used incorrectly, it can can cause severe data corruption. Unlike normal commands, it doesn't save the configuration changes, so you use saveconfig for that, after you're sure you have the correct states. Read the man page before using them for any other purpose.

Next you start the second plex:

vinum -> start myvol.p1
Reviving myvol.p1.s0 in the background
vinum[446]: reviving myvol.p1.s0
vinum -> vinum[446]: myvol.p1.s0 is up    some time later
l                                            command for previous prompt 
3 drives:
D a              State: up  /dev/da1s2h    A: 3582/4094 MB (87%)
D b              State: up  /dev/da2s2h    A: 3070/4094 MB (74%)
D c              State: up  /dev/da3s2h    A: 3582/4094 MB (87%)

1 volumes:
V myvol          State: up  Plexes:      2 Size:   1024 MB

2 plexes:
P myvol.p0     C State: up  Subdisks:    2 Size:   1024 MB
P myvol.p1     C State: up  Subdisks:    1 Size:   1024 MB

3 subdisks:
S myvol.p0.s0    State: up  D: a           Size    512 MB
S myvol.p1.s0    State: up  D: b           Size   1024 MB
S myvol.p0.s1    State: up  D: c           Size    512 MB

The message telling you that myvol.p1.s0 is up comes after the prompt, so the next command doesn't have a prompt. At this point you have a fully mirrored, functional volume, 1 GB in size. If you now look at the contents, though, you see:

# df /mnt
Filesystem        1048576-blocks  Used  Avail  Capacity  Mounted on
/dev/vinum/myvol             503     1    461        0%  /mnt

The volume is now 1 GB in size, but the file system on the volume is still only 512 MB. To expand it, use growfs:

# umount /mnt
# growfs /dev/vinum/myvol
We strongly recommend you to make a backup before growing the Filesystem
 Did you backup your data (Yes/No) ? Yes
new file systemsize is: 524288 frags
Warning: 261920 sector(s) cannot be allocated.
growfs: 896.1MB (1835232 sectors) block size 16384, fragment size 2048
        using 7 cylinder groups of 128.02MB, 8193 blks, 16512 inodes.
super-block backups (for fsck -b #) at:
 1048736, 1310912, 1573088
# mount /dev/vinum/myvol /mnt
# df /mnt
Filesystem        1048576-blocks  Used  Avail  Capacity  Mounted on
/dev/vinum/myvol             881     1    809        0%  /mnt

Optimizing performance

The mirrored volumes in the previous example are more resistant to failure than unmirrored volumes, but their performance is less: each write to the volume requires a write to both drives, using up a greater proportion of the total disk bandwidth. Performance considerations demand a different approach: instead of mirroring, the data is striped across as many disk drives as possible. The following configuration shows a volume with a plex striped across four disk drives:

drive c device /dev/da3s2h
drive d device /dev/da4s2h
volume stripe
  plex org striped 480k
    sd length 128m drive a
    sd length 128m drive b
    sd length 128m drive c
    sd length 128m drive d

When creating striped plexes for the UFS file system, ensure that the stripe size is a multiple of the file system block size (normally 16 kB), but not a power of 2. UFS frequently allocates cylinder groups with lengths that are a power of 2, and if you allocate stripes that are also a power of 2, you may end up with all in odes on the same drive, which would significantly impact performance under some circumstances. Files are allocated in blocks, so having a stripe size that is not a multiple of the block size can cause significant fragmentation of I/O requests and consequent drop in performance. See the man page for more details.

Vinum requires that a striped plex have an integral number of stripes. You don't have to calculate the size exactly, though: if the size of the plex is not a multiple of the stripe size, Vinum trims off the remaining partial stripe and prints a console message:

vinum: removing 256 blocks of partial stripe at the end of stripe.p0

As before, it is not necessary to define the drives that are already known to Vinum. After processing this definition, the configuration looks like:

4 drives:                
D a               State: up  /dev/da1s2h            A: 2942/4094 MB  (71%)
D b               State: up  /dev/da2s2h            A: 2430/4094 MB  (59%)
D c               State: up  /dev/da3s2h            A: 3966/4094 MB  (96%)
D d               State: up  /dev/da4s2h            A: 3966/4094 MB  (96%)

3 volumes:                
V myvol           State: up            Plexes:    2 Size:  1024 MB
V mirror          State: up            Plexes:    2 Size:   512 MB
V stripe          State: up            Plexes:    1 Size:   511 MB

5 plexes:                
P myvol.p0      C State: up            Subdisks:  1 Size:   512 MB
P mirror.p0     C State: up            Subdisks:  1 Size:   512 MB
P mirror.p1     C State: initializing  Subdisks:  1 Size:   512 MB
P myvol.p1      C State: up            Subdisks:  1 Size:  1024 MB
P stripe.p0     S State: up            Subdisks:  4 Size:   511 MB

8 subdisks:                  
S myvol.p0.s0     State: up            D: a         Size:   512 MB
S mirror.p0.s0    State: up            D: a         Size:   512 MB
S mirror.p1.s0    State: empty         D: b         Size:   512 MB
S myvol.p1.s0     State: up            D: b         Size:  1024 MB
S myvol.p0.s1     State: up            D: c         Size:   512 MB
S stripe.p0.s0    State: up            D: a         Size:   127 MB
S stripe.p0.s1    State: up            D: b         Size:   127 MB
S stripe.p0.s2    State: up            D: c         Size:   127 MB
S stripe.p0.s3    State: up            D: d         Size:   127 MB

This volume is represented in Figure 12-7. The darkness of the stripes indicates the position within the plex address space: the lightest stripes come first, the darkest last.

A striped Vinum volume


. 12.7. A striped Vinum volume

Resilience and performance

With sufficient hardware, it is possible to build volumes that show both increased resilience and increased performance compared to standard UNIX partitions. Mirrored disks will always give better performance than RAID-5, so a typical configuration file might be:

drive e device /dev/da5s2h
drive f device /dev/da6s2h
drive g device /dev/da7s2h
drive h device /dev/da8s2h
drive i device /dev/da9s2h
drive j device /dev/da10s2h
volume raid10 setupstate
  plex org striped 480k
    sd length 102480k drive a
    sd length 102480k drive b
    sd length 102480k drive c
    sd length 102480k drive d
    sd length 102480k drive e
  plex org striped 480k
    sd length 102480k drive f
    sd length 102480k drive g
    sd length 102480k drive h
    sd length 102480k drive i
    sd length 102480k drive j

In this example, we have added another five disks for the second plex, so the volume is spread over ten spindles. We have also used the setupstate keyword so that all components come up. The volume looks like this:

vinum -> l -r raid10
V raid10          State: up  Plexes:    2 Size:  499 MB
P raid10.p0     S State: up  Subdisks:  5 Size:  499 MB
P raid10.p1     S State: up  Subdisks:  5 Size:  499 MB
S raid10.p0.s0    State: up  D: a         Size:   99 MB
S raid10.p0.s1    State: up  D: b         Size:   99 MB
S raid10.p0.s2    State: up  D: c         Size:   99 MB
S raid10.p0.s3    State: up  D: d         Size:   99 MB
S raid10.p0.s4    State: up  D: e         Size:   99 MB
S raid10.p1.s0    State: up  D: f         Size:   99 MB
S raid10.p1.s1    State: up  D: g         Size:   99 MB
S raid10.p1.s2    State: up  D: h         Size:   99 MB
S raid10.p1.s3    State: up  D: i         Size:   99 MB
S raid10.p1.s4    State: up  D: j         Size:   99 MB

This assumes the availability of ten disks. It's not essential to have all the components on different disks. You could put the subdisks of the second plex on the same drives as the subdisks of the first plex. If you do so, you should put corresponding subdisks on different drives:

plex org striped 480k
  sd length 102480k drive a
  sd length 102480k drive b
  sd length 102480k drive c
  sd length 102480k drive d
  sd length 102480k drive e
plex org striped 480k
  sd length 102480k drive c
  sd length 102480k drive d
  sd length 102480k drive e
  sd length 102480k drive a
  sd length 102480k drive b

The subdisks of the second plex are offset by two drives from those of the first plex: this helps ensure that the failure of a drive does not cause the same part of both plexes to become unreachable, which would destroy the file system.

Figure 12-8 represents the structure of this volume.

A mirrored, striped Vinum volume


. 12.8. A mirrored, striped Vinum volume

Vinum configuration database

Vinum stores configuration information on each drive in essentially the same form as in the configuration files. You can display it with the dumpconfig command. When reading from the configuration database, Vinum recognizes a number of keywords that are not allowed in the configuration files, because they would compromise data integrity. For example, after adding the second plex to myvol, the disk configuration would contain the following text:

vinum -> dumpconfig
Drive a:  Device /dev/da1s2h
          Created on bumble.example.org at Tue Nov 26 14:35:12 2002
          Config last updated Tue Nov 26 16:12:35 2002
          Size:   4293563904 bytes  (4094 MB)
volume myvol state up
plex name myvol.p0 state up org concat vol myvol
plex name myvol.p1 state up org concat vol myvol
sd name myvol.p0.s0 drive a plex myvol.p0 len 1048576s driveoffset 265s state up plexoffset 0s
sd name myvol.p1.s0 drive b plex myvol.p1 len 2097152s driveoffset 265s state up plexoffset 0s
sd name myvol.p0.s1 drive c plex myvol.p0 len 1048576s driveoffset 265s state up plexoffset 1048576s

Drive /dev/da1s2h: 4094 MB (4293563904 bytes)
Drive b:  Device /dev/da2s2h
          Created on bumble.example.org at Tue Nov 26 14:35:27 2002
          Config last updated Tue Nov 26 16:12:35 2002
          Size:   4293563904 bytes (4094 MB)
volume myvol state up
plex name myvol.p0 state up org concat vol myvol
plex name myvol.p1 state up org concat vol myvol
sd name myvol.p0.s0 drive a plex myvol.p0 len 1048576s driveoffset 265s state up plexoffset 0s
sd name myvol.p1.s0 drive b plex myvol.p1 len 2097152s driveoffset 265s state up plexoffset 0s
sd name myvol.p0.s1 drive c plex myvol.p0 len 1048576s driveoffset 265s state up plexoffset 1048576s

The obvious differences here are the presence of explicit location information and naming (both of which are also allowed, but discouraged, for use by the user) and the information on the states (which are not available to the user). Vinum does not store information about drives in the configuration information: it finds the drives by scanning the configured disk drives for partitions with a Vinum label. This enables Vinum to identify drives correctly even if they have been assigned different UNIX drive IDs.

When you start Vinum with the vinum start command, Vinum reads the configuration database from one of the Vinum drives. Under normal circumstances, each drive contains an identical copy of the configuration database, so it does not matter which drive is read. After a crash, however, Vinum must determine which drive was updated most recently and read the configuration from this drive. It then updates the configuration, if necessary, from progressively older drives.

Installing FreeBSD on Vinum

Installing FreeBSD on Vinum is complicated by the fact that sysinstall and the loader don't support Vinum, so it is not possible to install directly on a Vinum volume. Instead, you need to install a conventional system and then convert it to Vinum. That's not as difficult as it might sound.

A typical disk installation lays out disk partitions in the following manner:

Typical partition layout without Vinum
da0s3a:/file sistemda0s3c: entire disk
da0s3d:swap
da0s3e:/usr file sistem
da0s3f:/var file sistem

This layout shows three file system partitions and a swap partition, which is not the layout recommended on page 68. We'll look at the reasons for this below.

Each partition corresponds logically to a Vinum subdisk. You could enclose all these subdisks in a Vinum drive. The only problem is that Vinum stores its configuration information at the beginning of the drive, and that's where the root file system is. One way to solve this problem is to put the swap partition first and make it 265 sectors longer than needed. You can do this from sysinstall simply by creating the swap partition before any other partition. Consider installing FreeBSD on a 4 GB drive. Create, in sequence, a swap partition of 256 MB, a root file system of 256 MB, a /usr file system of 2 GB, and a /var file system to take up the rest. It's important to create the swap partition at the beginning of the disk, so you create that first. After installation, the output of bsdlabel looks like this:

8 partitions:
#        size   offset  fstype  [fsize  bsize bps/cpg]
  a:   524288   532480  4.2BSD    2048  16384   94
  b:   532215      265    swap
  c:  8386733        0  unused       0      0     #"raw" part, don't edit
  e:  4194304  1056768  4.2BSD    2048  16384   89
  f:  3135661  5251072  4.2BSD    2048  16384   89

To convert to Vinum, use bsdlabel with the -e (edit label) option to create a volume of type vinum that maps the c partition:

h: 8386733

After this, you have the following situation:

Partition layout with Vinum
da0s3b: swapda0s3c: entire diskda0s3h: vinum drive
da0s3a:/file system
da0s3a:/usr file
da0s3f:/var file sistem

The shaded area at the top of the Vinum partition represents the configuration information, which cuts into the swap partition. To fix that, we redefine the swap partition to start after the Vinum configuration information and to be 265 sectors shorter. The file systems are relatively trivial to recreate: take the size and offset values from the bsdlabel output above and use them in a Vinum configuration file:

drive rootdev device /dev/da0s2h
volume swap
  plex org concat
#  b:     532215                  265          swap
  sd len  532215s    driveoffset  265s      drive rootdev
volume root
  plex org concat
#  a:     524288                  532480       4.2BSD  2048  16384  94
  sd len  524288s    driveoffset  532480s   drive rootdev
volume usr
  plex org concat
#  e:     4194304                 1056768      4.2BSD  2048  16384  89
  sd len  4194304s   driveoffset  1056768s   drive rootdev
volume var
  plex org concat
#  f:     3135661                 5251072      4.2BSD  2048  16384  89
  sd len  3135661s   driveoffset  5251072s
drive   rootdev

The comments are the corresponding lines from the bsdlabel output. They show the corresponding values for size and offset. Run vinum create against this file, and confirm that you have the volumes /, /usr and /var.

Next, ensure that you are set up to start Vinum with the new method. You should have the following lines in /boot/loader.conf:

vinum_load="YES"
vinum.autostart="YES"

Then reboot to single-user mode, start Vinum and run fsck against the volumes, using the -n option to tell fsck not to correct any errors it finds. You should see something like this:

# fsck -n -t ufs /dev/vinum/usr
** /dev/vinum/usr (NO WRITE)
** Last Mounted on /usr
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
35323 files, 314115 used, 718036 free (4132 frags, 89238 blocks, 0.4% fragmentation)

If there are any errors, they will probably be because you have miscalculated size or offset. You'll see something like this:

# fsck -n -t ufs /dev/vinum/usr
** /dev/vinum/usr (NO WRITE) 
Cannot find file system superblock
/dev/vinum/usr: CANNOT FIGURE OUT FILE SYSTEM PARTITION

You need to do this in single-user mode because the volumes are shadowing file systems, and it's normal for open file systems to fail fsck, since some of the state is in buffer cache.

If all is well, remount the root file system read-write:

# mount -u /

Then edit /etc/fstab to point to the new devices. For this example, /etc/fstab might initially contain:

# $Id: fstab,v 1.3 2002/11/14 06:48:16 grog Exp $
# Device     Mountpoint  FStype  Options  Dump Pass#
/dev/da0s4a  /           ufs     rw       1    1
/dev/da0s4b  none        swap    sw       0    0
/dev/da0s4e  /usr        ufs     rw       1    1
/dev/da0s4f  /var        ufs     rw       1    1

Change it to reflect the Vinum volumes:

# $Id: fstab,v 1.3 2002/11/14 06:48:16 grog Exp $  
# Device         Mountpoint  FStype  Options  Dump Pass#
/dev/vinum/swap  none        swap    sw       0    0
/dev/vinum/root  /           ufs     rw       1    1
/dev/vinum/usr   /usr        ufs     rw       1    1
/dev/vinum/var   /var        ufs     rw       1    1
/dev/da0s4b      none        swap    sw       0    0
/dev/da0s4e      /usr        ufs     rw       1    1
/dev/da0s4f      /var        ufs     rw       1    1

Then reboot again to mount the root file system from /dev/vinum/root. You can also optionally remove all the UFS partitions except the root partition. The loader doesn't know about Vinum, so it must boot from the UFS partition.

Once you have reached this stage, you can add additional plexes to the volumes, or you can extend the plexes (and thus the size of the file system) by adding subdisks to the plexes, as discussed on page 229.

Recovering from drive failures

One of the purposes of Vinum is to be able to recover from hardware problems. If you have chosen a redundant storage configuration, the failure of a single component will not stop the volume from working. In many cases, you can replace the components without down time.

If a drive fails, perform the following steps:

  1. Replace the physical drive.
  2. Partition the new drive. Some restrictions apply:
    • If you have hot-plugged the drive, it must have the same ID, the Vinum drive must be on the same partition, and it must have the same size.
    • If you have had to stop the system to replace the drive, the old drive will not be associated with a device name, and you can put it anywhere. Create a Vinum partition that is at least large enough to take all the subdisks in their original positions on the drive. Vinum currently does not compact free space when replacing a drive. An easy way to ensure this is to make the new drive at least as large as the old drive.

      If you want to have this freedom with a hot-pluggable drive, you must stop Vinum and restart it.

  3. If you have restarted Vinum, create a new drive. For example, if the replacement drive data3 is on the physical partition /dev/da3s1h , create a configuration file, say configfile, with the single line
    drive data3 device /dev/da3s1h
    

    Then enter:

    # vinum create configfile
    
  4. Start the plexes that were down. For example, vinum list might show:
    vinum -> l -r test
    V test         State: up        Plexes:       2 Size:  30 MB
    Ptest.p0     C State: up        Subdisks:     1 Size:  30 MB
    Ptest.p1     C State: faulty    Subdisks:     1 Size:  30 MB
    Stest.p0.s0    State: up        PO:        0  B Size:  30 MB
    Stest.p1.s0    State: obsolete  PO:        0  B Size:  30 MB
    vinum -> start test.p1.s0
    Reviving test.p1.s0 in the background
    vinum -> vinum[295]: reviving test.p1.s0   this message appears after the prompt
    (some time later)
    vinum[295]: test.p1.s0 is up
    
Failed boot disk

If you're running your root file system on a Vinum volume, you can survive the failure of the boot volume if it is mirrored with at least two concatenated plexes each containing only one subdisk. Under normal circumstances, you can carry on running as if nothing had happened, but obviously you will no longer be able to reboot from that disk. Instead, boot from the other disk.

The root file system also has individual UFS partitions, so you have a choice of what you mount. For example, if your root file system has UFS partitions /dev/da0s4a and /dev/da1s4a, you can mount either of these partitions or /dev/vinum/root. Never mount more than one of them, otherwise you can cause data corruption.

An even more insidious way to corrupt the root file system is to mount /dev/da0s4a or /dev/da1s4a and modify it. In this case, the two partitions are no longer the same, but there's no way for Vinum to know that. If this happens, you must mark the other subdisk as crashed with the vinum stop command.

Migrating Vinum to a new machine

Sometimes you might want to move a set of Vinum disks to a different FreeBSD machine. This is simple, as long as there are no name conflicts between the objects on the Vinum disks and any other Vinum objects you may already have on the system. Simply connect the disks and start Vinum. You don't need to put the disks in any particular location, and you don't need to run vinum create: Vinum stores the configuration on the drives themselves, and when it starts, it locates it accordingly.

Things you shouldn't do with Vinum

The vinum command offers a large number of subcommands intended for specific purposes. It's easy to abuse them. Here are some things you should not do:

13. Writing CD-Rs

Creating an ISO-9660 image; Burning the CD-R; Copying CD-ROMs;

Under FreeBSD, data on conventional hard disks is stored in the UNIX File System or UFS format. CD-ROMs and CD-Rs use a different file system, the ISO 9660 format, which is compatible with other systems. This is not a problem when you mount a CD-ROM: FreeBSD includes a read-only ISO 9660 file system. When you want to write a CD-R, however, things are a little more complicated: the medium requires you to write the entire file system at once, and since the file system is stored in a different format, you can't just copy the UFS file system. Instead, you must first create an image of the file system that you want to put on the CD-R, and then you copy it. We'll look at these steps in more detail below.

Creating an ISO-9660 image

The first step is to create the ISO 9660 file system image, frequently simply called an ISO image. There are a number of ports available in the Ports Collection; here we'll look at mkisofs, which is part of the cdrtools port. Installation isn't quite as straightforward as you might expect: you need a special fag to install mkisofs:

# cd /usr/ports/sysutils/cdrtools
# make install -DMKISOFS

mkisofs has a bewildering number of parameters. Here are the important ones:

This is a lot of stuff to type in every time. It's easier to write a Make file and use make:

APPLID   =   "Dummy application "
BOOT =
#To make it bootable, put in something like this:
#Note that the -b option is there as well
#BOOT   =   "-b floppies/boot.flp "
ISO     =   /var/tmp/isoimage
PREPARER =  "me "
VOLID   =   "Volume 0000 "
DIR     =   .

cdrom:
      mkisofs -A ${APPLID} ${BOOT} -J -o ${ISO} -f \
      -p ${PREPARER} -r -T -V ${VOLID} ${DIR}

For example, to make a bootable CD-R of the FreeBSD release, you would first perform the make world and make release. Assuming that the release directory is /home/release, you will find the directory trees for the first two CD-ROMs in /home/re-lease/R/cdrom/disc1 and /home/release/R/cdrom/disc2. You could do this:

# make cdrom DIR=/home/release/R/cdrom/disc1
mkisofs -A "Dummy application "  -J -o ../iso -table-name .MAP  -p "Greg Lehey " -r T
-V "Volume 000"
 6.40%  done,  estimate  finish  Sun  Aug  27  13:34:54  2000
12.79%  done,  estimate  finish  Sun  Aug  27  13:35:02  2000
19.19%  done,  estimate  finish  Sun  Aug  27  13:35:05  2000
25.57%  done,  estimate  finish  Sun  Aug  27  13:35:10  2000
31.97%  done,  estimate  finish  Sun  Aug  27  13:35:10  2000
38.36%  done,  estimate  finish  Sun  Aug  27  13:35:10  2000
44.75%  done,  estimate  finish  Sun  Aug  27  13:35:10  2000
51.15%  done,  estimate  finish  Sun  Aug  27  13:35:12  2000
57.54%  done,  estimate  finish  Sun  Aug  27  13:35:12  2000
63.94%  done,  estimate  finish  Sun  Aug  27  13:35:12  2000
70.34%  done,  estimate  finish  Sun  Aug  27  13:35:11  2000
76.72%  done,  estimate  finish  Sun  Aug  27  13:35:13  2000
83.12%  done,  estimate  finish  Sun  Aug  27  13:35:12  2000
89.52%  done,  estimate  finish  Sun  Aug  27  13:35:13  2000
95.90%  done,  estimate  finish  Sun  Aug  27  13:35:13  2000
Total translation table size: 35119 
Total rockridge attributes bytes: 59724 
Total directory bytes: 104448
Path table size(bytes): 256
Max brk space used 86224
78211 extents written (152 Mb)

The progress reports are rather boring nowadays, considering that the whole process only takes a couple of minutes, but the summary information at the bottom can be of interest.

Testing the CD-R

So now you have an ISO image. How do you know it's correct? It's just a single file, and it could have just about anything on it. You can burn a CD, of course, but if it's junk, you have another coaster. If you're not sure, it's better to look inside first. You can do that by using it as the basis for an md vnode device.

The md driver creates a number of different kinds of pseudo-device. See the man page md (4) for more details. We use the vnode device, a special file that refers to file system files. Support for md is included in the GENERIC kernel, but if you've built a kernel without the md driver, you can load it as a kld. If you're not sure, try loading the kld anyway. Then you associate a vnode device with the ISO image iso-image using the program mdconfig:

# kldload md                         load the kld module if necessary
kldload: can't load md: File exists  already loaded or in the kernel
# mdconfig -a -t vnode -f iso-image  configure the device
md0                                  this is the name assigned
# mount -t cd9660 /dev/md0 /mnt      mount it

After this, you will be able to access the image at /mnt as a normal file system. Don't forget to un mount and un configure the file when you're finished:

# umount /mnt
# mdconfig -d -u 0

Older releases of FreeBSD used the vn driver, which used different syntax.

Burning the CD-R

Once you have created and tested an ISO image, you can copy it to CD-R. For SCSI burners, you use cdrecord; ATA (IDE) CD-R burners you use burncd. In the following sections we'll look at both programs.

Burning a CD-R on an ATA burner

To burn a CD-R in an ATA (or IDE) burner, use burncd, which is part of the base system. Typically you'll only have one CD-R burner on the system, so it will be called /dev/acd0. You'll have something like this in your dmesg output:

acd0: CD-RW  <RWD RW4224> at atal-slave BIOSPIO

burncd has both fags and commands. For our purposes, the most important fags are:

The most important commands for writing ISO 9660 CD-ROMs are:

If burncd doesn't recognize a command, it assumes it's a file name. If it does, it assumes it isn't a file name. This can be confusing: there are other commands not mentioned in the list above, for example raw. If you have an ISO file called raw, youll have to rename it before you can burn it with burncd.

Before you start, you should decide on the recording speed. If your machine is fast enough, use the rated recording speed. In the case of the example machine, that's an 8x speed (i.e. it records at eight times the speed at which a normal audio CD is played). Before you do this, though, you should make sure that your system can keep a sufficient data rate so that there is always data available to transfer to the CD-R. If it can't keep up, you'll get an underrun, a gap in the data, and your CD-R is worthless (a coaster).

To make sure you don't make coasters, you should do a test run. The system goes through all the motions, but it doesn't actually write anything to the CD-R blank. Nevertheless, it tests all aspects of the burn, so you must have a valid CD-R blank in the drive, otherwise the attempt will fail. To test burn an image called iso, enter:

# burncd -f /dev/acd0c -t -v -s 8 data iso fixate
adding type 0x08 file iso size 184576 KB 92288 blocks
next writeable LBA 0
addr = 0 size = 189005824 blocks = 92288
writing from file iso size 184576 KB
written this track 6880 KB (3%) total 6880 KB

At this point, burncd overwrites the line with progress indications until it is find. Finally, you see:

written this track 184576 KB (100%) total 184576 KB
fixating CD, please wait..
burncd: ioctl(CDRIOCFIXATE): Input/output error

This last line appears a little alarming. It's not really serious, though: the CD has not really been written, so it's not possible to read from it. A number of CD-R drives return error conditions under these circumstances.

If everything was OK in the test run, you can repeat the command without the -t fag:

# burncd -f /dev/acd0c -v -s 8 data iso fixate

The output is identical, but this time you should not get the error message.

Burning a CD-R on a SCSI burner

If you have a SCSI burner, use cdrecord, which is part of the cdrtools port we installed on page 243. cdrecord has a rather strange habit of not using device names: instead, it accesses the device directly by its SCSI parameters (bus, unit and LUN). You can get these parameters from the dmesg output in /var/run/dmesg.boot, but there's an easier way:

# cdrecord -scanbus
Cdrecord 1.9 (i386-uriknown-freebsd4.1) Copyright (C) 1995-2000 Jorg Schilling
Using libscg version 'schily-0.1'
scsibus0:
  0,0,0  0)  'MATSHITA'  'CD-R   CW-7503  ' '1.06'  Removable CD-ROM
cdrecord: Warning: controller returns zero sized CD capabilities page,
cdrecord: Warning: controller returns wrong size for CD capabilities page,
cdrecord: Warning: controller returns wrong page 0 for CD capabilities page (2A).
  0,1,0  1)  IEAC    '  'CD-ROM CD-532S  ' '1.0A'  Removable CD-ROM
  0,2,0  2)  *      
  0,3,0  3)  *      
  0,4,0  4)  'SONY    '  'SDT-10000       ' '0101'  Removable Tape
  0,5,0  5)  *      
  0,6,0  6)  *      
  0,7,0  7)  *      
  0,8,0  8)  'QUANTUM '  'QM318000TD-SW   ' 'N491'  Disk

This output doesn't tell you exactly which devices are CD-Rs, and it also doesn't look at any except the first SCSI bus. Alternatively, you can use the standard system utility camcontrol:

<MATSHITA CD-R   CW-7503 1.06>   at  scbus0  target  0  lun  0  (pass0,cd0)
<TEAC CD-ROM CD-532S 1.0A>       at  scbus0  target  1  lun  0  (pass1,cd1)
<SONY SDT-10000 0101>            at  scbus0  target  4  lun  0  (sa0,pass2)
<QUANTUM QM318000TD-SW N491>     at  scbus0  target  8  lun  0  (pass3,da0)
<EXABYTE EXB-8505SMBANSH2 0793>  at  scbus1  target  1  lun  0  (sa1,pass4)
<Quantum DLT4000 CC1E>           at  scbus1  target  3  lun  0  (sa2,pass5)
<AIWA GD-8000 0119>              at  scbus1  target  4  lun  0  (sa3,pass6)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  0  (pass7,cd2)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  1  (pass8,cd3)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  2  (pass9,cd4)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  3  (pass10,cd5)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  4  (pass11,cd6)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  5  (pass12,cd7)
<NRC MBR-7 110>                  at  scbus1  target  6  lun  6  (pass13,cd8)

Either way, you need to recognize the CD-R device, which in this case is relatively easy: it's the Matsushita CW-7503 ("MATSHITA ") at the very beginning of each list. cdrecord refers to this device as 0,0,0 (bus 0, target 0, LUN 0).

The next thing to look at is the recording speed. If your machine is fast enough, use the rated recording speed. In the case of the example machine, that's an 8x speed (i.e. it records at 8 times the speed at which a normal audio CD is played). Before you do this, though, you should make sure that your system can keep a sufficient data rate so that there is always data available when to go on the CD. If you can't keep up, you'll get an underrun, a gap in the data, and your CD-R is worthless (a coaster).

To make sure you don't make coasters, you should do a dummy run. The system goes through all the motions, but it doesn't actually write anything to the CD-R blank. Nevertheless, it tests all aspects of the burn, so you must have a valid CD-R blank in the drive, otherwise the attempt will fail. To burn an image called iso, enter:

# cdrecord -dummy -v dev=0,0,0 -speed=8 iso
Cdrecord 1.9 (i386-unknown-freebsd5.0) Copyright (C) 1995-2000 Jorg Schilling
TOC Type: 1 = CD-ROM
scsidev: '0,0,0'
scsibus: 0 target: 0 lun: 0
Using libscg version 'schily-0.1'
atapi: 0
Device type    : Removable CD-ROM
Version        : 2
Response Format: 2
Capabilities   : SYNC LINKED
Vendor_info    : 'MATSHITA'
Identifikation : 'CD-R  CW-7503 '
Revision       : '1.06'
Device seems to be: Generic mmc CD-R.
Using generic SCSI-3/mmc CD-R driver (mmc_cdr).
Driver flags   : SWABAUDIO
FIFO size      : 4194304 = 4096 KB
Track 01: data 152 MB
Total size:    175 MB (17:22.84) = 78213 sectors
Lout start:    175 MB (17:24/63) = 78213 sectors
Current Secsize: 2048 ATIP info from disk:
  Indicated writing power: 5
  Is not unrestricted
  Is not erasable
  ATIP start of lead in:  -11080 (97:34/20)
  ATIP start of lead out: 335100 (74:30/00)
Disk type:    Long strategy type (Cyanine, AZO or similar) Manuf. index: 11
Manufacturer: Mitsubishi Chemical Corporation
Blocks total: 335100 Blocks current: 335100 Blocks remaining: 256887
RBlocks total: 342460 RBlocks current: 342460 RBlocks remaining: 264247
Starting to write CD/DVD at speed 8 in dummy mode for single session.
Last chance to quit, starting dummy write in 1 seconds.
Waiting for reader process to fill input buffer ... input buffer ready.
Starting new track at sector: 0
Track 01: 0 of 152 MB written (fifo 100%).

At this point, cdrecord overwrites the last line with progress indications until it is finished. If you're watching, keep an eye on the fifo information at the end of the line. This gives you an idea how well the system is keeping up with the burner. If the utilization drops to 0, you will get an underrun, and the blank would have become a coaster if this were for real.

Finally, you see:

Track 01: 152 of 152 MB written (fifo 100%).
Track 01: Total bytes read/written: 160176128/160176128 (78211 sectors).
Writing  time: 136.918s
Fixating...
WARNING: Some drives don't like fixation in dummy mode. Fixating time: 35.963s
cdrecord: fifo had 2523 puts and 2523 gets.
cdrecord: fifo was 0 times empty and 2451 times full, min fill was 96%.

The summary information at the end shows that at some point the fifo dropped below 100% full, but this is far from being a problem. If, on the other hand, there was a lot of disk activity at the same time, you might find the fifo level dropping much lower.

When you're sure that you won't have any problems, you can do the real thing: just repeat the command without the -dummy option. The output looks almost identical.

Copying CD-ROMs

Frequently you'll want to make a verbatim copy of another CD. There are copyright implications here, of course, but many CD-ROMs are not restricted. In particular, you may make copies of FreeBSD CD-ROMs for your personal use.

CD-ROMs are already in ISO format, of course, so to get a file /iso, as in the examples above, you could just perform a literal copy with dd:

# dd if=/dev/cd0c of=iso bs=128k

The bs=128k tells dd to copy in blocks of 128 kB. It's not strictly necessary, but if you omit it, it will perform a separate transfer for every sector, and on a slow machine it can be much less efficient.

There's an even easier way, though, if you have two CD-ROM drives: you can frequently copy directly from one drive to the other, without storing on disk at all. To do this, of course, you need to be very sure that your CD-ROM drive is fast enough. In particular, if it spins down during the copy, you will almost certainly have underruns and a useless copy. Be very sure to do a dummy run first. Let's assume that your second CD-ROM drive is /dev/cd1c (a SCSI drive). For IDE drives, write:

# burncd -f /dev/acd0c -t -v -s 8 data /dev/cd1c fixate

In this example, the -f option indicates that /dev/acd0c is the (IDE) CD-R burner. /dev/cd1c is the (SCSI) CD-ROM drive with the original CD-ROM. You don't need to mount /dev/cd1c, since it's being accessed as raw data, not a file system.

When you're sure this will work, remove the -t tag and repeat. For SCSI, enter

# cdrecord -dummy -v dev=0,0,0 -speed=8 /dev/cd1c

When it completes satisfactorily, remove the -dummy and repeat.

14. Tapes, backups and floppy disks

Backing up your data; Using floppy disks under FreeBSD.

In Chapter 11 we looked at hard disks. In this chapter, we'll consider how to guard against data loss, and how to transfer data from one location to another. These are functions that UNIX traditionally performs with tapes, and we'll look at them in the next sections. Because FreeBSD runs on PCs, however, you can't completely escape floppy disks, though it would be an excellent idea. We'll look at floppies on page 256.

Backing up your data

No matter how reliable your system, you are never completely protected against loss of data. The most common reasons are hardware failure and human error. By comparison, it's very seldom that a software error causes data loss, but this, too, can happen.

UNIX talks about archives, which are copies of disk data in a form suitable for writing on a serial medium such as tape. You can, however, write them to disk files a well, and that's what people do when they want to move a source tree from one system to another. You'll also hear the term tarball for an archive made by the tar program, which we discuss below.

What backup medium?

Traditionally, PCs use floppy disks as a removable storage medium. We'll look at floppies below, but you can sum the section up in one statement: don't use floppy disks.

Floppy disks are particularly unsuited as a backup medium for modern computers. Consider even a minimal system with a 2 GB hard disk. Storing 2 GB of data on floppies requires about 1,500 floppies, which, at $0.30 each, would cost you $450. Copying the data to a floppy takes about 50 seconds per floppy, so the raw backup time would be about 21 hours, plus the time it takes you to change the floppies, which could easily take another three or more hours. During this time you have to sit by the computer playing disk jockey, a total of three days' work during which you could hardly do anything else. When you try to read in the data again, there's a virtual certainty that one of the floppies has a data error, especially if you read them with a different drive.

By contrast, a single DDS or Exabyte cassette stores several gigabytes and costs about $6. The backup time for 2 GB is about 90 minutes, and the operation can be performed completely unattended.

A number of cheaper tape drives are also available, such as Travan tapes. FreeBSD supports them, but for one reason or another, they are not popular. FreeBSD once used to have support for "floppy tape ", run off a floppy controller, but these tapes were very unreliable, and they are no longer supported.

You can also use writeable "CD-ROMs" (CD-Rs) for backup purposes. By modern standards, the media are small (up to 700 MB), but they have the advantage of being readily accessible on other systems. We looked at CD-Rs in Chapter 13.

Tape devices

FreeBSD tape devices have names like /dev/nsa0 (see page 196). Each letter has a significance:

Backup software

FreeBSD does not require special "backup software." The base operating system supplies all the programs you need. The tape driver is part of the kernel, and the system includes a number of backup programs. The most popular are:

Backup strategies are frequently the subject of religious wars. I personally find that tar does everything I want, but you'll find plenty of people who recommend dump or amanda instead. In the following section, we'll look at the basics of using tar. See the man page dump(8) for more information on dump.

tar

tar, the tape archiver, performs the following functions:

tar does not compress the data. The resulting archive is slightly larger than the sum of the files that it contains, since it also contains a certain amount of header information. You can, however, use the gzip program to compress a tar archive, and tar invokes it for you automatically with the -z option. The size of the resultant archives depends strongly on the data you put in them. JPEG images, for example, hardly compress at all, while text compresses quite well and can be as much as 90% smaller than the original file.

Creating a tar archive

Create an archive with the c option. Unlike most UNIX programs, tar does not require a hyphen (-) in front of the options. For example, to save your complete kernel source tree, you could write:

# tar cvf source-archive.tar /usr/src/sys
tar: Removing leading / from absolute path names in the archive.
usr/src/sys/
usr/src/sys/CVS/
usr/src/sys/CVS/Root
usr/src/sys/CVS/Repository
usr/src/sys/CVS/Entries
usr/src/sys/compile/
usr/src/sys/compile/CVS/
(etc)

The parameters have the following meaning:

The message on the first line (Removing leading / ...) indicates that, although the directory name was specified as /usr/src/sys, tar treats it as usr/src/sys.. This makes it possible to restore the files into another directory at a later time.

You can back up to tape in exactly the same way:

# tar cvf /dev/nsa0 /usr/src/sys

There is a simpler way, how ever: if you don't specify a file name, tar looks for the environment variable TAPE. If it finds it, it interprets it as the name of the tape drive. You can make things a lot easier by setting the following line in the configuration file for your shell (.profile for sh, .bashrc for bash, .login for csh and tcsh):

TAPE=/dev/nsa0 export TAPE   for sh and bash
setenv TAPE /dev/nsa0        for csh and tcsh

After this, the previous example simplifies to:

# tar cv /usr/src/sys
Listing an archive

To list an archive, use the option t:

#   tar t                        from tape
usr/src/sys/
usr/src/sys/CVS/
usr/src/sys/CVS/Root
usr/src/sys/CVS/Repository
usr/src/sys/CVS/Entries
usr/src/sys/compile/
usr/src/sys/compile/CVS/
usr/src/sys/compile/CVS/Root
(etc)
#   tar tvf source-archive.tar   from disk
drwxrwxrwx root/bin       0 Oct 25 15:07 1997 usr/src/sys/
drwxrwxrwx root/bin       0 Oct 25 15:08 1997 usr/src/sys/CVS/
-rw-rw-rw- root/wheel     9 Sep 30 23:13 1996 usr/src/sys/CVS/Root
-rw-rw-rw- root/wheel    17 Sep 30 23:13 1996 usr/src/sys/CVS/Repository
-rw-rw-rw- root/bin     346 Oct 25 15:08 1997 usr/src/sys/CVS/Entries
drwxrwxrwx roo t/bin      0 Oct 27 17:11 1997 usr/src/sys/compile/
drwxrwxrwx root/bin       0 Jul 30 10:52 1997 usr/src/sys/compile/CVS/
(etc)

This example shows the use of the v (verbose) option with t. If you don't use it, tar displays only the names of the files (first example, from tape). If you do use it, tar also displays the permissions, ownerships, sizes and last modification date in a form reminiscent of ls -l (second example, which is from the disk file source-archive.tar).

Extracting files

To extract a file from the archive, use the x option:

#tar xv usr/src/sys/Makefile   from tape
usr/src/sys/Makefile           confirms that the file was extracted

As with the c option, if you don't use the v option, tar does not list any file names. If you omit the names of the files to extract, tar extracts the complete archive.

Compressed archives

You can combine gzip with tar by specifying the z option. For example, to create the archive source-archive.tar.gz in compressed format, write:

# tar czf source-archive.tar.gz /usr/src/sys

You must specify the z option when listing or extracting compressed archives, and you must not do so when listing or extracting non-compressed archives. Otherwise you get messages like:

# tar tzvf source-archive.tar
gzip: stdin: not in gzip format tar: child returned status 1
# tar tvf source-archive.tar.gz
tar: only read 2302 bytes from archive source-archive.tar.gz

Using floppy disks under FreeBSD

I don't like floppy disks. UNIX doesn't like floppy disks. Probably you don't like floppy disks either, but we occasionally have to live with them.

FreeBSD uses floppy disks for one thing only: for initially booting the system on systems that can't boot from CD-ROM. We've already seen that they're unsuitable for archival data storage and data transfer. For this purpose, FreeBSD uses tapes and CD-ROMs, which are much more reliable, and for the data volumes involved in modern computers, they're cheaper and faster.

So why use floppies? The only good reasons are:

In the following sections, we'll look at how to handle floppies under FreeBSD, with particular regard to coexisting with Microsoft. Here's an over view:

Formatting a floppy

Even if you buy preformatted floppies, it's a good idea to reformat them. Track alignment can vary significantly between individual floppy drives, and the result can be that your drive doesn't write quite on top of the pre-written tracks. I have seen read failure rates as high as 2% on pre-formatted floppies: in other words, after writing 100 floppies with valuable data, the chances are that two of them have read errors. You can reduce this problem by reformatting the floppy in the drive in which it is to be written, but you can't eliminate it.

On Microsoft platforms, you format floppies with the FORMAT program, which performs two different functions when invoked on floppies: it performs both a low-level format, which rewrites the physical sector information, and then it performs what it calls a high-level format, which writes the information necessary for Microsoft platforms to use it as a file system. UNIX calls the second operation creating a file system. It's not always necessary to have a file system on the diskette-in fact, as we'll see, it can be a disadvantage. In addition, FreeBSD offers different kinds of file system, so it performs the two functions with different programs. In this section, we'll look at fdformat, which performs the low-level format. We'll look at how to create a UFS or Microsoft file system in the next section.

To format a diskette in the first floppy drive, /dev/fd0, you would enter:

$ fdformat /dev/fd0
Format 1440K floppy '/dev/fdC?  (y/n): y
Processing --------------------------------

Each hyphen character (-) represents two tracks. As the format proceeds, the hyphens change to an F (Format) and then to V (Verify) in turn, so at the end the line reads

Processing VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV done.

File systems on floppy

It's possible to use floppies as file systems under FreeBSD. You can create a UFS file system on a floppy just like on a hard disk. This is not necessarily a good idea: the UFS file system is designed for performance, not maximum capacity. By default, it doesn't use the last 8% of disk space, and it includes a lot of structure information that further reduces the space available on the disk. Here's an example of creating a file system, mounting it on the directory /A , and listing the remaining space available on an empty 3,5" floppy. Since release 5, FreeBSD no longer requires a partition table on a floppy, so you don't need to run bsdlabel (the replacement for the older disklabel program).

# newfs   -O1 /dev/fd0   create a new file system
/dev/fd0: 1.4MB (2880 sectors) block size 16384, fragment size 2048
    using 2 cylinder groups of 1.00MB, 64 blks, 128 inodes.
super-block backups (for fsck -b #) at:
32, 2080
# mount /dev/fd0 /A      mount the floppy on /A
# df -k /A               display the space available
Filesystem   1024-blocks  Used  Avail  Capacity  Mounted on
/dev/fd0     1326          2    1218     0%        /A

Let's look at this in a little more detail:

The output of df looks terrible! Our fbppy only has 1218 kB left for normal user data, even though there is nothing on it and even df claims that it can really store 1326 kB. This is because UFS keeps a default of 8% of the space free for performance reasons. You can change this, however, with tunefs, the file system tune program1):

# umount /A                  first unmount the floppy
# tunefs -m 0 /dev/fd0       and changethe minimum free to 0
tunefs: minimum percentage of free space changes from 8% to 0%
tunefs: should optimize for space with minfree  < 8%
# tunefs -o space /dev/fd0   changethe optimization
tunefs: optimization preference changes from time to space
# mount /dev/fd0 /A          mount the file system again
# df /A                      and takeanother look
Filesystem   1024-blocks  Used  Avail  Capacity  Mounted on
/dev/fd0            1326   2    1324    0%         /A

Still, this is a far cry from the claimed data storage of a Microsoft disk. In fact, Microsoft disks can't store the full 1.4 MB either: they also need space for storing directories and allocation tables. The moral of the story: only use file systems on floppy if you don't have any alternative.

Microsoft file systems

To create a Microsoft FAT12, FAT16 or FAT32 file system, use the newfs_msdos command:

$ newfs_msdos -f 1440 /dev/fd0

The specification -f 1440 tells newfs_msdos that this is a 1.4 MB floppy. Alternatively, you can use the mformat command:

$ mformat A:

You can specify the number of tracks with the -t option, and the number of sectors with the -s option. To explicitly specify a fbppy with 80 tracks and 18 sectors (a standard 3,5" 1.44 MB floppy), you could enter:

$ mformat -t 80 -s 18 A:

mformat is one of the mtools that we look at in the next section.

Other uses of floppies

Well, you could take the disks out of the cover and use them as a kind of frisbee. But there is one other useful thing you can do with floppies: as an archive medium, they don't need a file system on them. They just need to be low-level formatted. For example, to write the contents of the current directory onto a floppy, you could enter:

$ tar cvfM /dev/fd0.
./
.xfmrc
.x6530mcdkey
.uwmrc
.twmrc
.rnsoft
.rnlast
...etc
Prepare volume #2 for /dev/fd0 and hit return:

Note also the solitary dot (.) at the end of the command line. That's the name of the current directory, and that's what you're backing up. Note also the option M, which is short for --multi-volume. There's a very good chance that you'll run out of space on a floppy, and this option says that you have a sufficient supply of floppies to perform the complete backup.

To extract the data again, use tar with the x option:

$ tar xvfM /dev/fd0.
./
.xfmrc
.x6530modkey
.uwmrc
...etc

See the man page tar(1) for other things you can do with tar.

Accessing Microsoft floppies

Of course, most of the time you get data on a floppy, it's not in tar format: it has a Microsoft file system on it. We've already seen the Microsoft file system type on page 190, but that's a bit of overkill if you just want to copy files from floppy. In this case, use the mtools package from the Ports Collection. mtools is an implementation of the MS-DOS programs ATTRIB, CD, COPY, DEL, DIR, FORMAT, LABEL, MD, RD, READ, REN, and TYPE under UNIX. To avoid confusion with existing utilities, the UNIX versions of these commands start with the letter m. They are also written in lower case. For example, to list the contents of a floppy and copy one of the files to the current (FreeBSD) directory, you might enter:

$ mdir               list the current directory on A
Volume in drive A is MESSED OS Directory for A:/
Directory for A:/
IO        SYS  33430  4-09-91   5:00a
MSDOS     SYS  37394  4-09-91   5:00a
COMMAND   COM  47845  12-23-92  5:22p
NFS            <DIR>  12-24-92  11:03a
DOSEDIT   COM  1728   10-07-83  7:40a
CONFIG    SYS  792    10-07-94  7:31p
AUTOEXEC  BAT  191    12-24-92  11:10a
MOUSE          <DIR>  12-24-92  11:09a
    12 File(s) 82944 bytes free
$ mcd nfs            change to directory A:\NFS 
$ mdir               and list the directory
Volume in drive A is MESSED OS
Directory for A:/NFS
.              <DIR>  12-24-92  11:03a
..             <DIR>  12-24-92  11:03a
HOSTS          5985   10-07-94  7:34p
NETWORK   BAT  103    12-24-92  12:28p
DRIVES    BAT  98     11-07-94  5:24p
   and many more
    51 File(s) 82944 bytes free
$ mtype drives.bat   type the contents of DRIVES.BAT
Net use c: presto:/usr/dos
C:
Cd\nfs
#net use f:Porsche:/dos
#net use g:Porsche:/usr
$ mcopy a:hosts.
Copying HOSTS
$ ls-l hosts         and list it
-rw-rw-rw- 1 root wheel 5985 Jan 28 18:04 hosts

You must specify the drive letter to mcopy, because it uses this indication to decide whether the file name is a UNIX or a Microsoft file name. You can copy files from FreeBSD to the floppy as well, of course.

A word of warning. UNIX uses a different text data format from Microsoft: in UNIX, lines end with a single character, called New line, and represented by the characters \n in the C programming language. It corresponds to the ASCII character Line Feed (represented by ^J). Microsoft uses two characters, a Carriage Return (^M) followed by a Line Feed. This unfortunate difference causes a number of unexpected compatibility problems, since both characters are usually invisible on the screen.

In FreeBSD, you won't normally have many problems. Occasionally a program complains about non-printable characters in an input line. Some, like Emacs, show them. For example, Emacs shows our last file, drives.bat, like this:

net use c: presto:/usr/dos~M
c:M
cd \nfs"M
#net use f: porsche:/dos~M
#net use g: porsche:/usr~M

This may seem relatively harmless, but it confuses some programs, including the C compiler and pagers like more, which may react in confusing ways. You can remove them with the -t option of mcopy:

$ mcopy -t a:drives.bat

Transferring files in the other direction is more likely to cause problems. For example, you might edit this file under FreeBSD and then copy it back to the diskette. The results depend on the editor, but assuming we changed all occurrences of the word porsche to freedom, and then copied the file back to the diskette, Microsoft might then find:

C:> type drives.bat
net use c: presto:/usr/dos
c:
cd \nfs
#net use f: freedom:/dos
#net use g: freedom:/usr

This is a typical result of removing the Carriage Return characters. The -t option to mcopy can help here, too. If you use it when copying to a Microsoft file system, it reinserts the Carriage Return characters.

15. Printers

Printer configuration; Starting the spooler; Testing the spooler; Troubleshooting; Using the spooler; PostScript; PDF.

In this chapter, we'll look at some aspects of using printers with FreeBSD. As a user, you don't access printers directly. Instead, a series of processes, collectively called the spooler, manage print data. One process, lpr, writes user print data to disk, and another, lpd, copies the print data to the printers. This method enables processes to write print data even if the printers are busy and ensures optimum printer availability.

In this section, we'll look briefly at what you need to do to set up printers. For more details, look in the online handbook section on printing.

lpd is the central spooler process. It is responsible for a number of things:

Through a configuration file, and by providing the special filter programs, you can enable the spooler to do all or some subset of the above for a great variety of printer hardware.

This may sound like overkill if you are the only user on the system. It is possible to access the printer directly, but it's not a good idea:

Printer configuration

There are three commonly used ways to connect a printer to a computer:

It's pretty straightforward to connect a parallel printer. You don't need to do anything special to configure the line printer driver lpt: it's in the kernel by default. All you need to do is to plug in the cable between the printer and the computer. If you have more than one parallel interface, of course, you'll have to decide which one to use. Parallel printer devices are called /dev/lptn, where n is the number, starting with 0. USB devices have names like /dev/ulptn. See Table 10-4 on page 195 for further details.

Configuring an Ethernet-connected printer is more complicated. You obviously need an IP address, which you configure on the printer. Most modern printers then appear like a remote computer to the spooler. We look at spooling to remote computers on page 266.

Testing the printer

When you have connected and powered on a parallel port printer, run the built-in test if one is supplied: typically there's a function that produces a printout describing the printer's features. After that, check the communication between the computer and the printer.

# lptest > /dev/lpt0

If you have a pure PostScript printer, one which can't print anything else, you won't get any out put. Even here, though, you should see some reaction on the status display.

Configuring /etc/printcap

The next step is to configure the central configuration file, /etc/printcap. This file is not the easiest to read, but after a while you'll get used to it. Here are some typical entries:

lp|lj|ps|local LaserJet 6MP printer:\
  :lp=/dev/lpt0:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs:sh:mx#0:\
  :if=/usr/local/libexec/lpfilter:

rlp|sample remote printer:\
  :rm=freebie:sd=/var/spool/output/freebie:lf=/var/log/lpd-errs:\
  :rp=lp:

Let's look at this in detail:

Remote printing

In a network, you don't need to have a printer on every machine; you can print on another machine (which may be a printer) on the same network. There are a couple of things to consider:

Spooler filters

Probably the least intelligible entry in the configuration file on page 265 was the if entry. It specifies the name of an input filter, a program through which Lpd passes the complete print data before printing.

What does it do that for? There can be a number of reasons. Maybe you have data in a format that isn't fit to print. For example, it might be PostScript, and your printer might not understand PostScript. Or it could be the other way around: your printer understands only PostScript, and the input isn't PostScript.

There's a more likely reason to require a filter, though: most printers still emulate the old teletypes, so they require a carriage return character (Ctrl-M or ^M) to start at the beginning of the line, and a new line character (Ctrl-J or ^J) to advance to the next line. UNIX uses only ^J, so if you copy data to it, you're liable to see a staircase effect. For example, may tell you:

$ ps
 PID  TT  STAT     TIME  COMMAND
2252  pi  Ss    0:01.35  /bin/bash
2287  p1  IW    0:04.77  e/etc/printcap
2346  p1  R+    0:00.05  ps

When you try to print it, however, you get:

 PID  TT  STAT     TIME COMMAND
2252  p1  Ss    0:01.35 /bin/bash
2287  p1  IW    0

The rest of the page is empty: you've gone off the right margin. There are a number of ways to solve this problem:

There are a couple of options for the print filter. One of them, taken from the online handbook, sends out a LaserJet control sequence before every job. Put the following shell script in /usr/local/libexec/lpfilter:

#!/bin/sh
printf "\033&k2G" && cat && printf "\f" && exit 0 exit 2
15.1. Simple print filter (html, txt)

This approach does not work well with some printers, such as my HP LaserJet 6MP, which can print both PostScript and LaserJet (natural) formats at random. They do this by recognizing the text at the beginning of the job. This particular filter confuses them by sending a LaserJet command code, so the printer prints the PostScript as if it were plain text.

In this kind of situation, the standard filters are no longer sufficient. You can solve the problem with the port apsfilter, which is in the Ports Collection.

Starting the spooler

As we saw above, the line printer daemon lpd is responsible for printing spooled jobs. By default it isn't started at boot time. If you're root, you can start it by name:

# lpd

Normally, however, you will want it to be started automatically when the system starts up. You do this by setting the variable lpd_enable in /etc/rc.conf:

lpd_enable="YES"    # Run the line printer daemon

See page 552 for more details of /etc/rc.conf.

You can also add another line referring to the line printer daemon to /etc/rc.conf:

lpd_flags=""    # Flags to lpd (if enabled).

You don't normally need this line. See the man page for lpd for details of the fags.

Testing the spooler

To test the spooler, you can run the lptest program again. This time, however, instead of sending it directly to the printer, you send it to the spooler:

$ lptest 80 5 | lpr

The results should look like:

!"#$%&'()* + ,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_,abcdefghijklmnop
"#$%&'()* + ,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_,abcdefghijklmnopq
#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK^OPQRSTUVWXYZ[\]^_,abcdefghijklmnopqr
$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_,abcdefghijklmnopqrs
%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK^OPQRSTUVWXYZ[\]^_,abcdefghijklmnopqrst

Troubleshooting

Here's a list of the most common problems and how to solve them.

15.1. Common printer problems
ProblemCause
The printer prints, but the last page doesn't appear. The status shows that the printer still has data in the buffer. After several minutes, the last page may appear.Your output data is not ejecting the last page. The printer is configured to either wait for an explicit eject request (the ASCII Form feed character, Ctrl-L) or to eject after a certain period of time.

You have a choice as to what you do about this. Usually you can configure the printer, or you could get the print filter to print a form feed character at the end of the job. Listing 15-1 already does this that's the printf "\f".

The lines wander off to the right edge of the paper and are never seen againThis is the staircase effect. Refer to page 268 for a couple of solutions.
Individual characters or whole sections of text are missing.This problem occurs almost only on serial printers. It's a result of incorrect handshakingsee page 330 and the online handbook for more details.
The output contained completely unintelligible random characters.On a serial printer ,if the characters appear slowly, and there's a predominance of the characters {|}~, this probably means that you have set up the communication parameters in correctly. Check the online handbook for a solution. Makes sure you don't confuse this problem with the following one.
The text was legible, but it bore no relationship to what you wanted to print.One possibility is that you are sending PostScript output to your printer. See the discussion on page 271 to check if it is PostScript. If it is, your printer is not interpreting it correctly, either because it doesn't understand PostScript, or because it has been confused (see the discussion on page 268 for one reason).
The display on the printer shows that data are arriving, but the printer doesn't print anything.You might be sending normal text to a PostScript printer that doesn't understand normal text. In this case, too, you will need a filter to convert the text to PostScriptthe opposite of the previous problem.

Alternatively, your printer port may not be interrupting correctly. This will not stop the printer from printing, but it can take up to 20 minutes to print a page. You can fix this by issuing the following command, which puts the printer /dev/lpt0 into polled mode:

# lptcontrol -p
You get the message lpr: cannot create freebie/.seqYou have forgotten to create the spool directory /var/spool/output/freebie.

Using the spooler

Using the spooler is relatively simple. Instead of outputting data directly to the printer, you pipe it to the spooler lpr command. For example, here is the same print command, first printing directly to the printer, and secondly via the spooler:

# ps waux > /dev/lpt0
$ ps waux | lpr

Note the difference in prompt: you have to be root to write directly to the printer, but normally anybody can write to the spooler. The spooler creates a job from this data. You can look at the current print queue with the lpq program:

$ lpq
waiting for lp to become ready (offline ?)
Rank  Owner   Job  Files             Total Size
1st   grog    313  (standard input)  9151  bytes
2nd   grog    30   (standard input)  3319m bytes
3rd   Yvonne  31   (standard input)  3395  bytes
4th   root    0    (standard input)  2611  bytes

The first line is a warning that lpd can't currently print. Take it seriously. In this example, the printer was deliberately turned off so that the queue did not change from one example to the next.

Normally, the job numbers increase sequentially: this particular example came from three different machines. You can get more detail with the -l option:

$ lpq -l
waiting for lp to become ready (offline ?)
grog: 1st           [job 313freebie.example.org]
  (standard input)  9151 bytes
grog: 2nd           [job 030presto.example.org]
  (standard input)  3319 bytes
yvonne: 3rd         [job 031presto.example.org]
  (standard input)  3395 bytes
root: 4th           [job 000bumble.example.org]
  (standard input)  2611 bytes

Removing print jobs

Sometimes you may want to delete spool output without printing it. You don't need to do this because of a printer configuration error: just turn the printer off, fix the configuration error, and turn the printer on again. The job should then be printed correctly. But if you discover that the print job itself contains garbage, you can remove it with the lprm program. First, though, you need to know the job number. Assuming the list we have above, we might want to remove job 30:

  lprm 30
dfA030presto.example.org dequeued
cfA030presto.example.org dequeued
  lpq
waiting for lp to become ready (offline ?)
Rank  Owner   Job  Files             Total Size
1st   grog    313  (standard input)  9151 bytes
2nd   yvonne  31   (standard input)  3395 bytes
3rd   root    0    (standard input)  2611 bytes

If the printer is offline, it may take some time for the lprm to complete.

PostScript

We've encountered the term PostScript several times already. It's a Page Description Language. With it, you can transmit detailed documents such as this book electronically and print them out in exactly the same form elsewhere1). PostScript is a very popular format on the World Wide Web, and web browsers like Netscape usually print in PostScript format.

Most other document formats describe special print features with escape sequences, special commands that start with a special character. For example, the HP LaserJet and PCL formats use the ASCII ESC character (Oxlb) to indicate the beginning of an escape sequence. PostScript uses the opposite approach: unless defined otherwise, the contents of a PostScript file are commands, and the printable data is enclosed in parentheses. PostScript documents start with something like:

%!PS-Adobe-3.0
%%Creator: groff version 1.10
%%CreationDate: Fri Oct 31 18:36:45 1997
%%DocumentNeededResources: font Symbol
%%+ font Courier
%%+ font Times-Roman
%%DocumentSuppliedResources: file images/vipw.ps
%%Pages: 32
%%PageOrder: Ascend
%%Orientation: Portrait
%%EndComments
%%BeginProlog

This is the prologue (the beginning) of the PostScript output for this chapter. The prologue of such a program can be several hundred kilobytes long if it includes embedded fonts or images. Amore typical size is about 500 lines.

You can do a number of things with PostScript:

Viewing with gv

gv is part of the instant workstation port that we discussed on page 93. To view a file with gv, simply start it:

$ gv filename &

If you don't specify a file name, you get a blank display. You can then open a file window by pressing o, after which you can select files and display them. Figure 15-1 shows the display of a draft version of this page with an overlaid open window at the top right. The Open File window contains a field at the top into which you can type the name of a file. Alternatively, the columns below, with scroll bars, allow you to browse the current directory and the parent directories.

The window below shows the text of the previous page (roughly) on the right hand side. Instead of scroll bars, there is a scroll area below the text Save Marked. You can scroll the image in all directions by selecting the box with the left mouse button and moving around. At top left are menu buttons that you can select with the left mouse button. Note also the button 1.414 at the top of the window: this is the magnification of the image. You can change it by selecting this button: a menu appears and gives you a range of magnifications to choose from.

The column to the right of these buttons is a list of page numbers. You can select a page number with the middle mouse button. You can also get an enlargement display of the text area around the mouse cursor by pressing the left button.

gv display


. 15.1. gv display

Printing with ghostscript

If your printer doesn't support PostScript, you can still print some semblance of the intended text with the help of ghostscript. The results are very acceptable with modern laser and inkjet printers, less so with older dot matrix printers.

To print on your particular printer, you first need to find a driver for it in ghostscript. In this context, the term driver means some code inside ghostscript that converts the data into something that the printer can print.

We've already seen how to use /etc/printcap. In this case, we'll need an input filter, a script or program that transforms the PostScript data into a form that the printer understands. The entry in /etc/printcap is pretty much the same for all printers:

ps|HP Office Jet 725 with PostScript:\
  :lp=/dev/lpt0:sd=/var/spool/output/colour:lf=/var/log/lpd-errs:sh:mx#0:\
  :if=/usr/local/libexec/psfilter:

This entry defines a printer called ps. The comment states that it's an HP Office Jet, but that's only a comment. Obviously you should choose a comment that matches the printer you really have.

The printer is connected to /dev/lpt0,the first parallel printer. Spool data is collected in the directory /var/spool/output/colour. You must create this directory, or printing will fail, and depending on what you use to print, you may not even see any error messages. They also don't appear on the log file, which in this case is /var/log/lpd-errs.

The important entry is in the last line, which refers to the input filter /usr/local/libexec/ps-filter. This file contains the instructions to convert the PostScript into something that the printer can understand. For example, for the HP Office Jet we're talking about here, it contains:

#!/bin/sh
/usr/local/bin/gs -sDEVICE=pcl3 -q -sPaperSize=a4 -dNOPAUSE -sOutputFile=- -

These options state:

Which driver?

The previous example used the driver for the HP DeskJet. Well, to be more precise, it used one of a plethora of drivers available. You can find more information in the HTML driver documentation at /usr/local/share/ghostscript/7.05/doc/Devices.htm. The 7.05 in the name refers to the release of ghostscript, which will change.

The documentation isn't the easiest to read. It's probably older than your printer, so there's a good chance that it won't mention your specific printer model. You may need to experiment a little before you get things working the way you want.

Printer drivers for DeskJets

There are at least six sets of drivers for HP DeskJets. They're all described in Devices.htm, but the following summary may help:

If you're using a DeskJet, you have the choice. Unfortunately, there's no way to know which is best until you've tried them all. Similar considerations apply to other makes of printer.

uniprint drivers

The uniprint drivers have a somewhat different kind of interface. They're described towards the end of the same Devices.htm file. To use them, change the driver specification as in the following example, that refers to an Epson

#!/bin/sh
/usr/local/bin/gs @stc500ph.upp -q -sPaperSize=a4 -dNOPAUSE -sOutputFile=- - -c quit

The differences here are:

Which drivers?

Another problem you might encounter is that it's possible to specify the drivers you want in your ghostscript executable when you build the port. It's quite possible that the drivers described in Devices.htm don't exist on your system. To find out, run ghostscript interactively with the -h (help) option:

$ gs -h
GNU Ghostscript 7.05 (2002-04-22)
Copyright (C) 2002 artofcode LLC, Benicia, CA.   All rights reserved.
Usage: gs  [switches]   [file1.ps file2.ps ...]
Most frequently used switches:  (you can use # in place of =)
  -dNOPAUSE   no pause after page           -q       'quiet', fewer messages
  -g<width>x<height>   page size in pixels  -r<res>  pixels/inch resolution
  -sDEVICE=<devname>   select device        -dBATCH  exit after last file
  -sOutputFile=<file> select output file: - for stdout,   |command for pipe,
                                         embed %d or %ld for page #
Input formats: PostScript PostScriptLevell PostScriptLevel2 PDF Available devices:
  x11 x11alpha x11cmyk x11gray2 x11gray4 x11mono x11rg16x x11rg32x md2k
  md5k md50Mono md50Eco md1xMono bj10e bj10v bj10vh bj200 bjc600 bjc800
  lips2p lips3 lips4 bjc880j lips4v uniprint dmprt epag escpage lp2000
  alc8600 alc8500 alc2000 alc4000 lp8800c lp8300c lp8500c lp3000c lp8200c
  lp8000c epl5900 epl5800 epl2050 epl2050p epl2120 lp7500 lp2400 lp2200
  lp9400 lp8900 lp8700 lp8100 lp7700 lp8600f lp8400f lp8300f lp1900
lp9600s
  lp9300 lp9600 lp8600 lp1800 mjc180 mjc360 mjc720 mj500c deskjet djet500
cdeskjet cdjcolor cdjmono cdj550 cdj670 cdj850 cdj880 cdj890 cdj1600
  cdj970 laserjet ljetplus ljet2p ljet3 ljet3d ljet4 ljet4d cljet5
  cljet5c cljet5pr lj5mono lj5gray pj pjxl pjxl300 pxlmono pxlcolor pcl3 hpdj ijs
  npdl rpdl gdi bmpmono bmpgray bmp16 bmp256 bmp16m bmp32b bmpsep1 bmpsep8
  faxg3 faxg32d faxg4 jpeg jpeggray pcxmono pcxgray pcx16 pcx256 pcx24b
  pcxcmyk pdfwrite bit bitrgb bitcmyk pbm pbmraw pgm pgmraw pgnm pgnmraw pnm 
pnmraw ppm ppmraw pkm pkmraw pksm pksmraw pngmono pnggray png16
  png256 png16m psmono psgray psrgb pswrite epswrite tiffcrle tiffg3
  tiffg32d tiffg4 tiff12nc tiff24nc tifflzw tiffpack nullpage
Search path:
  .  :  /opt/lib/ghostscript  :  /opt/lib/ghostscript/fonts  :
  /opt/lib/ghostscript/garamond  :  /usr/local/share/ghostscript/7.05/lib  :
  /usr/local/share/ghostscript/fonts
For more information, see /usr/local/share/ghostscript/7.05/doc/Use.htm.
Report bugs to bug-gs@ghostscript.com, using the form in Bug-form.htm.

PDF

PDF, or Portable Document Format, is a newer format for transferring print documents. Like PostScript, it comes from Adobe, and it is becoming increasingly important as a document interchange format on the Internet.

There are two ways to handle PDF:

Un like PostScript, an editor is available for PDF (Acrobat, the big brother of Acrobat Reader). Unfortunately, it's proprietary and not free, and worse still, it's not available for FreeBSD.

16. Networks and the Internet

Network layering; The physical network connection; Ethernet; Wireless LANs; The reference network;

In this part of the book we'll look at the fastest-growing part of the industry: networks, and in particular the Internet.

The industry has seen many different kinds of network software:

In this part of the book, we'll look only at the Internet Protocols. Thanks to its background, FreeBSD is a particularly powerful contender in this area, and we'll go into a lot of detail about how to set up and operate networks and network services. In the chapters following, we'll look at:

The rest of this chapter looks at the theoretical background of the Internet Protocols and Ethernet. You can set up networking without understanding any of it, as long as you and your hardware don't make any mistakes. This is the approach most commercial systems take. It's rather like crossing a lake on a set of stepping stones, blindfolded. In this book, I take a different approach: in the following discussion, you'll be inside with the action, not on the outside looking in through a window. It might seem unusual at first, but once you get used to it, you'll find it much less frustrating.

Network layering

One of the problems with networks is that they can be looked at from a number of different levels. End-users of PCs access the World Wide Web (WWW), and often enough they call it the Internet. That's just plain wrong. At the other end of the scale is the Link Layer, the viewpoint you'll take when you first create a connection to another machine.

Years ago, the International Standards Organization came up with the idea of a seven-layered model of networks, often called the OSI reference model. Why OSI and not ISO? OSI stands for Open Systems Interconnect. Since its introduction, it has become clear that it doesn't map very well to modern networks. W. Richard Stevens presents a better layering in TCP/IP Illustrated, Volume 1, page 6, shown here in Figure 16-1 .

Four-layer network model


. 16.1. Four-layer network model

We'll look at these layers from the bottom up:

The link layer

Data on the Internet is split up into packets, also called datagrams, which can be transmitted independently of each other. The link layer is responsible for getting packets between two systems that are connected to each other. The most trivial case is a point-to-point network, a physical connection where any data sent down the line arrives at the other end. More generally, though, multiple systems are connected to the network, as in an Ethernet. This causes a problem: how does each system know what is intended for it?

IP solves this problem by including a packet header in each IP packet. Consider the header something like the information you write on the outside of a letter envelope: address to send to, return address, delivery instructions. In the case of IP, the addresses are 32-bit digits that are conventionally represented in dotted decimal notation: the value of each byte is converted into decimal. The four values are written separated by dots. Thus the hexadecimal address 0xdf932501 would normally be represented as 223.147.37.1.

UNIX uses the notation 0x in a number to represent a hexadecimal number. The usage comes from the C programming language.

As we will see in Chapter 23, it makes debugging much easier if we understand the structure of the datagrams, so I'll show some of the more common ones in this chapter. Figure 16-2 shows the structure of an IP header.

IP Header


. 16.2. IP Header

We'll only look at some of these fields; for the rest, see TCP/IP Illustrated, Volume 1.

The network layer

The main purpose of the network layer is to ensure that packets get delivered to the correct recipient when it is not directly connected to the sender. This function is usually called routing.

Imagine routing to be similar to a postal system: if you want to send a letter to somebody you don't see often, you put the letter in a letter box. The people or machines who handle the letter look at the address and either deliver it personally or forward it to somebody else who is closer to the recipient, until finally somebody delivers it.

Have you ever received a letter that has been posted months ago? Did you wonder where they hid it all that time? Chances are it's been sent round in circles a couple of times. That's what can happen in the Internet if the routing information is incorrect, and that's why all packets have a time to live field. If it can't deliver a packet, the Internet Protocol simply drops (forgets about) it. You may find parallels to physical mail here, too.

It's not usually acceptable to lose data. We'll see how we avoid doing so in the next section.

The transport layer

The transport layer is responsible for end-to-end communication. The IP address just identifies the interface to which the data is sent. What happens when it gets there? There could be a large number of processes using the link. The IP header doesn't contain sufficient information to deliver messages to specific users within a system, so two additional protocols have been implemented to handle the details of communications between "end users."1). These end users connect to the network via ports, or communication end points, within individual machines.

TCP

The Transmission Control Protocol, or TCP, is a so-called reliable protocol: it ensures that data gets to its destination, and if it doesn't, it sends another copy. If it can't get through after a large number of tries (14 tries and nearly 10 minutes), it gives up, but it doesn't pretend the data got through. To perform this service, TCP is also connection oriented: before you can send data with TCP, you must establish a connection, which is conceptually similar to opening a file.

To implement this protocol, TCP packets include a TCP header after the IP header, as shown in Figure 16-3. This figure ignores the possible options that follow the IP header. The offset of the TCP header, shown here as 20, is really specified by the value of the IP

Header length field in the first byte of the packet. This is only a 4 bit field, so it is counted in words of 32 bits: for a 20 byte header, it has the value 5.

TCP Header with IP header


. 16.3. TCP Header with IP header

A number of fields are of interest when debugging network connections:

These three values are used to ensure efficient and reliable transmission of data. For each connection, TCP maintains a copy of the highest acknowledgment number received from the other side and a copy of all data that the other side has not acknowledged receiving. It does not send more than window size bytes of data beyond this value. If it does not receive an acknowledgment of transmitted data within a predetermined time, usually one second, it sends all the unacknowledged data again and again at increasingly large intervals. If it can't transmit the data after about ten minutes, it gives up and closes the connection.

UDP

The User Datagram Protocol, or UDP, Is different: it's an unreliable protocol. It sends data out and never cares whether it gets to its destination or not. So why do we use it if it's unreliable? Its faster, and thus cheaper. Consider it a junk mail delivery agent: who cares if you get this week's AOL junk CD-ROM or not? There will be another one in next week's mail. Since it doesn't need to reply, UDP is connectionless: you can just send a message off with UDP without worrying about establishing a connection first. For example, the rwhod daemon broadcasts summary information about a system on the LAN every few minutes. In the unlikely event that a message gets lost, it's not serious: another one will come soon.

UDP Header with IP header


. 16.4. UDP Header with IP header

Port assignment and Internet services

A port is simply a 16 bit number assigned to specific processes and which represents the source and destination end points of a specific connection. A process can either request to be connected to a specific port, or the system can assign one that is not in use.

RFC 1700 defines a number of well-known ports that are used to request specific services from a machine. On a UNIX machine, these are provided by daemons that listen on this port number- another words, when a message comes in on this port number, the IP software passes it to them, and they process it. These ports are defined in the file /etc/services. Here's an excerpt:

# Network services, Internet style
#
# WELL KNOWN PORT NUMBERS
#
ftp     21/tcp                #File Transfer [Control]
ssh     22/tcp                #Secure Shell Login
ssh     22/udp                #Secure Shell Login
telnet  23/tcp
smtp    25/tcp  mail          #Simple Mail Transfer
smtp    25/udp  mail          #Simple Mail Transfer
domain  53/tcp                #Domain Name Server
domain  53/udp                #Domain Name Server
...
http    80/tcp  www www-http  #World Wide Web HTTP
http    80/udp  www www-http  #World Wide Web HTTP

This file has a relatively simple format: the first column is a service name, and the second column contains the port number and the name of the service (either tcp or udp). Optionally, alternative names for the service may follow. In this example, smtp may also be called mail, and http may also be called www.

When the system starts up, it starts specific daemons. For example, if you're running mail, you may start up sendmail as a daemon. Any mail requests coming in on port 25 (smtp) will then be routed to sendmail for processing.

Network connections

You can identify a TCP connection uniquely by five parameters:

When you set up a connection, you specify the destination IP address and port number, and implicitly also the protocol. Your system supplies the source IP address; that's obvious enough. But where does the source port number come from? The system literally picks one out of a hat; it chooses an unused port number somewhere above the "magic" value 1024. You can look at this information with netstat:

$ netstat
Proto  Recv-Q  Send-Q  Local Address  Foreign Address       (state)
tcp4        0       0  presto.smtp    203.130.236.50.1825   ESTABLISHED
tcp4        0       0  presto.3312    andante.ssh           ESTABLISHED
tcp4        0       0  presto.2593    hub.freebsd.org.ssh   ESTABLISHED
tcp4        0       0  presto.smtp    www.auug.org.au.3691  ESTABLISHED

As you can see, this is the view on a system called presto. We'll see presto again in our sample network below. Normally you'll see a lot more connections here. For each connection, the protocol is tcp4 (TCP on IPv4). The first line shows a connection to the port smtp on presto from port 1825 on a machine with the IP address 203.130.236.50.

netstat shows the IP address in this case because the machine in question does not have reverse DNS mapping. This machine is sending a mail message to presto. The second and third lines show outgoing connections from presto to port ssh on the systems andante and hub.freebsd.org. The last is another incoming mail message from http://www.auug.org.au. Graphically, you could display the connection between presto and http://www.auug.org.au like this:



. 16.5.

Note that the port number for smtp is 25.

For various reasons, it's not always possible to connect directly in this manner:

The physical network connection

The most obvious thing about your network connection is what it looks like. It usually involves some kind of cable going out of your computer1), but there the similarity ends. FreeBSD supports most modern network interfaces:

The decision on which WAN connection you use depends primarily on the system you are connecting to, in many cases an Internet Service Provider or ISP. We'll look at ISPs in Chapter 18.

Ethernet

In the early 1970s, the Xerox Company chartered a group of researchers at its Palo Alto Research Center (PARC ) to brainstorm the Office of the Future. This innovative group created the mouse, the window interface metaphor and an integrated, object-oriented programming environment called Smalltalk. In addition, a young MIT engineer in the group named Bob Metcalfe came up with the concept that is the basis of modern local area networking, the Ethernet. The Ethernet protocol is a low-level broadcast packet-delivery system that employed the revolutionary idea that it was easier to resend packets that didn't arrive than it was to make sure all packets arrived. There are other network hardware systems out there, IBM's Token Ring architecture and Fiber Channel, for example, but by far the most popular is the Ethernet system in its various hardware incarnations. Ethernet is by far the most common local area network medium. There are three types:

  1. Originally, Ethernet ran at 10 Mb/s over a single thick coaxial cable, usually bright yellow in colour. This kind of Ethernet is often referred to as thick Ethernet, also called 10B5, and the line interface is called .AUI You may also hear the term yellow string (for tying computers together), though this term is not limited to thick Ethernet. Thick Ethernet is now obsolete: it is expensive, difficult to lay, and relatively unreliable. It requires 50 ohm resistors at each end of the cable to transmit signals correctly. If you leave these out, you won't get degraded performance: the network Will Not Work at all.
  2. As the name suggests, thin Ethernet is thin coaxial cable, and otherwise quite like thick Ethernet. It is significantly cheaper (thus the term Cheapernet), and the only disadvantage over thick Ethernet is that the cables can't be quite as long. The cable is called RG58, and the cable connectors are called BNC. Both terms are frequently used to refer to this kind of connection, as is 10 Base 2. You'll still see thin Ethernet around, but since it's effectively obsolete. Performance is poor, and it's no cheaper than 100 Mb/s Ethernet. Like thick Ethernet, all machines are connected by a single cable with terminators at each end.
  3. Modern Ethernets run at up to 1000 Mb/s over multi-pair cables called UTP, for Unshielded Twisted Pair. Twisted pair means that each pair of wires are twisted to minimize external electrical influenceafter all, the frequencies on a 1000 Mb/s Ethernet are way up in the UHF range. Unlike coaxial connections, where all machines are connected to a single cable, UTP connects individual machines to a hub or a switch, a box that distributes the signals. We'll discuss the difference between a hub and a switch on page 288. You'll also hear the terms 10BaseTP, 100BaseTP and 1000BaseTP

    Compared to coaxial Ethernet, UTP cables are much cheaper, and they are more reliable. If you damage or disconnect a coaxial cable, the whole network goes down. If you damage a UTP cable, you only lose the one machine connected to it. On the down side, UTP requires switches or hubs, which cost money, though the price has decreased to the point where it's cheaper to buy a cheap switch and UTP cables rather than the RG58 cable alone. UTP systems employ a star architecture rather than the string of coaxial stations with terminators. You can connect many switches together simply by reversing the connections at one end of a switch-to-switch link. In addition, UTP is the only medium currently available that supports 100 Mb/s Ethernet.

How Ethernet works

A large number of systems can be connected to a single Ethernet. Each system has a 48 bit address, the so-called Ethernet address. Ethernet addresses are usually written in bytes separated by colons (:), for example 0:a0:24:37:0d:2b. All data sent over the Ethernet contains two addresses: the Ethernet address of the sender and the Ethernet address of the receiver. Normally, each system responds only to messages sent to it or to a special broadcast address.

You'll also frequently hear the term MAC address. MAC stands for Media Access Control and thus means the address used to access the network link layer. For Ethernets I prefer to use the more exact term Ethernet address.

The fact that multiple machines are on the same network gives rise to a problem: obviously only one system can transmit at anyone time, or the data will be garbled. But how do you synchronize the systems? In traditional Ethernets, the answer is simple, but possibly surprising: trial and error. Before any interface transmits, it checks that the network is idlen. In the Ethernet specification, this is called Carrier Sense. Unfortunately, this isn't enough: two systems might start sending at the same time. To solve this problem, while it sends, each system checks that it can still recognize what it is sending. If it can't, it assumes that another system has started sending at the same time this is called a collision. When a collision occurs, both systems stop sending, wait a random amount of time, and try again. You'll see this method referred to as CSMA/CD (Carrier Sense Multiple Access/Collision Detect).

There are a number of problems with this approach:

With the point-to-point connections on a UTP-based network, you would think it would be possible to change some of this. After all, the connections look pretty much like the same wire that joins two modems to get her, and modems don't have collisions, and they do run in full-duplex mode. The problem is the hub: if you send a packet out to a hub, it doesn't know which connector to send it down, so it sends it down all of them, thus imitating the old Ethernet. To send it just to the destination, it would need to analyze the Ethernet address in every packet and know where to send it.

This is what a switch does: it learns the Ethernet addresses of each interface on the network and uses this information to send packets to only the line to which that interface is connected. There could be more than one if switches are cascaded. This also means that the line can run in full-duplex mode.

Nowadays the price differential between switches and hubs is very small; go into a computer market and you'll see that the prices overlap. If at all possible, buy a switch.

Transmitting Internet data across an Ethernet has another problem. Ethernet evolved independently of the Internet standards. As a result, Ethernets can carry different kinds of traffic. In particular, Microsoft uses a protocol called NetBIOS, and Novell uses a protocol called ZPX. In addition, Internet addresses are only 32 bits, and it would be impossible to map them to Ethernet addresses even if they were the same length. The result? You guessed it, another header. Figure 16-6 shows an Ethernet packet carrying an IP datagram.

Finding Ethernet addresses

So we send messages to Ethernet interfaces by setting the correct Ethernet address in the header. But how do we find the Ethernet address? All our IP packets use IP addresses. And it's not a good solution to just statically assign Ethernet addresses to IP addresses: first, there would be problems if an interface board or an IP address was changed, and secondly multiple boards can have the same IP address.

Ethernet frame with TCP datagram


. 16.6. Ethernet frame with TCP datagram

The chosen solution is the Address Resolution Protocol, usually called ARP. ARP sends out a message on the Ethernet broadcast address saying effectively "Who has IP address 223.147.37.1? Tell me your Ethernet address." The message is sent on the broadcast address, so each system on the net receives it. In each machine, the ARP protocol checks the specified IP address with the IP address of the interface that received the packet. If they match, the machine replies with the message "I am IP 223.147.37.1, my Ethernet address is 00:a0:24:37:0d:2b"

What systems are on that Ethernet?

Multiple systems can be accessed via an Ethernet, so there must be some means for a system to determine which other systems are present on the network. There might be a lot of them, several hundred for example. You could keep a list, but the system has to determine the interface for every single packet, and a list that long would slow things down. The preferred method is to specify a range of IP addresses that can be reached via a specific interface. The computer works in binary, so one of the easiest functions to perform is a logical and. As a result, you specify the range by a network mask: the system considers all addresses in which a specific set of bits have a particular value to be reachable via the interface. The specific set of bits is called the interface address.

For example, let's look forward to the reference network on page 294 and consider the local network, which has the network address 223.147.37.0 and the netmask 255.255.255.0. The value 255 means that every bit in the byte is set. The logical and function says "if a specific bit is set in both operands, set the result bit to 1; otherwise set it to 0." Figure 16-7 shows how the system creates a network address from the IP address 223.147.37.5 and the net mask 255.255.255.0.

Net mask


. 16.7. Net mask

The result is the same as the IP address for the first three bytes, but the last byte is 0: 223.147.37.0.

This may seem unnecessarily complicated. An easier way to look at it is to say that the 1 bits of the net mask describe which part of the address is the network part, and the 0 bits describe which part represents hosts on the network.

Theoretically you could choose your network mask bits at random. In practice, it's clear that it makes more sense to make network masks a sequence of binary 1 bits followed by a sequence of binary 0 bits. It has become typical to abbreviate the network mask to the number of 1 bits. Thus the network mask 255.255.255.0, with 24 bits set and 8 bits not set, is abbreviated to /24.The / character is always part of the abbreviation.

Address classes

When the Internet Protocols were first introduced, they included the concept of a default netmask. These categories of address were called address classes. The following classes are defined in RFC 1375:

16.1. Address classes
ClassAddress rangeNetwork maskNetwork address bitsHost address bitsNumber of systems
A0-127255.0.0.0/82416777216
B128-191255.255.0.0/161665536
C192-207255.255.255.0/248256
F208-215255.255.255.240/28416
G216-219(reserved)
H220-221255.255.255.248/2938
K222-223255.255.255.254/3112
D224-239(multicast)
E240-255(reserved)

This method is no longer used for specifying net masks, though the software still defaults to these values, but it is used for allocating networks. In addition you will frequently hear the term Class C network to refer to a network with 256 addresses in the range 192-223. This usage goes back to before RFC 1375.

Unroutable addresses

On occasion you may want to have addresses which are not visible on the global Internet, either for security reasons or because you want to run Network Address Translation (see page 393). RFC 1918 provides for three address ranges that should not be routed: 10.0.0.0/8 (with last address 10.255.255.255), 172.16.0.0/12 (with last address 172.31.255.255), and 192.168.0.0/16 (with last address 192.168.255.255).

Wireless LANs

An obvious problem with Ethernet is that you need a cable. As more and more machines are installed, the cabling can become a nightmare. It's particularly inconvenient for laptops: the network cable restricts where you can use the machine.

Wireless network cards have been around for some time, but in the last few years they have become particularly popular. Modern cards are built around the IEEE 802.11 series of standards.

The 802 series of standards cover almost all networking devices; don't let the number 802 suggest wireless networking. Ethernet is 802.3, for example.

They are usually PCMCIA (PC Card) cards, though some PCI cards are also available. Currently you're liable to come across the following kinds of cards:

Most current cards are 802.11b and run at up to 11 Mb/s. We'll concentrate on them in the rest of this section. They operate in the 2.4 GHz band, which is shared with a number of other services, including some portable telephones and microwave ovens. This kind of portable telephone can completely disrupt a wireless network. Interference and range are serious issues: wireless networks are generally not as reliable as wired networks.

Wireless cards can operate in up to three different modes:

How wireless networks coexist

Wireless networks have a number of issues that don't affect Ethernets. In particular, multiple networks can share the same geographical space. In most large cities you'll find that practically the entire area is shared by multiple networks. This raises a number of issues:

Encryption

As mentioned above, security is a big issue in wireless networks. The encryption provided is called Wired Equivalent Privacy or WEP, and it's not very good. Everybody connecting to the network needs to know the WEP key, so if anybody loses permission to access the network (for example, when changing jobs), the WEP keys need to be changed, which is a serious administrative problem. In some cases it's completely impractical: if you want to access a wireless network in an airport or a coffee shop (where they're becoming more and more common), it's not practical to use a WEP key. In fact, nearly all such public access networks don't use encryption at all.

As if that weren't bad enough, the WEP algorithm is flawed. Depending on the circumstances, it can take less than 10 minutes to crack it. Don't trust it.

So how do you protect yourself? The best solution is, of course, don't use wireless networks for confidential work. If you have to use a wireless network, make sure that anything confidential is encrypted end-to-end, for example with an ssh tunnel, which we'll look at on page 424.

The reference network

One of the problems in talking about networks is that there are so many different kinds of network connection. To simplify things, this book bases on one of the most frequent environments: a number of computers connected together by an Ethernet LAN with a single gate way to the Internet. Figure 16-8 shows the layout of the network to which we will refer in the rest of this book.

Reference network


. 16.8. Reference network

This figure contains a lot of information, which we will examine in detail in the course of the text:

17. Configuring the local network

In this chapter: Network configuration with sysinstall. Manual network configuration: Automatic configuration with DHCP; Configuring PC Cardnetworking cards; Setting up wireless networking. Routing; ISPsroute setup. Looking at the routing tables; Packet forwarding. Configuration summary.

In Chapter 16 we looked at the basic concepts surrounding BSD networking. In this chapter and the following two, well look at what we need to do to configure a network, first manually, then automatically. Configuring PPP is still a whole lot more difficult than configuring an Ethernet, and they require more prerequisites, so well dedicate Chapter 20, to that issue.

In this chapter, well first look at example.org in the reference network on page 294, since its the easiest to set up. After that, well look at what additional information is needed to configure machines on example.net.

Network configuration with sysinstall

To configure a network, you must describe its configuration to the system. The system initialization routines that we discussed on page 528 include a significant portion that sets up the network environment. In addition, the system contains a number of standard IP configuration files that define your systems view of the network. If you didnt configure the network when you installed your system, you can still do it now. Log in as root and start sysinstall. Select the Index, then Network Interfaces. You will see the menu of Figure 17-1 , which is the same as in Figure 6-4 on page 97. On a standard 80x25 display it requires scrolling to see the entire menu. The only real network board on this list is xl0, the Ethernet board. The others are standard hardware that can also be used as network interfaces.

Network setup menu


. 17.1. Network setup menu

Choose the Ethernet board, xl0 You get a question about whether you want to use IPv6 configuration. In this book we doesnt d discuss IPv6, so answer No. Next you get a question about DHCP configuration. We discuss DHCP configuration on page 302. If you already have a DHCP server set up, you may prefer to answer yes to this question, which is all you need to do. If you answer No, the next menu asks us to set the internet parameters. Figure 17-2 shows the network configuration menu after filling in the values.

Network configuration menu


. 17.2. Network configuration menu

Specify the fully qualified local host name. When you tab to the Domain: field, the domain is filled in automatically. We have chosen to call this machine presto, and the domain is example.org. In other words, the full name of the machine is presto.example.org. Its IP address is 223.147.37.2. In this configuration, all access to the outside world goes via gw.example.org, which has the IP address 223.147.37.5. The name server is located on the same host, presto.example.org. If the name server isnt running when this information is needed, we must specify all addresses in numeric form, as shown.

What happens if you dont have a domain name? If youre connecting to the global Internet, you should go out and get one-see page 318. But in the meantime, dont fake it. Just leave the fields empty. If youre not connecting to the Internet, of course, it doesnt make much difference what name you choose.

As is usual for a class C network, the net mask is 255.255.255.0. You dont need to fill in this informationif you leave this field without filling it in, sysinstall inserts it for you. Normally, as in this case, you wouldnt need any additional options to ifconfig

sysinstall saves configuration information in /etc/rc.conf. When the system starts the startup scripts use this information to configure the network. It also optionally starts the interface immediately. In the next section well look at the commands it uses to perform this function.

Manual network configuration

Usually FreeBSD configures your network automatically when it boots. To do so, it uses the configuration files in/etc. So why do it manually? There are several reasons:

We spend a lot of time discussing this point on the FreeBSD mailing lists. One things for sure: neither method of configuration is perfect. Both menu-based and text-file-based configuration schemes offer you ample opportunity to shoot yourself in the foot. But at the moment, the configuration file system is easier to check if you understand whats going on.Thats the reason for the rest of this chapter.

In this section, well look at the manual way to do things first, and then well see how to put it in the configuration files so that it gets done automatically next time. You can find a summary of the configuration files and their contents on page 551.

Describing your network

We saw that systems connect to networks via network interfaces. The kernel detects the interfaces automatically when it starts, but you still need to tell it what interfaces are connected to which networks, and even more importantly, which address your system has on each network. In addition, if the network is a broadcast network, such as an Ethernet, you need to specify a range of addresses that can be reached directly on that network, network mask.

Ethernet interfaces

Once we have understood these concepts, its relatively simple to use the ifconfig program to set them. For example, for the Ethernet interface on system gw, with IP address 223.147.37.5, we need to configure interface dcO. The network mask is the standard value for a class C network, 255.255.255.0. Thatsall we need to know:

# ifconfig dc0 inet 223.147.37.5 net mask 255.255.255.0 up

In fact, this is more than you usually need. The inet tells the interface to use Internet protocol Version 4 (the default), and up tells it to bring it up (which it does anyway). In addition, this is a class C network address, so the net mask defaults to 255.255.255.0. As a result, you can abbreviate this to:

# ifconfig dc0 223.147.37.5

Note that this is different from what Linux requires. With Linux you must supply explicit net mask and broadcast address specifications.

As we saw on page 290, it has become typical to abbreviate net masks to the character / followed by the number of 1 bits set in the network mask. ifconfig understands this usage, so if you wanted to set a non-standard network mask of, say, 255.255.255.240, which has 28 bits set, you could write:

# ifconfig dc0 223.147.37.5/28
Point-to-point interfaces

With a point-to-point interface, the software currently requires you to specify the IP address of the other end of the link as well. As we shall see in Chapter 20, there is no good reason to do this, but ifconfig insists on it. In addition, we need the network mask for a non-broadcast medium. The value is obvious1): 1 you can reach exactly one address at the other end, so it must be 255.255.255.255. With this information, we could configure the PPP interface on gw:

# ifconfig tun0 139.130.136.133 139.130.136.129 net mask 255.255.255.255

In fact, this is almost never necessary; in Chapter 20 well see that the PPP software usually sets the configuration automatically.

The loopback interface

The IP protocols require you to use an address to communicate with every systemeven your own system. Theoretically, you could communicate with your system via the an Ethernet interface, but this is relatively slow: the data would have to go through the network stack. Instead, there is a special interface for communicating with other processes in the same system, the loopback interface. Its name is lo0, and it has the address 127.0.0.1. Its straightforward enough to configure:

# ifconfig lo0 127.0.0.1

In fact, though, you dont even need to do this much work: the system automatically sets it up at boot time.

Checking the interface configuration

ifconfig doesnt just set the configuration: you can also use it to check the configuration. Its a good idea to do this after you change something:

$ ifconfig
dc0:  flags=8843< UP, BROADCAST, RUNNING, SIMPLEX, MULTICAST > mtu 1500
    inet 223.147.37.5 net mask 0xffffff00 broadcast 223.147.37.255
    inet6 fe80::280:c6ff:fef9:d3fa%dc0 prefixlen 64 scopeid 0x1
    ether 00:80:c6:f9:d3:fa
    media: Ethernet autoselect (100baseTX < full-duplex >)
    status: active
lp0:  flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> mtu 1500
lo0:  flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 net mask 0xff000000
tun0: flags=8051<UP, POINTOPOINT, RUNNING, MULTICAST > mtu 1500
    inet 139.130.136.133 -- > 139.130.136.129 net mask 0xffffffff
Other BSD systems require you to write ifconfig -a. to list the configuration of all interfaces, and FreeBSD still accepts it. Some commercial UNIX systems dont understand even this fag.

There are a number of things to note here:

The configuration files

The system startup scripts summarize this configuration information in a number of configuration variables .See Chapter 29 for more details. At the moment, the following variables are of interest to us:

Previously, FreeBSD also required you to set a variable network_interfaces, a list of the names of the interfaces to be configured. This variable now defaults to the value auto to specify that all interfaces should be configured. You only need to change it if you specifically want to exclude an interface from configuration.

For gw, we put the following information in /etc/rc.conf:

hostname=gw.example.org ifconfig_dc0=inet 223.147.37.5

We dont configure the tunO interface here; as well see in Chapter 20, the PPP setup works differently.

Automatic configuration with DHCP

Maintaining the network configurations for a number of machines can be a pain, especially if they're laptops that come and go. There's analternative for larger networks: use DHCP, the Dynamic Host Configuration Protocol. DHCP enables a machine to get configuration information automatically from the network. The concept is expandable, but typically you get an IP address and net mask and the names of the default name servers and routers. In terms of the configuration we've seen so far, this replaces running the ifconfig and route programs, and also the file /etc/resolv.conf, which describes the locations of name servers. We'll look at it on page 366.

There are two parts to DHCP: the client and the server.

DHCP client

To get a configuration, you run dhclient. In previous releases of FreeBSD, dhclient printed out information about the addresses it received. In Release 5, it does not print anything. Simply start it with the name of the interface:

# dhclient dc0

To assign an address automatically at boot time, put the special value DHCP in the ifconfig_dc0 variable:

ifconfig_dc0=DHCP
DHCP server

DHCP requires a server. The server is not included as part of the base system; instead, install the net/isc-dhcp3 port:

# cd /usr/ports/net/isc-dhcp3
# make install

To configure dhcpd, edit the configuration file /usr/local/etc/isc-dhcpd.conf. Here's an example:

ddns-update-style ad-hoc;

# 100 Mb/s Ethernet
subnet 223.147.37.0 net mask 255.255.255.0 {
  range 223.147.37.90 223.147.37.110;
  option domain-name-servers freebie.example.com, presto.example.com;
  option domain-name " example.com ";
  option routers gw.example.com;
  option subnet-mask 255.255.255.0;
  option broadcast-address 223.147.37.255;
  default-lease-time 86400;
  max-lease-time 259200;
  use-host-decl-names on;  use the specified name as host name
  host andante {
    hardware ethernet 0:50:da:cf:7:35;
  }
}

This configuration file tells dhcpd:

The variables default-lease-time and max-lease-time, which are specified in seconds, determine how long it will be before a system checks its configuration. The values here represent one day and three days respectively.

use-host-decl-names tells dhcpd to use the name on the host line as the host name of the system. Otherwise you would need an additional option host-name specification for every system. For one machine it doesnt makemuch difference, but if you have twenty such machines, you'll notice the difference.

One of the problems with dhcpd is that by default it doesnt allocate a static IP address. Theoretically you could attach a laptop to the same DHCP server and get a different address every time, but in fact dhcpd does its best to keep the same address, and sometimes you may find it impossible to change its mind. In this configuration file, though, we have explicitly told dhcpd about andante, which is recognized by its Ethernet address. This works relatively well for fixed machines, but theres problem with laptops and PC Card: dhcpd recognizes the network interface, not the machine, and if you swap the interface card, the IP address moves tothe new machine.

Starting dhcpd

The dhcpd port installs a sample startup file in the directory /usr/local/etc/rc.d. It's called isc-dhcpd.sh.sample, a name which ensures that it won't get executed. This file doesn't normally require any configuration; simply copy it to isc-dhcpd.sh in the same directory. This enables the system startup to find it and start dhcpd.

To start dhcpd during normal system operation, just run this same script:

# /usr/local/etc/rc.d/isc-dhcpd.sh start
Mar  14  15:45:09  freebie dhcpd: Internet Software Consortium DHCP Server V3.0rc10
Mar  14  15:45:09  freebie dhcpd: Copyright 1995-2001 Internet Software Consortium.
Mar  14  15:45:09  freebie dhcpd: All rights reserved.
Mar  14  15:45:09  freebie dhcpd: For info, please visit http://www.isc.org/products/DHCP
Mar  14  15:45:09  freebie dhcpd: Wrote 0 deleted host decls to leases file.
Mar  14  15:45:09  freebie dhcpd: Wrote 0 new dynamic host decls to leases file.
Mar  14  15:45:09  freebie dhcpd: Wrote 14 leases to leases file.
Mar  14  15:45:09  freebie dhcpd: Listening on BPF/xl0/00:50:da:cf:07:35/223.147.37.0/24
Mar  14  15:45:09  freebie dhcpd: Sending on BPF/xl0/00:50:da:cf:07:35/223.147.37.0/24
Mar  14  15:45:09  freebie dhcpd: Sending on Socket/fallback/fallback-net

When you change the configuration file /usr/local/etc/isc-dhcpd.conf, you must restart dhcpd:

# /usr/local/etc/rc.d/isc-dhcpd.sh restart

Configuring PC Card networking cards

We've looked at PC Card devices on page 159, but there are some special issues involved in configuring networking cards. Of course, ifconfig works with PC Card networking cards in exactly the same way as it does with PCI and ISA cards, but you cant configure them in the same manner at startup, because they might not yet be present.

On inserting a PC Card device, you will see something like this on the console:

Manufacturer ID: 01015751
Product version: 5.0
Product name: 3Com Corporation | 3CCFE575BT | LAN Card bus Card | 001 |
Functions: Network Adaptor, Memory
CIS reading done
cardbus0: Resource not specified in CIS: id=14, size=80
cardbus0: Resource not specified in CIS: id=18, size=80
xl0: <3Com 3c575B Fast Ether link XL> port 0x1080-0x10bf mem 0x88002400-0x8800247
f,0x88002480-0x880024ff irq 11 at device 0.0 on cardbus0
xl0: Ethernet address: 00:10:4b:f8:fd:20
miibus0: <MII bus> on xl0
tdkphy0: <TDK 78Q2120 media interface> on miibus0
tdkphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto

After this, ifconfig shows:

$ ifconfig xl0
  xl0:  flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
  ether 00:10:4b:f8:fd:20
  media: Ethernet autoselect (100baseTX <full-duplex>)

The card is there, but its not configured. FreeBSD uses the devd daemon to perform user land configuration after a card has been attached. We've already looked at devd on page 159. When devd establishes that the card is a networking card, it calls /etc/pccard_ether to configure it. In the following, we'll see how /etc/pccard_ether configures our xlO interface. It performs the following steps:

That's a lot of choice. What do you use when? That depends on what you want to do. The first thing to note is that nothing happens unless your interface name is in the variable removable_interfaces, and the variable ifconfig_xl0 exists. The question is, what do you put in ifconfig_xl0?

In principle, its the same as with other network cards: either IP address and other options, or DHCP. The third alternative is important, though. Lets consider the case where you want to start a number of services when the system is connected. You might want to run ntpdate, then start ntpd and rwhod, and you may want to mount some NFS file systems. You can do all this at startup with normal network cards, but /etc/pccard_ether isn't clever enough to do all that. Instead, create a file called /etc/startjf.xl0 and give it the following contents:

dhclient xl0
ntpdate freebie
killall ntpd
ntpd &
killall rwhod
rwhod &
mount t nfs -a

Dont forget to start DHCP or otherwise set the IP address, because this method bypasses the standard startups.

In addition, you put this in /etc/rc.conf:

devd_enable=YES
ifconfig_xl0=NO
removable interfaces="wi0 xe0 xl0"

The values in the last line only need to include xl0, of course, but its good to put in every interface name that you would possibly use.

Detaching network cards

When you remove a network card, devd invokes /etc/pccard_ether again. The actions are similar to the one it performs when the card is attached:

If you travel elsewhere with a laptop and suspend the system, make sure you unmount any NFS file systems first. You can't do it once you're no longer connected to the network, and its possible that things will hang trying to access NFS-mounted files.

Setting up wireless networking

We saw in Chapter 16 that wireless cards have a few more tricks up their sleeves than conventional Ethernets. To set them up correctly, you need to know:

Each of these translates into an ifconfig command. Here are some typical examples:

ifconfig wi0 ssid Example                       join Example network
ifconfig wi0 media autoselect media opt -adhoc  set BSS mode
ifconfig wi0 channel 3                          select channel 3 (if not in BSS mode)
ifconfig wi0 wepmode on                         turn encryption on (if using WEP)
ifconfig wi0 wepkey 0x42726f6b21                encryption key (for WEP)

When setting media options, you must also select the media, even if it is unchanged; thus the media autoselect in the example above.

You have a choice of where to put these specifications. For example, if you were connecting to the Example network, which is IBSS, you could put this in your /etc/rc.conf

devd_enable=YES
ifconfig_wi0="192.168.27.4 ssid Example media autoselect media opt adhoc \
channel 3 wepmode on wepkey 0x42726f6b21 removable interfaces="wi0 xe0 xl0"

You don't need to do anything special to become an IBSS master in an IBSS network: if there is no master already, and your card supports it, your system will become the IBSS master.

If, on the other hand, you were connecting to a non-encrypted network, you would not need the WEP key, and you might enter:

ifconfig_wi0="192.168.27.4 ssid Example media autoselect media opt ibss-master channel 3 wepmode off"

What we can do now

At this point, we have configured the link layer. We can communicate with directly connected machines. To communicate with machines that are not directly connected, we need to set up routing. We'll look at that next.

Routing

Looking back at our example network on page 294, we'll reconsider a problem we met there: when a system receives normal data packet, what does it do with it? There are four possibilities:

  1. If the packet is a broadcast packet, or if its addressed to one of its interface addresses, it delivers it locally.
  2. If its addressed to a system to which it has a direct connection, it sends it to that system.
  3. If its not addressed to a system to which it is directly connected, but it knows a system that knows what to do with the packet, it sends the packet to that system.
  4. If none of the above apply, it discards the packet.
17.1. The routing table
DestinationGateway Net maskTypeInterface
127.0.0.1 127.0.0.1 255.0.0.0 Host lo0
223.147.37. 255.255.255.0Directdc0
139.130.136.129 139.130.136.133 255.255.255.255Host tun0
Default 139.130.136.1290.0.0.0Gatewaytun0

These decisions are the basis of routing. The implementation performs them with the aid of a routing table, which tells the system which addresses are available where. We've already seen the net mask in Chapter 16, on page 290. Well see that it also plays a significant role in the routing decision. Table 17-1 shows a symbolic view of the routing table for gv.example.org. It looks very similar to the ifconfig output in the previous section:

And that's all there is to it! Well, sort of. In our example configuration, we're hidden in one corner of the Internet, and there's only one way out to the rest of the network. Things look different when you are connected to more than one network. On page 310 we'll look at the differences we need for the ISP example.net. In the middle of the Internet, things are even more extreme. There may be dozens of interfaces, and the choice of a route for a particular address may be much more complicated. In such an environment, two problems occur:

Adding routes automatically

FreeBSD comes with all the currently available routing software, primarily the daemon routed. The newer gated used to be included as well, but it is no longer available for free. It is available from http://www.nexthop.com/products/howto_order.shtml. An alternative in the Ports Collection is zebra.

All these daemons have one thing in common: you don't need them. At any rate, you dont need them until you have at least two different connections to the Internet, and even then its not sure. As a result, we wont discuss them here. If you do need to run routing daemons, read all about them in TCP/IP Network Administration, by Craig Hunt.

From our point of view, however, the routing protocols have one particular significance: the system expects the routing table to be updated automatically. As a result, it is designed to use the information supplied by the routing protocols to perform the update. This information consists of two parts:

Adding routes manually

As we saw in the previous section, the routing software uses only addresses, and not the interface name. To add routes manually, we have to give the same information.

The program that adds routes manually is called route. We need it to add routes to systems other than those to which we are directly connected.

To set up the routing tables for the systems connected only to our reference network (freebie, presto, bumble and wait), we could write:

# route add default gw

During system startup, the script /etc/rc.network performs this operation automatically if you set the following variable in /etc/rc.conf:

default router="223.147.37.5"  # Set to default gateway (or NO).

Note that we enter the address of the default router as an IP address, not a name. This command is executed before the name server is running. We cant change the sequence in which we start the processes: depending on where our name server is, we may need to have the route in place to access the name server.

On system gw, the default route goes via the tunO interface:

#default router="139.130.136.129" # Set to default gateway (or NO).
gateway enable="YES "             # Set to YES if this host will be a gateway.

This is a PPP interface, so you don't need a default router entry; if you did, it would look like the commented-out entry above. Later we'll see how PPP sets the default route.

We need to enable gateway functionality on this system, since it receives data packets on behalf of other systems. Well look at this issue in more depth on page 313.

ISP's route setup

At the ISP site, things are slightly more complicated than at example.org. Lets look at the gateway machine free-gw.example.net. It has three connections, to the global Internet, to example.org and to another network, biguser.com (the network serviced by interface pppO). To add the routes requires something like the following commands:

# route add default 139.130.237.65             igw.example.net
# route add -net 223.147.37.0 139.130.136.133  gw.example.org
# route add -net 223.147.38.0 -iface ppp0      local ppp0 interface

The first line tells the system that the default route is via gw.example.org. The second shows that the network with the base IP address 223.147.37.0 (example.org) can be reached via the gateway address 139.130.136.133, which is the remote end of the PPP link connected via ppp3. In the case of biguser.com, we dont know the address of the remote end; possibly it changes every time its connected. As a result, we specify the name of the interface instead: we know it's always connected via pppO.

The procedure to add this information to /etc/rc.conf is similar to what we did for the interface addresses:

The variable static_routes contains a list of the static routes that are to be configured.

For each route, a variable corresponding to the route name specified in static_routes, with the text route_ prepended. Unlike the interfaces, you can assign any name you want to them, as long as it starts with route. It makes sense for them to be related to the domain name, but they don't have to. For example, we would have liked to have called our network freebie.org, but there's a good chance that this name has been taken, so we called it example.org instead. The old name live in the name of the route, route_freebie. In the case of biguser.com, we have called the route variable route_biguser.

We put the following entries into free-gw's /etc/rc.conf:

default router="139.130.237.65"  # Set to default gateway (or NO).
static_routes="freebie biguser"  # list of static routes 
route_freebie="-net 223.147.37.0 139.130.237.129" 
route_biguser="-net 223.147.38.0 139.130.237.9"

Looking at the routing tables

You can show the routing tables with the netstat tool. Option -r shows the routing tables. For example, on freebie you might see:

# net stat -r
Routing tables

Internet:
Destination  Gateway            Flags  Refs    Use  Netif  Expire
default      gw                 UGSc     9    8732    rl0  
localhost    localhost          UH       0    1255    lo0  
223.147.37   link#2             UC       0       0    
presto       0:0:c0:44:a5:68    UHLW    13  139702    rl0    1151
freebie      0:a0:24:37:d:2b    UHLW     3   38698    lo0  
wait         0:60:97:40:fb:e1   UHLW     6    1062    rl0     645
bumble       8:0:20:e:2c:98     UHLW     2      47    rl0    1195
gw           0:60:97:40:fb:e1   UHLW     6    1062    rl0     645
broadcast    ff:ff:ff:ff:ff:ff  UHLWb    2    5788    rl0  

Theres lot to notice about this information:

The first column is the name of a host or a network to which packets can be sent, or the keyword default.

The second column, the gateway, indicates the path to the destination. This field differs significantly even from older versions of UNIX. It can be the name of a host (for example, gw), a pointer to an interface (link#2, which means the second Internet interface; the output from ifconfig is in the same sequence), or an Ethernet address (8:0:20:e:2c:98). Older versions of UNIX do not use the last two forms.

Well look at the fags below. The most important ones to note are G (gateway) and H (host).

The fields Refs, Use and Expire are only of interest when you're running a routing protocol. See the man page netstat(l) for more details.

Netif is the name of the interface by which the gateway can be reached. In the case of a link, this is the interface, so the Netif field is empty.

The order of the entries is not important. The system searches the table for a best fit, not a first fit.

The default entry points to gw, as we would expect. The interface, rl0, is the interface by which gw can be reached.

You will also get some additional output for IPv6 ("Internet "). If you're not using IPv6, you can ignore it. If it gets on your nerves, you can limit your view to IPv4 by entering the command netstat -rfinet. The -f fag specifies which address family you're interested in, and inet specifies IPv4.

Flags

Compared to earlier versions of netstat, the current version displays many more fags. The following table gives you an overview.

17.2. net stat -r tags values
FlagNameMeaning
1RTF_PROTO1Protocol specific routing flag 1
2RTF_PROTO2Protocol specific routing flag 2
3RTF_PROTO3Protocol specific routing flag 3
BRTF_BLACKHOLEJust discard pkts (during updates)
bRTF_BROADCASTThe route represents a broadcast address
CRTF_CLONINGGenerate new routes on use
cRTF_PRCLONINGProtocol-specified generate new routes on use
DRTF_JDYNAMICCreated dynamically (by redirect)
GRTF_GATEWAYDestination requires forwarding by intermediary
HRTF_HOSTHost entry (net otherwise)
LRTF_LLINFOValid protocol to link address translation
MRTF_MODIFIEDModified dynamically (by redirect)
RRTF_REJECTHost or net unreachable
SRTF_STATICManually added
URTF_UPRoute usable
WRTF_WASCLONEDRoute was generated as a result of cloning
XRTF_XRESOLVEExternal daemon translates proto to link address

Packet forwarding

We saw above that when a system receives packet that is not intended for itself, it looks for a route to the destination. In fact, this is not always the case: by default, FreeBSD just silently drops the packet. This is desirable for security reasons, and indeed its required by RFC 1122, but if you want to access the Internet via another machine on your local net, its less than convenient.

The rationale for this is that most systems are only connected to one network, and it doesn't make sense to have packet forwarding enabled. Earlier systems made this a kernel option, so that disabling packet forwarding also made the kernel fractionally smaller. In current versions of FreeBSD, the code is always there, even if it is disabled.

Its straightforward enough to set up your machine as a router (or gateway): you can set it with the sysctl command:

# sysctl -w net.inet.ip.forwarding=1
net.inet.ip.forwarding: 0 -> 1

In /etc/rc.conf you can set this with the variable gateway_enable:

gateway_enable="YES "  # Set to YES if this host will be a gateway.

Configuration summary

In the course of this chapter, we've discussed a number of different configurations. In this section we'll summarize the configuration for for free-gw.example.net, since it is the most complicated. You enter the following information in your /etc/rc.conf:

Without the comments, this gives the following entries:

hostname="free-gw.example.net"
ifconfig_rl0="inet 139.130.237.117"
default router="139.130.237.65"  # Set to default gateway (or NO).
static_routes="freebie biguser"  # Set to static route list
route_freebie="-net 223.147.37.0 139.130.136.133"
route_biguser="-net 223.147.38.0 -iface ppp0"
gateway enable="YES "            # Set to YES if this host will be a gateway.

For machine configured with DHCP, you might have:

hostname="andante.example.net"
ifconfig_wi0=DHCP

18. Connecting to the Internet

The physical connection; Establishing yourself on the Internet; Choosing an Internet Service Provider; Whos that ISP?; Making the connection.

To implement the reference network shown in the previous chapter, we need to do a lot of things that interface with the outside world. They can take some time, so we should look at them first:

The physical connection

Just two or three years ago, the way to connect to the outside world was simple: a phone line. Since then, things have changed quite a bit, and you may have quite a choice:

Establishing yourself on the Internet

The first thing you need to decide is the extent of your presence on the Net. There are various possibilities:

Does it matter? Thats for you to decide. Its certainly a very good idea to have your own domain name. As time goes on, your email address will become more and more important. If you get a mail address like 4711@flybynight.net, and Flybynight goes broke, or you decide to change to a different ISP, your mail address is gone, and you have to explain that to everybody who might want to contact you. If, on the other hand, your name is Jerry Dunham, and you register a domain dunham.org, you can assign yourself any mail address in that domain.

But how do you go about it? One way would be to pay your ISP to do it for you. You dont need to do that: its easy enough to do yourself on the World-Wide Web. You must be connected to the Internet to perform these steps. This implies that you should first connect using your ISP's domain name, then establish your domain name, and change to that domain.

Which domain name?

Well continue to assume that your name is Jerry Dunham. If you live in, say, Austin, Texas, you have a number of domain names you can choose from: dunham.org, dunham.com, dunham.net, or even dunham.tx.us if you want to use the geographical domain.

If you live in, say, Capetown, people will probably suggest that you get the domain dunham.za, the geographical domain for South Africa. The problem with that is that you are limiting yourself to that country. If you move to, say, Holland, you would have to change to dunham.nla situation only fractionally better than being bound to an ISP. The same considerations apply to dunham.tx.us, of course.

Your choice of domain name also affects the way you apply. In the following sections, I assume you take my advice and apply for an organizational rather than a geographical domain.

Preparing for registration

Once upon a time, registration was handled by Inter NIC, a professional body. Since then it has been delegated to commercial companies, and the quality of service has suffered correspondingly: they don't even appear to know the technical terms. For example, you may find them referring to a domain name as a "Web Address." Things are still deteriorating at the time of writing: additional companies are being allowed to register domain names, and the field seems to attract a lot of cowboys.

Registering a domain name

The only prerequisites for registering a domain name are:

First, check that the name is available:

$ whois duriham.org
No match for " DUNHAM.ORG ".

The Inter NIC Registration Services Host contains ONLY Internet Information
(Networks, ASN's, Domains, and POC's).
Please use the whois server at nic.ddn.mil for MILNET Information.

Next, try to find a reputable registrar. Immediately after the transfer of registrars from Inter NIC, the only company to offer this service was Network Solutions, but now there are many. I do not recommend Network Solutions: they're expensive and incompetent. If, as I recommend, you set up your mail server to refuse mail from servers without reverse mapping, you will not be able to communicate with them, since they do not have reverse DNS on their mail servers, and they use unregistered names for them. Judge for yourself what this says about their technical competence.

One registrar that many FreeBSD people use is Gandi (http://www.gandi.net/), which is slightly associated with the FreeBSD project. So far nobody has found anything negative to say about them. Unlike Network Solutions, their web pages are also relatively simple to understand.

Getting IP addresses

Once upon a time, it was possible to get IP addresses from Inter NIC, but this practice is now restricted to large allocations for ISPs. Instead, get the addresses from your ISP. Routing considerations make it impractical to move IP addresses from one place to another. Ifyou move a long distance, you should expect to change your IP addresses in the same way as you would change your telephone number.

Choosing an Internet Service Provider

In most cases, you will get your connection to the Internet from an Internet Service Provider, or ISP. As the name suggests, an ISP will supply the means for you to connect your system or your local network to the Internet. They will probably also supply other services: most ISPs cant live on Internet connections alone.

In this chapter we'll look at the things you need to know about ISPs, and how to get the best deal. Well concentrate on what is still the most common setup, PPP over dialup line with a V90 modem (56 kbps), which will give you a peak data transfer rate of about 7kB/s.

Whos that ISP?

As the Internet, and in particular the number of dialup connections, explodes, a large number of people have had the idea to become involved. In the early days of public Internet access, many ISPs were small companies run by very technical people who have seen a market opportunity and have grabbed it. Other ISPs were small companies run by not-so technical people who have jumped on the bandwagon. Still other ISPs are run by large companies, in particular the cable TV companies and the telephone companies. Which is for you? How can you tell to which category an ISP belongs? Do you care?

You should care, of course. Lets consider what you want from an ISP, and what the ISP wants. You want a low-cost, high-reliability, high speed connection to the Internet. You may also want technical advice and value-added services such as DNS (see Chapter 21) and web pages.

The main priority of a small ISP (or any other ISP, for that matter) is to get a good night's sleep. Next, he wants to ensure the minimum number of nuisance customers. After that, he wants to ensure that he doesn't go out of business. Only then is he interested in the same things that you are.

In the last few years, a large number of ISPs have gone out of business, and many more have merged with other companies. In particular, large companies frequently bought out small techie ISPs and then ran them into the ground with their incompetence. For a humorous view of this phenomenon, see the ''User Friendly cartoon series starting at http://ars.userfriendly.org/cartoons/?id=19980824.

Questions to ask an ISP

So how do you choose an ISP? Dont forget the value of word-of-mouthit's the most common way to find an ISP. If you know somebody very technical, preferably a FreeBSD user, who is already connected, ask himhe'll certainly be able to tell you about his ISP. Otherwise, a lot depends on your level of technical understanding. It's easy to know more about the technical aspects of the Internet than your ISP, but it doesn't often help getting good service. Here are a few questions to ask any prospective ISP:

Making the connection

After calling a few ISPs, you should be able to make decision based on their replies to these questions. The next step is to gather the information needed to connect. Use Table 18-1 to collect the information you need. See Chapter 20 for information about authentication, user name and password.

18.1. Information for ISP setup
InformationFill in specific value
IP address of your end of the link
IP address of the other end of the link
Kind of authentication (CHAP,PAP, login)
User or system name
Password or key
Primary Name Server name
Primary Name Server IP address
Secondary Name Server name
Secondary Name Server IP address
Pop (Mail) Server Name
News Server Name

19. Serial communications

Terminology; Asynchronous and synchronous communication; Serial ports; Modems; Modem commands.

UNIX has always had a high level of support for serial lines, but their purpose has changed dramatically. In the early 70s, the standard "terminal" was a Teletype KSR35, a 10-character-per-second serial printer with keyboard. Early UNIX serial line support was geared towards supporting these devices, either directly connected, or via a modem.

Even in the early 80s, when 4.2BSD introduced network support, things didn't change much: the network support used different hardware. By this time, the Teletypes had been replaced with glass ttys, in other words serial terminals with a monitor instead of a printer. The speeds had gone up from the 110 bps of the Teletype to 9600 bps, but the underlying principles hadn't changed.

It wasn't until the last 10 years that the glass ttys were replaced by display boards directly connected to the system bus, or by other machines connected by Ethernet. The role of the serial port has changed completely: nowadays, they're used mainly for mice and dialup Internet connections.

This change in use has invalidated a few basic concepts. Only a few years ago, the standard "high-speed" modem was a V 22bis 2400 bps modem, even then too slow for an Internet connection. The standard data communication line was 56 kb/s, and it was invariably a leased line. As a result, the Internet grew up assuming that connections were leased lines, and therefore permanently connected. Even today, the Internet protocols do not deal well with dialup access.

On the other hand, UUCP did use dialup access. As a result, provisions for dialup access in UNIX tend to be derived from UUCP. This doesn't make for smooth integration.

In this chapter, well look at the way FreeBSD handles serial communications, at how modems work, and how the two fit together.

Terminology

Any serial connection has two ends, which may be computers, terminals, printers or modems. In modem terminology, the computers are Data Terminal Equipment or DTE (this terminology arose at a time when the device connected to a modem was usually a terminal), and modems are Data Communication Equipment or DCE. You'll also sometimes hear the name dataset for a modem.

Asynchronous and synchronous communication

There are two different ways to transmit serial data, called synchronous and asynchronous communication. They grew up in different worlds:

Asynchronous communication

Asynchronous communication predates computers. It was originally developed to run teletypewriters, electrical typewriters that were run Offa serial data stream, the best-known of which were made by the Teletype corporation. These machines were frequently used to provide a remote transcript of what somebody was typing miles away, so they would typically print one character at a time, stop, and wait for the next. In the early days of UNIX, the standard terminal was a Teletype model KSR35, commonly just called teletype or tty (pronounced "titty").

Heres picture of a typical byte encoding:

Asynchronous byte encoding


. 19.1. Asynchronous byte encoding

This figure shows an encoding for the letter j, in binary 01101011. We'll see a number of things about it:

Synchronous communication

By contrast with asynchronous communication, synchronous communication comes from the mainframe world, and it assumes that data does not come one byte at a time. Instead, it transmits data in blocks. Each block is preceded by one or two SYN characters that tell the receiver that data is coming, and that enable it to determine the correct orientation of the bits in the data.

All modern modems use synchronous communication on the phone line, because it is more efficient, and its the basis of protocols such as SNA and X 25, but you will almost never see any other use of it in UNIX systems.

Serial ports

Nowadays, all PCs come equipped with two serial ports, which are called COM1: and COM2: in the DOS world. UNIX names are different, and FreeBSD calls these same devices sio0 and sio1. It's possible to connect up to four direct serial ports on a standard PC, but due to the design of the board, each one requires a separate IRQ line. If you put two serial ports on the same interrupt line, neither of them will work.

The first two devices, sio0 and sio1, normally use the default IRQs 4 and 3. By default, however, PC manufacturers put COM3: and COM4: also at IRQs 4 and 3. How can this work? It can't, if you also have COM1: and COM2: enabled at those IRQs. However, DOS tends to do only one thing at a time, so you can use different ports at different times on the same IRQ, as long as the interrupts arent enabled on more than one of the ports at a time. This restriction is unacceptable for UNIX, so we have to put them somewhere else. The only unused interrupt available to 8-bit boards is IRQ 5, originally intended for a second parallel printer port.

There's a very good chance that IRQ 5 will already be occupied. What can you do? If one of the boards has a 16-bit or better interface, you can check if one of the interrupts 10 to 15 is available. All EISA and PCI boards fit into this category, and so do ISA boards with two connectors to the motherboard. Unfortunately, a lot of ISA serial cards only have an8-bit interface. The only alternative Isa intelligent serial board that only occupies a single interrupt. In this case, you will probably have to build a custom kernel. See the man page sio(4).

Connecting to the port

Theoretically, serial line can consist of only three wires: a Receive Data line, often abbreviated to RxD, a Transmit Data line (TxD), and a Signal Ground line (SG). In fact, it is possible to get a link to work like this, but there are a number of problems:

We solve these questions, and more, by the use of additional lines. The most common standard is RS-232, so known as EIA-232, a standard for DCE to DTE connection. In Europe, it is sometimes confused with the CCITT V 24 standard, though V24 does not in fact correspond exactly to RS-232. Most external modems display some of these signals on LED, but modem manufacturers love to create alternative abbreviations for signal names. Here are the signals that RS-232 defines, with some of the more common abbreviations that you may see on external modems.

19.1. RS-232 signals and modem LEDs
RS-232 namePinModem LEDPurpose
PG1Protective ground. Used for electrical grounding only.
TxD2TD D1Transmitted data: data coming from the DTE to the modem.
RxD3RD D2Received data: data coming from the modem to the DTE.
RTS4Request to send. Indicates that the device has data to output.
CTS5Clear to send. Indicates that the device can receive input.
DSR6MR PW ONData set ready. Indicates that the modem is powered on and has passed self-test. On some modems, PW indicates that power is on, and MR indicates that it is operative.
SG7Signal ground. Return for the other signals.
DCD8CD M5Carrier detect. Indicates that the modem has connection with another modem.
DTR20DTR S1Data terminal ready. Indicates that the terminal or computer is ready to talk to the modem.
RI22AARing indicator. Raised by a modem to indicate that an incoming call is ringing. The AA indicator on a modem will usually flash when the incoming call is ringing.
AA"Auto Answer. "Indicates that the modem will answer an incoming call.
HS"High Speed. " Indicates that the modem is running at a higher speed than its minimum. Individual modems interpret this differently, but you can assume that something is wrong if your modem has this indicator and its off during transmission.
MNPIndicates that error correction is active.
OH"Off hook". Indicates that the modem has some connection with the phone line.
PWIndicates that modem power is on. May or may not imply DSR.

The line DCD tells the DTE that the modem has established a connection. Well look at how to use this information on page 335.

In addition to these signals, synchronous modems supply clocks on pins 17 and 19. For more details about RS-232, see RS-232 Made easy by Martin Seyer.

When can I send data?

There are two ways to determine if the other end is prepared to accept data: hardware handshaking and software handshaking. Both are also referred to as flow control. In each case, the handshaking is symmetrical. Well look at it from the point of view of the DTE, because this is the more common viewpoint.

In hardware handshaking, the DCE raises CTS (Clear to Send) when it's ready to accept input. The DTE only transmits data when CTS is asserted from the other end. You'll often see that the DTE asserts RTS (Request to send) when it wants to send data. This is a throwback to the days of half-duplex modems, which could only transmit in one direction at a time: RTS was needed to switch the modem into send mode.

Software handshaking is also called X-on/X-off The DCE sends a character (X-off, which corresponds to Ctrl-S) when the buffer is full, and another (X-on, corresponding to Ctrl-Q) when there is space in the buffer again. You can also use this method on a terminal to temporarily stop the display of a lot of data, and then restart it. It's no longer a good choice for modems.

For hardware handshake to work, your modem must be configured correctly, and you must have the correct cables. If it isn't, the symptoms will be very slow response when transferring large quantities of data: at a higher level, TCP can recover from these overruns, but it takes at least a second to do so every time. Well see hatcheck that your modem has the correct kind of fow control on page 333.

Modems

A modem is a device that transfers digital data into a form suitable for transmission over a transmission line, which is usually a telephone line. Telephone lines are limited to a frequency of about 3.6 kHz, and this limited the speed of older modems to about 1200 bits per second. Modern modems use many sophisticated techniques to increase the speed way beyond this. Current modems transmit at 56 kilobits per second.

Lets consider the modem connection in the reference network on page 294, which is repeated in figure 19-2 . As we can see, there are three connections:

You'll also note another value specified here: the connection between the modems is 2,400 baud. Isnt a baud the same thing as a bit per second? No, not always. The term baud is a representation of the frequency of data on a serial line. On the connections between the systems and the modem, which handle raw digital data, it corresponds to the bit rate. On the modem line, it doesn't. Here, it indicates that 2,400 units of data are sent per second.

Unfortunately, many people use the term baud where bit rate should be used. This didn't make any difference in the old days with simple modems where the bit rate and baud rate were the same, but nowadays its confusing

Network modem connection


. 19.2. Network modem connection

Modem speeds

Two factors determine the data transmission speed of a modem: the protocol and the use of data compression. Table 19-2 on page 332 gives an overview of modem protocols and their speeds.

Currently, the most popular modem protocol is V90. V 90 pushes modem technology to the limit, and it only works when the other end of the link is a digital (ISDN) connection. You can't get a 56 kb/s connection with any kind of analogue modem at the other end. As a result, they're really only suitable for connection to a large ISP site. In addition, the actual speed of the connection depends greatly on the telephone line quality, so the difference between a V 90 and a V 34bis modem may not be as much as it appears.

Data compression

In addition, you usually have a choice of data compression: V 42bis or MNP-5. The choice depends on what the modem at the other end of the line does. You can set most modems to negotiate either protocol. These protocols include related error correction standards, called V 42 or MNP2-4 respectively. If you believe the sales claims, these compression protocols will give you up to 100% increase in data throughput. Consider this the upper limit; a lot of data is binary, and when ftp'ing a typical gzipped tar archive, you will probably get almost no speed improvement.

Data compression has one negative side: it increases the data rate, but it also increases latency, the time it takes for data to get from the local DTE to the remote DTE. The data doesn't take as long on the line, but it spends more time in the modems being compressed and uncompressed. If you're running a protocol like PPP that supplies optional compression in the software, you may find it advantageous to turn off compression. Well look at that again in Chapter 20, Configuring PPP.

19.2. Modem protocols and speeds
ProtocolSpeed (bps)
Bell 203300
V 21300
Bell 2121200
V 221200
V 22bis2400
V 329600
V 32bis14400
V 3428800
V 34bis33600
V 9056000

The link speed

The standard PC serial hardware can run at speeds that are a fraction of 115,200 bps (in other words, 115200 divided by a small integer). This gives the following combinations:

19.3. Serial line speeds
DivisorSpeed (bps)
1115200
257600
338400
428800
523 040
619 200

You'll notice that it cant run at 33600 or 56000 bps. Also, looking at the example above, you'll note that all three links run at different speeds. How can that work? Only afew years ago, it wouldn't, but modern modems can buffer data. For example, the ISP can send data to the modem far faster than the modem can send it to the other modem. It stores the data in internal memory until it can be transmitted. This can also happen at the other end. If you misconfigure your line so that the local link runs at 9600 bps, things will still work, but of course the total speed is the speed of the slowest link, in this case

9600 bps.

This flexibility brings a problem with it: the modem can't know inadvance how fast the connection to the computer is. It needs a way to find out. The modem solves the question of local line speed by a trick: all commands start with AT or at (you're not allowed to mix cases, like aT or At). It can recognize these characters even if they arrive at the wrong speed, and thus it can establish the speed of the connection.

Dialing out

Nowadays, all modems are capable of dialing. That wasnt always the case, and in some old documentation you may find references to an Auto-Call Unit or ACU, which is simply the dialler part of a modem connected via a separate port. Typically, one ACU could serve multiple modems.

Nearly every modern modem uses a command set designed by Hayes Corporation, which is thus called the Hayes Command Set. We'll look at it in the following section. It is also sometimes called the AT command set, because nearly all the commands start with the sequence AT. The CCITT also created an autodial recommendation, K25, which was occasionally implemented, but now appears to be dead.

Modem commands

Modern modems store their state in a number of registers, called S registers. The register use varies somewhat from manufacturer to manufacturer, but most modems have a number in common. They each store a one-byte value, ranging between 0 and 255. Here's a list of the more important ones for a Rockwell V 34 chip set. The name of the chip set is not the same as the name of the modem. You'll note that one of the commands enables you to find out the chip set version, as we'll see in the example that follows.

19.4. Selected S registers
Register numberPurpose
soNumber of rings until auto-answer 0 disables auto-answer. Set to 0 for no automatic answer, or 1 for auto-answer
s2The escape character, which lets you return from online mode to command mode. Normally, this character is a + To return to command mode, wait a second after any previous input, enter +++, and wait a second, after which the modem should reply with OK
s6The time, in seconds, to wait before blind dialing. If you have set your modem to not wait for a dial tone (maybe because it doesn't understand the dial tone), it will wait this long and then try to dial anyway
S7The number of seconds to wait after dialing before DCD must be asserted (before a connection is established). If this is set too short, you will not be able to establish a connection. If its too long, you will waste time when there is no answer or the line is busy
S11The duration of DTMF (dialing) tones. If these are set incorrectly, the telephone exchange may not understand the number you dial

The AT command set tells the modem to do something specific. Here are some of the more important ones.

19.5. Selected AT commands
CommandMeaning
A/Redial the last number
ATAAnswer an incoming call manually This is an alternative to auto-answer by setting S0
ATDnumberDial number This command has a large number of options, but if your modem is set up correctly, you probably wont need any of them
ATEnumberEnable command echo if number is 1, disable it if number is 0. The setting of this parameter can be important for some chat scripts, which may not respond correctly otherwise
ATH0Disconnect the line
ATInumberDisplay modem identification. The values of number vary from one modem to the next. See the examples below
ATLnumberSet the speaker volume number ranges from 0 to 3. 0 means "speaker off," 3 is the loudest
ATMnumberDetermine when the speaker is on.
  • 0 means "always off,"
  • 1 means "speaker on until connect,"
  • 2 means "speaker always on,"
  • 3 means "speaker off during dialing and receiving "
ATO0Go back online from command mode. You dont need this command when dialing: the modem automatically goes online when the connection is established
ATPSelect pulse dial. If your exchange doesnt understand DTMF (tone) dialing, you should set this mode. Never use it if your exchange understands DTMF: pulse dialing (also known as steam dialing) is much slower
ATQnumberIf number is 0, suppress result codes (like OK after every command) If number is 1, enable them. This value can be of importance for chat scripts
ATSr=nSet the value of S register r to n
ATSnumber?Display the contents of an S register. See the example below
ATTSet tone (DTMF) dialing
ATVnumberIf number is 0, return result codes in numeric form. If its 1, return text. Dont rely on either form to be consistent from one modem to the next
ATXnumberDetermine the form of the result codes. This depends a lot on the manufacturer, but its important for chat scripts. If you run into trouble, with chat scripts, check your modem documentation
ATZReset modem configuration to default values
AT&KnumberSelect flow control method. Normally, 3 enables RTS/CTS flow control, which is what you want
AT&RnumberIf number is 0, CTS is only asserted if the DTE asserts RTS, even if the modem is able to receive data If its set to 1, it behaves normally Make sure this value is set to 1
AT&TnumberPerform modem-specific test number. This command is the origin of the statement: "UNIX is a trademark of AT&T in the USA and other countries. AT&T is a modem test command "
AT&VView the current configuration. See the example below
AT&WnumberStore the current configuration as profile number. Most external modems can store two profiles, or configurations. If number is not specified, write the profile specified in a previous AT&Y command. See the example below
AT&YnumberDecide which profile (0 or 1) will be loaded when the modem is reset, and which will be written by the command AT&W

Dialing out manually

In this section, we'll look at what needs to be done to establish a dial-out connection. You don't normally do this yourself: some software will do it for you automatically its useful to know what goes on, though: it can be of immense help in solving connection problems.

There are two distinct things that you want to do with the modem: first, you send commands to the modem to set up the link. Once the link is established, you dont want to talk to the modem anymore, you want to talk to the system at the other end of the link.

In the old days, the system used a separate ACU to establish the connection, and the solution was simple: the system issued the dialing commands to the ACU and opened the modem in such a manner that the open did not complete until a connection had been established. Nowadays, the modem handles both dialing and the establishment of connection. But to do so, the system has to open the modem before communication has been established.

The terminal parameter clocal enables communication with a device that is not asserting DCD (such as a modem that hasnt established a connection yet). When it starts, the software sets clocal. When it has finished talking to the modem and wants to wait for the connection to be established, it resets (turns off) clocal and waits for DCD. You can check this with the stty command:

stty -f /dev/cuaa2 -a
ppp disc; speed 57600 baud; 0 rows; 0 columns;
lflags: -icanon -isig -iexten -echo -echoe -echok -echoke -echonl
        -echoctl -echoprt -altwerase -noflsh -tostop -flusho pendin
        -nokerninfo -extproc
iflags: -istrip -icrnl -inlcr -igncr -ixon -ixoff -ixany -imaxbel ignbrk
        -brkint -inpck ignpar -parmrk
oflags: -opost -onlcr -oxtabs
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb crtscts -dsrflow
        -dtrflow mdmbuf
cchars: discard=^O; dsusp=^Y; eof=^D; eol=<undef>;
        eol2=<undef>; erase=^?; intr=^C; kill=^U; lnext=^V;
        min=1; quit=^\; reprint=^R; start=^Q; status=^T;
        stop=^S; susp=^Z; time=0; werase=^W;

This example, taken when the modem is connected, shows clocal reset. As you can see, this is indicated by the text -clocal.

Theres problem here: what if this line is also enabled for dialup? As we shall see on page 338, there will be a getty process in the process of opening the line. It won't succeed until DCD is asserted, so we can dial with no problem. But when the connection is established, how do we stop getty from being there first?

The FreeBSD solution is to create separate devices for each case. For the second serial port, sio1, the system creates a file /dev/cuaal for dialing out, and /dev/ttydl for dialing in. If cuaal is open, an open on ttydl does not complete when connection is established.

Dialing outan example

For an example of what you might look at, lets consider a manual dialup to an ISP. This assumes that you are using user PPP (see page 348) and that have an entry ISP in your /etc/ppp/ppp.conf. If you don't have an entry for an ISP, you can still test the modem, but in this case you won't be able to dial. In this case, simply omit the text ISP.

ppp ISP
User Process PPP. Written by Toshiharu OHNO.
Using interface: tun0
Interactive mode
ppp ON freebie> term      go into direct connect mode
Enter to terminal mode.
Type `~?' for help.
at                           synchronize with the modem
OK
at&v                         look at the modem profile
ACTIVE PROFILE:
B0 E1 L0 Ml N1 Q0 T V1 W0 X4 Y0 &C1 &D2 &G0 &J0 &K4 &Q5 &R1 &S0 &T5 &X0 &Y0
S00:000 S01:000 S02:043 S03:013 S04:010 S05:008 S06:002 S07:060 S08:002 S09:006
S10:014 S11:090 S12:050 S18:000 S25:005 S26:001 S36:007 S37:000 S38:020 S44:020
S46:138 S48:007 S95:000

STORED PROFILE 0:
B0 E1 L0 M1 N1 Q0 T V1 W0 X4 Y0 &C1 &D2 &G0 &J0 &K4 &Q5 &R1 &S0 &T5 &X0
S00:000 S02:043 S06:002 S07:060 S08:002 S09:006 S10:014 S11:090 S12:050 S18:000
S36:007 S37:000 S40:104 S41:195 S46:138 S95:000

STORED PROFILE 1:
B0 E1 L1 M1 N1 Q0 T V1 W0 X4 Y0  &C1 &D2  &G0 &J0 &K3 &Q5 &R1 &S0 &T5 &X0
S00:000 S02:043 S06:002 S07:060 S08:002 S09:006 S10:014 S11:090 S12:050 S18:000
S36:007 S37:000 S40:104 S41:195 S46:138 S95:000

TELEPHONE NUMBERS:
0=T1234567890    1=
2=               3=

OK

The term profile refers to a set of the complete configuration information for the modem. External modems can usually store two different profiles. Some modems may not have any stored profiles, or they may have a different number. The AT&V command shows the current configuration (" active profile ") and the two stored profiles. The first line refects the parameters set with AT commands (for example, L0 means that the command ATL0, turn off the speaker, has been issued). The next two or three lines refect the values of the Sregisters. In addition, this modem can store up to four telephone numbers, a feature that is seldom of great interest.

If you look at this profile, you'll notice that the active profile includes the parameter &K4 This means "use XON/XOFF flow control " This is not desirable: it's better to use RTS/CTS flow control. To fix it,

at&k3    set RTS/CTS flow control
OK
at&w     write the active profile
OK
at&v     and check
ACTIVE PROFILE:
B0 E1 L0 M1 N1 Q0 T V1 W0 X4 Y0 &C1 &D2 &G0 &J0 &K3 &Q5 &R1 &S0 &T5 &X0 &Y0
S00:000 S01:000 S02:043 S03:013 S04:010 S05:008 S06:002 S07:060 S08:002 S09:006
S10:014 S11:090 S12:050 S18:000 S25:005 S26:001 S36:007 S37:000 S38:020 S44:020
S46:138 S48:007 S95:000

STORED PROFILE 0:
B0 E1 L0 M1 N1 Q0 T V1 W0 X4 Y0 &C1 &D2 &G0 &J0 &K3 &Q5 &R1 &S0 &T5 &X0
S00:000 S02:043 S06:002 S07:060 S08:002 S09:006 S10:014 S11:090 S12:050 S18:000
S36:007 S37:000 S40:104 S41:195 S46:138 S95:000

STORED PROFILE 1:
B0 E1 L1 M1 N1 Q0 T V1 W0 X4 Y0 &C1 &D2 &G0 &J0 &K3 &Q5 &R1 &S0 &T5 &X0
S00:000 S02:043 S06:002 S07:060 S08:002 S09:006 S10:014 S11:090 S12:050 S18:000
S36:007 S37:000 S40:104 S41:195 S46:138 S95:000

TELEPHONE NUMBERS:
0=T1234567890    1=
2=               3=

OK

The active profile includes the parameter &Y0, so the AT&W command writes back to stored profile 0.

The AT&V command doesn't show all the S registers. Some of them relate to the current state of the modem, and arent part of the configuration. For example, my modem includes an S register S86, the Call Failure Reason Code If a call fails, it could be interesting to look at it. To do so:

ats86?     show contents of S86
012        Connection dropped by other end

With this background, we can now proceed to establish a connection:

atd1234567     just dial
CONNECT 57600
ppp ON freebie>
PPP ON freebie>

Dialing in

Traditionally, UNIX distinguishes between local serial terminals and terminals connected by modem by whether they assert the DCD signal. It starts a getty (for Get TTY) process for each line getty opens the line, but for modems the line state is set in such a way that the call to open does not complete until the DCE asserts DCD. This is done by resetting the tag clocal. If you look at the line state with the stty program, it will show -clocal if the flag is reset.

To set up a line for dialing in, add information about the line in the file /etc/ttys. The default file contains a number of lines like:

ttyd0.    "/usr/libexec/getty std 9600".    unknown off secure

This information has the following meaning:

After changing /etc/ttys, send init (process 1) a HUP signal to tell it to re-read /etc/ttys:

# kill -1 1

20. Configuring PPP

Quick setup; How PPP works; The information you need to know; Setting up user PPP; Setting up kernel PPP; Things that can go wrong.

Two protocols support connection to the Internet via modem: SLIP (Serial Line Internet Protocol) and PPP (Point to Point Protocol). As the name suggests, SLIP supports only IP. It is an older, less rugged protocol. Its only advantage is that it may be available where PPP isn't. If you have the choice, always take PPP: it differs from SLIP in being able to handle multiple protocols simultaneously, and its also used on many DSL links (PPP over Ethernet or PPPoE). In this chapter, well look only at PPP.

PPP can perform a number of functions:

FreeBSD provides two versions of PPP:

If you have a DSL link, you dont have a choice of version: currently, only User PPP supports PPPoE.

Quick setup

The following sections go into some detail about how PPP works. Its not completely necessary to know it all to set up PPP. If youre in a hurry, you can move onto the configuration summaries on page 348 for user PPP, or page 359 for kernel PPP.

How PPP works

The following steps are necessary to set up a PPP connection:

The interfaces

Most network interfaces are dedicated to networking. For example, an Ethernet adapter can't be used for anything else. Serial lines are different: you could also use them to connect a mouse or even remote terminal. Theres another difference, too: you access serial lines via their device names. You access network interfaces via the ifconfig program, because they don't usually have device names - in technical jargon, they're in a separate name space from files. How do we solve this conflict?

The solution may seem a little surprising: PPP uses two different devices for each connection. You decide which serial line you want to use, and the software chooses a network interface for you, though you can override this choice if you're using user PPP. For example, your serial line might be called /dev/cuaa0, /dev/cuaal or /dev/cuaa2, while your interface will be called tun0 or tun1 (for user PPP), or ppp0 or ppp1 (for kernel PPP). Its possible to connect to a DSL line without PPP, but when you use PPPoE, you also have two devices, the Ethernet interface and tunO (Kernel PPP does not support PPPoE).

The tunnel device uses a device interface called /dev/ton, where n is a digit, to read and write to the other side of the corresponding network interface.

User PPP runs in user space, so it does require a device name for the network interface, for example tun0. It uses this device to read and write to the back end of the tunnel interface.

Dialing

If you're running a PPP connection over a dial-up link, you'll need to establish a telephone connection, which is still called dialing. Thats modem function, of course, and its not defined in the PPP standard.

User PPP includes both built-in dialing support and external dialing support, while kernel PPP supplies only the latter. In practice, the only difference is the way your configuration files look. Well look at these when we discuss the individual implementations.

You don't need to dial for a DSL connection.

Negotiation

Once the connection is established and the PPP processes can talk to each other, they negotiate what PPP features they will use.1) The negotiation is successful if the two sides can agree on a functional subset of the features both would like to have.

For each feature of the link, PPP negotiation can perform up to two actions. User PPP uses the following terms to describe them, viewed from the local end of a link: