Guidelines for using the ATOMIC lab machines

Postscript version

Scheduling and Machine Use

This section describes how to allocate a machine for your use, and the conventions to follow using that machine. If you are just using the machines to gather performance data and not doing any kernel development, it is sufficient to read this section.

Reserving a Lab Machine

Before using one of the machines in the lab, it must be reserved using the reservation form at http://vex.isi.edu:1080/cgi-bin/atomic-schedule/reservation_form. Be sure your name, email address and the date are all present, chose a type, time period and machines, and submit. If you will be using the scheduler frequently, talk to Ted Faber (faber@isi.edu) about setting defaults for name and email.

There are two types of reservations, shared and exclusive. A shared reservation indicates that you are using the machine, and others can also do so without interfering with you. An exclusive reservation means that no one else should use the machine, and that the person with the exclusive reservation may reboot the machine at any time. You should get an exclusive reservation if you are doing the following types of work:

Shared reservations should be used if you want to pin down a machine for something use as a debugging console, or user level development on a lab machine.

You can change or delete a reservation by submitting a new request. You can only change requests that were made in your name.

The use of time period blocks are an artifact of the scheduler originally being written as a demonstration scheduler for DARTNET. The blocks roughly correspond to the blocks DARTNET was being scheduled in at the time.

The scheduler is a good tool for making sure that the machines are allocated fairly in times of high use. Use common sense and courtesy in its use. If you want to try a ten minute test and the machines are all idle, you probably do not need a reservation. If you want to guarantee that the machine isn't rebooted while you video conference from there, make the reservation. On the other hand if you made an exclusive reservation two minutes before you plan to use a machine, log on and find someone using the machine, let them know you have reserved the machine before rebooting it.

An exclusive reservation gives you the right to reboot the machine and to have anyone else who is logged onto the machine log out. (Rebooting the machine is a pretty good way to remove all users, actually.)

The scheduler is maintained by Ted Faber. Contact him if you have comments about it.

Lab Machine Etiquette

You should return any machine you reserve exclusively to the same state you it was in when you reserved it. This state includes:

N.B.: restoring the FAST kernel is as simple as myri_configure FAST && reboot

Again, restoring the system state is tempered with common sense. If you have a long suite of experiments to run using a non-standard network configuration, its acceptable to note the change to the atomic-lab-users mailing list (described below) and leave things set up. Again, the important point is that people who may want to use the machines know how they are configured, and how to restore them to a default configuration.

Remote Lab Usage

For many people it's more convenient to use the lab from home or USC. Remember that the lab's golden rule is:
Leave Your Machines Like You Found Them

When you're remote, there's a second important rule:

Be Secure In All Lab Access

What do these rules really mean?

To Leave Your Machines Like You Found Them, you need to make sure that you can reboot your machines with the current default kernels. If you're remote, you should be sure that you're not going to hang the machine in some horrible way that prevents you from Leaving Them Like You Found Them. Remember, you can't physically go to the console and power cycle the thing if it hangs when you're remote. Therefore either

If these options fall through, you should be prepared to drive from where you are to ISI in order to power cycle/reboot the machine before your reservation expires. Simply extending your reservation to cover until the next time you plan to be in is considered antisocial.

To Be Secure in All Lab Access, you need to be aware that each su1 command requires you to type in the root password. Since there have been snooping attacks between campus and ISI before, this means you must insure that your network connection is encrypted. The best way to do this is to use the secure shell (ssh) to log in to your machines. Note that S/Key is not sufficient here, since it does not encrypt the connection after you've logged in.

Some documentation on ssh is available at http://www.isi.edu/action/secure-shell.html. ssh is supported on darkstar and zephyr, as well as on the FreeBSD lab machines. If you are using one of the suns that does not support ssh, use ssh to connect to darkstar or zephyr, then rlogin to the lab machine of your choice.

Finally, if your terminal is in a public or only moderately secure place, make sure you lock it (perhaps with xautolock or xlock) before leaving it unattended.

This section contributed by John Heidemann

Mailing Lists

There is an ISI mailing list, atomic-lab-users, that provides communication between users of the lab. The maintainers use it to propagate updates to users, and users should use it to coordinate large usages of the lab. For example if your project has a deadline arriving and you need access to many lab machines, it is polite to let the other users know so they can schedule their work accordingly. To get on the list, mail Ted Faber.

Software Installation on the FreeBSD Systems

In an effort to minimize confusion, we are trying to make sure that the same software is installed on all the machines. We are making use of both the FreeBSD package system and the shared repository at ~d7share/bin.FreeBSD.

Software for which a FreeBSD package (or port) exists and is adequate will be installed using that package (or port). Other shared software will be added in ~d7share/bin.FreeBSD. A list of the current packages (ports) installed on all machines is available by the command ~d7share/bin.FreeBSD/lab_check. Ted Faber checks these regularly to make sure that lab configurations are in synch. To get a package added please contact either Ted Faber or Anne Hutton. If they are unavailable, and you have superuser access to idn You can use the script ~d7share/bin.FreeBSD/lab_install to install new packages across machines. Please send mail to the above if you do so.

Please contact Ted Faber or Anne Hutton if you need one of the packages reconfigured (e.g., you want sudo privleges.

Other Information

Before changing the physical connections of ATOMIC lab machines, be sure that you read and understand the information at http://www.isi.edu/div7/atomic2/lab-rules.html, maintained by Nehal Bhau. Plugging the wrong equipment together can damage it, so please be careful.

The lab is primarily funded by LSAM, so if contention gets high, LSAM projects have priority. We have never had to prioritize machine access in this manner because everyone has been reasonable and considerate. Hopefully that will continue to be the case.

Kernel Development

This section describes how to configure a private ATOMIC source tree, and how to compile, install and run your kernel. If you are planning to do kernel development please use these recommended procedures.

Configuring a Source Tree

Since we may have many development efforts in progress at any time, we have adopted a policy to allow each to be carried out in its own source tree, with the option of combining them later. We have investigated the use of CVS for this, it appears to use quite a bit of disk space, so we have abandoned that.

The new source tree will be primarily composed of symbolic links to the tree in ~atomic/4.1.3-sys. The utility newtree found in ~faber/bin/newtree will make such a tree. It takes two arguments, the first being the tree to mirror and the second being the location to place the links. For example, to make a new tree in a subdirectory called source in my home directory, the line:

~faber/bin/newtree ~atomic/4.1.3-sys/ ~/source

will work. Newtree was written by Ted Faber as well. lndir will do much of this; however, newtree is somewhat optimized for our source environment, and catches a few cases that require some foresight when using lndir.

When you want to make changes to a file, first delete the symbolic link and then copy the file from the main source tree (~atomic/4.1.3-sys). This ensure that your changes are only visible to you. Even in your copied files you should protect your changes by #ifdefs to prevent such changes from propagating in unwanted ways. See below for how to define the macros used to protect your code. This is a necessary precaution since potentially any code may someday be integrated into the main source tree.

In some cases you will want to modify sources that are not in ~atomic/4.1.3-sys. That tree is pruned somewhat to make kernel compilation quicker, and so is not complete. The full source is in /nfs/atom/4.1.3-sys and you can put additional symbolic links to that directory in place by hand if you need them. Again, only write to copies of the files, never to the files in the main tree.

FreeBSD Machines

The sources for FreeBSD are mounted read-only on each lab machine from idn on /sys. Use newtree to manipulate them just like on the Suns.

Configuring and building a Kernel

To configure a kernel, you enter the directory sun4m/conf in your source tree and execute the command configure kernelname. Kernelname is a kernel configuration file. The easiest way to generate one is to edit one of the existing configuration files. There are several in the directory that you can copy. After config exits, and it can take a while, a directory with the same name as the configuration file will have been created in sun4m. Change to that directory and execute the make command to build the kernel, which can also take a while.

To be sure that your source tree is correct, it is a good idea to configure and build the GENERIC kernel.

See the config(8) manual page for the format of the configuration file. Note the option keyword which you will use to define macros in the #ifdefs surrounding your changed code.

Installing a Kernel

The convention that we follow in the ATOMIC lab is to have the currently running kernel hard linked to /vmunix. The other copy of the kernel is generally kept in the root directory as well. Keeping a separate copy of the kernel prevents it from being accidentally deleted while switching kernels.

The default ISI reboot sequence checks for a file called /vmunix-new, and if it is detected, installs it as the new kernel (storing the current kernel in /vmunix.prev) and reboots. Using this fact, you can make a hard link from a copy of your kernel in the root directory to /vmunix-new, and reboot the system (using reboot(8)) to install your kernel. Be sure that the kernel is readable, writable and executable by the owner, readable and executable by group and others, (protection bits 0755) and is owned by the root user and the wheel group.

If your kernel is named /vmunix.something the /usr/atomic/myri_configure command can be used to link your kernel to /vmunix-new. Simply run /usr/atomic/myri_configure something as the superuser. Myri_configure is documented in /usr/atomic/man and if you add that to your MANPATH, you should be able to see the page.

The preferred method of rebooting the system is the reboot(8) command. Using the boot monitor does not sync disks. Do not use fastboot(8) either. The time savings from not running fsck(8) is minimal, and it can cause a lot of trouble. If the system is hung up, and using reboot(8) is impossible, press the Stop and letter A keys on the console simultaneously to get the boot monitor's ok prompt and run the boot command. The console does not respond to that, power cycle it.

Averting Disaster

It is possible that your newly installed kernel panics, i.e., fails and reboots, when it is run. In order to restore the system, it is necessary to go back to running a stable kernel. All lab machines should have a safe kernel in /vmunix.GENERIC. Other choices for a stable kernel are /vmunix.IPATOM, /vmunix.FAST, and /vmunix.prev. To boot from a kernel other than /vmunix, get the boot monitor's ok prompt. At that prompt enter the command boot disk filename replacing filename with the name of one of your stable kernels. This should restore you to a running system, from which you can manually relink /vmunix to the stable kernel and reboot. Do not install the kernel using the /vmunix-new trick, because the swap and reboot occurs relatively late in the boot sequence.

The other trouble you can get into is beyond the scope of this document. Note that the commands from the boot prompt are documented on the boot(8) man page.

Conclusions

That should be enough to get you started on the machines. If you have problems not covered in this document, talk to Ted Faber (faber@isi.edu), John Heidemann (johnh@isi.edu) or Joe Touch (touch@isi.edu).
This page maintained by Ted Faber.
Last Modified: .pso date