The Opening of Solaris^TM

Peter Baer Galvin

Welcome to the new Sun Microsystems. A couple of years ago, Sun was as closed as a Cape Cod oyster. Communications were very controlled, and the Solaris source code was a very well-guarded secret. Fast forward to June, 2005. Now it seems that everyone within Sun has an opinion or technical detail that they are sharing through blogs.sun.com. Even more startling, Sun has followed through on their promise to open source the Solaris operating system. This month, I'll take a look at the OpenSolaris project, explore what's available, and walk you through downloading, building, and installing Solaris from the OpenSolaris project. What does OpenSolaris mean to Sun? What does it mean to you?

The Theory

Sun has long stated that they would make Solaris an open source project. Along the way, there were unfortunately delays and speed bumps, stretching out the delivery date. Most of these delays seem related to the legal aspects of code ownership and licensing format. The result is that Sun created a new open source license, the CDDL. This CDDL has been covered elsewhere with much detail and quite a bit of gnashing of teeth. I'll leave it to those in the open source community to debate, but the license seems reasonable, and very open, in general. For example (in my opinion -- I'm not a lawyer), anyone could take the OpenSolaris source code, modify it, and sell either the modification or the entire package.

Will the open source community embrace and extend OpenSolaris? That is the big, open question. There are signs that they will, though, and why not? Solaris 10 is "the best" operating system available. Now that developers can get their hands on it and modify it, I believe they will. Details of CDDL are available at:

http://www.opensolaris.org/os/licensing/

Overview

http://opensolaris.org is ground zero of the OpenSolaris project. There you can find instructions on downloading and building OpenSolaris. There are quite a few surprises in store for anyone exploring opensolaris.org. Not the least of which is that the project is well cooked and ready for prime time. Following the instructions on the site, it is rather easy to download the source for OpenSolaris and build the entire binary set that comprises commercial Solaris.

If your purpose is just to peruse the source code, a tarball download gets all of the available code. Note that this is not the same as all of the code that goes into Solaris. There are some files that Sun does not own the rights to redistribute as source code, so those are provided in a separate binary-only tarball. A count of the directory tree from the "closed" tarball shows ~6,500 entries, but many of those are empty directories. This is opposed to ~85,000 entries from the source code tree. So, in fact almost all of the OpenSolaris project distribution is available in source format.

To actually compile the source and create binaries, you first need a very recent build of Solaris. In fact, the version you need is the Solaris Express: Community Release (currently "Nevada build 16"), which is more recent than the one currently available via the standard Solaris Express. (That release is "Nevada build 15".) Apparently this build is needed to fix bugs in the ability to build OpenSolaris. Moving forward, any release from Nevada 16 onward should be able to build the OpenSolaris project.

Note that the Solaris Express release goes through multiple weeks of testing before it is made available, and that no such testing is done of the builds available via OpenSolaris. Clearly, this release should be used only for development. I intentionally do not call Nevada 16 either Solaris 10 or Solaris 11. All of the bits of a new minor or major release from Sun tend to get mixed together until decisions are made about release dates and feature sets. Within Sun engineering, all changes are added to the most recently opened "gate" (or development tree). That is, if a Solaris 11 gate is created, all changes must be added there first and then integrated into the older gates (i.e., Solaris 10 Update 1) over time. This required "soak" time of changes results in higher quality and lower bug counts. Nevada is the latest gate, and features from it may show up in Solaris 10 Update 1, Solaris 11, or both.

A further surprise is that the source can be compiled via GCC or the Sun Studio compiler suite, and that a license for Sun Studio is available at no cost. The site (http://www.opensolaris.org/os/community/tools/sun_studio_tools/) states that Sun Studio software "is being provided, free and unrestricted, to participants in the OpenSolaris Community". Whether this means that Sun Studio can be used for free on platforms other than Nevada 16, or for purposes other than compiling OpenSolaris, is currently unclear.

A final pleasant surprise is that the source code is very up to date. According to sources at Sun, the source code at OpenSolaris is about 2 weeks behind the bits that are currently checked in to the Solaris source at Sun. As of the time of this writing, the source was for Nevada build 17 plus some bits of build 18. That difference is expected to close over time, to the point that OpenSolaris gets the code from Sun at the same time as it is injected into the commercial Solaris source tree. Simply astounding!

In terms of unhappy surprises, there are still a few rough edges. For example, you can't create official "packages" out of any source builds at the moment. More is documented in the release notes that accompany the source downloads.

In general, the launch of OpenSolaris seems well thought out, and has a lot of momentum. Available with the project is a bug database, a source browser, and lots of community discussion.

The Download

With all of that background in mind, it is finally time to download OpenSolaris. There is a good overview of the steps needed to download and build OpenSolaris available at the main site, along with a roadmap of the future of OpenSolaris. The download process is simply a matter of reading the release notes, selecting the appropriate files to download, and waiting for all the bits to move. Sun actually provides three ways to move bits -- the Sun Download Center, BitTorrent, and genunix.org.

Actually, there are two choices to make. For example, OpenSolaris can be built from a set of pre-built archives, or you can build it from source. Also, you need to select the compiler.

The files to download include the current Solaris Express: Community Release build (i.e., Nevada Build 16 binaries), the chosen compilers (Sun Studio or GCC), tools to use to build OpenSolaris (the "ON" tools package), and the source or pre-built archive files. Unpack each file according to its kind and the instructions, and then it's (almost) time for the big build.

Before we look at building OpenSolaris, we should take a moment to admire the OpenSolaris source. The choice of parent directory is yours. I selected /opt/src/nevada. Note that selecting to download the source results in a directory tree of 1.8 GB, so there is a lot of source to explore. Actually, the tree includes "usr", which is the source tree; "archives", which is the output of a build; "closed", which includes the encumbered binaries that are not provided in source form; and a "proto" directory for use by Install(1). Using the online source browser is a nice way to find what you are looking for, without manually searching through the code. There you can search by definition, symbol, file path, and history, as well as do a full search on everything. For example, a "path" search for "usr/src/cmd/ls" reveals "ls.c" as well as some make files. Clicking on "ls.c" shows the formatted and highlighted full source for the ls command.

Build

Once a system is built based on Nevada 16, compiling the source into usable binaries can begin. Some patches are required, as spelled out in the release notes. There is a full Developer's Reference manual as well that describes the entire build process, archives, BFUs, and so on, available at:

http://www.opensolaris.org/os/community/onnv/devref_toc/

Next, an environment file is needed to guide the build. One can be copied from usr/src/tools/env/opensolaris.sh to opensolaris.sh (or some other name) and then modified to include path and permission information (the GATE, CODEMGR_WS, VERSION, and STAFFER variables).

The system I used for testing is a slow-ish (1 GHz) x86-based PC. Running the nightly command performs a full, silent build of the entire OpenSolaris source tree as in nightly ./opensolaris.sh &. On my system, it took between 3 and 4 hours. Once a full build is done, builds of only the changed components can be done via nightly -i ./opensolaris.sh &. A complete log is created in log/nightly.log, and the build progress can be monitored via ptree.

Install

Once a build is complete, the binaries can be used in situ or can be installed into the proper places on the current system. If you want to do a full install of the binaries built during a full build, it's best to run nightly with the -a and -z options to build gzip archives. Then the bfu command can install the archive on the system. A reboot then brings up OpenSolaris as built from source.

If you just want to build a new kernel (say kernel.peter), there are a few options and some best practices. First, perform a "nightly" to build the whole source, including the kernel. Second, source the build environment script file (in this example "opensolaris.sh") to set the variables in your shell. Third, use "Install -XG kernel.peter" to create a tar file in /tmp/Install.root of kernel.peter including all of the kernel modules you built. Fourth, untar the tarball in "/". And, finally, boot the system using the new kernel via the boot path:

kernel.peter/sparcv9/unix - for SPARC
kernel.peter/unix - for x86 32-bit
kernel.peter/amd64/unix - for x86 64-bit

Other Stuff

Open Solaris is very big. People outside of Sun are just starting to find their way around it. The Sun blogs help a lot. Specifically, check out those by Eric Boutilier and Bryan Cantrill. Eric Schrock is blogging about how to add a system call and source in general to OpenSolaris.

Another good way to track all of the blogging about OpenSolaris is to use the Technorati tags:

http://www.technorati.com/tag/opensolaris

http://www.sun.com/blueprints/0505/819-2679.html

Conclusions

We are indeed facing a brave, new, open Sun. There are early signs that Sun's efforts at building a vibrant open source community around OpenSolaris are working. Within days of the release of OpenSolaris, a team of OpenSolaris beta testers released the ShilliX project. The first piece of the project is an .iso file that can be used to create an OpenSolaris-based CD. This CD can be used as a "live CD" -- boot it and run OpenSolaris without having to install OpenSolaris to a hard drive. It can also be used as an install CD for OpenSolaris. Very handy indeed. Let's hope there is much more to follow from ShilliX and others. OpenSolaris should allow for a lot of innovation in and around the standard Sun Solaris.

Peter Baer Galvin (http://www.petergalvin.info) is the Chief Technologist for Corporate Technologies (www.cptech.com), a premier systems integrator and VAR. Before that, Peter was the systems manager for Brown University's Computer Science Department. He has written articles for Byte and other magazines, and previously wrote Pete's Wicked World, the security column, and Pete's Super Systems, the systems management column for Unix Insider (http://www.unixinsider.com). Peter is coauthor of the Operating Systems Concepts and Applied Operating Systems Concepts textbooks. As a consultant and trainer, Peter has taught tutorials and given talks on security and systems administration worldwide.