OpenBSD from a veteran Linux user perspective
For the first time I installed a BSD box on a machine I control. The experience has been eye-opening, especially since I consider myself an "old-school" Linux admin, and I've felt out of place with the latest changes on the system administration.
Linux is now easier to use than ever, but administration has become more difficult. There are many components, most of which are interconnected in modern ways. I'm not against progress, but I needed a bit of recycling. So instead of adapting myself to the new tools, I thought, why not look for modern tools which behave like old ones?
This article discusses some of the main differences between OpenBSD and Linux, from a Linux admin perspective.
There are some texts on the net discussing the philosophical differences between BSDs and Linux, but not many of them are really hands-on. This one is the best, and I recommend you to read it along with this one.
Since I am new to OpenBSD, I may get some things wrong. Please email me any corrections. However, my goal is to point out my first impressions, so if there are any Linux users reading and thinking about making the jump, they can know what to expect.
Final update: I've received a huge amount of feedback from this article, overwhelmingly positive, and very welcoming from the OpenBSD community.
This text's goal is to discuss an OpenBSD newbie's first impressions and, as such, I'm realizing that some parts aren't totally accurate. However, I want to keep that perspective, so in the future I'll prepare a more technical guide on how to migrate to OpenBSD for Linux users. Thanks everyone!
The "RAMDAC" running joke
First, some background about my Linux experience.
My first computer was a 386 with DOS and Windows 3.1. I had played with Spectrums, Commodores, and IBM PCs (8086). I followed the traditional Windows path: 3.1 -> 95 -> 98 -> ME -> 98 -> 2000. But I always liked computers, and the most visible part of them, besides the hardware, is the OS.
I tried to install my first Linux distro on 1999. It was a Red Hat Linux 5.2, if I remember correctly, and I got the CD from a magazine because I was still running a dial-up. I was 15 and I thought I knew computers, after all, I had assembled my own, an AMD K6-2 box, from parts.
Red Hat proved me wrong.
- Which is your chipset?
- Man, I don't know
- Which is the model of your RAMDAC?
- What is a RAMDAC?
- I need your monitor modelines. Don't get them wrong or you will physically damage your CRT
- Dude, I'm 15, I can't afford to break anything!
In the end I didn't break my monitor, but
got a black screen which said login:, and didn't know what to do, so I booted back
into Windows and played a bit of Warcraft 2.
In that age we only had one computer, so if you were installing something and needed help, you had to stop, reboot into Windows, dial up the modem, search the Usenet or forums, write down the solution on a piece of paper—no ubiquitous printers—, hope you got the commands right, reboot, start the installation over, reach the point where you previously were, and apply that solution. Not practical at all.
The best help we had were books, and those were expensive and difficult to find on a small town bookstore. For those of us not fortunate enough to buy/find books, we had hobbyist magazines. In Spain there were a few imported and poorly translated magazines which were expensive, but carried some CDs, the only practical way to get distros.
The first Linux I really was able to use was Mandrake 6.0. It had a graphical installer—not that having graphics
made any real difference on the final result— but it auto detected my hardware correctly and booted into X. Yeah!
Old Linux software!
A game called Nethack which had nothing to do with hacking! sysconfig!
Unfortunately, I couldn't connect to the internet because of my Winmodem, so after a few days of tinkering, Mandrake was wiped too.
Months later, I got myself a BeOS CD. It was like Linux, which for me, then, meant it was not Windows. The setup ran totally effortless and it even detected my Winmodem. The internet ran faster than on Windows. It had a great internet browser, mail and newsgroups clients. Oh, boy! I used BeOS for a long time almost exclusively and only booted Windows to play some games.
A couple years later I started Computer Engineering on college, so I wiped out everything and installed Linux. I got a new machine and a real network connection.
I've run lots of Debians, Red Hats, Mandrakes, Gentoos and Slackwares. We used Solaris and even some VAXes. I ran some servers for student organizations, and finally settled on Debian as "the best" distro: stable, easy to use, no need to compile on our 486, nice hardware detection and with a big community.
Finally, I moved to Ubuntu if only because its LTS releases. Around 2006 I got into Macs, which at first seemed like a nicer Linux, and now I appreciate the hardware+software combo for which I know I won't have to fight with its drivers.
In summary, I've seen a lot of UNIX, even more Linux, and administered a good chunk of them. My servers have always run some sort of Debian.
You could say that as I grew older I also grew tired of fighting with RAMDACs, modelines and Winmodems. Each age brought new "RAMDACs": CD recording, wireless card support, laptop hibernation, webcams, divx playing, DVD playing, NTFS support...
Linux always worked in the server but had some quirks in the desktop which made it somewhat unattractive for daily use, even when I run it exclusively on my laptop.
Nowadays, Macs offer a UNIX with some peace of mind, and the current status of Linux is good enough. Some of the friends I evangelized long ago—I quit doing that—still use Ubuntu and are happy with it. Linux may never triumph on the desktop (or laptop), but it's good enough for most.
Upgrading a G4 Mac Mini
Now jump to 2015. My home server, a G4 Mac Mini, was already two Debians behind. Some packages weren't ported to powerpc. I needed to perform a clean install and upgrade the whole system either way. But this time I didn't want to use a Linux installation which wants me to reboot every 5 days because of some critical patch. I'm looking at you, Ubuntu.
As you can imagine my operating system fascination didn't fade out, only my time. I had been closely following the BSDs and using a NetBSD shell account, installed Plan 9 on a virtual machine, and even wrote a toy OS project.
I'm not afraid of compiling stuff—I do it for a living— and may even be open to modifying some code if needed. Why not try something new?
Since I had the weird powerpc requirement, I ruled out most operating systems. Finally I decided to play relatively
safe and go for a BSD. FreeBSD is the most popular, has
more online HOWTOs, and probably more features (ZFS, Jails), though I probably would not be using them. OpenBSD
is more hackable, seems to have better documentation, and some cool people I know use it. I didn't want to quit using a pot
to start using a kettle, so I downloaded OpenBSD's install57.iso
It was impossible to boot the Mini with a USB stick; I'm unsure if it's the firmware's fault
or the fact that I was dd'ing the .iso file into the USB instead of the .fs one which doesn't seem to be available
for macppc.
I found some blank DVDs on a closet, borrowed a computer with a DVD drive—another medium I hadn't touched in years—, and burned the ISO image. The fact that I recorded the first disk with the ISO file on the root folder instead of properly burning the contents into the DVD warned me that this was going to be hard, but fun...
Surprisingly, the installation was straightforward, it detected the 10-yr-old hardware, and by following the instructions I managed to partition the disks and install the boot loader. The box was up and running.
Well, that wasn't so hard, was it? Now, to restore my old installation.
Hm, first of all,
bash needs to be installed from packages and goes into /usr/local/bin, so I had to
modify a lot of scripts which pointed to #!/usr/bin/env bash. The ps and tar commands have slightly different
switches which broke other scripts.
The base services are different; OpenBSD includes its own HTTP, SMTP and NTP servers. Configuration files
are in different places. Here goes my week...
GNU is really not UNIX
A quick note on the GNU/Linux naming discussion, since GNU is entering the equation now. I use the term "Linux" for simplicity. I know that's the kernel name. It also happens to be the popular name, even if not totally correct—according to some.
Here is some food for thought; why does the FSF deserve more credit on the name than, say, the Apache Foundation, or the FreeDesktop project, or BSD, for that matter? Why don't we include every key component on the name and call it GNU/FreeDesktop/Apache/OSI/BSD/.../Linux? Including only GNU would be unfair to other big contributors, wouldn't it? So let's please stop this fight.
That being said, the GNU tools and design philosophy make a noticeable difference in administration and userspace, and one can only appreciate it when switching to a different environment.
I don't want to overstate it, though. Thanks to POSIX, a Linux admin can run BSD with little extra effort, since most of the things are similar. There are, in fact, more similarities than differences. If FreeBSD and OpenBSD are brothers, then Linux is a close cousin.
ls is always ls. mkdir is mkdir.
But when you're being used to /dev/hda, free -m and cat /proc/cpuinfo you realize that having a different
kernel is naturally going to change some of the administration tools.
Some say that the GNU tools are bloated and that the BSD toolchain is more "pure UNIX". The reality is that it depends on the specific GNU tool, I've personally found that GNU tools are more complex because they're more powerful, though they are less UNIX-like (do one thing only and do it well) and more like complete solutions. That's fine; different, but fine. After all, GNU is not Unix!
In recent years, the Linux environment has grown in the GNU toolchain fashion, not the UNIX fashion. One may even say it has grown in the Windows fashion: be practical, be accommodating to all, be fast, be modern.
There have always been debates about "bloated and complex code". More recently, systemd. Previously, Apache, sysconfig, iptables/iptools.... The list goes on and on. Wheel out comp.os.linux and look at the flame wars.
No software fits all nor should be shamed for its design decisions. In the end, with a few critical exceptions like OpenSSL and the Heartbleed bug, it is just a matter of taste: does the admin prefer simple, pluggable services, or bit monolithic suites? Compatibility or modernness? Familiarity or shiny new things? Standards or NIH?
I had been riding the Linux wave for years, until I recently realized that my admin skills needed a total
recycling. In a few years we've gone from /etc/init.d/sshd restart to service sshd restart to systemctl start sshd.
That's a bit fast in my opinion, but I understand it's the price of progress, aimed to make computers boot
faster and theoretically easier to administer for newcomers. Old admins, on the other hand, have a harder time adapting.
Having to choose between recycling into an always changing Linux or a more stable UNIX environment, I chose OpenBSD. Given my history of trying all possible OSs, Let me state again that I'm not against the recent Linux direction. I just wanted to go out and see if there is a different way to do things.
Differences between OpenBSD and Linux
Maybe you're reading this article for its practical value and not for my ramblings, sorry. I thought I had to provide some context. I'm used to googling, I'm used to RTFMing, I'm used to reading source code to learn what software does. This context is important to judge if you would notice the same differences as I did.
Here's a list of things that surprised me the most after completing an OpenBSD install, adapting my old setup to the new environment and running it for a few days.
Simplicity
First of all, everything is much simpler, like the Linux old days. Not necessarily easier, but simpler. More minimalistic. I found this plays well with my mind. OpenBSD follows the UNIX philosophy more closely: lots of small components that do one thing and talk between them by passing text.
Because of that, some base components are not as feature-rich, on purpose. Since 99% of the servers don't need the flexibility of Apache, OpenBSD's httpd will work fine, be more secure, and probably faster. For those who need the big boys, just install Apache from the packages.
Having a developer-chosen default option for many servers is a time saver. The admin knows it will be well supported and documented, and tightly integrated with other components. The alternative, the Linux way, is to just use what everybody else uses (Apache), or choose one of the multiple options, always wondering if it's the right one—nginx? lighttpd? thttpd? You know what... nobody got ever fired for choosing Apache
Design decisions
Picking up on that thought, the system is very opinionated. When the community decides that some module sucks, they develop a new one from scratch. OpenBSD has its own NTPd, SMTPd and, more recently, HTTPd. They work great.
Likewise, the standard shell is pdksh. The OpenBSD FAQ states that
"Users familiar with bash are encouraged to try ksh(1) before loading bash on their system -- it does what most people desire of bash.",
which is a bit too bold. ksh does not support history substitution (sudo !!:1) which I use a lot, though I agree
that for many users it will be enough. Many people hate bash for a reason, I am not one of them.
Having a super powerful shell has saved me from writing perl scripts for system administration.
Bash can always be installed from packages, anyway.
This is a big difference from Linux, which is more like a "consensus" operating system. Developers have to keep compatibility and whenever there is a controversial design decision like systemd, dozens of projects decide to fork. Not good.
Strong opinions, on the other hand also lead to less support for some, like ext4, ZFS or Linux binary compatibility. For example, ext4 is officially supported read-only but in my case it didn't read some folders properly. FreeBSD plays better on that regard, though they also have more developer manpower. This leads to some use cases, like an OpenBSD desktop, being possible but not the best choice for this OS.
Finally, other decisions make little sense. According to disklabel(8), the /usr partition takes about 2G of disk space, not including /usr/src or /usr/obj. This means that there is little space to hold what is essentially the whole system plus ports. I had trouble compiling large ports since /usr ran out of disk space. If a large number of users will be compiling some ports, why not set a larger /usr by default?
Documentation
The man pages are excellent, a delight. Unlike Linux, they are not just a list of switches for the software, but a comprehensive guide to the tool, with lots of examples. They are much, much better—thankfully, because unlike Linux, again, there are not tons of help on public forums.
OpenBSD's man pages are so nice that RTFMing somebody on the internet is not condescending but selfless.
Granted, I wouldn't make a UNIX novice run OpenBSD from man pages, but for an experienced admin, they contain exactly the information they need.
Small differences in common tools
Using the BSD toolchain instead the GNU one means there are small differences between tools.
For example, some ps switches are missing, like the useful -f. The tar options for reading from stdin
are also slightly different. When ls is run by root, it automatically appends all hidden files.
df has -h (human) and -k (kilobytes), but no -m for megabytes.
If you've used MacOS you probably know a few of these.
Packages
OpenBSD has packages, like Linux. Unlike it, packages are only available for 3rd party software, not the base system.
OpenBSD's base system is more or less what gets installed from the CDs: kernel, shell, coreutils, a small part of X and essential servers (http, ntp, smtp, etc.) Everything else must be installed from packages.
The documentation recommends using packages, since it is not worth it to compile from ports—the package sources. However, packages don't get security updates. The only way to patch bugs is to compile the ports.
Fortunately, there is a simple way to use the best of both worlds: add FETCH_PACKAGES=yes to /etc/mk.conf
and install software from ports. The system will automatically fetch the package and save the compilation
time if there is a current binary available.
Another interesting tool is /usr/ports/infrastructure/bin/out-of-date, which checks which ports need
an update, so you can go to /usr/ports/<portname> and make update. This command plays well with previously
installed packages, so you don't have to worry deleting them first.
In summary, after you install the release, if you're interested in getting security updates until next
release, the officially recommended path is to follow -stable, use FETCH_PACKAGES and work from ports.
This is not very clear in the documentation but the folks at #openbsd
helped me figure it out.
As a colophon, if you're using x86 or amd64, m:tier provides binary updates for the base system and packages, much like Linux does. Otherwise, if there is any bug in the base system you'll need to recompile that part yourself. The amount of compiling needed will be determined by the patched component and any related software, so just read the instructions on the patch.
Configuration files
The base system config files are properly centralized in /etc, but
not the ports. The porting quality is excellent, better than any Linux distro. Every port is adapted
to the OpenBSD system and made sure it behaves correctly. However, some maintainers decide that all
the port files need to be contained in some folder, like transmission-daemon, which stores its
config into /var/transmission/.config/transmission-daemon/settings.json. It's a bit crazy to store
a system-wide daemon config file into /var which, according to man hier, contains
Multi-purpose log, temporary, transient, and spool files.
Apparently some daemons are chrooted by default, and there is a global "catch-all" README folder
on /usr/local/share/doc/pkg-readmes which contains specific info about packages. transmission-daemon
had no related info, so maybe I'll contact the maintainer.
Chroot
Speaking of roots, nearly all daemons in the base system are chrooted and privstep by default. The base system has a lot of hardening by default, which is one of the main reasons why OpenBSD has almost no remote holes on the default installation.
Since chrooting software in Linux can be cumbersome, it's very convenient to get it done for you, so thanks!
Experienced community
I feel like the learning curve is a feature, not a bug, intended to keep newcomers away. OpenBSD is unapologetically elitist. Honestly, I don't mind that. I've been administering systems for more than a decade and not all environments are for everybody.
OpenBSD can afford to be elitist because it is a small system, with a clear direction, the documentation is crystal clear, and it doesn't make vague promises.
make build
As you can see there is a big con to using OpenBSD coming from a Linux world, the process for patching security issues. On Linux I was used to run a single command and let any part of the system (base or 3rd party) update itself. With OpenBSD, it takes a lot more effort and time, especially in my old machine.
This process leaves the admin only one realistic option: follow the -stable branch, which is basically the same code as the CD release with small patches, and recompile stuff regularly. Otherwise, the installed system will be exposed to potential security holes until the next release.
I feel that this needs to be more prominent in the OpenBSD docs, especially on the Migrating to OpenBSD
section: if you want an updated and stable
system you'll need to recompile stuff constantly, there is no equivalent to apt-get upgrade.
To get a secure production system with OpenBSD, the officially recommended path is to:
- Install the CD release
- Download the source code
- Recompile the kernel (recommended by "following -stable")
- Recompile userland
- Download ports tree
- Add
FETCH_PACKAGES=yesto/etc/mk.confto let ports fetch packages, if available, and install software using the ports syntax. - Recompile when there is a security issue which affects your setup, though you may skip some compiling if using m:tier.
Of course, this is a feature, not a bug, but it's the biggest admin change from old Linux users.
That's a lot of effort compared to apt-get update && apt-get upgrade. Honestly, had I known it,
I would've more strongly considered keeping my Debian installation. I read all the online documentation before
installing OpenBSD, and I felt like this point wasn't really clear.
Since you can safely use -stable ports/packages with a -release base system, steps 3-4 can be avoided or shortened if you don't want to update your base to -stable. That's what I would recommend to former Linux users, but take this newbie's advice with a grain of salt.
In any case, for low-performing machines like mine, maybe the "recommended" path to follow -stable and rebuild the source for every errata is not the best one. For small fixes it may be better to just apply the patch and follow its instructions. Apparently in faster machines it's just more convenient to recompile the base system since it takes just a few minutes. Had I been using x86 or amd64, I'd have totally gone for m:tier, so you can dismiss this section if that's your case.
To be totally fair, it's rare for OpenBSD to have remote holes on the CD release, so one could be relatively safe by only upgrading from release to release. But the truth is that there is no simple way to binary patch for critical updates unless using the third-party m:tier on one of the supported architectures.
With that it mind, to summarize, there are the following options:
- Use a -release base and -stable ports (with
FETCH_PACKAGES=YES), cherry-picking patches from base and updating ports bymake update. This may be the recommended path for low-performing machines - Use a -stable base, too. You can then update the whole system with a handful of commands and won't need to follow patch instructions
- Use -release and update from m:tier
- Keep using -release until a next -release comes, unless there is an unlikely remote hole that forces you to recompile the base. This may be the best option for newbies if the only person using the box is the admin, so there is no way to suffer local attacks.
Update: OpenBSD now supports binary updates for security, like apt-get upgrade.
Check out the announcement
and the official documentation
which has been updated to reflect this much easier process: just run pkg_add -u
Conclusion
From a user perspective all of this is transparent; OpenBSD has a UNIX terminal or Xwindows session and everything works as expected. But a Linux admin will need to adapt to these new tools and allocate some more time for administration.
OpenBSD has pros and cons. Personally, my main pros are the excellent documentation, its minimalism and the choice of default daemons. The only con is the need to recompile to patch errata. If I had just one wish for OpenBSD, it would be a more straightforward updating system for security errata.
Now, the dreaded question. Is it worth it?
Honestly, I wasted too much time. Some of it was to be expected, since I needed to learn a different environment. Had I been 10 years younger, this wouldn't have been a problem, but my time is more limited now. The fact that I needed to compile things on an old machine probably didn't help. Keep that in mind when considering a BSD for an old, weird architecture.
After the initial investment, I want to see if maintenance is easier and release upgrades are smoother than with Debian. Manually upgrading things is a pain in the neck, but all other factors lead me to think that OpenBSD is a great server OS.
Maybe I was expecting something else from the docs I read? It is probably my fault, though. Anyway, I want to contribute to the available documentation by writing this document so that other Linux admins can make a more informed decision.
On the other hand, my geeky side is content. OpenBSD rocks. It is a different—a real—UNIX and I've really come to appreciate simple code and software. As an admin, having minimalistic, default servers is a blessing.
Again, should you try OpenBSD? The answer is yes, though be careful if you're either in a rush or have specific software requirements. The first days are a bit hard, and recompiling on a slow machine takes time.
If you like UNIX, it will open your eyes to the fact that there is more than one way to do things, and that system administration can still be simple while modern.