FreeBSD Jails

What is a Jail?

Those familiar with Java recognize the security concept of a sandbox. For those that aren't, it's the concept that everyone gets a unique, well-equipped sandbox to play in, and a person in one sandbox isn't allowed into anyone else's sandbox, not even to share anything with anyone else. On FreeBSD, jails implement this concept — they keep processes in their own part of the system, denying access to anything else. A jail requires its own dedicated IP address, though, which can make life difficult for those with limited address space. If this presents you with a hardship, consider at least using chroot. It won't afford you as much security, but it does help.

How does this help security? Take, for example, a box with an external FTP server and the company extranet. An exploit for the server is discovered, and a cracker manages to gain root access through the FTP daemon. If the FTP server is not run in a sandbox or jail, the cracker will have access to everything on the machine, including sensitive information destined for the company's partners through the extranet. If, however, the FTP server is run in a jail, the cracker will only have access to the FTP files.

There are, of course, still potential risks. If you run at secure level 0, the cracker can simply access the raw disk device and read data from there. The solution is obvious — on a box sensitive enough to require jails, use appropriate secure levels as well. This will eliminate a cracker's ability to read from or write to raw disk devices.

Configuring a Jail

Configuring a jail is pleasantly simple.

First, ensure that your system environment is jail-friendly. Because each jail requires its own IP address, the services on your box must be configured to listen to specific addresses, not just every available address. For example, if the box's addresses are 199.232.41.26 (main) and 199.232.41.27 (jail), to get inetd to listen only on 199.232.41.26, add inetd_flags="-wW -a 199.232.41.26" to /etc/rc.conf. If you fail to do this, conflicts may occur over the aliased IP address.

For some daemons, this is not an easy process — sendmail and rpcbind are two examples. If you're using these services on your box, you might consider simply running them inside of a jail of their own. After configuring all of the non-jailed daemons to listen to a specific address, reboot the machine. This will put everything into a known state, eliminating any potential for confusion.

With the proper host environment in place, create the directory that will house the jail. In this example, it's /usr/jail/ftp. Now go to /usr/src and run:

# make world DESTDIR=/usr/jail/ftp
# cd etc
# make distribution DESTDIR=/usr/jail/ftp
# cd /usr/jail/ftp/dev
# sh
# MAKEDEV jail

# cd ..
# ln -sf /dev/null kernel

These commands build the jail and populate it with all of the tools that your processes will need to run. Actually, they put in a lot more than just what your processes will need. For example, perl, gcc, and sendmail will all be installed, but you probably don't need them in your jail. Keep in mind, though, that it's a lot easier to take stuff out until something breaks than it is to put stuff back in until everything works.

To configure the jail environment, you might want to copy /stand/sysinstall into /usr/jail/ftp/stand, to provide you with an easy configuration interface. I'll show you how to use it in just a second.

With the system rebooted, you're now ready to configure the jail environment. Start the jail for the first time by running:

# jail /usr/jail/ftp jail.hostname.com 199.232.41.27 /bin/sh

This will put you at a shell prompt in your jail environment. From here, you can run /stand/sysinstall (literally, since / now refers to the jail's root directory, not the system's.)

There are several configuration tasks to perform, such as setting the root password (don't make it the same as the main system root password!), adding user accounts, and configuring /etc/resolv.conf. Read man 8 jail for more configuration tasks that you'll need to perform. Keep in mind that you want to be able to log in to the environment, so consider running an SSH daemon inside the jail.

Once you're done configuring the jail environment, exit the shell and the jail will be shut down.

You're almost ready to start the jail "for real." First, add the appropriate IP address alias. For our example, this is done via:

ifconfig fxp0 inet alias 199.232.41.27 255.255.255.255

You can configure this in /etc/rc.conf to be done automatically at boot.

Now, let's start the jail! This is done with two commands:

# mount -t procfs proc /usr/jail/ftp/proc
# jail /usr/jail/ftp jail.hostname.com 199.232.41.27 /bin/sh /etc/rc

You'll see some warning messages scroll by, but don't worry about them. You can now see all the daemons running inside the jail, as indicated by the J flag shown in the ps output. If you enabled SSH within the jail, you can ssh to the jail environment.

Since normal shutdown commands like halt don't work in the jail, you must take a special measure to shut the jail down. First, log in to the environment and become root. You can then kill all the processes inside the jail via kill -TERM -1 or kill t -KILL -1. You can also do this from outside the jail by manually killing any PID within the jail.

You should now be able comfortably to create and use jails to secure everything from FTP daemons to DNS servers. Before we finish, there's just one more note.

The security.jail.set_hostname_allowed (or jail.set_hostname_allowed on pre-5.0 machines) sysctl variable determines whether or not the superuser within the jail can set the hostname. This is enabled by default, and you might consider disabling it by placing security.jail.set_hostname_allowed=0 in /usr/jail/ftp/etc/sysctl.conf. Remember that it's jail.set_hostname_allowed=0 on machines running FreeBSD-4.x!

Conclusion

You've now seen both how jails work and how to set one up. Whether you're running multiple public services from the same box, providing login shells to untrusted users, or offering a public service on an otherwise private machine, using jails properly will help you sleep at night.

Mike DeGraw-Bertsch is a security and Unix system administration consultant in the Boston, Mass. area. When he's not at a job, writing, hacking with Perl, or playing with his wireless network, he can usually be found playing goal in ice hockey.

Return to the BSD DevCenter.

Comments on this article

Showing messages 1 through 13 of 13.

• core dump when starting jail
  2003-11-02 18:23:51  anonymous2 [View]
  
  
  I followed your instructions, as far as I know, and ended up with mount.core, swapon.core, and umount.core
  
  The messages that came up with trying to start the jail are:
  Segmentation fault (core dumped)
  Skipping disk checks ...
  Segmentation fault (core dumped)
  Segmentation fault (core dumped)
  Mounting /etc/fstab filesystems failed, startup aborted
  
  I'm really new to this... any ideas where to look for the problem?
  
  Thanks,
  Chris
• core dump when starting jail
  2003-11-04 12:22:27  anonymous2 [View]
  
  
  Nevermind, I got it working. Was missing the fstab file.

• Freebsd Jail To good to be true?
  2003-10-27 00:31:11  anonymous2 [View]
  
  
  I work at a ISP as a assistant admin.
  they have been having alot of problems with security so we have implemented a new security scheme using jails and consistent patching (reson for the hire) anyway i have talked to some oracle admins who advised against using freebsd jails and said your services will fail ( stability ).
  i hear only good things about freebsd jail, so what am asking what are the bad things other from some services not running in a jail. for example
  we were able to jail qmail and other services which i had a hard time using chroot with.
• Freebsd Jail To good to be true?
  2004-05-23 13:39:49  Nemesi [View]
  
  
  Hallo,
  
  I coordinate the IT services of a group of large research centers.
  
  Since 2 years all of our primary services (mail/qmail+courier, web/apache2+mysql+imp+egroupware, dns/isc-bind, dhcp/isc-dhcpd, proxy/squid, vmps/OpenVMPS, kerberos/mit-krb5, ldap/openldap, etc etc) run on FreeBSD.
  
  Since the beginning some, nowadays ALL the services run on jail'ed "virtual machines", on separate IP addresses, on separate disk partitions... running into jails.
  
  Essentially all the daemons worked without any problem at first shot by setting up a "full jail environment" (that is an entire envirinment containing /etcv, the libraries, the binaries, etc etc), most of them worked happily into a "microenvironment" built into a directory and started as jail, only for a few we decided to apply local patches to further minimize the amount of "stuff" inside the jail. In example we have a local patch for dhcpd that makes it "self-jail" at startup (but after loading the dynamic libraries....) so that into the dhcpd jail there are in total less than 20 files.
  
  Since one year ALL the services (about 30 daemons running on a cluster of 8 different machines) run into a jail environment. We never experienced a single problem that we could correlate with the jails (of course some daemons had some stability problems by themselves, namely openldap and apache2) but nothing that could be in any way correlated with the use of the jails.
  
  We also have a few instances of Oracle running on a separate set of machines (an OpenVMS cluster, one linux box and one OSX box). There yes: we had a number of issues. But those were all installations on platforms "recommended" by Oracle thus I would say that the problem is Oracle.
  
  If for some reason you need Oracle then consider that it is anyway going to be "officially supported" only on the systems that they choose configured in the ways that they choose. And I think that at Oracle they don't even know what FreeBSD is. Thus the minimization of the issues will be running Oracle on dedicated machines on the OS they want you to... and of course behind a very well configured (FreeBSD) firewall.
  
  A.
  
  

• Addition
  2003-10-08 07:54:24  rubeng [View]
  
  
  
  Good article. I would like to clarify 2 points however:
  
  1) It is not necessary to install "world" to run a jail. All you need are the programs (and libraries) that you actually want to run inside the jail.
  
  2) A jail does not really need a seperate IP address. Actually, you can run many jails on the same IP address, as long as the services in each jail run on different port nrs.
  
  An example document on how to create a BIND (mini-)jail can be found here:
  
  http://www.bzerk.org/documents/bindjail/
  
  Ruben
  

• Fatal bug
  2003-09-15 03:32:11  anonymous2 [View]
  
  
  A number of programs will not work in a jail, due to a long-standing bug:
  
  http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/26506
  
  I ran into this with snmpwalk. And the patch in that PR did not work :-(
  
  Lupe Christoph

• Useful ; thanks for the article
  2003-09-06 05:05:54  anonymous2 [View]
  
  
  Useful article, however I'd like to read more about the advantages of jail and how it can be used. I can see what it's basic advantages are, if a user compromises something inside the jail it won't affect the real system. However, if there are services running inside the jail, I guess the attacker can do enough harm inside the jail; which will affect the services and the (sensitive) data they serve.
  
  Also, is a jail inside a jail possible ? Chroot inside jail ? (My assumption so far is that it is possible).

• cool
  2003-09-05 13:11:47  anonymous2 [View]
  
  
  very nice, a better chroot , for root :)

• Chroot jail HOWTO for Linux
  2003-09-05 13:00:09  anonymous2 [View]
  
  
  I had to figure out how to set up chroot
  jails that supported remote login on Linux
  recently, and documented the procedure. It's at
  
  http://www.kegel.com/crosstool/current/doc/chroot-login-howto.html

• Forgot chroot!
  2003-09-05 09:41:45  anonymous2 [View]
  
  
  Yeah, I'd really like a simple explanation of how to use chroot to set up a basic jail environment (albeit less secure).

• chroot
  2003-09-05 03:01:18  otto [View]
  
  
  Great article, Mike, but where's the chroot info you promised?
  
  Cheers,
  -otto.
• chroot
  2003-09-05 14:44:26  anonymous2 [View]
  
  
  Where did he promise chroot info? He said "consider using" it, not "I'll tell you how to use it."
• chroot
  2003-09-05 14:52:57  chromatic | [View]
  
  
  Oops. Mike asked me to edit that sentence in the article but I forgot to reply to the parent comment. This was my fault.