FreeBSD_Jails.txt

# FreeBSD Jails
via https://clinta.github.io/freebsd-jails-the-hard-way/

* VirtualBox
  * Intall, with ZFS, admin user as sudo admin
  * ssh -p 2222 [email protected]

* Jail Host, for use with PostgreSQL
  * https://www.textplain.net/tutorials/2015/running-postgresql-in-freebsd-jails/
  * /etc/sysctl.conf: security.jail.sysvipc_allowed=1
  * /etc/rc.conf: jail_sysvipc_allow="YES"
  * /boot/loader.conf:

      kern.ipc.semmni=256
      kern.ipc.semmns=512
      kern.ipc.semmnu=256

  * Reboot!
  * If you run multiple PG jails on host, each pgsql user MUST have a
    unique UID so kernel can manage separate shared memory for each!
    * In each jail: vipw -d /var/jails/jailname/etc/, change UID 70 to
      something like: 70+iplast or 70 + jail number:

      pgsql:*:70203:70:PostgreSQL pseudo-user:/usr/local/pgsql:/bin/sh

* Setup global jail settings in /etc/jail.conf:
      exec.start = "/bin/sh /etc/rc";
      exec.stop = "/bin/sh /etc/rc.shutdown";
      exec.clean;
      mount.devfs;

* Mount FreeBSD base install:
  * scp scp -P 2222 FreeBSD-10.2-RELEASE-amd64-disc1.iso [email protected]:/tmp
  * mv /tmp/FreeBSD-10.2-RELEASE-amd64-disc1.iso $JAILS
  * From https://www.freebsd.org/doc/handbook/jails-build.html
    * mdconfig -f $JAILS/FreeBSD-10.2-RELEASE-amd64-disc1.iso
    * mount -t cd9660 /dev/md0 /mnt   ("md0" from mdconfig output)
  * Or Fetch them via FTP
    * fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/10.2-RELEASE/base.txz -o /tmp/base.txz

##  Version 1: Full Jails. As root:

* As Root:
  * export JAILS=/var/jails
  * export ZJAILS=zroot/jails
  * zfs create -o mountpoint=$JAILS $ZJAILS
  * zfs create $ZJAILS/j01
  * Install and Verify:

    tar -xvf /mnt/usr/freebsd-dist/base.txz -C $JAILS/j01
    tar -xvf /mnt/usr/freebsd-dist/lib32.txz -C $JAILS/j01
    tar -xvf /mnt/usr/freebsd-dist/ports.txz -C $JAILS/j01
    UNAME_r=10.2-RELEASE freebsd-update -b $JAILS/j01 fetch install
    UNAME_r=10.2-RELEASE freebsd-update -b $JAILS/j01 IDS
    cp /etc/resolv.conf     $JAILS/j01/etc/resolv.conf
    cp /etc/localtime       $JAILS/j01/etc/localtime
    echo hostname=\"j01\" > $JAILS/j01/etc/rc.conf

  * /etc/jail.conf: (Append jail config)
      j01 {
          host.hostname = "j01.jailhouse.io";
          path = "/var/jails/j01";
          interface = "em0"; ## Or whatever
          ip4.addr = 10.0.0.11; ## Whatever
      }
  * Start the jail: jail -c j01

##  Version 2: J02 via a Template Jail

* Create a Template Jail
  * export JAILS=/var/jails
  * export ZJAILS=zroot/jails
  * zfs create -p $ZJAILS/template-10.2-RELEASE
  * Install and Verify:

      tar -xvf /mnt/usr/freebsd-dist/base.txz -C $JAILS/j01
      tar -xvf /mnt/usr/freebsd-dist/lib32.txz -C $JAILS/j01
      tar -xvf /mnt/usr/freebsd-dist/ports.txz -C $JAILS/j01
      UNAME_r=10.2-RELEASE freebsd-update -b $JAILS/j01 fetch install
      UNAME_r=10.2-RELEASE freebsd-update -b $JAILS/j01 IDS
      cp /etc/resolv.conf     $JAILS/j01/etc/resolv.conf
      cp /etc/localtime       $JAILS/j01/etc/localtime
      echo hostname=\"j01\" > $JAILS/j01/etc/rc.conf

  * zfs snapshot $ZJAILS/template-10.2-RELEASE@p10 (p10 is the patch level)

* Deploy as j02:
  * Create jail FS
      zfs clone $ZJAILS/template-10.2-RELEASE@p10 $JAILS/j02
      echo hostname=\"j01\" > $JAILS/j01/etc/rc.conf
  * Configure the jail in /etc/jails.conf:
      j02 {
          host.hostname = "j02.jailhouse.io";
          path = "/var/jails/j02";
          interface = "em0"; ## Or whatever
          ip4.addr = 10.0.0.12; ## Whatever
      }

  * Start the jail: jail -c j02

##  Version 3: J03 via nullfs mounts, sharing common files

* The approach:
  * Create a "shared" template file system
  * Create a "private" jail-local file system
  * Jail will have a /data mount point for the private FS
  * Move writable dirs from shared to private, symlink original to /data/
  * Create jail data file system as j03.fs
  * Create $JAILS/j03/data tree as jail home
  * Use MullFS to mount shared as RO, and data as RW.

* Create template:
      zfs clone $ZJAILS/template-10.2-RELEASE@p10 $ZJAIL/shared-10.2-RELEASE
      zfs create -p $ZJAILS/private-10.2-RELEASE

      mkdir -p $JAILS/private-10.2-RELEASE/usr/ports/distfiles \
               $JAILS/private-10.2-RELEASE/home \
               $JAILS/private-10.2-RELEASE/portsbuild
      mv $JAILS/shared-10.2-RELEASE/etc       $JAILS/private-10.2-RELEASE/etc
      mv $JAILS/shared-10.2-RELEASE/usr/local $JAILS/private-10.2-RELEASE/usr/local
      mv $JAILS/shared-10.2-RELEASE/tmp       $JAILS/private-10.2-RELEASE/tmp
      mv $JAILS/shared-10.2-RELEASE/var       $JAILS/private-10.2-RELEASE/var
      mv $JAILS/shared-10.2-RELEASE/root      $JAILS/private-10.2-RELEASE/root

* ln writable dirs in shared to PRIVATE/data/ (with RELATIVE paths!!)
      cd $JAILS/shared-10.2-RELEASE
      mkdir data
      ln -s data/etc etc
      ln -s data/home home
      ln -s data/root root
      ln -s data/usr/local usr/local
      ln -s data/usr/ports/distfiles usr/ports/distfiles
      ln -s data/tmp tmp
      ln -s data/var var

* Finish and snapshot
      echo "WRKDIRPREFIX=/data/portbuild" >> $JAILS/private-10.2-RELEASE/etc/make.conf
      zfs snapshot $ZJAILS/private-10.2-RELEASE@skeleton

* Create J03 from shared template
      zfs clone $ZJAILS/private-10.2-RELEASE@skeleton $JAILS/j03.fs
      mkdir -p $JAILS/j03/data
      echo hostname=\"j03\" > $JAILS/j03/etc/rc.conf

  * Create $JAILS/j03.fstab
      /var/jails/shared-10.2-RELEASE/ /var/jails/pg0103/     nullfs  ro  0 0
      /var/jails/pg0103.fs            /var/jails/pg0103/data nullfs  rw  0 0

  * Configure the jail in /etc/jails.conf:
      j03 {
          host.hostname = "j03.jailhouse.io";
          path = "/var/jails/j03";
          interface = "em0"; ## Or whatever
          ip4.addr = 10.0.0.13; ## Whatever
      }

  * Start the jail: jail -c j03

##  Version 4: J04 via ezjail

https://www.freebsd.org/doc/handbook/jails-ezjail.html

* pkg install ezjail
* vi /etc/rc.conf:

    cloned_interfaces="lo1"
    ezjail_enable="YES"

* service ezjail start
* ezjail-admin install -p

* ezjail-admin create j04 'lo1|127.0.1.1,em0|10.0.0.14'

* Jail config:  /usr/local/etc/ezjail/j04

* ezjail-admin start   j04

* Configure jail:
  * ezjail-admin console j04
  * passwd ### Set the root password
  * Set up Time Zone
  * Set up /etc/resolv.conf
  * Set up /etc/hosts
  * Set up /etc/rc.conf

* Update ports: ezjail-admin update -P

* ezjail-admin stop j04

## PostgreSQL

http://loga.us/2015/02/09/postgresql-9-x-streaming-repilication-on-freebsd-part-i/

* Install

    portsnap extract
    portsnap fetch update
    cd /usr/ports/devel/p5-File-HStore/ && make install clean

  * In each jail: vipw -d /var/jails/jailname/etc/, change UID 70 to
    something like: 70+iplast or 70 + jail number:

    pgsql:*:70203:70:PostgreSQL pseudo-user:/usr/local/pgsql:/bin/sh

  * add 'postgresql_enable="YES"' to /etc/rc.conf

  * /usr/local/etc/rc.d/postgresql initdb