Skip to content

Instantly share code, notes, and snippets.

@No9

No9/iocage-meetup.md

Last active July 21, 2020 03:16
Show Gist options
  • Save No9/b7cfe8dc2d0f02854422af3b6edad6ad to your computer and use it in GitHub Desktop.
Save No9/b7cfe8dc2d0f02854422af3b6edad6ad to your computer and use it in GitHub Desktop.
Download ZIP
Some notes for the BSD Dublin Meetup 2017-04-20
Raw
iocage-meetup.md

iocage overview

Introduction

OS level virtualisation is attracting a lot of attention at the moment. A list of the technologies is available here http://blog.aquasec.com/a-brief-history-of-containers-from-1970s-chroot-to-docker-2016

A far more detailed history of the formative years is available here

Bryan Cantrill on Jails and Solaris Zones http://paperswelove.org/2016/video/bryan-cantrill-jails-and-solaris-zones/

To understand how to build jails using the OS level commands https://clinta.github.io/freebsd-jails-the-hard-way/

Look at IOCage

Create a $10 Freebsd ZFS server on Digital Ocean https://www.digitalocean.com/community/tutorials/how-to-create-your-first-digitalocean-droplet-virtual-server

You will need to generate a key first if you don't have one

ssh-keygen -t rsa -C "[email protected]"

This will put it into your home folder ~/.ssh/id_rsa.pub

  • Configure the environment
sudo pkg install git
sudo pkg install python36
sudo python3.6 -m ensurepip
git clone --recursive https://github.com/iocage/iocage
sudo pip3.6 install Cython
curl http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.0-RELEASE/src.txz > src.txz 
sudo tar -C / -xvf src.txz
cd iocage/py-libzfs && python3.6 setup.py build 
sudo python3.6 setup.py install
cd .. 
sudo pip3.6 install .
  • Configure the network Based on this document for networking jails

https://www.kirkg.us/posts/how-to-configure-a-freebsd-jail-on-a-digital-ocean-droplet/

Add the following to /etc/rc.conf

iocage_enable="YES"
cloned_interfaces="lo1"
ipv4_addrs_lo1="192.168.0.1-9/29"
pf_enable="YES"

Create /etc/pf.conf and add the following

IP_PUB="206.189.5.9"

# Packet normalization
scrub in all

# Allow outbound connections from within the jails
nat on vtnet0 from lo1:network to any -> $IP_PUB port 0:65535

# webserver jail at 192.168.0.2
rdr on vtnet0 proto tcp from any to $IP_PUB port 443 -> 192.168.0.2
# just an example in case you want to redirect to another port within your jail
rdr on vtnet0 proto tcp from any to $IP_PUB port 80 -> 192.168.0.2 port 80

Load the pf configuration

$ sudo service pf start
$ sudo pfctl -f /etc/pf.conf

Bring up the clone interface

$ sudo service netif cloneup
$ ifconfig 
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 82:11:d6:bb:5a:ff
	inet 146.185.155.138 netmask 0xffffff00 broadcast 146.185.155.255 
	inet 10.14.0.6 netmask 0xffff0000 broadcast 10.14.255.255 
	inet6 fe80::8011:d6ff:febb:5aff%vtnet0 prefixlen 64 scopeid 0x1 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	groups: lo 
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 192.168.0.1 netmask 0xfffffff8
        inet 192.168.0.3 netmask 0xffffffff
        inet 192.168.0.4 netmask 0xffffffff
        inet 192.168.0.5 netmask 0xffffffff
        inet 192.168.0.6 netmask 0xffffffff
        inet 192.168.0.7 netmask 0xffffffff
        inet 192.168.0.8 netmask 0xffffffff
        inet 192.168.0.9 netmask 0xffffffff
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
  • Create the jail
$ sudo iocage fetch
$ sudo iocage create -n nginx ip4_addr="lo1|192.168.0.2/24" -r 11.1-RELEASE
  • start the jail
$ sudo iocage start myjail
  • verify the jail exists
$ jls
   JID  IP Address      Hostname                      Path
     1  172.16.1.1       8b4702b2-03a1-4989-8367-e068c /iocage/jails/8b4702b2-03a1-4989-8367-e068cf572a4d/root
  • Add a user including the wheel group for su access
$ sudo jexec 1 touch /etc/fstab
$ sudo jexec 1 passwd
$ sudo jexec 1 adduser
Login group is tester. Invite tester into other groups? []: wheel
  • login to the jail with a non-root account and look around.
$ sudo jexec 1 login
$ telnet digitalocean.com 80
$ su
# exit
$ exit
  • set the jail to start on reboot
$ sudo iocate set boot=on myjail

Additional Reading

Copy jails between hosts

https://www.linkedin.com/pulse/freebsd-jails-zfs-axel-s-gruner

https://groups.google.com/forum/#!topic/iocage/nTcpXp7LiW0

A JSON API for jails https://api.sysadm.us/classes/iocage.html

VNET build for virtual box https://lists.freebsd.org/pipermail/freebsd-virtualization/2011-January/000633.html

@dariocravero
Copy link

IOCCreate class used in the cli's create method to create a jail!

@dariocravero
Copy link

http://galeracluster.com/documentation-webpages/jails.html

@dariocravero
Copy link

dariocravero commented Apr 20, 2017 •
edited
Loading

from that article above
/etc/rc.conf

cloned_interfaces="${cloned_interfaces} lo1"

# Firewall Configuration
pf_enable="YES"
pf_rules="/etc/pf.conf"
pflog_enable="YES"
pflog_logfile="/var/log/pf.log"

service netif cloneup

/etc/pf.conf

# External Network Interface
ext_if="em0"

# Internal Network Interface
int_if="lo1"

# IP Addresses
external_addr="172.16.3.212"
internal_addr="10.1.1.10/24"
jail_IP_address="10.1.1.10"

# Variables for Galera Cluster
wsrep_ports="{3306,4567,4568,4444}"
table <wsrep_cluster_address> persist {192.168.1.1,192.168.1.2,192.168.1.3}

# Translation
nat on $ext_if from $internal_addr to any -> ($ext_if)

# Redirects
rdr on $ext_if proto tcp from any to $external_addr/32 port 3306 -> $jail_IP_address port 3306
rdr on $ext_if proto tcp from any to $external_addr/32 port 4567 -> $jail_IP_address port 4567
rdr on $ext_if proto tcp from any to $external_addr/32 port 4568 -> $jail_IP_address port 4568
rdr on $ext_if proto tcp from any to $external_addr/32 port 4444 -> $jail_IP_address port 4444

pass in proto tcp from <wsrep_cluster_address> to any port $wsrep_ports keep state

pfctl -v -nf /etc/pf.conf
service pf start
service pflog start

when creating the jail, we have to set it up with the cloned network interface instead:

iocage create tag=myjail ip4_addr="lo1|10.1.1.10/24" -r 11.0-RELEASE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment