Skip to content

Instantly share code, notes, and snippets.

@dch

dch/freebsd_packaging_notes.md

Last active March 24, 2018 01:30
Show Gist options
  • Save dch/ec2693c051c66dcd2f17b30fc575a910 to your computer and use it in GitHub Desktop.
Save dch/ec2693c051c66dcd2f17b30fc575a910 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
freebsd_packaging_notes.md

Packaging

FreeBSD 10+ provides a new pkg(7) tool and a new efficient binary format for storing package info, with similar functionality to apt or yum in Linux distros.

Despite this, building custom packages on FreeBSD is a very simple task and we should use it as much as possible, both to set build -specific options, and to make our own private ports, instead of just relying on upstream packages and changes.

poudriere is the tool of choice to do this. It's job is to build each package and all its dependencies within a clean empty FreeBSD jail, and to provide annotated build logs and final packages that are directly usable as an external package repository.

setting up poudriere

basic configuration

# the master config file /usr/local/etc/poudriere.conf
ZPOOL=zroot
FREEBSD_HOST=http://ftp.de.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere
USE_PORTLINT=yes
USE_TMPFS=yes
DISTFILES_CACHE=/usr/ports/distfiles
SVN_HOST=svn.FreeBSD.org
CHECK_CHANGED_OPTIONS=verbose
PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/pkg.example.org.key
WRKDIR_ARCHIVE_FORMAT=txz
URL_BASE=https://pkg.example.org/poudriere
KEEP_OLD_PACKAGES=yes
BUILDER_HOSTNAME=pkg.example.org

set make.conf

Tweaking /usr/local/etc/poudriere.d/make.conf ensures we can build dtrace-enabled ports and avoid openssl as a dependency:

BATCH=yes
DEVELOPER=yes
PROXYDEPS_FATAL=yes
KEEP_OLD_PACKAGES=yes
USE_TMPFS=all
TMPFS_LIMIT=2
WRKDIRPREFIX=/tmp
CCACHE_DIR=/var/cache/ccache
# Erlang/OTP with DTrace
STRIP=
WITH_CTF=1
CFLAGS+=-fno-omit-frame-pointer
# OpenSSL avoidance
DEFAULT_VERSIONS+=ssl=libressl
OPTIONS_UNSET= GSSAPI_BASE
OPTIONS_SET=   GSSAPI_MIT
# enable our custom ports category
VALID_CATEGORIES+=example

Enable ln -s /usr/local/etc/poudriere.d/make.conf /etc/make.conf if you are lazy and want to keep the same options for manual port testing.

set up signing keys for our packages

mkdir -p /usr/local/etc/ssl/keys /usr/local/etc/ssl/certs
chmod 600 /usr/local/etc/ssl/keys
openssl genrsa -out /usr/local/etc/ssl/keys/pkg.example.org.key 4096
openssl rsa -in /usr/local/etc/ssl/keys/pkg.example.org.key \
  -pubout > /usr/local/etc/ssl/certs/pkg.example.org.cert

set up ccache for faster builds

zfs create -o canmount=off zroot/var/cache
zfs create zroot/var/cache/ccache

set up our ports tree

We actually have 2 ports trees - a public one, in /usr/ports, and a private one, in /usr/ports/example:

poudriere ports -c -F -f none -M /usr/ports -m git -n default
mkdir -p /usr/local/etc/poudriere.d/ports/default/
echo git> /usr/local/etc/poudriere.d/ports/default/method
echo /usr/ports> /usr/local/etc/poudriere.d/ports/default/mnt
git clone git://github.com/example/ports /usr/ports
cd /usr/ports
git remote add upstream git://github.com/freebsd/freebsd-ports
git fetch --all
git clone [email protected]:example/wharf /usr/ports/example
# source tarballs from upstream repos are downloaded & cached here
zfs create zroot/usr/ports/distfiles

install poudriere

pkg install -y dialog4ports poudriere ccache
poudriere jail -c -j 11_amd64  -v 11.0-RELEASE -a amd64

set up and run a custom port build

# make a list of builds we are interested in having a custom port for
echo lang/erlang-runtime18 > /usr/local/etc/poudriere.d/packages.lst

using custom port builds

We have some port, say, lang/erlang where we wish to enable dtrace, hipe, and dirty schedulers. For dtrace support, make sure kernel support is started via kldload dtraceall or the build will fail mysteriously. This is typically enabled by adding dtraceall_load="YES" to /boot/loader.conf.

# we could simply customise all packages listed in packages.lst
poudriere options -f /usr/local/etc/poudriere.d/packages.lst
# or just customise a single package and assume defaults for its descendants
poudriere options -cn lang/erlang-runtime18
# look in here for the specific set options
/usr/local/etc/poudriere.d/options/lang_erlang-runtime18/options

# clear and re-do options for a specific set of ports
poudriere options -cn editors/vim lang/erlang net-mgmt/collectd5 lang/erlang-runtime18 devel/protobuf-c

keeping things updated

  • the port tree itself needs to be merged/rebased into an appropriate upstream ports branch
  • the jail needs to be updated
  • the host system also needs to be updated
# ports tree
cd /usr/ports
git fetch --all
# float our custom patches up to the top again
# any patches that have been merged in upstream
# fail, so just `git rebase --skip` over them.
git rebase upstream/master
# .... hackety hackety
git push --force origin/master
# update jail
poudriere jail -u -j 11_amd64
# rebuild its packages
poudriere bulk -f /usr/local/etc/poudriere.d/custom.pkg -j 11_amd64

exporting our packages

  • rsync and file:/// urls are perfectly fine, as is zfs send etc
  • www/h2o has a configuration for exporting our custom packages over HTTP(S)

Advanced pkg repo security

Some package repositories need security. While a certificate-based approach may be possible, I've successfully used ssh://, http basic authentication, and even a humble rsync and local file:// url.

using file URL and rsync

The details of rsync or zfs send|recv are left to the user. Ensure softlinks are preserved appropriately. Within your /usr/local/etc/pkg/repos/repo.conf` file, simply use an appropriate path:

example: {
        url: file:///usr/local/poudriere/data/packages/11_amd64-default
        mirror_type: http
        signature_type: pubkey
        pubkey: /usr/local/etc/ssl/certs/poudriere.cert
        enabled: yes
}

ssh repo security

Use ssh:// urls, including username, url, and path, to retrieve packages securely, in your /usr/local/etc/pkg/repos/repo.conf file. Specific ssh keys can be configured in /root/.ssh/config as required. Ensure you restrict access on the server side too, via sshd_config.

url: ssh://pkg.example.org:/usr/local/poudriere/data/packages/11_amd64-default

http basic auth

The underlying libfetch library is used by pkg to do the dirty work. See https://www.freebsd.org/cgi/man.cgi?query=fetch&sektion=3 for details.

Add this to your h2o.conf file for serving packages, making use the jail name you are using in poudriere matches the file.dir location -- in this case, it's 11_amd64-default:

    pkg.example.org:
        paths:
            "/poudriere":
                file.dir: "/usr/local/share/poudriere/html/"
            "/poudriere/data":
                file.dir: "/usr/local/poudriere/data/logs/bulk/"
            "/FreeBSD:11:amd64":
                mruby.handler: |
                    require "htpasswd.rb"
                    Htpasswd.new("/usr/local/etc/h2o/private/htpasswd", "example")
                file.dir: "/usr/local/poudriere/data/packages/11_amd64-default/"

When running pkg update and similar, export HTTP_AUTH into the environment to be consumed by pkg: HTTP_AUTH="basic:example:user:passwd" pkg update

integrating custom private ports into the ports tree

  • we use git to edit and maintain any public-facing port changes in /usr/ports
  • git clone git://github.com/example/wharf /usr/ports/example
  • add /usr/ports/Makefile.local to add our example category into the ports tree
  • there is a VALID_CATEGORIES+=example in /etc/make.conf and /usr/local/etc/poudriere.d/make.conf for poudriere automated package builds and normal by-hand builds as well

tips

  • there is a single Makefile for each port to describe the build process
  • see simple for a minimal port that uses a git tag to build from
  • wharf is the equivalent of the https://github.com/freebsd/freebsd-ports repo to provide Makefiles for build and packaging scripts for our private ports

pkg(8) hacks

FreeBSD's pkg(8) tool supports a few handy package management commands:

pkg lock -y <thing>
pkg unlock -y <thing>
# get a list of all manually installed packages (i.e. "really useful" vs automatic dependencies)
pkg leaf
# what package installed this file?
pkg which /usr/local/bin/erl
# read any major ports-related notes for upgrading packages since a given date
pkg updating -d 20160101

You can also retrieve an older version of a package from the local cache in /var/cache/pkg/<name-version>.txz and install it directly via pkg add <path>, although generally it's better to keep packages consistent within the set built at the same time on the package server pkg.example.org and use them from there.

resources

packaging

canonical

useful but not up to date:

main ports tree diff

diff --git a/.gitignore b/.gitignore
index 13ad354..51ffbb7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,3 +6,4 @@
 *.rej
 *.orig
 *.sw[p-z]
+/example
diff --git a/Makefile.local b/Makefile.local
new file mode 100644
index 0000000..790f37f
--- /dev/null
+++ b/Makefile.local
@@ -0,0 +1,3 @@
+# $FreeBSD$
+#
+SUBDIR                  += example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment