smartos-live src development workflow

workflow.md

Workflow

This workflow was designed specifically to aid in development and testing the components within smartos-live/src. SmartOS is designed to be a read-only operating system that is loaded onto a ramdisk during boot. While this design is very helpful in a datacenter, it is not well suited for local incremental development. This workflow will temporarily alter the behavior of SmartOS to allow a quick and iterative workflow for development.

Goals

The goals of this workflow are to allow the following:

1. Development occurs directly on the developers workstation, in his or her native editor and toolchain.
2. All sources within the smartos-live/src directory are compiled when changed, including c files.
3. Incremental build output changes are synced and reflected on the global zone in realtime, without a reboot.
4. Incremental builds and syncronizations are quick enough that experimentations can be tested immediately.

Overview

End Result

After the initial setup, the workflow will flow between three different contexts: the desktop workstation, a SmartOS build zone, and the global zone where the changes are tested.

Desktop

Your desktop is where the development occurs. This workflow allows edits to be made directly on the source that can be committed and pushed directly. The main advantage here is that a native editor/environment can be used and ultimately there is no source that needs to be synced back onto the repository before the code can be pushed.

The desktop will need to act as an nfs server, and export the smartos-live/src directory. This directory will later be mounted inside of the SmartOS build zone.

SmartOS build zone

A dedicated build zone is the easiest way to create an environment where the code can be built. Due to the way the project is setup, a full build is required once. After that point, incremental builds can be isolated to the smartos-live/src directory. From this point forward, builds are very quick.

One of the primary reasons for using a full build zone is the ability to modify c source files and to generate the props.js file without difficulty. In addition, there are many small files/utilites in the smartos-live/src project that require modifications when implementing a feature. At least for me, it's easiest to be able to build it all at once instead of trying to sync each individual change.

For the build, we'll actually follow the default setup instructions from the wiki with one slight exception: after the smartos-live project is cloned and checked out to the proper branch, we'll mount over the smartos-live/src directory with the nfs export from the desktop. This allows changes on the desktop workstation to be included in the build.

Global Zone

The global zone will be converted into a writable filesystem that will allow us to sync the changes from the incremental builds into the running filesystem for testing. Since we will only be concerned with /usr, this is accomplished by copying the contents of /usr into a persistent directory, and then mounting that directory back on top of /usr. At this point, we can rsync the iterative build particles onto the persistent directory, which will then be reflected in the global zone. Of course, any daemons using releavant code will need to be restarted.

Setup

VMware

There is nothing inherently special about the VMware setup. I launched a virtual machine and followed this guide. If the vmware image is recent enough, you won't need to reboot into the custom build (which we'll go over below).

Workstation

The workstation needs to export the smartos-live/src directory via an nfs share. I use a mac, so the steps go like this:

$ echo '"/Users/tylerflint/Work/clones/smartos-live/src" -network 172.16.89.0 -mask 255.255.255.0 -mapall=501:20' >> /etc/exports
$ sudo nfsd update

NOTE: The location of your local code path will differ, and the vmware network may be different as well.

Build Zone

1. From the global zone, create a build zone

2. Import the recommended build image

```bash
$ imgadm import a1d74530-4212-11e3-8a71-a7247697c8f2
```

2. Create the build zone

```bash
$ vmadm create <<-END
{
  "brand": "joyent",
  "max_physical_memory": 8192,
  "tmpfs": 8192,
  "fs_allowed": "ufs,pcfs,tmpfs",
  "image_uuid": "a1d74530-4212-11e3-8a71-a7247697c8f2",
  "quota": 15,
  "alias": "build",
  "hostname": "build",
  "resolvers": ["8.8.8.8", "8.8.4.4"],
  "nics": [
      {
          "nic_tag": "admin",
          "ip": "dhcp"
      }
  ],
  "internal_metadata": {
      "root_pw": "password",
      "admin_pw": "password"
  }
}
END
```

2. Configure the build zone

3. zlogin into the build zone

```bash
$ zlogin $(vmadm list | grep build | awk '{ print $1 }')
```

2. install git

```bash
$ pkgin in git
```

3. clone the smartos-live source

```bash
$ git clone https://github.com/tylerflint/smartos-live.git
$ cd smartos-live
```

4. checkout the relevant branch

```bash
$ git checkout  origin/OS-2647
```

5. copy the configure script

```bash
$ cp sample.configure.smartos configure.smartos
```

6. mount the src directory from the workstation

```bash
$ /usr/sbin/mount -Fnfs -o rw,intr 172.16.89.1:/Users/tylerflint/Work/clones/smartos-live/src /root/smartos-live/src
```

3. Build the initial image

4. configure

```bash
./configure
```

2. build

```bash
export MAX_JOBS=8
gmake world && gmake live
```

Global Zone

1. Update the boot image

This is only necessary if the boot image and the image created from the recent build have diverged significantly.

1. make a directory to mount the boot image into

```bash
$ mkdir -p /var/tmp/boot
```

2. mount the image

```bash
$ mount -F pcfs /dev/dsk/c0t0d0p1 /var/tmp/boot
```

3. remove the existing platform files

```bash
$ rm -rf /var/tmp/boot/platform/i86pc
```

4. copy the recent build platform onto the boot platform

```bash
$ cp -r /zones/$(vmadm list | grep build | awk '{ print $1 }')/root/root/smartos-live/output/platform-20141127T182625Z/i86pc /var/tmp/boot/platform
```

5. reboot into the new image

```bash
$ reboot
```

2. Convert the global zone /usr into a writable filesystem

NOTE: This will need to be done with each reboot.

1. Copy the existing contents of /usr into a writable directory

```bash
$ cp -RP /usr /var/tmp/usr
```

2. mount the writable directory on top of the live /usr directory

```bash
$ mount -O -F lofs /var/tmp/usr /usr
```

Iterate!

Workstation

Modify existing files, add new files, remove old files, etc.

Build

1. Mount smartos-live/src if not already mounted

$ /usr/sbin/mount -Fnfs -o rw,intr 172.16.89.1:/Users/tylerflint/Work/clones/smartos-live/src /root/smartos-live/src

2. cd into src directory

$ cd smartos-live/src

3. build

$ gmake

4. install

$ gmake install

Global zone

1. Sync changes

$ rsync \
    -rlptgo \
    --safe-links \
    /zones/$(vmadm list | grep build | awk '{ print $1 }')/root/root/smartos-live/proto/usr/ \
    /var/tmp/usr

Summary

The initial setup takes a bit of time, but after the initial setup and build, the iterative builds and are quite quick. I've found that it takes aproximately 15 minutes for the initial setup, and about 3 hours for the initial build. I usually kickoff an initial build before I go to bed.

In my experience, a source code change from my workstation can be on the global zone within 10 seconds.

This is a work in progress. Any suggestions would be appreciated.