jkeiser

Parallelization in Chef Recipes

Chef presently runs the resources in a recipe serially, one after the next. In this proposal, user-selected groups of resources will run their actions in parallel.

MVP (Minimum Viable Product) Features

in_parallel

To run a group of resources in parallel, you write it this way:

jkeiser

=ERROR REPORT==== 4-Feb-2014::19:56:26 ===
** State machine <0.210.0> terminating
** Last message in was {'EXIT',<0.208.0>,killed}
** When State == ready
**      Data  == {state,undefined,<0.211.0>,5000,
                        [{<<"application_name">>,<<>>},
                        {<<"client_encoding">>,<<"UTF8">>},
                        {<<"DateStyle">>,<<"ISO, MDY">>},
                        {<<"integer_datetimes">>,<<"on">>},
                        {<<"IntervalStyle">>,<<"postgres">>},

jkeiser

[gfm] Introducing Chef Metal 0.2! Chef Metal is a framework that lets you manage your *clusters* with Chef the same way you manage machines: with recipes. Combined with the power of Chef, Metal's `machine` resource helps you to describe, version, deploy and manage everything from simple to complex clusters with a common set of tools.

To get it, gem install chef-metal. To hack, go to https://github.com/opscode/chef-metal. Currently supported provisioners include LXC containers, EC2, DigitalOcean, and Vagrant. If you just want to skip all that, follow the quick start in the README.

The current release is an alpha. You can see a lot of our plans, and some concrete examples, in the [requirements doc](https://github.com/opscode/chef-metal/blob/master/docs/r

jkeiser

# 1. Create
chef_role 'blah' do
  user_permissions 'cosmo', :all
  user_permissions 'jerry', :read, :write, :delete
  group_permissions 'errbody', :read
  group_permissions 'clients', :read, :write
end

# 2. Modify

jkeiser

machine 'server' do
  recipe 'chef-server'
  action :nothing
end.run_action(:create)

# Get 'server' node from current 
server_node = Chef::Node.load('server')

# Retrieve admin .pem file from server
machine_file '/path/to/admin.pem' do

jkeiser

root@test:/mnt/host_src/chef-metal-lxc# ruby -I lib lxccheck.rb
received 1 INSIDE!
received 2 OUTSIDE!
received 3 INSIDE!
received 4 OUTSIDE!
received 5 INSIDE!
received 6 OUTSIDE!
received 7 INSIDE!
received 8 OUTSIDE!
received 9 INSIDE!

jkeiser

root@test:/mnt/host_src/chef-metal-lxc/test# strace -f -e process chef-client -z -l debug -o lxctests::simple2
execve("/usr/local/bin/chef-client", ["chef-client", "-z", "-l", "debug", "-o", "lxctests::simple2"], [/* 23 vars */]) = 0
arch_prctl(ARCH_SET_FS, 0x7fc526822740) = 0
clone(child_stack=0x7fc52682cfb0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc52682d9d0, tls=0x7fc52682d700, child_tidptr=0x7fc52682d9d0) = 32664
Process 32664 attached
[pid 32664] _exit(0)                    = ?
[pid 32664] +++ exited with 0 +++
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fc526822a10) = 32665
clone(child_stack=0x7fc52682cfb0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc52682d9d0, tls=0x7fc52682d700, child_tidptr=0x7fc52682d9d0) = 32666
Process 32666 attached

jkeiser

[pid  3801] clone(Process 3802 attached
[pid  3802] getpid()                    = 1864
[pid  3802] read(23,  <unfinished ...>
[pid  3801] <... clone resumed> child_stack=0x7fff0cd40130, flags=CLONE_PARENT|SIGCHLD) = 3802
[pid  3802] <... read resumed> "\0\0\0\0", 4) = 4
[pid  3802] personality(PER_LINUX)      = 0
[pid  3802] open("/proc/sys/kernel/cap_last_cap", O_RDONLY) = 22
[pid  3802] read(22, "36\n", 31)        = 3
[pid  3802] close(22)                   = 0
[pid  3802] prctl(PR_CAPBSET_DROP, 0x10, 0, 0, 0) = 0

jkeiser

module LXC
  module Extra
    def execute(&block)
      r,w = IO.pipe
      pid = attach do
        ENV.clear
        ENV['PATH'] = '/usr/bin:/bin:/usr/sbin:/sbin'
        ENV['TERM'] = 'xterm-256color'
        ENV['SHELL'] = '/bin/bash'
        r.close

jkeiser

require 'lxc'
require 'socket'
s = TCPServer.new(9050)
c = LXC::Container.new('simple')
c.stop if c.running?
c.start