mgmt-grouping-analysis

#
#	mgmt grouping analysis - 29/mar/2016
#	By: James Shubin <[email protected]>
#	https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/
#

* Comparison of different backends for package installation
* All times are in seconds. All tests ran with warm caches. Longer is worse.
* Data was collected from multiple runs but only one sample of each shown here.
* Accompanying spreadsheet with full data is also available.

#
#	versions
#

$ time puppet --version 
4.2.1

real	0m0.659s
user	0m0.525s
sys	0m0.064s

$ time ./mgmt --version
mgmt version 0.0.3-1-g6f3ac4b

real	0m0.007s
user	0m0.006s
sys	0m0.002s

$ time pkcon --version
1.0.11

real	0m0.013s
user	0m0.006s
sys	0m0.005s

$ time dnf --version
1.1.7
  Installed: dnf-0:1.1.7-2.fc23.noarch at 2016-03-17 13:37
  Built    : Fedora Project at 2016-03-09 16:45

  Installed: rpm-0:4.13.0-0.rc1.12.fc23.x86_64 at 2016-03-03 09:39
  Built    : Fedora Project at 2016-02-29 09:53

real	0m0.438s
user	0m0.379s
sys	0m0.036s

#
#	single package, package check, package already installed
#

$ time sudo puppet apply pkg.pp 
Notice: Compiled catalog for computer.example.com in environment production in 0.69 seconds
Notice: Applied catalog in 0.25 seconds

real	0m8.860s
user	0m4.325s
sys	0m1.262s

$ time sudo ./mgmt run --file examples/pkg1.yaml --converged-timeout=0
21:01:37 main.go:63: This is: mgmt, version: 0.0.3-1-g6f3ac4b
21:01:37 main.go:64: Main: Start: 1459299697979476457
21:01:37 main.go:190: Main: Running...
21:01:37 main.go:113: Etcd: Starting...
21:01:37 main.go:117: Main: Waiting...
21:01:37 etcd.go:113: Etcd: Watching...
21:01:37 etcd.go:113: Etcd: Watching...
21:01:37 configwatch.go:54: Watching: examples/pkg1.yaml
21:01:38 config.go:272: Compile: Adding AutoEdges...
21:01:38 config.go:533: Compile: Grouping: Algorithm: nonReachabilityGrouper...
21:01:38 main.go:171: Graph: Vertices(1), Edges(0)
21:01:38 main.go:174: Graphviz: No filename given!
21:01:38 pgraph.go:764: State: graphStateNil -> graphStateStarting
21:01:38 pgraph.go:825: State: graphStateStarting -> graphStateStarted
21:01:38 main.go:117: Main: Waiting...
21:01:38 pkg.go:245: Pkg[powertop]: CheckApply(true)
21:01:38 main.go:79: Converged for 0 seconds, exiting!
21:01:38 main.go:55: Interrupted by exit signal
21:01:38 pgraph.go:796: Pkg[powertop]: Exited
21:01:38 main.go:203: Goodbye!

real	0m0.387s
user	0m0.030s
sys	0m0.016s

$ time sudo pkcon install powertop
Resolving                     [=========================]         
Starting                      [=========================]         
Finished                      [=========================]         
Fatal error: powertop-2.8-1.fc23.x86_64 is already installed

real	0m0.171s
user	0m0.026s
sys	0m0.019s

$ time sudo dnf install -y powertop
Last metadata expiration check: 1:23:41 ago on Tue Mar 29 20:04:29 2016.
Package powertop-2.8-1.fc23.x86_64 is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!

real	0m2.980s
user	0m2.778s
sys	0m0.189s

#
#	single package, package install, package not installed
#

$ time sudo puppet apply pkg.pp 
Notice: Compiled catalog for computer.example.com in environment production in 0.69 seconds
Notice: /Stage[main]/Main/Package[powertop]/ensure: created
Notice: Applied catalog in 10.13 seconds

real	0m18.254s
user	0m9.211s
sys	0m2.074s

$ time sudo ./mgmt run --file examples/pkg1.yaml --converged-timeout=0
21:04:18 main.go:63: This is: mgmt, version: 0.0.3-1-g6f3ac4b
21:04:18 main.go:64: Main: Start: 1459299858287120473
21:04:18 main.go:190: Main: Running...
21:04:18 main.go:113: Etcd: Starting...
21:04:18 main.go:117: Main: Waiting...
21:04:18 etcd.go:113: Etcd: Watching...
21:04:18 etcd.go:113: Etcd: Watching...
21:04:18 configwatch.go:54: Watching: examples/pkg1.yaml
21:04:20 config.go:272: Compile: Adding AutoEdges...
21:04:20 config.go:533: Compile: Grouping: Algorithm: nonReachabilityGrouper...
21:04:20 main.go:171: Graph: Vertices(1), Edges(0)
21:04:20 main.go:174: Graphviz: No filename given!
21:04:20 pgraph.go:764: State: graphStateNil -> graphStateStarting
21:04:20 pgraph.go:825: State: graphStateStarting -> graphStateStarted
21:04:20 main.go:117: Main: Waiting...
21:04:20 pkg.go:245: Pkg[powertop]: CheckApply(true)
21:04:20 pkg.go:303: Pkg[powertop]: Apply
21:04:20 pkg.go:317: Pkg[powertop]: Set: installed...
21:04:25 packagekit.go:399: PackageKit: Woops: Signal.Path: /8442_beabdaea
21:04:25 packagekit.go:399: PackageKit: Woops: Signal.Path: /8443_acbadcbd
21:04:31 pkg.go:335: Pkg[powertop]: Set: installed success!
21:04:31 main.go:79: Converged for 0 seconds, exiting!
21:04:31 main.go:55: Interrupted by exit signal
21:04:31 pgraph.go:796: Pkg[powertop]: Exited
21:04:31 main.go:203: Goodbye!

real	0m13.320s
user	0m0.023s
sys	0m0.021s

$ time sudo pkcon install powertop 
Resolving                     [=========================]         
Loading cache                 [=========================]         
Testing changes               [=========================]         
Finished                      [=========================]         
Installing                    [=========================]         
Querying                      [=========================]         
Downloading packages          [=========================]         
Testing changes               [=========================]         
Installing packages           [=========================]         
Finished                      [=========================]         

real	0m7.291s
user	0m0.052s
sys	0m0.029s

$ time sudo dnf install -y powertop
Last metadata expiration check: 1:22:03 ago on Tue Mar 29 20:04:29 2016.
Dependencies resolved.
==========================================================================
 Package          Arch           Version            Repository       Size
==========================================================================
Installing:
 powertop         x86_64         2.8-1.fc23         updates         228 k

Transaction Summary
==========================================================================
Install  1 Package

Total download size: 228 k
Installed size: 576 k
Downloading Packages:
powertop-2.8-1.fc23.x86_64.rpm            212 kB/s | 228 kB     00:01    
--------------------------------------------------------------------------
Total                                     125 kB/s | 228 kB     00:01     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Installing  : powertop-2.8-1.fc23.x86_64                            1/1 
  Verifying   : powertop-2.8-1.fc23.x86_64                            1/1 

Installed:
  powertop.x86_64 2.8-1.fc23                                              

Complete!

real	0m10.406s
user	0m4.954s
sys	0m0.836s

#
#	three packages, package check, packages already installed
#

$ time sudo puppet apply pkg3.pp 
Notice: Compiled catalog for computer.example.com in environment production in 0.75 seconds
Notice: Applied catalog in 0.26 seconds

real	0m8.571s
user	0m4.533s
sys	0m1.251s

$ time sudo ./mgmt run --file examples/autogroup2.yaml --converged-timeout=0
21:11:42 main.go:63: This is: mgmt, version: 0.0.3-1-g6f3ac4b
21:11:42 main.go:64: Main: Start: 1459300302188722023
21:11:42 main.go:190: Main: Running...
21:11:42 main.go:113: Etcd: Starting...
21:11:42 main.go:117: Main: Waiting...
21:11:42 etcd.go:113: Etcd: Watching...
21:11:42 etcd.go:113: Etcd: Watching...
21:11:42 configwatch.go:54: Watching: examples/autogroup2.yaml
21:11:42 config.go:272: Compile: Adding AutoEdges...
21:11:42 config.go:533: Compile: Grouping: Algorithm: nonReachabilityGrouper...
21:11:42 config.go:533: Compile: Grouping: Success for: Pkg[powertop] into Pkg[cowsay]
21:11:42 config.go:533: Compile: Grouping: Success for: Pkg[sl] into Pkg[cowsay]
21:11:42 main.go:171: Graph: Vertices(1), Edges(0)
21:11:42 main.go:174: Graphviz: No filename given!
21:11:42 pgraph.go:764: State: graphStateNil -> graphStateStarting
21:11:42 pgraph.go:825: State: graphStateStarting -> graphStateStarted
21:11:42 main.go:117: Main: Waiting...
21:11:42 pkg.go:245: Pkg[autogroup:(cowsay,powertop,sl)]: CheckApply(true)
21:11:42 main.go:79: Converged for 0 seconds, exiting!
21:11:42 main.go:55: Interrupted by exit signal
21:11:42 pgraph.go:796: Pkg[cowsay]: Exited
21:11:42 main.go:203: Goodbye!

real	0m0.740s
user	0m0.046s
sys	0m0.016s

$ time sudo pkcon install powertop sl cowsay
Resolving                     [=========================]         
Testing changes               [=========================]         
Finished                      [=========================]         
Fatal error: powertop-2.8-1.fc23.x86_64 is already installed

real	0m1.140s
user	0m0.031s
sys	0m0.017s

$ time sudo dnf install -y powertop sl cowsay
Last metadata expiration check: 1:28:09 ago on Tue Mar 29 20:04:29 2016.
Package powertop-2.8-1.fc23.x86_64 is already installed, skipping.
Package sl-5.02-2.fc23.x86_64 is already installed, skipping.
Package cowsay-3.03-19.fc23.noarch is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!

real	0m2.980s
user	0m2.763s
sys	0m0.194s

#
#	three packages, package install, packages not installed
#

$ time sudo puppet apply pkg3.pp 
Notice: Compiled catalog for computer.example.com in environment production in 0.68 seconds
Notice: /Stage[main]/Main/Package[powertop]/ensure: created
Notice: /Stage[main]/Main/Package[sl]/ensure: created
Notice: /Stage[main]/Main/Package[cowsay]/ensure: created
Notice: Applied catalog in 33.02 seconds

real	0m41.229s
user	0m19.085s
sys	0m4.046s

$ time sudo ./mgmt run --file examples/autogroup2.yaml --converged-timeout=0
21:16:00 main.go:63: This is: mgmt, version: 0.0.3-1-g6f3ac4b
21:16:00 main.go:64: Main: Start: 1459300560994114252
21:16:00 main.go:190: Main: Running...
21:16:00 main.go:113: Etcd: Starting...
21:16:00 main.go:117: Main: Waiting...
21:16:00 etcd.go:113: Etcd: Watching...
21:16:00 etcd.go:113: Etcd: Watching...
21:16:00 configwatch.go:54: Watching: examples/autogroup2.yaml
21:16:03 config.go:272: Compile: Adding AutoEdges...
21:16:03 config.go:533: Compile: Grouping: Algorithm: nonReachabilityGrouper...
21:16:03 config.go:533: Compile: Grouping: Success for: Pkg[powertop] into Pkg[cowsay]
21:16:03 config.go:533: Compile: Grouping: Success for: Pkg[sl] into Pkg[cowsay]
21:16:03 main.go:171: Graph: Vertices(1), Edges(0)
21:16:03 main.go:174: Graphviz: No filename given!
21:16:03 pgraph.go:764: State: graphStateNil -> graphStateStarting
21:16:03 pgraph.go:825: State: graphStateStarting -> graphStateStarted
21:16:03 main.go:117: Main: Waiting...
21:16:03 pkg.go:245: Pkg[autogroup:(cowsay,powertop,sl)]: CheckApply(true)
21:16:03 pkg.go:303: Pkg[autogroup:(cowsay,powertop,sl)]: Apply
21:16:03 pkg.go:317: Pkg[autogroup:(cowsay,powertop,sl)]: Set: installed...
21:16:08 packagekit.go:399: PackageKit: Woops: Signal.Path: /8547_cbeaddda
21:16:08 packagekit.go:399: PackageKit: Woops: Signal.Path: /8548_bcaadbce
21:16:16 pkg.go:335: Pkg[autogroup:(cowsay,powertop,sl)]: Set: installed success!
21:16:16 main.go:79: Converged for 0 seconds, exiting!
21:16:16 main.go:55: Interrupted by exit signal
21:16:16 pgraph.go:796: Pkg[cowsay]: Exited
21:16:16 main.go:203: Goodbye!

real	0m15.621s
user	0m0.040s
sys	0m0.038s

$ time sudo pkcon install powertop sl cowsay
Resolving                     [=========================]         
Starting                      [=========================]         
Testing changes               [=========================]         
Finished                      [=========================]         
Installing                    [=========================]         
Querying                      [=========================]         
Downloading packages          [=========================]         
Testing changes               [=========================]         
Installing packages           [=========================]         
Finished                      [=========================]         

real	0m14.755s
user	0m0.060s
sys	0m0.025s

$ time sudo dnf install -y powertop sl cowsay
Last metadata expiration check: 1:30:10 ago on Tue Mar 29 20:04:29 2016.
Dependencies resolved.
==========================================================================
 Package         Arch          Version               Repository      Size
==========================================================================
Installing:
 cowsay          noarch        3.03-19.fc23          updates         30 k
 powertop        x86_64        2.8-1.fc23            updates        228 k
 sl              x86_64        5.02-2.fc23           fedora          18 k

Transaction Summary
==========================================================================
Install  3 Packages

Total download size: 276 k
Installed size: 624 k
Downloading Packages:
(1/3): cowsay-3.03-19.fc23.noarch.rpm      86 kB/s |  30 kB     00:00    
(2/3): sl-5.02-2.fc23.x86_64.rpm           52 kB/s |  18 kB     00:00    
(3/3): powertop-2.8-1.fc23.x86_64.rpm     252 kB/s | 228 kB     00:00    
--------------------------------------------------------------------------
Total                                      76 kB/s | 276 kB     00:03     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Installing  : cowsay-3.03-19.fc23.noarch                            1/3 
  Installing  : powertop-2.8-1.fc23.x86_64                            2/3 
  Installing  : sl-5.02-2.fc23.x86_64                                 3/3 
  Verifying   : sl-5.02-2.fc23.x86_64                                 1/3 
  Verifying   : powertop-2.8-1.fc23.x86_64                            2/3 
  Verifying   : cowsay-3.03-19.fc23.noarch                            3/3 

Installed:
  cowsay.noarch 3.03-19.fc23          powertop.x86_64 2.8-1.fc23         
  sl.x86_64 5.02-2.fc23              

Complete!

real	0m12.801s
user	0m5.347s
sys	0m1.089s

#
#	output over time during a puppet run (filtered)
#

$ ps auxww | grep dnf
root     12118 27.0  1.4 417060 110864 ?       Ds   21:57   0:03 /usr/bin/python3 /usr/bin/dnf -d 0 -e 0 -y install powertop
$ ps auxww | grep dnf
root     12713 32.7  2.0 475204 159840 ?       Rs   21:57   0:02 /usr/bin/python3 /usr/bin/dnf -d 0 -e 0 -y install sl
$ ps auxww | grep dnf
root     13126  0.0  0.7 275324 55608 ?        Rs   21:57   0:00 /usr/bin/python3 /usr/bin/dnf -d 0 -e 0 -y install cowsay

#
#	pkg.pp
#

package { 'powertop':
	ensure => present,
}

#
#	pkg3.pp
#

package { ['powertop', 'sl', 'cowsay']:
	ensure => present,
}

#
#	pkg1.yaml
#

---
graph: mygraph
resources:
  pkg:
  - name: powertop
    state: installed
edges: []

#
#	autogroup2.yaml
#

---
graph: mygraph
resources:
  pkg:
  - name: powertop
    meta:
      autogroup: true
    state: installed
  - name: sl
    meta:
      autogroup: true
    state: installed
  - name: cowsay
    meta:
      autogroup: true
    state: installed
edges: []

#
#	data
#

               | powertop check            | powertop install             | powertop,sl,cowsay check  | powertop,sl,cowsay install   |
|--------------+---------------------------+------------------------------+---------------------------+------------------------------|
| Tool / Try   |   1 |   2 |   3 | average |    1 |    2 |    3 | average |   1 |   2 |   3 | average |    1 |    2 |    3 | average |
|--------------+-----+-----+-----+---------+------+------+------+---------+-----+-----+-----+---------+------+------+------+---------|
| puppet 4.2.1 | 8.4 | 8.5 | 8.8 |     8.6 | 18.2 | 18.5 | 19.4 |    18.7 | 8.7 | 8.2 | 8.5 |     8.5 |   40 | 45.3 | 41.2 |    42.2 |
| mgmt 0.0.3   | 0.3 | 0.4 | 0.3 |     0.3 | 12.7 | 13.3 | 13.1 |    13.0 | 0.7 | 0.6 | 0.7 |     0.7 | 15.8 | 15.6 | 21.5 |    17.6 |
| pkcon 1.0.11 | 0.2 | 0.2 | 0.2 |     0.2 |    9 |  8.7 |  7.2 |     8.3 | N/A | N/A | N/A |     N/A | 15.7 | 14.7 | 16.3 |    15.6 |
| dnf 1.1.7    |   3 | 2.9 |   3 |     3.0 | 10.8 | 10.4 | 12.1 |    11.1 | 2.8 | 2.9 | 2.9 |     2.9 | 12.3 | 12.8 | 14.1 |    13.1 |

NOTE: The N/A values are because pkcon doesn't check and install all three
packages if one is already present, therefore the results there are erroneous!

#
#	EOF
#