Skip to content

Instantly share code, notes, and snippets.

@vladak

vladak/ips-mediator-hell.md

Last active August 24, 2021 08:01
Show Gist options
  • Save vladak/805caeb0a376801402985d59efe9983e to your computer and use it in GitHub Desktop.
Save vladak/805caeb0a376801402985d59efe9983e to your computer and use it in GitHub Desktop.
Download ZIP
IPS mediators - mixing version and implementation
Raw
ips-mediator-hell.md

Question: is mixing mediator-version with mediator-implementation in Solaris IPS a good idea ?

TLDR answer: no, generally it's not

Case 1

Suppose we have 2 packages that mediate usr/include/moomin (also they deliver symlinks in lib which is not really mediated since the link names differ), the foo package mediates using the mediator-version, the bar package mediates using mediator-implementation:

uls-0:/builds/vkotal/ips$ grep -v ^# foo.p5m 
set name=pkg.fmri [email protected],5.11-0
set name=pkg.summary value="This is foo"
set name=pkg.description value="foo foo foo"
set name=info.classification \
    value=org.opensolaris.category.2008:Applications/Accessories
link path=lib/foo.so.1.0 target=libc.so.1 mediator=moomin mediator-version=1.1
dir  path=usr/foo owner=root group=bin mode=0755
dir  path=usr/foo/include owner=root group=bin mode=0755
file usr/foo/include/foo.h path=usr/foo/include/foo.h owner=root group=bin \
    mode=0644
link path=usr/include/moomin target=../foo/include mediator=moomin \
    mediator-version=1.1
    
uls-0:/builds/vkotal/ips$ grep -v ^# bar.p5m 
set name=pkg.fmri [email protected],5.11-0
set name=pkg.summary value="This is bar"
set name=pkg.description value="bar bar"
set name=info.classification \
    value=org.opensolaris.category.2008:Applications/Accessories
link path=lib/bar.so.1.0 target=libc.so.1 mediator=moomin \
    mediator-implementation=bar
dir  path=usr/bar owner=root group=bin mode=0755
dir  path=usr/bar/include owner=root group=bin mode=0755
file usr/bar/include/bar.h path=usr/bar/include/bar.h owner=root group=bin \
    mode=0644
link path=usr/include/moomin target=../bar/include mediator=moomin \
    mediator-implementation=bar

So let's build the packages:

uls-0:/builds/vkotal/ips$ make
pkgrepo create my-repository
pkgrepo -s my-repository set publisher/prefix=mypublisher
pkgfmt foo.p5m
pkgsend -s my-repository publish -d proto/foo foo.p5m
pkg://mypublisher/[email protected],5.11-0:20191017T124523Z
PUBLISHED
pkgfmt bar.p5m
pkgsend -s my-repository publish -d proto/bar bar.p5m
pkg://mypublisher/[email protected],5.11-0:20191017T124525Z
PUBLISHED
pkgrepo verify -s my-repository
Initiating repository verification.
pkgrepo list -s my-repository
PUBLISHER   NAME                                          O VERSION
mypublisher bar                                             1.0-0:20191017T124525Z
mypublisher foo                                             1.0-0:20191017T124523Z

and install them (on a virtual machine):

root@admin-ST056:~#    pkg set-publisher -g /net/uls-0/builds/vkotal/ips/my-repository mypublisher
root@admin-ST056:~#    pkg list -avf 'pkg://mypublisher/*'
FMRI                                                                         IFO
pkg://mypublisher/[email protected]:20191017T124525Z                                 ---
pkg://mypublisher/[email protected]:20191017T124523Z                                 ---
root@admin-ST056:~#    pkg install --no-backup-be --no-index pkg://mypublisher/foo pkg://mypublisher/bar
           Packages to install:  2
           Mediators to change:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                                2/2           2/2      0.0/0.0      --

PHASE                                          ITEMS
Installing new actions                         16/16
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Updating package cache                           2/2 
root@admin-ST056:~#    pkg contents -m foo
set name=pkg.fmri value=pkg://mypublisher/[email protected],5.11-0:20191017T124523Z
set name=pkg.summary value="This is foo"
set name=pkg.description value="foo foo foo"
set name=info.classification value=org.opensolaris.category.2008:Applications/Accessories
link mediator=moomin mediator-version=1.1 path=lib/foo.so.1.0 target=libc.so.1
dir group=bin mode=0755 owner=root path=usr/foo
dir group=bin mode=0755 owner=root path=usr/foo/include
file 396356402134b87f36e746effc384f082700b78c chash=a1294b0e7bd0ba0f47af6e5e12094fdf8c40c2da group=bin mode=0644 owner=root path=usr/foo/include/foo.h pkg.content-hash=file:sha512t_256:274efb49e8763557cc8d993b35d812a19de84345d6d3c93c115174698a3a922f pkg.content-hash=gzip:sha512t_256:91ee95424ac2e997e88ac168b44fa1434aee3d7d034d36a6f86a4436e784f54e pkg.csize=43 pkg.size=23
link mediator=moomin mediator-version=1.1 path=usr/include/moomin target=../foo/include
root@admin-ST056:~#    pkg contents -m bar
set name=pkg.fmri value=pkg://mypublisher/[email protected],5.11-0:20191017T124525Z
set name=pkg.summary value="This is bar"
set name=pkg.description value="bar bar"
set name=info.classification value=org.opensolaris.category.2008:Applications/Accessories
link mediator=moomin mediator-implementation=bar path=lib/bar.so.1.0 target=libc.so.1
dir group=bin mode=0755 owner=root path=usr/bar
dir group=bin mode=0755 owner=root path=usr/bar/include
file 396356402134b87f36e746effc384f082700b78c chash=a1294b0e7bd0ba0f47af6e5e12094fdf8c40c2da group=bin mode=0644 owner=root path=usr/bar/include/bar.h pkg.content-hash=file:sha512t_256:274efb49e8763557cc8d993b35d812a19de84345d6d3c93c115174698a3a922f pkg.content-hash=gzip:sha512t_256:91ee95424ac2e997e88ac168b44fa1434aee3d7d034d36a6f86a4436e784f54e pkg.csize=43 pkg.size=23
link mediator=moomin mediator-implementation=bar path=usr/include/moomin target=../bar/include

Check all is well:

root@admin-ST056:~#    pkg mediator moomin
MEDIATOR      VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
moomin        system    1.1     system     
root@admin-ST056:~#    ls /usr/{foo,bar}
/usr/bar:
include

/usr/foo:
include
root@admin-ST056:~#    ls -ald /usr/include/moomin
lrwxrwxrwx   1 root     root          14 Oct 17 13:08 /usr/include/moomin -> ../foo/include

Evidently, the foo got precedence. Now change the mediator implementation only:

root@admin-ST056:~#    pkg set-mediator -vvv --no-backup-be -I bar moomin
            Packages to change:         3
           Mediators to change:         1
     Estimated space available:  51.52 GB
Estimated space to be consumed: 649.75 MB
       Create boot environment:        No
Create backup boot environment:        No
          Rebuild boot archive:        No

Changed mediators:
  mediator moomin:
           version: 1.1 (system default) -> None
    implementation: None -> bar (local default)

Changed packages:
mypublisher
  bar
    1.0-0
  foo
    1.0-0
  foo
    1.0-0

Actions:
  link mediator=moomin mediator-version=1.1 path=lib/foo.so.1.0 target=libc.so.1 -> None
  link mediator=moomin mediator-version=1.1 path=lib/foo.so.1.0 target=libc.so.1 -> None
  None -> link mediator=moomin mediator-implementation=bar path=usr/include/moomin target=../bar/include
  None -> link mediator=moomin mediator-implementation=bar path=lib/bar.so.1.0 target=libc.so.1
PHASE                                          ITEMS
Removing old actions                             2/2
Updating modified actions                        2/2
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Updating package cache                           2/2 
root@admin-ST056:~#    pkg mediator moomin
MEDIATOR      VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
moomin        system            local      bar
root@admin-ST056:~# ls -ald /usr/include/moomin
lrwxrwxrwx   1 root     root          14 Oct 17 13:16 /usr/include/moomin -> ../bar/include

All looks good. Now change version only:

root@admin-ST056:~# pkg set-mediator -vvv --no-backup-be -V 1.1 moomin
            Packages to change:         2
           Mediators to change:         1
     Estimated space available:  51.52 GB
Estimated space to be consumed: 649.75 MB
       Create boot environment:        No
Create backup boot environment:        No
          Rebuild boot archive:        No

Changed mediators:
  mediator moomin:
           version: None -> 1.1 (local default)
    implementation: bar (local default) -> bar (local default)

Changed packages:
mypublisher
  bar
    1.0-0
  bar
    1.0-0

Actions:
  link mediator=moomin mediator-implementation=bar path=lib/bar.so.1.0 target=libc.so.1 -> None
  link mediator=moomin mediator-implementation=bar path=usr/include/moomin target=../bar/include -> None
  link mediator=moomin mediator-implementation=bar path=usr/include/moomin target=../bar/include -> None
  link mediator=moomin mediator-implementation=bar path=lib/bar.so.1.0 target=libc.so.1 -> None
PHASE                                          ITEMS
Removing old actions                             4/4
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Updating package cache                           2/2

We got invalid combination (or at least what we think is invalid since no package delivers the link with implementation=bar and version=1.1):

root@admin-ST056:~# pkg mediator moomin
MEDIATOR      VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
moomin        local     1.1     local      bar

and the symlink just disappeared:

root@admin-ST056:~# ls -ald /usr/include/moomin
/usr/include/moomin: No such file or directory

Case 2

Now let's modify the packages to use both implementation and version where there is no overlap:

uls-0:/builds/vkotal/ips$ make clean && make
rm -rf my-repository
pkgrepo create my-repository
pkgrepo -s my-repository set publisher/prefix=mypublisher
pkgfmt foo.p5m
pkgsend -s my-repository publish -d proto/foo foo.p5m
pkg://mypublisher/[email protected],5.11-0:20191017T134656Z
PUBLISHED
pkgfmt bar.p5m
pkgsend -s my-repository publish -d proto/bar bar.p5m
pkg://mypublisher/[email protected],5.11-0:20191017T134658Z
PUBLISHED
pkgrepo verify -s my-repository
Initiating repository verification.
pkgrepo list -s my-repository
PUBLISHER   NAME                                          O VERSION
mypublisher bar                                             1.0-0:20191017T134658Z
mypublisher foo                                             1.0-0:20191017T134656Z

and install them:

root@admin-ST056:~#    pkg set-publisher -g /net/uls-0/builds/vkotal/ips/my-repository mypublisher
root@admin-ST056:~#    pkg list -avf 'pkg://mypublisher/*'
FMRI                                                                         IFO
pkg://mypublisher/[email protected]:20191017T134658Z                                 ---
pkg://mypublisher/[email protected]:20191017T134656Z                                 ---

and verify:

root@admin-ST056:~#    pkg install --no-backup-be --no-index pkg://mypublisher/foo pkg://mypublisher/bar
           Packages to install:  2
           Mediators to change:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                                2/2           2/2      0.0/0.0      --

PHASE                                          ITEMS
Installing new actions                         16/16
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Updating package cache                           2/2 
root@admin-ST056:~#    pkg contents -m foo
set name=pkg.fmri value=pkg://mypublisher/[email protected],5.11-0:20191017T134656Z
set name=pkg.summary value="This is foo"
set name=pkg.description value="foo foo foo"
set name=info.classification value=org.opensolaris.category.2008:Applications/Accessories
link mediator=moomin mediator-implementation=foo mediator-version=1.1 path=lib/foo.so.1.0 target=libc.so.1
dir group=bin mode=0755 owner=root path=usr/foo
dir group=bin mode=0755 owner=root path=usr/foo/include
file 396356402134b87f36e746effc384f082700b78c chash=a1294b0e7bd0ba0f47af6e5e12094fdf8c40c2da group=bin mode=0644 owner=root path=usr/foo/include/foo.h pkg.content-hash=file:sha512t_256:274efb49e8763557cc8d993b35d812a19de84345d6d3c93c115174698a3a922f pkg.content-hash=gzip:sha512t_256:91ee95424ac2e997e88ac168b44fa1434aee3d7d034d36a6f86a4436e784f54e pkg.csize=43 pkg.size=23
link mediator=moomin mediator-implementation=foo mediator-version=1.1 path=usr/include/moomin target=../foo/include
root@admin-ST056:~#    pkg contents -m bar
set name=pkg.fmri value=pkg://mypublisher/[email protected],5.11-0:20191017T134658Z
set name=pkg.summary value="This is bar"
set name=pkg.description value="bar bar"
set name=info.classification value=org.opensolaris.category.2008:Applications/Accessories
link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=lib/bar.so.1.0 target=libc.so.1
dir group=bin mode=0755 owner=root path=usr/bar
dir group=bin mode=0755 owner=root path=usr/bar/include
file 396356402134b87f36e746effc384f082700b78c chash=a1294b0e7bd0ba0f47af6e5e12094fdf8c40c2da group=bin mode=0644 owner=root path=usr/bar/include/bar.h pkg.content-hash=file:sha512t_256:274efb49e8763557cc8d993b35d812a19de84345d6d3c93c115174698a3a922f pkg.content-hash=gzip:sha512t_256:91ee95424ac2e997e88ac168b44fa1434aee3d7d034d36a6f86a4436e784f54e pkg.csize=43 pkg.size=23
link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=usr/include/moomin target=../bar/include
root@admin-ST056:~#    pkg mediator moomin
MEDIATOR      VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
moomin        system    1.1     system     foo
root@admin-ST056:~#    ls -ald /usr/include/moomin
lrwxrwxrwx   1 root     root          14 Oct 17 14:13 /usr/include/moomin -> ../foo/include

now switch using implementation only:

root@admin-ST056:~# pkg set-mediator -vvv --no-backup-be -I bar moomin
            Packages to change:         3
           Mediators to change:         1
     Estimated space available:  51.52 GB
Estimated space to be consumed: 649.75 MB
       Create boot environment:        No
Create backup boot environment:        No
          Rebuild boot archive:        No

Changed mediators:
  mediator moomin:
           version: 1.1 (system default) -> 1.0 (system default)
    implementation: foo (system default) -> bar (local default)

Changed packages:
mypublisher
  bar
    1.0-0
  foo
    1.0-0
  foo
    1.0-0

Actions:
  link mediator=moomin mediator-implementation=foo mediator-version=1.1 path=lib/foo.so.1.0 target=libc.so.1 -> None
  link mediator=moomin mediator-implementation=foo mediator-version=1.1 path=lib/foo.so.1.0 target=libc.so.1 -> None
  None -> link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=usr/include/moomin target=../bar/include
  None -> link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=lib/bar.so.1.0 target=libc.so.1
PHASE                                          ITEMS
Removing old actions                             2/2
Updating modified actions                        2/2
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Updating package cache                           2/2

That looks sensible:

root@admin-ST056:~# pkg mediator moomin
MEDIATOR      VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
moomin        system    1.0     local      bar

so now switch version only:

root@admin-ST056:~# pkg set-mediator -vvv --no-backup-be -V 1.1 moomin
            Packages to change:         2
           Mediators to change:         1
     Estimated space available:  51.52 GB
Estimated space to be consumed: 649.75 MB
       Create boot environment:        No
Create backup boot environment:        No
          Rebuild boot archive:        No

Changed mediators:
  mediator moomin:
           version: 1.0 (system default) -> 1.1 (local default)
    implementation: bar (local default) -> bar (local default)

Changed packages:
mypublisher
  bar
    1.0-0
  bar
    1.0-0

Actions:
  link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=lib/bar.so.1.0 target=libc.so.1 -> None
  link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=usr/include/moomin target=../bar/include -> None
  link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=lib/bar.so.1.0 target=libc.so.1 -> None
  link mediator=moomin mediator-implementation=bar mediator-version=1.0 path=usr/include/moomin target=../bar/include -> None
PHASE                                          ITEMS
Removing old actions                             4/4
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Updating package cache                           2/2

Similar thing happened as with Case 1:

root@admin-ST056:~# pkg mediator moomin
MEDIATOR      VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
moomin        local     1.1     local      bar
root@admin-ST056:~# ls -ald /usr/include/moomin
/usr/include/moomin: No such file or directory
@orcl-jlana
Copy link

you do not deliver symlink /usr/include/moomin with implementation=bar and version=1.1 in any package therefore the result looks correct to me.

BTW version could be part of mediator-implementation by append @version, i.e you can use [email protected] - additional level of confusion ;-)

@vladak
Copy link
Author

vladak commented Aug 23, 2021 •
edited
Loading

you do not deliver symlink /usr/include/moomin with implementation=bar and version=1.1 in any package therefore the result looks correct to me.

Yep, the gist contains this note:

We got invalid combination (or at least what we think is invalid since no package delivers the link with implementation=bar and version=1.1):

the point being that this is not fool proof. There is nothing in IPS stopping you from entering an invalid state and the combination of the mediator types actually makes this worse. The problem is not that this is not implemented according to the specification, the problem is that the combination makes working with the mediators even more fragile. Not only it is necessary to make sure there is not any typo in the values, one has to pay close attention to how exactly the values are switched.

This is really the classical case of:

  • A: "I ran this command and it broke my system."
  • B: "Of course ! Everyone knows that this is how the command should not be run. It is implemented according to the specification and therefore 100% correct."

BTW version could be part of mediator-implementation by append @version, i.e you can use [email protected] - additional level of confusion ;-)

This stems from the fact that mediator-implementation and mediator-version do not work well together - both as an idea and implementation. Having the version embedded in the value of the implementation field is rather a workaround used by some packages. One could argue that the approach of embedding the version in mediator-implementation value is more robust because one has to think solely of the value of one mediator field, not values of both mediator fields plus making sure both fields are actually specified.

@orcl-jlana
Copy link

I agree that set-mediator is a fragile command.

@vladak
Copy link
Author

vladak commented Aug 24, 2021

This is not just about single command. In order to make the work with mediators more robust, an inordinate amount of checking and logic would have to be put into the implementation of the set-mediator command (and possibly other commands, like package publishing and repo construction). My take on this is that mediators should have been made first class objects in IPS and no amount of fixing of the current design will help prevent users shooting themselves into the foot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment