ROOTLESS

Organize user workspace in place of / Should look similar to:

├── run
│   └── kata-containers
├── usr
│   └── share
│       └── kata-containers
│           ├── kata-containers-image_clearlinux_1.6.0-rc2_agent_a41f322af86.img
│           ├── kata-containers.img
│           ├── kata-containers-initrd_alpine_1.6.0-rc2_agent_a41f322af86.initrd
│           ├── kata-containers-initrd.img
│           ├── vmlinux-4.19.24.31-21.1.container
│           ├── vmlinux.container
│           ├── vmlinuz-4.19.24.31-21.1.container
│           └── vmlinuz.container
└── var
    ├── lib
    └── run
        └── kata-containers

Error: container create failed: mkdir /var/run/netns: permission denied
this error is occuring because the networking namespace being created is in a location that the non-root, slirp4netns interface podman uses for rootless networking, cannot access. This needs to be in a location accessible to the user

modify configuration.toml
- ~~uncomment disable_new_netns = true~~
- ~~set internetworking_model="none"~~
- ~~run podman with --net none~~
- uncomment disable_vhost_net = true

Modify the containernetworking/plugins/pkg/testutils/netns_linux.go file, so that the creation of the new netns is to a /tmp dir that can be accessed by the user

//const nsRunDir = "/var/run/netns"
const nsRunDir = "/tmp/katapod/var/run/netns"

build that subpackage, and import it into kata-runtime/katautil/network for creating a new namespace Need a way to configure the path (go between root and non root) note: the ns package from containernetworking is still required for getting the currentNS

instead of calling NewNS from the ns package, call it from the testutils package

if config.NetNSPath == "" {
            n, err := testutils.NewNS()
            if err != nil {
                    return err
            }
            config.NetNSPath = n.Path()
            config.NetNsCreated = true

            return nil
}

Make file paths in kata vc configurable, ~~or change permissions of /var/ to be non root~~
- modified makefile to point defaultRootDir to be /tmp/katapod/
- modified any /var or /run to be in /tmp/katapod (see note above)
- virtcontainers/store/filesystem_backend.go
- virtcontinaers/kata-agent.go
- cli/main.go
- pkg/katautils/config-settings.go
- pkg/katautils/oci.go

Update configuration.toml so that the hypervisor kernel and image are pointed to a user owned location

[gnbeyer@podman-rootless runtime]$ podman run -it --net none --runtime /usr/local/bin/kata-runtime alpine uname
Error: container create failed: qemu-lite-system-x86_64: -object memory-backend-file,id=mem0,mem-path=/usr/share/kata-containers/kata-containers-image_clearlinux_1.6.0-rc2_agent_a41f322af86.img,size=134217728: can't open backing store /usr/share/kata-containers/kata-containers-image_clearlinux_1.6.0-rc2_agent_a41f322af86.img for guest RAM: Permission denied                                                

: internal libpod error

# for example:
[hypervisor.qemu]
path = "/usr/bin/qemu-lite-system-x86_64"
kernel = "/tmp/katapod/usr/share/kata-containers/vmlinuz.container"
image = "/tmp/katapod/usr/share/kata-containers/kata-containers.img"
machine_type = "pc"

rootfs mount type:

WARN[0000] Failed to add conmon to cgroupfs sandbox cgroup: mkdir /sys/fs/cgroup/systemd/libpod_parent: permission denied 
DEBU[0001] Received container pid: -1                   
DEBU[0001] Cleaning up container 3eda4b5173336d6cd8499dd40f2422cd89b75d99deb09cbbf679b5b1b124a793 
DEBU[0001] Network is already cleaned up, skipping...   
DEBU[0001] unmounted container "3eda4b5173336d6cd8499dd40f2422cd89b75d99deb09cbbf679b5b1b124a793" 
DEBU[0001] [graphdriver] trying provided driver "vfs"   
ERRO[0001] container create failed: rpc error: code = Unknown desc = mkdir /home/gnbeyer: read-only file system

journalctl

Apr 03 20:30:10 podman-rootless kata-runtime[11373]: time="2019-04-03T20:30:10.26476542Z" level=info msg="proxy started" arch=amd64 command=create container=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 name=kata-runtime pid=11373 proxy-pid=11393 proxy-url="unix:///home/gnbeyer/sd/run/vc/sbs/60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201/proxy.sock" sandbox=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 source=virtcontainers subsystem=kata_agent
Apr 03 20:30:10 podman-rootless kata-runtime[11373]: time="2019-04-03T20:30:10.264803758Z" level=info msg="New client" arch=amd64 command=create container=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 name=kata-runtime pid=11373 source=virtcontainers subsystem=kata_agent url="unix:///home/gnbeyer/sd/run/vc/sbs/60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201/proxy.sock"
Apr 03 20:30:11 podman-rootless kata-runtime[11373]: time="2019-04-03T20:30:11.566555256Z" level=info msg="New client" arch=amd64 command=create container=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 name=kata-runtime pid=11373 source=virtcontainers subsystem=kata_agent url="unix:///home/gnbeyer/sd/run/vc/sbs/60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201/proxy.sock"
Apr 03 20:30:11 podman-rootless kata-runtime[11373]: time="2019-04-03T20:30:11.596863983Z" level=info msg="cleanup agent" arch=amd64 command=create container=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 name=kata-runtime path=/home/gnbeyer/sd/kata-containers/shared/sandboxes/60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 pid=11373 source=virtcontainers subsystem=kata_agent
Apr 03 20:30:11 podman-rootless kata-runtime[11373]: time="2019-04-03T20:30:11.597472136Z" level=error msg="rpc error: code = Unknown desc = mkdir /home/gnbeyer: read-only file system" arch=amd64 command=create container=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 name=kata-runtime pid=11373 source=runtime
Apr 03 20:30:11 podman-rootless kata-proxy[11393]: time="2019-04-03T20:30:11.615621466Z" level=fatal msg="channel error" error="accept unix /home/gnbeyer/sd/run/vc/sbs/60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201/proxy.sock: use of closed network connection" name=kata-proxy pid=11393 sandbox=60fdd77febbb7ffd8b522a088249d74bfb5dfa2625c33a1569905d5c62eb7201 source=proxy

From the tracing:
/grpc.AgentService/CreateSandbox
Service:kata-runtimeDuration:19.54msStart Time:1.37s
Tags:component=gRPCspan.kind=clientresponse_code=Unknownresponse_class=0xxerror=true
Process:client-uuid=7776a2a14a7194c2hostname=podman-rootlessip=10.0.2.15jaeger.version=Go-2.14.0
Logs (1)
1.39s
event	
"error"
message	
"rpc error: code = Unknown desc = mkdir /home/gnbeyer: read-only file system"
Log timestamps are relative to the start time of the full trace.

Hack: modify the root filesystem as rw (reverting commit #6498466 change) in the qemu_amd64

-       {"rootflags", "dax,data=ordered,errors=remount-ro ro"},
+       {"rootflags", "dax,data=ordered,errors=remount-ro rw"},

kata-runtime attempting to create cgroup in /sys/fs/cgroups

time="2019-04-04T19:47:07.042291826Z" level=error msg="Could not create cgroup for /vc: mkdir /sys/fs/cgroup/systemd/vc: permission denied" arch=amd64 command=create container=0932377c9ce95b10cc9c8f49885a2ab783d0e9161cbb10f97aa8431ae8d2fdfa name=kata-runtime pid=9514 source=runtime

container method newCGroups cgroup, err := cgroupsNewFunc(cgroups.V1, cgroups.StaticPath(c.state.CgroupPath), &resources) /sys/fs/groups is passed from the oci config.json

comment that portion of code out

kataruntime/virtcontainers/containers.go
// comment this out
if err = c.newCgroups(); err != nil {
      return
}

CONTAINER CREATED

[uname@host runtime]$ podman run -it --runtime=/usr/local/bin/kata-runtime alpine sh
/ # uname -a
Linux 90f35f9fb979 4.19.24-21.1.container #1 SMP Thu Jan 1 00:00:00 UTC 1970 x86_64 Linux
/ #

gabibeyer/kata_podman_rootless.md

ROOTLESS

CONTAINER CREATED