REX-Ray+CSI: Once more unto the breach

README.md

REX-Ray+CSI: Once more unto the breach

This document outlines the roadmap for supporting CSI with libStorage and REX-Ray:

Release	Description
R1	Initial CSI compatibility
R2	CNCF-ready
R3	Reference CSI implementation

R1 - Initial CSI compatibility

The first release will support both CSI architectures: centralized and decentralized, as well as the hybrid model necessary to satisfy Mesos. Many thanks to @codenrhoden for providing the illustrations!

Arch	Diagram
Centralized
Decentralized
Hybrid

Currently the only analogue to the CSI centralized model is K8s where there is a CO master. Docker and Mesos both function in a decentralized model with the libStorage API controller providing optional centralization.

Work	Summary	Story Points
#1	New REX-Ray module for CSI Ident+Node+Controller Services	21
		21

R2 - CNCF-ready

The second release will represent a project ready to donate to the CNCF as the default implementation of CSI. The primary feature of this release will be support for out-of-tree drivers via Go plug-ins, making it trivial to deliver CSI compatibility for a given storage platform via REX-Ray.

Work	Summary	Story Points
#1	Convert existing drivers into plug-ins	40
#2	Deprecate libStorage executor	21
#3	Update build process to account for plug-ins	21
		82

R3 - Reference implementation

Rounding out the roadmap is R3, the release that transforms REX-Ray from a program that embeds the libStorage API to the reference implementation of CSI. This release introduces support for distributed configuration via one or more well-known providers such as Etcd or abstraction libraries like Docker's libkv.

The introduction of distributed configuration support will sunset the libStorage API and complete the transformation of REX-Ray into a reference implementation of the CSI specification.

Work	Summary	Story Points
#1	Distributed configuration support	40
#2	Relocate CSI endpoint from REX-Ray to libStorage	20
		60

csi+docker.puml

@startuml

database storage [
    Backend
    Storage
    <U+0020><U+0020><U+0020><U+0020>API
]

rectangle "Docker Node 1" as dnode1 {
    frame "Docker" as dnode1d {
        component "dockerd" as dnode1dd
    }
    frame "REX-Ray" as dnode1rr {
        component dnode1csi [
            **CSI Node+Controller**
            ---
            libStorage Client
        ]
        component "libS Server" as dnode1lss
    }
    dnode1dd <-> dnode1csi: gRPC
    dnode1csi <-> dnode1lss: HTTP(s)\nJSON
}

rectangle "Docker Node 2" as dnode2 {
    frame "Docker" as dnode2d {
        component "dockerd" as dnode2dd
    }
    frame "REX-Ray" as dnode2rr {
        component dnode2csi [
            **CSI Node+Controller**
            ---
            libStorage Client
        ]
        component "libS Server" as dnode2lss
    }
    dnode2dd <-> dnode2csi: gRPC
    dnode2csi <-> dnode2lss: HTTP(s)\nJSON
}

storage <-> dnode1lss
storage <-> dnode2lss

@enduml

csi+docker.svg

csi+k8s.puml

@startuml

database storage [
    Backend
    Storage
    <U+0020><U+0020><U+0020><U+0020>API
]

rectangle "k8s Master" as knode1 {
    frame "k8s" as knode1k {
        component "kube-controller-manager" as knode1kd
    }
    frame "REX-Ray" as knode1rr {
        component knode1csi [
            **CSI Controller**
            ---
            libStorage Client
        ]
        component "libS Server" as knode1lss
    }
    knode1kd <-> knode1csi: gRPC
    knode1csi <-> knode1lss: HTTP(s)\nJSON
}

rectangle "k8s Minion" as knode2 {
    frame "k8s" as knode2k {
        component "kubelet" as knode2kd
    }
    frame "REX-Ray" as knode2rr {
        component knode2csi [
            **CSI Node**
            ---
            libStorage Client
        ]
    }
    knode2kd <-> knode2csi: gRPC
    knode2csi <-> knode1lss: HTTP(s)\nJSON
}

knode1lss -> storage
knode1 -[hidden]> storage
storage -[hidden]> knode2
storage -[hidden]> knode2csi

@enduml

csi+k8s.svg

csi+mesos.puml

@startuml

database storage [
    Backend
    Storage
    <U+0020><U+0020><U+0020><U+0020>API
]

rectangle "Mesos Node 1" as mnode1 {
    frame "Mesos" as mnode1m {
        component "mesosd" as mnode1md
    }
    frame "REX-Ray" as mnode1rr {
        component mnode1csi [
            **CSI Node+Controller**
            ---
            libStorage Client
        ]
        component "libS Server" as mnode1lss
    }
    mnode1md <-> mnode1csi: gRPC
    mnode1csi <-> mnode1lss: HTTP(s)\nJSON
}

rectangle "Mesos Node 2" as mnode2 {
    frame "Mesos" as mnode2m {
        component "mesosd" as mnode2md
    }
    frame "REX-Ray" as mnode2rr {
        component mnode2csi [
            **CSI Node+Controller**
            ---
            libStorage Client
        ]
    }
    mnode2md <-> mnode2csi: gRPC
    mnode2csi <-> mnode1lss: HTTP(s)\nJSON
}

mnode1lss -> storage
mnode1 -[hidden]> storage
storage -[hidden]> mnode2
storage -[hidden]> mnode2csi

@enduml

csi+mesos.svg