REX-Ray+CSI

README.md

REX-Ray+CSI

The introduction of the Container Storage Interface (CSI) provides an exciting opportunity to the world of Container Orchestrator (CO) and storage. This document reviews how CSI adds value to REX-Ray, strengthening its position as the leading CO storage management solution.

Note: Before reading this document please take time to review the several REX-Ray/CSI architecture and component types.

The Proposal

This section illustrates the proposed solution for introducing CSI to REX-Ray and libStorage:

1. Create a new libStorage interface, EndpointProvider:

type EndpointProvider interface {
	// Close immediately closes all active net.Listeners and any
	// connections in state StateNew, StateActive, or StateIdle. For a
	// graceful shutdown, use Shutdown.
	Close() error

	// Serve accepts incoming connections on the Listener l, creating a
	// new service goroutine for each.
	Serve(l net.Listener) error

	// Shutdown gracefully shuts down the server without interrupting any
	// active connections. Shutdown works by first closing all open
	// listeners, then closing all idle connections, and then waiting
	// indefinitely for connections to return to idle and then shut down.
	// If the provided context expires before the shutdown is complete,
	// then the context's error is returned.
	Shutdown(ctx context.Context) error
}

2. Refactor the libStorage API (HTTP/REST) components into an implementation of the EndpointProvider interface.
3. Relocate the Docker Volume Driver endpoint from REX-Ray to libStorage and then refactor the endpoint provider as an implementation of the EndpointProvider interface.
4. Create CSI implementations of the EndpointProvider interface:
   • csi.node
   • csi.controller
   • csi.node+controller
5. Introduce the following configuration enhancements:
   • Use libstorage.server.endpoints.NAME.type to specify the endpoint's provider type
   • Use libstorage.server.endpoints.NAME.services to define which configured storage service(s) an endpoint will expose. Please note that if an endpoint provider is not capable of consuming more than one service the property can be defined as a string, otherwise it would be []string.

Diagrams

This section visualizes the proposed changes as two diagrams that represent a centralized architecture with Docker as the CO, without and with CSI support.

Centralized Architecture w Docker

The first diagram illustrates the centralized architecture as it is today using Docker as the CO:

Centralized Architecture w Docker & CSI

The second diagram also outlines the centralized architecture, but this time includes the model as it would appear with the changes required to support CSI described above:

Endpoint Providers

This section lists the supported endpoint provider types:

Type	Multi-Service Support	Description
libstorage	✓	The libStorage API (HTTP/REST) endpoint provider
docker		The Docker Volume Driver (HTTP/REST) endpoint provider
csi		The CSI node+controller (gRPC) endpoint provider
csi.node		The CSI node (gRPC) endpoint provider
csi.controller		The CSI controller (gRPC) endpoint provider

Estimates

This section estimates the story points for the issues above:

Issue	Summary	Story Points
#1	EndpointProvider interface	8
#2	Refactor libStorage w EndpointProvider interface	21
#3	Relocate/Refactor Docker Vol Driver	13
#4	Create CSI endpoint providers	21
#4	Enhance libStorage configuration	13
		76

docker-rr-cz.puml

@startuml
frame "Master Host" as masterHost {
    frame "REX-Ray\nController" as masterControllerProc {
        rectangle  "libStorage\nPackage" as masterLibstoragePkg {
            rectangle "libStorage\nController" as masterControllerLibStorageEndpoint
        }
        rectangle masterController [
            **libStorage Package**
            ==
            libStorage Controller
        ]
    }
}
frame "Node Host(s)" as nodeHost {
    rectangle "Docker" as nodeClient
    frame "REX-Ray\nAgent" as nodeAgentProc {
        rectangle nodeAgentEndpoint [
            **REX-Ray Package**
            ==
            Docker Volume
            Driver Endpoint
        ]
        rectangle masterControllerClient [
            **libStorage Package**
            ==
            libStorage API
            Client
        ]
    }
    nodeClient -> nodeAgentEndpoint: "Docker Volume\nDriver API (HTTP/REST)"
    nodeAgentEndpoint -> masterControllerClient
    masterHost -[hidden]d-> nodeHost
}
masterControllerClient -u-> masterControllerLibStorageEndpoint: "libStorage API\n(HTTP/REST)"
@enduml

docker-rr-cz.svg

docker-rr.csi-cz.puml

@startuml
frame "Master Host" as masterHost {
    rectangle "Docker" as masterClient
    frame "REX-Ray\nController" as masterControllerProc {
        rectangle  "libStorage\nPackage" as masterLibstoragePkg {
            rectangle "CSI\nController\nService" as masterControllerCSIEndpoint
            rectangle "libStorage\nController" as masterControllerLibStorageEndpoint
        }
    }
    masterClient -> masterControllerCSIEndpoint: "CSI\n(gRPC)"
}
frame "Node Host(s)" as nodeHost {
    rectangle "Docker" as nodeClient
    frame "REX-Ray\nAgent" as nodeAgentProc {
        rectangle "libStorage\nPackage" as nodeLibstoragePkg {
            rectangle "CSI\nNode\nService" as nodeAgentEndpoint
            rectangle "libStorage\nAPI\nClient" as masterControllerClient
        }
    }
    nodeClient -> nodeAgentEndpoint: "CSI\n(gRPC)"
    nodeAgentEndpoint -> masterControllerClient
}
masterControllerClient -u-> masterControllerLibStorageEndpoint: "libStorage API\n(HTTP/REST)"
@enduml

docker-rr.csi-cz.svg