|
|
|
[TOC] |
|
|
|
> additional important docs: [config references](https://github.com/docker/notary/tree/master/docs/reference) |
|
|
|
## 0) Setting up DB(s) |
|
1. Simply boot up two database instances (`i1` for server and `i2` for signer). |
|
2. create a database named `notaryserver` on `i1` and create `notarysigner` on `i2`. |
|
3. run SQL commands from [here](https://github.com/docker/notary/tree/master/migrations) |
|
4. later configure the database urls in the `server-config.json` and `signer-config.json` |
|
5. sit back and have a donut. |
|
|
|
## 1) Setting up signer |
|
1. `git clone https://github.com/cyc115/notary && git checkout cyc_prod_ready` |
|
3. run `signer` with `docker-compose -f docker-compose.signer.yml up --build ` |
|
6. AWS instance need to expose inbound TCP `7789` for server-signer communication and outbound database port (default is `5432`) |
|
|
|
### signer config file |
|
``` |
|
{ |
|
"server": { |
|
"grpc_addr": ":7899", |
|
"tls_cert_file": "./notary-signer.crt", |
|
"tls_key_file": "./notary-signer.key", |
|
"client_ca_file": "./notary-server.crt" |
|
}, |
|
"logging": { |
|
"level": "debug" |
|
}, |
|
"storage": { |
|
"backend": "postgres", |
|
"db_url": "postgres://user:pass@endpoint:port/notarysigner" |
|
} |
|
} |
|
``` |
|
|
|
## 2) Setting up server |
|
1. `git clone https://github.com/cyc115/notary && git checkout cyc_prod_ready` |
|
2. build the server with `docker-compose -f ./docker-compose.server.yml up --build` |
|
3. Assign AWS in/out-bound rules to `signer` and database |
|
4. ~~set up DNS in container to use `8.8.8.8` in `/etc/resolv.conf` to make sure signer is discoverable~~ (might not be necessary) |
|
5. set up `notarysigner` in `/etc/hosts` to make sure the host name is known by server |
|
6. create tls server key and cert pair using template below |
|
7. test with notary cli client |
|
|
|
### server config file (checkout up-to-date version in repo) |
|
``` |
|
{ |
|
"server": { |
|
"http_addr": ":4443", |
|
"tls_key_file" : "./cyc/key.pem", |
|
"tls_cert_file" : "./cyc/certificate.pem" |
|
}, |
|
"trust_service": { |
|
"type": "remote", |
|
"hostname": "notarysigner", |
|
"port": "7899", |
|
"tls_ca_file": "./root-ca.crt", |
|
"key_algorithm": "ecdsa", |
|
"tls_client_cert" :"./notary-server.crt", |
|
"tls_client_key": "./notary-server.key" |
|
}, |
|
"logging": { |
|
"level": "info" |
|
}, |
|
"storage": { |
|
"backend": "postgres", |
|
"db_url": "postgres://xx:xx@xxx/notaryserver" |
|
} |
|
} |
|
``` |
|
> **note: ** in case of the following error: **`2017/06/19 22:37:13 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp: lookup notarysigner on 127.0.0.11:53: no such host"; Reconnecting to {notarysigner:7899 <nil>}`. Set `8.8.8.8` as DNS either on host's `/etc/resolv.conf`, or in the corresponding docker-compose file. |
|
|
|
## 3) set up daemon client |
|
1. checkout `cyc_api_client` branch |
|
3. config `Daemon server` TLS, `escrow`in `clientapi-server-config.toml` |
|
4. run `sudo docker-compose -f clientapi.yml up --build` to start `daemon` and `escrow` |
|
6. AWS instance need to expose TCP inbound (4449) to allow cli connection |
|
|
|
> - **NOTE:** **Keys stored by `escrow` is stored in the mounted directory `.secrete`. Key's encryption passphrase is stored as clear text in `clientapi-server-config.toml`. ** |
|
> - **IDEA** : implement a (reusable) configuration generation tool using vault as backend, this tool should take in a templated config file (json, yml, toml, etc), a secrete store service ( hasicorp vault, environment var, or a file from a mounted volume), smash them together and spit out ready to use configuration files. |
|
> - Another way is to use `docker-compose` ` [extend](https://blog.docker.com/2015/04/easily-configure-apps-for-multiple-environments-with-compose-1-2-and-much-more/) but it lacks flexibility. |
|
|
|
|
|
### clientapi-server.config |
|
``` |
|
[server] |
|
grpc_addr = ":4449" |
|
tls_cert_file = "./clientapi-server.crt" |
|
tls_key_file = "./clientapi-server.key" |
|
|
|
[logging] |
|
level = "info" |
|
|
|
[upstream] |
|
addr = "https://54.174.67.41:4443" |
|
tls_ca_file = "./clientapi-server.crt" |
|
# addr = "notary-server:4443" |
|
|
|
[key_storage] |
|
type = "remote" |
|
addr = "notaryescrow:4450" |
|
tls_ca_file = "./root-ca.crt" |
|
tls_cert_file = "./clientapi-server.crt.back" |
|
tls_key_file = "./clientapi-server.key.back" |
|
secret = "fixturesecret" |
|
``` |
|
|
|
## 3) set up notary cli on local machine |
|
1. checkout `cyc_api_client` |
|
2. run `make client` |
|
3. make sure the local `~/.notary/config.json` file contains the correct daemon address. (Example shown below under `files` header) |
|
example CLI config: |
|
``` |
|
{ |
|
"trust_dir" : "~/.docker/trust", |
|
"remote_server": { |
|
"url": "http://54.218.252.249:4443", |
|
//tls stuff |
|
} |
|
} |
|
``` |
|
|
|
## 4) set up token based authentication |
|
|
|
### 4.0) Authentication flow |
|
```sequence |
|
Daemon->NotaryServer: publish |
|
NotaryServer->Daemon: 401 unauthorized |
|
Daemon->AuthServer: authenticate |
|
AuthServer->Daemon: Authorize + JWT |
|
Daemon->NotaryServer: Publish w/ JWT |
|
NotaryServer->NotaryServer: verify JWT w/ crt |
|
NotaryServer->Daemon: 200 ok |
|
``` |
|
|
|
### 4.1) Authorization server |
|
|
|
The Authorization server provides the following via `GET` request. |
|
|
|
```javascript |
|
{ |
|
"token": "eyIn0.C1w.XXX", // the only required field |
|
"access_token": "eyIn0.C1w.XXX", |
|
"expires_in": 300, // time to expire |
|
"issued_at": "2017-07-03T16:16:59.188578836Z" |
|
} |
|
``` |
|
- **token** : An opaque Bearer token that clients should supply to subsequent requests in the Authorization header. |
|
- **access_token** : For compatibility with OAuth 2.0. When both **token** and **access_token** are specified, they should be equivalent; if they differ the client's choice is undefined. |
|
- **expires_in**: (Optional) The duration in seconds since the token was issued that it will remain valid. When omitted, this defaults to 60 seconds. For compatibility with older clients, a token should never be returned with less than 60 seconds to live. |
|
- **issued_at**: (Optional) The RFC3339-serialized UTC standard time at which a given token was issued. If issued_at is omitted, the expiration is from when the token exchange completed. |
|
- **refresh_token**: (Optional) Token which can be used to get additional access tokens for the same subject with different scopes. This token should be kept secure by the client and only sent to the authorization server which issues bearer tokens. This field will only be set when `offline_token=true` is provided in the request. |
|
|
|
|
|
### 4.2) JWT format |
|
the client would make GET request to the server with the following param: |
|
``` |
|
https://auth.docker.io/token |
|
?service=registry.docker.io |
|
&scope=repository:library/hello-world:pull,push` |
|
``` |
|
the decrypted JWT token contains a `header`, `claim-set`, and `signature` : |
|
#### header : |
|
```javascript |
|
{ |
|
"alg": "ES256", |
|
"typ": "JWT", |
|
"x5c": ["MIICL...yk="] // list of cert in trust chain |
|
} |
|
``` |
|
|
|
#### claim-set |
|
```javascript |
|
{ |
|
"access": [{ //maybe empty if unauthorized |
|
"type": "repository", |
|
"name": "library/hello-world", |
|
"actions": [ |
|
"pull" |
|
] |
|
}], |
|
"aud": "registry.docker.io", //corresponds to the service in the GET param |
|
"exp": 1497390356, |
|
"iat": 1497390056, |
|
"iss": "auth.docker.io", // iss must match |
|
"jti": "kWB3WGHvHxtO8GETn__x", |
|
"nbf": 1497389756, |
|
"sub": "" // sub must match |
|
} |
|
``` |
|
|
|
### 4.3) Notary Daemon setup |
|
|
|
Include the following in `clientapi-server-config.toml` |
|
|
|
```toml |
|
[auth] |
|
type = "token" |
|
[auth.options] |
|
realm = "https://auth.docker.io/token" // where to fetch token |
|
service = "notary.docker.io" . // request the audience field in the JWT, must match with server config |
|
issuer = "auth.docker.io" // must match with server config |
|
rootcertbundle = "./dockerauth.crt" // path relative to notary/fixtures |
|
``` |
|
|
|
|
|
### 4.4) Notary Server setup |
|
|
|
add the following to `server-config.json` |
|
```javascript |
|
"auth" : { |
|
"type": "token", |
|
"options": { |
|
"realm": "https://auth.docker.io", |
|
"service": "notary.docker.io", // the audience field in the token |
|
"issuer": "auth.docker.io", |
|
"rootcertbundle": "./fixtures/xxx.crt" . // path relative to notary |
|
} |
|
} |
|
``` |
|
|
|
| Parameter | Required | Description | |
|
|-----------|----------|-------------------------------------------------------| |
|
| `realm` | yes | The realm in which the registry server authenticates. | |
|
| `service` | yes | The service being authenticated. | |
|
| `issuer` | yes | The name of the token issuer. The issuer inserts this into the token so it must match the value configured for the issuer. | |
|
| `rootcertbundle` | yes | The absolute path to the root certificate bundle. This bundle contains the public part of the certificates used to sign authentication tokens. | |
|
|
|
## files |
|
|
|
### `~/.notary/config.json` |
|
|
|
```javascript |
|
{ |
|
"trust_dir": "~/.docker/trust", |
|
"remote_server": { |
|
"url": "https://54.174.67.41:4443", //notary-server |
|
"root_ca": "./self-sign.pem" // ca certificate used by conventional cli client to verify notary server |
|
}, |
|
// api client related |
|
"api": { |
|
"addr" : "54.174.67.41:4449", |
|
"insecure" : false, |
|
"tls_ca_file": "/Users/xjqw46/.notary/self-sign.pem" |
|
} |
|
} |
|
``` |
|
<table> |
|
<tr> |
|
<th>Parameter</th> |
|
<th>Required</th> |
|
<th>Description</th> |
|
</tr> |
|
<tr> |
|
<td valign="top"><code>url</code></td> |
|
<td valign="top">no</td> |
|
<td valign="top">URL of the Notary server: defaults to https://notary.docker.io |
|
This configuration option can be overridden with the command line flag |
|
`-s` or `--server`.</td> |
|
</tr> |
|
<tr> |
|
<td valign="top"><code>root_ca</code></td> |
|
<td valign="top">no</td> |
|
<td valign="top"><p>The path to the file containing the root CA with which to verify |
|
the TLS certificate of the Notary server, for example if it is self-signed. |
|
The path is relative to the directory of the configuration file.</p> |
|
<p>This configuration option can overridden with the command line flag |
|
`--tlscacert`, which would specify a path relative to the current working |
|
directory where the Notary client is invoked.</p> |
|
<p> <b>does not seem to work, require client to pass in rootcacert via flag </b></p> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td valign="top"><code>tls_client_cert</code></td> |
|
<td valign="top">no</td> |
|
<td valign="top"><p>The path to the client certificate to use for mutual TLS with |
|
the Notary server. Must be provided along with <code>tls_client_key</code> |
|
or not provided at all. The path is relative to the directory of the |
|
configuration file.</p> |
|
<p>This configuration option can overridden with the command line flag |
|
`--tlscert`, which would specify a path relative to the current working |
|
directory where the Notary client is invoked.</p></td> |
|
</tr> |
|
<tr> |
|
<td valign="top"><code>tls_client_key</code></td> |
|
<td valign="top">no</td> |
|
<td valign="top"><p>The path to the client key to use for mutual TLS with |
|
the Notary server. Must be provided along with <code>tls_client_cert</code> |
|
or not provided at all. The path is relative to the directory of the |
|
configuration file.</p> |
|
<p>This configuration option can overridden with the command line flag |
|
`--tlskey`, which would specify a path relative to the current working |
|
directory where the Notary client is invoked.</p></td> |
|
</tr> |
|
</table> |
|
|
|
|
|
### sample server cert config (client facing TLS cert) |
|
``` |
|
[ req ] |
|
default_bits = 4096 |
|
distinguished_name = req_distinguished_name |
|
req_extensions = v3_req |
|
x509_extensions = v3_ca |
|
prompt = no |
|
|
|
[ v3_req ] |
|
# required |
|
subjectAltName = @alt_names |
|
basicConstraints = CA:TRUE |
|
keyUsage = digitalSignature, keyEncipherment |
|
|
|
[v3_ca] |
|
# required |
|
basicConstraints = critical, CA:TRUE, pathlen:1 |
|
keyUsage = critical, keyCertSign |
|
subjectAltName = @alt_names |
|
|
|
[alt_names] |
|
IP.1 = 54.174.67.41 |
|
DNS.1 = notaryserver |
|
DNS.2 = notary-server |
|
DNS.3 = notary-signer |
|
DNS.4 = notarysigner |
|
|
|
[ req_distinguished_name ] |
|
0.organizationName = Moto |
|
organizationalUnitName = unit name |
|
emailAddress = [email protected] |
|
localityName = Chicago |
|
stateOrProvinceName = IL |
|
countryName = US |
|
commonName = 54.174.67.41 |
|
``` |
|
|
|
## example ca certificate used by server-signer verification |
|
``` |
|
Certificate: |
|
Data: |
|
Version: 3 (0x2) |
|
Serial Number: |
|
8f:c7:59:4b:8d:49:44:23 |
|
Signature Algorithm: sha256WithRSAEncryption |
|
Issuer: C=US, ST=CA, L=San Francisco, O=Docker, CN=Notary Testing CA |
|
Validity |
|
Not Before: Feb 17 00:43:13 2017 GMT |
|
Not After : Feb 15 00:43:13 2027 GMT |
|
Subject: C=US, ST=CA, L=San Francisco, O=Docker, CN=Notary Testing CA |
|
Subject Public Key Info: |
|
Public Key Algorithm: rsaEncryption |
|
Public-Key: (4096 bit) |
|
Modulus: |
|
7f:53:f4:3a:95:e1:e4:05:9c:fc:de:07:18:8c:09: |
|
.... |
|
7d:d3:4e:fb:95:9c:ed:2b:9a:c4:bb:03:dd:38:75: |
|
07:9e:37 |
|
Exponent: 65537 (0x10001) |
|
X509v3 extensions: |
|
X509v3 Basic Constraints: critical |
|
CA:TRUE, pathlen:1 |
|
X509v3 Key Usage: critical |
|
Non Repudiation, Certificate Sign, CRL Sign |
|
X509v3 Subject Key Identifier: |
|
C5:DA:E5:75:FE:6D:AE:49:7F:46:11:B8:BE:94:AA:3B:54:41:2E:59 |
|
Signature Algorithm: sha256WithRSAEncryption |
|
79:c9:b7:04:62:d6:8a:09:ac:34:87:8e:f8:f9:f4:d6:ff:84: |
|
.... |
|
4b:02:26:91:11:c5:de:a1 |
|
|
|
``` |
|
## Other stuff |
|
- Generating wildcard certificate |
|
- Guideline for generating CA pin-able certificate |
|
- setting up a health check |
|
- manage logs |
|
|
|
 |
