Minio server with the Security Token Service (STS) and AssumeRole for temporary session tokens

README.md

MinIO Security Token Service (STS)

This is an example of setting up a Minio server with the Security Token Service (STS) and AssumeRole for temporary session tokens.

You must create a new Minio user to use STS, the default Minio access/secret won't work. The new user must have access to the objects you will be creating sessions for, the permissions of the created session are the intersection of the permissions of the STS user and the inline permissions requested when the session is created

Tested on 2020-04-06 with the current version of Minio. This is probably RELEASE.2020-04-04T05-39-31Z, though if using Homebrew on Mac OSX minio --version outputs DEVELOPMENT.GOGET so who knows.

Example: start the Minio server and configure the client

This will serve your home directory:

export MINIO_ACCESS_KEY=minio
export MINIO_SECRET_KEY=minio123
minio server ~/

Create a mc client config called local:

mc config host add local http://localhost:9000 minio minio123

Example: create a user stsadmin with full read-only access to all buckets and objects

mc admin user add local stsadmin stsadmin-secret
mc admin policy add local readall s3-policy-readall.json
mc admin policy set local readall user=stsadmin

Example: create a token

Run create-token.py to create a session token valid for 15 minutes for a bucket and optional prefix. The token along with connection credentials will be printed to stdout. This example will give access to keys matching the prefix media/* in bucket tmp:

TOKEN=$(./create-token.py --endpoint http://localhost:9000 --accesskey stsadmin --secretkey stsadmin-secret --bucket tmp --prefix 'media/*')
echo "$TOKEN"

Example: test the token

Test the token by using list-or-get-object.py. This script reads the connection credentials output by the previous script on stdin.

List objects (trailing /):

$ echo "$TOKEN" | ./list-or-get-object.py tmp/media/

Listing tmp/media/
- ETag: '"00000000000000000000000000000000-1"'
  Key: media/hello.txt
  LastModified: 2020-04-06 18:29:55.394000+00:00
  Owner:
      DisplayName: ''
      ID: 02d6176db174dc93cb1b899f7c6078f08654445fe8cf1b6ce98d8855f66bdbf4
  Size: 6
  StorageClass: STANDARD

Get an object (no trailing /):

$ echo "$TOKEN" | ./list-or-get-object.py tmp/media/hello.txt

Getting tmp/media/hello.txt
b'hello\n'

List a disallowed path:

$ echo "$TOKEN" | ./list-or-get-object.py tmp/other/

Listing tmp/other/
Traceback (most recent call last):
    ...
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied.

Expired token:

$ echo "$TOKEN" | ./list-or-get-object.py tmp/media/

Listing tmp/media/
Traceback (most recent call last):
    ...
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: The access key ID you provided does not exist in our records.

create-token.py

#!/usr/bin/env python

# https://github.com/minio/minio/blob/master/docs/sts/assume-role.md
# https://docs.minio.io/docs/how-to-use-aws-sdk-for-python-with-minio-server
# https://docs.min.io/docs/minio-select-api-quickstart-guide.html

import argparse
import boto3
import json

parser = argparse.ArgumentParser(
    'create-token.py', description=(
        'STS token creator. Create read-only temporary access tokens for S3'))
parser.add_argument('--endpoint', help='S3 server endpoint', required=True)
parser.add_argument('--region', help='S3 region', default="")
parser.add_argument(
    '--accesskey', help='Access key ID for the STS admin user', required=True)
parser.add_argument(
    '--secretkey', help='Secret access key ID for STS admin user',
    required=True)
parser.add_argument('--bucket', help='S3 bucket', required=True)
parser.add_argument(
    '--prefix', default='*', help=(
        'Prefix inside bucket, for example * (default), '
        'prefix/*, prefix/file.name'))

args = parser.parse_args()

sts_admin = boto3.client(
    'sts',
    endpoint_url=args.endpoint,
    aws_access_key_id=args.accesskey,
    aws_secret_access_key=args.secretkey,
    region_name=args.region)

bucket_name = args.bucket
prefix = args.prefix

# Access policies
# https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#policies_session
# https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html
# https://aws.amazon.com/premiumsupport/knowledge-center/s3-folder-user-access/
policy = {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListObjectsInBucket",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": ["arn:aws:s3:::{}".format(bucket_name)],
            "Condition": {
                "StringLike": { "s3:prefix": [prefix]}
            },
        },
        {
            "Sid": "GetObjectsInBucket",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": ["arn:aws:s3:::{}/{}".format(bucket_name, prefix)],
        },
    ],
}

# https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html#STS.Client.assume_role
response = sts_admin.assume_role(
    RoleArn='arn:x:ignored:by:minio:',
    RoleSessionName='ignored-by-minio',
    # PolicyArns=[{'arn': 'string'}],
    Policy=json.dumps(policy),
    DurationSeconds=900,
)

boto3_client_args = dict(
    endpoint_url=args.endpoint,
    aws_access_key_id=response['Credentials']['AccessKeyId'],
    aws_secret_access_key=response['Credentials']['SecretAccessKey'],
    aws_session_token=response['Credentials']['SessionToken'],
    region_name=args.region,
)
# E.g. s3 = boto3.client('s3', **boto3_client_args)
print(json.dumps(boto3_client_args))

list-or-get-object.py

#!/usr/bin/env python

import boto3
import json
import sys
import yaml


s3path = sys.argv[1]
s = sys.stdin.read()
connection_args = json.loads(s)
connection_args.pop('expiration', None)

session = boto3.client('s3', **connection_args)

bucket, prefix = s3path.split('/', 1)
if s3path.endswith('/'):
    print('Listing {}'.format(s3path))
    r = session.list_objects(Bucket=bucket, Prefix=prefix)
    print(yaml.dump(r['Contents']))
else:
    print('Getting {}'.format(s3path))
    r = session.get_object(Bucket=bucket, Key=prefix)
    print(r['Body'].read())

s3-policy-readall.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": ""
        }
    ]
}