Skip to content

Instantly share code, notes, and snippets.

@nicowilliams
Last active October 27, 2025 16:29
Show Gist options
  • Select an option

  • Save nicowilliams/c812b7354f69330392d99a825a4242dc to your computer and use it in GitHub Desktop.

Select an option

Save nicowilliams/c812b7354f69330392d99a825a4242dc to your computer and use it in GitHub Desktop.
On the design and implementation of Two Sigma's generic authorization system
title author date
On the design and implementation of Two Sigma's generic authorization system
Nicolas Williams
October 22, 2025

Abstract

TS Entitlements is a dial-tone and very fast application-level authorization system designed and implemented at Two Sigma Investments, LP which is inspired by labeled security systems, mainly SMACK (the Simplified Mandatory Access Control Kernel), traditional filesystem ACL systems such as NTFS ACLs, role-based access control systems (RBAC), and attribute-based access control systems (ABAC). It could be said that TS Entitlements evolves and "unifies" all of these.

The core primitives in TS Entitlements are a subject-verb-label authorization check where the label is a name stored in or derived from an actual object's metadata, and granting/revocation of access by creating/deleting grantee-role-label triples where "roles" are named sets of "verbs". A "subject" is a user's authentication context (or capability), while a "grantee" can be a user ID, a user group, or other special entities. With these three primitives and a very simple schema we manage to build much richer policies and schema including ABAC functionality, and we let applications have varying levels of access controls granularity, all while yielding excellent authorization check performance.

Disclaimer

This document is being distributed for informational and educational purposes only and is not an offer to sell or the solicitation of an offer to buy any securities or other instruments. The information contained herein is not intended to provide, and should not be relied upon for, investment advice. The views expressed herein are not necessarily the views of Two Sigma Investments, LP or any of its affiliates (collectively, “Two Sigma”). Such views reflect the assumptions of the author(s) of the document and are subject to change without notice. The document may employ data derived from third-party sources. No representation is made by Two Sigma as to the accuracy of such information and the use of such information in no way implies an endorsement of the source of such information or its validity.

The copyrights and/or trademarks in some of the images, logos or other material used herein may be owned by entities other than Two Sigma. If so, such copyrights and/or trademarks are most likely owned by the entity that created the material and are used purely for identification and comment as fair use under international copyright and/or trademark laws. Use of such image, copyright or trademark does not imply any association with such organization (or endorsement of such organization) by Two Sigma, nor vice versa.

Table of Contents

  1. Introduction
  2. Generic authorization systems
  3. Applicability
  4. TS Entitlements
    1. Abstract API
    2. Verbs, roles, and labels
    3. Architecture
    4. Enterprise directory schema for TS Entitlements
    5. Schema optimized for access control evaluation
    6. checkCore() in SQL
    7. Incremental materialized view maintenance
    8. Making the C-coded checkCore() fast
    9. Dial-tone architecture
    10. Data distribution
    11. ABAC functionality in TS Entitlements
    12. Dog-fooding
    13. Odds and ends
    14. Negative permissions
    15. UI/UX considerations
    16. Sample verbs and roles
    17. Auditing extant authorizations
  5. Limitations of the checkCore() subject-verb-object model
  6. Comparison to other generic authorization systems
    1. Comparison to MLS
      1. Abstract API for MLS
    2. Comparison to SMACK
      1. Abstract API for SMACK
    3. Comparison to filesystem ACLs
      1. Abstract API for filesystem ACLs
    4. Comparison to RBAC
      1. Abstract API for RBAC
    5. Comparison to OAuth-style protocols
    6. Comparison to ABAC systems
  7. Constructing new primitives from the checkCore() primitive
  8. Conclusion and future directions
    1. Federated entitlements using URIs, OAuth, and/or PKIX
      1. Use-case: federated access controls for healthcare data
  9. References

Introduction

This blog/paper is about TS Entitlements, an authorization system that I had a large part in designing, implementing, operating, and maintaining at Two Sigma Investments, LP.

Throughout I'll be using "abstract APIs" to help show how TS Entitlements works and how it is an evolution of several other generic authorization systems.

In the sense of the "ACLs don't" paper, TS Entitlements is currently an "ACL" system, though nothing1 precludes a capability model from being added, possibly OAuth-style.

TS Entitlements stores its authorization data in an enterprise directory that users interact with. Naturally that directory and/or its schema are subject to change, as long as the TS Entitlements authorization data can be represented in the directory. Multiple radically different directories can be supported.

The API that applications use to perform authorization checks uses authorization data that has been extracted and transformed into a schema optimized for authorization check performance.

The API provides a subject-verb-label metaphor for access control checks, but a grantee-role-label metaphor for granting and revocation. Here a "label" is a name string that stands for the object to be protected, not unlike SMACK. Typically the application stores the label in the object's metadata, though some applications derive the label from the object's metadata (generally some name). Between the use of roles in granting, having symbolic labels, and sets of grants, TS Entitlements resembles RBAC, SMACK, and filesystem ACLs, all with enhancements. Combining the core abstraction into richer abstractions yields ABAC functionality.

The difference in directory-side and API-side authorization data requires an extract-transform-load (ETL) step in the product's architecture. This ETL process is as a compiler that translates data expressed in a convenient schema to the schema that is convenient for the API's implementation. The API-side schema is extremely simple and requires creativity to express ABAC policies and attributes, while the enterprise directory schema can be richer to hide that creativity from the user interface (UI) and user experience (UX).

Generic authorization systems

An authorization system allows users to express who can do what to what things and where, with primitives for granting access, revocation, and checking if some access is allowed. There are many types of authorization systems, some quite specialized and even esoteric, while others are quite generic. Some applications require very specialized authorization functionality (for example, TPMs), while others can reasonably use generic authorization systems. TS Entitlements is a generic authorization system.

Generic authorization systems include:

  • access control lists (ACLs)
  • role-based access control (RBAC)
  • labeled security, typically involving mandatory access controls (MAC)

among others.

ACLs typically consist of lists of grant/deny ACL entries (ACEs), with each entry naming a user, a group of users, a host, or other grantee entity, and some set of verbs. Typical systems include the Windows NTFS ACL system, where there are only 16 verbs (encoded as a 16-bit bit-mask) that one can grant. Typically the primitives in ACL systems are: setacl(file, acl), getacl(file) -> acl, and check(user, file).

RBAC systems typically involve granting users or groups of users one or more roles enterprise- or domain-wide. In RBAC systems roles are not always well-defined in their semantics because typically the check() primitive is of the form check(subject, role), and so one has to understand all the check() invocations used by applications. RBAC systems are also often coarse-grained systems where there is no way to limit roles to subsets of large administrative domains (e.g., Solaris RBAC).

Labeled security systems are generally MAC systems, though that's not required. A MAC system forces a separation of who can set or change an object's label (and what the default label is) from the owner of the object -- this is typically desirable in military organizations where classification and declassification authority is highly limited. But the core concept of labeled security is not so much mandatory security as it is separating the storage of the authorization metadata from the objects it protects by giving that authorization metadata a name that objects store in their metadata. This is especially true in SMACK where labels are free-form text strings. Older labeled security systems embed all the authorization metadata in the label itself, so the label is not exactly a "name", but typically labels have names anyways.

TS Entitlements is inspired by all of these and is an evolution of all of them as follows:

  • Take an NTFS ACL, remove the DENY option thus making it a set of entries rather than a list of entries, refer to roles (which are named sets of verbs) instead of verbs in the entries, allow unbounded roles and verbs, give the ACL a name, store the result in a directory, and store the name in objects' (e.g., files) metadata instead of the actual ACL.

  • Take RBAC, add a notion of verbs, make roles properly a named set of verbs, add a notion of label to allow for fine-grained access control.

  • Take SMACK, allow larger labels, replace the authorization metadata database which stores {label, rwx, grantee} with a set of {label, role, grantee}, and expand this database to a set of {label, verb, user} grants for fast checking in the kernel or application.

Applicability

TS Entitlements is a generic authorization system, and it can be used creatively in many applications.

However, there are limits to the applicability of generic authorization systems, including TS Entitlements.

For example TPM 2.0 authorization policies can't be made to refer to TS Entitlements policies except through signed policy tickets from an online service intended for this, but that entails trade-offs that might not be acceptable. Another example is that application-specific authorization systems can have lower cognitive load than generic authorization systems, and sometimes that lower cognitive load is sufficient to justify an application-specific authorization system.

In other words generic authorization systems like TS Entitlements are not universal in applicability, therefore they are not a panacea. Still, TS Entitlements is applicable to -and used by- a large swath of applications.

TS Entitlements

A key goal of TS Entitlements is to support a fast check() predicate that represents actions as verbs with well-defined semantics, with an arbitrary number of verbs, and both fine-grained access controls as well as coarse-grained access controls, with those access controls stored in an enterprise directory.

We cannot store information about all objects in a directory, as that would not scale, but we want to be able to get a rough idea of what each user can do just by auditing the authorization metadata stored in the directory. To do this we must have a) documentation embedded in the authorization data as to what kinds of objects will be protected by each grant, and b) relatively few grants. A full audit of authorization data does still require iterating objects in all applications, naturally, but a partial audit based only on the metadata stored in the directory can paint a pretty complete picture by itself.

Abstract API

As mentioned above, TS Entitlements is a generic authorization system with three core primitives:

  • check(subject, verb, label) -> Result<Error>
  • checkCore(subject, verb, label) -> boolean -- core authorization check function used by check() to implement more complex policies
  • grant(label, role, grantee)
  • revoke(label, role, grantee)

and a few additional supporting primitives:

  • query(label, role, [grantee]) -> Set<grantee>
  • query(subject) -> Set<{label, verb}>
  • query(subject) -> Set<{label, role}>
  • createRole(role)
  • addVerb2Role(role, verb)
  • createLabel(label)
  • etc.

As well there are primitives that TS Entitlements using applications must provide for themselves, such as:

  1. setlabel(object, label) and getlabel(object) -> label (fine-grained),
  2. setlabel(volume, label) and getlabel(volume) -> label (coarse-grained),
  3. derivelabel(object) -> label (fine-grained),
  4. derivelabel(volume) -> label (coarse-grained).

(1) and (2) are for applications that can store a label in object metadata; (3) and (4) are for applications that cannot. The former is always preferable to the latter.

Note that a grantee is an entity like a user or a group of users, while a subject is anything from an authenticated identity to an authentication context. For example, check() can use a [JWT] as a subject, can validate the token, and extract its claims. Currently there is no capability support in check(), but its design does not preclude it, as a) a capability could be used as a subject, b) the result of check() can be extended to produce capability request messages to be sent to a peer (e.g., an HTTP redirect to a token issuer).

Verbs, roles, and labels

Verbs are English-language verbs, capitalized, such as READ. We do qualify verbs with "application" names in order to distinguish different semantics for the same verbs, such as filesystem:READ and database:READ.

Roles are English-language actor words, with the first letter capitalized, such as Reader, Writer, Administrator, Owner, and so on. As with verbs we qualify roles with application names, but note that there is no requirement that roles contain only verbs qualified with the same application name.

Labels have some hierarchy to them. Currently we only have one namespace component, but we could add more hierarchy. A typical label might be SecurityDevelopment::TSEntitlements. Applications can implement a most-specific label search using some separator for components of the label, such as FooNamespace::a/b/c, where / is used as a separator.

Architecture

We have several components:

  • an enterprise directory service
  • services that perform extract, transform, and load (ETL) operations with the authorization metadata from the directory, then produce and distribute small databases optimized for check() speed, as well as produce and distribute incremental updates to those small databases
  • a client that fetches those small databases and the incremental updates to them, applying them to the local copies they keep
  • a C-coded API that implements check() and query()
  • foreign function interface (FFI) bindings of that C-coded API for multiple programming languages

We also have a REST/gRPC application service that exports check() and query() functions for applications that cannot use the C-coded API.

Data flows from the directory to the ETL service to the clients to the API.

Enterprise directory schema for TS Entitlements

The directory schema will be very specific to the directory being used. At Two Sigma we use an internal, proprietary directory, therefore I will not describe the exact schema we use, but I will show a mostly equivalent SQL schema that can be used to represent TS Entitlements:

CREATE TYPE entity_type AS ENUM (
    'user', 'user_group', 'realm', 'compartment', 'special'
    -- special is for entities like 'ANYONE', 'SELF',
    -- 'COMPARTMENTED', etc.
);
CREATE TABLE entities (
    _name text NOT NULL,
    _type entity_type NOT NULL,
    _id bigint AUTOINCREMENT,
    
    -- Static attributes of users that are best
    -- not represented as group memberships should
    -- be added here:
    
    PRIMARY KEY (_name, _type),
    UNIQUE (_id)
);

CREATE TABLE user_group_user_memberships (
    _user_name text,
    _user_type entity_type CHECK ('user'),
    _group_name text,
    _group_type entity_type CHECK ('group'),
    FOREIGN KEY (_user_name, _user_type)
        REFERENCES entities (_name, _type),
    FOREIGN KEY (_group_name, _group_type)
        REFERENCES entities (_name, _type),
    PRIMARY KEY (_user_name, _user_type,
                 _group_name, _group_type)
);

CREATE TABLE user_group_group_memberships (
    _child_name text,
    _child_type entity_type CHECK ('group'),
    _parent_name text,
    _parent_type entity_type CHECK ('group'),
    FOREIGN KEY (_child_name, _child_type)
        REFERENCES entities (_name, _type),
    FOREIGN KEY (_parent_name, _parent_type)
        REFERENCES entities (_name, _type),
    PRIMARY KEY (_child_name, _child_type,
                 _parent_name, _parent_type)
);

CREATE TABLE apps (_app text PRIMARY KEY, notes text);
CREATE TABLE verbs (
    _verb text PRIMARY KEY,
    _app TEXT NOT NULL FOREIGN KEY REFERENCES apps (_app),
    _notes TEXT
    
    -- Static attributes of verbs should be
    -- added here:
);
                   
CREATE TABLE roles (
    _role TEXT PRIMARY KEY,
    _app TEXT NOT NULL FOREIGN KEY REFERENCES apps (_app),
    _notes TEXT
    
    -- Note that grants to specific verbs can
    -- indirect through multiple roles, therefore
    -- any attributes of roles must either affect
    -- only the granting/revocation process, or must
    -- "flow" additively to verbs.
);

CREATE TABLE role2verb (
    _role text FOREIGN KEY REFERENCES role (_role),
    _verb text FOREIGN KEY REFERENCES verb (_verb),
    PRIMARY KEY (_role, _verb)
);

CREATE TABLE labels (
    _label text PRIMARY KEY,
    _notes text
    
    -- Attributes of labels go here:
);

CREATE TABLE grants (
    _label text FOREIGN KEY REFERENCES labels (_label),
    _role text,
    _grantee bigint FOREIGN KEY REFERENCES entities (_id),
    
    -- Grant attributes go here, such as time-of-day
    -- contraints.
    _expires timestamp -- the grant is "deleted" from the
                       -- data used by the API when it
                       -- expires,
    PRIMARY KEY (_label, _role, _id)
);   

Schema optimized for access control evaluation

We use a variant of the following SQL schema for various systems, both relational as well as a bespoke system based on the Constant DataBase (CDB) hash-table-in-a-file file format:

CREATE TABLE subject2id (
    subject_name TEXT,
    subject_type entity_type,
    subject_id integer NOT NULL UNIQUE,
    PRIMARY KEY (subject_name, subject_type)
);
CREATE TABLE subject2groups (
    id integer,
    group_id integer,
    PRIMARY KEY (id, group_id)
);
CREATE TABLE grants (
    label TEXT,
    verb TEXT,
    grantee integer,
    PRIMARY KEY (label, verb, grantee)
);

where the roles referenced by grants have been expanded into verbs, and where the group nesting transitive closure is expanded in the subject2groups table. Every subject will have an entry in subject2groups for it being a member of itself.

This is used to make checkCore() very fast.

checkCore() in SQL

We do have a SQL-coded checkCore() for use in database applications. It looks like this:

    SELECT EXISTS (
        SELECT 1
        FROM entities e
        JOIN subject2groups s2g
             ON e._id = s2g._id
        JOIN grant2grantees g2g
             ON s2g._group_id = g2g._grantee
        WHERE e._type = 'user' AND e._name = _the_subject AND
              g2g._label = _the_label AND g2g._verb = _the_verb
    );

where

  • subject2groups is a MATERIALIZED VIEW mapping every user ID to every group's ID that they belong to directly or indirectly via nested group memberships, and
  • grant2grantees is a MATERIALIZED VIEW with roles expanded to verbs.

Note in particular that the subject2groups materialized view lists each subject's user group memberships, but also their own user entity ID as well as the ANYONE special entity's ID. This allows the JOIN in the above query to match grants to the user as well as grants to the ANYONE special entity, not just grants to user groups.

These expansions of the nested group transitive closure and the role-verb assignments plus the materializations of these expansions makes check() quite fast.

I elide the SQL definitions of these materialized views. The interested reader can define them themselves using the descriptions above.

Incremental materialized view maintenance

Our materialized views are updated using incremental updates from the enterprise directory service as we go so as to avoid having to re-materialize the views. The code to update the materializations is hand-coded. To avoid restrictions imposed by the RDBMS when using MATERIALIZED VIEWs we use regular tables and implement our own materialization functionality, including REFRESH MATERIALIZED VIEW <name> CONCURRENTLY. We use something similar to mat_views.sql.

Making the C-coded checkCore() fast

Abstractly the key to making check() fast is that our access controls are sets of entries rather than lists of entries.

Concretely though we also expand the set of {label, role, grantee} entries to {label, verb, grantee} entries, and we make it very fast to find subjects' lists of grantee IDs corresponding to the user IDs and the group IDs, just as in the SQL-coded checkCore() above, and we implement exactly the same logic as in that SQL-coded checkCore(). Naturally these subject2groups and grant2grantees databases are all indexed.

The C-coded checkCore(subject, verb, label} then does the following:

  1. looks up the {label, verb}'s set of grantees (which is stored as a sorted array of IDs),
  2. looks up the subject's set of grantee IDs (which is also stored as a sorted array of IDs),
  3. if the intersection of the two sorted entity ID arrays is empty then access is denied, else access is granted

If one of the two grantee ID arrays is much smaller than the other then we iterate through the smaller one and join it to the larger one with a binary search, stopping when we find one match or no matches. If both grantee ID arrays are similar in size then we walk a cursor through each looking for a match. Users typically have a few hundred group memberships, but {label, verb}s typically have only a dozen grantees, therefore a typical check() is O(N log M) where N << M, which is quite fast.

Typical checkCore() calls take on the order of 15us to complete. Although the system is fast by design, we do leave some performance on the table because it's plenty fast enough, and we didn't optimize it further. For example, we use 64-bit entity IDs but 32-bit entity IDs would have sufficed, therefore we waste a lot of memory bandwidth. For another example we make sure that we can align these arrays on 64-bit boundaries w/o additional memory allocations, but we don't make sure that the arrays are indeed always aligned on 64-bit boundaries on disk, thus we sometimes incur memmove() penalties. We could probably reduce the cost of checkCore() a great deal, but we just haven't needed to.

The local indexed database consists of CDB ("constant database") files [CDB], using TinyCDB as the implementation. A CDB file is a read-only hash table in a file. The keys are either integer entity IDs or entity names (name and entity type). The values are sorted arrays of entity IDs or sorted arrays of entity names (name and type), with the latter only needed for query().

Incremental updates are used to generate new CDBs based on the current ones and those updates, and then the new CDBs which are then renamed into place.

The API takes great pains to ensure that only up to two instances of CDBs are open at any given time, one being the current CDBs and the other being the previous CDBs, all while being multi-threaded and thread-hot. We use a user-land read-copy-update (RCU) like facility to do this.

These CDBs total around 100MB, and they are mapped into memory.

One CDB maps subject names to their entity IDs. Another maps subject entity IDs to their list of supplementary entity IDs (group memberships, with nested user group transitive closure expanded). And another maps {label, verb} grants (i.e., with roles expanded) to direct grantees. Thus the core check() primitive simple does three CDB lookups and a fast sorted array empty intersection check.

Dial-tone architecture

To make the system dial-tone we distribute the authorization data in its optimized and indexed form to all clients so that check() can be 100% local, with incremental updates distributed and applied to the local caches asynchronously from the directory where the authorization metadata is stored.

We do also have a REST/gRPC service that can be called from clients that do not have local data or which cannot use our C implementation of the TS Entitlements API. There are a variety of programming languages with a strong preference for not using the C run-time, such as Rust and Go. The REST/gRPC service can run locally, using local data, to maintain the dial-tone characterization of the system's availability.

Data distribution

The authorization data (CDB files) and incremental updates thereto are distributed over HTTPS. The ETL service writes incremental updates, metadata, and heartbeat messages to a file that the clients fetch with HTTP GET range and conditional request headers using weak ETags. The Range: header specifies a starting offset and leaves the end offset unspecified, and the server maintains the GET response body open as long as the file to which the updates are written remains in place -- the GET response ends when the file is deleted or renamed away. Clients can use the ETL service's promise of heartbeats, and missing heartbeats, to decide to reconnect.

ABAC functionality in TS Entitlements

We currently support use of several attributes of the subject, the label, and the grant in making access control decisions:

  • User attributes (contextual):
    • the realm of the client user (e.g., Kerberos realm name, JWT issuer name, etc.)
    • the compartment of the client user workload
    • whether the user has performed multi-factor authentication (MFA)
  • User attributes:
    • on-call schedule, enabling the user to elide related approval requirements during off hours when on-call for a product
  • Grant attributes
    • the set of realms that the grant is constrained to
    • whether the grant is "compartmented" and the set of compartments the grant is constrained to
    • whether the grantee requires approval from an approver
    • whether the grantee requires multi-factor authentication (MFA)

A realm is an administrative domain, and corresponds to such things as: Kerberos realm names, PKIX issuer names, JWT issuer names, etc.

A compartment is a set of nodes or clusters on which corresponding workloads run. These are used for isolation -- a sort of application-level firewall.

We express constraint attributes on grants as grants to special grantees representing:

  • realms
  • compartments
  • "needs approval"
  • "needs MFA"

We also represent some attributes as grants to normal grantees, such as:

  • "is on-call"

On-call grants are made and revoked when each user's on-call shifts start and end.

A user who is on-call for some product might get to perform operations after hours without express approval in spite of any "needs approval" policies, though subject to ex-post review the next day.

Attributes of subjects are denoted by the authentication system used. For example, when using [OAuth] w/ [JWT] or [CWT] we encode the compartment of the user as the string value of a cmpt claim. The realm of a user is taken from the domain, realm, or issuer of the user's ticket, certificate, or token, depending on the authentication method used. We rely on identifying claims to look up the user's entitlements for the purpose of evaluating check() calls.

Some attributes require the ability to indicate "access granted subject to conditions ...", where conditions include:

  • needs approval (two-person system)
  • needs MFA

We have a way to indicate that use of compartments is optional, which we used when migrating applications to the compartment system. Applications that are in the process of migrating might allow but warn about access from outside approved compartments so that operators can investigate and alter compartment composition until the application is ready to run with mandatory compartment enforcement.

Thus our check() primitive in reality does not return boolean:

  • In C check() returns an integer status code representing success (access granted), failure (access denied), various errors, and even results like conditional access grant with multiple conditions such as "needs approval", "needs MFA", etc.
  • In languages that support exceptions check() returns boolean or void and throws exceptions for errors and conditional access grants.
  • In languages that support algebraic data types but not exceptions we use something like Either with an error object in the failure case.

Many more attributes could be implemented in this way. The important thing is that we built a rich ABAC system using our simple check(subject, verb, label) primitive.

We also intend to eventually support attributes of the grants themselves, such as grant expiration.

We use natural language concepts like subject, verb, object (by proxy, via label). What should we call attributes? I think we can call attributes:

  • adjectives when attached to grantees or labels
  • adverbs when attached to grants

We could add a clearance_level attribute to match the Multi-Level Security [MLS] labeled security system. Here users would have a specific and global clearance level, and the labels attached to objects would have a minimum clearance level attribute. Users would have to have a clearance level at least as high as the label requires in order to even see that the object exists, let alone access it in any way. We don't need this MLS functionality, but other organizations do.

Dog-fooding

It is possible to use this system to permission granting and revocation operations by using verbs such as:

  • tsents:OWN -- denotes ownership by the grantee
  • tsents:GRANT -- if granted denotes permission of the grantee to grant roles they have on the label to others except for granting of roles containing this verb
  • tsents:DELEGATE -- if granted denotes permission of the grantee to grant roles containing tsents:GRANT on the label to others
  • etc. See Sample verbs and roles.

Odds and ends

Special grantee entities include:

  • ANYONE -- grants to ANYONE are grants to any authenticated user from any of the realms also granted the same verbs
  • SELF -- grants to SELF cause check() to succeed when the subject is the same user that is running the relying party software (great for developers)
  • COMPARTMENTED -- denotes that the {label, role} requires that subjects be workloads executing in granted compartments
  • TWOPARTY -- denotes that the {label, role} requires that subjects have approval by a second party
  • MULTIFACTOR -- denotes that the {label, role} requires that subjects perform multi-factor authentication

We could also have a PUBLIC special grantee entity to denote that no authentication is required.

Special labels include:

  • SELF:: -- a label grants all verbs to the SELF special grantee entity (great for developers)

TS Entitlements does support using JSON text representations of serialized sets of grants as a label. This is useful for applications that can store grants but not labels in their object metadata, as well as for applications that cannot trust the enterprise directory service for whatever reason.

Negative permissions

Although TS Entitlements does not encode negative entries like filesystem ACLs do, it is still possible to encode negative permissions by having verbs which, if granted, cause a denial. Applications would check() if the subject has been granted some verb denoting negative permissions, denying if granted, allowing otherwise.

Naturally negative permissions must be used very carefully, if at all!

For example:

  • foo:DENY_LAUNCH_MISSILES would deny the grantee permission to launch missiles, highlighting the dangers of negative permissions!

A more realistic example is our ONCALL verb that denotes exemption from needing a reviewer after hours when on-call.

UI/UX considerations

We have two types of UIs related to TS Entitlements:

  • those that concern viewing, creating, or managing verbs, roles, labels, and grants, and
  • those that are part of the applications that use TS Entitlements and which concern picking labels for new objects, setting labels on objects, and viewing the labels set on objects.

The former we provide with TS Entitlements.

The latter are clearly application-specific, as it should be. Though for RESTful HTTP applications we could specify conventions for those operations using appropriate HTTP verbs and URI local-part naming conventions.

We currently use <app>:<NAME> as the naming convention for verbs, and <app>:<Name> as the naming convention for roles. A form that does not require an <app> prefix but which uses site-local configuration for discovering each verb's and role's <app> might be useful. A URN- and/or URI-based naming convention for verbs and roles might be appropriate for interchange purposes and/or federation.

We currently use <verb>^<label> and <role>^<label> to denote a grant of a verb or role in some UIs. For example Repository:PULL^monorepo::some/code/base. This is exactly backwards of how it should have been! The label should have come first, then the verb or role. The ^ character might not have been the best choice either.

Default label selection and label selection pickers are noticeably absent in our implementation. This is a cause of trouble. Applications that have any notion of "directory" can simply apply a directory's label to objects create in it by default. But not all applications have "directories". An API for finding a default label for a given user creating objects in a given application would be nice. A convention for this can be created using an <app>:DEFAULT verb and a query() call to find labels that grant <app>:DEFAULT to the user creating an object, but such a convention would allow for multiple default labels, in which case the application might need to interact with the user or pick the first such label in lexicographic order (say). Alternatively we could have a separate schema and APIs for encoding and querying such things.

A JSON Schema for interchange of grants on labels would be nice, but we do not currently have one.

Sample verbs and roles

Sample generic verbs that one might want:

  • Ownership, management, auditing:

    • tsents:OWN -- denotes the ability to add/remove owners of the label on which it is granted; it might also denote having all verbs for all objects protected by the same label
    • tsents:GRANT -- see Dog fooding
    • tsents:DELEGATE -- see Dog fooding
    • tsents:MANAGE -- denotes the ability to do all things that owners can do except add/remove owners
    • tsents:AUDIT -- denotes the ability to list objects protected by the label and to see all grants on the label
  • Filesystem-like verbs:

    • generic:READ
    • generic:CREATE
    • generic:WRITE
    • generic:APPEND
    • generic:ACCESS -- like the Unix x permissions bit on directories
    • generic:EXECUTE
    • generic:LABEL -- denotes the right to change the label of objects protected by the label this is granted on
    • etc.
  • Verbs for simple HTTP applications:

    • http:<method> for every HTTP method
  • Verbs version control repositories:

    • vc:CREATE
    • vc:PULL
    • vc:PUSH
    • vc:PUSH_TAG
    • vc:LABEL
    • etc.
  • App-specific generic verbs that most apps should want:

    • <app>:READ -- denotes read capability
    • <app>:WRITE -- may denote create/write/append/modify capabilities
    • <app>:LABEL -- denotes the ability to set the label of an object
    • <app>:DEFAULT denotes that the label is a "default label" for objects created by the grantee (see above)
    • <app>:ADMIN -- denotes the ability to perform administrative operations on an object, such as start or stop a service, etc.

Sample roles:

  • tsents:Owner, tsents:Manager, tsents:Auditor
  • generic:Reader (this might include http:HEAD and http:GET, as well as vc:PULL)
  • generic:Writer (this might include http:POST, PUT, DELETE, and PATCH, as well as vc:PUSH)
  • generic:Administrator
  • etc.

Auditing extant authorizations

Auditing who can do what to what resources has two steps:

  1. Iterate labels and grants in the enterprise directory back-ending TS Entitlements,
  2. Iterate objects of applications of interest and check their labels

Partial audits might skip step (2), using documentation on the labels in the enterprise directory to infer the extent of the access granted to each user. Step (2), after all, can be quite time consuming.

Partial audits might skip step (1). Auditing only the applications' objects and relying on the documentation on their labels in the enterprise directory to ensure that the documentation is accurate can help validate partial audits that only perform step (1).

Partial audits might only look at objects of a specific application and all the access granted to user by the objects' labels, referencing the documentation and grants in the enterprise directory.

Limitations of the checkCore() subject-verb-object model

We get a great deal of mileage out of the subject-verb-object model in TS Entitlements, as it lets us unify -to a large degree- labeled security, MAC, DAC, RBAC, and filesystem ACLs, and it lets us have rich ABAC policies.

But there are some limits. Generic time-of-day restrictions can be difficult or impractical to express in the simple API-side schema by using special verbs and/or special grantees -- they can be expressed as grants that are automatically made and revoked around the time ranges in question, but this requires the incremental propagation system to be functioning. Indeed, we use that automatic grant/revocation approach implementing on-call policies.

The subject-verb-object model at the core of TS Entitlements does bleed into the policies as seen by users even though we can can have a more user-friendly schema at the enterprise directory than on the API side. To simplify further for users requires applications to layer their own UIs for access control management.

The subject-verb-object model is naturally stateless, unless one wishes to rely on making changes to the authorization policy store as a way to store state at some cost in latency (and unclean data should a process fail to clean up such state). Stateful policies can be implemented with this model only with adjunct systems to help maintain state. For example, a policy that some action can only be taken pursuant to recorded, official approval by some suitable officer will typically require a system in which to record requests, reviews, and approvals. Nonetheless a policy that approvals must come from specific approvers can be implemented with the subject-verb-object model -as indeed, we have done in TS Entitlements- using groups to define approvers who are granted a generic:Approver role that contains a generic:APPROVE verb, even though the approval state has to be recorded elsewhere.

Careful management of user group membership can be used to reflect organizational structure into the authorization system, naturally. Still a policy like "only managing directors may review and approve" mapping to a grant to a Managing Directors group is not as clean as a policy where the system for expressing it specifically allows a "managing directors only" constraint checkbox as a first-class feature. The UI can map a "managing directors only" (and other such) checkbox to grants of user groups that reflect the organizational structure, but that bit of complexity -though it can be hidden from the user- has to be present in the system in ways that are not as pure as a first-class feature might be instead.

Policies that involve cryptographic capabilities are almost entirely right out. For example, "encryption to groups" policies, TPM 2.0 EA policies, etc. Similarly, "smart contracts" on blockchains cannot be modeled with checkCore(). Cryptographic authorization systems simply cannot be implemented in terms of checkCore(). Though signed attestations of authorization can be used in TPM 2.0 EA policies (see the TPM2_PolicySigned() TPM 2.0 command), and the signers of those attestations can implement authorization checks using TS Entitlements.

In short, policies which are easily expressed in natural language need not trivially map to the subject-verb-object model, requiring creative thinking to fit that model. All policies expressible in MLS, SMACK, RBAC, and filesystem ACLs can be expressed in TS Entitlements, and most ABAC policies as well, but there will likely be some ABAC policies that cannot be implemented either at all with TS Entitlements or without additional components -- the latter is the case with policies that require keeping state.

Comparison to other generic authorization systems

TS Entitlements is inspired by SMACK, which is essentially an evolution of or inspired by MLS. But TS Entitlements is also easy to relate to filesystem ACLs, RBAC, and ABAC systems, and it is inspired by my experience and work on those systems as well. Being an evolution of all those of those generic systems means that TS Entitlements is a sort of a unification of them -- not a drop-in replacement, but abstractly at least it unifies all of those styles of generic authorization systems.

Note that the abstract APIs that we impute to other authorization systems below are not really specified anywhere. I use these to illustrate them, but the reader should refer to each system's specifications -and the documentation of their implementations- for more and more-accurate details.

Abstracting authorization APIs

Throughout this document I've been using a pseudo-code representation of abstract APIs. Abstract APIs are a useful tool for thinking about authorization systems as they serve as abstract documentation of what an implementation must provide, and also of how applications can or must use a given authorization system.

Comparison to MLS

Multi-level security [MLS] is a labeled security technology whereby all workloads (user processes) and all objects are "labeled". Labels in MLS consist of a triple {doi, level, Set<compartments>}, where the domain of interpretation (DOI) identifies the meaning of the other two items in the triple, with level being a very coarse-grained clearance level (e.g., "Secret", "Top Secret", "Ultra"), and the compartments being akin to user groups (e.g., U.S. DoD Army, U.S. DoD Navy, UK Marines, etc.). These labels are generally fixed in size, which limits the number of compartments that can be used. The DOI helps translate labels across administrative domains in a way that makes up a bit for the limited number of compartments.

Although labels in MLS have structure, they also have names, with a database for doing lookups between names and label values and vice-versa. MLS label names are for humans and need not reflect the actual compartments and level granted by the named label. Label names are only used for display purposes or when setting a label, which are relatively rare events and which need not be fast operations as they involve user interaction.

Storing the set of granted compartments in a fixed-sized label, and labeling both user processes as well as objects, is an optimization that makes checking for access very fast: the access control kernel simply compares the DOI of the process and the object, and if the DOIs are the same then it checks that the process' level is at least as high as the object's minimum level, and if so then it does a bit-mask XOR of the user process' and the object's labels' compartment sets, and if that results in at a non-zero value then access is granted if it is also granted by applicable discretionary access controls, while in all other cases access is denied. If the DOIs don't match that triggers a slow path where the user process' label is translated to the object's label's DOI or vice-versa, then the labels are checked as before.

In TS Entitlements we dispense with fixed-sized labels and do not attempt to encode the grants in the label itself. This frees us to use free-form labels, as in SMACK. Also as in SMACK the price we pay is that we must have a fast local database of grants in order to make access control checks fast enough. Although it's worth noting that TS Entitlements does support labels that are JSON texts representing serialized sets of grants -- this is for applications that cannot store labels but can store ACLs, as well as for applications whose owners do not wish to store authorization data in the enterprise directory.

By not storing grants in the label we allow exponentially larger policies than MLS can implement, and we do away with the notion of domain of interpretation (DOI).

One important difference is that TS Entitlements is used primarily as a discretionary access control (DAC) system, though it can also be used as a MAC system. Recall that MAC systems generally are layered on top of DAC systems as an additional access control where the owner of the protected object does not necessarily manage its MAC policy. Applications that use TS Entitlements can do the same, storing two labels per-object: one denoting DAC policy and the other denoting MAC policy. A MAC policy would mainly deal in just a very few verbs such as: one denoting the ability to change the MAC label, and one denoting the ability to get any access granted by the DAC label.

Abstract API for MLS

  • check(process, object) - boolean
  • check(subject_label, object_label) -> boolean
  • setProcessLabel(process, label)
  • getProcessLabel(process) - label
  • setObjectLabel(object, label)
  • getObjectLabel(object) -> label
  • getLabelDisplayName(label) -> name
  • createLabelDisplayName(name, label)
  • getLabelLevel(label) -> level
  • getLabelCompartments(label) -> Set<compartment>
  • getCompartmentName(compartment) -> name
  • createCompartment(name) -> compartment
  • makeLabel(level, Set<compartment>) -> label
  • addUserToCompartment(user, compartment)
  • removeUserFromCompartment(user, compartment)
  • setUserLevel(user, level)

Comparison to SMACK

SMACK is a system that replaces {level, compartments} with {name}, but retains fixed-sized labels in order to interoperate with protocols where labels are carried on the wire. This allows SMACK to replace structured labels with free-form, textual labels, just as in TS Entitlements.

To make authorization checks fast SMACK requires a local, in-memory, in-kernel database of grants. The grants involve a small number of verbs.

Differences between SMACK and TS Entitlements:

  • TS Entitlements removes the label length restrictions because it does not need to interoperate with protocols that carry e.g., CIPSO labels on the wire.
  • TS Entitlements allows arbitrary numbers of verbs.
  • TS Entitlements separates the local database used for authorization checks from the source of truth from which the former is derived. This allows the source of truth's schema to be richer.
  • TS Entitlements allows grants to be of roles rather than verbs in the source of truth. This allows roles to be refactored (verbs to be added to or removed from roles) without having to change the grants in the source of truth or the application source code that calls the check() primitive.

As in the MLS case, SMACK is a MAC system, while TS Entitlements is mainly used as a DAC system, though it can also be used as a MAC system.

Essentially TS Entitlements is a more sophisticated grant management system for what is otherwise remarkably similar to SMACK.

Abstract API for Smack

  • check(process, object) -> boolean
  • check(user, label, verb) -> boolean
  • setLabel(object, label)
  • getLabel(object) -> label
  • grantUser(user, label, Set<verb>))
  • grantGroup(group, label, Set<verb>))
  • listGrants(label) -> {Set<{user, Set<verb>}>, Set<{group, Set<verb>}>}

Comparison to filesystem ACLs

Filesystem ACLs generally come in two flavors:

  • NTFS-style ACLs. These include ZFS- and NFSv4-style ACLs. The differences between the three are quite minor and not relevant here.
  • POSIX Draft ACLs. This is a draft standard that was never finished but which Linux supports very well.

NTFS-style ACLs can have negative entries that deny certain verbs to specific users (or groups of users). Generally NTFS-style ACLs are reordered when they are set on a file such that DENY ACEs come first followed by ACEs that grant access. ACL evaluation is linear due to the kernel not knowing whether the ACL's ACEs have been sorted: for each ACE in order the system will check if the user process credentials (user ID, supplemental user group ID list) and the requested verb matches the ACE. The first matching ACE determines the result: if it's a DENY ACE then access is denied, else it is granted. If no ACE matches then access is denied. If the ACL could be counted on to be sorted with DENY ACEs first, and also by SACL vs DACL, and with each set of ACEs, DENY and GRANT, further sorted by grantee security identifier (SID) then the kernel could use a fast set intersection algorithm just like TS Entitlements uses. A further optimization would be to intern SIDs so that smaller (32-bit) integers could be used in the processes' access tokens and files' security descriptors (which include the ACLs). ZFS, for example, does intern SIDs using 64-bit "FUID" identifiers, but ZFS ACLs cannot be counted on to be sorted anymore than NTFS ACLs can be.

POSIX Draft ACLs do not have deny entries, but only the verbs granted by the most-specific matching ACE are granted.

NTFS-/ZFS-/NFSv4-style ACLs support 16 verbs using 16-bit bitmasks. Windows repurposes the security descriptor (which includes the ACL) in other applications, with different meanings for the 16-bit bitmask's verbs for different applications.

POSIX Draft ACLs support 3 verbs (read, write, execute/access) using 3-bit bitmasks.

In all cases the ACL is metadata stored along with the file's (or other object's) metadata.

TS Entitlements:

  • Moves the ACL definition to an enterprise directory and replaces the contents of the ACL in file/object metadata with the ACL's name.
  • Allows unlimited numbers of verbs.
  • Adds a notion of role that is a set of verbs to make grants more flexible, allowing for refactoring of roles and grants w/o having to change application code (and vice-versa), and relieving users of having to think in terms of verbs when roles are better.

NTFS-style ACLs support both, DAC and MAC. TS Entitlements does as well, though it is currently only used for DAC: apps can store separate labels for DAC and MAC, or if need be we could implement a pseudo-label that combines two labels.

The core idea here is that it makes little sense to store ACLs in files (and other objects) when typically most files a user owns have the same ACLs. The downside of naming ACLs and storing those names instead of the ACLs' contents in the files' metadata is this: users have to be able to pick from a reasonably small size set of labels when setting an object's label, and they need a notion a context-specific default label. In a traditional filesystem the context-specific default ACL is provided by the directory containing any new file or directory, and users don't have to know any names for ACLs. In this sense TS Entitlements complicates the user experience, though the antidote for this is to ensure that the cardinality of frequently-used labels for each user is small.

Abstract API for filesystem ACLs

  • check(process, object) -> boolean
  • check(user, supplementary groups, ..., ACL, verb) -> boolean
  • setacl(object, acl)
  • getacl(object) -> acl

ACL construction typically has no API as such, but we can construct an abstract API for it anyways:

  • makeEmptyACL() -> acl
  • grantUser(user, acl, Set<verb>)) -> acl
  • grantGroup(group, acl, Set<verb>)) -> acl
  • listGrants(acl) -> {Set<{user, Set<verb>}>, Set<{group, Set<verb>}>}

Comparison to RBAC

The [RBAC] systems that I am intimately familiar with, Solaris RBAC and UName*It's2 RBAC, are very coarse-grained: grants are of roles across a large domain of objects. In Solaris RBAC the scope of a grant is either domain-wide or host-local. In UName*It grant scope is domain- or subdomain-wide. In both cases there are no verbs, with the scope of roles in the sense of actions allowed is documented in natural language.

TS Entitlements allows many grants of the same role on different sets of objects. The set of objects that a grant relates to is identified by the objects themselves as their metadata records the name of a label. In this way TS Entitlements allows for very fine-grained RBAC-style access control, though it also allows very coarse-grained access control -- the degree of granularity is controlled by applications' developers and/or resource owners/operators, or both.

Abstract API for RBAC

There are numerous RBAC systems, so constructing an abstract API that is faithful to all of them will not be possible. Here is an approximation:

  • check(user, role) -> boolean or check(user, role, scope)
  • createRole(role)
  • grantRole(user, role, scope)
  • revokeRole(user, role, scope)

where scope is some sort of fairly coarse-grained identifier for a large domain of objects, such as a NIS/NIS+/LDAP/DNS/etc. domain.

Comparison to OAuth-style protocols

[OAuth]-style protocols generally involve a cryptographic token that conveys "claims" about the authenticated subject where the claims should be sufficient for the relying party to perform authorization, and the claims may or may not identify the subject (i.e., the subject's identity may be withheld from the relying party). The claims could be voluminous if the issuer is configured to return information about large scopes of objects at the relying party that the subject may access, or the claims can be small and relating to just one object at the relying party. For the latter case the application may direct the subject's user-agent to fetch a new token for each object accessed by the subject.

TS Entitlements is not currently an OAuth-style protocol, though it can be a protocol (see also below), and OAuth-style protocols that can use "labels" to identify objects could work with TS Entitlements. This would be one way of federating TS Entitlements. This would also add a capability model to TS Entitlements without changing its API substantially as the subject argument to check() can already be a JWT, therefore the only thing missing is for check() to be able to throw an exception or return an error such that the application can issue an appropriate redirect (in HTTP applications anyways) to an issuer that can produce a token that contains the requested access claim.

In our current implementation there is no federation support, and relying parties must have a great deal of authorization information locally available. We currently rely on OAuth tokens identifying the subject in order to then use the TS Entitlements API as it can then find the subject's group memberships etc.

Constructing new primitives from the checkCore() primitive

The ABAC functionality in TS Entitlements is constructed from the checkCore() primitive and pseudo-verbs and special grantees. That is, the check() primitive that applications invoke itself invokes checkCore() at least once, or possibly multiple times. Note that because of caching this does not significantly slow down check().

We can think of a number of authorization functions that we don't yet currently implement but which we could implement entirely based on checkCore(). For example, we might want a form that takes two subjects, one being an impersonator and the other being the subject to be impersonated by the impersonator. Such a check() construction might check whether the impersonated is allowed the requested access and also whether impersonator is allowed to generic:IMPERSONATE (or perhaps whether the impersonator is allowed the requested access, or both). A similar feature for cross-origin request sharing (CORS) could see us implement a subject type corresponding to Referer (i.e., URI authority) to an user entity ID for running automation (we call these "role accounts") that is then used as a subject.

Similarly we can implement useful functions in terms of query(), such as functions that return a small number of labels from which a user might pick one in a picker UI element.

As noted above we could also have a MAC variant of check() that takes two labels, and which implements mandatory access control policies using a MAC label, and discretionary policies using a DAC label. Or we could use a single composite label that encodes both MAC and DAC labels, but overloading label names like this makes for a bad UI as users would be expected to understand the MAC label naming convention. Still, using a MAC label naming convention might be appropriate where applications can only store a single label per-object.

There are ABAC attributes that we don't but we easily could implement, such as the following taken from the Wikipedia page on ABAC:

  • time-of-day restrictions,
  • department restrictions,
  • clearance level restrictions,
  • object type restrictions.

Some of these additional ABAC features can be denoted by user group memberships and grants to them, others can be denoted by special entities and grants thereto, and some might require schema changes (e.g., clearance level). I believe we can implement all the kinds of attributes and policies given as examples on the Wikipedia ABAC page except for generalized time-of-day restrictions, and most useful attributes and policies that one could think of, though in some cases that might require adding functions to the API.

The reason that generalized time-of-day restrictions are hard to implement solely in terms of checkCore() and special verbs and/or special grantees is that it is difficult to define arbitrary time ranges with just those few tools. For example, one might have generic:AFTER_<NNNN> and generic:BEFORE_<NNNN> pseudo-verbs to define time ranges, but having to checkCore() all of those verbs will get expensive. On the other hand one could have a generic:BUSINESS_HOURS_ONLY pseudo-verb where the time range "business hours" is defined elsewhere, and just a few such ranges might suffice.

Any desirable policy that one can imagine implementing in terms of checkCore() can therefore be so implemented, either in TS Entitlements' check() itself or in the applications that use it. Naturally it is best to have a single library that provides all the variants of check() and all the policy logic, but applications can use the TS Entitlements check() primitive to implement policies which we don't already implement in TS Entitlements. The checkCore() primitive, and the check() function, are something of a "mecano set" that can be used to construct much richer and more complex policies than just checkCore().

Comparison to ABAC systems

I have no experience with any other [ABAC] systems, and for that reason I will not do an in-depth comparison here. But from what I can tell ABAC is always a feature that is added to or used in conjunction with other authorization systems. That is, ABAC systems are not generally sufficient on their own. In TS Entitlements ABAC is indeed an add-on feature.

Conclusion and future directions

TS Entitlements is a relational and generic authorization system using natural language analogies (subject-verb-object, or, rather, subject-verb-label) for representation of extant access controls and to check access controls as needed. This system has served Two Sigma Investments, LP well for over a decade. There are other authorization schemes in use at Two Sigma that are specific to certain applications and use-cases, but by and large TS Entitlements is widely used.

Because the primitives in TS Entitlements are powerful but simple and can be combined to create richer and more complex policies, TS Entitlements is quite flexible. Though any such system has some inherent limitations.

TS Entitlements being an evolution of several radically different general-purpose authorization systems makes it likely that its design is widely applicable.

Being inherently relational, TS Entitlements implementations can be created quite simply in SQL and other relational languages. Outside of any RDBMS the complexity of any implementation of a TS Entitlements alike system will lie mainly in a) any synchronization of enterprise directories and/or "triggers" if applicable, b) the extraction of data and incremental updates streams, c) transformation to forms optimized for fast check() implementations, d) distribution of the same, and lastly e) the implementation of the API. Using SQLite3 for a local database can greatly simplify (c), (d), and/or (e) by using existing tooling (like SQLite3 itself) and enabling a trivial SQL-based implementation of checkCore(). In other words, the most complex part of any implementation of a TS Entitlements alike system will be integration with the implement's choice of enterprise directory.

Perhaps the single most important thing to do to TS Entitlements to make it generally usable outside Two Sigma is to add support for federation using multiple different distributed sources of truth, possibly designing, specifying, and standardizing a set of APIs, conventions (for standard verbs and roles), and protocols.

Federated entitlements using URIs, OAuth, and/or PKIX

In order to support federated entitlements, it might be a good idea to have URN/URI forms for verbs and roles (URNs for well-known, pre-defined ones, URIs otherwise), and URI forms for labels and grants, along with new URI schemes for verbs, roles, and labels/grants. Users and application developers would still use <app>:<VERB> and <app>:<Role> naming, but would configure mappings of those to URNs/URIs.

For example, a pre-defined verb like tsents:OWN could have a canonical URN like urn:ietf:rfc:XXXX:tsents:OWN if the system were published as an Internet RFC, and a custom verb like Kafka:CREATE_TOPIC could be verb://domain.example/Kafka:CREATE_TOPIC. Similarly for generic roles.

For labels (and grants) we might have a label: scheme with an authority component: label://domain.example/Some/Label/Here. A label: scheme should be able to map label: URIs to https: URIs using an appropriate service discovery mechanism.

For example, label://domain.example/Some/Label/Here might map to https://authz.domain.example/labels/Some/Label/Here.

A service discovery mechanism for mapping label: scheme URIs to https: scheme URIs would have to deliver to the client:

  • one or more authorities to use for the https: URIs corresponding to a label: scheme URI's authority,
  • an optional URI local-part prefix to prepend to the label: URI's local-part

Service discovery could be done with a DNS-based discovery mechanism and/or with .well-known/ https:-scheme URIs with the label:-scheme URI's authority. For example, a GET of https://domain.example/.well-known/entitlements.json might yield a JSON text with a key listing origins and another specifying a local-part prefix. Or a DNS URI-type Resource Record lookup might be used to find the origins, and some as-yet-unspecified DNS Resource Record type might be used to discover any URI local-part prefixes, or simply hard-code a prefix as /entitlements/ or /authorization/.

For grants we might use a label: scheme URI w/ query parameters (q-params) for the role or verb. Thus an HTTP GET of an https: URI mapped from label://domain.example/Some/Label/Here?role=tsents:Owner would return the grantees if the user-agent were authorized to see them, while a PUT, POST, or PATCH might create or alter the grant (if the user-agent were authorized to alter the set of grants). A HEAD of label://domain.example/Some/Label/Here?verb=tsents:OWN&subject=some_username_here would implement check("some_username_here", "tsents:OWN", "label://domain.example/Some/Label/Here"), while a GET of the same would implement the same function and also return additional information, such as related grants to enable caching for client-side evaluation of related check() calls the client might want to do soon after the GET. A GET of a label: URI w/ grant q-params might also return a cryptographic token that the user-agent can present to applications, and which token could act as an access certificate.

As noted above OAuth-style protocols could also be adapted to federate TS Entitlements. For example, when performing some operation on an HTTP resource the origin could redirect the user-agent to fetch a code via OIDC that can be used to demonstrate that the user has the requested access. The redirect would be to a URI that denotes the arguments to the check() call to be done by the code issuer, and the resulting code and token would indicate whether the user-agent has the requested access. Token requests need not be HTTP redirects.

A set of claims could be specified for JSON Web Tokens (JWTs) to carry assertions of one or more grants.

A PKIX access certificate attribute types could be specified to carry one or more assertions of grants for applications that use PKIX access certificates.

A complete set of protocols could be designed and standardized using the above sketch as a starting point.

Use-case: federated access controls for healthcare data

Imagine a world where users can create labels for their healthcare data then supply those labels to healthcare providers for use in authorizing access to their data. Thus a user might have a label for data related to some surgery, and different labels for data related to unrelated surgeries, dental health, general health, endocrinology, etc. A user could then manage grants on their data, making it very easy to share data with -for example- a doctor for a second opinion, or when moving from one practice to another.

Using URIs for labels and grants would allow for third party authorization providers. Aggressive caching could be used to cope with provider failures and to recover by moving to other providers if need be.

A standard protocol for this use-case could revolutionize healthcare and privacy.

Other use-cases abound.

References

[NTFS], the access control system for Windows' NTFS filesystem, Active Directory, and other products.

[[IEEE]] Institute of Electrical and Electronics Engineers, "IEEE 1003.1e and 1003.2c: Draft Standard for Information Technology--Portable Operating System Interface (POSIX)--Part 1: System Application Program Interface (API) and Part 2: Shell and Utilities, draft 17", October 1997.

[NFSv4 ACLs] "Network File System (NFS) Version 4 Minor Version 1 Protocol" section 6.2.1, S. Shepler (Editor) et. al., January 2010.

[NFSv4 POSIX Draft ACLs] "POSIX Draft ACL support for Network File System Version 4, Minor Version 2", R. Macklem, March 2025.

[ZFS ACLs] "Oracle Solaris 11.1 Administration: ZFS File Systems", "Solaris ACL Model", Oracle, January 2005.

[RBAC] "Role-based access control", Wikipedia.

[ABAC] "Attribute-based access control", Wikipedia.

[MLS] "Multilevel security", Wikipedia

[SMACK], "The Simplified Mandatory Access Control Kernel (Whitepaper)", C. Schaufler, April 2008.

[OAUTH] "The OAuth 2.0 Authorization Framework", D. Hardt (Editor), October 2012.

[JWT] "JSON Web Token (JWT)", M. Jones et. al., May 2015.

[CWT] "CBOR Web Token (CWT)", M. Jones et. al., May 2015.

[ACLs don't] "ACLs don’t", Tyler Close, January 2009.

[CDB] "cdb", D. J. Bernstein, unknown publication date.

Footnotes

  1. In a capability system a trusted component makes an authorization check and issues an unforgeable capability that can be passed from process to process and which causes access to be granted to any process holding that capability, and actual access controls consist of validating the capability and checking that it specifically permits the requested access. Any "ACL" system can be turned into a capability system. Capability systems avoid confused-deputy vulnerabilities, and preserve privacy (because capabilities need not name the subject whose access they attest to).

  2. UName*It was an enterprise directory service that was available in the 1990s. It's authorization system was an RBAC system with pre-defined roles, no verbs, whose granularity was at the level of "domains", where a domain corresponds to a DNS domainname with any number of hosts and users and other resources in it.

Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
\hypertarget{abstract}{%
\section{Abstract}\label{abstract}}
TS Entitlements is a dial-tone and very fast application-level
authorization system designed and implemented at Two Sigma Investments,
LP which is inspired by labeled security systems, mainly
\protect\hyperlink{SMACK-ref}{SMACK} (the Simplified Mandatory Access
Control Kernel), traditional filesystem ACL systems such as
\protect\hyperlink{NTFS-ACLs-ref}{NTFS ACLs}, role-based access control
systems (\protect\hyperlink{RBAC-ref}{RBAC}), and attribute-based access
control systems (\protect\hyperlink{ABAC-ref}{ABAC}). It could be said
that TS Entitlements evolves and ``unifies'' all of these.
The core primitives in TS Entitlements are a subject-verb-label
authorization check where the label is a name stored in or derived from
an actual object's metadata, and granting/revocation of access by
creating/deleting grantee-role-label triples where ``roles'' are named
sets of ``verbs''. A ``subject'' is a user's authentication context (or
capability), while a ``grantee'' can be a user ID, a user group, or
other special entities. With these three primitives and a very simple
schema we manage to build much richer policies and schema including ABAC
functionality, and we let applications have varying levels of access
controls granularity, all while yielding excellent authorization check
performance.
\hypertarget{disclaimer}{%
\section{Disclaimer}\label{disclaimer}}
This document is being distributed for informational and educational
purposes only and is not an offer to sell or the solicitation of an
offer to buy any securities or other instruments. The information
contained herein is not intended to provide, and should not be relied
upon for, investment advice. The views expressed herein are not
necessarily the views of Two Sigma Investments, LP or any of its
affiliates (collectively, ``Two Sigma''). Such views reflect the
assumptions of the author(s) of the document and are subject to change
without notice. The document may employ data derived from third-party
sources. No representation is made by Two Sigma as to the accuracy of
such information and the use of such information in no way implies an
endorsement of the source of such information or its validity.
The copyrights and/or trademarks in some of the images, logos or other
material used herein may be owned by entities other than Two Sigma. If
so, such copyrights and/or trademarks are most likely owned by the
entity that created the material and are used purely for identification
and comment as fair use under international copyright and/or trademark
laws. Use of such image, copyright or trademark does not imply any
association with such organization (or endorsement of such organization)
by Two Sigma, nor vice versa.
\hypertarget{table-of-contents}{%
\section{Table of Contents}\label{table-of-contents}}
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
\protect\hyperlink{introduction}{Introduction}
\item
\protect\hyperlink{generic-authorization-systems}{Generic
authorization systems}
\item
\protect\hyperlink{applicability}{Applicability}
\item
\protect\hyperlink{ts-entitlements}{TS Entitlements}
\begin{enumerate}
\def\labelenumii{\arabic{enumii}.}
\tightlist
\item
\protect\hyperlink{abstract-api}{Abstract API}
\item
\protect\hyperlink{verbs-roles-and-labels}{Verbs, roles, and labels}
\item
\protect\hyperlink{architecture}{Architecture}
\item
\protect\hyperlink{enterprise-directory-schema-for-ts-entitlements}{Enterprise
directory schema for TS Entitlements}
\item
\protect\hyperlink{schema-optimized-for-access-control-evaluation}{Schema
optimized for access control evaluation}
\item
\protect\hyperlink{checkcore-in-sql}{\texttt{checkCore()} in SQL}
\item
\protect\hyperlink{incremental-materialized-view-maintenance}{Incremental
materialized view maintenance}
\item
\protect\hyperlink{making-the-c-coded-checkcore-fast}{Making the
C-coded \texttt{checkCore()} \emph{fast}}
\item
\protect\hyperlink{dial-tone-architecture}{Dial-tone architecture}
\item
\protect\hyperlink{data-distribution}{Data distribution}
\item
\protect\hyperlink{abac-functionality-in-ts-entitlements}{ABAC
functionality in TS Entitlements}
\item
\protect\hyperlink{dog-fooding}{Dog-fooding}
\item
\protect\hyperlink{odds-and-ends}{Odds and ends}
\item
\protect\hyperlink{negative-permissions}{Negative permissions}
\item
\protect\hyperlink{uiux-considerations}{UI/UX considerations}
\item
\protect\hyperlink{sample-verbs-and-roles}{Sample verbs and roles}
\item
\protect\hyperlink{auditing-extant-authorizations}{Auditing extant
authorizations}
\end{enumerate}
\item
\protect\hyperlink{limitations-of-the-checkcore-subject-verb-object-model}{Limitations
of the \texttt{checkCore()} subject-verb-object model}
\item
\protect\hyperlink{comparison-to-other-generic-authorization-systems}{Comparison
to other generic authorization systems}
\begin{enumerate}
\def\labelenumii{\arabic{enumii}.}
\tightlist
\item
\protect\hyperlink{comparison-to-mls}{Comparison to MLS}
\begin{enumerate}
\def\labelenumiii{\arabic{enumiii}.}
\tightlist
\item
\protect\hyperlink{abstract-api-for-mls}{Abstract API for MLS}
\end{enumerate}
\item
\protect\hyperlink{comparison-to-smack}{Comparison to SMACK}
\begin{enumerate}
\def\labelenumiii{\arabic{enumiii}.}
\tightlist
\item
\protect\hyperlink{abstract-api-for-smack}{Abstract API for SMACK}
\end{enumerate}
\item
\protect\hyperlink{comparison-to-filesystem-acls}{Comparison to
filesystem ACLs}
\begin{enumerate}
\def\labelenumiii{\arabic{enumiii}.}
\tightlist
\item
\protect\hyperlink{abstract-api-for-filesystem-acls}{Abstract API
for filesystem ACLs}
\end{enumerate}
\item
\protect\hyperlink{comparison-to-rbac}{Comparison to RBAC}
\begin{enumerate}
\def\labelenumiii{\arabic{enumiii}.}
\tightlist
\item
\protect\hyperlink{abstract-api-for-rbac}{Abstract API for RBAC}
\end{enumerate}
\item
\protect\hyperlink{comparison-to-oauth-style-protocols}{Comparison
to OAuth-style protocols}
\item
\protect\hyperlink{comparison-to-abac-systems}{Comparison to ABAC
systems}
\end{enumerate}
\item
\protect\hyperlink{constructing-new-primitives-from-the-checkcore-primitive}{Constructing
new primitives from the \texttt{checkCore()} primitive}
\item
\protect\hyperlink{conclusion-and-future-directions}{Conclusion and
future directions}
\begin{enumerate}
\def\labelenumii{\arabic{enumii}.}
\tightlist
\item
\protect\hyperlink{federated-entitlements-using-uris-oauth-andor-pkix}{Federated
entitlements using URIs, OAuth, and/or PKIX}
\begin{enumerate}
\def\labelenumiii{\arabic{enumiii}.}
\tightlist
\item
\protect\hyperlink{use-case-federated-access-controls-for-healthcare-data}{Use-case:
federated access controls for healthcare data}
\end{enumerate}
\end{enumerate}
\item
\protect\hyperlink{references}{References}
\end{enumerate}
\hypertarget{introduction}{%
\section{Introduction}\label{introduction}}
This blog/paper is about TS Entitlements, an authorization system that I
had a large part in designing, implementing, operating, and maintaining
at Two Sigma Investments, LP.
Throughout I'll be using ``abstract APIs'' to help show how TS
Entitlements works and how it is an evolution of several other generic
authorization systems.
In the sense of the \protect\hyperlink{ACLs-dont-ref}{``ACLs don't''}
paper, TS Entitlements is currently an ``ACL'' system, though
nothing\footnote{In a capability system a trusted component makes an
authorization check and issues an unforgeable capability that can be
passed from process to process and which causes access to be granted
to any process holding that capability, and actual access controls
consist of validating the capability and checking that it specifically
permits the requested access. Any ``ACL'' system can be turned into a
capability system. Capability systems avoid confused-deputy
vulnerabilities, and preserve privacy (because capabilities need not
name the subject whose access they attest to).} precludes a capability
model from being added, possibly OAuth-style.
TS Entitlements stores its authorization data in an enterprise directory
that users interact with. Naturally that directory and/or its schema are
subject to change, as long as the TS Entitlements authorization data can
be represented in the directory. Multiple radically different
directories can be supported.
The API that applications use to perform authorization checks uses
authorization data that has been extracted and transformed into a schema
optimized for authorization check performance.
The API provides a subject-verb-label metaphor for access control
checks, but a grantee-role-label metaphor for granting and revocation.
Here a ``label'' is a name string that stands for the object to be
protected, not unlike SMACK. Typically the application stores the label
in the object's metadata, though some applications \emph{derive} the
label from the object's metadata (generally some name). Between the use
of roles in granting, having symbolic labels, and sets of grants, TS
Entitlements resembles RBAC, SMACK, and filesystem ACLs, all with
enhancements. Combining the core abstraction into richer abstractions
yields ABAC functionality.
The difference in directory-side and API-side authorization data
requires an extract-transform-load (ETL) step in the product's
architecture. This ETL process is as a compiler that translates data
expressed in a convenient schema to the schema that is convenient for
the API's implementation. The API-side schema is extremely simple and
requires creativity to express ABAC policies and attributes, while the
enterprise directory schema can be richer to hide that creativity from
the user interface (UI) and user experience (UX).
\hypertarget{generic-authorization-systems}{%
\section{Generic authorization
systems}\label{generic-authorization-systems}}
An authorization system allows users to express who can do what to what
things and where, with primitives for granting access, revocation, and
checking if some access is allowed. There are many types of
authorization systems, some quite specialized and even esoteric, while
others are quite generic. Some applications require very specialized
authorization functionality (for example, TPMs), while others can
reasonably use generic authorization systems. TS Entitlements is a
\emph{generic} authorization system.
Generic authorization systems include:
\begin{itemize}
\tightlist
\item
access control lists (ACLs)
\item
role-based access control (RBAC)
\item
labeled security, typically involving mandatory access controls (MAC)
\end{itemize}
among others.
ACLs typically consist of lists of grant/deny ACL entries (ACEs), with
each entry naming a user, a group of users, a host, or other grantee
entity, and some set of verbs. Typical systems include the Windows NTFS
ACL system, where there are only 16 verbs (encoded as a 16-bit bit-mask)
that one can grant. Typically the primitives in ACL systems are:
\texttt{setacl(file,\ acl)},
\texttt{getacl(file)\ -\textgreater{}\ acl}, and
\texttt{check(user,\ file)}.
RBAC systems typically involve granting users or groups of users one or
more roles enterprise- or domain-wide. In RBAC systems roles are not
always well-defined in their semantics because typically the
\texttt{check()} primitive is of the form
\texttt{check(subject,\ role)}, and so one has to understand all the
\texttt{check()} invocations used by applications. RBAC systems are also
often coarse-grained systems where there is no way to limit roles to
subsets of large administrative domains (e.g.,
\href{https://docs.oracle.com/cd/E23824_01/html/821-1456/rbac-1.html}{Solaris
RBAC}).
Labeled security systems are generally MAC systems, though that's not
required. A MAC system forces a separation of who can set or change an
object's label (and what the default label is) from the owner of the
object -- this is typically desirable in military organizations where
classification and declassification authority is highly limited. But the
core concept of labeled security is not so much mandatory security as it
is separating the storage of the authorization metadata from the objects
it protects by giving that authorization metadata a name that objects
store in their metadata. This is especially true in SMACK where labels
are free-form text strings. Older labeled security systems embed all the
authorization metadata in the label itself, so the label is not exactly
a ``name'', but typically labels have names anyways.
TS Entitlements is inspired by all of these and is an evolution of all
of them as follows:
\begin{itemize}
\item
Take an NTFS ACL, remove the \texttt{DENY} option thus making it a set
of entries rather than a list of entries, refer to roles (which are
named sets of verbs) instead of verbs in the entries, allow unbounded
roles and verbs, give the ACL a name, store the result in a directory,
and store the name in objects' (e.g., files) metadata instead of the
actual ACL.
\item
Take RBAC, add a notion of verbs, make roles properly a named set of
verbs, add a notion of label to allow for fine-grained access control.
\item
Take SMACK, allow larger labels, replace the authorization metadata
database which stores \texttt{\{label,\ rwx,\ grantee\}} with a set of
\texttt{\{label,\ role,\ grantee\}}, and expand this database to a set
of \texttt{\{label,\ verb,\ user\}} grants for fast checking in the
kernel or application.
\end{itemize}
\hypertarget{applicability}{%
\section{Applicability}\label{applicability}}
TS Entitlements is a generic authorization system, and it can be used
creatively in many applications.
However, there are limits to the applicability of generic authorization
systems, including TS Entitlements.
For example TPM 2.0 authorization policies can't be made to refer to TS
Entitlements policies except through signed policy tickets from an
online service intended for this, but that entails trade-offs that might
not be acceptable. Another example is that application-specific
authorization systems can have lower cognitive load than generic
authorization systems, and sometimes that lower cognitive load is
sufficient to justify an application-specific authorization system.
In other words generic authorization systems like TS Entitlements are
not universal in applicability, therefore they are not a panacea. Still,
TS Entitlements is applicable to -and used by- a large swath of
applications.
\hypertarget{ts-entitlements}{%
\section{TS Entitlements}\label{ts-entitlements}}
A key goal of TS Entitlements is to support a fast \texttt{check()}
predicate that represents actions as verbs with well-defined semantics,
with an arbitrary number of verbs, and both fine-grained access controls
as well as coarse-grained access controls, with those access controls
stored in an enterprise directory.
We cannot store information about all objects in a directory, as that
would not scale, but we want to be able to get a rough idea of what each
user can do just by auditing the authorization metadata stored in the
directory. To do this we must have a) documentation embedded in the
authorization data as to what kinds of objects will be protected by each
grant, and b) relatively few grants. A full audit of authorization data
does still require iterating objects in all applications, naturally, but
a partial audit based only on the metadata stored in the directory can
paint a pretty complete picture by itself.
\hypertarget{abstract-api}{%
\subsection{Abstract API}\label{abstract-api}}
As mentioned above, TS Entitlements is a generic authorization system
with three core primitives:
\begin{itemize}
\tightlist
\item
\texttt{check(subject,\ verb,\ label)\ -\textgreater{}\ Result\textless{}Error\textgreater{}}
\item
\texttt{checkCore(subject,\ verb,\ label)\ -\textgreater{}\ boolean}
-- core authorization check function used by \texttt{check()} to
implement more complex policies
\item
\texttt{grant(label,\ role,\ grantee)}
\item
\texttt{revoke(label,\ role,\ grantee)}
\end{itemize}
and a few additional supporting primitives:
\begin{itemize}
\tightlist
\item
\texttt{query(label,\ role,\ {[}grantee{]})\ -\textgreater{}\ Set\textless{}grantee\textgreater{}}
\item
\texttt{query(subject)\ -\textgreater{}\ Set\textless{}\{label,\ verb\}\textgreater{}}
\item
\texttt{query(subject)\ -\textgreater{}\ Set\textless{}\{label,\ role\}\textgreater{}}
\item
\texttt{createRole(role)}
\item
\texttt{addVerb2Role(role,\ verb)}
\item
\texttt{createLabel(label)}
\item
etc.
\end{itemize}
As well there are primitives that TS Entitlements using
\emph{applications} must provide for themselves, such as:
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
\texttt{setlabel(object,\ label)} and
\texttt{getlabel(object)\ -\textgreater{}\ label} (fine-grained),
\item
\texttt{setlabel(volume,\ label)} and
\texttt{getlabel(volume)\ -\textgreater{}\ label} (coarse-grained),
\item
\texttt{derivelabel(object)\ -\textgreater{}\ label} (fine-grained),
\item
\texttt{derivelabel(volume)\ -\textgreater{}\ label} (coarse-grained).
\end{enumerate}
\begin{enumerate}
\def\labelenumi{(\arabic{enumi})}
\tightlist
\item
and (2) are for applications that can store a label in object
metadata; (3) and (4) are for applications that cannot. The former is
always preferable to the latter.
\end{enumerate}
Note that a \texttt{grantee} is an entity like a user or a group of
users, while a \texttt{subject} is anything from an authenticated
identity to an authentication context. For example, \texttt{check()} can
use a \protect\hyperlink{JWT-ref}{{[}JWT{]}} as a subject, can validate
the token, and extract its claims. Currently there is no capability
support in \texttt{check()}, but its design does not preclude it, as a)
a capability could be used as a subject, b) the result of
\texttt{check()} can be extended to produce capability request messages
to be sent to a peer (e.g., an HTTP redirect to a token issuer).
\hypertarget{verbs-roles-and-labels}{%
\subsection{Verbs, roles, and labels}\label{verbs-roles-and-labels}}
Verbs are English-language verbs, capitalized, such as \texttt{READ}. We
do qualify verbs with ``application'' names in order to distinguish
different semantics for the same verbs, such as \texttt{filesystem:READ}
and \texttt{database:READ}.
Roles are English-language actor words, with the first letter
capitalized, such as \texttt{Reader}, \texttt{Writer},
\texttt{Administrator}, \texttt{Owner}, and so on. As with verbs we
qualify roles with application names, but note that there is no
requirement that roles contain only verbs qualified with the same
application name.
Labels have some hierarchy to them. Currently we only have one namespace
component, but we could add more hierarchy. A typical label might be
\texttt{SecurityDevelopment::TSEntitlements}. Applications can implement
a most-specific label search using some separator for components of the
label, such as \texttt{FooNamespace::a/b/c}, where \texttt{/} is used as
a separator.
\hypertarget{architecture}{%
\subsection{Architecture}\label{architecture}}
We have several components:
\begin{itemize}
\tightlist
\item
an enterprise directory service
\item
services that perform extract, transform, and load (ETL) operations
with the authorization metadata from the directory, then produce and
distribute small databases optimized for \texttt{check()} speed, as
well as produce and distribute incremental updates to those small
databases
\item
a client that fetches those small databases and the incremental
updates to them, applying them to the local copies they keep
\item
a C-coded API that implements \texttt{check()} and \texttt{query()}
\item
foreign function interface (FFI) bindings of that C-coded API for
multiple programming languages
\end{itemize}
We also have a REST/gRPC application service that exports
\texttt{check()} and \texttt{query()} functions for applications that
cannot use the C-coded API.
Data flows from the directory to the ETL service to the clients to the
API.
\hypertarget{enterprise-directory-schema-for-ts-entitlements}{%
\subsection{Enterprise directory schema for TS
Entitlements}\label{enterprise-directory-schema-for-ts-entitlements}}
The directory schema will be very specific to the directory being used.
At Two Sigma we use an internal, proprietary directory, therefore I will
not describe the exact schema we use, but I will show a mostly
equivalent SQL schema that can be used to represent TS Entitlements:
\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{CREATE} \KeywordTok{TYPE}\NormalTok{ entity\_type }\KeywordTok{AS}\NormalTok{ ENUM (}
\StringTok{\textquotesingle{}user\textquotesingle{}}\NormalTok{, }\StringTok{\textquotesingle{}user\_group\textquotesingle{}}\NormalTok{, }\StringTok{\textquotesingle{}realm\textquotesingle{}}\NormalTok{, }\StringTok{\textquotesingle{}compartment\textquotesingle{}}\NormalTok{, }\StringTok{\textquotesingle{}special\textquotesingle{}}
\CommentTok{{-}{-} special is for entities like \textquotesingle{}ANYONE\textquotesingle{}, \textquotesingle{}SELF\textquotesingle{},}
\CommentTok{{-}{-} \textquotesingle{}COMPARTMENTED\textquotesingle{}, etc.}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ entities (}
\NormalTok{ \_name text }\KeywordTok{NOT} \KeywordTok{NULL}\NormalTok{,}
\NormalTok{ \_type entity\_type }\KeywordTok{NOT} \KeywordTok{NULL}\NormalTok{,}
\NormalTok{ \_id bigint AUTOINCREMENT,}
\CommentTok{{-}{-} Static attributes of users that are best}
\CommentTok{{-}{-} not represented as group memberships should}
\CommentTok{{-}{-} be added here:}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (\_name, \_type),}
\KeywordTok{UNIQUE}\NormalTok{ (\_id)}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ user\_group\_user\_memberships (}
\NormalTok{ \_user\_name text,}
\NormalTok{ \_user\_type entity\_type }\KeywordTok{CHECK}\NormalTok{ (}\StringTok{\textquotesingle{}user\textquotesingle{}}\NormalTok{),}
\NormalTok{ \_group\_name text,}
\NormalTok{ \_group\_type entity\_type }\KeywordTok{CHECK}\NormalTok{ (}\StringTok{\textquotesingle{}group\textquotesingle{}}\NormalTok{),}
\KeywordTok{FOREIGN} \KeywordTok{KEY}\NormalTok{ (\_user\_name, \_user\_type)}
\KeywordTok{REFERENCES}\NormalTok{ entities (\_name, \_type),}
\KeywordTok{FOREIGN} \KeywordTok{KEY}\NormalTok{ (\_group\_name, \_group\_type)}
\KeywordTok{REFERENCES}\NormalTok{ entities (\_name, \_type),}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (\_user\_name, \_user\_type,}
\NormalTok{ \_group\_name, \_group\_type)}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ user\_group\_group\_memberships (}
\NormalTok{ \_child\_name text,}
\NormalTok{ \_child\_type entity\_type }\KeywordTok{CHECK}\NormalTok{ (}\StringTok{\textquotesingle{}group\textquotesingle{}}\NormalTok{),}
\NormalTok{ \_parent\_name text,}
\NormalTok{ \_parent\_type entity\_type }\KeywordTok{CHECK}\NormalTok{ (}\StringTok{\textquotesingle{}group\textquotesingle{}}\NormalTok{),}
\KeywordTok{FOREIGN} \KeywordTok{KEY}\NormalTok{ (\_child\_name, \_child\_type)}
\KeywordTok{REFERENCES}\NormalTok{ entities (\_name, \_type),}
\KeywordTok{FOREIGN} \KeywordTok{KEY}\NormalTok{ (\_parent\_name, \_parent\_type)}
\KeywordTok{REFERENCES}\NormalTok{ entities (\_name, \_type),}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (\_child\_name, \_child\_type,}
\NormalTok{ \_parent\_name, \_parent\_type)}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ apps (\_app text }\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{, notes text);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ verbs (}
\NormalTok{ \_verb text }\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{,}
\NormalTok{ \_app TEXT }\KeywordTok{NOT} \KeywordTok{NULL} \KeywordTok{FOREIGN} \KeywordTok{KEY} \KeywordTok{REFERENCES}\NormalTok{ apps (\_app),}
\NormalTok{ \_notes TEXT}
\CommentTok{{-}{-} Static attributes of verbs should be}
\CommentTok{{-}{-} added here:}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE} \KeywordTok{roles}\NormalTok{ (}
\NormalTok{ \_role TEXT }\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{,}
\NormalTok{ \_app TEXT }\KeywordTok{NOT} \KeywordTok{NULL} \KeywordTok{FOREIGN} \KeywordTok{KEY} \KeywordTok{REFERENCES}\NormalTok{ apps (\_app),}
\NormalTok{ \_notes TEXT}
\CommentTok{{-}{-} Note that grants to specific verbs can}
\CommentTok{{-}{-} indirect through multiple roles, therefore}
\CommentTok{{-}{-} any attributes of roles must either affect}
\CommentTok{{-}{-} only the granting/revocation process, or must}
\CommentTok{{-}{-} "flow" additively to verbs.}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ role2verb (}
\NormalTok{ \_role text }\KeywordTok{FOREIGN} \KeywordTok{KEY} \KeywordTok{REFERENCES} \KeywordTok{role}\NormalTok{ (\_role),}
\NormalTok{ \_verb text }\KeywordTok{FOREIGN} \KeywordTok{KEY} \KeywordTok{REFERENCES}\NormalTok{ verb (\_verb),}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (\_role, \_verb)}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ labels (}
\NormalTok{ \_label text }\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{,}
\NormalTok{ \_notes text}
\CommentTok{{-}{-} Attributes of labels go here:}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ grants (}
\NormalTok{ \_label text }\KeywordTok{FOREIGN} \KeywordTok{KEY} \KeywordTok{REFERENCES}\NormalTok{ labels (\_label),}
\NormalTok{ \_role text,}
\NormalTok{ \_grantee bigint }\KeywordTok{FOREIGN} \KeywordTok{KEY} \KeywordTok{REFERENCES}\NormalTok{ entities (\_id),}
\CommentTok{{-}{-} Grant attributes go here, such as time{-}of{-}day}
\CommentTok{{-}{-} contraints.}
\NormalTok{ \_expires }\DataTypeTok{timestamp} \CommentTok{{-}{-} the grant is "deleted" from the}
\CommentTok{{-}{-} data used by the API when it}
\CommentTok{{-}{-} expires,}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (\_label, \_role, \_id)}
\NormalTok{); }
\end{Highlighting}
\end{Shaded}
\hypertarget{schema-optimized-for-access-control-evaluation}{%
\subsection{Schema optimized for access control
evaluation}\label{schema-optimized-for-access-control-evaluation}}
We use a variant of the following SQL schema for various systems, both
relational as well as a bespoke system based on the Constant DataBase
(CDB) hash-table-in-a-file file format:
\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ subject2id (}
\NormalTok{ subject\_name TEXT,}
\NormalTok{ subject\_type entity\_type,}
\NormalTok{ subject\_id }\DataTypeTok{integer} \KeywordTok{NOT} \KeywordTok{NULL} \KeywordTok{UNIQUE}\NormalTok{,}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (subject\_name, subject\_type)}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ subject2groups (}
\KeywordTok{id} \DataTypeTok{integer}\NormalTok{,}
\FunctionTok{group\_id} \DataTypeTok{integer}\NormalTok{,}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (}\KeywordTok{id}\NormalTok{, }\FunctionTok{group\_id}\NormalTok{)}
\NormalTok{);}
\KeywordTok{CREATE} \KeywordTok{TABLE}\NormalTok{ grants (}
\KeywordTok{label}\NormalTok{ TEXT,}
\NormalTok{ verb TEXT,}
\NormalTok{ grantee }\DataTypeTok{integer}\NormalTok{,}
\KeywordTok{PRIMARY} \KeywordTok{KEY}\NormalTok{ (}\KeywordTok{label}\NormalTok{, verb, grantee)}
\NormalTok{);}
\end{Highlighting}
\end{Shaded}
where the roles referenced by grants have been expanded into verbs, and
where the group nesting transitive closure is expanded in the
\texttt{subject2groups} table. Every subject will have an entry in
\texttt{subject2groups} for it being a member of itself.
This is used to make \texttt{checkCore()} very fast.
\hypertarget{checkcore-in-sql}{%
\subsection{\texorpdfstring{\texttt{checkCore()} in
SQL}{checkCore() in SQL}}\label{checkcore-in-sql}}
We do have a SQL-coded \texttt{checkCore()} for use in database
applications. It looks like this:
\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{SELECT} \KeywordTok{EXISTS}\NormalTok{ (}
\KeywordTok{SELECT} \DecValTok{1}
\KeywordTok{FROM}\NormalTok{ entities e}
\KeywordTok{JOIN}\NormalTok{ subject2groups s2g}
\KeywordTok{ON}\NormalTok{ e.\_id }\OperatorTok{=}\NormalTok{ s2g.\_id}
\KeywordTok{JOIN}\NormalTok{ grant2grantees g2g}
\KeywordTok{ON}\NormalTok{ s2g.\_group\_id }\OperatorTok{=}\NormalTok{ g2g.\_grantee}
\KeywordTok{WHERE}\NormalTok{ e.\_type }\OperatorTok{=} \StringTok{\textquotesingle{}user\textquotesingle{}} \KeywordTok{AND}\NormalTok{ e.\_name }\OperatorTok{=}\NormalTok{ \_the\_subject }\KeywordTok{AND}
\NormalTok{ g2g.\_label }\OperatorTok{=}\NormalTok{ \_the\_label }\KeywordTok{AND}\NormalTok{ g2g.\_verb }\OperatorTok{=}\NormalTok{ \_the\_verb}
\NormalTok{ );}
\end{Highlighting}
\end{Shaded}
where
\begin{itemize}
\tightlist
\item
\texttt{subject2groups} is a \texttt{MATERIALIZED\ VIEW} mapping every
user ID to every group's ID that they belong to directly or indirectly
via nested group memberships, and
\item
\texttt{grant2grantees} is a \texttt{MATERIALIZED\ VIEW} with roles
expanded to verbs.
\end{itemize}
Note in particular that the \texttt{subject2groups} materialized view
lists each subject's user group memberships, but also their own user
entity ID as well as the \texttt{ANYONE} special entity's ID. This
allows the \texttt{JOIN} in the above query to match grants to the user
as well as grants to the \texttt{ANYONE} special entity, not just grants
to user groups.
These expansions of the nested group transitive closure and the
role-verb assignments plus the materializations of these expansions
makes \texttt{check()} quite fast.
I elide the SQL definitions of these materialized views. The interested
reader can define them themselves using the descriptions above.
\hypertarget{incremental-materialized-view-maintenance}{%
\subsection{Incremental materialized view
maintenance}\label{incremental-materialized-view-maintenance}}
Our materialized views are updated using incremental updates from the
enterprise directory service as we go so as to avoid having to
re-materialize the views. The code to update the materializations is
hand-coded. To avoid restrictions imposed by the RDBMS when using
\texttt{MATERIALIZED\ VIEW}s we use regular tables and implement our own
materialization functionality, including
\texttt{REFRESH\ MATERIALIZED\ VIEW\ \textless{}name\textgreater{}\ CONCURRENTLY}.
We use something similar to
\href{https://github.com/twosigma/postgresql-contrib/blob/master/mat_views.sql}{mat\_views.sql}.
\hypertarget{making-the-c-coded-checkcore-fast}{%
\subsection{\texorpdfstring{Making the C-coded \texttt{checkCore()}
\emph{fast}}{Making the C-coded checkCore() fast}}\label{making-the-c-coded-checkcore-fast}}
Abstractly the key to making \texttt{check()} fast is that our access
controls are \emph{sets} of entries rather than \emph{lists} of entries.
Concretely though we also expand the set of
\texttt{\{label,\ role,\ grantee\}} entries to
\texttt{\{label,\ verb,\ grantee\}} entries, and we make it very fast to
find subjects' lists of grantee IDs corresponding to the user IDs and
the group IDs, just as in the SQL-coded \texttt{checkCore()} above, and
we implement exactly the same logic as in that SQL-coded
\texttt{checkCore()}. Naturally these \texttt{subject2groups} and
\texttt{grant2grantees} databases are all indexed.
The C-coded \texttt{checkCore(subject,\ verb,\ label\}} then does the
following:
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
looks up the \texttt{\{label,\ verb\}}'s set of grantees (which is
stored as a sorted array of IDs),
\item
looks up the \texttt{subject}'s set of grantee IDs (which is also
stored as a sorted array of IDs),
\item
if the intersection of the two sorted entity ID arrays is empty then
access is denied, else access is granted
\end{enumerate}
If one of the two grantee ID arrays is much smaller than the other then
we iterate through the smaller one and join it to the larger one with a
binary search, stopping when we find one match or no matches. If both
grantee ID arrays are similar in size then we walk a cursor through each
looking for a match. Users typically have a few hundred group
memberships, but \texttt{\{label,\ verb\}}s typically have only a dozen
grantees, therefore a typical \texttt{check()} is \texttt{O(N\ log\ M)}
where \texttt{N\ \textless{}\textless{}\ M}, which is quite fast.
Typical \texttt{checkCore()} calls take on the order of 15us to
complete. Although the system is fast by design, we do leave some
performance on the table because it's plenty fast enough, and we didn't
optimize it further. For example, we use 64-bit entity IDs but 32-bit
entity IDs would have sufficed, therefore we waste a lot of memory
bandwidth. For another example we make sure that we can align these
arrays on 64-bit boundaries w/o additional memory allocations, but we
don't make sure that the arrays are indeed always aligned on 64-bit
boundaries on disk, thus we sometimes incur \texttt{memmove()}
penalties. We could probably reduce the cost of \texttt{checkCore()} a
great deal, but we just haven't needed to.
The local indexed database consists of CDB (``constant database'') files
\protect\hyperlink{CDB-ref}{{[}CDB{]}}, using
\href{https://www.corpit.ru/mjt/tinycdb.html}{TinyCDB} as the
implementation. A CDB file is a read-only hash table in a file. The keys
are either integer entity IDs or entity names (name and entity type).
The values are sorted arrays of entity IDs or sorted arrays of entity
names (name and type), with the latter only needed for \texttt{query()}.
Incremental updates are used to generate new CDBs based on the current
ones and those updates, and then the new CDBs which are then renamed
into place.
The API takes great pains to ensure that only up to two instances of
CDBs are open at any given time, one being the current CDBs and the
other being the previous CDBs, all while being multi-threaded and
thread-hot. We use a user-land read-copy-update (RCU) like facility to
do this.
These CDBs total around 100MB, and they are mapped into memory.
One CDB maps subject names to their entity IDs. Another maps subject
entity IDs to their list of supplementary entity IDs (group memberships,
with nested user group transitive closure expanded). And another maps
\texttt{\{label,\ verb\}} grants (i.e., with \texttt{role}s expanded) to
direct grantees. Thus the core \texttt{check()} primitive simple does
three CDB lookups and a fast sorted array empty intersection check.
\hypertarget{dial-tone-architecture}{%
\subsection{Dial-tone architecture}\label{dial-tone-architecture}}
To make the system dial-tone we distribute the authorization data in its
optimized and indexed form to all clients so that \texttt{check()} can
be 100\% local, with incremental updates distributed and applied to the
local caches asynchronously from the directory where the authorization
metadata is stored.
We do also have a REST/gRPC service that can be called from clients that
do not have local data or which cannot use our C implementation of the
TS Entitlements API. There are a variety of programming languages with a
strong preference for not using the C run-time, such as Rust and Go. The
REST/gRPC service can run locally, using local data, to maintain the
dial-tone characterization of the system's availability.
\hypertarget{data-distribution}{%
\subsection{Data distribution}\label{data-distribution}}
The authorization data (CDB files) and incremental updates thereto are
distributed over HTTPS. The ETL service writes incremental updates,
metadata, and heartbeat messages to a file that the clients fetch with
HTTP \texttt{GET} range and conditional request headers using weak
ETags. The \texttt{Range:} header specifies a starting offset and leaves
the end offset unspecified, and the server maintains the \texttt{GET}
response body open as long as the file to which the updates are written
remains in place -- the \texttt{GET} response ends when the file is
deleted or renamed away. Clients can use the ETL service's promise of
heartbeats, and missing heartbeats, to decide to reconnect.
\hypertarget{abac-functionality-in-ts-entitlements}{%
\subsection{ABAC functionality in TS
Entitlements}\label{abac-functionality-in-ts-entitlements}}
We currently support use of several attributes of the subject, the
label, and the grant in making access control decisions:
\begin{itemize}
\tightlist
\item
User attributes (contextual):
\begin{itemize}
\tightlist
\item
the realm of the client user (e.g., Kerberos realm name, JWT issuer
name, etc.)
\item
the compartment of the client user workload
\item
whether the user has performed multi-factor authentication (MFA)
\end{itemize}
\item
User attributes:
\begin{itemize}
\tightlist
\item
on-call schedule, enabling the user to elide related approval
requirements during off hours when on-call for a product
\end{itemize}
\item
Grant attributes
\begin{itemize}
\tightlist
\item
the set of realms that the grant is constrained to
\item
whether the grant is ``compartmented'' and the set of compartments
the grant is constrained to
\item
whether the grantee requires approval from an approver
\item
whether the grantee requires multi-factor authentication (MFA)
\end{itemize}
\end{itemize}
A realm is an administrative domain, and corresponds to such things as:
Kerberos realm names, PKIX issuer names, JWT issuer names, etc.
A compartment is a set of nodes or clusters on which corresponding
workloads run. These are used for isolation -- a sort of
application-level firewall.
We express constraint attributes on grants as grants to special grantees
representing:
\begin{itemize}
\tightlist
\item
realms
\item
compartments
\item
``needs approval''
\item
``needs MFA''
\end{itemize}
We also represent some attributes as grants to normal grantees, such as:
\begin{itemize}
\tightlist
\item
``is on-call''
\end{itemize}
On-call grants are made and revoked when each user's on-call shifts
start and end.
A user who is on-call for some product might get to perform operations
after hours without express approval in spite of any ``needs approval''
policies, though subject to ex-post review the next day.
Attributes of subjects are denoted by the authentication system used.
For example, when using \protect\hyperlink{OAuth-ref}{{[}OAuth{]}} w/
\protect\hyperlink{JWT-ref}{{[}JWT{]}} or
\protect\hyperlink{CWT}{{[}CWT{]}} we encode the compartment of the user
as the string value of a \texttt{cmpt} claim. The realm of a user is
taken from the domain, realm, or issuer of the user's ticket,
certificate, or token, depending on the authentication method used. We
rely on identifying claims to look up the user's entitlements for the
purpose of evaluating \texttt{check()} calls.
Some attributes require the ability to indicate ``access granted subject
to conditions \ldots{}'', where conditions include:
\begin{itemize}
\tightlist
\item
needs approval (two-person system)
\item
needs MFA
\end{itemize}
We have a way to indicate that use of compartments is optional, which we
used when migrating applications to the compartment system. Applications
that are in the process of migrating might allow but warn about access
from outside approved compartments so that operators can investigate and
alter compartment composition until the application is ready to run with
mandatory compartment enforcement.
Thus our \texttt{check()} primitive in reality does not return
\texttt{boolean}:
\begin{itemize}
\tightlist
\item
In C \texttt{check()} returns an integer status code representing
success (access granted), failure (access denied), various errors, and
even results like conditional access grant with multiple conditions
such as ``needs approval'', ``needs MFA'', etc.
\item
In languages that support exceptions \texttt{check()} returns
\texttt{boolean} or \texttt{void} and throws exceptions for errors and
conditional access grants.
\item
In languages that support algebraic data types but not exceptions we
use something like \texttt{Either} with an error object in the failure
case.
\end{itemize}
Many more attributes could be implemented in this way. The important
thing is that we built a rich ABAC system using our simple
\texttt{check(subject,\ verb,\ label)} primitive.
We also intend to eventually support attributes of the grants
themselves, such as grant expiration.
We use natural language concepts like \texttt{subject}, \texttt{verb},
\texttt{object} (by proxy, via \texttt{label}). What should we call
attributes? I think we can call attributes:
\begin{itemize}
\tightlist
\item
\texttt{adjective}s when attached to grantees or labels
\item
\texttt{adverb}s when attached to grants
\end{itemize}
We could add a \texttt{clearance\_level} attribute to match the
Multi-Level Security \protect\hyperlink{MLS-ref}{{[}MLS{]}} labeled
security system. Here users would have a specific and global clearance
level, and the labels attached to objects would have a minimum clearance
level attribute. Users would have to have a clearance level at least as
high as the label requires in order to even see that the object exists,
let alone access it in any way. We don't need this MLS functionality,
but other organizations do.
\hypertarget{dog-fooding}{%
\subsection{Dog-fooding}\label{dog-fooding}}
It is possible to use this system to permission granting and revocation
operations by using verbs such as:
\begin{itemize}
\tightlist
\item
\texttt{tsents:OWN} -- denotes ownership by the grantee
\item
\texttt{tsents:GRANT} -- if granted denotes permission of the grantee
to grant roles they have on the label to others except for granting of
roles containing this verb
\item
\texttt{tsents:DELEGATE} -- if granted denotes permission of the
grantee to grant roles containing \texttt{tsents:GRANT} on the label
to others
\item
etc. See \protect\hyperlink{sample-verbs-and-roles}{Sample verbs and
roles}.
\end{itemize}
\hypertarget{odds-and-ends}{%
\subsection{Odds and ends}\label{odds-and-ends}}
Special grantee entities include:
\begin{itemize}
\tightlist
\item
\texttt{ANYONE} -- grants to \texttt{ANYONE} are grants to any
authenticated user from any of the realms also granted the same verbs
\item
\texttt{SELF} -- grants to \texttt{SELF} cause \texttt{check()} to
succeed when the subject is the same user that is running the relying
party software (great for developers)
\item
\texttt{COMPARTMENTED} -- denotes that the \texttt{\{label,\ role\}}
requires that subjects be workloads executing in granted compartments
\item
\texttt{TWOPARTY} -- denotes that the \texttt{\{label,\ role\}}
requires that subjects have approval by a second party
\item
\texttt{MULTIFACTOR} -- denotes that the \texttt{\{label,\ role\}}
requires that subjects perform multi-factor authentication
\end{itemize}
We could also have a \texttt{PUBLIC} special grantee entity to denote
that no authentication is required.
Special labels include:
\begin{itemize}
\tightlist
\item
\texttt{SELF::} -- a label grants all verbs to the \texttt{SELF}
special grantee entity (great for developers)
\end{itemize}
TS Entitlements does support using JSON text representations of
serialized sets of grants as a label. This is useful for applications
that can store grants but not labels in their object metadata, as well
as for applications that cannot trust the enterprise directory service
for whatever reason.
\hypertarget{negative-permissions}{%
\subsection{Negative permissions}\label{negative-permissions}}
Although TS Entitlements does not encode negative entries like
filesystem ACLs do, it is still possible to encode negative permissions
by having verbs which, if granted, cause a denial. Applications would
\texttt{check()} if the subject has been granted some verb denoting
negative permissions, denying if granted, allowing otherwise.
Naturally negative permissions must be used very carefully, if at all!
For example:
\begin{itemize}
\tightlist
\item
\texttt{foo:DENY\_LAUNCH\_MISSILES} would deny the grantee permission
to launch missiles, highlighting the dangers of negative permissions!
\end{itemize}
A more realistic example is our \texttt{ONCALL} verb that denotes
exemption from needing a reviewer after hours when on-call.
\hypertarget{uiux-considerations}{%
\subsection{UI/UX considerations}\label{uiux-considerations}}
We have two types of UIs related to TS Entitlements:
\begin{itemize}
\tightlist
\item
those that concern viewing, creating, or managing verbs, roles,
labels, and grants, and
\item
those that are part of the applications that use TS Entitlements and
which concern picking labels for new objects, setting labels on
objects, and viewing the labels set on objects.
\end{itemize}
The former we provide with TS Entitlements.
The latter are clearly application-specific, as it should be. Though for
RESTful HTTP applications we could specify conventions for those
operations using appropriate HTTP verbs and URI local-part naming
conventions.
We currently use
\texttt{\textless{}app\textgreater{}:\textless{}NAME\textgreater{}} as
the naming convention for verbs, and
\texttt{\textless{}app\textgreater{}:\textless{}Name\textgreater{}} as
the naming convention for roles. A form that does not require an
\texttt{\textless{}app\textgreater{}} prefix but which uses site-local
configuration for discovering each verb's and role's
\texttt{\textless{}app\textgreater{}} might be useful. A URN- and/or
URI-based naming convention for verbs and roles might be appropriate for
interchange purposes and/or
\protect\hyperlink{federated-entitlements-using-uris-oauth-andor-pkix}{federation}.
We currently use
\texttt{\textless{}verb\textgreater{}\^{}\textless{}label\textgreater{}}
and
\texttt{\textless{}role\textgreater{}\^{}\textless{}label\textgreater{}}
to denote a grant of a verb or role in some UIs. For example
\texttt{Repository:PULL\^{}monorepo::some/code/base}. This is exactly
backwards of how it should have been! The label should have come first,
then the verb or role. The \texttt{\^{}} character might not have been
the best choice either.
Default label selection and label selection pickers are noticeably
absent in our implementation. This is a cause of trouble. Applications
that have any notion of ``directory'' can simply apply a directory's
label to objects create in it by default. But not all applications have
``directories''. An API for finding a default label for a given user
creating objects in a given application would be nice. A convention for
this can be created using an
\texttt{\textless{}app\textgreater{}:DEFAULT} verb and a
\texttt{query()} call to find labels that grant
\texttt{\textless{}app\textgreater{}:DEFAULT} to the user creating an
object, but such a convention would allow for multiple default labels,
in which case the application might need to interact with the user or
pick the first such label in lexicographic order (say). Alternatively we
could have a separate schema and APIs for encoding and querying such
things.
A JSON Schema for interchange of grants on labels would be nice, but we
do not currently have one.
\hypertarget{sample-verbs-and-roles}{%
\subsection{Sample verbs and roles}\label{sample-verbs-and-roles}}
Sample generic verbs that one might want:
\begin{itemize}
\tightlist
\item
Ownership, management, auditing:
\begin{itemize}
\tightlist
\item
\texttt{tsents:OWN} -- denotes the ability to add/remove owners of
the \emph{label} on which it is granted; it might also denote having
all verbs for all objects protected by the same label
\item
\texttt{tsents:GRANT} -- see \protect\hyperlink{dog-fooding}{Dog
fooding}
\item
\texttt{tsents:DELEGATE} -- see \protect\hyperlink{dog-fooding}{Dog
fooding}
\item
\texttt{tsents:MANAGE} -- denotes the ability to do all things that
owners can do except add/remove owners
\item
\texttt{tsents:AUDIT} -- denotes the ability to list objects
protected by the label and to see all grants on the label
\end{itemize}
\item
Filesystem-like verbs:
\begin{itemize}
\tightlist
\item
\texttt{generic:READ}
\item
\texttt{generic:CREATE}
\item
\texttt{generic:WRITE}
\item
\texttt{generic:APPEND}
\item
\texttt{generic:ACCESS} -- like the Unix \texttt{x} permissions bit
on directories
\item
\texttt{generic:EXECUTE}
\item
\texttt{generic:LABEL} -- denotes the right to change the label of
objects protected by the label this is granted on
\item
etc.
\end{itemize}
\item
Verbs for simple HTTP applications:
\begin{itemize}
\tightlist
\item
\texttt{http:\textless{}method\textgreater{}} for every HTTP method
\end{itemize}
\item
Verbs version control repositories:
\begin{itemize}
\tightlist
\item
\texttt{vc:CREATE}
\item
\texttt{vc:PULL}
\item
\texttt{vc:PUSH}
\item
\texttt{vc:PUSH\_TAG}
\item
\texttt{vc:LABEL}
\item
etc.
\end{itemize}
\item
App-specific generic verbs that most apps should want:
\begin{itemize}
\tightlist
\item
\texttt{\textless{}app\textgreater{}:READ} -- denotes read
capability
\item
\texttt{\textless{}app\textgreater{}:WRITE} -- may denote
create/write/append/modify capabilities
\item
\texttt{\textless{}app\textgreater{}:LABEL} -- denotes the ability
to set the label of an object
\item
\texttt{\textless{}app\textgreater{}:DEFAULT} denotes that the label
is a ``default label'' for objects created by the grantee (see
\protect\hyperlink{uiux-considerations}{above})
\item
\texttt{\textless{}app\textgreater{}:ADMIN} -- denotes the ability
to perform administrative operations on an object, such as start or
stop a service, etc.
\end{itemize}
\end{itemize}
Sample roles:
\begin{itemize}
\tightlist
\item
\texttt{tsents:Owner}, \texttt{tsents:Manager},
\texttt{tsents:Auditor}
\item
\texttt{generic:Reader} (this might include \texttt{http:HEAD} and
\texttt{http:GET}, as well as \texttt{vc:PULL})
\item
\texttt{generic:Writer} (this might include \texttt{http:POST},
\texttt{PUT}, \texttt{DELETE}, and \texttt{PATCH}, as well as
\texttt{vc:PUSH})
\item
\texttt{generic:Administrator}
\item
etc.
\end{itemize}
\hypertarget{auditing-extant-authorizations}{%
\subsection{Auditing extant
authorizations}\label{auditing-extant-authorizations}}
Auditing who can do what to what resources has two steps:
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
Iterate labels and grants in the enterprise directory back-ending TS
Entitlements,
\item
Iterate objects of applications of interest and check their labels
\end{enumerate}
Partial audits might skip step (2), using documentation on the labels in
the enterprise directory to infer the extent of the access granted to
each user. Step (2), after all, can be quite time consuming.
Partial audits might skip step (1). Auditing only the applications'
objects and relying on the documentation on their labels in the
enterprise directory to ensure that the documentation is accurate can
help validate partial audits that only perform step (1).
Partial audits might only look at objects of a specific application and
all the access granted to user by the objects' labels, referencing the
documentation and grants in the enterprise directory.
\hypertarget{limitations-of-the-checkcore-subject-verb-object-model}{%
\section{\texorpdfstring{Limitations of the \texttt{checkCore()}
subject-verb-object
model}{Limitations of the checkCore() subject-verb-object model}}\label{limitations-of-the-checkcore-subject-verb-object-model}}
We get a great deal of mileage out of the subject-verb-object model in
TS Entitlements, as it lets us unify -to a large degree- labeled
security, MAC, DAC, RBAC, and filesystem ACLs, and it lets us have rich
ABAC policies.
But there are some limits. Generic time-of-day restrictions can be
difficult or impractical to express in the simple API-side schema by
using special verbs and/or special grantees -- they can be expressed as
grants that are automatically made and revoked around the time ranges in
question, but this requires the incremental propagation system to be
functioning. Indeed, we use that automatic grant/revocation approach
implementing on-call policies.
The subject-verb-object model at the core of TS Entitlements does bleed
into the policies as seen by users even though we can can have a more
user-friendly schema at the enterprise directory than on the API side.
To simplify further for users requires applications to layer their own
UIs for access control management.
The subject-verb-object model is naturally stateless, unless one wishes
to rely on making changes to the authorization policy store as a way to
store state at some cost in latency (and unclean data should a process
fail to clean up such state). Stateful policies can be implemented with
this model only with adjunct systems to help maintain state. For
example, a policy that some action can only be taken pursuant to
recorded, official approval by some suitable officer will typically
require a system in which to record requests, reviews, and approvals.
Nonetheless a policy that approvals must come from specific approvers
can be implemented with the subject-verb-object model -as indeed, we
have done in TS Entitlements- using groups to define approvers who are
granted a \texttt{generic:Approver} role that contains a
\texttt{generic:APPROVE} verb, even though the approval state has to be
recorded elsewhere.
Careful management of user group membership can be used to reflect
organizational structure into the authorization system, naturally. Still
a policy like ``only managing directors may review and approve'' mapping
to a grant to a \texttt{Managing\ Directors} group is not as clean as a
policy where the system for expressing it specifically allows a
``managing directors only'' constraint checkbox as a first-class
feature. The UI can map a ``managing directors only'' (and other such)
checkbox to grants of user groups that reflect the organizational
structure, but that bit of complexity -though it can be hidden from the
user- has to be present in the system in ways that are not as pure as a
first-class feature might be instead.
Policies that involve cryptographic capabilities are almost entirely
right out. For example, ``encryption to groups'' policies, TPM 2.0 EA
policies, etc. Similarly, ``smart contracts'' on blockchains cannot be
modeled with \texttt{checkCore()}. Cryptographic authorization systems
simply cannot be implemented in terms of \texttt{checkCore()}. Though
signed attestations of authorization can be used in TPM 2.0 EA policies
(see the \texttt{TPM2\_PolicySigned()} TPM 2.0 command), and the signers
of those attestations can implement authorization checks using TS
Entitlements.
In short, policies which are easily expressed in natural language need
not trivially map to the subject-verb-object model, requiring creative
thinking to fit that model. All policies expressible in MLS, SMACK,
RBAC, and filesystem ACLs can be expressed in TS Entitlements, and most
ABAC policies as well, but there will likely be some ABAC policies that
cannot be implemented either at all with TS Entitlements or without
additional components -- the latter is the case with policies that
require keeping state.
\hypertarget{comparison-to-other-generic-authorization-systems}{%
\section{Comparison to other generic authorization
systems}\label{comparison-to-other-generic-authorization-systems}}
TS Entitlements is inspired by SMACK, which is essentially an evolution
of or inspired by MLS. But TS Entitlements is also easy to relate to
filesystem ACLs, RBAC, and ABAC systems, and it is inspired by my
experience and work on those systems as well. Being an evolution of all
those of those generic systems means that TS Entitlements is a sort of a
unification of them -- not a drop-in replacement, but abstractly at
least it unifies all of those styles of generic authorization systems.
Note that the abstract APIs that we impute to other authorization
systems below are not really specified anywhere. I use these to
illustrate them, but the reader should refer to each system's
specifications -and the documentation of their implementations- for more
and more-accurate details.
\hypertarget{abstracting-authorization-apis}{%
\subsection{Abstracting authorization
APIs}\label{abstracting-authorization-apis}}
Throughout this document I've been using a pseudo-code representation of
abstract APIs. Abstract APIs are a useful tool for thinking about
authorization systems as they serve as abstract documentation of what an
implementation must provide, and also of how applications can or must
use a given authorization system.
\hypertarget{comparison-to-mls}{%
\subsection{Comparison to MLS}\label{comparison-to-mls}}
Multi-level security \protect\hyperlink{MLS-ref}{{[}MLS{]}} is a labeled
security technology whereby all workloads (user processes) and all
objects are ``labeled''. Labels in MLS consist of a triple
\texttt{\{doi,\ level,\ Set\textless{}compartments\textgreater{}\}},
where the domain of interpretation (DOI) identifies the meaning of the
other two items in the triple, with \texttt{level} being a very
coarse-grained clearance level (e.g., ``Secret'', ``Top Secret'',
``Ultra''), and the compartments being akin to user groups (e.g., U.S.
DoD Army, U.S. DoD Navy, UK Marines, etc.). These labels are generally
fixed in size, which limits the number of compartments that can be used.
The DOI helps translate labels across administrative domains in a way
that makes up a bit for the limited number of compartments.
Although labels in MLS have structure, they also have names, with a
database for doing lookups between names and label values and
vice-versa. MLS label names are for humans and need not reflect the
actual compartments and level granted by the named label. Label names
are only used for display purposes or when setting a label, which are
relatively rare events and which need not be fast operations as they
involve user interaction.
Storing the set of granted compartments in a fixed-sized label, and
labeling both user processes as well as objects, is an optimization that
makes checking for access very fast: the access control kernel simply
compares the DOI of the process and the object, and if the DOIs are the
same then it checks that the process' level is at least as high as the
object's minimum level, and if so then it does a bit-mask XOR of the
user process' and the object's labels' compartment sets, and if that
results in at a non-zero value then access is granted if it is also
granted by applicable discretionary access controls, while in all other
cases access is denied. If the DOIs don't match that triggers a slow
path where the user process' label is translated to the object's label's
DOI or vice-versa, then the labels are checked as before.
In TS Entitlements we dispense with fixed-sized labels and do not
attempt to encode the grants in the label itself. This frees us to use
free-form labels, as in SMACK. Also as in SMACK the price we pay is that
we must have a fast local database of grants in order to make access
control checks fast enough. Although it's worth noting that TS
Entitlements does support labels that are JSON texts representing
serialized sets of grants -- this is for applications that cannot store
labels but can store ACLs, as well as for applications whose owners do
not wish to store authorization data in the enterprise directory.
By not storing grants in the label we allow exponentially larger
policies than MLS can implement, and we do away with the notion of
domain of interpretation (DOI).
One important difference is that TS Entitlements is used primarily as a
discretionary access control (DAC) system, though it can also be used as
a MAC system. Recall that MAC systems generally are layered on top of
DAC systems as an additional access control where the owner of the
protected object does not necessarily manage its MAC policy.
Applications that use TS Entitlements can do the same, storing two
labels per-object: one denoting DAC policy and the other denoting MAC
policy. A MAC policy would mainly deal in just a very few verbs such as:
one denoting the ability to change the MAC label, and one denoting the
ability to get any access granted by the DAC label.
\hypertarget{abstract-api-for-mls}{%
\subsubsection{Abstract API for MLS}\label{abstract-api-for-mls}}
\begin{itemize}
\tightlist
\item
\texttt{check(process,\ object)\ -\ boolean}
\item
\texttt{check(subject\_label,\ object\_label)\ -\textgreater{}\ boolean}
\item
\texttt{setProcessLabel(process,\ label)}
\item
\texttt{getProcessLabel(process)\ -\ label}
\item
\texttt{setObjectLabel(object,\ label)}
\item
\texttt{getObjectLabel(object)\ -\textgreater{}\ label}
\item
\texttt{getLabelDisplayName(label)\ -\textgreater{}\ name}
\item
\texttt{createLabelDisplayName(name,\ label)}
\item
\texttt{getLabelLevel(label)\ -\textgreater{}\ level}
\item
\texttt{getLabelCompartments(label)\ -\textgreater{}\ Set\textless{}compartment\textgreater{}}
\item
\texttt{getCompartmentName(compartment)\ -\textgreater{}\ name}
\item
\texttt{createCompartment(name)\ -\textgreater{}\ compartment}
\item
\texttt{makeLabel(level,\ Set\textless{}compartment\textgreater{})\ -\textgreater{}\ label}
\item
\texttt{addUserToCompartment(user,\ compartment)}
\item
\texttt{removeUserFromCompartment(user,\ compartment)}
\item
\texttt{setUserLevel(user,\ level)}
\end{itemize}
\hypertarget{comparison-to-smack}{%
\subsection{Comparison to SMACK}\label{comparison-to-smack}}
SMACK is a system that replaces \texttt{\{level,\ compartments\}} with
\texttt{\{name\}}, but retains fixed-sized labels in order to
interoperate with protocols where labels are carried on the wire. This
allows SMACK to replace structured labels with free-form, textual
labels, just as in TS Entitlements.
To make authorization checks fast SMACK requires a local, in-memory,
in-kernel database of grants. The grants involve a small number of
verbs.
Differences between SMACK and TS Entitlements:
\begin{itemize}
\tightlist
\item
TS Entitlements removes the label length restrictions because it does
not need to interoperate with protocols that carry e.g., CIPSO labels
on the wire.
\item
TS Entitlements allows arbitrary numbers of verbs.
\item
TS Entitlements separates the local database used for authorization
checks from the source of truth from which the former is derived. This
allows the source of truth's schema to be richer.
\item
TS Entitlements allows grants to be of \emph{roles} rather than verbs
in the source of truth. This allows roles to be refactored (verbs to
be added to or removed from roles) without having to change the grants
in the source of truth or the application source code that calls the
\texttt{check()} primitive.
\end{itemize}
As in the MLS case, SMACK is a MAC system, while TS Entitlements is
mainly used as a DAC system, though it can also be used as a MAC system.
Essentially TS Entitlements is a more sophisticated grant management
system for what is otherwise remarkably similar to SMACK.
\hypertarget{abstract-api-for-smack}{%
\subsubsection{Abstract API for Smack}\label{abstract-api-for-smack}}
\begin{itemize}
\tightlist
\item
\texttt{check(process,\ object)\ -\textgreater{}\ boolean}
\item
\texttt{check(user,\ label,\ verb)\ -\textgreater{}\ boolean}
\item
\texttt{setLabel(object,\ label)}
\item
\texttt{getLabel(object)\ -\textgreater{}\ label}
\item
\texttt{grantUser(user,\ label,\ Set\textless{}verb\textgreater{}))}
\item
\texttt{grantGroup(group,\ label,\ Set\textless{}verb\textgreater{}))}
\item
\texttt{listGrants(label)\ -\textgreater{}\ \{Set\textless{}\{user,\ Set\textless{}verb\textgreater{}\}\textgreater{},\ Set\textless{}\{group,\ Set\textless{}verb\textgreater{}\}\textgreater{}\}}
\end{itemize}
\hypertarget{comparison-to-filesystem-acls}{%
\subsection{Comparison to filesystem
ACLs}\label{comparison-to-filesystem-acls}}
Filesystem ACLs generally come in two flavors:
\begin{itemize}
\tightlist
\item
\protect\hyperlink{NTFS-ACLs-ref}{NTFS-style ACLs}. These include
\protect\hyperlink{ZFS-ACLs-ref}{ZFS}- and
\protect\hyperlink{NFSv4-ACLs-ref}{NFSv4}-style ACLs. The differences
between the three are quite minor and not relevant here.
\item
\protect\hyperlink{POSIX-Draft-ACLs-ref}{POSIX Draft ACLs}. This is a
draft standard that was never finished but which Linux supports very
well.
\end{itemize}
NTFS-style ACLs can have negative entries that deny certain verbs to
specific users (or groups of users). Generally NTFS-style ACLs are
reordered when they are set on a file such that \texttt{DENY} ACEs come
first followed by ACEs that grant access. ACL evaluation is linear due
to the kernel not knowing whether the ACL's ACEs have been sorted: for
each ACE in order the system will check if the user process credentials
(user ID, supplemental user group ID list) and the requested verb
matches the ACE. The first matching ACE determines the result: if it's a
\texttt{DENY} ACE then access is denied, else it is granted. If no ACE
matches then access is denied. If the ACL could be counted on to be
sorted with \texttt{DENY} ACEs first, and also by SACL vs DACL, and with
each set of ACEs, \texttt{DENY} and \texttt{GRANT}, further sorted by
grantee security identifier (SID) then the kernel could use a fast set
intersection algorithm just like TS Entitlements uses. A further
optimization would be to intern SIDs so that smaller (32-bit) integers
could be used in the processes' access tokens and files' security
descriptors (which include the ACLs). ZFS, for example, does intern SIDs
using 64-bit ``FUID'' identifiers, but ZFS ACLs cannot be counted on to
be sorted anymore than NTFS ACLs can be.
POSIX Draft ACLs do not have deny entries, but only the verbs granted by
the most-specific matching ACE are granted.
NTFS-/ZFS-/NFSv4-style ACLs support 16 verbs using 16-bit bitmasks.
Windows repurposes the security descriptor (which includes the ACL) in
other applications, with different meanings for the 16-bit bitmask's
verbs for different applications.
POSIX Draft ACLs support 3 verbs (read, write, execute/access) using
3-bit bitmasks.
In all cases the ACL is metadata stored along with the file's (or other
object's) metadata.
TS Entitlements:
\begin{itemize}
\tightlist
\item
Moves the ACL definition to an enterprise directory and replaces the
contents of the ACL in file/object metadata with the ACL's
\emph{name}.
\item
Allows unlimited numbers of verbs.
\item
Adds a notion of \emph{role} that is a set of verbs to make grants
more flexible, allowing for refactoring of roles and grants w/o having
to change application code (and vice-versa), and relieving users of
having to think in terms of verbs when roles are better.
\end{itemize}
NTFS-style ACLs support both, DAC \emph{and} MAC. TS Entitlements does
as well, though it is currently only used for DAC: apps can store
separate labels for DAC and MAC, or if need be we could implement a
pseudo-label that combines two labels.
The core idea here is that it makes little sense to store ACLs in files
(and other objects) when typically most files a user owns have the same
ACLs. The downside of naming ACLs and storing those names instead of the
ACLs' contents in the files' metadata is this: users have to be able to
pick from a reasonably small size set of labels when setting an object's
label, and they need a notion a context-specific default label. In a
traditional filesystem the context-specific default ACL is provided by
the directory containing any new file or directory, and users don't have
to know any names for ACLs. In this sense TS Entitlements complicates
the user experience, though the antidote for this is to ensure that the
cardinality of frequently-used labels for each user is small.
\hypertarget{abstract-api-for-filesystem-acls}{%
\subsubsection{Abstract API for filesystem
ACLs}\label{abstract-api-for-filesystem-acls}}
\begin{itemize}
\tightlist
\item
\texttt{check(process,\ object)\ -\textgreater{}\ boolean}
\item
\texttt{check(user,\ supplementary\ groups,\ ...,\ ACL,\ verb)\ -\textgreater{}\ boolean}
\item
\texttt{setacl(object,\ acl)}
\item
\texttt{getacl(object)\ -\textgreater{}\ acl}
\end{itemize}
ACL construction typically has no API as such, but we can construct an
abstract API for it anyways:
\begin{itemize}
\tightlist
\item
\texttt{makeEmptyACL()\ -\textgreater{}\ acl}
\item
\texttt{grantUser(user,\ acl,\ Set\textless{}verb\textgreater{}))\ -\textgreater{}\ acl}
\item
\texttt{grantGroup(group,\ acl,\ Set\textless{}verb\textgreater{}))\ -\textgreater{}\ acl}
\item
\texttt{listGrants(acl)\ -\textgreater{}\ \{Set\textless{}\{user,\ Set\textless{}verb\textgreater{}\}\textgreater{},\ Set\textless{}\{group,\ Set\textless{}verb\textgreater{}\}\textgreater{}\}}
\end{itemize}
\hypertarget{comparison-to-rbac}{%
\subsection{Comparison to RBAC}\label{comparison-to-rbac}}
The \protect\hyperlink{RBAC-ref}{{[}RBAC{]}} systems that I am
intimately familiar with, Solaris RBAC and UName*It's\footnote{UName*It
was an enterprise directory service that was available in the 1990s.
It's authorization system was an RBAC system with pre-defined roles,
no verbs, whose granularity was at the level of ``domains'', where a
domain corresponds to a DNS domainname with any number of hosts and
users and other resources in it.} RBAC, are very coarse-grained:
grants are of \emph{roles} across a large \emph{domain} of objects. In
Solaris RBAC the scope of a grant is either domain-wide or host-local.
In UName*It grant scope is domain- or subdomain-wide. In both cases
there are no verbs, with the scope of roles in the sense of actions
allowed is documented in natural language.
TS Entitlements allows many grants of the same role on different sets of
objects. The set of objects that a grant relates to is identified by the
objects themselves as their metadata records the name of a label. In
this way TS Entitlements allows for very fine-grained RBAC-style access
control, though it also allows very coarse-grained access control -- the
degree of granularity is controlled by applications' developers and/or
resource owners/operators, or both.
\hypertarget{abstract-api-for-rbac}{%
\subsubsection{Abstract API for RBAC}\label{abstract-api-for-rbac}}
There are numerous RBAC systems, so constructing an abstract API that is
faithful to all of them will not be possible. Here is an approximation:
\begin{itemize}
\tightlist
\item
\texttt{check(user,\ role)\ -\textgreater{}\ boolean} or
\texttt{check(user,\ role,\ scope)}
\item
\texttt{createRole(role)}
\item
\texttt{grantRole(user,\ role,\ scope)}
\item
\texttt{revokeRole(user,\ role,\ scope)}
\end{itemize}
where \texttt{scope} is some sort of fairly coarse-grained identifier
for a large domain of objects, such as a NIS/NIS+/LDAP/DNS/etc. domain.
\hypertarget{comparison-to-oauth-style-protocols}{%
\subsection{Comparison to OAuth-style
protocols}\label{comparison-to-oauth-style-protocols}}
\protect\hyperlink{OAuth-ref}{{[}OAuth{]}}-style protocols generally
involve a cryptographic token that conveys ``claims'' about the
authenticated subject where the claims should be sufficient for the
relying party to perform authorization, and the claims may or may not
identify the subject (i.e., the subject's identity may be withheld from
the relying party). The claims could be voluminous if the issuer is
configured to return information about large scopes of objects at the
relying party that the subject may access, or the claims can be small
and relating to just one object at the relying party. For the latter
case the application may direct the subject's user-agent to fetch a new
token for each object accessed by the subject.
TS Entitlements is not currently an OAuth-style protocol, though it can
be a protocol (see also
\protect\hyperlink{federated-entitlements-using-uris}{below}), and
OAuth-style protocols that can use ``labels'' to identify objects could
work with TS Entitlements. This would be one way of federating TS
Entitlements. This would also add a capability model to TS Entitlements
without changing its API substantially as the \texttt{subject} argument
to \texttt{check()} can already be a JWT, therefore the only thing
missing is for \texttt{check()} to be able to throw an exception or
return an error such that the application can issue an appropriate
redirect (in HTTP applications anyways) to an issuer that can produce a
token that contains the requested access claim.
In our current implementation there is no federation support, and
relying parties must have a great deal of authorization information
locally available. We currently rely on OAuth tokens identifying the
subject in order to then use the TS Entitlements API as it can then find
the subject's group memberships etc.
\hypertarget{constructing-new-primitives-from-the-checkcore-primitive}{%
\section{\texorpdfstring{Constructing new primitives from the
\texttt{checkCore()}
primitive}{Constructing new primitives from the checkCore() primitive}}\label{constructing-new-primitives-from-the-checkcore-primitive}}
The ABAC functionality in TS Entitlements is constructed from the
\texttt{checkCore()} primitive and pseudo-verbs and special grantees.
That is, the \texttt{check()} primitive that applications invoke itself
invokes \texttt{checkCore()} at least once, or possibly multiple times.
Note that because of caching this does not significantly slow down
\texttt{check()}.
We can think of a number of authorization functions that we don't yet
currently implement but which we could implement entirely based on
\texttt{checkCore()}. For example, we might want a form that takes two
subjects, one being an impersonator and the other being the subject to
be impersonated by the impersonator. Such a \texttt{check()}
construction might check whether the impersonated is allowed the
requested access \emph{and} also whether impersonator is allowed to
\texttt{generic:IMPERSONATE} (or perhaps whether the impersonator is
allowed the requested access, or both). A similar feature for
cross-origin request sharing (CORS) could see us implement a subject
type corresponding to \texttt{Referer} (i.e., URI authority) to an user
entity ID for running automation (we call these ``role accounts'') that
is then used as a subject.
Similarly we can implement useful functions in terms of
\texttt{query()}, such as functions that return a small number of labels
from which a user might pick one in a picker UI element.
As noted above we could also have a MAC variant of \texttt{check()} that
takes two labels, and which implements mandatory access control policies
using a MAC label, and discretionary policies using a DAC label. Or we
could use a single composite label that encodes both MAC and DAC labels,
but overloading label names like this makes for a bad UI as users would
be expected to understand the MAC label naming convention. Still, using
a MAC label naming convention might be appropriate where applications
can only store a single label per-object.
There are ABAC attributes that we don't but we easily could implement,
such as the following taken from the
\href{https://en.wikipedia.org/wiki/Attribute-based_access_control}{Wikipedia
page on ABAC}:
\begin{itemize}
\tightlist
\item
time-of-day restrictions,
\item
department restrictions,
\item
clearance level restrictions,
\item
object type restrictions.
\end{itemize}
Some of these additional ABAC features can be denoted by user group
memberships and grants to them, others can be denoted by special
entities and grants thereto, and some might require schema changes
(e.g., clearance level). I believe we can implement all the kinds of
attributes and policies given as examples on the
\href{https://en.wikipedia.org/wiki/Attribute-based_access_control}{Wikipedia
ABAC page} except for generalized time-of-day restrictions, and most
useful attributes and policies that one could think of, though in some
cases that might require adding functions to the API.
The reason that generalized time-of-day restrictions are hard to
implement solely in terms of \texttt{checkCore()} and special verbs
and/or special grantees is that it is difficult to define arbitrary time
ranges with just those few tools. For example, one might have
\texttt{generic:AFTER\_\textless{}NNNN\textgreater{}} and
\texttt{generic:BEFORE\_\textless{}NNNN\textgreater{}} pseudo-verbs to
define time ranges, but having to \texttt{checkCore()} all of those
verbs will get expensive. On the other hand one could have a
\texttt{generic:BUSINESS\_HOURS\_ONLY} pseudo-verb where the time range
``business hours'' is defined elsewhere, and just a few such ranges
might suffice.
Any desirable policy that one can imagine implementing in terms of
\texttt{checkCore()} can therefore be so implemented, either in TS
Entitlements' \texttt{check()} itself or in the applications that use
it. Naturally it is best to have a single library that provides all the
variants of \texttt{check()} and all the policy logic, but applications
\emph{can} use the TS Entitlements \texttt{check()} primitive to
implement policies which we don't already implement in TS Entitlements.
The \texttt{checkCore()} primitive, and the \texttt{check()} function,
are something of a ``mecano set'' that can be used to construct much
richer and more complex policies than just \texttt{checkCore()}.
\hypertarget{comparison-to-abac-systems}{%
\subsection{Comparison to ABAC
systems}\label{comparison-to-abac-systems}}
I have no experience with any other
\protect\hyperlink{ABAC-ref}{{[}ABAC{]}} systems, and for that reason I
will not do an in-depth comparison here. But from what I can tell ABAC
is always a feature that is added to or used in conjunction with other
authorization systems. That is, ABAC systems are not generally
sufficient on their own. In TS Entitlements ABAC is indeed an add-on
feature.
\hypertarget{conclusion-and-future-directions}{%
\section{Conclusion and future
directions}\label{conclusion-and-future-directions}}
TS Entitlements is a relational and generic authorization system using
natural language analogies (subject-verb-object, or, rather,
subject-verb-label) for representation of extant access controls and to
check access controls as needed. This system has served Two Sigma
Investments, LP well for over a decade. There are other authorization
schemes in use at Two Sigma that are specific to certain applications
and use-cases, but by and large TS Entitlements is widely used.
Because the primitives in TS Entitlements are powerful but simple and
can be combined to create richer and more complex policies, TS
Entitlements is quite flexible. Though any such system has some
\protect\hyperlink{limitations-of-the-checkcore-subject-verb-object-model}{inherent
limitations}.
TS Entitlements being an evolution of several radically different
general-purpose authorization systems makes it likely that its design is
widely applicable.
Being inherently relational, TS Entitlements implementations can be
created quite simply in SQL and other relational languages. Outside of
any RDBMS the complexity of any implementation of a TS Entitlements
alike system will lie mainly in a) any synchronization of enterprise
directories and/or ``triggers'' if applicable, b) the extraction of data
and incremental updates streams, c) transformation to forms optimized
for fast \texttt{check()} implementations, d) distribution of the same,
and lastly e) the implementation of the API. Using
\href{https://sqlite.org}{SQLite3} for a local database can greatly
simplify (c), (d), and/or (e) by using existing tooling (like SQLite3
itself) and enabling a trivial SQL-based implementation of
\texttt{checkCore()}. In other words, the most complex part of any
implementation of a TS Entitlements alike system will be integration
with the implement's choice of enterprise directory.
Perhaps the single most important thing to do to TS Entitlements to make
it generally usable outside Two Sigma is to add support for
\protect\hyperlink{federated-entitlements-using-uris-oauth-andor-pkix}{federation}
using multiple different distributed sources of truth, possibly
designing, specifying, and standardizing a set of APIs, conventions (for
standard verbs and roles), and protocols.
\hypertarget{federated-entitlements-using-uris-oauth-andor-pkix}{%
\subsection{Federated entitlements using URIs, OAuth, and/or
PKIX}\label{federated-entitlements-using-uris-oauth-andor-pkix}}
In order to support federated entitlements, it might be a good idea to
have URN/URI forms for verbs and roles (URNs for well-known, pre-defined
ones, URIs otherwise), and URI forms for labels and grants, along with
new URI schemes for verbs, roles, and labels/grants. Users and
application developers would still use
\texttt{\textless{}app\textgreater{}:\textless{}VERB\textgreater{}} and
\texttt{\textless{}app\textgreater{}:\textless{}Role\textgreater{}}
naming, but would configure mappings of those to URNs/URIs.
For example, a pre-defined verb like \texttt{tsents:OWN} could have a
canonical URN like \texttt{urn:ietf:rfc:XXXX:tsents:OWN} if the system
were published as an Internet RFC, and a custom verb like
\texttt{Kafka:CREATE\_TOPIC} could be
\texttt{verb://domain.example/Kafka:CREATE\_TOPIC}. Similarly for
\protect\hyperlink{sample-verbs-and-roles}{generic roles}.
For labels (and grants) we might have a \texttt{label:} scheme with an
authority component: \texttt{label://domain.example/Some/Label/Here}. A
\texttt{label:} scheme should be able to map \texttt{label:} URIs to
\texttt{https:} URIs using an appropriate service discovery mechanism.
For example, \texttt{label://domain.example/Some/Label/Here} might map
to \texttt{https://authz.domain.example/labels/Some/Label/Here}.
A service discovery mechanism for mapping \texttt{label:} scheme URIs to
\texttt{https:} scheme URIs would have to deliver to the client:
\begin{itemize}
\tightlist
\item
one or more authorities to use for the \texttt{https:} URIs
corresponding to a \texttt{label:} scheme URI's authority,
\item
an optional URI local-part prefix to prepend to the \texttt{label:}
URI's local-part
\end{itemize}
Service discovery could be done with a DNS-based discovery mechanism
and/or with \texttt{.well-known/} \texttt{https:}-scheme URIs with the
\texttt{label:}-scheme URI's authority. For example, a \texttt{GET} of
\texttt{https://domain.example/.well-known/entitlements.json} might
yield a JSON text with a key listing origins and another specifying a
local-part prefix. Or a DNS URI-type Resource Record lookup might be
used to find the origins, and some as-yet-unspecified DNS Resource
Record type might be used to discover any URI local-part prefixes, or
simply hard-code a prefix as \texttt{/entitlements/} or
\texttt{/authorization/}.
For grants we might use a \texttt{label:} scheme URI w/ query parameters
(q-params) for the role or verb. Thus an HTTP \texttt{GET} of an
\texttt{https:} URI mapped from
\texttt{label://domain.example/Some/Label/Here?role=tsents:Owner} would
return the grantees if the user-agent were authorized to see them, while
a \texttt{PUT}, \texttt{POST}, or \texttt{PATCH} might create or alter
the grant (if the user-agent were authorized to alter the set of
grants). A \texttt{HEAD} of
\texttt{label://domain.example/Some/Label/Here?verb=tsents:OWN\&subject=some\_username\_here}
would implement
\texttt{check("some\_username\_here",\ "tsents:OWN",\ "label://domain.example/Some/Label/Here")},
while a \texttt{GET} of the same would implement the same function
\emph{and also} return additional information, such as related grants to
enable caching for client-side evaluation of related \texttt{check()}
calls the client might want to do soon after the \texttt{GET}. A
\texttt{GET} of a \texttt{label:} URI w/ grant q-params might also
return a cryptographic token that the user-agent can present to
applications, and which token could act as an access certificate.
As noted \protect\hyperlink{comparison-to-oauth-style-protocols}{above}
OAuth-style protocols could also be adapted to federate TS Entitlements.
For example, when performing some operation on an HTTP resource the
origin could redirect the user-agent to fetch a code via OIDC that can
be used to demonstrate that the user has the requested access. The
redirect would be to a URI that denotes the arguments to the
\texttt{check()} call to be done by the code issuer, and the resulting
code and token would indicate whether the user-agent has the requested
access. Token requests need not be HTTP redirects.
A set of claims could be specified for JSON Web Tokens (JWTs) to carry
assertions of one or more grants.
A PKIX access certificate attribute types could be specified to carry
one or more assertions of grants for applications that use PKIX access
certificates.
A complete set of protocols could be designed and standardized using the
above sketch as a starting point.
\hypertarget{use-case-federated-access-controls-for-healthcare-data}{%
\subsubsection{Use-case: federated access controls for healthcare
data}\label{use-case-federated-access-controls-for-healthcare-data}}
Imagine a world where users can create labels for their healthcare data
then supply those labels to healthcare providers for use in authorizing
access to their data. Thus a user might have a label for data related to
some surgery, and different labels for data related to unrelated
surgeries, dental health, general health, endocrinology, etc. A user
could then manage grants on their data, making it very easy to share
data with -for example- a doctor for a second opinion, or when moving
from one practice to another.
Using URIs for labels and grants would allow for third party
authorization providers. Aggressive caching could be used to cope with
provider failures and to recover by moving to other providers if need
be.
A standard protocol for this use-case could revolutionize healthcare and
privacy.
Other use-cases abound.
\hypertarget{references}{%
\section{References}\label{references}}
\href{https://learn.microsoft.com/en-us/windows/win32/secauthz/access-control-lists}{{[}NTFS{]}},
the access control system for Windows' NTFS filesystem, Active
Directory, and other products.
{[}{[}IEEE{]}{]} Institute of Electrical and Electronics Engineers,
``IEEE 1003.1e and 1003.2c: Draft Standard for Information
Technology--Portable Operating System Interface (POSIX)--Part 1: System
Application Program Interface (API) and Part 2: Shell and Utilities,
draft 17'', October 1997.
\href{https://datatracker.ietf.org/doc/html/rfc5661\#section-6.2.1}{{[}NFSv4
ACLs{]}} ``Network File System (NFS) Version 4 Minor Version 1
Protocol'' section 6.2.1, S. Shepler (Editor) et. al., January 2010.
\href{https://datatracker.ietf.org/doc/html/draft-rmacklem-nfsv4-posix-acls-10}{{[}NFSv4
POSIX Draft ACLs{]}} ``POSIX Draft ACL support for Network File System
Version 4, Minor Version 2'', R. Macklem, March 2025.
\href{https://docs.oracle.com/cd/E26502_01/html/E29007/gbacb.html}{{[}ZFS
ACLs{]}} ``Oracle Solaris 11.1 Administration: ZFS File Systems'',
``Solaris ACL Model'', Oracle, January 2005.
\href{https://en.wikipedia.org/wiki/Role-based_access_control}{{[}RBAC{]}}
``Role-based access control'', Wikipedia.
\href{https://en.wikipedia.org/wiki/Attribute-based_access_control}{{[}ABAC{]}}
``Attribute-based access control'', Wikipedia.
\href{https://en.wikipedia.org/wiki/Multilevel_security}{{[}MLS{]}}
``Multilevel security'', Wikipedia
\href{https://www.kernel.org/doc/html/v4.18/admin-guide/LSM/Smack.html\#the-simplified-mandatory-access-control-kernel-whitepaper}{{[}SMACK{]}},
``The Simplified Mandatory Access Control Kernel (Whitepaper)'', C.
Schaufler, April 2008.
\href{https://datatracker.ietf.org/doc/html/rfc6749}{{[}OAUTH{]}} ``The
OAuth 2.0 Authorization Framework'', D. Hardt (Editor), October 2012.
\href{https://datatracker.ietf.org/doc/html/rfc7519}{{[}JWT{]}} ``JSON
Web Token (JWT)'', M. Jones et. al., May 2015.
\href{https://datatracker.ietf.org/doc/html/rfc8392}{{[}CWT{]}} ``CBOR
Web Token (CWT)'', M. Jones et. al., May 2015.
\href{https://waterken.sourceforge.net/aclsdont/current.pdf}{{[}ACLs
don't{]}} ``ACLs don't'', Tyler Close, January 2009.
\href{https://cr.yp.to/cdb.html}{{[}CDB{]}} ``cdb'', D. J. Bernstein,
unknown publication date.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment