Skip to content

Instantly share code, notes, and snippets.

@bengolder

bengolder/audit_log.md

Created April 26, 2017 22:29
Show Gist options
  • Save bengolder/635d84ed072dc43aceb82e3b4385bd1e to your computer and use it in GitHub Desktop.
Save bengolder/635d84ed072dc43aceb82e3b4385bd1e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
audit_log.md

Access Controls and Auditability

Goals:

  • Default to requiring authentication, intentionally override where necessary
  • Default to checking for object or queryset permissions, intentionally override where necessary
  • Default to logging access and actions for later audit, override where necessary
  • Ensure maintainability by creating one well-known way to implement, and adding checks to ensure proper implementation

What do we want in our audit logs?

List (index) view:

  • time, person accessed field, field, and field for applicants: id, id, id, id

Detail view (assumes full CJI & PII access)

  • time, person accessed details of applicant: id

Restricted detail view (assumes some PII/CJI access)

  • time, person accessed field, field, and field of applicant: id

Adding a status update (creating a new related PII/CJI object, where that is considered the latest version of that info) Updating contact info (these should be related objects)

  • time, person added a new latest resource for applicant: id

Adding a comment (where latest doesn't not imply a state for the applicant)

  • time, person added a new resource(id) for applicant: id

Sending a communication

  • time, person sent resource(id) to applicant(id) using resource(id)

String representations:

  • 2017-04-24 21:35:02.799233+00:00, user(4) from organization(5) with permission "application_review" in session(14), sent email(3210) to applicant(1023) using email address(321)
  • 2017-04-24 21:35:02.799233+00:00, [email protected] from Code for America, using ip(69.12.169.82) and device('iphone') with permission "admin", sent email(3210) to applicant(1023) using email address(321)
  • 2017-04-24 21:35:02.799233+00:00, [email protected] from Code for America, using session(14) with permission "admin", sent email(3210) to applicant(1023) using email address(321)

UserProfile

  • user
  • organization
class UserSession(models.Model):
    created = models.DateTime()
    updated = models.DateTime()
    user_profile = models.ForeignKey(
        'user_accounts.UserProfile',
        on_delete=models.PROTECT)
    django_session = models.ForeignKey(
        'sessions.Session',
        on_delete=models.PROTECT)
    # these are intended to collect a method of access at a point in time
    ip = models.IPAddressField()
    user_agent = models.TextField()

class UserSessionAction(models.Model):
    time = models.DateTime()
    session = models.ForeignKey('user_accounts.UserSession')


from django.contrib.postgres.fields import ArrayField
class ReadApplicantPIIListAction(UserSessionAction):
    applicants_affected = ArrayField(
        models.ForeignKey('intake.Applicant', on_delete=models.PROTECT))
    # content types?
    fields_accessed = ArrayField(TextField())

Queries to consider: every event pertaining to an applicant every event pertaining to an organization every event pertaining to users in a group every event pertaining to users with a given level of permission every event pertaining to a type of PII every event pertaining to a session

UI outputs to consider (all relevant to an optional timespan): a history of all actions happening to an applicant a history of all actions by a user

  • time, person created a new resource for applicant: id

  • time, person created a new resource for applicant: id

  • time, person edited field on resource

  • time, person deleted resource

  • time, person added data to field on resource

  • time, person added data to field on resource

Questions

  • can I add meta attributes to database fields to indicate that they are PII or CJI?
  • How should I convey the relationships, for example, whether the us

### DjangoRESTFramework as a starting point




### Middleware as a code coverage check

How will we ensure that we do not leave particular views unchecked for access controls and auditability?

Middleware is applied universally across the application, to all requests and responses. In our base views or when intentionally opening access or bypassing audit logginge, we can add a property to the `Request` object to record that we've either processed this request through access controls or intentionally bypassed them. Checking for the positive presence of this attribute in middleware, we can ensure that no request accidentally bypasses access controls or audit logging.

```python
# in intake/middleware.py
class EnforceAuditabilityAndAccessControlsMiddleware(MiddlewareBase):

    def __call__(self, request):
        # before calling view
        response = self.get_response(request)
        # after calling view (request should have been modified by view)
        access_controlled = getattr(
            request, 'is_configured_for_access_control', False)
        if not access_controlled:
            raise exceptions.InsufficientAccessControls(
                "This request does not utilize sufficient audit logging")
        audit_ready = getattr(request, 'is_configured_for_audit', False)
        if not audit_ready:
            raise exceptions.InsufficientAuditability(
                "This request does not utilize sufficient audit logging")
        return response

Testing for proper implementation

A base test case that can be inherited to run tests on each child test case

Making a Base View for Audit Logs

Goals:

  1. As a CMR dev, I want to make sure that I don't accidentally grant too much access to data.
  • implement middleware that checks to see if a given request has had permissions properly checked. [2 hr]
  • implement a base view with a clear API for checking authentication, query scope, and object permissions. [6 hr]
  1. As a CMR dev, I don't want to have to think too much about access control logging or auditability.
  • Implement audit logging in the base view, based on permissions checks [6 hour]
  1. As CMR staff, I would like to be able to read audit data, so that I can properly check if access controls are working as expected.
  • make sure that audit logs are in a format that is usable when needed [3 hr]

Things to check:

  • Django REST Framework Permissions

How Django REST Framework handles permissions

  • It has a base view and a base Permission class
  • The permission class has methods for handling object-based permissions
  • it has explicit methods for checking object p

Django REST Framework looks great, and seems to offer an excellent set of basic functionality for systematically checking permissions

Events The following events shall be logged:

  1. Successful and unsuccessful system log-on attempts.
  2. Successful and unsuccessful attempts to use: a. access permission on a user account, file, directory or other system resource; b. create permission on a user account, file, directory or other system resource; c. write permission on a user account, file, directory or other system resource; d. delete permission on a user account, file, directory or other system resource; e. change permission on a user account, file, directory or other system resource.
  3. Successful and unsuccessful attempts to change account passwords.
  4. Successful and unsuccessful actions by privileged accounts.
  5. Successful and unsuccessful attempts for users to: a. access the audit log file; b. modify the audit log file; c. destroy the audit log file."

Content The following content shall be included with every audited event:

  1. Date and time of the event.
  2. The component of the information system (e.g., software component, hardware component) where the event occurred.
  3. Type of event.
  4. User/subject identity.
  5. Outcome (success or failure) of the event."
2017-04-24T22:01:05.277685+00:00 heroku[router]: at=info method=GET path="/partners/san_diego_pubdef/" host=clearmyrecord.codeforamerica.org request_id=aee28572-beb0-46c5-b737-df5a5e2dad53 fwd="52.90.33.223" dyno=web.1 connect=0ms service=69ms status=200 bytes=15278 protocol=https

2017-04-24T22:01:05.277685+00:00
heroku[router]:
    at=info
    method=GET 
    path="/partners/san_diego_pubdef/"
    host=clearmyrecord.codeforamerica.org
    request_id=aee28572-beb0-46c5-b737-df5a5e2dad53
    fwd="52.90.33.223"
    dyno=web.1
    connect=0ms
    service=69ms
    status=200
    bytes=15278
    protocol=https


2017-04-24T21:35:02.799233+00:00 heroku[router]:
    at=info
    method=GET
    path="/application/1560/"
    host=clearmyrecord.codeforamerica.org request_id=e337be7c-eced-403b-9753-dd6c2cc0efa4
    fwd="54.209.231.248"
    dyno=web.1
    connect=0ms
    service=72ms
    status=302
    bytes=409
    protocol=https

2017-04-24T21:35:02.882063+00:00 heroku[router]:
    at=info
    method=GET
    path="/accounts/login/?next=/application/1560/"
    host=clearmyrecord.codeforamerica.org
    request_id=61781369-377c-4307-acd9-f939be30143b
    fwd="54.209.231.248"
    dyno=web.2
    connect=0ms
    service=78ms
    status=200
    bytes=10404
    protocol=https
2017-04-24T21:43:22.286352+00:00 app[worker.1]:
    [2017-04-24 21:43:22,286: INFO/MainProcess]
        Received task:
            intake.tasks.celery_request[11bac2a7-1908-4915-9ccf-d9cfcaa8baaf]

2017-04-24T21:43:23.940170+00:00 app[worker.1]:
    [2017-04-24 21:43:23,939:INFO/PoolWorker-4]
        Task intake.tasks.celery_request[11bac2a7-1908-4915-9ccf-d9cfcaa8baaf] succeeded in 1.0378765090135857s: None

Martin Fowler log example

<log_datetime> <user> <action_description> <old_value> <new_value> <date_of_action>

2017-04-24T21:43:23.940170+00:00
    [email protected]
    edited 'email' on user_accounts.Organization(id=4)
    from '[email protected]' to '[email protected]'

django admin



2017-04-24T19:08:18.057846+00:00 heroku[router]:
    at=info
    method=POST
    path="/apply/"
    host=clearmyrecord.codeforamerica.org request_id=408bfcf1-fc62-426b-9b63-cdbe91fffbeb
    fwd="50.161.171.204"
    dyno=web.1
    connect=1ms
    service=31ms
    status=403
    bytes=6532
    protocol=https
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment