Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save DavidMStraub/5a7f1f4b1e374db4445baa93654b88f4 to your computer and use it in GitHub Desktop.

Select an option

Save DavidMStraub/5a7f1f4b1e374db4445baa93654b88f4 to your computer and use it in GitHub Desktop.
Gramps Web API advanced permissions evaluation

[Generated on 2025-01-05 by Claude Sonnet 4.5 with access to the Gramps Web API repsotitory.]

Fine-Grained Permission System - Evaluation & Recommendations

Current System Overview

Existing Architecture

Role-Based Access Control (RBAC):

  • 6 user roles: Guest (0), Member (1), Contributor (2), Editor (3), Owner (4), Admin (5)
  • Roles grant hierarchical permissions with each level inheriting from the previous
  • Permissions stored in JWT claims for stateless authorization

Privacy Control:

  • PERM_VIEW_PRIVATE: Enables/disables viewing of Gramps objects marked private
  • When disabled, users get a ModifiedPrivateProxyDb wrapping the database
  • Based on Gramps' native PrivateProxyDb which filters private records

Object-Level Permissions:

  • PERM_ADD_OBJ: Add new genealogical objects (Contributor+)
  • PERM_EDIT_OBJ: Edit existing objects (Editor+)
  • PERM_DEL_OBJ: Delete objects (Editor+)

Current Limitations:

  • Binary privacy: either see all private records or none
  • No per-object-type permissions (e.g., edit people but not sources)
  • No relationship-based access (e.g., view only close relatives)
  • No branch-specific editing rights

Requirements Analysis

Desired Capabilities

  1. Relationship-Based Viewing: View details for people within N generations
  2. Branch-Specific Editing: Edit only a specific lineage/branch
  3. Graduated Privacy Levels: More than just public/private
  4. Context-Aware Access: Different rights for different parts of the tree

Architecture Options

Option 1: Extended Proxy Database System

Approach: Build custom proxy classes extending Gramps' existing proxy system

Implementation:

  • Create RelationshipProxyDb wrapping the base database
  • Filter objects based on relationship calculations at query time
  • Similar to existing PrivateProxyDb but with genealogical logic

Pros:

  • Consistent with current architecture
  • Leverages proven Gramps proxy patterns
  • Minimal changes to API endpoints
  • Database-agnostic (works with all filters/exports)

Cons:

  • Performance overhead for complex relationship calculations
  • Relationship computation on every database access
  • Limited flexibility for non-relationship-based rules

Complexity: Medium Performance Impact: Medium-High (relationship calculations are expensive)


Option 2: User-Object Permission Matrix

Approach: Store explicit user-object permissions in a database table

Implementation:

  • New user_object_permissions table with columns: user_id, object_handle, object_type, permission_level
  • Pre-compute and cache accessible objects per user
  • Check permissions in middleware before returning data

Pros:

  • Fast lookups (indexed database queries)
  • Maximum flexibility (any permission pattern)
  • Can handle complex rules once computed
  • Explicit audit trail

Cons:

  • Significant storage overhead (potentially millions of rows)
  • Complex maintenance (updates when objects added/modified)
  • Relationship changes require permission recalculation
  • Requires schema migrations

Complexity: High Performance Impact: Low (after initial computation)


Option 3: Dynamic Filter-Based Permissions

Approach: Store Gramps filter definitions per user role/permission level

Implementation:

  • Extend user table with viewing_filter and editing_filter JSON columns
  • Filters use Gramps' native filter engine (already supports relationship rules)
  • Apply user's filter when calling get_db_handle()
  • Reuse existing filter infrastructure (ancestors, descendants, etc.)

Pros:

  • Leverages existing, well-tested Gramps filter system
  • Filters already support relationship-based rules
  • User-configurable without code changes
  • Works with existing FilterProxyDb infrastructure

Cons:

  • Filter complexity can be hard for admins to configure
  • Some performance impact (filter evaluation at query time)
  • Limited by Gramps filter capabilities

Complexity: Medium Performance Impact: Medium


Option 4: Attribute-Based Access Control (ABAC)

Approach: Define permissions based on attributes of user, object, and context

Implementation:

  • Policy engine evaluating rules like:
    • "User can view Person if relationship_distance <= user.max_viewing_distance"
    • "User can edit Person if person.handle in user.editable_branch_roots + descendants"
  • Policies stored as evaluatable expressions (e.g., Python expressions or policy DSL)

Pros:

  • Most flexible and powerful approach
  • Supports complex conditional logic
  • Industry-standard pattern (similar to AWS IAM policies)
  • Can combine multiple criteria

Cons:

  • Most complex to implement
  • Significant learning curve for administrators
  • Policy language/engine required
  • Potential security risks if not carefully implemented

Complexity: Very High Performance Impact: Medium


Recommended Approach

Hybrid: Option 3 (Filter-Based) + Enhanced Proxy System

This combines the strengths of the existing architecture with proven Gramps functionality.

Handling Related Objects (Events, Notes, Sources, Media)

Challenge: When filtering People, what happens to associated objects?

Example Scenarios:

  • Person passes filter → should their birth event be visible?
  • Person fails filter → can you still see an event where they're a witness?
  • Can you view a note attached to a filtered-out person?

Solution: Transitive Inclusion with ReferencedBySelectionProxyDb

Gramps already provides ReferencedBySelectionProxyDb which:

  • Takes a set of "approved" object handles
  • Includes ALL objects referenced by those approved objects
  • Used in exports with person filters

Implementation Strategy:

class FilteredProxyDb(ProxyDbBase):
    def __init__(self, db, person_filter):
        super().__init__(db)
        
        # Apply person filter to get approved people
        approved_people = apply_person_filter(db, person_filter)
        
        # Use ReferencedBySelectionProxyDb for transitive inclusion
        self._proxy = ReferencedBySelectionProxyDb(
            db,
            person_list=approved_people,
            note_list=[],  # Will be auto-included if referenced
            event_list=[],
            source_list=[],
            # ...
        )
    
    # Delegate all methods to self._proxy

Access Rules:

  1. People: Only those matching the filter
  2. Events: Included if referenced by filtered people (birth, death, etc.)
  3. Families: Included if either parent matches filter
  4. Notes: Included if attached to any included object
  5. Sources/Citations: Included if cited by any included object
  6. Media: Included if referenced by any included object
  7. Places: Included if used in any included event

Edge Cases:

Scenario Behavior
Person A matches filter, married to Person B who doesn't Family visible, B appears as spouse but B's details hidden when accessing directly
Event has multiple participants, only some match filter Event visible, but participant list filtered
Note shared between filtered and non-filtered people Note visible (conservative approach)

Configuration Options:

For maximum flexibility, add a policy setting:

# In user configuration
related_objects_policy: str = "transitive"  # or "strict"
  • "transitive" (recommended): Include all objects referenced by filtered people
  • "strict": Also filter related objects (e.g., only show events for filtered people)

Why Transitive is Better:

  • Prevents broken references in UI
  • Maintains data integrity and context
  • Matches user expectations (if I can see a person, I expect to see their birth event)
  • Consistent with Gramps export behavior

Phase 1: Foundation (2-3 weeks)

  1. Extend User Model

    class User:
        viewing_filter_json: str  # Serialized Gramps filter
        editing_filter_json: str  # For write permissions
        privacy_level: int  # 0=all, 1=non-private, 2=custom
        related_objects_policy: str  # "transitive" (default) or "strict"
  2. Create FilteredProxyDb Class

    • Extends ModifiedPrivateProxyDb
    • Applies user's viewing filter to People
    • Uses ReferencedBySelectionProxyDb for transitive inclusion
    • Caches relationship calculations per request
  3. Update get_db_handle() Logic

    def get_db_handle(readonly=True):
        claims = get_jwt()
        if claims.get('viewing_filter'):
            filter_def = claims['viewing_filter']
            return FilteredProxyDb(base_db, filter_def)
        elif not claims.get(PERM_VIEW_PRIVATE):
            return ModifiedPrivateProxyDb(base_db)
        return base_db

Phase 2: UI & Administration (2-3 weeks)

  1. Filter Builder UI for admins

    • Preset templates: "Descendants of X within N generations"
    • Visual filter editor leveraging existing Gramps UI patterns
  2. User Management Extensions

    • Assign filters to users/roles
    • "Anchor person" field (which person is the user in the tree?)
  3. API Endpoints

    • PUT /users/{username}/viewing-filter
    • GET /users/{username}/accessible-people (preview)

Phase 3: Advanced Features (3-4 weeks)

  1. Multi-Level Privacy

    • Extend privacy flag from boolean to 0-3 levels
    • Filter by privacy level: "show level 0-1 only"
  2. Branch-Specific Editing

    • Separate editing_filter_json field
    • Apply to write operations (POST/PUT/DELETE)
    • Validation: ensure user can view what they're editing
  3. Performance Optimization

    • Cache relationship calculations in Redis
    • Pre-compute common filter results
    • Lazy-load filtered object lists

Key Design Decisions

Decision Point 1: Filter Storage Format

Options:

  • A) JSON representation of Gramps filter rules ✓ RECOMMENDED
  • B) Custom DSL
  • C) Python code snippets

Rationale: JSON format allows validation, is language-agnostic for frontend, and maps directly to existing Gramps filter system.

Decision Point 2: Performance vs. Flexibility

Options:

  • A) Always compute filters dynamically ✓ RECOMMENDED for MVP
  • B) Pre-compute and cache filtered object lists
  • C) Hybrid: cache for common filters, compute for custom

Rationale: Start with dynamic computation for correctness, add caching later based on profiling. Premature optimization here risks bugs.

Decision Point 3: Backward Compatibility

Options:

  • A) Maintain current privacy system, add filters alongside ✓ RECOMMENDED
  • B) Migrate all users to filter-based system
  • C) Parallel systems with toggle

Rationale: Don't break existing deployments. Null viewing_filter_json → use legacy PERM_VIEW_PRIVATE.

Decision Point 4: Edit Permission Scope

Options:

  • A) Separate viewing and editing filters ✓ RECOMMENDED
  • B) Same filter for both (edit implies view)
  • C) Edit requires superset of view permissions

Rationale: Different use cases: users who can view extended family but only edit immediate family.

Example Use Cases & Implementation

Use Case 1: View Close Relatives Only

Requirement: Member can view people within 3 generations

Implementation:

{
  "rules": [
    {
      "name": "IsLessThanNthGenerationAncestorOf",
      "values": ["<USER_PERSON_ID>", "4"]
    },
    {
      "name": "IsLessThanNthGenerationDescendantOf",
      "values": ["<USER_PERSON_ID>", "4"]
    }
  ],
  "operator": "or"
}

Admin workflow:

  1. Identify user's person record in tree
  2. Select "View close relatives" template
  3. Set generations = 3
  4. Save to user's viewing_filter_json

Use Case 2: Edit Only Own Branch

Requirement: Contributor can edit descendants of their grandparent

Implementation:

{
  "editing_filter_json": {
    "rules": [
      {
        "name": "IsDescendantOf",
        "values": ["<BRANCH_ROOT_ID>", "1"]
      }
    ]
  }
}

Middleware check on POST/PUT/DELETE:

def require_edit_permission(handle, obj_type):
    if has_permissions([PERM_EDIT_OBJ]):
        # No restrictions for Editors+
        return
    
    user_id = get_jwt_identity()
    editing_filter = get_user_editing_filter(user_id)
    if not editing_filter:
        abort(403)
    
    if not object_matches_filter(handle, obj_type, editing_filter):
        abort(403, "Object not in your editable branch")

Use Case 3: Graduated Privacy Levels

Requirement: Different privacy tiers (public → family → close family → private)

Implementation:

  1. Database: Add privacy_level integer field to objects (0-3)
  2. User setting: max_privacy_level (Guest=0, Member=1, Contributor=2, Editor+=3)
  3. Proxy filter: Filter objects where obj.privacy_level <= user.max_privacy_level

Migration: Current boolean privateprivacy_level = 3 if private else 0

Security Considerations

  1. JWT Token Size: Embedding full filters in JWT could exceed size limits

    • Solution: Store filter ID in JWT, fetch full filter server-side
  2. Filter Injection: Malicious filter JSON could compromise security

    • Solution: Validate filter JSON against schema before saving
    • Solution: Use read-only filter evaluation (no dynamic code execution)
  3. Performance DoS: Complex filters could overload server

    • Solution: Set max filter complexity (e.g., max 10 rules)
    • Solution: Timeout filter evaluation after N seconds
  4. Write Authorization: Ensure edit filters are properly enforced

    • Solution: Double-check permissions in write operations (defense in depth)
    • Solution: Audit log all permission changes
  5. Information Leakage via Related Objects

    • Risk: User accesses filtered-out person via shared note/event
    • Solution: Transitive policy prevents direct access, only shows object in context
    • Solution: API endpoints check object type and enforce primary filter
  6. Family Relationship Exposure

    • Risk: Seeing family record reveals existence of filtered-out spouse
    • Solution: Proxy database can redact spouse details in family objects
    • Alternative: Family only visible if both spouses pass filter (stricter)

Migration Path

For Existing Installations

Phase 1 - Preparation:

  1. Database migration adds new columns (default NULL)
  2. Existing behavior unchanged (NULL filters = use legacy permissions)

Phase 2 - Opt-In:

  1. Admins can assign filters to specific users
  2. Both systems work in parallel

Phase 3 - Full Migration (Optional):

  1. Convert PERM_VIEW_PRIVATE to filter equivalent
  2. Deprecate legacy permission flags

Performance Benchmarks (Estimated)

Based on typical Gramps database with 10,000 people:

Operation Current Filter-Based (uncached) With Caching
Get person list 50ms 150ms 60ms
Get single person 5ms 8ms 5ms
Relationship calc N/A 200ms (initial) 10ms
Search people 100ms 200ms 110ms

Mitigation strategies:

  • Cache relationship graphs per session
  • Use lazy evaluation (only filter when accessing)
  • Index commonly used relationship paths

Alternative: Quick Win Solutions

If full implementation is too complex, consider these incremental improvements:

Quick Win 1: "Living People" Mode

  • Add PERM_VIEW_LIVING permission
  • Use Gramps' existing LivingProxyDb
  • Restricts viewing to deceased only

Effort: 1 day Benefit: Privacy for living relatives

Quick Win 2: Preset Viewing Scopes

  • Add enum: viewing_scope: "all" | "non-private" | "living-only" | "deceased-only"
  • Map to existing proxy classes
  • No custom filters, just fixed options

Effort: 3 days Benefit: Simple multi-level access without complexity

Quick Win 3: "Home Person" Restriction

  • Add home_person_handle to User model
  • New permission PERM_VIEW_EXTENDED_FAMILY
  • If set, only show people within N degrees

Effort: 1 week Benefit: Relationship-based viewing without full filter system

Questions for You

Before proceeding with implementation, please clarify:

  1. Priority Use Cases: Which is most important?

    • Viewing restrictions (relationship-based)?
    • Editing restrictions (branch-based)?
    • Privacy levels (graduated access)?
  2. User Base: How many users per tree typically?

    • Affects caching strategy and storage considerations
  3. Performance Requirements: What's acceptable latency for filtered queries?

    • Helps choose between caching strategies
  4. Administrative Complexity: Who will manage these permissions?

    • Affects UI complexity requirements
  5. Backward Compatibility: Must all existing installations work unchanged?

    • Determines migration approach
  6. Timeline: What's the target timeframe for MVP?

    • Helps scope initial implementation
  7. Related Objects Policy: For filtered people, should associated objects (events, notes, sources) be:

    • A) Always included (transitive - recommended for usability)
    • B) Also filtered (stricter privacy, but may break context)
    • C) Configurable per user/role?
  8. Information Disclosure Tolerance: Is it acceptable for users to:

    • Know filtered-out people exist (via family relationships)?
    • See event/note titles but not details for filtered objects?
    • Or should filtered objects be completely invisible?

Conclusion

The Filter-Based Permissions (Option 3) approach offers the best balance of:

  • Leveraging existing Gramps infrastructure
  • Flexibility for various use cases
  • Reasonable implementation complexity
  • Good performance with caching

Starting with the "Quick Win" solutions can provide immediate value while the full system is developed.

Recommended Next Steps:

  1. Review and discuss this document
  2. Prioritize use cases
  3. Implement one "Quick Win" for immediate improvement
  4. Begin Phase 1 of full filter-based system
  5. Iterate based on user feedback
@banalg

banalg commented Jul 25, 2026

Copy link
Copy Markdown

brainstormin synthesis with Gemini on 2026-07-25

Hi David,

First, thank you for the incredible work on Gramps Web.

I recently read your Gist exploring advanced filter-based permissions (Option 3). I’d like to propose an architectural alternative that builds upon your ideas—specifically your mention of using ReferencedBySelectionProxyDb—but shifts the computational load away from the SQL database to ensure high performance.

The concept is a Hybrid ReBAC + ABAC model utilizing string-based entitlements (similar to AWS IAM policies) and an in-memory graph.

Here is how it would work:

1. String-Based Entitlements (AWS ARN style)

Instead of complex JSON filters or recursive SQL queries, permissions are defined as strings using a standard funnel approach (Macro to Micro), adding explicit Allow/Deny rules and Condition filters.

Schema: EFFECT:DATASET:ACTION:ROOT_HANDLE:DIRECTION:DEPTH:CONDITION

  • EFFECT: allow or deny
  • CONDITION (optional): alive, deceased, private, public, or *

These strings are stored natively in the Gramps Web database (keeping the system standalone), could be grouped in roles, and can easily be mapped via OIDC custom claims for users with external IdPs.

Here are some practical examples:

Role / Use Case Entitlement Strings Explanation
Global Auditor allow:*:r:*:*:*:* Read-only access to all datasets on the instance.
Historical Archive allow:MyTree:r:*:*:*:deceased Can view the entire tree, but living persons are filtered out (GDPR friendly).
Strict Privacy Shield allow:MyTree:r:*:*:*:*
deny:MyTree:*:*:*:*:private
Can read the tree, but an explicit DENY is enforced on any node flagged as "private" in Gramps.
Branch Contributor allow:MyTree:rw:I0142:DESC:*:* Can read and edit all descendants of a specific historical patriarch/matriarch (Handle I0142).
Direct Ancestry allow:MyTree:r:current:ASC:*:* Can read the logged-in user's direct ancestors infinitely backward, but cannot see cousins or siblings.

2. Standard Templates for New Trees

Gramps Web could propose default roles when a new Tree is created, automatically generating these baseline entitlements:

Default Role Entitlement Strings
Tree Admin allow:{Dataset}:admin:*:*:*:*
Tree Editor allow:{Dataset}:rw:*:*:*:*
Tree Reader (Public) allow:{Dataset}:r:*:*:*:public
Lineage Reader allow:{Dataset}:r:current:ASC:*:*
allow:{Dataset}:r:current:DESC:*:*
Self Only (Strict) allow:{Dataset}:r:current:*:0:*

3. In-Memory Topology via networkx (The PDP, Policy Decision Point)

Running recursive SQL queries on every HTTP request will inevitably cause performance bottlenecks. Instead, upon loading a Dataset, we extract only the bare topology (nodes = Handles, edges = Parent/Child relationships) into a single, shared Python networkx directed graph in RAM.

Crucially, we also append lightweight node attributes during initialization (e.g., G.add_node("I0001", alive=True, private=False)).

Because this topology represents the static resource, the memory footprint is negligible. An entire tree of 100,000 individuals with basic boolean attributes takes less than 50MB. Graph traversals to compute the allowed node list take less than a millisecond.

4. Fail-Fast Middleware (The PEP, Policy Enforcement Point) & Data Leakage Prevention

A proxy intercepts API requests and applies the policies:

  1. It short-circuits instantly if the DATASET doesn't match or if an action is insufficient.
  2. It traverses the networkx graph based on the user's strings, computing the allow list of Person Handles, subtracting any deny matches, and filtering out nodes that don't match the CONDITION (like deceased).
  3. The Data Leakage Solution: We pass this final, highly accurate whitelist of Handles to your native ReferencedBySelectionProxyDb.

By doing so, the native proxy automatically filters out all Media, Notes, and Events that do not belong to the authorized sub-graph or belong to a denied/living person.

This architecture keeps the database schema simple, resolves the GDPR/living persons issue elegantly, guarantees lightning-fast API responses, and natively solves the data leakage problem.

I would love to hear your thoughts on this approach and whether you think this would be a viable direction for a PoC.

Best regards,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment