Securing AI Document Systems: Implementing the Four-Perimeter Framework with Permit.io

Project Repo

Repo link: github

My Journey

Implementing the AI Document Assistant with robust security controls was both challenging and rewarding.

Getting Started: Understanding the Four-Perimeter Framework I began by diving deep into Permit.io's Four-Perimeter Framework to understand how these security layers could be implemented in a practical application. This required shifting my thinking from "can the user authenticate?" to "what specific actions should this user be allowed to perform on this specific resource?"

Architectural Decisions: I chose FastAPI for the backend due to its performance and built-in OpenAPI documentation. For AI capabilities, LangChain provided the flexibility to integrate with various LLMs while creating abstractions for document processing. The Permit.io SDK offered the authorization layer that connected everything together.

The most critical architectural decision was designing the security perimeters as separate, modular components that could intercept and authorize requests at different stages of the AI pipeline.

Challenges and Solutions

Challenge 1: Integrating Authorization with RAG Architecture

Implementing authorization in a RAG (Retrieval-Augmented Generation) system was complex because I needed to control not just what documents a user could see, but what content the AI model could access on their behalf.

Solution: I split the RAG protection into pre-query and post-query filtering. This allowed me to filter document IDs before retrieval and then sanitize the content before passing it to the LLM.

Challenge 2: Stubbing External Systems for Testing

Operations like "analyze" and "translate" would normally call external APIs, but implementing these connections would have been overkill for a proof of concept.

Solution: I created a simulated external access control system that would deny these operations by default, demonstrating the separation of duties principle without implementing actual external integrations.

Challenge 3: Handling Authentication Errors

I struggled with 401 and 403 errors during testing. It wasn't always clear whether the issue was with authentication or authorization.

Solution: I improved error handling and debugging in the auth endpoints, which helped identify that the OAuth token URL path was incorrect. This seemingly small detail was causing the whole authentication flow to fail.

Challenge 4: Developing Without External APIs

Working with OpenAI and Permit.io in a development environment proved challenging when API keys weren't available or rate limits were hit.

Solution: I implemented fallback mechanisms that used mock responses when external services were unavailable, making development smooth even without constant API access.

What I Learned

This project taught me several valuable lessons:

• Security is a Feature, Not an Afterthought: Integrating security controls from the beginning led to a cleaner, more robust architecture.




• Separation of Duties is Powerful: Having operations that even admin users can't perform without approval creates significantly stronger security guarantees.




• Role-Based vs. Attribute-Based Access Control: I gained a deeper understanding of how RBAC is insufficient for AI systems, where ABAC (Attribute-Based Access Control) provides the necessary granularity.




• Error Handling is Critical for Security UX: Clear, specific error messages make security issues easier to debug without revealing sensitive implementation details.

The most important insight was realizing that authorization for AI systems needs to happen at multiple layers. Simply checking if a user can access a document isn't enough - you need controls at the prompt level, the data access level, the external API level, and the response level to create truly secure AI systems.

This approach to AI security feels like the right direction for building enterprise AI applications that can handle sensitive data while respecting organizational boundaries and compliance requirements.

Authorization for AI Applications with Permit.io

I implemented Permit.io's Four-Perimeter Framework to create defense-in-depth for AI systems.

1. Prompt Filtering:
   
   I built a prompt filtering layer that validates user prompts before they reach the AI models. This ensures users can only submit prompts for operations they're authorized to perform, based on their role and document sensitivity.




2. RAG Data Protection:
   
   I created two-phase protection for document access:
   
   Pre-query filtering to control which documents can be retrieved
   
   Post-query filtering to sanitize retrieved content
   
   This ensures the AI model only accesses document content the user is authorized to see.




3. External Access Control:
   
   For operations requiring external APIs (analyze, translate), I implemented controls that enforce separation of duties. Even admin users need explicit approval for these sensitive operations, demonstrating the principle that privileged access doesn't mean unlimited access.




4. Response Enforcement:
   
   The final security layer filters AI-generated responses before returning them to users, ensuring content is appropriate for the user's permission level.

Role-Based Permission Design

I implemented three distinct user roles with different permission levels:

Admin users: Can upload documents and perform most operations except those requiring external access

Premium users: Can read documents and perform basic AI operations but can't upload

Basic users: Can only read their own documents and use the simplest AI operation (question answering)

Implementing authorization for AI applications taught me that AI systems need much more fine-grained controls than traditional apps. Context (document sensitivity, operation type) is crucial for authorization decisions. Even admin users should face restrictions for high-risk operations. Well-designed permission denials improve security understanding

Using Permit.io's attribute-based access control, I was able to implement these nuanced authorization controls that go far beyond simple role-based access, providing the security needed for enterprise AI applications handling sensitive data.