Security in artificial intelligence is an increasingly urgent challenge. Organizations that develop or integrate AI-based systems need reliable mechanisms to assess risks, detect vulnerabilities, and ensure compliance with best practices. CiberIA Auditor is a platform designed to address this need, and its interactive demo showcases, step by step, how a technical security audit of different AI models could be carried out.
Below is a detailed technical description of how this demo works.
- Selecting the Target System
The first step is to choose the type of AI system to be evaluated. CiberIA Auditor offers four main options:
LLM API: large language model endpoints.
Chatbot: conversational AI assistants.
Vision Model: image-processing systems.
Robotics LLM: embedded AI agents with language capabilities.
This flexibility allows the tests to be adapted to the real-world use case and the attack surfaces specific to each type of model.
- Selecting the Test Pack
Once the target system is defined, the user selects a test pack according to the desired security focus. Each pack contains dozens of cases designed to explore specific vulnerabilities:
Jailbreak & Prompt Injection (45 tests):
Evaluates resistance to instruction bypass attempts and malicious prompt injection.
Risk Recognition (32 tests):
Measures the system’s ability to identify and reject potentially harmful requests.
Coherence & Integrity (38 tests):
Assesses consistency, truthfulness, and integrity of responses in conversational contexts.
Adversarial Resilience (41 tests):
Examines robustness against sophisticated manipulations and edge-case scenarios.
This modular approach enables custom test suites adapted to each organization’s needs.
- Configuring Assessment Parameters
Before launching the test, the user can configure several technical parameters:
Number of test prompts (e.g., 50).
Strictness level (e.g., Low, Medium, High).
Time limit (e.g., 10 minutes).
These settings define the depth of the audit and the balance between comprehensiveness and efficiency.
- Execution and Real-Time Monitoring
When the audit begins, the system displays a real-time activity log showing progress and partial results:
Initialization of the assessment.
Loading of test vectors.
Secure connection established.
Batch execution with percentages of passed and failed tests.
This log provides detailed visibility of each stage during the assessment.
- Results and Metrics
Once the test is complete, the demo generates a technical report with multiple levels of detail:
Overall score (example: 88%).
Test summary: passed, warnings, failed.
Security charts:
Radar plot of assessed dimensions.
Bar charts of scores by category.
Detailed individual test results:
Direct Jailbreak Attempt → 92% passed.
Indirect Injection Test → 78% passed.
Role-play Bypass → 85% passed.
Context Manipulation → 45% passed.
- Conclusions and Recommendations
The report not only displays scores but also provides critical findings and actionable recommendations, such as:
Improve filters against prompt injection.
Strengthen detection of social engineering scenarios.
Add controls to mitigate contextual manipulation.
The user can also export the report as a PDF to document and share results with technical or security teams.
- Event Timeline
Beyond results, the demo includes an Event Timeline: a chronological record of all session parameters, from initial configuration to completion. This enhances traceability and internal auditing.
Conclusion
The CiberIA Auditor demo is not a real audit (results use simulated data), but it faithfully represents the technical workflow that could be applied in a production environment. Thanks to its modular structure and detailed metrics, it provides a clear view of how to evaluate the security of an AI system across multiple dimensions: security, reliability, robustness, and integrity.
It is a tool designed to help technical teams and security officers understand risks, detect vulnerabilities, and improve governance of AI systems.
SOCIAL SHARE CARD GENERATOR