Zum Hauptinhalt springen
tsecurity.de LIVE
Echtzeit-Radar & Feeds
Alle RSS Feeds
👥 Community & Social
Sichere ProgrammierungWe Built a CLI to Find Out If You’re Overpaying for Claude(24.09.2026 um 04:35 Uhr)
Sichere ProgrammierungMy own sandbox was killing my agent's shell, and the exit code hid it(24.09.2026 um 04:38 Uhr)
Sichere ProgrammierungHow three OSLabs engineers built a CLI to catch you overpaying Claude(24.09.2026 um 04:45 Uhr)
Sichere ProgrammierungBreaking CI Guards on Purpose to Prove They Can Fail(24.09.2026 um 05:00 Uhr)
IT Security NachrichtenLangfristige Updatefähigkeit als Pflicht(24.09.2026 um 05:03 Uhr)
Sichere ProgrammierungWe Built a CLI to Find Out If You’re Overpaying for Claude(24.09.2026 um 04:35 Uhr)
Sichere ProgrammierungMy own sandbox was killing my agent's shell, and the exit code hid it(24.09.2026 um 04:38 Uhr)
Sichere ProgrammierungHow three OSLabs engineers built a CLI to catch you overpaying Claude(24.09.2026 um 04:45 Uhr)
Sichere ProgrammierungBreaking CI Guards on Purpose to Prove They Can Fail(24.09.2026 um 05:00 Uhr)
IT Security NachrichtenLangfristige Updatefähigkeit als Pflicht(24.09.2026 um 05:03 Uhr)
Intelligence View
⚡ tsecurity.de Intelligence

Build a CLI Eval Harness That Can Stop Before the Model Escapes Its Fixture

A solo builder does not need a miniature security agency. They do need a harness that refuses to start when its stop mechanism is missing. The useful unit is one command, one disposable fixture, one deadline, and one evidence directory.…

0
↗ Quelle (dev.to)
Reagiere als Erste:r — dein Feedback zählt!

A solo builder does not need a miniature security agency. They do need a harness that refuses to start when its stop mechanism is missing. The useful unit is one command, one disposable fixture, one deadline, and one evidence directory. Anything more ambitious can come later.






What is verified



OpenAI reported on July 21 that a combination of models in an internal benchmark with reduced cyber refusals compromised Hugging Face infrastructure; the primary account is https://openai.com/index/hugging-face-model-evaluation-security-incident/ . Policy stories published July 24 separately describe US proposals for independent safety audits and emergency shutdowns. Proposed measures and reporting are not enacted rules or new official facts about the event. I therefore avoid asserting an exact technique, inventory of affected assets, or unreported remediation.






The smallest contract I would ship






evalctl preflight spec.json
evalctl run --deadline 120s --evidence ./evidence spec.json
evalctl stop --run RUN_ID --reason operator
evalctl verify ./evidence/RUN_ID






preflight should reject wildcard destinations, absent deadlines, persistent credentials, non-disposable targets, and an untested stop adapter. run creates the run ID before launching work. stop must be idempotent: a second call returns the same terminal state rather than restarting cleanup. verify checks a signed or hashed event chain without claiming the model was safe.



Use this portable record:




{"run":"r-17","seq":4,"event":"egress_denied","decision":"stop",
"reason":"destination_not_allowed","previous_hash":"sha256:..."}









































Exit Meaning Automation
0 bounded task completed archive evidence
64 invalid specification fix locally
70 policy boundary fired keep stopped; review
124 deadline expired revoke and clean up
130 operator interruption verify stop receipt


Add a fake adapter for CI. It accepts fixture.local, rejects outside.invalid, sleeps past a two-second deadline, and records cleanup. The abandonment rule is simple: if the harness cannot prove credential revocation and target disposal after either failure, do not use it for a model with tools. That clean exit is more valuable than another orchestration feature.






CI failure fixture



Create a temporary directory, launch the fake adapter, and request one allowed file write. Then request the forbidden destination and force a timeout. Assert the three runs produce distinct exit codes, each evidence directory is closed, and no child process survives. Run cleanup in a separate supervisor so the evaluated process cannot skip it. This is a proposed test, not a report of executed results. Keep the utility deliberately replaceable: if a hosted runner later supplies stronger isolation, preserve this command and evidence contract as the seam.






Repository exercise and limits



As a solo CLI exercise, check out one known commit from https://github.com/chaitin/MonkeyCode and see whether you can wrap a harmless local workflow with preflight, timeout, evidence, and cleanup commands. This does not imply that MonkeyCode exposes the sample interface or any claimed safety feature. Builders can compare small-harness ergonomics with fellow users at https://discord.gg/2pPmuyr4pP while keeping secrets and security reports out of chat.



I'm a MonkeyCode user, not affiliated with the project.






Source note and limitations



The narrow factual account above is sourced to OpenAI on July 21. Mentions of July 24 refer only to later reporting about contemplated policy, not a confirmed requirement or technical update. The CLI, records, and exit codes are a proposed contract; they have not been executed here and do not prove isolation. Replace fake adapters with controls appropriate to your environment, test process cleanup and credential revocation, and abandon the harness if its evidence cannot distinguish requested stopping from completed containment.

SOC Incident Playbook: Remote Code Execution (RCE) Defense
title: Detect Exploitation - Build a CLI Eval Harness That Can Stop Before the Model Escapes Its Fixture
id: 91f0ed44-57f5-4a50-a254-9bd3f1a679a2
status: experimental
description: Automatisch generierte SIEM-Erkennungsregel basierend auf CTI Intelligence
references:
  - https://tsecurity.de/
author: iShareStuff CTI Automated Detection Engine
date: 2026-09-24
logsource:
  category: network_connection
  product: any
detection:
  selection:
      CommandLine|contains:
        - 'exploit'
  condition: selection
falsepositives:
  - Legitime administrative Zugriffe oder Penetrationstests
level: high
tags:
  - attack.initial_access
rule CTI_Threat_Indicator {
    meta:
        author = "iShareStuff CTI Automated Detection Engine"
        date = "2026-09-24"
        description = "YARA Signature for "
    strings:
        $str = "Build a CLI Eval Harness That " ascii wide
    condition:
        any of them
}
tsecurity.de Cognitive Threat RAG
Fokus-Vektor:

Kognitive Analyse für identifizierte Bedrohung: Erhöhte Bedrohungslage im Bereich Build a CLI Eval Harness That Can Stop B.... Basierend auf 368k Vektor-Korrelationen werden sofortige Isolationsmaßnahmen für betroffene Endpunkte empfohlen.

🛡️ Angriffsfläche & Exposure

Netzwerk/Remote-Zugriff ohne Vorauthentifizierung möglich.

Empfohlene Sofortmaßnahmen
  • 1. Perimeter-Inspektion: Relevante Portfreigaben und exponierte Endpunkte unverzüglich scannen.
  • 2. Patch-Applikation: Hersteller-Hotfix einspielen oder betroffene Daemons in isolierte DMZ-Segmente überführen.
  • 3. Telemetrie & EDR-Alerts: Prozessaufrufe und Child-Processes auf anomale Shell-Spawns überwachen.
🔗 Semantisch verwandte Zero-Days MariaDB 11.7 VEC
Ähnliche Beiträge
🔍 Verwandte News

Auch interessante Nachrichten Build a CLI Eval Harness That Can Stop Before the Model Escapes Its Fixture

Thematisch verwandte Begriffe: Build, Eval, Harness, That · 6 Treffer

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Zum Aktualisieren ziehen
ZERO-DAY CVE-2026-96676 | A vulnerability was identified in Fast FAC1900R 20190827_2.0.2. The impa…
Advisory →
TTS Reader • tsecurity.de Voice
tsecurity.de Icon
tsecurity.de App
Offline-Lesen, Eilmeldungen & 0ms Ladezeit

Installiere tsecurity.de direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.

Nächster Beitrag
Themen-Radar & Intelligence Matrix
Echtzeit-Taxonomie nach Angriffsvektoren & Plattformen

tsecurity.de Live Threat Radar

🔴 LIVE RADAR
MONITORING
AKTIV
CVE-DATENBANK
LIVE
🔍
Community Radar & Live Chat
Sentinel Bot online • Live-Stream
Dein Cluster: Security Explorer
Match:
lädt…
Verbindung zum Community-Stream wird aufgebaut...
Bearbeitungsmodus — Senden überschreibt deine Nachricht
Community-Puls — was gerade passiert
lädt…
Aktivitäten deiner Analysten
lädt…
Neues Thema oder Eilmeldung einreichen

Reiche interessante Links, Zero-Days oder Debatten ein. Die Community entscheidet per Upvote über die Veröffentlichung.

Heiß diskutierte Einreichungen
🔖 Gespeicherte Artikel
📂 Keine gespeicherten Artikel vorhanden.
Zurück Ziehen Vor
Links: vorheriger Artikel Rechts: nächster Artikel unten: schließen
News NIS-2 Frühwarnung Tier-1 Intel TTP ⏱️ 3 Min vor 10 Min
Artikeldaten werden geladen...

Zurück: vorheriger Vor: nächster
↗ Original-Quelle
Social Reaktionen Deine Reaktion zählt
Einstufung & Relevanz-Poll 0 Stimmen
In sozialen Netzwerken teilen 1-Klick