Setting you up to fail

I saw the following in a job description:

Design agents that utilise LLMs for reasoning, document extraction, and the end-to-end automation of complex enterprise processes. You will implement AI best practices to ensure our autonomous agents act with precision, security and industrial-grade reliability.

(emphasis mine)

This is setting you up to fail and is a classical XY problem waiting to happen.

If you don't know the , CMS systems built around git (yes, it works, but CMS's are meant for people who like WYSIWYG editors) and fun tools like . And LLMs are a lot of things, but they are not precise, secure nor reliable. And so, because the CTO (or more likely the Marketing Manager turned 10x Vibe Coder) has already said the solution is LLMs, you are now going to see the following problems:

Precision

Well, how about counting letters in words or asking for words with 5 letters or painting any picture with text in it. It's really easy to see that LLMs are not precise. If they were precise you wouldn't need and many more . This makes it a cat and mouse game and it won't be long till consultant agencies will step up, since they are already skilled in the tax evasion game, the stay compliant game and the aggressive accounting game.

Reliable

Related to precision and following instructions, reliability is often achieved in software by being deterministic: same input yields the same output. This isn't true for LLMs, even at temperature 0, drift occurs. This isn't a bug, it's a feature of the architecture.

Contrary to popular belief, LLMs are ill-equipped for reliable automation of repetitive tasks. There's no telling when it will do something it shouldn't, what the consequences are, if it will do it again and how likely that will be.

When routing your emails and flagging something as urgent, you may find errors not that important. You can laugh about .

You have been warned.