8 small data transforms I don't want to write as shell glue anymore

🦀🐍 Practical fimod examples for small CI/config data transforms with a Python-like taste.

In the first article,
I wrote about why I built fimod: I kept seeing tiny python3 -c scripts in CI pipelines, doing just enough JSON/YAML/CSV work to be useful, and just enough shell quoting to become annoying.

This article is more concrete.

Here are 8 small transforms that I do not want to keep rewriting as shell glue, ad-hoc Python snippets, or copied scripts between repositories.

fimod is not trying to replace jq or yq. Those are excellent tools. I see fimod more as a small data-shaping Swiss army knife: a few megabytes, Python-like expressions, common structured formats, reusable molds, and a controlled execution surface.

📚 Website/docs: https://pytgaen.github.io/fimod/

⭐ Repository: https://github.com/pytgaen/fimod

1. Extract one value for a shell script

Sometimes a CI job only needs one value from a structured file.

{"name":"demo","version":"1.2.3"}

With fimod:

fimod s -i package.json \
  -e 'data["version"]' \
  --output-format txt

Output:

1.2.3

That makes it easy to use in shell:

VERSION=$(fimod s -i package.json -e 'data["version"]' --output-format txt)

No JSON boilerplate, no quotes around the string, no temporary script.

2. Validate a config file in CI

For boolean checks, --check suppresses stdout and maps truthiness to the exit code.

fimod s -i deploy.yaml \
  -e 'all(k in data for k in ["image", "replicas", "port"])' \
  --check

For clearer error messages, use gk_assert:

fimod s -i deploy.yaml -e '
def transform(data, **_):
    gk_assert("image" in data, "missing image")
    gk_assert("replicas" in data, "missing replicas")
    gk_assert("port" in data, "missing port")
    return True
' --check

On failure, the error messages go to stderr and the process exits non-zero — the shape CI expects.

3. Read an HTTP API directly

For small API transforms, fimod can read HTTPS URLs directly:

fimod s -i https://api.github.com/repos/pytgaen/fimod \
  -e '{"name": data["name"], "stars": data["stargazers_count"]}'

Example output:

{
  "name": "fimod",
  "stars": 4
}

This is not meant to replace a full HTTP client in an application. But for CI metadata, release scripts, or quick API-to-config glue, it avoids another curl | jq | sed chain.

4. Extract named regex captures

fimod injects regex helpers into every transform. re_search returns structured data, including named captures.

echo '{"tag":"release-v2.4.1"}' \
  | fimod s \
      -e 're_search(r"(?P<major>[0-9]+)\.(?P<minor>[0-9]+)", data["tag"])["named"]'

Output:

{
  "major": "2",
  "minor": "4"
}

Under the hood, the regex helpers use Rust's fancy-regex crate, with PCRE-like features such as lookahead, lookbehind, backreferences, and named captures.

5. Flatten nested fields to CSV

APIs often return nested JSON, while the next step wants a flat CSV artifact.

[
  {"name":"Alice","email":"[email protected]","address":{"city":"Paris"}},
  {"name":"Bob","email":"[email protected]","address":{}}
]

fimod s -i users.json \
  -e '[{"name": u["name"], "email": u["email"], "city": dp_get(u, "address.city", "unknown")} for u in data]' \
  -o contacts.csv

Output:

name,email,city
Alice,[email protected],Paris
Bob,[email protected],unknown

dp_get avoids a small pile of defensive nested dict.get(...) calls.

6. Hash sensitive fields before exporting

For anonymized fixtures or safer artifacts, hashing is built in:

fimod s -i people.csv \
  -e '[{**row, "email": hs_sha256(row["email"])} for row in data]' \
  -o people-anon.csv

Input:

name,email
Alice,[email protected]
Bob,[email protected]

Output shape:

name,email
Alice,ff8d9819fc0e12bf0d24892e45987e249a28dce836a85cad60e28eaaa8c6d976
Bob,5ff860bf1190596c7188ab851db691f0f3169c453936e9e1eba2f9a47f7a0018

The point is not that hashing is hard in Python. It is not. The point is that in a small CI transform, I do not want to import and wire another script just for that.

7. Generate text/config with MiniJinja

fimod is not limited to data-to-data transforms. It can also render text from structured data using MiniJinja-powered helpers.

echo '{"app":"api","host":"localhost","port":8080}' \
  | fimod s --output-format txt \
      -e 'tpl_render_str("APP={{ app }}\nURL=http://{{ host }}:{{ port }}\n", data)'

Output:

APP=api
URL=http://localhost:8080

This opens the door to .env files, Markdown snippets, Dockerfiles, small config files, or release notes generated from structured data.

For larger templates, fimod also supports rendering template files from directory molds with tpl_render_from_mold(...).

8. Reuse a mold from a registry

One-liners are nice, but the more interesting part is sharing transforms.

A mold is a reusable transform script. A registry lets you call molds by name with @name instead of copying scripts between repositories.

For example, with the example registry configured, you can call the shared pick_fields mold by name:

fimod s -i contacts.csv \
  -m @pick_fields \
  --arg fields=name,email \
  -o contacts-public.json

This works because fimod resolves @pick_fields from a configured registry. In a fresh environment, you can configure the default registry once, or point this command at the official example registry explicitly with FIMOD_REGISTRY=https://github.com/pytgaen/fimod/tree/main/molds.

Input:

name,email,role
Alice,[email protected],admin
Bob,[email protected],user

Output:

[
  {
    "name": "Alice",
    "email": "[email protected]"
  },
  {
    "name": "Bob",
    "email": "[email protected]"
  }
]

In a team, that registry can be your own Git repository. That is where fimod becomes more than a one-liner tool: reviewed transforms can be reused across projects without copy-pasting another tiny Python script everywhere.

So when would I use this?

Fimod is a great fit for CI/config data plumbing: one small but powerful binary that can read many structured formats, pull input from HTTP when the data lives behind an API, and reuse molds when a transform should become shared project knowledge.

The registry and sandbox matter for that last part. They make sharing a transform less about copying a script around, and more about resolving known code from a known place, with a controlled execution surface.

I would reach for it when I need a small, explicit data transform in a pipeline:

• extract one value;
• validate a config;
• reshape JSON/YAML/CSV/TOML;
• read a small API response;
• hash or mask fields;
• generate a tiny text artifact;
• reuse a shared transform as a mold.

That is where I want fimod to shine: boring CI/config transforms with less glue.

If you try one of these examples, I would start with a small file you already manipulate in a script today.

• ⭐ If the idea looks useful, a GitHub star helps the project be discovered.
• 💬 If you have a quick reaction, a comment here on dev.to is perfect.
• 🐛 If an example breaks or the docs are unclear, a short GitHub issue with the command you tried is more than enough.