🔧 ProgrammierungWhat If AI Had a Digital Endocrine System?(16.09.2026 um 03:15 Uhr)
🔧 ProgrammierungMy AWS Learning Journey: CloudWatch, Lambda, IAM & CloudFront(16.09.2026 um 03:17 Uhr)
🔧 AI Nachrichten Flint: Turning Skills into Personal Assets(16.09.2026 um 03:17 Uhr)
🔧 ProgrammierungAWS Rhyming Game Contest – Exploring Amazon Kinesis(16.09.2026 um 03:22 Uhr)
🔧 ProgrammierungChandra Meets CodeDeploy: My First AWS Deployment Journey(16.09.2026 um 03:30 Uhr)
📰 IT Security NachrichtenHow A.I. Risks Are Splitting Silicon Valley and Washington(16.09.2026 um 03:24 Uhr)
🔧 ProgrammierungWhat If AI Had a Digital Endocrine System?(16.09.2026 um 03:15 Uhr)
🔧 ProgrammierungMy AWS Learning Journey: CloudWatch, Lambda, IAM & CloudFront(16.09.2026 um 03:17 Uhr)
🔧 AI Nachrichten Flint: Turning Skills into Personal Assets(16.09.2026 um 03:17 Uhr)
🔧 ProgrammierungAWS Rhyming Game Contest – Exploring Amazon Kinesis(16.09.2026 um 03:22 Uhr)
🔧 ProgrammierungChandra Meets CodeDeploy: My First AWS Deployment Journey(16.09.2026 um 03:30 Uhr)
📰 IT Security NachrichtenHow A.I. Risks Are Splitting Silicon Valley and Washington(16.09.2026 um 03:24 Uhr)

🔧 Programmierung 🕛 vor 3 Monaten 7 Min Lesezeit
0

CrawlForge v4.2.2: New CLI + 3 Tools for Local AI Scraping

↗ Quelle (dev.to)
🗣️ Stimme:
📑 Inhaltsübersicht

Today we are shipping CrawlForge v4.2.2, our biggest release since launch. It brings three new tools, a standalone command-line interface, and a quiet shift in how we think about web scraping for AI: most of it should run locally, on your own machine, without API keys.



This post is the umbrella for everything in 4.2.2. Three deep-dive guides follow in the next nine days.








TL;DR: v4.2.2 adds @crawlforge/cli (a standalone CLI -- no MCP client needed), extract_with_llm (local LLM extraction via Ollama, no OpenAI/Anthropic key required), scrape_template (one-line scrapers for 10 popular sites), and list_ollama_models (free model discovery). Tool count goes from 20 to 23. Free tier still includes 1,000 credits. Install: npm install -g @crawlforge/cli.








Table of Contents




  • What Shipped

  • The New CrawlForge CLI

  • Extract With LLM: Local AI Extraction

  • Scrape Template: Ten Sites, One Call

  • list_ollama_models: Free Model Discovery

  • Old Workflow vs v4.2.2 Workflow

  • Credit Costs

  • How to Upgrade

  • What Is Next









What Shipped



v4.2.2 adds four things:





  1. @crawlforge/cli -- a standalone command-line tool exposing all 23 CrawlForge tools to your shell. No MCP client required.


  2. extract_with_llm -- LLM-powered structured extraction that defaults to local Ollama. No external API key needed.


  3. scrape_template -- pre-built scrapers for Amazon, LinkedIn, GitHub, YouTube, Reddit, Hacker News, Stack Overflow, npm, Product Hunt, and Twitter/X.


  4. list_ollama_models -- a free discovery tool that lists models on your local Ollama instance.



Tool count goes from 20 to 23. The CLI is brand new -- it is not a tool, it is a delivery channel.




CODE
+----------------+       +-------------------+       +----------------+
| Your Shell | <--> | @crawlforge/cli | <--> | CrawlForge |
| (cron, CI) | | (JSON in/out) | | API + Tools |
+----------------+ +-------------------+ +----------------+
^
No MCP handshake.
Just HTTPS + stdout.












The New CrawlForge CLI



The CLI is the shortest path from intent to scraped data. You install it once, set an environment variable, and every CrawlForge tool becomes a command:




CODE
npm install -g @crawlforge/cli
export CRAWLFORGE_API_KEY="cf_live_your_key_here"

crawlforge scrape https://example.com
crawlforge search "best MCP servers 2026"
crawlforge research "AI agent frameworks" --depth 3






Why does this matter? Because MCP is great for AI agents, but a lot of scraping work is not an AI agent task. It is a cron job. A CI step. A one-off pull from your terminal. For that, you want JSON on stdout that pipes into jq, not a JSON-RPC handshake.




Why have a CLI when MCP already exists?

MCP is optimized for AI agents picking tools dynamically. The CLI is optimized for humans typing commands and scripts piping JSON. Different shapes for different jobs:
































Workflow Best fit
Claude/Cursor agent MCP
Cron job CLI
GitHub Actions step CLI
One-off terminal CLI
Server in a loop Raw API


All three paths hit the same backend, share the same credit balance, and use the same API key.







Read the .









Scrape Template: Ten Sites, One Call



scrape_template is for the long tail of scraping requests that all look the same: "get me product data from Amazon", "get me a GitHub repo's metadata", "get me the top posts on Hacker News today". You should not need to write CSS selectors for these. We did it once, we maintain it, you call it.




CODE
crawlforge template amazon --url "https://www.amazon.com/dp/B0CHX1W1XY"
crawlforge template github --url "https://github.com/anthropics/anthropic-sdk-python"
crawlforge template hackernews --top 10






Ten templates ship in this release:































































Template What it returns Credits
amazon Product title, price, rating, reviews, images 1
linkedin Profile name, headline, experience, skills 1
github Repo metadata, stars, languages, README 1
youtube Video title, views, channel, transcript 1
reddit Post title, score, comments, top replies 1
hackernews Story title, points, URL, comments 1
stackoverflow Question, answers, accepted, vote counts 1
npm Package metadata, weekly downloads, versions 1
producthunt Product name, tagline, upvotes, makers 1
tweet Tweet text, author, engagement, replies 1


Full walkthrough with code: .


  • Webhook delivery for batch_scrape -- get results pushed to your endpoint when long-running jobs complete.


  • CLI watch mode -- crawlforge track --watch for live diffs on monitored pages.






  • Ready to try the new tools? Free tier still includes 1,000 credits and no credit card.





  • Vollständiger Original-Artikel
    Den kompletten Beitrag mit allen Details direkt auf dev.to lesen.
    ↗ Original-Artikel auf dev.to lesen
    Wie bewertest du diesen Beitrag?
    1 Klick Feedback
    Teilen mit Netzwerk & Team:

    Community-Analysen & Experten-Meinungen 0

    Verfasse deine eigene Analyse, teile Workarounds oder diskutiere diesen Vorfall im Blog.
    Noch keine Community-Analyse verfasst. Markiere einen Textabschnitt oder klicke oben auf Eigene Analyse verfassen“!
    Community Pulse: Relevanz-Einschätzung
    1 Klick Experten-Votum
    🔴 Akute Relevanz 0%
    🟡 In Evaluierung 0%
    🟢 Keine Auswirkung 0%
    Spannende Innovation 0%
    Verwandte Story-Cluster & Quellen (Vektor-KI)
    Port 8095 Engine
    1 Quelle
    What If AI Had a Digital Endocrine System?
    1 Quelle
    My AWS Learning Journey: CloudWatch, Lambda, IAM & CloudFront
    1 Quelle
    Flint: Turning Skills into Personal Assets
    Ähnliche Beiträge
    🔍 Verwandte News

    Auch interessante Nachrichten CrawlForge v4.2.2: New CLI + 3 Tools for Local AI Scraping

    Thematisch verwandte Begriffe: CrawlForge, v422, Tools, Local · 6 Treffer

    Laden...

    Videos werden geladen ...

    Laden...

    Beiträge werden geladen ...

    Laden...

    Videos werden geladen ...

    Laden...

    Beiträge werden geladen ...

    Laden...

    Videos werden geladen ...

    Laden...

    Beiträge werden geladen ...

    Laden...

    Videos werden geladen ...

    Laden...

    Beiträge werden geladen ...

    Laden...

    Videos werden geladen ...