My Headless LinkedIn Bot Fails Silently Every Night — Until I Point SigNoz at It
Subtitle: From ChromeDriver 151 vs Brave 150 to correlated traces: what I learn instrumenting a Selenium cron job with OpenTelemetry
Every morning I open yesterday's log file and scroll, guessing.
Some days the bot sends zero connection requests. Other days it hits 27 of 30 and stops. The script runs headless on a schedule — no browser window, no Slack alert, just a text file that might not tell the whole story.
I built this for a SigNoz hackathon. The goal isn't a polished observability tutorial. It's to stop flying blind on a real Python + Selenium bot running on my Mac. Here's what actually breaks, what I instrument, and what shows up in SigNoz when I finally look.
What I Run
The bot sends LinkedIn connection requests up to a daily cap. I run headless Brave on macOS via launchd:
./schedule/install_daily_schedule.sh install
# At login + hourly while the Mac is on; at most one successful run per day
Relevant settings from .env:
BROWSER_HEADLESS=true
USE_BROWSER_PROFILE=true
USE_NETWORK_GROW=true # fallback when people search is exhausted
AUTO_ACCEPT_INVITATIONS=true # accept pending invites on Grow first
LINKEDIN_CONNECTIONS=30
LINKEDIN_DRIVER_AUTO_FIX_ATTEMPTS=2
SIGNOZ_ENABLED=true
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
OTEL_SERVICE_NAME=linkedin-connection-bot
When people search runs out, the bot switches to My Network → Grow, accepts incoming invitations, then sends Connect requests. Resume state lets a crashed run pick up on Grow without redoing search. A wrapper script (schedule/daily_run.sh) handles lock files, stale process cleanup, and one chromedriver auto-fix retry.
Simple on paper. Messy in production.
The Failures I Can't See From One Log Line
ChromeDriver 151 vs Brave 150
On July 17, most hourly launchd checks looked like this:
Critical error: Failed to initialize brave: ...
This version of ChromeDriver only supports Chrome version 151
Current browser version is 150.0.7871.125
Brave hadn't auto-updated. Selenium had cached ChromeDriver 151. The bot never got past browser launch — zero connections, 784 lines of stack trace.
I add self-healing in Scripts/browser_utils.py: detect the mismatch, kill stale chromedriver and Brave processes, re-download the matching driver:
def auto_fix_chromedriver_mismatch(browser_name, log_func=None):
cleanup_stale_bot_processes(browser_name, log_func=log_func)
path = resolve_chromedriver_path(browser_name, force_refresh=True)
_log(log_func, f"Auto-fix: resolved chromedriver at {path}")
return path
That fixed launches once Brave caught up. But I still want to know when it happens without grepping logs at 7 a.m.
Almost Done: 27/30 and a Hangup
Late on July 17, after auto-fix worked, the bot resumed Grow-only mode and sent connections steadily — until:
Progress: 27/30 connections sent (grow)
Progress: 28/30 connections sent (grow)
Cleaning up bot browser processes...
./schedule/daily_run.sh: line 19: 52556 Hangup: 1 "$VENV_PYTHON" run_bot.py
Two short of the cap. Killed by SIGHUP. Without exported counters, the only signal is a progress line buried in a 700-line file.
Stale Locks
While one process is still running, the next hourly check logs: Another bot run is already in progress (pid 51947). Skipping. Earlier that evening I saw: Removing stale lock file. The lock logic is correct. Interrupted runs still leave footguns. Metrics like bot.driver_errors and bot.session_crashes surface this on a dashboard instead of me tailing files.
One Module, Three Signals
Everything lives in Scripts/telemetry.py — OTLP gRPC to port 4317, gated by SIGNOZ_ENABLED. Each run gets a 12-character bot.run_id on traces, metrics, and logs.
Spans: bot.run, bot.configure_browser, bot.login, bot.search_page, bot.accept_invitations, bot.connect_from_grow, bot.recover_session.
Metrics: bot.connections_sent (label source=search|grow), bot.invitations_accepted, bot.driver_errors (error_type=version_mismatch), bot.session_crashes, bot.run_success / bot.run_failure, bot.run_duration_seconds.
Main loop wiring:
tel.init_telemetry()
tel.log_event(logger, "Bot run started", phase="start")
with tel.span("bot.run", phase="run"):
configurations()
login(webpage, user)
tel.record_run_outcome(success=not run_failed, connections_sent=connections_sent)
tel.record_run_duration(time.time() - run_started_at)
On driver mismatch: tel.record_driver_error(error_type="version_mismatch") before auto-fix runs.
Logs append run_id and phase for trace correlation in SigNoz Logs Explorer:
Verify telemetry smoke test | run_id=87626c6dc93a phase=verify
trace_id=7b69827901b867b9f0658ed774ff684d span_id=0e08651f63956788
One lesson I keep relearning: logs need explicit OTLP export. telemetry.py wires a LoggingHandler plus OTLPLogExporter — stdout files alone aren't enough.
Three-Pillar Correlation — The Part I Care About Most
SigNoz's real payoff for me isn't three separate tabs. It's jumping between logs, traces, and metrics without rebuilding queries. ).
How my bot wires it
| Pillar | What I emit | Correlation hook | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Traces | Spans like bot.run, bot.login, bot.connect_from_grow | bot.run_id span attribute; root span wraps the whole cron run | ||||||||||||||||||
| Metrics | bot.connections_sent, bot.run_success, etc. | Same bot.run_id label on every counter/histogram sample | ||||||||||||||||||
| Logs | log_event() lines with run_id= and phase= in the body | LoggingInstrumentor injects trace_id / span_id when the log fires inside an active span (), which gives me operation-level rates alongside my custom bot.* counters.
|
| # | File | Use in section |
|---|---|---|
| 1 | ui-logs-linkedin-bot.png | Instrumentation / SigNoz verification — Logs Explorer with run_id, phase, trace correlation |
| 2 | ui-traces-bot-run.png | SigNoz verification — bot.verify_telemetry spans |
| 3 | ui-metrics-connections-sent.png | SigNoz verification — bot.connections_sent by source |
| 4 | ui-metrics-run-success.png | SigNoz verification — bot.run_success over 24h |
| 5 | ui-services-overview.png | SigNoz verification — Services page with linkedin-connection-bot |
tags: OpenTelemetry · SigNoz · Selenium · Observability · Python
Community-Analysen & Experten-Meinungen 0
Verwandte Story-Cluster & Quellen (Vektor-KI)
Ähnliche Beiträge
Auch interessante Nachrichten My Headless LinkedIn Bot Fails Silently - Until I Point SigNoz at It
Thematisch verwandte Begriffe: Headless, LinkedIn, Fails, Silently · 6 Treffer
The Gemini desktop app is now available for Windows
Header and Footer not showing in Excel
Burn Out, Or Fade Away
VPN-Problem mit Windows Update vom September-Patchday - Swiss IT Magazine
Remote Desktop Services hängt: KB5124008-Fix 2026 - WindowsPower.de
Fehlerhaftes Sicherheits-Update lässt Windows verstummen - PC-WELT
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
🔖 Gespeicherte Artikel
myDraft App
Installiere die App direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.
Community Radar & Live Chat
Aktivitäten deiner Analysten
Neues Thema oder Eilmeldung einreichen
Reiche interessante Links, Zero-Days oder Debatten ein. Die Community entscheidet per Upvote über die Veröffentlichung.
SOCIAL SHARE CARD GENERATOR