Zum Hauptinhalt springen
••••••••••••••••••••
Intelligence View
⚡ tsecurity.de Intelligence

Building a Distributed GitHub Streak Monitor with Cloudflare Workers

Building a Distributed GitHub Streak Monitor with Cloudflare Workers Don't let your GitHub streak disappear! I built Streaky - a web app that automatically…

Beitrag
0
Seite
0
↗ Quelle (dev.to)
Social ReaktionenReagiere als Erste:r — dein Feedback zählt!




Building a Distributed GitHub Streak Monitor with Cloudflare Workers



Don't let your GitHub streak disappear! I built Streaky - a web app that automatically tracks your daily contributions and sends notifications to Discord & Telegram before your streak breaks.



Live Demo: streakyy.vercel.app


Source Code: github.com/0xReLogic/Streaky





The Problem



GitHub streaks represent consistency in coding habits. But life gets busy, and it's easy to forget to commit on a hectic day. I wanted an automated solution that:




  • Respects user privacy (zero-knowledge architecture)

  • Scales efficiently (distributed processing)

  • Supports multiple notification platforms

  • Requires zero maintenance





Tech Stack





  • Frontend: Next.js 15 (App Router), React 19, TypeScript, Tailwind CSS


  • Backend: Cloudflare Workers + D1 (SQLite)


  • Infrastructure: Rust notification proxy on Koyeb


  • Auth: GitHub OAuth via NextAuth.js v5


  • Security: AES-256-GCM encryption





Architecture Overview





┌─────────────────┐
│ Next.js 15 │ ← User Interface (Vercel)
│ + NextAuth │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Cloudflare │ ← API & Distributed Cron
│ Workers + D1 │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Rust Proxy │ ← Notification Delivery
│ (Koyeb) │
└─────────────────┘







Challenge #1: Distributed Cron Processing





The Problem



Cloudflare Workers have a 30-second CPU time limit per request. Processing thousands of users sequentially would exceed this limit.





The Solution: Service Bindings



I used Cloudflare's Service Bindings to spawn separate Worker instances for each user, enabling true parallel processing:




// Main cron handler
export default {
async scheduled(event: ScheduledEvent, env: Env, ctx: ExecutionContext) {
// Get all pending users
const users = await getPendingUsers(env);

// Process each user in parallel
for (const user of users) {
ctx.waitUntil(
// Each user gets their own Worker instance
env.SELF.fetch('http://internal/api/cron/process-user', {
method: 'POST',
headers: { 'X-Cron-Secret': env.SERVER_SECRET },
body: JSON.stringify({ userId: user.id })
})
);
}
}
}






Configuration in wrangler.toml:




[[services]]
binding = "SELF"
service = "streaky-backend"
environment = "production"






Key Benefits:




  • Isolated execution per user


  • Service Bindings bypass CPU time limits - each spawned Worker gets its own 30s budget

  • No timeout errors (TLE) even with thousands of users

  • Clean Analytics Engine metrics

  • Scalable to thousands of users






Challenge #2: Idempotent Queue System






The Problem



Cron jobs can overlap or retry, potentially processing the same user multiple times and sending duplicate notifications.






The Solution: D1-Based Queue with Atomic Operations






-- Queue table with status tracking
CREATE TABLE cron_queue (
user_id TEXT PRIMARY KEY,
status TEXT NOT NULL DEFAULT 'pending',
batch_id TEXT,
claimed_at TIMESTAMP,
processed_at TIMESTAMP
);

-- Atomic claim operation
UPDATE cron_queue
SET status = 'processing',
claimed_at = CURRENT_TIMESTAMP,
batch_id = ?
WHERE user_id = ?
AND status = 'pending';






TypeScript implementation:




async function claimNextPendingUser(env: Env): Promise<QueueItem | null> {
const batchId = crypto.randomUUID();

// Atomic claim with status check
const result = await env.DB.prepare(`
UPDATE cron_queue
SET status = 'processing',
claimed_at = CURRENT_TIMESTAMP,
batch_id = ?
WHERE user_id IN (
SELECT user_id FROM cron_queue
WHERE status = 'pending'
LIMIT 1
)
RETURNING *
`
).bind(batchId).first();

return result;
}






Key Benefits:




  • No duplicate processing

  • Safe concurrent execution

  • Automatic retry for failed jobs

  • Clean state management






Challenge #3: Zero-Knowledge Security






The Problem



Storing Discord/Telegram webhooks and GitHub tokens securely without exposing them.






The Solution: Multi-Layer Security



1. GitHub Tokens: Never Stored




// OAuth refresh flow - tokens never hit database
const session = await getServerSession(authOptions);
const accessToken = session?.accessToken; // From NextAuth session only






2. Webhooks: AES-256-GCM Encryption




async function encryptWebhook(webhook: string, key: string): Promise<string> {
const iv = crypto.getRandomValues(new Uint8Array(12));
const encodedKey = await crypto.subtle.importKey(
'raw',
Buffer.from(key, 'hex'),
{ name: 'AES-GCM' },
false,
['encrypt']
);

const encrypted = await crypto.subtle.encrypt(
{ name: 'AES-GCM', iv },
encodedKey,
new TextEncoder().encode(webhook)
);

return Buffer.concat([iv, new Uint8Array(encrypted)]).toString('base64');
}






3. Isolated Notification Proxy (Rust on Koyeb)



Why not send notifications directly from Cloudflare Workers?



The Problem: Cloudflare Workers use shared IP pools. Multiple Workers from different users share the same IP addresses, which triggers rate limits from Discord/Telegram APIs.



The Solution: Dedicated Rust proxy with its own IP address.




// Rust proxy receives encrypted webhooks
// Decrypts and sends notifications
// No logging of sensitive data
async fn send_notification(
encrypted_webhook: String,
message: String,
) -> Result<(), Error> {
let webhook = decrypt(&encrypted_webhook)?;
send_to_platform(&webhook, &message).await?;
Ok(())
}






Key Benefits:




  • Zero-knowledge architecture

  • Credentials encrypted at rest

  • Isolated notification delivery

  • Dedicated IP (no rate limit issues)

  • No sensitive data in logs






Challenge #4: Database Optimization






The Problem



Frequent cron queries could slow down the database and increase costs.






The Solution: Strategic Indexes






-- Index for queue processing
CREATE INDEX idx_cron_queue_status
ON cron_queue(status)
WHERE status = 'pending';

-- Index for user lookups
CREATE INDEX idx_users_github_id
ON users(github_id);

-- Index for notification history
CREATE INDEX idx_notifications_user_created
ON notifications(user_id, created_at DESC);






Performance Results:




  • Queue queries: <5ms

  • User lookups: <3ms

  • Notification history: <10ms






Lessons Learned






1. Service Bindings Are Powerful



Cloudflare's Service Bindings enable true distributed processing without the complexity of message queues or external orchestration.






2. Idempotency Is Critical



In distributed systems, operations must be idempotent. The atomic queue system prevents duplicate processing elegantly.






3. Security By Design



Building zero-knowledge architecture from the start is easier than retrofitting it later. Encrypt early, encrypt often.






4. D1 Is Production-Ready



Cloudflare D1 (SQLite) handles the workload beautifully with proper indexes. No need for complex database setups.






5. Monorepo Management



Managing TypeScript, Rust, and Python in one repo requires discipline but enables code sharing and consistent tooling.






Future Improvements




  • Custom reminder times (not fixed at 12:00 UTC)

  • Multiple reminder times per day

  • Timezone-aware notifications

  • Contribution goal tracking

  • Weekly/monthly statistics






Try It Out!



Live Demo: streakyy.vercel.app


Source Code: github.com/0xReLogic/Streaky



The project is fully open-source under MIT license. Contributions, feedback, and stars are welcome!

🔍 CTI & Forensik

Cyber Threat Intelligence & Forensik

ATT&CK-Navigator · IoC-Radar · Exploit-Belege
CTI Threat Relationship Graph
Akteure · Techniken · Beziehungen
3 Knoten · 2 Relationen
CVE / Incident Threat Actor Software MITRE ATT&CK CWE Weakness IoC
Ähnliche Beiträge
🔍 Verwandte News

Auch interessante Nachrichten Building a Distributed GitHub Streak Monitor with Cloudflare Workers

Thematisch verwandte Begriffe: Building, Distributed, GitHub, Streak · 6 Treffer

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

Laden...

Beiträge werden geladen ...

Laden...

Videos werden geladen ...

💬 Kommentare werden geladen…
Zum Aktualisieren ziehen
tsecurity.de Icon
Offline-Lesen, Eilmeldungen & 0ms Ladezeit

Installiere tsecurity.de direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.

Nächster Beitrag