Scaling to 6,000
PDFs Per Hour
How I optimized a document generation system to process 6,000 PDFs per hour on a single Railway worker, a throughput number that typically requires 5-10 workers.
The Challenge
KamkmPDF started as a simple Next.js app with a cron job. It worked for 50 users, but when hundreds started uploading simultaneously, the queue backed up. Most teams would deploy 5-10 workers. I wanted to see how far a single optimized worker could go.
Starting Point
- Throughput200-300/hr
- Generation time5-10s
- Concurrent jobs3 before crash
- AI keys1 (rate limits)
Final Results
- Throughput6,000/hr
- Generation time<2s
- Concurrent jobs10 stable
- AI keys11 with failover
Key Optimizations
Four major changes that transformed performance
Go Worker Rewrite
Migrated from TypeScript to Go for better concurrency and memory efficiency using goroutines.
2x throughput increaseCerebras AI Integration
Implemented priority-based AI model selection with Cerebras GPT-oss-120b as primary for high-speed inference.
5-10x faster than GPT-4Database Optimization
Atomic job claiming with FOR UPDATE SKIP LOCKED, connection pooling, and strategic indexing.
Query time: 100ms → 5msBrowser Pooling
Reusable Puppeteer instances instead of spawning new browsers per job.
10 concurrent PDFs, same memoryArchitecture Highlights
Built for horizontal scaling from day one
Queue System
- PostgreSQL-backed queue with atomic job claiming
- Round-robin API key distribution for AI redundancy
- Multi-tier PDF fallback (Puppeteer → PDFKit → Static)
Worker Design
- Browser pool with automatic cleanup and monitoring
- Horizontal scaling ready - just add more workers
Technology Stack
Carefully chosen for maximum throughput
Frontend
Worker
Database
AI
PDF Generation
Storage
Cost Efficiency
Enterprise throughput, startup budget
Infrastructure
- Single Railway worker~$20/mo
- PostgreSQL database~$15/mo
- No Redis or queues$0
AI Costs
- Cerebras GPT-oss-120b~$0.002/PDF
- At 6,000 PDFs/hour~$12/hr
- vs GPT-4 equivalent5x savings
Total: ~$35/month infrastructure + usage-based AI costs. Competitors typically spend $200-500/month for similar throughput.
Ready to Build Something Amazing?
This case study demonstrates my ability to optimize systems for maximum performance without unnecessary infrastructure complexity. Let's discuss your scaling challenges.