// 01 · Headless service
Single binary, no embedded UI. Your app never imports GFire as a library — only HTTP and curl.
Background jobs
without a library.
GFire is a headless Go service: apps enqueue work over HTTP, workers spawn external handler binaries, state lives in PostgreSQL, Redis, or ValKey. v1.0.0 — production-ready: bulk enqueue, recurring cron, Prometheus metrics, CLI, and E2E tests included.
curl -fsSL https://get.gfire.hermesrodriguez.com/install.sh | sh
// 02 · Multi-backend storage
PostgreSQL with SKIP LOCKED, Redis/ValKey with BRPOP and Lua scripts. Pick the backend that fits your stack.
// 03 · Horizontal scale
Peer nodes, shared storage, no Raft. Add pods; they coordinate through the storage layer.
// Recent releases
- Bulk enqueue with partial acceptance (POST /v1/jobs/enqueue/batch)
- Recurring cron jobs with distributed lock (robfig/cron)
- CLI: gfire migrate, queue list, server status
- Prometheus GET /metrics endpoint
- Idempotency-Key for client retry deduplication
- OpenAPI spec at GET /openapi.json
- E2E test: docker-compose → curl → CLI verification
- Band 7 audit hardening — request IDs, logging, metrics polish
- Server shutdown grace period and drain
- Band 3 engine — workers, retry, cancel, DLQ, result capture
- REST API and CLI available for early testing
// How it works
Technical notes · v1.0.0
// Production-ready job orchestration
GFire is a headless job orchestration service: any client with HTTP can enqueue work without importing a runtime-specific library. v1.0.0 ships the full stack — PostgreSQL (SKIP LOCKED), Redis, and ValKey storage backends, a REST API with bulk enqueue and idempotency keys, recurring cron jobs with distributed locks, a CLI, Prometheus metrics, and an E2E test suite. Language-agnostic by design; handlers are external binaries.
// Redis / ValKey architecture
go-redis/v9
One driver for Redis and ValKey — swap the address, keep the implementation. Open-source forks and managed Redis stay compatible.
BRPOP (push via blocking)
Workers block on the queue instead of polling. Less CPU and network churn; jobs wake a worker as soon as they arrive.
Lua scripts
N peer GFire nodes share storage — no Raft between pods. Lua atomically moves jobs (e.g. enqueued → processing) so race-free claims live in the data layer.
Sorted sets (ZSET)
Delayed and scheduled jobs use scores = Unix timestamps. The scheduler pulls only work that is due — scales with large backlogs.
// How GFire compares
| Tool | Interaction model | Storage | Language coupling |
|---|---|---|---|
| GFire | HTTP / headless service | PostgreSQL, Redis, ValKey | High — language agnostic |
| Faktory | Protocol / headless | Redis | High — custom TCP protocol |
| Asynq | Go library (embedded) | Redis | Low — Go only |
| River | Go library (embedded) | PostgreSQL | Low — Go only |
| Sidekiq | Ruby gem (embedded) | Redis | Low — Ruby only |
| Celery | Python library (embedded) | Redis, RabbitMQ, SQL | Medium — Python-first |
GFire is sidecar-ready and standalone: enqueue with curl or any HTTP/JSON client — no vendor SDK inside your app. Unlike Faktory’s TCP protocol or in-process libraries (Sidekiq, Asynq, River), handlers stay external binaries.
Full compare matrix in the repo
// External handlers
GFire does not run your business logic in-process. Workers dequeue from storage, then spawn a subprocess per job (`cmd` in YAML). Handlers can be Go, Python, Node, shell — GFire tracks exit status, retries, and continuations. Handlers never talk to Redis/PostgreSQL directly; the service owns the storage contract.
Instruction cards (~1 KB)
Jobs are small triggers, not fat payloads. Heavy data lives in S3 or your database; GFire carries the instruction. Keeps Redis memory predictable and avoids large-message instability.
// Quick start
Install the CLI and verify the binary (today: gfire version).
$ curl -fsSL https://get.gfire.hermesrodriguez.com/install.sh | sh $ gfire version
v1.0.0 — stable release. Full REST API, worker engine, and CLI included.
Planned milestones:ROADMAP.md
// Repositories
| Repo | Role |
|---|---|
| hrodrig/gfire | Service, storage backends, releases |
| SPECIFICATIONS.md | Behavior contract and design |