Honker

honker adds Postgres-style NOTIFY/LISTEN semantics to SQLite, with a durable pub/sub, task queue, and event streams on the side, without client polling or a daemon/broker. Cross-process wake latency is ~0.7 ms p50 on an M-series laptop.

In its basic form it’s a plain SQLite loadable extension, so any language that can SELECT load_extension('/path/to/libhonker_ext') gets the same queue, streams, and notifications on the same file. Bindings for Python, Node, Rust, Go, Ruby, Bun, Elixir, C++, .NET / C#, Java, and Kotlin share one on-disk format.

SQLite is backing real work now — Bluesky’s PDS, Fly’s LiteFS, Turso, weekend projects that somehow ended up in production. Once real work flows through a SQLite-backed app, you need a queue. The usual answer is “add Redis + Celery.” That works, but introduces a second datastore with its own backup story, a dual-write problem between your business table and the queue, and the operational overhead of running a broker.

honker takes the approach that if SQLite is the primary datastore, the queue should live in the same file. That means INSERT INTO orders and queue.enqueue(...) commit in the same transaction. Rollback drops both. The queue is just rows in a table with a partial index.

What you can use it for

• Cross-process NOTIFY/LISTEN-style signaling on a single SQLite file, without Redis or a broker
• Durable work queues with retries, priority, delayed jobs, expiration, and dead letters
• Transactional outbox patterns where your business write and enqueue commit together or roll back together
• Durable streams with per-consumer offsets
• Time-trigger scheduling with a leader-elected scheduler: 5-field cron, 6-field cron, and @every <n><unit>
• Named locks, rate limiting, and opt-in task result storage
• Thin cross-language bindings over the same extension-owned SQLite layout

One example

Enqueue atomically with a business write, then consume. Same .db file, same on-disk format, now including Java and Kotlin.

Python

import honker

db = honker.open("app.db")
q = db.queue("emails")

# Enqueue in the same transaction as the business write.
with db.transaction() as tx:
    tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)", [42, 99])
    q.enqueue({"to": "alice@example.com", "order_id": 42}, tx=tx)

# Worker wakes on database updates or due deadlines.
async for job in q.claim("worker-1"):
    await send_email(job.payload)
    job.ack()

Or with Huey-style decorators:

@q.task(retries=3, timeout_s=30)
def send_email(to, subject):
    ...
    return {"sent_at": time.time()}

r = send_email("alice@example.com", "Hi")   # enqueues, returns TaskResult
print(r.get(timeout=10))                    # blocks until worker runs it

Node

const { open } = require('@russellthehippo/honker-node');
const db = open('app.db');
const q = db.queue('emails');

// Enqueue in the same transaction as the business write.
const tx = db.transaction();
tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)", [42, 99]);
q.enqueueTx(tx, { to: 'alice@example.com', order_id: 42 });
tx.commit();

// Worker wakes on database updates or due deadlines.
const waker = q.claimWaker();
while (true) {
  const job = await waker.next('worker-1');
  if (!job) break;
  await sendEmail(job.payload);
  job.ack();
}

Rust

use honker::{Database, QueueOpts, EnqueueOpts};
use serde_json::json;

let db = Database::open("app.db")?;
let q = db.queue("emails", QueueOpts::default());

let tx = db.transaction()?;
tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)",
           rusqlite::params![42, 99])?;
q.enqueue_tx(&tx,
    &json!({"to": "alice@example.com", "order_id": 42}),
    EnqueueOpts::default())?;
tx.commit()?;

if let Some(job) = q.claim_one("worker-1")? {
    send_email(&job.payload)?;
    job.ack()?;
}

Go

import honker "github.com/russellromney/honker-go"

db, _ := honker.Open("app.db", "./libhonker_ext.dylib")
defer db.Close()

q := db.Queue("emails", honker.QueueOptions{})

tx, _ := db.Begin()
tx.Exec("INSERT INTO orders (id, total) VALUES (?, ?)", 42, 99)
q.EnqueueTx(tx, map[string]any{
    "to": "alice@example.com", "order_id": 42,
}, honker.EnqueueOptions{})
tx.Commit()

if job, _ := q.ClaimOne("worker-1"); job != nil {
    var p map[string]any
    job.UnmarshalPayload(&p)
    sendEmail(p)
    job.Ack()
}

Ruby

require "honker"

db = Honker::Database.new("app.db", extension_path: "./libhonker_ext.dylib")
q = db.queue("emails")

db.transaction do |tx|
  tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)", [42, 99])
  q.enqueue({to: "alice@example.com", order_id: 42}, tx: tx)
end

if (job = q.claim_one("worker-1"))
  send_email(job.payload)
  job.ack
end

Bun

import { open } from "@russellthehippo/honker-bun";

const db = open("app.db", "./libhonker_ext.dylib");
const q = db.queue("emails");

const tx = db.transaction();
tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)", [42, 99]);
q.enqueue({ to: "alice@example.com", order_id: 42 }, { tx });
tx.commit();

const job = q.claimOne("worker-1");
if (job) {
  await sendEmail(job.payload as { to: string });
  job.ack();
}

Elixir

{:ok, db} = Honker.open("app.db", extension_path: "./libhonker_ext.dylib")

Honker.Transaction.transaction(db, fn tx ->
  Honker.Transaction.execute(tx, "INSERT INTO orders (id, total) VALUES (?, ?)", [42, 99])
  Honker.Queue.enqueue_tx(tx, "emails", %{to: "alice@example.com", order_id: 42}, [], [])
end)

case Honker.Queue.claim_one(db, "emails", "worker-1") do
  {:ok, nil} -> :ok
  {:ok, job} ->
    send_email(job.payload)
    Honker.Job.ack(db, job)
end

.NET

using Honker;

using var db = Database.Open("app.db");
var q = db.Queue("emails");

using (var tx = db.BeginTransaction())
{
    tx.Execute("INSERT INTO orders (id, total) VALUES (@p0, @p1)", 42, 99);
    q.Enqueue(new { to = "alice@example.com", order_id = 42 }, transaction: tx);
    tx.Commit();
}

var job = q.ClaimOne("worker-1");
if (job is not null)
{
    await SendEmail(job.Payload);
    job.Ack();
}

Java

import dev.honker.Honker;

try (var db = Honker.open("app.db")) {
    var q = db.queue("emails");

    db.transactionVoid(tx -> {
        tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)",
            java.util.List.of(42, 99));
        q.enqueue(tx, "{\"to\":\"alice@example.com\",\"order_id\":42}");
    });

    var job = q.claimOne("worker-1").orElseThrow();
    sendEmail(job.payloadJson());
    job.ack();
}

Kotlin

import dev.honker.kotlin.honker

honker("app.db").use { db ->
    val q = db.queue("emails")

    db.transactionVoid { tx ->
        tx.execute("INSERT INTO orders (id, total) VALUES (?, ?)", listOf(42, 99))
        q.enqueue(tx, """{"to":"alice@example.com","order_id":42}""")
    }

    val job = q.claimOne("worker-1").orElseThrow()
    sendEmail(job.payloadJson())
    job.ack()
}

C++

#include "honker.hpp"

int main() {
    honker::Database db{"app.db", "./libhonker_ext.dylib"};
    auto q = db.queue("emails");

    {
        honker::Transaction tx{db.raw()};
        sqlite3_exec(
            tx.raw(),
            "INSERT INTO orders (id, total) VALUES (42, 99)",
            nullptr, nullptr, nullptr
        );
        q.enqueue_tx(tx, R"({"to":"alice","order_id":42})");
        tx.commit();
    }

    auto job = q.claim_one("worker-1");
    if (job) {
        send_email(job->payload());
        job->ack();
    }
}

SQL

.load ./libhonker_ext
SELECT honker_bootstrap();

BEGIN;
INSERT INTO orders (id, total) VALUES (42, 99);
SELECT honker_enqueue('emails', '{"to":"alice","order_id":42}',
                      NULL, NULL, 0, 3, NULL);
COMMIT;

SELECT honker_claim_batch('emails', 'worker-1', 32, 300);
SELECT honker_ack_batch('[1,2,3]', 'worker-1');

How it works

honker polls SQLite’s PRAGMA data_version every millisecond. That’s a monotonic counter SQLite increments on every commit from any connection, journal mode, or process — a ~3 µs read for a precise wake signal. A background thread fans the tick out to every subscriber, which runs SELECT ... WHERE id > last_seen and yields new rows. One poller thread per database regardless of subscriber count.

Idle cost is that one lightweight SELECT per millisecond per database — no page-cache pressure, no writer-lock contention, no kernel file watcher in the mix. Listener count scales for free because the wake signal is one shared poll, not one query per listener.

Advanced users can opt into experimental kernel filesystem-event or mmap WAL-index backends; see Watcher backends.

The queue, stream, and pub/sub primitives are all INSERTs into tables managed by the extension. Calling queue.enqueue(payload, tx=tx) inside your business transaction means the job row is ACID with the INSERT INTO orders that preceded it. Rollback drops the job along with everything else.

Honker intentionally does not support SQLite in-memory database filenames (:memory:, file::memory:?cache=shared, or file:<name>?mode=memory&cache=shared). Bare :memory: is per connection, and SQLite’s shared-memory URI forms only share state inside one process. For production-faithful tests, use a temporary file-backed .db; it preserves the same locking, wake, crash/reopen, and multi-process semantics as production.

Prior art

pg_notify gives you fast cross-process triggers but no retry or visibility. Huey is the SQLite-backed Python task queue honker draws the most from. pg-boss and Oban are the Postgres-side gold standards. If you already run Postgres, use those.

Install

Python

pip install honker

Node

npm install @russellthehippo/honker-node

Rust

cargo add honker

Go

go get github.com/russellromney/honker-go

Ruby

gem install honker

Bun

bun add @russellthehippo/honker-bun

Elixir

mix.exs

{:honker, "~> 0.1"}

.NET

dotnet add package Honker

Java

<dependency>
  <groupId>dev.honker</groupId>
  <artifactId>honker</artifactId>
  <version>0.1.0</version>
</dependency>

Kotlin

<dependency>
  <groupId>dev.honker</groupId>
  <artifactId>honker-kotlin</artifactId>
  <version>0.1.0</version>
</dependency>

C++

git clone https://github.com/russellromney/honker-cpp.git
cd honker-cpp
zig build

SQLite extension

# Build from source — it's one crate
cargo build --release -p honker-extension
# → target/release/libhonker_ext.{dylib,so}