merkql

Tamper-evident audit logs

Regulatory environments (SOX, HIPAA, PCI-DSS, GDPR) require demonstrating that audit records haven't been modified after the fact. merkql provides a cryptographic inclusion proof for every record—hand an auditor a proof and a root hash and they can independently verify authenticity without access to your systems.

// Write an audit event
let producer = Broker::producer(&broker);
producer.send(&ProducerRecord::new("audit",
    Some("user-1001".into()),
    r#"{"action":"access_patient_record","id":"R-2847"}"#
)).unwrap();

// Later: generate a proof for any record
let topic = broker.topic("audit").unwrap();
let partition = topic.partition(0).unwrap().read().unwrap();
let proof = partition.proof(0).unwrap().unwrap();

// Anyone can verify independently
assert!(MerkleTree::verify_proof(&proof, partition.store()).unwrap());

Event sourcing without infrastructure

Event-sourced applications need an append-only log with multiple consumer groups, offset tracking, retention, and durability. merkql provides all of these as a library—no Docker, no cluster management, no network configuration.

let config = BrokerConfig {
    compression: Compression::Lz4,
    default_retention: RetentionConfig { max_records: Some(100_000) },
    ..BrokerConfig::new("./data/events")
};
let broker = Broker::open(config).unwrap();

// Each projection gets its own consumer group
let mut projections = Broker::consumer(&broker, ConsumerConfig {
    group_id: "order-summary-projection".into(),
    auto_commit: false,
    offset_reset: OffsetReset::Earliest,
});

Integration testing

Replace a real Kafka cluster in your test suite. Same produce/subscribe/poll/commit lifecycle, starts in microseconds from a temp directory, provides the same consumer group semantics—without Docker, port conflicts, or test isolation problems.

#[test]
fn order_processing_pipeline() {
    let dir = tempfile::tempdir().unwrap();
    let broker = Broker::open(BrokerConfig::new(dir.path())).unwrap();

    // Simulate upstream events
    let producer = Broker::producer(&broker);
    for order in test_orders() {
        producer.send(&ProducerRecord::new("orders", None, order)).unwrap();
    }

    // Run your real consumer logic
    let mut consumer = Broker::consumer(&broker, config);
    consumer.subscribe(&["orders"]).unwrap();
    let records = consumer.poll(Duration::from_millis(100)).unwrap();
    assert_eq!(records.len(), test_orders().len());
}

Edge and embedded systems

IoT gateways, POS terminals, medical devices, vehicle telemetry—anywhere you need local event buffering with integrity guarantees and no network dependencies. LZ4 compression keeps storage manageable. Retention policies prevent unbounded growth.

Local development

Run your Kafka-based microservices locally without Docker Compose. Same topic/partition/consumer-group semantics, zero startup time.