STRMCourse

Real-time data streaming

Lessons9modules

Total90mfull study

Quick7mtrailer

Projects9docker labs

hello-stream · single-node Redpanda + Python four-line discipline

Greenfield producer/consumer that demonstrates idempotence + acks=all + manual commit + read_committed.

snap/streaming:helloRepo · streaming-hello

$git clonehttps://github.com/snap-dev/streaming-hello.git

docker-compose.yml

services:
  redpanda:
    image: redpandadata/redpanda:v24.3.1
    command: [redpanda, start, --smp=1, --memory=1G, --overprovisioned, --node-id=0,
              --kafka-addr=PLAINTEXT://0.0.0.0:9092,
              --advertise-kafka-addr=PLAINTEXT://redpanda:9092]
    healthcheck:
      test: ["CMD-SHELL", "rpk cluster info -X brokers=redpanda:9092 >/dev/null"]
      interval: 5s
      retries: 12
    ports: ["9092:9092", "9644:9644"]

  app:
    image: python:3.11-slim
    working_dir: /app
    depends_on:
      redpanda: { condition: service_healthy }
    volumes: ["./src:/app:ro"]
    environment:
      BOOTSTRAP: redpanda:9092
    command: >-
      bash -c "pip install -q confluent-kafka==2.6.0 &&
               python produce_consume.py"

Run

~/streaming-hello · zsh

$ docker compose up --abort-on-container-exit

5 events produced, 5 consumed; same key consistently lands on same partition; offsets monotonic per partition.

What you'll observe

Redpanda passes the rpk cluster info healthcheck before app starts

Producer config has enable.idempotence=true and acks=all

Consumer config has enable.auto.commit=false and isolation.level=read_committed

Same key (order-0) consistently lands on the same partition

Container exits with code 0 within 30 seconds

Lift this to your work

Drop into any new microservice that needs to publish or consume events. Replace localhost:9092 with your bootstrap.servers, add SASL/SSL config from your secrets manager, and you have the production shape on day one. The 'four-line discipline' (idempotent + acks=all + manual commit + read_committed) is the producer/consumer review checklist Snap engineers paste into PRs.