01 / Python · FastAPI
Application programming interface
Accepts a request and returns HTTP 202 after saving durable work. Readiness checks database access.
Build in public / DevOps
Platform Engineering Lab
A small application with the engineering around it made visible: delivery, security, observability and recovery.
Interactive walkthrough
Follow one request
This demonstration runs in your browser. It does not connect to Kafka, a database or a Kubernetes cluster. Timing and stages are illustrative.
The request moves from acceptance through storage, messaging and completion.
Reduced motion: advance each stage when ready.
- 01AcceptedWaiting
- 02Stored + outboxWaiting
- 03Published to KafkaWaiting
- 04Worker processedWaiting
- 05CompletedWaiting
Ready. Submit a request to begin.
The real application
Durability before convenience
02 / PostgreSQL · outbox
Durable storage
Stores the request and pending event in one transaction. A broker outage keeps the event available for retry.
03 / Apache Kafka
Event delivery
The relay waits for a broker acknowledgement. Events carry OpenTelemetry trace context across the queue.
04 / Python · worker
Background processing
Processes the event, completes the request and then commits the consumer offset. Replays have idempotent database effects.
Delivery is at least once. Duplicate events are expected; the database completion update is idempotent. This lab uses a single Kafka broker and database instance.
Code to verified artifact
- 01 Review. Review the change, then manually start Continuous Integration after confirming the included runner allowance.
- 02 Validate. Unit tests and Ruff check application behavior and code.
- 03 Scan. Semgrep checks source code; Trivy scans dependencies and the container image.
- 04 Exercise. Docker Compose runs the complete request flow in CI.
- 05 Record. Export the Software Bill of Materials and scan reports as artifacts.
The workflow must run successfully before these checks can be reported as passed. Kubernetes reconciliation through Argo CD is a later phase.
Run the local stack
cd experiments/devops-platform-lab
python3 scripts/bootstrap.py
docker compose up --build -d
python3 scripts/smoke.pyOpen the API documentation at localhost:8000/docs and Grafana at localhost:3001. All exposed ports bind to your machine's loopback interface.
Setup and shutdown instructionsImplementation ledger
Source availability and runtime evidence are separate.
Python + PostgreSQL + Kafka
FastAPI, transactional outbox, manual offsets, idempotent completion.
Source availableContinuous Integration + security
Tests, Ruff, Semgrep, Trivy, Software Bill of Materials, Compose smoke test.
Workflow availableMetrics + distributed tracing
Prometheus, Grafana, OpenTelemetry Collector and Tempo configuration.
Runtime check pendingSite Reliability Engineering
Illustrative service objectives, alert rules, incident drill and runbook.
Source availableInfrastructure as Code + configuration
Local Terraform and Ansible; cloud references remain unapplied.
Next phaseService mesh + secrets + eBPF
Istio, Vault, Cilium and Hubble need a verified local cluster.
Next phaseFinOps + incident assistance + IDP
Financial Operations, grounded incident assistance and Internal Developer Platform golden path.
Next phaseDynamic security + infrastructure scans
Open Worldwide Application Security Project ZAP and Checkov.
Next phaseReliability you can inspect
Prometheus scrapes request counts, latency and worker health. Grafana displays metrics; Tempo stores distributed traces. Alerts and a worker/broker incident drill are configured for local validation.
Read the incident runbookCost-conscious by design
The platform runs locally and does not provision cloud resources or call paid model APIs. Local hardware, electricity and existing website hosting remain your responsibility. No measured cloud savings or service-level achievement is claimed.