Local Deploy Standards
Local deploy exists to support integration tests and agent development. It must not assume one developer, one checkout, or one fixed port.
Rules
- No fixed host ports. Do not reserve
localhost:4566,8080,3000, or any other global port in checked-in manifests. - No fixed global names. Cluster names, namespaces, containers, and temp files must be derived from the checkout path or an explicit caller-provided name.
- Every
upcommand must emit a machine-readable endpoint file under.lake/. Humans can read stdout; agents should read the file. - Every
downcommand must tear down only the resources created by the same checkout. Never delete a shared cluster or a hard-coded namespace. - Bind to an ephemeral host port (Docker
-p 4566, no fixed left-hand side) and read the allocated port back, rather than reserving a global one. - Prefer the lightest thing that works: a single container needs Docker, not a Kubernetes cluster. Do not stand up kind/k8s for one emulator.
Mise Layering
mise installinstalls only the base development tools needed for normal Rust work. Docker is assumed present (not a mise tool).- Any local-deploy-only tools (cloud emulators, load-test tools) are
task-scoped
toolson the deploy tasks, never top-level[tools]. - CI should not run local deploy unless the job explicitly owns the environment.
Current Contract
mise run test-env-up runs a checkout-scoped LocalStack container (DynamoDB +
S3) directly in Docker — no kind/k8s. The container is named per checkout
(path hash) and bound to an ephemeral port, both discovered dynamically. It
writes .lake/test-env.env with:
LAKE_DYNAMODB_ENDPOINT=http://127.0.0.1:<dynamic-port>
(the same endpoint serves S3, since LocalStack multiplexes all services on one
port). mise run test-env-down removes that checkout’s container.
The ignored lake-meta LocalStack test creates both Dynamo layouts, exercises
dual CAS and lease-guarded transactions, backfills in bounded pages, finalizes,
then reads through the prefix-query path:
set -a; source .lake/test-env.env; set +a
cargo test -p lake-meta --test dynamo_localstack -- --ignored --nocapture
The community image localstack/localstack:3 is pinned deliberately —
:latest now requires a LOCALSTACK_AUTH_TOKEN and exits without one.
Query execution defaults to a 1 GiB process-wide DataFusion memory pool and
an 8 GiB aggregate spill budget. Production deployments should mount fast
ephemeral storage and set LAKE_QUERY_MEMORY_BYTES,
LAKE_QUERY_SPILL_BYTES, and LAKE_QUERY_SPILL_DIR for the node size. Both
byte budgets must be at least 16 MiB. The spill root is not durable state:
DataFusion creates a randomized child directory per Query runtime and removes
its files when operators and the runtime are dropped.