Skip to content
lake
Browse this documentation section

Local Deploy Standards

Local deploy exists to support integration tests and agent development. It must not assume one developer, one checkout, or one fixed port.

Rules

Mise Layering

Current Contract

mise run test-env-up runs a checkout-scoped LocalStack container (DynamoDB + S3) directly in Docker — no kind/k8s. The container is named per checkout (path hash) and bound to an ephemeral port, both discovered dynamically. It writes .lake/test-env.env with:

LAKE_DYNAMODB_ENDPOINT=http://127.0.0.1:<dynamic-port>

(the same endpoint serves S3, since LocalStack multiplexes all services on one port). mise run test-env-down removes that checkout’s container.

The ignored lake-meta LocalStack test creates both Dynamo layouts, exercises dual CAS and lease-guarded transactions, backfills in bounded pages, finalizes, then reads through the prefix-query path:

set -a; source .lake/test-env.env; set +a
cargo test -p lake-meta --test dynamo_localstack -- --ignored --nocapture

The community image localstack/localstack:3 is pinned deliberately — :latest now requires a LOCALSTACK_AUTH_TOKEN and exits without one.

Query execution defaults to a 1 GiB process-wide DataFusion memory pool and an 8 GiB aggregate spill budget. Production deployments should mount fast ephemeral storage and set LAKE_QUERY_MEMORY_BYTES, LAKE_QUERY_SPILL_BYTES, and LAKE_QUERY_SPILL_DIR for the node size. Both byte budgets must be at least 16 MiB. The spill root is not durable state: DataFusion creates a randomized child directory per Query runtime and removes its files when operators and the runtime are dropped.