security-best-practices.md

dstack Production Security Best Practices

This document describes security considerations for deploying dstack apps in production.

Security Audit

dstack has been audited by zkSecurity. The audit covered the KMS, guest agent, and attestation verification components. See the full audit report for findings and remediation status.

Always pin image hash in your docker-compose.yaml

When deploying applications in a TEE environment, it's critical to ensure the integrity and immutability of your container images. Using image digests (SHA256 hashes) instead of tags cryptographically ensures that the exact same image is always pulled, preventing supply chain attacks. This proves to users that your App is anchored to a specific code version.

❌ Bad example:

services:
  nginx:
    image: nginx:latest

services:
  nginx:
    image: nginx:1.27.5

✅ Good example:

services:
  nginx:
    image: nginx@sha256:eee5eae48e79b2e75178328c7c585b89d676eaae616f03f9a1813aaed820745a

Reproducibility

If your App is intended for end users who need to verify what code your App is running, then the verifiability of Docker images is crucial. dstack anchors the code running inside the CVM through the hash of app-compose.json. However, at the same time, the App needs to provide users with a reproducible build method. There are multiple ways to achieve reproducible image builds, and dstack provides a reference example: dstack-ingress

Authenticated envs and user_config

dstack provides encrypted environment variable functionality. Although the CVM physical machine controller cannot view encrypted environment variables, they may forge encrypted environment variables because the CVM encryption public key is known to everyone. Therefore, Apps need to perform auth checks on encrypted environment variables at the application layer. LAUNCH_TOKEN pattern is one method to prevent unauthorized envs replacement. For details, refer to the deployment script of dstack-gateway.

Newer dstack OS images support the LAUNCH_TOKEN pattern natively via requirements.launch_token_hash in app-compose.json. When this field is set, the guest reads the launch token from user_config at JSON path dstack.launch_token and refuses to boot — before any keys are provisioned — unless its digest matches the hash pinned in the (compose-hash-measured) app-compose.json. When the field is absent, user_config is not parsed and stays fully application-defined. Set manifest_version to "3" (string) when using requirements so older guests fail closed instead of silently ignoring it.

The digest is domain-separated so it stays distinct from the legacy plain-sha256(token) convention and from generic precomputed tables:

LAUNCH_TOKEN_HASH=$(printf 'dstack-launch-token/v1:%s' "$TOKEN" | sha256sum | cut -d' ' -f1)

Because launch_token_hash is public, a guessable token can be recovered offline by brute force. Guests reject tokens shorter than 32 bytes, but length alone does not guarantee entropy — always generate the token randomly, e.g. tr -dc 'a-zA-Z0-9' < /dev/urandom | head -c 32.

Also understand the protection boundary of this mechanism: the guest verifies the token before any keys are provisioned, which means the token must reach the guest through user_config — a channel the host can read. The requirement therefore stops parties who only know the public app-compose.json from launching the app, but once a host has hosted a deployment it learns the token and can later relaunch instances of that compose with substituted encrypted envs. Mitigations: generate a fresh token per deployment and remove stale compose hashes from the on-chain whitelist; if the token must stay secret from the host, use the app-layer APP_LAUNCH_TOKEN encrypted-env pattern above instead (its check necessarily runs after key provisioning).

If you use dstack-vmm's built-in UI, the prelaunch script has already been automatically filled in for you:

You only need to add the APP_LAUNCH_TOKEN environment variable to enable LAUNCH_TOKEN checking.

user_config is not encrypted, and similarly requires integrity checks at the application layer. For example, you can store a USER_CONFIG_HASH in encrypted environment variables and verify it in the prelaunch script.

Don't put secrets in docker-compose.yaml

CVM needs to ensure verifiability, so app-compose.json is public by default, containing the prelaunch script and docker-compose.yaml. You should not put secrets in docker-compose.yaml for best security practice. Use encrypted environment variables instead.

In case by any chance you really do not want to expose your compose file, you can disable exposing app-compose.json by setting public_tcbinfo=false in app-compose.json. Example app-compose.json:

{
    ...
    "public_tcbinfo": false
    ...
}

But keep in mind, even if you disable exposing app-compose.json, it is just hidden from the public API, the physical machine controller can still access it on the file system.

Do not use development trust settings in production

Development settings are intentionally easy to audit, but they are not production-safe. A production deployment should satisfy all of the following:

• KMS quote verification remains enabled. Do not deploy production KMS with quote_enabled = false.
• KMS authorization uses webhook/on-chain policy. Do not use auth_api.type = "dev" with real key material.
• The KMS contract pins a concrete gateway app id. Do not use gateway_app_id = "any" for production traffic.
• TEE quotes are evaluated by deployment policy, including TCB status and expected OS/application measurements.

The KMS TLS listener may keep rpc.tls.mutual.mandatory = false because bootstrap, temp-CA bootstrap, and public metadata endpoints need to be reachable before a client has an RA-TLS certificate. GetTempCaCert returns temp CA private material for the bootstrap flow; treat it as bootstrap-sensitive.

App key release and KMS key handover still require verified caller attestation from the RA-TLS client certificate. Certificate signing verifies the CSR signature and embedded attestation before signing.

Keep private material owner-only

Secret-bearing files should be owner-only (0600) wherever possible, including app keys, decrypted env files, KMS root keys, gateway WireGuard/TLS keys, and ACME credentials. Preserve restrictive permissions when copying volumes, backing up /etc/kms/certs, or moving gateway and certbot state between hosts. Public issue #606 tracks the remaining low-cost hardening work in dstack-managed file writes.

docker logs is public available by default

Similarly, to facilitate App observability, docker logs are public by default. You can disable exposing docker logs by setting public_logs=false. Example app-compose.json:

{
    ...
    "public_logs": false
    ...
}

Don't expose unexpected ports

In dstack CVM, dstack-guest-agent listens on port 8090, allowing public access to basic CVM information.

In docker-compose.yaml, all declared ports will be exposed to the public internet. Do not expose unnecessary ports.

For example:

# This will expose port 80 to the public
services:
  nginx:
    image: nginx@sha256:eee5eae48e79b2e75178328c7c585b89d676eaae616f03f9a1813aaed820745a
    ports:
      - "80:80"

# This will not expose port 80 to the public
services:
  nginx:
    image: nginx@sha256:eee5eae48e79b2e75178328c7c585b89d676eaae616f03f9a1813aaed820745a

# This will not expose port 80 to the public
services:
  nginx:
    image: nginx@sha256:eee5eae48e79b2e75178328c7c585b89d676eaae616f03f9a1813aaed820745a
    ports:
      - "127.0.0.1:80:80"

Note that when setting network_mode: host, all ports listened to within the container will be exposed to the public internet.

# This will expose port 80 to the public
services:
  nginx:
    image: nginx@sha256:eee5eae48e79b2e75178328c7c585b89d676eaae616f03f9a1813aaed820745a
    network_mode: host