Skip to content

test(sandbox): exercise a sandbox session end to end - #403

Open
ItamarZand88 wants to merge 1 commit into
itamar/alien-75-sandbox-6-typescriptfrom
itamar/alien-75-sandbox-7-e2e
Open

test(sandbox): exercise a sandbox session end to end#403
ItamarZand88 wants to merge 1 commit into
itamar/alien-75-sandbox-6-typescriptfrom
itamar/alien-75-sandbox-7-e2e

Conversation

@ItamarZand88

@ItamarZand88 ItamarZand88 commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Exercises a sandbox end to end: the test app declares one, and the suite drives a real session through it rather than asserting on rendered templates.

What the test does when it runs:

  1. Deploys the comprehensive test app, which now declares a sandbox alongside its other resources.
  2. Creates a session through the binding and runs a command in it.
  3. Writes a file, reads it back, and checks the content — which is the only thing that proves the agent, the capability and the transport all line up.
  4. Terminates the session and confirms it is gone.

What I did

The assertions are on observed behaviour, not on artifacts. A test that checks a template contains a string would pass against a sandbox that never starts, so this one runs code inside the session and reads back what it wrote.

Files touched

  • crates/alien-test/ — the sandbox case in the deployment suite.
  • tests/e2e/test-apps/comprehensive-typescript/ — the declaration and the handler the test drives.

How I tested

The suite itself is the test: it creates a session, runs a command, round-trips a file and
terminates, asserting on what came back rather than on rendered artifacts.

I have not run it against a live deployment in this change — it needs cloud credentials and a
free slot. It should be run before merge, and the thing to watch is teardown: an accepted delete
is not a completed one, so confirm the session is actually gone rather than trusting the call.

@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds a Local-only end-to-end sandbox binding check and revises teardown to confirm deletion before reporting success.

  • Declares and links sandbox resources in both comprehensive test applications.
  • Exercises command execution and file round-tripping through Rust and TypeScript handlers.
  • Polls session state after termination, retries transient lookup failures, and prioritizes teardown failures.
  • Integrates the sandbox check into the Local binding support matrix and test runner.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
crates/alien-test/src/e2e.rs Adds Sandbox to the binding model and enables it only for the Local platform.
crates/alien-test/tests/common/bindings.rs Adds the HTTP-level sandbox check and validates its standard success response.
crates/alien-test/tests/common/runner.rs Dispatches supported Sandbox checks through the existing binding runner.
tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Exercises a Rust sandbox session and reliably propagates or confirms teardown outcomes.
tests/e2e/test-apps/comprehensive-typescript/src/handlers/sandbox.ts Implements the equivalent TypeScript sandbox lifecycle with cleanup-first failure reporting.
tests/e2e/test-apps/comprehensive-rust/alien.ts Declares, links, authorizes, and provisions the Local-only Rust sandbox fixture.
tests/e2e/test-apps/comprehensive-typescript/alien.ts Declares, links, authorizes, and provisions the Local-only TypeScript sandbox fixture.

Sequence Diagram

sequenceDiagram
    participant Runner as E2E runner
    participant App as Comprehensive app
    participant Sandbox as Sandbox binding
    Runner->>App: POST /sandbox-test/alien-sandbox
    App->>Sandbox: create session
    App->>Sandbox: run command
    App->>Sandbox: write file
    App->>Sandbox: read file
    App->>Sandbox: terminate session
    loop Until absent/terminated or timeout
        App->>Sandbox: get session
    end
    App-->>Runner: Success or teardown failure
Loading

Reviews (18): Last reviewed commit: "test(sandbox): exercise a sandbox sessio..." | Re-trigger Greptile

Comment thread tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Outdated
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch 2 times, most recently from 2cbe0ec to 4a813ec Compare August 11, 2026 21:26
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 4a813ec to 9e50cc3 Compare August 11, 2026 22:16
greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 11, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 9e50cc3 to d835538 Compare August 11, 2026 22:24
@greptile-apps
greptile-apps Bot dismissed their stale review August 11, 2026 22:24

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 11, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from d835538 to 4948041 Compare August 12, 2026 06:32
@greptile-apps
greptile-apps Bot dismissed their stale review August 12, 2026 06:32

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 12, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 4948041 to 1b3c226 Compare August 12, 2026 06:55
@greptile-apps
greptile-apps Bot dismissed their stale review August 12, 2026 06:55

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 12, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 1b3c226 to 40e88e5 Compare August 12, 2026 07:05
@greptile-apps
greptile-apps Bot dismissed their stale review August 12, 2026 07:05

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 12, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 40e88e5 to e80dd48 Compare August 12, 2026 07:22
@greptile-apps
greptile-apps Bot dismissed their stale review August 12, 2026 07:22

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

Comment thread tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Outdated
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch 2 times, most recently from 42c6245 to c4dcd66 Compare August 12, 2026 08:11
Comment thread tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Outdated
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from c4dcd66 to 85eb01b Compare August 12, 2026 08:15
Comment thread tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Outdated
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch 2 times, most recently from d7befbd to 6d38ac5 Compare August 12, 2026 11:01
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 6d38ac5 to 88c7ec1 Compare August 12, 2026 11:18
greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 12, 2026
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 88c7ec1 to 9f347e7 Compare August 12, 2026 12:14
@greptile-apps
greptile-apps Bot dismissed their stale review August 12, 2026 12:14

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

Comment thread tests/e2e/test-apps/comprehensive-rust/src/handlers/sandbox.rs Outdated
@ItamarZand88
ItamarZand88 force-pushed the itamar/alien-75-sandbox-7-e2e branch from 9f347e7 to 0feee35 Compare August 12, 2026 12:39
@ItamarZand88

Copy link
Copy Markdown
Contributor Author

@greptile-apps review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant