Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ All notable changes to SwiftAgentKit will be documented in this file.
- **ContextSift: stop the `artifact_read` retrieval loop.** A large *active* tool result was bounded to `maxActiveResultChars` and spilled to an artifact with a "use artifact_read" hint; when the model then called `artifact_read`, that retrieval result was *itself* re-truncated and re-spilled — so the model kept calling `artifact_read` to "get the full output" that never fully surfaced (observed as ~6 repeated `artifact_read` calls until `maxTurns` ran out). Fix: retrieval tools (`artifact_read`, `artifact_search`) are now exempt from active-display truncation and receipt spilling — their output is shown in full (the tools already page via `offset`/`limit`). Also raised the default `maxActiveResultChars` 2000 → 8000 so ordinary tool outputs fit inline without any artifact round-trip. Regression coverage added (an active `artifact_read` result larger than the bound is shown in full, never re-truncated).

### Added
- **`PythonTool` (`run_python`) in `SwiftAgentKitTools`.** A first-class tool that runs Python 3 in a dedicated virtual environment: it ensures the venv exists, pip-installs any requested `packages` into it (isolated from system Python), stages the `code` to a temp file, runs it, and returns combined stdout/stderr + exit code. `requiresConfirmation`, configurable `venvPath`/`timeoutSeconds`/`maxOutputChars`, wall-clock timeout, macOS-only. Turns "make the model orchestrate venv + pip via raw shell" into one reliable capability. New tests (confirmation flag; a gated live test creating a venv, installing a package into it, and running code — proving isolation).
- **Goal-completion verifier loop — don't trust the model's "done" signal.** New `AgentCallbacks.verifyCompletion(query, proposedAnswer, state) -> GoalVerdict`: when the model finishes a turn with no tool calls, the agent asks the verifier whether the goal is actually met before stopping. `.satisfied` stops; `.unsatisfied(reason:)` feeds the reason back as a nudge and **keeps the loop going**; `.blocked(reason:)` stops early and surfaces the blocker (a "big failure to mention") instead of burning turns. Bounded by `AgentConfig.maxVerificationRetries` (default 3) and `maxTurns`. New events `completionVerificationFailed` / `completionBlocked`. This turns "loop until the model says done" into "loop until the goal is verifiably done, or a real blocker" — the verifier can be a deterministic check (does the output file exist? do tests pass?) or a judge-model call. Regression coverage: retries until satisfied; stops immediately when satisfied; stops early on blocked; bounded so an always-unsatisfied verifier can't loop forever.
- **`Agent.lastPromptTokens`** — estimated token count of the most recent prompt *actually sent* to the model (after context management/trimming), so apps can show real context usage instead of raw-history size.
- **`AgentState.mutate(forKey:default:_:)`** — atomic read-modify-write under the state lock, so parallel tools can update the same key (append to an array, bump a counter) without a lost update. The existing accessors were already individually thread-safe (each guarded by the internal lock; `snapshot()` returns a value-type copy) — this closes the compound-update gap and documents that `@unchecked Sendable` is correct here. Concurrency stress coverage added (2000 concurrent increments → exactly 2000; mixed read/write/snapshot/mutate hammer).
Expand Down
134 changes: 134 additions & 0 deletions Sources/SwiftAgentKitTools/PythonTool.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
//
// PythonTool.swift
// SwiftAgentKitTools
//
// Run Python in an isolated virtual environment. Encapsulates venv creation and
// package installation so on-device Python is a first-class capability rather
// than something the model has to orchestrate via raw shell commands.
// macOS-only (`Process` + venv).
//

#if os(macOS)
import Foundation
import SwiftAgentKit

/// Runs Python 3 code in a dedicated virtual environment and returns its combined
/// stdout/stderr + exit code. Ensures the venv exists and pip-installs any
/// requested packages first (isolated from system Python). `requiresConfirmation`
/// — running code is as powerful as the shell.
public struct PythonTool: AgentTool {
public let name = "run_python"
public let description = """
Run Python 3 code in an isolated virtual environment and return its combined \
stdout/stderr and exit code. Put the full script in `code`. List any \
third-party `packages` you import (e.g. numpy, pandas, matplotlib, fpdf2) — \
they are pip-installed into the environment first. Prefer this over run_shell \
for Python. Every call must be approved.
"""
public let parameters = ToolParameters(
properties: [
"code": ToolParameterProperty(type: "string", description: "The Python 3 source to run."),
"packages": ToolParameterProperty(
type: "array",
description: "Third-party pip packages to ensure are installed before running.",
itemsType: "string"),
],
required: ["code"]
)

public var requiresConfirmation: Bool { true }

private let venvPath: URL
private let timeoutSeconds: Double
private let maxOutputChars: Int

/// - Parameter venvPath: where the virtual environment lives (created if
/// missing). Apps typically point this at a per-app location.
public init(venvPath: URL, timeoutSeconds: Double = 180, maxOutputChars: Int = 10_000) {
self.venvPath = venvPath
self.timeoutSeconds = timeoutSeconds
self.maxOutputChars = maxOutputChars
}

private var pythonPath: String { venvPath.appendingPathComponent("bin/python3").path }

public func execute(parameters: [String: Any]) async throws -> AgentToolResult {
guard let code = (parameters["code"] as? String), !code.isEmpty else {
return .error(toolCallId: "", toolName: name, message: "run_python requires `code`.")
}
let packages = stringArray(parameters["packages"])

// 1. Ensure the venv exists.
if !FileManager.default.isExecutableFile(atPath: pythonPath) {
_ = await runCommand("python3 -m venv \(quote(venvPath.path))")
if !FileManager.default.isExecutableFile(atPath: pythonPath) {
return .error(toolCallId: "", toolName: name,
message: "Could not create a Python venv at \(venvPath.path). Is python3 installed?")
}
}

// 2. Ensure requested packages (one pip call; fast when already satisfied).
if !packages.isEmpty {
let install = await runCommand(
"\(quote(pythonPath)) -m pip install --quiet \(packages.map(quote).joined(separator: " "))")
if install.exitCode != 0 {
return .error(toolCallId: "", toolName: name,
message: "pip install failed (exit \(install.exitCode)):\n\(install.output)")
}
}

// 3. Stage the code to a temp file and run it with the venv python.
let scriptURL = FileManager.default.temporaryDirectory
.appendingPathComponent("sak-python-\(UUID().uuidString).py")
do {
try code.write(to: scriptURL, atomically: true, encoding: .utf8)
} catch {
return .error(toolCallId: "", toolName: name, message: "Failed to stage script: \(error.localizedDescription)")
}
defer { try? FileManager.default.removeItem(at: scriptURL) }

let run = await runCommand("\(quote(pythonPath)) \(quote(scriptURL.path))")
var output = run.output
if output.count > maxOutputChars {
output = String(output.prefix(maxOutputChars)) + "\n… [output truncated]"
}
var header = "exit \(run.exitCode)"
if run.timedOut { header += " (terminated after \(Int(timeoutSeconds))s timeout)" }
return .success(toolCallId: "", toolName: name, result: output.isEmpty ? header : "\(header)\n\(output)")
}

// MARK: - Private

/// Run a command via a login shell (so PATH resolves python3), reading to EOF
/// on a detached task raced against a wall-clock timeout (no pipe deadlock).
private func runCommand(_ command: String) async -> (exitCode: Int32, output: String, timedOut: Bool) {
let process = Process()
process.executableURL = URL(fileURLWithPath: "/bin/zsh")
process.arguments = ["-lc", command]
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
do {
try process.run()
} catch {
return (-1, "Failed to launch: \(error.localizedDescription)", false)
}
let handle = pipe.fileHandleForReading
let readTask = Task.detached { handle.readDataToEndOfFile() }
let timeoutTask = Task { [timeoutSeconds] in
try? await Task.sleep(nanoseconds: UInt64(timeoutSeconds * 1_000_000_000))
if process.isRunning { process.terminate() }
}
let data = await readTask.value
timeoutTask.cancel()
process.waitUntilExit()
let timedOut = process.terminationReason == .uncaughtSignal
return (process.terminationStatus, String(data: data, encoding: .utf8) ?? "", timedOut)
}

/// Single-quote a shell argument safely.
private func quote(_ s: String) -> String {
"'" + s.replacingOccurrences(of: "'", with: "'\\''") + "'"
}
}
#endif
35 changes: 35 additions & 0 deletions Tests/SwiftAgentKitToolsTests/SwiftAgentKitToolsTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -322,3 +322,38 @@ func liveAgentUsesIsolatedVenv() async throws {
let venvHasCowsay = shellCapture("\(venvPy) -c 'import cowsay; print(\"has_cowsay\")'")
#expect(venvHasCowsay.contains("has_cowsay"))
}

// MARK: - PythonTool

#if os(macOS)
@Test func pythonToolRequiresConfirmation() {
#expect(PythonTool(venvPath: tempDir().appendingPathComponent("venv")).requiresConfirmation == true)
}

/// Live: run_python creates a venv, installs a package into it, and runs code —
/// isolated from system Python. Gated (creates a temp venv, installs cowsay).
/// SAK_LIVE_TESTS=1 swift test --filter pythonToolRunsInIsolatedVenv
@Test(.enabled(if: ProcessInfo.processInfo.environment["SAK_LIVE_TESTS"] == "1"))
func pythonToolRunsInIsolatedVenv() async throws {
let dir = tempDir()
let venv = dir.appendingPathComponent("venv")
let tool = PythonTool(venvPath: venv)

// Plain code (no packages): venv is created, code runs.
let r1 = try await tool.execute(parameters: ["code": "print('PY_OK', 6*7)"])
#expect(r1.isError == false)
#expect(r1.result.contains("PY_OK 42"))
#expect(r1.result.contains("exit 0"))
#expect(FileManager.default.isExecutableFile(atPath: venv.appendingPathComponent("bin/python3").path))

// With a package: it's installed into THIS venv and importable.
let r2 = try await tool.execute(parameters: [
"code": "import cowsay; print('COWSAY_OK')",
"packages": ["cowsay"],
])
#expect(r2.result.contains("COWSAY_OK"))
// Proof of isolation: the package landed in the temp venv, not system Python.
let venvHas = shellCapture("\(venv.appendingPathComponent("bin/python3").path) -c 'import cowsay; print(\"in_venv\")'")
#expect(venvHas.contains("in_venv"))
}
#endif
Loading