diff --git a/CHANGELOG.md b/CHANGELOG.md index b40b0b9..71c765e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ All notable changes to SwiftAgentKit will be documented in this file. - **ContextSift: stop the `artifact_read` retrieval loop.** A large *active* tool result was bounded to `maxActiveResultChars` and spilled to an artifact with a "use artifact_read" hint; when the model then called `artifact_read`, that retrieval result was *itself* re-truncated and re-spilled — so the model kept calling `artifact_read` to "get the full output" that never fully surfaced (observed as ~6 repeated `artifact_read` calls until `maxTurns` ran out). Fix: retrieval tools (`artifact_read`, `artifact_search`) are now exempt from active-display truncation and receipt spilling — their output is shown in full (the tools already page via `offset`/`limit`). Also raised the default `maxActiveResultChars` 2000 → 8000 so ordinary tool outputs fit inline without any artifact round-trip. Regression coverage added (an active `artifact_read` result larger than the bound is shown in full, never re-truncated). ### Added +- **`PythonTool` (`run_python`) in `SwiftAgentKitTools`.** A first-class tool that runs Python 3 in a dedicated virtual environment: it ensures the venv exists, pip-installs any requested `packages` into it (isolated from system Python), stages the `code` to a temp file, runs it, and returns combined stdout/stderr + exit code. `requiresConfirmation`, configurable `venvPath`/`timeoutSeconds`/`maxOutputChars`, wall-clock timeout, macOS-only. Turns "make the model orchestrate venv + pip via raw shell" into one reliable capability. New tests (confirmation flag; a gated live test creating a venv, installing a package into it, and running code — proving isolation). - **Goal-completion verifier loop — don't trust the model's "done" signal.** New `AgentCallbacks.verifyCompletion(query, proposedAnswer, state) -> GoalVerdict`: when the model finishes a turn with no tool calls, the agent asks the verifier whether the goal is actually met before stopping. `.satisfied` stops; `.unsatisfied(reason:)` feeds the reason back as a nudge and **keeps the loop going**; `.blocked(reason:)` stops early and surfaces the blocker (a "big failure to mention") instead of burning turns. Bounded by `AgentConfig.maxVerificationRetries` (default 3) and `maxTurns`. New events `completionVerificationFailed` / `completionBlocked`. This turns "loop until the model says done" into "loop until the goal is verifiably done, or a real blocker" — the verifier can be a deterministic check (does the output file exist? do tests pass?) or a judge-model call. Regression coverage: retries until satisfied; stops immediately when satisfied; stops early on blocked; bounded so an always-unsatisfied verifier can't loop forever. - **`Agent.lastPromptTokens`** — estimated token count of the most recent prompt *actually sent* to the model (after context management/trimming), so apps can show real context usage instead of raw-history size. - **`AgentState.mutate(forKey:default:_:)`** — atomic read-modify-write under the state lock, so parallel tools can update the same key (append to an array, bump a counter) without a lost update. The existing accessors were already individually thread-safe (each guarded by the internal lock; `snapshot()` returns a value-type copy) — this closes the compound-update gap and documents that `@unchecked Sendable` is correct here. Concurrency stress coverage added (2000 concurrent increments → exactly 2000; mixed read/write/snapshot/mutate hammer). diff --git a/Sources/SwiftAgentKitTools/PythonTool.swift b/Sources/SwiftAgentKitTools/PythonTool.swift new file mode 100644 index 0000000..a584a9e --- /dev/null +++ b/Sources/SwiftAgentKitTools/PythonTool.swift @@ -0,0 +1,134 @@ +// +// PythonTool.swift +// SwiftAgentKitTools +// +// Run Python in an isolated virtual environment. Encapsulates venv creation and +// package installation so on-device Python is a first-class capability rather +// than something the model has to orchestrate via raw shell commands. +// macOS-only (`Process` + venv). +// + +#if os(macOS) +import Foundation +import SwiftAgentKit + +/// Runs Python 3 code in a dedicated virtual environment and returns its combined +/// stdout/stderr + exit code. Ensures the venv exists and pip-installs any +/// requested packages first (isolated from system Python). `requiresConfirmation` +/// — running code is as powerful as the shell. +public struct PythonTool: AgentTool { + public let name = "run_python" + public let description = """ + Run Python 3 code in an isolated virtual environment and return its combined \ + stdout/stderr and exit code. Put the full script in `code`. List any \ + third-party `packages` you import (e.g. numpy, pandas, matplotlib, fpdf2) — \ + they are pip-installed into the environment first. Prefer this over run_shell \ + for Python. Every call must be approved. + """ + public let parameters = ToolParameters( + properties: [ + "code": ToolParameterProperty(type: "string", description: "The Python 3 source to run."), + "packages": ToolParameterProperty( + type: "array", + description: "Third-party pip packages to ensure are installed before running.", + itemsType: "string"), + ], + required: ["code"] + ) + + public var requiresConfirmation: Bool { true } + + private let venvPath: URL + private let timeoutSeconds: Double + private let maxOutputChars: Int + + /// - Parameter venvPath: where the virtual environment lives (created if + /// missing). Apps typically point this at a per-app location. + public init(venvPath: URL, timeoutSeconds: Double = 180, maxOutputChars: Int = 10_000) { + self.venvPath = venvPath + self.timeoutSeconds = timeoutSeconds + self.maxOutputChars = maxOutputChars + } + + private var pythonPath: String { venvPath.appendingPathComponent("bin/python3").path } + + public func execute(parameters: [String: Any]) async throws -> AgentToolResult { + guard let code = (parameters["code"] as? String), !code.isEmpty else { + return .error(toolCallId: "", toolName: name, message: "run_python requires `code`.") + } + let packages = stringArray(parameters["packages"]) + + // 1. Ensure the venv exists. + if !FileManager.default.isExecutableFile(atPath: pythonPath) { + _ = await runCommand("python3 -m venv \(quote(venvPath.path))") + if !FileManager.default.isExecutableFile(atPath: pythonPath) { + return .error(toolCallId: "", toolName: name, + message: "Could not create a Python venv at \(venvPath.path). Is python3 installed?") + } + } + + // 2. Ensure requested packages (one pip call; fast when already satisfied). + if !packages.isEmpty { + let install = await runCommand( + "\(quote(pythonPath)) -m pip install --quiet \(packages.map(quote).joined(separator: " "))") + if install.exitCode != 0 { + return .error(toolCallId: "", toolName: name, + message: "pip install failed (exit \(install.exitCode)):\n\(install.output)") + } + } + + // 3. Stage the code to a temp file and run it with the venv python. + let scriptURL = FileManager.default.temporaryDirectory + .appendingPathComponent("sak-python-\(UUID().uuidString).py") + do { + try code.write(to: scriptURL, atomically: true, encoding: .utf8) + } catch { + return .error(toolCallId: "", toolName: name, message: "Failed to stage script: \(error.localizedDescription)") + } + defer { try? FileManager.default.removeItem(at: scriptURL) } + + let run = await runCommand("\(quote(pythonPath)) \(quote(scriptURL.path))") + var output = run.output + if output.count > maxOutputChars { + output = String(output.prefix(maxOutputChars)) + "\n… [output truncated]" + } + var header = "exit \(run.exitCode)" + if run.timedOut { header += " (terminated after \(Int(timeoutSeconds))s timeout)" } + return .success(toolCallId: "", toolName: name, result: output.isEmpty ? header : "\(header)\n\(output)") + } + + // MARK: - Private + + /// Run a command via a login shell (so PATH resolves python3), reading to EOF + /// on a detached task raced against a wall-clock timeout (no pipe deadlock). + private func runCommand(_ command: String) async -> (exitCode: Int32, output: String, timedOut: Bool) { + let process = Process() + process.executableURL = URL(fileURLWithPath: "/bin/zsh") + process.arguments = ["-lc", command] + let pipe = Pipe() + process.standardOutput = pipe + process.standardError = pipe + do { + try process.run() + } catch { + return (-1, "Failed to launch: \(error.localizedDescription)", false) + } + let handle = pipe.fileHandleForReading + let readTask = Task.detached { handle.readDataToEndOfFile() } + let timeoutTask = Task { [timeoutSeconds] in + try? await Task.sleep(nanoseconds: UInt64(timeoutSeconds * 1_000_000_000)) + if process.isRunning { process.terminate() } + } + let data = await readTask.value + timeoutTask.cancel() + process.waitUntilExit() + let timedOut = process.terminationReason == .uncaughtSignal + return (process.terminationStatus, String(data: data, encoding: .utf8) ?? "", timedOut) + } + + /// Single-quote a shell argument safely. + private func quote(_ s: String) -> String { + "'" + s.replacingOccurrences(of: "'", with: "'\\''") + "'" + } +} +#endif diff --git a/Tests/SwiftAgentKitToolsTests/SwiftAgentKitToolsTests.swift b/Tests/SwiftAgentKitToolsTests/SwiftAgentKitToolsTests.swift index 0cf8401..da6cb62 100644 --- a/Tests/SwiftAgentKitToolsTests/SwiftAgentKitToolsTests.swift +++ b/Tests/SwiftAgentKitToolsTests/SwiftAgentKitToolsTests.swift @@ -322,3 +322,38 @@ func liveAgentUsesIsolatedVenv() async throws { let venvHasCowsay = shellCapture("\(venvPy) -c 'import cowsay; print(\"has_cowsay\")'") #expect(venvHasCowsay.contains("has_cowsay")) } + +// MARK: - PythonTool + +#if os(macOS) +@Test func pythonToolRequiresConfirmation() { + #expect(PythonTool(venvPath: tempDir().appendingPathComponent("venv")).requiresConfirmation == true) +} + +/// Live: run_python creates a venv, installs a package into it, and runs code — +/// isolated from system Python. Gated (creates a temp venv, installs cowsay). +/// SAK_LIVE_TESTS=1 swift test --filter pythonToolRunsInIsolatedVenv +@Test(.enabled(if: ProcessInfo.processInfo.environment["SAK_LIVE_TESTS"] == "1")) +func pythonToolRunsInIsolatedVenv() async throws { + let dir = tempDir() + let venv = dir.appendingPathComponent("venv") + let tool = PythonTool(venvPath: venv) + + // Plain code (no packages): venv is created, code runs. + let r1 = try await tool.execute(parameters: ["code": "print('PY_OK', 6*7)"]) + #expect(r1.isError == false) + #expect(r1.result.contains("PY_OK 42")) + #expect(r1.result.contains("exit 0")) + #expect(FileManager.default.isExecutableFile(atPath: venv.appendingPathComponent("bin/python3").path)) + + // With a package: it's installed into THIS venv and importable. + let r2 = try await tool.execute(parameters: [ + "code": "import cowsay; print('COWSAY_OK')", + "packages": ["cowsay"], + ]) + #expect(r2.result.contains("COWSAY_OK")) + // Proof of isolation: the package landed in the temp venv, not system Python. + let venvHas = shellCapture("\(venv.appendingPathComponent("bin/python3").path) -c 'import cowsay; print(\"in_venv\")'") + #expect(venvHas.contains("in_venv")) +} +#endif