Skip to content

Implement prompt execution provenance and observability #69

Description

@andrewperis

Parent epic: #4

Summary

Implement the Prompt SDK provenance and observability records required to trace how a prompt execution was produced without logging secrets or unnecessary sensitive/private content.

Scope

  • Record prompt ID/version and Prompt Specification version.
  • Record hashes/identifiers for rendered prompt and relevant context package inputs as policy permits.
  • Record provider/model/adapter identity and normalized generation parameters.
  • Record execution timestamps, duration, usage metadata, result identifiers, and status.
  • Record validation/lint and structured-output validation outcomes.
  • Support correlation/execution IDs for downstream manifest/build orchestration.
  • Define redaction and safe-logging rules.
  • Keep observability sinks pluggable rather than requiring a specific telemetry vendor.

Deliverables

  • Execution provenance schema/model.
  • SDK instrumentation hooks.
  • Redaction/safe-logging policy.
  • Default local/test observer implementation.
  • Tests and examples.

Acceptance Criteria

  • Every SDK execution can produce a stable execution/provenance record.
  • Prompt and context versions/identifiers are traceable.
  • Provider/model and relevant parameters are traceable.
  • Secrets and prohibited sensitive data are not logged by default.
  • Observability can be integrated without a specific vendor dependency.
  • Records are suitable for later Comic Manifest/Build Orchestrator provenance.
  • Tests verify redaction and metadata completeness.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions