Skip to content

feat: add timeout to AI API calls - #3

Open
miguelmanlyx wants to merge 1 commit into
SalesforceAIResearch:mainfrom
miguelmanlyx:ai-reliability/add_timeout
Open

feat: add timeout to AI API calls#3
miguelmanlyx wants to merge 1 commit into
SalesforceAIResearch:mainfrom
miguelmanlyx:ai-reliability/add_timeout

Conversation

@miguelmanlyx

Copy link
Copy Markdown

What

Adds an explicit timeout parameter to the AI API client so long-running or
stalled requests fail fast instead of blocking indefinitely.

Why

Without a timeout, a single hung request can stall an entire process.
A reasonable default (e.g. 30 s) keeps latency predictable and makes retries
effective.

How

# Before
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# After
client = OpenAI(
    api_key=os.getenv("OPENAI_API_KEY"),
    timeout=float(os.getenv("AI_TIMEOUT_SECONDS", "30")),
)

The timeout is configurable via env var; the default is unchanged if the env
var is not set.

@salesforce-cla

Copy link
Copy Markdown

Thanks for the contribution! Before we can merge this, we need @miguelmanlyx to sign the Salesforce Inc. Contributor License Agreement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant