Skip to content

feat(ai): GeminiLanguageModel thought summaries - #16623

Draft
andrewheard wants to merge 1 commit into
ah/ai-glm-tool-callingfrom
ah/glm-thought-summaries
Draft

feat(ai): GeminiLanguageModel thought summaries#16623
andrewheard wants to merge 1 commit into
ah/ai-glm-tool-callingfrom
ah/glm-thought-summaries

Conversation

@andrewheard

Copy link
Copy Markdown
Contributor

Added support for configuring Gemini thinking levels (reasoningLevel) and thought summaries within Apple's FoundationModels framework. These can be configured at the model level, with dynamic session profiles, or per request.

Apple's FoundationModels framework models reasoning depth using ContextOptions.ReasoningLevel (.light, .moderate, .deep), but does not have a native representation for configuring Gemini-specific thought summaries. This PR bridges the two systems cleanly:

  1. It aligns reasoning depth with Apple's standard ContextOptions.ReasoningLevel across profiles and request options.
  2. It introduces Gemini-specific configurations for thought summary generation (summaries: .auto / .off) that align with the Gemini Interactions API naming.
  3. It exposes thought summaries directly on response types, stream snapshots, transcripts, and observable session properties.

Key Features & Architecture

1. Model Configuration (GeminiLanguageModel.Thinking)

  • Adds GeminiLanguageModel.Thinking to set the default thought summary mode (.auto or .off) on GeminiLanguageModel and FirebaseAI.

2. Dynamic Profile Modifiers (.geminiThinking)

  • Adds geminiThinking(summaries:) and geminiThinking(perform:) to LanguageModelSession.DynamicProfile.
  • Automatically attaches GeminiRequestMetadata to outgoing prompts in session history without mutating or re-instantiating the underlying model.
  • Automatically maintains session.properties.geminiThoughtSummary as @Observable state across conversation turns.

3. Direct, Zero-Synchronization Inspection

  • Exposes geminiThoughtSummary: String? directly on:
    • LanguageModelSession.Response
    • LanguageModelSession.ResponseStream.Snapshot (for streaming)
    • Transcript
  • Eliminates the need for manual locks, Mutex, or async stream tapping to retrieve the generated reasoning summary.

4. Per-Turn Request Metadata (GeminiRequestMetadata)

  • Introduces GeminiRequestMetadata conforming to ConvertibleToGeneratedContent and ConvertibleFromGeneratedContent.
  • Stored under the "gemini" metadata namespace to keep the session metadata clean.
  • Supports dictionary helper .gemini(thinkingSummaries: .auto) for convenient per-request configuration.

5. Standard Apple Reasoning Level Mapping

  • Translates Apple's ContextOptions.ReasoningLevel directly to Gemini's ThinkingConfig.thinkingLevel:
    • .light -> .low
    • .moderate -> .medium
    • .deep -> .high
    • .custom(...) -> case-insensitive string mapping ("MINIMAL", "LOW", "MEDIUM", "HIGH").

6. Strict Precedence Hierarchy

The request translator resolves thinking configurations hierarchically:

  • Per-Turn Request Metadata > Profile Prompt Metadata > Model Default

Usage

Examples

Example 1: Model Default Configuration

Configure thought summaries globally when initializing the model:

import FirebaseAI
import FoundationModels

let model = FirebaseAI.geminiLanguageModel(
  modelID: "gemini-3.5-flash-lite",
  thinking: GeminiLanguageModel.Thinking(summaries: .auto)
)

let session = LanguageModelSession(model: model)
let response = try await session.respond(
  to: "How many r's are in strawberry? Think step by step.",
  contextOptions: ContextOptions(reasoningLevel: .deep)
)

if let thoughts = response.geminiThoughtSummary {
  print("Model Thinking:\n\(thoughts)")
}
print("Final Answer:\n\(response.content)")

Example 2: Dynamic Profile with SwiftUI Observability

Bind a SwiftUI view directly to session.properties.geminiThoughtSummary without
any manual state management or synchronization:

import FoundationModels
import GeminiLanguageModel
import SwiftUI

struct AssistantView: View {
  @State private var session: LanguageModelSession

  init(model: GeminiLanguageModel) {
    let profile = LanguageModelSession.Profile {
      Instructions("You are a helpful science tutor.")
    }
    .model(model)
    .reasoningLevel(.deep)
    .geminiThinking(summaries: .auto)

    _session = State(initialValue: LanguageModelSession(profile: profile))
  }

  var body: some View {
    VStack(alignment: .leading, spacing: 12) {
      if let thought = session.properties.geminiThoughtSummary {
        GroupBox("Thinking Process") {
          Text(thought)
            .font(.caption)
            .foregroundStyle(.secondary)
        }
      }

      // Main chat interface...
    }
  }
}

Example 3: Live Thought Streaming with perform:

Receive thought updates during the generation cycle using the observer modifier:

let profile = LanguageModelSession.Profile {
  Instructions("You are a helpful math tutor.")
}
.model(model)
.reasoningLevel(.deep)
.geminiThinking { thoughtSummary in
  print("Intermediate Thought:\n\(thoughtSummary)")
}

let session = LanguageModelSession(profile: profile)
let response = try await session.respond(to: "Solve 47 * 89 step by step.")

Example 4: Streaming Snapshot Inspection

Inspect partial thought summaries in real time while streaming response tokens:

let stream = session.streamResponse(to: "Explain quantum superposition.")

for try await snapshot in stream {
  if let partialThought = snapshot.geminiThoughtSummary {
    print("Streaming Thoughts: \(partialThought)")
  }
  print("Streaming Content: \(snapshot.content)")
}

Example 5: Per-Turn Request Metadata Override

Enable thought summaries and deep reasoning for a single difficult query without
altering session- or model-level settings:

let session = LanguageModelSession(model: model)

let response = try await session.respond(
  contextOptions: ContextOptions(reasoningLevel: .deep),
  metadata: .gemini(thinkingSummaries: .auto)
) {
  "Solve this logic puzzle: You have 3 light switches outside a closed room..."
}

#expect(response.geminiThoughtSummary != nil)

#no-changelog

Add support for configuring and observing Gemini internal reasoning and
thought summaries across model, profile, and request layers.

- Add `GeminiLanguageModel.Thinking` to configure default thought summary
  modes (`.auto`, `.off`) on `GeminiLanguageModel` and `FirebaseAI`.
- Add `LanguageModelSession.DynamicProfile` extensions `.geminiThinking`
  to enable thought summaries and observe incoming reasoning thoughts
  without mutating the underlying model instance.
- Expose `session.properties.geminiThoughtSummary` as observable state
  managed automatically during session turns.
- Provide zero-synchronization `geminiThoughtSummary` properties on
  `LanguageModelSession.Response`, `ResponseStream.Snapshot`, and
  `Transcript`.
- Introduce `GeminiRequestMetadata` to enable per-turn thought summary
  overrides via `session.respond(metadata:)`.
- Map Apple's `ContextOptions.ReasoningLevel` (`.light`, `.moderate`,
  `.deep`, `.custom`) to Gemini's thinking level in the request
  translator.
- Add unit and integration tests covering thought summaries, profile
  modifiers, request metadata precedence, and deep reasoning.
@gemini-code-assist

Copy link
Copy Markdown
Contributor
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@andrewheard
andrewheard force-pushed the ah/glm-thought-summaries branch from 9d8872a to 7733eb2 Compare September 8, 2026 01:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant