Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[submodule "ai-core/subprojects/llama.cpp"]
path = ai-core/subprojects/llama.cpp
[submodule "ai-agent-local/subprojects/llama.cpp"]
path = ai-agent-local/subprojects/llama.cpp
url = https://github.com/appdevforall/llama.cpp.git
branch = androidide-custom
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@ See the official [plugin documentation](https://www.appdevforall.org/codeonthego
| [`compose-preview/`](compose-preview/) | Renders Jetpack Compose `@Preview` functions on-device — no full app build or run. |
| [`ai-literacy-course/`](ai-literacy-course/) | Bundles Learn AI Anywhere's offline "Introduction to AI" course (26 videos + interactive activities) and plays it full-screen, fully offline. |
| [`layout-editor/`](layout-editor/) | Visual drag-and-drop editor for Android XML layouts. |
| [`ai-core/`](ai-core/) | Shared on-device LLM inference backend (bundled llama.cpp AAR) plus a Gemini API backend, exposed to other plugins as a runtime service. |
| [`ai-assistant/`](ai-assistant/) | In-IDE AI chat assistant with tool calling; talks to `ai-core` for inference over local or Gemini models. |
| [`ai-core/`](ai-core/) | The **Agent** chat (tool-calling assistant) plus the shared LLM inference **router** other plugins consume. Ships no model — install at least one backend plugin below. Mandatory for every AI feature. |
| [`ai-agent-local/`](ai-agent-local/) | On-device `.gguf` inference backend for `ai-core` (bundled llama.cpp AAR). Registers as `local`; needs no network. |
| [`ai-agent-gemini/`](ai-agent-gemini/) | Google Gemini API inference backend for `ai-core`. Registers as `gemini`; needs an API key and network access. |
| [`flutter-template/`](flutter-template/) | Adds Flutter starter project templates (Basic, BLoC, Provider, GetX, Riverpod) to the New Project screen. |
| [`code-suggestions-plugin/`](code-suggestions-plugin/) | Inline ghost-text code completions powered by AI. |
| [`speech-to-text-plugin/`](speech-to-text-plugin/) | Voice-to-code: converts speech to code with AI generation. |
Expand Down
File renamed without changes.
69 changes: 69 additions & 0 deletions ai-agent-gemini/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# AI Agent Gemini plugin for CodeOnTheGo

Google Gemini API inference for CodeOnTheGo's AI plugins. Registers itself as the
`gemini` backend with [`ai-core`](../ai-core/)'s `LlmInferenceService`, which is
what `ai-core`'s Agent chat, `code-suggestions-plugin`, `speech-to-text-plugin` and
`vector-search-plugin` actually talk to.

Calls the Generative Language REST API directly over `HttpURLConnection` rather
than the google-genai SDK: the SDK bundles OkHttp 4.x, but plugins run in the
host IDE's classloader where `okhttp3` resolves to the host's older OkHttp, and
that mismatch crashed generation with a `NoSuchMethodError`.

## Building

Prerequisites: Android SDK (API 33+), JDK 17. Create `local.properties` with
`sdk.dir=...`. This plugin uses the shared wrapper at the repo root:

```bash
cd ai-agent-gemini
../gradlew assemblePlugin # release -> build/plugin/ai-agent-gemini.cgp
../gradlew assemblePluginDebug # debug variant
```

## API key handling

The key is entered in **AI Core → Agent settings**, not here. It is stored
encrypted (AES/GCM under a hardware-backed Android Keystore secret) and sent as
an `x-goog-api-key` **header**, never in a URL query string.

`security/SecureApiKeyStore.kt` is the only copy of the crypto — this plugin owns
both the write and the read, so there are no constants to keep in sync with
another plugin. A key written under an earlier plugin id is adopted once by
`preferences/GeminiPreferences.kt` and re-encrypted here.

## Installation

Install **`ai-core` as well** — without the router this plugin has nothing to
register with. Order does not matter: this plugin re-registers when it sees
ai-core activate. Copy `build/plugin/ai-agent-gemini.cgp` to the device, install
via CodeOnTheGo's Plugin Manager, then restart the IDE.

## Cross-plugin contract

This plugin's own settings pane calls `GeminiBackend.listModels()` and
`listModels(String)` directly (see `BackendGeminiCatalogGateway` in
`settings/GeminiCatalogGateway.kt`) to populate the model picker and to verify a
key before it is saved. Those two signatures, and the `ListModels HTTP <code>`
message shape thrown by `fetchAvailableModels`, are a contract — the pane is
mounted by ai-core across the plugin classloader boundary, so
`proguard-rules.pro` pins the class and its public methods.

## Key classes

Every source file sits in a package named for its layer; nothing is loose at the
root of `com/itsaky/androidide/plugins/aiagentgemini/`.

- `plugin/GeminiPlugin.kt` — plugin entry point; registers the backend with ai-core
- `backend/GeminiBackend.kt` — the REST transport, streaming (SSE) and model catalog
- `errors/GeminiErrorFormatter.kt` — turns an API failure into one translated sentence
- `security/SecureApiKeyStore.kt` — AES/GCM at rest
- `preferences/GeminiPreferences.kt` — this plugin's settings store, plus the
one-time adoption of settings written under earlier plugin ids
- `prompt/GeminiSystemPrompt.kt` — the system prompt this cloud model is given
- `logging/` — `LOG_PREFIX` (`AiAgentGemini`), prefixing every logcat tag this plugin writes
- `settings/` — the settings pane this backend contributes to the selector

## License

GPL-3.0 — same as AndroidIDE / CodeOnTheGo.
126 changes: 126 additions & 0 deletions ai-agent-gemini/ai-agent-gemini.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AI Agent Gemini Plugin</title>
<style>
body {
background: #ffffff;
color: #000000;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
line-height: 1.6;
margin: 0;
padding: 1.25rem;
max-width: 820px;
}
h1 { font-size: 1.6rem; margin: 0 0 0.75rem; }
h2 { font-size: 1.2rem; margin: 1.75rem 0 0.5rem; border-bottom: 1px solid #e0e0e0; padding-bottom: 0.2rem; }
code {
background: #f4f4f4;
padding: 0.1rem 0.3rem;
border-radius: 3px;
font-family: "SF Mono", Menlo, Consolas, monospace;
font-size: 0.9em;
}
ul { padding-left: 1.25rem; }
li { margin: 0.3rem 0; }
table { border-collapse: collapse; margin: 0.5rem 0; width: 100%; }
th, td { border: 1px solid #ddd; padding: 0.4rem 0.6rem; text-align: left; vertical-align: top; }
th { background: #f4f4f4; }
.note {
border-left: 3px solid #999;
padding: 0.25rem 0.75rem;
margin: 1rem 0;
background: #fafafa;
}
</style>
</head>
<body>
<h1>AI Agent Gemini Plugin</h1>

<h2>Executive overview</h2>
<p><b>AI Agent Gemini</b> adds Google's <b>Gemini</b> models to CodeOnTheGo's
AI features. It registers itself as the <code>gemini</code> inference backend
with <b>AI Core</b>, which routes requests from the <b>Agent</b> chat, <b>Code
Suggestions</b>, <b>Speech to Text</b> and <b>Vector Search</b>. It also
contributes <b>its own settings pane</b> — the API key field and the model
picker — which AI Core mounts inside Agent settings.</p>
<p>Install <b>AI Core</b> alongside it — without the router this plugin has
nothing to register with. Install order does not matter.</p>
<p>Using this backend sends prompts, and any file contents a plugin includes in
them, to Google over HTTPS. Choose <b>AI Agent Local</b> instead if inference
must stay on the device.</p>

<h2>Core functionality</h2>
<ul>
<li><b>Cloud inference</b> via direct calls to the Gemini REST API, with text
completion, chat with history, and server-sent-events streaming.</li>
<li><b>Live model catalog</b> — lists the models the saved key can actually
use, so a retired model name is never offered.</li>
<li><b>Key verification before saving</b> — a newly entered key is checked
against the catalog, distinguishing a refused key from an unreachable
network.</li>
<li><b>Encrypted at rest</b> — the API key is stored as AES/GCM ciphertext
under a hardware-backed Android Keystore secret, and a pre-existing
plaintext key is upgraded in place on first read.</li>
<li><b>Translated, safe error messages</b> — an API failure becomes one
user-facing sentence; the raw HTTP error body stays in the log and never
reaches the chat transcript.</li>
</ul>

<h2>Technical architecture</h2>
<table>
<tr><th>Component</th><th>Role</th></tr>
<tr><td><code>GeminiPlugin</code></td><td>Plugin entry point. Registers the
backend with AI Core on activation, re-registering if AI Core activates
later; cancels in-flight requests and drops the decrypted key on
dispose.</td></tr>
<tr><td><code>GeminiBackend</code></td><td>The transport. Calls the
Generative Language REST API over <code>HttpURLConnection</code>, parses the
streaming response, and fetches the model catalog.</td></tr>
<tr><td><code>GeminiErrorFormatter</code></td><td>Classifies a failure
(retired model, quota, refused key, outage, unreachable) so it can be
reported as one translated sentence.</td></tr>
<tr><td><code>SecureApiKeyStore</code></td><td>AES/GCM encryption of the API
key under a hardware-backed Android Keystore secret owned by this
plugin.</td></tr>
<tr><td><code>GeminiSettingsFragment</code></td><td>The settings pane AI Core
mounts: key entry and verification, visibility toggle, and the model picker
driven by the live catalog.</td></tr>
</table>
<p><b>No third-party HTTP SDK.</b> The google-genai SDK bundles OkHttp 4.x, but
plugins run in the host IDE's classloader where <code>okhttp3</code> resolves to
the host's older OkHttp — a mismatch that crashed generation with a
<code>NoSuchMethodError</code>. <code>HttpURLConnection</code> has no
third-party dependency and works regardless of the host's OkHttp version.</p>

<h2>Usage</h2>
<ol>
<li>Obtain a Gemini API key from Google AI Studio.</li>
<li>Install <b>AI Core</b> and <b>AI Agent Gemini</b> via the Plugin
Manager, then restart the IDE.</li>
<li>Open <b>Preferences &rarr; Configuration &rarr; Agent</b> and select the
<b>gemini</b> backend. This plugin's own pane appears below it.</li>
<li>Enter the key and tap <b>Save</b> — it is verified against the API before
it is stored — then pick a model from the live list.</li>
</ol>
<div class="note">
The key is sent as an <code>x-goog-api-key</code> request header, never in a
URL query string — query strings leak into logs, proxies and crash reports.
</div>

<h2>Key benefits</h2>
<ul>
<li><b>High capability</b> — frontier-class models on a device that could not
run them locally.</li>
<li><b>Small footprint</b> — no bundled model or native library.</li>
<li><b>Credential hygiene</b> — encrypted at rest, header-only in transit,
dropped from memory when the plugin unloads.</li>
<li><b>No dead model names</b> — the picker reflects the live catalog for your
specific key.</li>
<li><b>Coexists with the local backend</b> — install both and switch in AI
Settings.</li>
</ul>
</body>
</html>
52 changes: 21 additions & 31 deletions ai-assistant/build.gradle.kts → ai-agent-gemini/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,23 @@ plugins {
}

pluginBuilder {
pluginName = "ai-assistant"
pluginName = "ai-agent-gemini"
}

android {
namespace = "com.itsaky.androidide.plugins.aiassistant"
namespace = "com.itsaky.androidide.plugins.aiagentgemini"
compileSdk = 36

defaultConfig {
applicationId = "com.itsaky.androidide.plugins.aiassistant"
applicationId = "com.itsaky.androidide.plugins.aiagentgemini"
minSdk = 33
targetSdk = 36
versionCode = 2
versionName = "1.1.0"
}

buildFeatures {
viewBinding = true
buildConfig = true
versionCode = 1
versionName = "1.0.0"
}

buildTypes {
release {
// Disable minification to avoid lambda obfuscation issues with ClassLoader isolation
isMinifyEnabled = false
isShrinkResources = false
signingConfig = signingConfigs.getByName("debug")
Expand All @@ -46,18 +40,15 @@ android {
}
}

testOptions {
unitTests.isReturnDefaultValues = true
}

packaging {
resources {
excludes += setOf(
"META-INF/DEPENDENCIES",
"META-INF/LICENSE",
"META-INF/LICENSE.txt",
"META-INF/NOTICE",
"META-INF/NOTICE.txt"
"META-INF/NOTICE.txt",
"META-INF/INDEX.LIST"
)
}
}
Expand All @@ -66,26 +57,25 @@ android {
dependencies {
compileOnly(files("../libs/plugin-api.jar"))

// Use 'implementation' (not 'compileOnly') for androidx libraries.
// This is required for XML layouts: AAPT2 needs these dependencies at compile-time to process
// resource attributes and resolve xmlns declarations. This is standard across all CoGo plugins
// with XML layouts (random-xkcd, sketch-to-ui-plugin, Beepy). See investigation in Task 4.
// 'implementation' (not 'compileOnly') for the androidx/Material libraries: AAPT2 needs them
// at compile time to process the settings pane's layout, as in every CoGo plugin with XML.
implementation("androidx.appcompat:appcompat:1.6.1")
implementation("androidx.fragment:fragment-ktx:1.8.8")
implementation("com.google.android.material:material:1.10.0")
implementation("androidx.recyclerview:recyclerview:1.3.2")
implementation("androidx.constraintlayout:constraintlayout:2.1.4")

// Markdown rendering - plugin-specific library
implementation("io.noties.markwon:core:4.6.2")

// JSON serialization for session persistence
implementation("com.google.code.gson:gson:2.10.1")
implementation("org.jetbrains.kotlin:kotlin-stdlib:2.3.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")

testImplementation(files("../libs/plugin-api.jar"))
testImplementation("junit:junit:4.13.2")
testImplementation("io.mockk:mockk:1.13.8")
testImplementation("org.json:json:20240303")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.8.1")
testImplementation("androidx.arch.core:core-testing:2.2.0")
testImplementation("org.json:json:20231013")
}

// SecureApiKeyStore is no longer duplicated: this plugin holds the only copy, so there is nothing
// left to drift against. The parity check that guarded the ai-assistant copy went with that plugin.

// AAR metadata checks are disabled by convention for these application-as-library plugins.
tasks.matching {
it.name.contains("checkDebugAarMetadata") ||
it.name.contains("checkReleaseAarMetadata")
}.configureEach { enabled = false }
File renamed without changes.
21 changes: 21 additions & 0 deletions ai-agent-gemini/proguard-rules.pro
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# AI Agent Gemini Plugin ProGuard Rules

# Keep plugin entry point
-keep public class com.itsaky.androidide.plugins.aiagentgemini.plugin.GeminiPlugin {
public <methods>;
}

# Keep the backend: AI Core's settings pane calls listModels across the plugin
# classloader boundary, because listModels is not on LlmBackend. Renaming or
# stripping it breaks the model picker and key verification silently.
-keep public class com.itsaky.androidide.plugins.aiagentgemini.backend.GeminiBackend {
public <methods>;
}

# Keep the settings fragment: it is instantiated by name from getSettingsFragmentClassName().
-keep public class com.itsaky.androidide.plugins.aiagentgemini.settings.GeminiSettingsFragment {
public <init>(...);
}

# Keep plugin-api interfaces
-keep interface com.itsaky.androidide.plugins.** { *; }
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ dependencyResolutionManagement {
}
}

rootProject.name = "ai-assistant"
rootProject.name = "ai-agent-gemini"
Loading
Loading