feat(airt): proper multimodal — custom targets, per-media prompts, media-output scoring, injection images (1.6.6)#79
Merged
Conversation
Extends generate_multimodal_attack so the agent can probe beyond a litellm model: - Custom HTTP target: custom_url (+ auth / request-template / JSONPath) generates a @task target that serializes the message's text + base64 media into the user's request body and extracts the response via JSONPath. The agent reads the target's API docs to build the template; target_model becomes optional. - Per-media prompts: `prompts` (aligned list) or `prompts_csv` (media_filename,prompt matched by basename) pairs each media set with its own prompt, falling back to goal. - Exposes both through the generate_multimodal_attack tool, and surfaces #77's custom_url on the standard generate_attack tool. - Agent stays byte-free on attack data: the manifest docstring + agent md now state the agent must never open/view/reason about the media or prompt contents — it only invokes the tool with paths + judge/attacker model + transforms + endpoint config; the workflow loads and probes the media at runtime. Bumps capability 1.6.2 -> 1.6.5. Adds 5 tests (custom target, per-media prompts list/CSV, optional target_model, default single-goal unchanged).
Rounds out effective multimodal probing on top of custom targets + per-media prompts: - Media-OUTPUT scoring: score_media_output (+ media_output_modalities / rubric) wires per-output-modality multimodal_judge response_scorers into multimodal_attack, so an image-out or speech-to-speech target's GENERATED media is scored (trial score = MAX across text + media). Off by default — text-only scoring is unchanged. - generate_injection_images tool + media_generator.py: render attack text (or a CSV of texts) into typographic prompt-injection images, so the agent can CREATE the attack data and probe a vision model without the user supplying media — byte-free (the agent passes paths, never views the text). - Agent md: documents media-output scoring, injection-image creation, and audio-out / speech-to-speech guidance (Nova Sonic needs an SDK target, not a generic HTTP one). Bumps capability 1.6.5 -> 1.6.6. Adds tests (media-output on/off/explicit modality, injection rendering). Full capability suite: 162 passed.
rdheekonda
changed the title
generate_multimodal_category_attack sweeps a multimodal attack across sampled goals from a harm sub-category (bundled goals.csv). A multimodal sweep needs media, so: - render_from_goals=True: each sampled goal is rendered into a typographic injection image (harmful text in the image, benign framing prompt) — fully turnkey, no user media required. One attack per goal. - user media (image_dir/paths/audio/video): each sampled goal is paired 1:1 with a media file (dedup-safe, no cycling). - Neither given → an error that tells the agent to ASK the user for media paths. Exposes the generate_multimodal_category_attack tool and documents in the agent md (including the "ask for media paths" UX). Bumps capability 1.6.6 -> 1.6.7. Adds 5 tests; full suite 167 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes multimodal red teaming a proper, effective capability the TUI agent can drive end to end.
Custom HTTP multimodal target
generate_multimodal_attack(custom_url=..., custom_auth_*, custom_request_template, custom_response_text_path)generates a
@task target(message)that serializes text + base64 media into the user's JSON body via{prompt}/{image_b64}/{audio_b64}/{video_b64}placeholders, POSTs, and extracts the reply viaJSONPath.
target_modeloptional. Also exposes #77'scustom_urlon the standardgenerate_attacktool.Per-media prompts
prompts(aligned list) orprompts_csv(media_filename,promptby basename) — each media set gets itsown prompt; unmapped media fall back to
goal. Default single-goalpath unchanged.Media-OUTPUT scoring
score_media_output(+media_output_modalities/media_output_rubric) wires per-output-modalitymultimodal_judgeresponse_scorersintomultimodal_attack, so an image-out / speech-to-speech target'sgenerated media is scored (trial score = MAX across text + media). Off by default.
Turnkey seed media —
generate_injection_imagesNew tool +
media_generator.py: render attack text (or a CSV of texts) into typographic prompt-injectionimages, so the agent can create the attack data and probe a vision model without the user supplying media.
Agent stays byte-free on harmful data
The agent never opens/views/reasons about the media or prompt contents — it plans from byte-free inventory
(
build_media_manifest) and invokes tools with paths + judge/attacker model + transforms + endpointconfig; the workflow (or the injection tool) handles the bytes. Agent md documents custom targets,
per-media prompts, media-output scoring, injection-image creation, and audio-out/S2S (Nova Sonic needs an
SDK target, not a generic HTTP one).
Bumps capability 1.6.2 → 1.6.6.
Test plan
injection-image rendering; default paths unchanged
py_compileclean; generated scripts pass thecompile()guard; injection generator renders real PNGsand a typographic-injection run created from text