feat: Send PDF and Office files to Responses API models end-to-end by danielle-mustillo · Pull Request #3200 · kagent-dev/kagent

danielle-mustillo · 2026-10-14T14:22:32Z

🤖 written by Claude (start)

Problem

Models did not get the contents of PDF and Office files, or of files at a link.

A PDF or Office file sent as bytes became a "could not be read as text" note, also on Responses API models that can read it.
The chat UI did not accept PDF or Office files.
A file link typed in the chat stayed plain text, so a model that can fetch files did not get the file.
An A2A url part reached only Gemini and Responses API models. Other models dropped it with no error.

Resolution

A Responses API model (apiFormat: responses) reads PDF, .docx, .xlsx and .pptx files sent as bytes.
The chat UI accepts these four file types.
The chat also sends a typed link to a known file type as a url part, unless the host is an IP address or a LAN name. Responses API and Gemini models fetch the file.
Other models get a url part as its bare URL in the text, so the agent's fetch tool can read it.

User Experience

A kagent UI user can now attach report.pdf, or type Summarize https://example.com/q3-report.pdf, and a Responses API agent answers from the file's contents.

A kagent A2A user can now send a url part to any model. Models that cannot fetch files get the URL as text and do not drop it.

Changes

The user sends	Responses API model	Gemini	Other models
PDF, .docx, .xlsx or .pptx as bytes	the model reads the file (new 🆕)	"could not be read" note	"could not be read" note
Text, CSV, JSON or image as bytes	unchanged	unchanged	unchanged
Typed link to a known file type	the model fetches the file (new 🆕)	the model fetches the file (new 🆕)	unchanged: the URL stays in the text
A2A url part	the model fetches the file	the model fetches the file	bare URL in the text (new 🆕)
Link from an older turn	bare URL in the text (new 🆕)	bare URL in the text (new 🆕)	bare URL in the text
Files as bytes over 50 MB in one chat	newest files sent, older ones a note (new 🆕)	unchanged	unchanged

Limits:

A bad link fails only the turn that sends it. A follow-up about that link does not see the file again.
A damaged file fails each later turn until the 50 MB cap drops it.
Gemini can ignore a link to a type it does not document, such as .docx. It gives no error.
A GitHub blob/ link sends the web page's HTML, not the file.
Azure documents only PDF.

Tested

Kind cluster at a6704386, Azure OpenAI gpt-4.1-mini. A Responses API agent read the four types, .txt and a PDF link. A Chat Completions agent got the note, and with a fetch tool it fetched the typed link.

🤖 written by Claude (end)

A Responses API model now gets PDF, Word, Excel and PowerPoint bytes as an input_file part, and the chat UI accepts these types. Other models still get a "could not be read" note. A link that the model cannot send now fails the turn it was sent on with an error that names the link. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

The model does not see input_file's filename, so a text part before the file names it, as it already did for links. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

The chat sends each http(s) link to a known file type in the message text also as a url part, with no chip. A model that reads links (Responses API, Gemini) fetches the file. For other models the runtime drops the part, since its URL is in the text, or replaces it with the bare URL, so the agent's fetch tool can read it. This replaces the error for a link the model cannot read. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

The model reads the message as markdown, so links are now found the same way: bare URLs, <autolinks> and [markdown](links), but none inside code. A URL that ends in a bracket or a full stop keeps the right end. Tests cover the composer, the chip rules and the parser cases. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

…rn step filesToText copies only the user turns that change, and turnFilesToText decides each part. The comments now say which parts are left as they are. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

danielle-mustillo · 2026-10-15T03:35:54Z

Before/after on a kind cluster with an Azure OpenAI gpt-4.1-mini agent (apiFormat: responses). Base b42a6c53, head 3e437e3c.

Composer: PDF and .docx now stage as chips. Before, both were "not a supported file type".
Attached files: the model now reads both files and gives each secret word. Before, it never got the files and asked for them.
Link in the text: the model now reads a linked PDF and quotes its title. Before, the link stayed text and the model asked to download it.

pr-3200-reel.mp4

Before: the composer rejects PDF and .docx

After: the composer stages PDF and .docx

Before: the model asks for the files

After: the model reads each secret word

Before: the model asks to download the link

After: the model quotes the linked PDF

🤖 written by Claude

…PI models Also assert OpenAIModel implements FileSender, and stop the mock chat client from showing a chip for a link in the text. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

…he link media type helper Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

Charlesthebird

This automated review is a first pass. A manual review will follow once the blockers are addressed.

Blocks merge: the UI turns every typed link with a known file extension into a url part, and Responses API models fetch it again on every turn. One unreachable or non-file link then breaks the whole chat.

⚠️ What I checked

✅ Go tests in fileextract, models and a2a
✅ UI tests for attachments, a2aGrpcChatClient, useChat and ChatComposer
✅ Non-Responses adapters (Anthropic, Bedrock, Ollama) get url parts as bare URL text
✅ filesToText copies changed turns instead of mutating session history
Links in textLinks: internal hosts, .md/.yaml blob pages
Replay of url parts and file bytes on later turns

🤖 written by Claude

Links in older user turns are now sent as bare URL text, for all models. A tool result no longer counts as the newest user turn, so a link stays a file for the whole turn that sends it. Responses API models get at most 50 MB of files per request, newest first. Older files over the cap become a note. A PDF must look like a PDF and an Office file like a zip, else it gets the "could not be read" note. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

The provider fetches a link itself and cannot reach these hosts. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

Charlesthebird

Thanks for adding this and including the screenshots and video!

danielle-mustillo added the work-in-progress label Oct 14, 2026

github-actions Bot added the enhancement New feature or request label Oct 14, 2026

danielle-mustillo mentioned this pull request Oct 14, 2026

feat(ui): Send a pasted link as a file #3201

Closed

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 14, 2026

danielle-mustillo self-assigned this Oct 14, 2026

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 14, 2026

danielle-mustillo added 2 commits October 14, 2026 21:40

fix(adk): name byte files for Responses API models in a text part

e96dc7e

The model does not see input_file's filename, so a text part before the file names it, as it already did for links. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

danielle-mustillo force-pushed the danielle-mustillo/pdf-office-files branch from a670438 to 90e9d5d Compare October 15, 2026 01:40

danielle-mustillo added 2 commits October 14, 2026 22:27

This comment has been minimized.

Sign in to view

danielle-mustillo marked this pull request as ready for review October 15, 2026 03:47

danielle-mustillo requested review from Charlesthebird and supreme-gg-gg as code owners October 15, 2026 03:47

danielle-mustillo changed the title ~~feat: Send PDF and Office files to Responses API models~~ feat: Send PDF and Office files to Responses API models end-to-end Oct 15, 2026

danielle-mustillo changed the title ~~feat: Send PDF and Office files to Responses API models end-to-end~~ feat: Send PDF and Office files to Responses API models end-to-end Oct 15, 2026

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 15, 2026

danielle-mustillo added 3 commits October 15, 2026 10:34

fix: accept PDF and Office media types with parameters on Responses A…

bdda4ab

…PI models Also assert OpenAIModel implements FileSender, and stop the mock chat client from showing a chip for a link in the text. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

refactor: cut the moved newest-turn helper, a repeated data URL and t…

8e6088d

…he link media type helper Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

fix(ui): keep the chip of an agent-sent link that its text also names

133c20a

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

danielle-mustillo force-pushed the danielle-mustillo/pdf-office-files branch from 0c5afed to 133c20a Compare October 15, 2026 14:34

Charlesthebird reviewed Oct 15, 2026

View reviewed changes

Comment thread ui/src/api/chat/attachments.ts Resolved

Comment thread go/adk/pkg/models/openai_responses.go Resolved

Comment thread go/adk/pkg/models/openai_responses.go Resolved

danielle-mustillo added 2 commits October 16, 2026 16:15

fix(ui): keep links to IP addresses and LAN hosts as text

6f1489f

The provider fetches a link itself and cannot reach these hosts. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 16, 2026

danielle-mustillo commented Oct 16, 2026

View reviewed changes

Comment thread go/adk/pkg/models/openai_responses.go Resolved

Charlesthebird approved these changes Oct 16, 2026

View reviewed changes

danielle-mustillo requested a review from moezdil October 16, 2026 21:00

EItanya added this pull request to the merge queue Oct 19, 2026

Merged via the queue into kagent-dev:main with commit 04db728 Oct 19, 2026
33 checks passed

This was referenced Oct 25, 2026

[Automated] Draft docs (kagent): feat: add token passthrough for remote MCP servers kagent-dev/website#646

Merged

docs(1.x): Send files to Responses API models, and drop tokenExchange kagent-dev/website#648

Merged

tparis-provops mentioned this pull request Nov 1, 2026

[FEATURE] Harness runtimes (Claude Code, Codex) accept image parts in a user message #3467

Open

BrewTestBot mentioned this pull request Nov 9, 2026

kagent 1.0.0 Homebrew/homebrew-core#324443

Merged

Navigation Menu

Repository navigation

feat: Send PDF and Office files to Responses API models end-to-end - #3200

feat: Send PDF and Office files to Responses API models end-to-end#3200
EItanya merged 10 commits into
kagent-dev:mainfrom
danielle-mustillo:danielle-mustillo/pdf-office-files

danielle-mustillo commented Oct 14, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

danielle-mustillo commented Oct 15, 2026 •

edited

Loading

Uh oh!

Charlesthebird left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Charlesthebird left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

danielle-mustillo commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Resolution

User Experience

Changes

Tested

Uh oh!

This comment has been minimized.

danielle-mustillo commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before: the composer rejects PDF and .docx

After: the composer stages PDF and .docx

Before: the model asks for the files

After: the model reads each secret word

Before: the model asks to download the link

After: the model quotes the linked PDF

Uh oh!

Charlesthebird left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Charlesthebird left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

danielle-mustillo commented Oct 14, 2026 •

edited

Loading

danielle-mustillo commented Oct 15, 2026 •

edited

Loading