Send the picture with the words
Send the picture beside the shopper’s words. Agentcard reads the two together and names the product the picture shows, then searches with the shopper’s words and that name in place of “that”: “where can I get the Patagonia Down Sweater Jacket”. The cards, the reply and the conversation’s history all read those words.- SDK
- MCP
- API
Pass the picture as
image to buy.turn:agentcard.ts
Send the picture as data
A picture your app holds in memory goes as base64 text, the picture’s bytes written out as letters and digits, with its type. Agentcard takesimage/jpeg, image/png, image/webp and image/gif, up to 3,000,000 bytes once decoded. A full-size phone photo can be larger, so scale the picture down before you send it. A picture sent by address needs an https address of at most 2,048 characters.
- SDK
- MCP
- API
agentcard.ts
Send a picture with no words
A shopper who only shows a picture still gets a search. Agentcard searches as if the shopper had written “find” and the product’s name, and the answer has the same shape: a reply and the products found.- SDK
- MCP
- API
buy.turn needs the shopper’s words, so give it a short ask with the picture:agentcard.ts
client.buy itself can send the picture with no ask at all:agentcard.ts
Handle an unclear picture
Agentcard searches for a product only when it is sure what the picture shows. A picture of a crowded shelf, or of several products when the words do not say which, leaves the search on the shopper’s words alone, so “where can I get that jacket” searches for jackets. A picture with no words that Agentcard cannot read is refused:Send it only with a new request
Send the picture with the shopper’s message that asks for something, and with nothing else. A pick of a card, a yes to a total and every later message go without it, so a shopper who taps a card never uploads the picture again. Agentcard reads the picture during the call and never keeps it.- SDK
- MCP
- API
buy.turn sends the picture with the turn’s first request for products and with nothing else, so your code does nothing more.Fix a refused picture
A picture Agentcard cannot take is refused with400, and the error names the field:
- SDK
- MCP
- API
buy.turn checks the picture’s address, its type and the size of its data before any call, so a picture those checks refuse throws a TypeError naming the field. buy.turn('that', { image: { url: 'http://frames.yourapp.com/a.jpg' } }) throws:https address instead:agentcard.ts
image.data is at most 3 MB decoded, so handle that refusal as well as the TypeError.