> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agentcard.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Shop from a picture the shopper shows

> Send the picture a shopper points at with their message, and Agentcard searches for the product it shows.

Send the picture your shopper shows with their message, and Agentcard searches for the product in it. A viewer who points their phone at a jacket on screen and says "where can I get that jacket" gets cards for that jacket, not for every jacket.

```ts agentcard.ts theme={null}
const turn = buy.turn('where can I get that jacket', {
  image: { url: 'https://frames.yourapp.com/show/s01e04/00-12-31.jpg' },
});
```

Agentcard reads the picture and the shopper's words together and names the product the picture shows. The search then runs on the shopper's words with that product in place of "that": "where can I get the Patagonia Down Sweater Jacket". The cards, the reply and the conversation's history all read those words.

## Send the picture as data

Send a picture your app holds in memory as base64 data, with its type:

```ts agentcard.ts theme={null}
const turn = buy.turn('how much is this', {
  image: { data: frame.toString('base64'), media_type: 'image/jpeg' },
});
```

Agentcard takes `image/jpeg`, `image/png`, `image/webp` and `image/gif`, up to 3 MB once decoded. A full-resolution phone photo can be larger, so scale the picture down before you send it.

## Send a picture with no words

A shopper who only shows a picture still gets a search. `buy.turn` needs the shopper's words, so give it a short ask with the picture:

```ts agentcard.ts theme={null}
const turn = buy.turn('find this', {
  image: { url: 'https://frames.yourapp.com/show/s01e04/00-12-31.jpg' },
});
```

An app that calls `client.buy` itself sends the picture with no `ask` at all, and Agentcard searches for the product the picture shows, as if the shopper had written "find" and its name:

```ts agentcard.ts theme={null}
const turn = await client.buy({
  image: { url: 'https://frames.yourapp.com/show/s01e04/00-12-31.jpg' },
  external_user_id: 'user-42',
  resume: true,
});
```

## Handle an unclear picture

Agentcard searches for a product only when it is sure what the picture shows. A picture of a crowded shelf, or of several products when the words do not say which, leaves the search on the shopper's words alone, so "where can I get that jacket" searches for jackets. A picture with no words that Agentcard cannot read is refused, and the shopper's next message with words searches as usual.

## Check what the turn sends

The picture goes with the turn's first request for products, and with nothing else. A pick of a card, a yes to a total and every later message send no picture, so a shopper who taps a card never uploads the frame again. Agentcard reads the picture during the turn and never keeps it.

An app that calls `client.buy` itself sends the picture as the `image` field of the request. A request that sends `image` with `confirm` is refused.

## Fix a refused picture

`buy.turn` checks the picture's address, its type and the size of its data before any call, so a picture those checks refuse throws a TypeError naming the field. `buy.turn('that', { image: { url: 'http://frames.yourapp.com/a.jpg' } })` throws:

```text theme={null}
TypeError: Pass image.url as an https URL of at most 2048 characters.
```

Send the picture from an `https` address instead:

```ts agentcard.ts theme={null}
const turn = buy.turn('that', { image: { url: 'https://frames.yourapp.com/a.jpg' } });
```

Those checks are the SDK's own, and `POST /buy` refuses a picture the SDK passes: the request takes 3,000,000 decoded bytes, where `buy.turn` takes 3,145,728, so a picture between the two throws nothing and the call answers with a `400`. Handle the request's refusal as well as the TypeError.

A request your app sends itself gets a `400` that names the field:

```json theme={null}
{
  "error": "image.data is at most 3 MB decoded",
  "code": "invalid_request",
  "hint": "POST /buy { \"ask\": \"where can I get that jacket\", \"image\": { \"url\": \"https://...\" } }"
}
```

| Error | What to do |
| - | - |
| `image.url must be an https URL` | Send the picture from an `https` address, or as data. |
| `image takes exactly one of image.url or image.data` | Send the address or the data, not both. |
| `image.data is at most 3 MB decoded` | Scale the picture down and send it again. |
| `image.media_type must be one of image/jpeg, image/png, image/webp, image/gif` | Convert the picture to one of those types. |
| `image goes with an ask, never with confirm` | Send the picture with the shopper's message, and confirm without it. |
| `image.embedding: embeddings are not accepted yet; send image.url or image.data` | Send the picture itself instead of a vector your app computed. |
| `ask is required: Agentcard could not tell what the picture shows, so send the shopper's words with it` | Send the picture again with the shopper's words. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.