Scan

The Scan tab identifies whatever is in a photo and returns a structured result: name, category, confidence, description and a few fun facts. It is backed by the identify edge function.

Change the prompt or model

Change the identify prompt or model in this project. Update INSTRUCTION or identifyModel in supabase/functions/_utils/ai.config.json, then redeploy identify.

The prompt is a local constant at the top of supabase/functions/identify/index.ts:

supabase/functions/identify/index.ts


const INSTRUCTION =
  'Identify the single main subject of this image. Give its common name, a broad ' +
  'category (e.g. Animal, Plant, Food, Vehicle, Landmark, Everyday Object), your ' +
  'confidence as a number between 0 and 1, a one or two sentence description, and ' +
  '2 to 4 short fun facts.';

The model is not a local constant. IDENTIFY_MODEL is imported from _utils/ai.ts, which reads it from the same supabase/functions/_utils/ai.config.json that config.js uses. Change identifyModel there, pick a model whose input modalities include image, then redeploy:

supabase/functions/_utils/ai.config.json


"identifyModel": "google/gemini-3-flash-preview"


supabase functions deploy identify

Repurposing it

Repurpose the Scan feature in this project for a different subject. Change INSTRUCTION and IdentifySchema in the identify edge function together, and update ScanResultCard.tsx to match.

To turn Scan into something else, such as a plant-disease checker or a wine-label reader, change INSTRUCTION and IdentifySchema together, update the labels in src/components/scan/ScanResultCard.tsx, and redeploy. Existing rows keep their old shape, so clear the scans table in development or handle both shapes when rendering.

The result shape

The function asks for structured output and validates it with Zod, so the shape is guaranteed:

supabase/functions/identify/index.ts


const IdentifySchema = z.object({
  name: z.string(),
  category: z.string(),
  confidence: z.number().min(0).max(1),
  description: z.string(),
  funFacts: z.array(z.string()),
});

It is stored as jsonb in scans.result.

Gotchas

Categories are free text, not an enum. The prompt suggests Animal, Plant, Food, Vehicle, Landmark, Everyday Object, but a model can return anything, and the category chips filter on whatever comes back. For a stable set, tighten INSTRUCTION and change category to a Zod enum.

There is no live camera preview. Scan uses the system camera and picker through expo-image-picker, so there is no expo-camera dependency to maintain. The viewfinder graphic is the modal’s framing, not a feed.

Photos are resized to a 1024px long edge before upload, because vision models bill by image size.

Scans count toward the free monthly allowance, alongside chat messages. Out of allowance returns 402 quota_exceeded and the upgrade paywall. See Quota & Paywall.

Error codes

Code	HTTP	Meaning
`invalid_image`	400	The submitted URL isn’t a `chat-media` object.
`image_unavailable`	400	The object couldn’t be downloaded server-side.
`quota_exceeded`	402	Free allowance used up.
`provider_not_configured`	500	`OPENROUTER_API_KEY` isn’t set.

Files

Piece	Where
Screen	`src/app/(protected)/(tabs)/scan.tsx`
Components	`src/components/scan/`
Upload helper	`src/lib/storage/chatMedia.ts`
History hook	`src/hooks/scans.ts`
Function	`supabase/functions/identify/index.ts`
Table	`scans`

How it works

Structured output. The function tries generateObject first. If the provider cannot do structured output, it falls back to plain JSON-mode prompting and parses the response through the same schema. The response body carries a mode field of generateObject or json-fallback, so you can tell which path ran.

History. Owner-scoped by row-level security, so useScans() needs no user_id filter.