An e-commerce platform is building a visual product search system. Users upload photos of items they want to find (e.g., a photo of a lamp they saw at a friend's house). Claude must identify: (1) item category, (2) style attributes (mid-century, industrial, etc.), (3) approximate price range, and (4) 3-5 specific product recommendations from the platform's catalog. The catalog has 2 million SKUs. The team debates whether vision processing should happen in Claude or in a dedicated computer vision pipeline before Claude. Which architecture is correct?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — use Claude vision to interpret category, style, and price, then hand those structured attributes to embedding-based product retrieval for the actual catalog search. That keeps semantic understanding where it's strong and keeps 2 million SKU matching in a purpose-built search system.
Full Explanation
The correct architecture is the one that uses Claude vision for category, style attributes, and approximate price range, then passes the structured output to an embedding-based product retrieval system for specific recommendations. This implements the correct separation of concerns: Claude's vision capabilities are genuinely superior for semantic interpretation tasks because they require understanding context, cultural style references, and visual nuances that classification-only computer vision pipelines struggle with. Searching 2 million SKUs is not a Claude task; it requires an embedding-based vector search system operating at catalog scale. Claude correctly outputs structured attributes, which then parameterize a retrieval query against the product catalog. The architecture that uses Claude's native vision for all four tasks in a single call with a RAG-retrieved shortlist injected into context is wrong because RAG-retrieving a shortlist from a 2 million SKU catalog adds retrieval complexity and still can't surface the right products without the structured interpretation step first. The architecture that uses a dedicated computer vision pipeline to process the image into a structured attribute vector, then uses Claude text-only to interpret attributes and generate recommendations, is wrong because it discards Claude's superior style interpretation by reducing the image to a structured vector first, losing the semantic richness that makes the product search accurate. The architecture that uses Claude vision with the full 2 million SKU catalog chunked into multiple calls is wrong because it's economically and technically infeasible.