Image Intelligence Pipeline (Cloudinary Asset Describer)
Multimodal workflow that automates layout-aware tagging and context extraction for 200+ media assets at ultra-low cost. The same architecture maps directly onto Azure Computer Vision + Azure OpenAI Vision + Azure Blob Storage for enterprise deployments.
Project details
The Challenge: Breaking the Manual Tagging Bottleneck
A high-performance portfolio requires more than just images; it needs Design Intelligence. Features like smart cropping, dynamic color themes, and accessibility require precise metadata (focal points, dominant colors, and alt text). Manually populating these for hundreds of assets across different file types (Images, Videos, PDFs) was a significant operational hurdle.
The Architecture: Multimodal Schema Enforcement
I developed a specialized middleware that leverages a fast multimodal LLM's native vision capabilities (Gemini 3 Flash Preview was used in the prototype; the same flow runs on Azure OpenAI GPT-4o + Azure Computer Vision in the production-grade Microsoft architecture). Unlike generic captioning tools, this system enforces strict schema validation for 3 distinct asset categories:
Visual Strategy (Images)
Motion Analysis (Video)
Document Parsing (PDF)
The Efficiency Engine
To ensure maximum efficiency, the application includes a Deduplication Engine. By hashing asset metadata (bytes/dimensions), the tool identifies redundant files and maps existing descriptions to new instances, eliminating wasted API tokens while maintaining consistent cataloging.
Why a Fast Multimodal LLM?
For high-volume vision tasks like layout token extraction, a fast multimodal model provides the ideal balance of near-instant latency and sub-cent pricing, enabling a 'Zero-Cost' automation feel at enterprise scales. The prototype used Gemini 3 Flash Preview; the production-grade Microsoft equivalent is Azure OpenAI GPT-4o-mini with vision, which carries the same speed/cost profile under enterprise governance.
Multimodal LLM (vision)
Fast, cheap vision model for tagging. Prototype: Gemini 3 Flash. Production target: Azure OpenAI GPT-4o-mini with vision.
Cloudinary Admin API (Azure Blob Storage equivalent in production)
Used for searching assets and updating context metadata. The same workflow runs against Azure Blob Storage with metadata indexed in Azure AI Search.
Multimodal Schemas
Strict KV-pair extraction for structured JSON updates.
