Scalable Multimodal Video Search on AWS

🔑 Enhanced Key Takeaways

• •Amazon Nova Multimodal Embeddings model unifies text, documents, images, video, and audio into a single embedding space for agentic RAG and semantic search[4].
• •Nova 2 Omni processes text, images, video, and speech inputs while natively generating text and images, supporting tasks like video analysis and natural language image editing[2][4].
• •Video inputs to Nova models are resized to 672x672 pixels with dynamic sampling: 1 FPS up to 16 minutes (960 frames max for Lite/Pro) or 3,200 frames for Premier, via base64 (25 MB) or S3 URI (1 GB)[1].

🛠️ Technical Deep Dive

• •Amazon Nova video understanding supports multi-aspect ratio resizing to 672x672 square dimensions before model input[1].
• •Dynamic frame sampling: 1 FPS for videos ≤16 minutes (Lite/Pro), reducing rate for longer videos to cap at 960 frames; Premier caps at 3,200 frames[1].
• •Input methods: base64 (payload ≤25 MB) or S3 URI (≤1 GB), with no difference in token count[1].
• •Nova 2 Omni enables cross-modal reasoning, temporal understanding in videos, and combines video with speech for improved performance[3].
• •Nova Multimodal Embeddings powers semantic search across modalities in a unified vector space[4].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline