Descriptions:
Abed Matini, senior backend developer at Ogilvy, presents a framework-free hybrid RAG system designed to sidestep two common production pain points: the “multimodal tax” of uploading documents directly into an LLM’s context at query time (burning tokens before a single question is asked), and the complexity of managing too many integrated search tools in a production chatbot. The solution combines semantic vector search with SQL-based Reciprocal Rank Fusion (RRF), keeping everything in PostgreSQL without requiring a dedicated vector database.
The talk includes a live demo of an FAQ assistant chatbot built for an employee handbook, running entirely locally via Ollama with CPU-only inference — no GPU required. Matini walks through the full system stack: Python, FastAPI, React, PostgreSQL, Docker, and LangFuse for observability. A central focus is document chunking strategy selection — heading-based, paragraph-based, fixed 512-character windows with 64% overlap, and sentence-based — with concrete examples showing how each approach handles different document structures and where each breaks down.
The observability layer via LangFuse tracks chat latency, token usage, and model behavior, while the system also incorporates prompt injection detection patterns and a guardrail layer for risky queries. An admin dashboard lets operators upload documents in PDF, Word, PowerPoint, or image formats and select chunking strategies per document type. The full codebase is designed to run in GitHub Codespaces with a single command, making it readily reproducible for teams wanting to evaluate or extend the approach.
📺 Source: AI Engineer · Published June 28, 2026
🏷️ Format: Hands On Build
