← Presentation home · Part 8 of 13
Where normal RAG quietly breaks
Editorial update: The original slide below overstates API and automation limitations. Open WebUI has APIs; the revised notes explain the actual pipeline checks and reasons to consider a separate proxy.
Where a RAG setup can quietly lose information
A useful answer depends on what survives the whole pipeline: text extraction, chunking, embedding, search, prompt assembly and generation. A successful HTTP response does not prove that every stage worked.
Check the embedding input
An embedding model has an input limit. If a chunk exceeds it, part of the text may be truncated. Compare the actual chunk length and embedding limit in the versions you run. Characters and tokens are different units.
Check the generation context
The question, instructions, chat history and retrieved passages share the model's context budget. Inspect the active context setting and assembled prompt rather than assuming a fixed default applies to every Ollama release or machine.
Built-in RAG can be automated
The earlier notes overstated Open WebUI's limitations. It has API endpoints and knowledge-base management features. Scripting a document workflow does not inherently require a separate proxy.
When a separate retrieval service helps
A proxy can be useful when several clients need the same retrieval rules, when you need to inspect exactly what is sent to the model, or when the existing application's extension points do not fit the workflow. It also adds another service to secure and maintain.
Test the result
- Use questions whose answers occur near the end of a chunk.
- Inspect the passages retrieved for each question.
- Confirm those passages reach the generation request intact.
- Check answers and citations against the original documents.
The original slide is a record of the talk. These notes correct its claims that built-in retrieval has no API or automation surface.