Teaching AI Something It Was Never Trained On

Deep dive

Teaching AI Something It Was Never Trained On

TL;DR

By the end of this tutorial, you'll watch a local AI answer detailed questions about a programming language that did not exist until I wrote it last week. Nothing has been trained or fine-tuned. It is all retrieval. No cloud. No API keys. No data leaving the room.

What this is about

This is a tutorial about RAG (Retrieval-Augmented Generation, a method that hands an AI relevant documents at question time instead of baking knowledge into the model through training). RAG is how you give an AI a document it has never seen and make it an instant expert.

Most RAG demos use documents the model has probably half-seen during training already (Wikipedia, popular manuals, big open datasets). That makes it impossible to tell whether the model is *reading the document* or just *remembering it*. The honest part of the demo gets lost.

So we removed all doubt. We built a real, working symbolic programming language from scratch. No English keywords. Every control structure is a symbol. No LLM has ever heard of it. If the model answers correctly, it answered from the documents we gave it. Full stop.

The three live tests

The whole session is built around three demonstrations. Hold this checklist in your head, everything else explains how these three moments are possible.

1. Ask a real question. Watch the AI cite the documentation and produce correct code.
2. Ask about a feature that does not exist yet. Watch it admit the gap instead of inventing.
3. Ask something totally off-topic. Watch it refuse cleanly.

Each one tests a different property of a healthy RAG system. The first tests retrieval quality. The second tests hallucination resistance. The third tests topical containment. Pass all three and the system is doing real work, not theater.

What you do not need to know going in

No prior AI knowledge required. You do not need to know how transformers work, what an embedding is, what fine-tuning is, what vector search is. We define each term the first time it appears. If you can use ChatGPT you can follow this.

What you will know coming out

Why fine-tuning is the wrong tool for injecting facts (and what the research says about that). What RAG is in three sentences. How RAG-as-a-feature differs from RAG-as-infrastructure. The exact local stack, Ollama, OpenWebUI, an embedding model, that you can run on your own hardware. And why a 730-line Perl script can outperform a Python tower of four cloud services.

Why this matters

This is not just an academic demo. This is the architecture I shipped at the University of Toronto Faculty of Dentistry, a real production AI tool serving real users, processing real policy questions, with PHI (Personal Health Information) rules that legally forbid sending data to a cloud. The "never trained on" framing is a teaching device. The architecture underneath is what lets you use AI in a regulated environment without leaking anything.

FAQ

Is this just ChatGPT with extra steps? No. ChatGPT is a cloud service. Everything here runs on one machine on local hardware. The model, the search, the documents, all of it.

Did you train a model for this? No, and that is the entire point. Training was the wrong tool. The next deep-dive explains why.

Will I be able to build this myself after? Yes, if you can install Docker and run a couple of terminal commands. The "10-minute setup" deep-dive (slide 07) is literally a copy-paste install.

If you only remember one thing

You are about to watch a model answer correctly about a thing it has never seen. The trick is not in the model. The trick is in what you give it to read.