Quick keys
— Navigate with ← → or spacebar.
F
fullscreen ·
S
speaker view ·
O
overview ·
Alt+Click
zoom
Navigation
×
# Day 2: Tool Use (Function Calling) + RAG <!-- .slide: data-background-gradient="linear-gradient(135deg, rgba(2,132,199,0.14) 0%, rgba(249,115,22,0.10) 45%, rgba(16,185,129,0.10) 100%)" --> <div style="margin-top: 1.8em; text-align: center; opacity: 0.9;"> SCIPE Workshop on Large Language Models </div> <div style="margin-top: 1.4em; display: flex; justify-content: center; gap: 1.2rem; flex-wrap: wrap;"> <div style="display: flex; align-items: center; gap: 0.55rem; padding: 0.6rem 0.85rem; border-radius: 999px; background: rgba(2,132,199,0.10); border: 1px solid rgba(2,132,199,0.22);"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" aria-hidden="true"> <path d="M14.7 6.1c.3.7.4 1.4.3 2.1l4.2 4.2c.6.6.6 1.5 0 2.1l-2.3 2.3c-.6.6-1.5.6-2.1 0L10.6 15c-.7.1-1.4 0-2.1-.3L6 17.1l1.2 1.2-1.4 1.4-3.6-3.6 1.4-1.4 1.2 1.2 2.4-2.4c-.7-1.9-.2-4.1 1.4-5.7 1.2-1.2 2.8-1.7 4.4-1.5l-2.5 2.5 1.8 1.8 2.4-2.5z" fill="rgba(2,132,199,0.95)"/> </svg> <div style="font-weight: 650; color: rgba(2,132,199,0.95);">Tool Use</div> </div> <div style="display: flex; align-items: center; gap: 0.55rem; padding: 0.6rem 0.85rem; border-radius: 999px; background: rgba(249,115,22,0.10); border: 1px solid rgba(249,115,22,0.24);"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" aria-hidden="true"> <path d="M10 4a6 6 0 1 0 3.5 10.9l4.3 4.3 1.4-1.4-4.3-4.3A6 6 0 0 0 10 4zm0 2a4 4 0 1 1 0 8 4 4 0 0 1 0-8z" fill="rgba(194,65,12,0.95)"/> <path d="M6 20h10" stroke="rgba(194,65,12,0.55)" stroke-width="2" stroke-linecap="round"/> </svg> <div style="font-weight: 650; color: rgba(194,65,12,0.95);">RAG</div> </div> </div> <div style="margin-top: 2.0em; opacity: 0.75; font-size: 0.85em; text-align: center;"> Case Western Reserve University • January 17, 2026 </div> <div style="text-align: center; margin-top: 2.2em;"> <div style="display: flex; align-items: center; justify-content: center; gap: 2.2rem; flex-wrap: wrap;"> <img src="/assets/slides/2026-01-16-state-of-llms/logo/cwru_logo.webp" alt="CWRU" style="height: 70px; width: auto; padding: 0.3em 0.4em;" /> <img src="/assets/slides/2026-01-16-state-of-llms/logo/uoc_logo.webp" alt="University of Cincinnati" style="height: 70px; width: auto; padding: 0.3em 0.4em;" /> <img src="/assets/slides/2026-01-16-state-of-llms/logo/osu_logo.webp" alt="Ohio State University" style="height: 70px; width: auto; padding: 0.3em 0.4em;" /> <img src="/assets/slides/2026-01-16-state-of-llms/logo/NSF_Official_logo_Low_Res_150ppi-300x300.webp" alt="NSF" style="height: 70px; width: auto; padding: 0.3em 0.4em;" /> </div> </div> --- <style> :root { --tool-accent: rgba(2,132,199,1); --tool-soft: rgba(2,132,199,0.10); --rag-accent: rgba(249,115,22,1); --rag-soft: rgba(249,115,22,0.10); --mix-accent: rgba(16,185,129,1); --mix-soft: rgba(16,185,129,0.10); --ink: rgba(15,23,42,1); --muted: rgba(15,23,42,0.72); --card-bg: rgba(255,255,255,0.72); --card-border: rgba(15,23,42,0.12); } .reveal .c-grid { display: grid; gap: 1.2rem; } .reveal .c-grid-2 { grid-template-columns: 1fr 1fr; } .reveal .c-grid-3 { grid-template-columns: 1fr 1fr 1fr; } .reveal .c-card { background: var(--card-bg); border: 1px solid var(--card-border); border-radius: 18px; padding: 1.15rem 1.25rem; box-shadow: 0 16px 34px rgba(15,23,42,0.07); } .reveal .c-kicker { font-size: 0.55em; letter-spacing: 0.14em; text-transform: uppercase; opacity: 0.7; margin-bottom: 0.4rem; } .reveal .c-muted { color: var(--muted); } .reveal .c-titleRow { display: flex; align-items: center; gap: 0.6rem; margin-bottom: 0.65rem; } .reveal .c-titleRow strong { font-weight: 700; } .reveal .c-pill { display: inline-flex; align-items: center; gap: 0.4rem; padding: 0.25rem 0.55rem; border-radius: 999px; font-size: 0.55em; font-weight: 700; border: 1px solid rgba(15,23,42,0.14); background: rgba(255,255,255,0.75); letter-spacing: 0.04em; text-transform: uppercase; } .reveal .c-pill.tool { border-color: rgba(2,132,199,0.24); background: var(--tool-soft); color: rgba(2,132,199,1); } .reveal .c-pill.rag { border-color: rgba(249,115,22,0.26); background: var(--rag-soft); color: rgba(194,65,12,1); } .reveal .c-pill.mix { border-color: rgba(16,185,129,0.26); background: var(--mix-soft); color: rgba(5,150,105,1); } .reveal .c-list { margin: 0.65rem 0 0 0; padding-left: 1.15rem; font-size: 0.82em; } .reveal .c-list li { margin: 0.28rem 0; } .reveal .c-callout { border-left: 6px solid rgba(2,132,199,0.55); background: rgba(2,132,199,0.06); padding: 0.9rem 1.0rem; border-radius: 16px; font-size: 0.82em; } .reveal .c-callout.warn { border-left-color: rgba(249,115,22,0.70); background: rgba(249,115,22,0.08); } .reveal .c-callout.ok { border-left-color: rgba(16,185,129,0.70); background: rgba(16,185,129,0.08); } .reveal .c-flow { display: flex; align-items: stretch; justify-content: center; gap: 0.65rem; flex-wrap: wrap; margin-top: 1.0rem; } .reveal .c-node { min-width: 240px; padding: 0.85rem 0.95rem; border-radius: 16px; border: 1px solid rgba(15,23,42,0.12); background: rgba(255,255,255,0.72); text-align: center; font-size: 0.78em; } .reveal .c-node .small { display: block; margin-top: 0.35rem; font-size: 0.78em; opacity: 0.75; } .reveal .c-arrow { display: flex; align-items: center; font-size: 1.2em; opacity: 0.4; padding: 0 0.15rem; } .reveal .c-pre { margin: 0.5rem 0 0 0; padding: 0.85rem 0.9rem; border-radius: 16px; background: rgba(15,23,42,0.05); border: 1px solid rgba(15,23,42,0.10); font-size: 0.68em; overflow: hidden; } .reveal .c-pre code { white-space: pre; } .reveal .c-svg { width: 100%; height: auto; display: block; margin: 0 auto; } .reveal .c-foot { font-size: 0.55em; opacity: 0.75; margin-top: 0.85rem; } @media (max-width: 1100px) { .reveal .c-grid-3 { grid-template-columns: 1fr; } .reveal .c-grid-2 { grid-template-columns: 1fr; } } </style> ## Today: from “chat” to “systems” <div class="c-grid c-grid-3" style="margin-top: 1.1rem;"> <div class="c-card"> <div class="c-kicker">Part 1</div> <div class="c-titleRow"> <span class="c-pill tool">Tools</span> <strong>Function calling</strong> </div> <div class="c-muted" style="font-size: 0.78em;">Turn natural language into structured, executable actions.</div> <ul class="c-list"> <li>Tool schemas and validation</li> <li>Core tool-call loop</li> <li>Reliability patterns</li> <li>Security / trust boundaries</li> </ul> </div> <div class="c-card"> <div class="c-kicker">Part 2</div> <div class="c-titleRow"> <span class="c-pill rag">RAG</span> <strong>Retrieval-augmented generation</strong> </div> <div class="c-muted" style="font-size: 0.78em;">Ground the model in your data, with citations.</div> <ul class="c-list"> <li>Index-time vs query-time</li> <li>Vector search intuition</li> <li>Chunking, reranking, packing</li> <li>Failure modes & evaluation</li> </ul> </div> <div class="c-card"> <div class="c-kicker">Part 3</div> <div class="c-titleRow"> <span class="c-pill mix">Combine</span> <strong>Agentic / tool-augmented RAG</strong> </div> <div class="c-muted" style="font-size: 0.78em;">Retrieval is “just another tool” — but needs guardrails.</div> <ul class="c-list"> <li>Query rewriting tool</li> <li>Retrieval + rerank tool</li> <li>Grounded answer formatting</li> <li>Observability and caching</li> </ul> </div> </div> <div class="c-callout ok" style="margin-top: 1.2rem;"> <strong>Goal:</strong> reliable answers grounded in data — plus actions that are auditable. </div> --- ## LLM-only vs LLM-as-a-component <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: start;"> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill" style="border-color: rgba(15,23,42,0.16); background: rgba(15,23,42,0.05); color: rgba(15,23,42,0.75);">Model only</span> <strong>Text in → text out</strong> </div> <ul class="c-list"> <li>Can be wrong confidently (hallucinations)</li> <li>Can’t see private or fresh data</li> <li>Can’t take actions or verify outcomes</li> <li>Hard to audit “why this answer?”</li> </ul> <div class="c-callout warn" style="margin-top: 0.95rem;"> <strong>Good for:</strong> brainstorming, drafting, summarizing what’s already provided. </div> </div> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill mix">System</span> <strong>LLM + tools + retrieval</strong> </div> <div class="c-flow" style="margin-top: 0.4rem;"> <div class="c-node"> User request <span class="small">natural language</span> </div> <div class="c-arrow">→</div> <div class="c-node"> LLM <span class="small">decide + call</span> </div> <div class="c-arrow">→</div> <div class="c-node"> Tools / RAG <span class="small">data + actions</span> </div> <div class="c-arrow">→</div> <div class="c-node"> Grounded answer <span class="small">with evidence</span> </div> </div> <div class="c-callout ok" style="margin-top: 0.95rem;"> <strong>Good for:</strong> production apps (search, analytics, workflows, assistants). </div> </div> </div> --- # Tool Use (Function Calling) <!-- .slide: data-background-gradient="linear-gradient(135deg, rgba(2,132,199,0.16) 0%, rgba(255,255,255,0) 60%)" --> <div class="c-callout" style="max-width: 58rem; margin: 1.2rem auto;"> the model emits <strong>structured arguments</strong> for a tool you own, your runtime validates them, executes, and returns <strong>structured results</strong>. </div> --- ## Why tools? <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: start;"> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill tool">Reason</span> <strong>LLMs don’t have guaranteed access to truth</strong> </div> <ul class="c-list"> <li><strong>Freshness:</strong> news, prices, inventory, policies change</li> <li><strong>Private data:</strong> your docs, tickets, logs, databases</li> <li><strong>Determinism:</strong> math, code execution, ETL, search</li> <li><strong>Actions:</strong> create a ticket, send email, run workflow</li> </ul> </div> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill tool">Pattern</span> <strong>Turn “intent” into “API calls”</strong> </div> <div class="c-flow" style="margin-top: 0.2rem;"> <div class="c-node"> Intent <span class="small">“Book a room”</span> </div> <div class="c-arrow">→</div> <div class="c-node" style="border-color: rgba(2,132,199,0.22); background: rgba(2,132,199,0.06);"> Tool call <span class="small"><code>create_reservation(...)</code></span> </div> <div class="c-arrow">→</div> <div class="c-node"> Result <span class="small">confirmation + details</span> </div> </div> <div class="c-foot"> Reliability comes from <strong>schemas</strong>, <strong>validation</strong>, and <strong>guardrails</strong>, not from “asking nicely”. </div> </div> </div> --- ## The contract: schemas + validation <div class="c-grid c-grid-2" style="margin-top: 1.0rem; align-items: start;"> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill tool">Tool</span> <strong>Definition (what the model can call)</strong> </div> <pre class="c-pre" style="font-size: 0.58em;"><code class="language-json">{ "name": "search_docs", "description": "Search internal docs and return relevant chunks.", "parameters": { "type": "object", "properties": { "query": { "type": "string" }, "top_k": { "type": "integer", "minimum": 1, "maximum": 10 } }, "required": ["query"] } }</code></pre> <div class="c-foot"> Runtime responsibilities: <strong>validate types</strong>, enforce limits, handle auth/timeouts, log calls. </div> </div> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill tool">Call</span> <strong>Model emits structured arguments</strong> </div> <pre class="c-pre" style="font-size: 0.58em;"><code class="language-json">{ "tool": "search_docs", "arguments": { "query": "What is retrieval-augmented generation?", "top_k": 5 } }</code></pre> <div class="c-callout ok" style="margin-top: 0.9rem;"> <strong>Key idea:</strong> the model “writes JSON” — your system decides what gets executed. </div> </div> </div> --- ## Core loop (tool calling) <pre class="c-pre" style="margin-top: 1.0rem;"><code class="language-mermaid">%%{init: { 'theme': 'default', 'themeVariables': { 'fontSize': '26px' }, 'sequence': { 'messageFontSize': 24, 'noteFontSize': 22 }, 'config': { 'scale': 2 } }}%% sequenceDiagram participant U as User participant M as LLM participant R as Runtime participant T as Tool/API U->>M: Question + constraints M->>R: tool_call(name, args) R->>T: Execute tool(args) T-->>R: tool_result(data) R-->>M: tool_result(data) M-->>U: Final answer (grounded)</code></pre> <div class="c-foot"> The runtime is where <strong>permissions</strong>, <strong>validation</strong>, and <strong>auditing</strong> live. </div> --- ## Agentic loop (plan → act → observe) <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: center;"> <div class="c-card"> <div class="c-titleRow"> <span class="c-pill tool">Loop</span> <strong>Multiple tool calls for harder tasks</strong> </div> <ul class="c-list"> <li>decompose into steps</li> <li>call tools for each step</li> <li>update state (memory, scratchpad, citations)</li> <li>stop when success criteria met</li> </ul> <div class="c-callout warn" style="margin-top: 0.95rem;"> <strong>Risk:</strong> more steps → more opportunities for mistakes and injection. </div> </div> <div class="c-card" style="text-align: center;"> <img src="/assets/slides/2026-01-17-tool-use-rag/plan-act-observe-loop.webp" alt="Plan act observe loop diagram" class="c-svg" style="max-width: 100%; height: auto;" /> </div> </div> --- ## Reliability patterns (what makes tools work in production) <div class="c-grid c-grid-3" style="margin-top: 1.1rem;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Validate</span><strong>Strict schemas</strong></div> <div class="c-muted" style="font-size: 0.78em;">Reject / repair malformed args. Enforce bounds.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Retry</span><strong>Deterministic fallbacks</strong></div> <div class="c-muted" style="font-size: 0.78em;">Timeouts, retries, alternate tools, safe defaults.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Idempotent</span><strong>Safe side-effects</strong></div> <div class="c-muted" style="font-size: 0.78em;">Use request IDs; don’t double-charge / double-create.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Cache</span><strong>Memoize results</strong></div> <div class="c-muted" style="font-size: 0.78em;">Speed + cost control; cache by (tool, args, user).</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Trace</span><strong>Logs + spans</strong></div> <div class="c-muted" style="font-size: 0.78em;">Know which tool calls created which answers.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Guard</span><strong>Least privilege</strong></div> <div class="c-muted" style="font-size: 0.78em;">Allow-list tools; per-user auth; redact secrets.</div> </div> </div> --- ## Security: trust boundaries (prompt injection is a systems problem) <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: start;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Threat</span><strong>Untrusted inputs everywhere</strong></div> <ul class="c-list"> <li>user prompt can try to override system goals</li> <li>tool outputs can contain adversarial text (“ignore prior instructions…”)</li> <li>retrieved documents can smuggle instructions (RAG injection)</li> </ul> <div class="c-callout warn" style="margin-top: 0.95rem;"> <strong>Rule:</strong> treat tool output and retrieved text as <strong>data</strong>, not instructions. </div> </div> <div class="c-card" style="text-align: center;"> <img src="/assets/slides/2026-01-17-tool-use-rag/trust-boundary.webp" alt="Trust boundary diagram" class="c-svg" style="max-width: 100%; height: auto;" /> </div> </div> --- # Retrieval-Augmented Generation (RAG) <!-- .slide: data-background-gradient="linear-gradient(135deg, rgba(249,115,22,0.16) 0%, rgba(255,255,255,0) 62%)" --> <div class="c-callout" style="border-left-color: rgba(249,115,22,0.75); background: rgba(249,115,22,0.07); max-width: 58rem; margin: 1.2rem auto;"> retrieve the most relevant chunks from your corpus and feed them to the model as context. </div> --- ## RAG: Big Picture <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: start;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Index time</span><strong>Build the knowledge base</strong></div> <ul class="c-list"> <li>collect documents (PDF, HTML, wiki, tickets)</li> <li>chunk (size + overlap) and add metadata</li> <li>embed chunks into vectors</li> <li>store in a vector index</li> </ul> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Query time</span><strong>Answer with evidence</strong></div> <ul class="c-list"> <li>embed the user query</li> <li>retrieve top-k chunks (vector / hybrid)</li> <li>rerank + pack into context window</li> <li>generate an answer with citations</li> </ul> </div> </div> <div class="c-foot"> RAG is a <strong>data pipeline</strong> and a <strong>prompting pattern</strong> — not a single library call. </div> --- ## RAG pipeline (end-to-end) <pre class="c-pre" style="margin-top: 1.0rem;"><code class="language-mermaid">%%{init: { 'theme': 'default', 'themeVariables': { 'fontSize': '24px' }, 'flowchart': { 'nodeSpacing': 50, 'rankSpacing': 55 }, 'config': { 'scale': 2 } }}%% flowchart LR subgraph I[Index time] D[Docs] --> C[Chunk + metadata] C --> E[Embed] E --> V[(Vector index)] end subgraph Q[Query time] U[User question] --> QE[Query embed] QE --> R[Retrieve top-k] V --> R R --> RR[Rerank / filter] RR --> P[Pack context] P --> L[LLM] L --> A[Answer + citations] end</code></pre> --- ## Vector search intuition <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: center;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Idea</span><strong>Nearest neighbors in embedding space</strong></div> <ul class="c-list"> <li>each chunk becomes a vector</li> <li>query becomes a vector</li> <li>retrieve chunks closest to the query vector</li> </ul> <div class="c-callout ok" style="margin-top: 0.95rem;"> mix semantic search with keyword filters (hybrid) and reranking. </div> </div> <div class="c-card" style="text-align: center;"> <img src="/assets/slides/2026-01-17-tool-use-rag/vector-search.webp" alt="Embedding space scatterplot with query and retrieved neighbors" class="c-svg" style="max-width: 100%; height: auto;" /> </div> </div> --- ## Context packing (why “just add more docs” fails) <div class="c-grid c-grid-2" style="margin-top: 1.1rem; align-items: start;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Constraint</span><strong>Context window is finite</strong></div> <ul class="c-list"> <li>too much context → relevant pieces get diluted</li> <li>too little context → missing evidence</li> <li>ordering matters (high-signal first)</li> </ul> <div class="c-callout warn" style="margin-top: 0.95rem;"> <strong>Practical rule:</strong> retrieve, filter, rerank, then pack — don’t “dump”. </div> </div> <div class="c-card"> <div style="font-size: 0.75em; opacity: 0.75; margin-bottom: 0.5rem;">Context window (tokens)</div> <div style="display: flex; height: 46px; border-radius: 14px; overflow: hidden; border: 1px solid rgba(15,23,42,0.12); background: rgba(255,255,255,0.65);"> <div style="flex: 3; background: rgba(249,115,22,0.22); border-right: 1px solid rgba(15,23,42,0.10); display: flex; align-items: center; justify-content: center; font-size: 0.68em; font-weight: 700; color: rgba(194,65,12,0.95);">Top chunks</div> <div style="flex: 2; background: rgba(2,132,199,0.12); border-right: 1px solid rgba(15,23,42,0.10); display: flex; align-items: center; justify-content: center; font-size: 0.68em; font-weight: 700; color: rgba(2,132,199,0.95);">Definitions</div> <div style="flex: 2; background: rgba(16,185,129,0.12); border-right: 1px solid rgba(15,23,42,0.10); display: flex; align-items: center; justify-content: center; font-size: 0.68em; font-weight: 700; color: rgba(5,150,105,0.95);">Constraints</div> <div style="flex: 1; background: rgba(15,23,42,0.06); display: flex; align-items: center; justify-content: center; font-size: 0.68em; font-weight: 700; color: rgba(15,23,42,0.70);">…</div> </div> <div class="c-foot"> Good packers use <strong>dedup</strong>, <strong>MMR</strong>, and <strong>metadata filters</strong> (recency, source). </div> </div> </div> --- ## Failure modes (and the fix is rarely “bigger model”) <div class="c-grid c-grid-2" style="margin-top: 1.1rem;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Miss</span><strong>No relevant chunks retrieved</strong></div> <div class="c-muted" style="font-size: 0.78em;">Fix: better chunking, query rewrite, hybrid retrieval, metadata filters.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Buried</span><strong>Relevant chunk is low-ranked</strong></div> <div class="c-muted" style="font-size: 0.78em;">Fix: reranking (cross-encoder), MMR, per-section retrieval.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Drift</span><strong>Answer ignores evidence</strong></div> <div class="c-muted" style="font-size: 0.78em;">Fix: stronger grounding prompt, citations required, constrained format.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">Injection</span><strong>Retrieved text attacks the model</strong></div> <div class="c-muted" style="font-size: 0.78em;">Fix: isolate retrieved content, strip instructions, run safety filters.</div> </div> </div> <!-- --- ## Evaluation checklist <div class="c-card" style="margin-top: 1.1rem;"> <div class="c-titleRow"><span class="c-pill mix">Measure</span><strong>Retrieval + generation + cost</strong></div> <div style="display: grid; grid-template-columns: 1fr 1fr; gap: 1.0rem; margin-top: 0.8rem; font-size: 0.78em;"> <div style="padding: 0.85rem 0.95rem; border-radius: 16px; border: 1px solid rgba(15,23,42,0.10); background: rgba(255,255,255,0.65);"> <div style="font-weight: 800; margin-bottom: 0.4rem;">Retrieval quality</div> <div class="c-muted">Recall@k, MRR, nDCG, source coverage, filter accuracy</div> </div> <div style="padding: 0.85rem 0.95rem; border-radius: 16px; border: 1px solid rgba(15,23,42,0.10); background: rgba(255,255,255,0.65);"> <div style="font-weight: 800; margin-bottom: 0.4rem;">Answer quality</div> <div class="c-muted">Correctness, faithfulness, citation correctness, refusal when unknown</div> </div> <div style="padding: 0.85rem 0.95rem; border-radius: 16px; border: 1px solid rgba(15,23,42,0.10); background: rgba(255,255,255,0.65);"> <div style="font-weight: 800; margin-bottom: 0.4rem;">Latency & cost</div> <div class="c-muted">p50/p95 latency, tokens, tool costs, cache hit rate</div> </div> <div style="padding: 0.85rem 0.95rem; border-radius: 16px; border: 1px solid rgba(15,23,42,0.10); background: rgba(255,255,255,0.65);"> <div style="font-weight: 800; margin-bottom: 0.4rem;">Safety</div> <div class="c-muted">Injection robustness, data leakage, policy adherence, least privilege</div> </div> </div> </div> --> --- # Tool‑Augmented RAG <!-- .slide: data-background-gradient="linear-gradient(135deg, rgba(16,185,129,0.16) 0%, rgba(255,255,255,0) 65%)" --> <div class="c-callout ok" style="max-width: 58rem; margin: 1.2rem auto;"> <strong>Idea:</strong> treat retrieval as one tool among many — and let the agent decide when to retrieve, rewrite, verify, and cite. </div> --- ## Agentic RAG (one practical architecture) <pre class="c-pre" style="margin-top: 1.0rem;"><code class="language-mermaid">%%{init: { 'theme': 'default', 'themeVariables': { 'fontSize': '22px' }, 'flowchart': { 'nodeSpacing': 40, 'rankSpacing': 50 }, 'config': { 'scale': 2 } }}%% flowchart TD U[User question] --> M[LLM] M -->|rewrite| Q[Query rewrite tool] Q --> R[Retriever tool] R --> RR[Reranker / filters] RR --> P[Context packer] P --> M M -->|optional| V[Verifier tool] V --> M M --> A[Answer + citations]</code></pre> <div class="c-foot"> The win is <strong>selective context</strong> + <strong>audited steps</strong> — the risk is uncontrolled loops. </div> --- ## Key takeaways <div class="c-grid c-grid-2" style="margin-top: 1.1rem;"> <div class="c-card"> <div class="c-titleRow"><span class="c-pill tool">Tools</span><strong>Reliability comes from engineering</strong></div> <div class="c-muted" style="font-size: 0.80em;">Schemas, validation, auth, retries, tracing, and least privilege.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill rag">RAG</span><strong>Grounding is a pipeline</strong></div> <div class="c-muted" style="font-size: 0.80em;">Chunking + retrieval + reranking + packing + prompting + evaluation.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill mix">Combine</span><strong>Retrieval is a tool call</strong></div> <div class="c-muted" style="font-size: 0.80em;">Agentic RAG is powerful when guarded by runtime policy.</div> </div> <div class="c-card"> <div class="c-titleRow"><span class="c-pill mix">Next</span><strong>Hands-on</strong></div> <div class="c-muted" style="font-size: 0.80em;">Build a small app</div> </div> </div> --- # Thank You! <div style="display: flex; flex-direction: column; align-items: center; justify-content: center; margin-top: 4rem;"> <h3 style="margin-bottom: 2rem; font-size: 2em;">Questions?</h3> <div style="text-align: center;"> <img src="/assets/slides/2026-01-16-state-of-llms/SCIPE_WORKSHOP.webp" alt="SCIPE Workshop QR Code" style="width: 27rem; max-width: 100%; height: auto; margin: 2rem auto;" /> </div> <div style="font-size: 1.5em; font-weight: bold; text-align: center; margin-top: 2rem; opacity: 0.95;"> Now let's start coding! </div> </div> </div>