Tag: LLMs

ACL 2026 presentation on Quantize What Counts: More for Keys, Less for Values, explaining key-value norm disparity, key-prioritized quantization, and practical KV-cache compression guidance.

Serving reasoning LLMs efficiently and reliably: lossless DFloat11 compression, KV-cache quantization, and Bayes@N evaluation and ranking under test-time scaling.

Serving reasoning LLMs efficiently and reliably: lossless DFloat11 compression, KV-cache quantization, and Bayes@N evaluation and ranking under test-time scaling.

A geometry-driven mixed-precision KV-cache quantization poster showing that keys carry more information than values, so key-favored bit allocation preserves accuracy while reducing memory.

SCIPE Workshop on Large Language Models • Final Presentation

Day 3 of the SCIPE workshop on large language models: attention under the hood and open research directions — why Pass@k is unstable, Bayes@N posteriors, rankings with uncertainty, and ranking reasoning LLMs under test-time scaling.

Day 2 of the SCIPE workshop on large language models: moving from chat to systems with tool use and function calling — schemas and validation, the agentic plan-act-observe loop, prompt injection as a systems problem, and retrieval-augmented generation.

Day 1 of the SCIPE workshop on large language models: what language modelling actually models, how LLMs get from tokens to meaning, the stochastic-parrot debate, agentic LLMs, and a roadmap of the field.

NeurIPS 2025 presentation on Dynamic-Length Float (DFloat11/DF11): a lossless format that Huffman-codes BFloat16 exponents down to ~11 bits, cutting model size ~30% with bit-for-bit identical outputs and a GPU kernel that makes compressed inference fast.

NeurIPS 2025 poster on DFloat11: a lossless compression framework that shrinks LLMs and diffusion transformers to ~70% of their size with bit-for-bit identical outputs, plus a GPU kernel that decompresses on the fly.

M.Sc. thesis defense on evaluating and ranking reasoning LLMs under a fixed test-time sampling budget, covering Pass@k's instability, Bayesian alternatives, and how many samples a reliable ranking actually needs.

BFloat16 uses 8 bits to store exponents, but those 8 bits carry only about 2.6 bits of actual information in trained neural networks. Regardless of the initialization and training recipe.

A principled Bayesian framework that replaces Pass@k with posterior estimates, credible intervals, and stable rankings for LLM evaluation

Explore how Item Response Theory (IRT) and other psychometric models can simulate and analyze LLM evaluation datasets. Learn how difficulty, discrimination, and guessing parameters reveal model reasoning patterns, with interactive examples across multiple reading levels.

Explore how simulating LLM responses to evaluation datasets with stochastic sampling is like flipping biased coins—revealing variability, bias, and the importance of multiple trials for reliable benchmarking.

A Bayesian framework for evaluating large language models that replaces unstable Pass@k metrics with posterior estimates and credible intervals. The method improves sample efficiency, supports graded outcomes, and enables statistically sound model comparisons.

Key-favored KV-cache quantization for LLMs: theory shows keys have larger norms and should get more bits; empirics show 4b-K/2b-V preserves up to 98.3% accuracy while cutting memory.

DFloat11 compresses LLMs to 70% of their original size while maintaining bit-for-bit identical outputs. A lossless compression framework with efficient GPU inference that enables running Llama 3.1 405B on a single node.

Reasoning in Large Language Models: chain-of-thought prompting (zero-shot CoT, re-reading, self-ask, least-to-most, self-consistency), retrieval-augmented reasoning, reasoning vs. reciting, bounded rationality, and interpretability.