Test-Time Scaling Under Budget (Slides)

# Test-Time Scaling Under Budget

<div style="text-align: left; font-size: 0.75em;">
<div style="margin-bottom: 1.5em;">
<strong>Candidate:</strong> Mohsen Hariri<br>
<span style="opacity: 0.7; font-size: 0.9em;">M.Sc. in Computer Science</span>
</div>

<div style="margin-bottom: 1.5em;">
<strong>Advisor:</strong> Prof. Vipin Chaudhary
</div>

<div style="opacity: 0.8; font-size: 0.9em;">
Case Western Reserve University<br>
<span style="opacity: 0.7;">November 21, 2025</span>
</div>
</div>

</div>

---

<!-- ## About This Slide Deck

### Previous Solutions & Limitations

| Solution | Limitations |
|:---|:---|
| **Microsoft PowerPoint** | Difficult to learn, no customization, no LLM-integration |
| **Google Slides** | Harder to customize, no LLM-integration |
| **LaTeX Beamer** | Complex packages, portability issues, no LLM-integration |
| **Reveal.js** | Portable (HTML), but interactive usage is limited, no LLM-integration |

</div>

### Building a New System

- **Markdown** (based on Reveal.js)
- **LaTeX** (math support)
- **Mermaid** (diagrams)
- **HTML and CSS** customization
- **LLM-based** content generation
- **Interactive and fast development** (live reload, hotkeys, etc)

</div>

--- -->

## Research Contributions: Compression & Test-Time Scaling (2025)

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Compression & Quantization</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>Quantize What Counts: More for Keys, Less for Values</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Mohsen Hariri, Alan Luo, Weicong Chen, Shaochen Zhong, Tianyi Zhang, Qifan Wang, Xia Hu, Xiaotian Han, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Under Review: ACL (October Cycle)</div>
  </li>
  <li style="margin-bottom: 1rem; opacity: 0.3;">
    <strong>Precision Where It Counts: Norm-Aware KVQuant</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Mohsen Hariri, Weicong Chen, Sreehari Sankar, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress • Target: ACL (January Cycle)</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>70% Size, 100% Accuracy: Lossless LLM Compression (DFloat11)</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Tianyi Zhang, Mohsen Hariri, Shaochen Zhong, Vipin Chaudhary, Yang Sui, Xia Hu, Anshumali Shrivastava</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">NeurIPS 2025 • Collaboration with Rice University</div>
  </li>
  <li style="margin-bottom: 1rem; opacity: 0.3;">
    <strong>Future Directions</strong>
    <div style="opacity: 0.7; margin-left: 1rem;">DF11 in KV cache</div>
  </li>
</ul>
</div>

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Reasoning LLMs & Test-Time Scaling</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>Don't Pass@k: A Bayesian Framework for LLM Evaluation</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Mohsen Hariri, Amirhossein Samandar, Michael Hinczewski, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Under Review: ICLR • Collaboration with Physics Department</div>
  </li>
  <li style="margin-bottom: 1rem; opacity: 0.3;">
    <strong>Ranking without Averaging: A Bradley-Terry-Bayesian Framework</strong>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress</div>
  </li>
  <li style="margin-bottom: 1rem; opacity: 0.3;">
    <strong>Practices for Evaluating Reasoning Models</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Reassessing Reasoning Evaluation in LLMs: Metrics, Ranking, and Reproducibility</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress</div>
  </li>
  <li style="margin-bottom: 1rem; opacity: 0.3;">
    <strong>Next Steps</strong>
    <div style="opacity: 0.7; margin-left: 1rem;">Rubric-based evaluation (extended categorical evaluation)</div>
    <div style="opacity: 0.7; margin-left: 1rem;">Prior (role of post-training and compression in LM evaluation)</div>
  </li>
</ul>
</div>

</div>

---

## Research Contributions: Applications & Biomedical (2024-25)

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Applications</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>LoRATK: LoRA Once, Backdoor Everywhere</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">H. Liu, S. Zhong, X. Sun, M. Tian, M. Hariri, Z. Liu, R. Tang, Z. Jiang, J. Yuan, Y. Chuang, L. Li, S. Choi, R. Chen, V. Chaudhary, X. Hu</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">EMNLP 2025 • Collaboration with Rice University</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>K4: Online Log Anomaly Detection Via Unsupervised Typicality Learning</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Weicong Chen, Vikash Singh, Zahra Rahmani, Debargha Ganguly, Mohsen Hariri, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">IEEE HiPC 2025</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Masked-speech recognition using human and synthetic cloned speech</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Lauren Calandruccio, Mohsen Hariri, Emily Buss, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Trends in Hearing 2025 • Collaboration with Psychological Sciences Department</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Sweeping Promptable Spoofs under the DirtyRAG</strong>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress • Collaboration with Rice University</div>
  </li>
</ul>
</div>

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Biomedical</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>Novel adaptation of video segmentation to 3D MRI: efficient zero-shot knee segmentation with SAM2</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Andrew Seohwan Yu, Mohsen Hariri, Xuecen Zhang, Mingrui Yang, Vipin Chaudhary, Xiaojuan Li</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">SPIE Medical Imaging 2024</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Radiomics to detect inflammation and fibrosis in Crohn’s disease</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">P. Chirra, J. Sleiman, N. Gandhi, I. Gordon, M. Hariri, M. Baker, R. Ottichilo, D. Bruining, J. Kurowski, S. Viswanath, F. Rieder</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Journal of Crohn's and Colitis, 2025</div>
  </li>
</ul>
</div>

</div>

---

## Table of Contents

<div style="background: rgba(255, 255, 255, 0.05); padding: 1.5rem; border-radius: 12px; text-align: left; border-left: 4px solid var(--r-main-color);">
<h3 style="margin: 0 0 0.5rem 0; font-size: 1.1em; color: var(--r-heading-color);">1. Evaluation of LLM Reasoning</h3>
<div style="font-size: 0.75em; opacity: 0.8;">A Bayesian Approach to Ranking & Uncertainty</div>
</div>

<div style="background: rgba(255, 255, 255, 0.05); padding: 1.5rem; border-radius: 12px; text-align: left; border-left: 4px solid var(--r-main-color);">
<h3 style="margin: 0 0 0.5rem 0; font-size: 1.1em; color: var(--r-heading-color);">2. Numerical Data Type & Entropy</h3>
<div style="font-size: 0.75em; opacity: 0.8;">Lossless Compression of LLMs</div>
</div>

<div style="background: rgba(255, 255, 255, 0.05); padding: 1.5rem; border-radius: 12px; text-align: left; border-left: 4px solid var(--r-main-color);">
<h3 style="margin: 0 0 0.5rem 0; font-size: 1.1em; color: var(--r-heading-color);">3. Demystifying 2.6 Bits</h3>
<div style="font-size: 0.75em; opacity: 0.8;">The Role of Initialization & Optimizers</div>
</div>

<div style="background: rgba(255, 255, 255, 0.05); padding: 1.5rem; border-radius: 12px; text-align: left; border-left: 4px solid var(--r-main-color);">
<h3 style="margin: 0 0 0.5rem 0; font-size: 1.1em; color: var(--r-heading-color);">4. KV Cache Compression</h3>
<div style="font-size: 0.75em; opacity: 0.8;">Spectral Norms & Spectral Norms of KV Cache</div>
</div>

</div>

---

<h2 style="font-size: 2.2em; margin: 0 0 2rem 0;">Evaluation of LLM Reasoning</h2>

<h3 style="font-size: 1.3em; font-weight: normal; opacity: 0.8;">A Bayesian Approach</h3>

</div>

---

## Part I: Evaluation of LLM Reasoning, A Bayesian Approach

<div style="border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem; margin-bottom: 1rem;">
<h4 style="margin: 0; opacity: 0.8; text-transform: uppercase; letter-spacing: 1px; font-size: 1.1em;">Contents:</h4>
</div>
<div style="border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem; margin-bottom: 1rem;">
<h4 style="margin: 0; opacity: 0.8; text-transform: uppercase; letter-spacing: 1px; font-size: 1.1em;">Resources and References:</h4>
</div>

<div>
<h3 style="font-size: 1.4em; color: var(--r-heading-color); margin: 0;"> LLM Evaluation Metric </h3>
</div>

<div>
<div style="margin-bottom: 0.5rem;"><strong>Paper:</strong></div>
<div style="margin-bottom: 0.5rem;">Don’t Pass@𝑘: A Bayesian Framework for Large Language Model Evaluation</div>
<div style="font-style: italic; margin-bottom: 0.5rem; opacity: 0.9;">Mohsen Hariri, Amirhossein Samandar, Michael Hinczewski, Vipin Chaudhary (2025)</div>
<a href="https://www.arxiv.org/abs/2510.04265" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://www.arxiv.org/abs/2510.04265</a>
</div>

<div>
<h3 style="font-size: 1.4em; color: var(--r-heading-color); margin: 0;"> Simulating LLM Evaluation </h3>
</div>

<div>
<div style="margin-bottom: 0.5rem;"><strong>Posts:</strong></div>

<div style="margin-bottom: 1rem;">
<div>Simulating LLM Evaluation Datasets Using Psychometric Models (2025-10-23)</div>
<a href="https://mohsenhariri.github.io/posts/2025-10-22-item-response-theory-llms.html" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://mohsenhariri.github.io/posts/2025-10-22-item-response-theory-llms.html</a>
</div>

<div style="margin-bottom: 1rem;">
<div>Simulating LLM Answers to Evaluation Datasets (2025-10-22)</div>
<a href="https://mohsenhariri.github.io/posts/2025-10-22-simulating-llm-answers-to-eval-datasets.html" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://mohsenhariri.github.io/posts/2025-10-22-simulating-llm-answers-to-eval-datasets.html</a>
</div>

<div style="margin-bottom: 0.5rem;"><strong>Code:</strong></div>
<div>
<a href="https://github.com/mohsenhariri/scorio" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://github.com/mohsenhariri/scorio</a>
</div>
</div>

</div>

---

## Motivation: The Problem with Pass@k

- **Pass@k**: Probability of at least one correct answer in $k$ attempts from $N$ total.
- **Instability**: High variance when $k \approx N$.
- **Misleading Rankings**: Small fluctuations distort comparisons.
- **No Uncertainty**: Lacks variance; relies on bootstrapping.

---

## A Bayesian Approach

Treat evaluation as a **statistical inference problem**.

- **Goal**: Estimate underlying success probability $\pi$ and uncertainty.
- **Model**: Categorical outcomes (correct, wrong, format error, etc.).
- **Prior**: Dirichlet prior (uniform or informed).
- **Posterior**: Closed-form mean $\mu$ and variance $\sigma^2$.

> "Replace Pass@k with posterior estimates of a model’s underlying success probability."

---

## The Framework

$R \in \{0, \dots, C\}^{M \times N}$

- $M$ questions, $N$ trials per question.
- Outcomes are **categorical** ($C+1$ categories).
- **Weighted Metric** $\bar{\pi}$:
  $$ \bar{\pi} = \frac{1}{M} \sum_{\alpha=1}^M \mathbf{w} \cdot \boldsymbol{\pi}_\alpha $$
- $\mathbf{w}$: Weight vector (e.g., $w_1=1$ for correct, $w_0=0$ for wrong).

---

## Bayesian Estimator

Given results $R$ and prior $R^0$:

**Posterior Mean ($\mu$)**:
$$ \mu = w_0 + \frac{1}{MT} \sum_{\alpha=1}^M \sum_{j=0}^C \nu_{\alpha j}(w_j - w_0) $$

**Posterior Variance ($\sigma^2$)**:
$$ \sigma^2 = \frac{1}{M^2 (T+1)} \sum_{\alpha=1}^M \left[ \sum_{j=0}^C \frac{\nu_{\alpha j}}{T} (w_j - w_0)^2 - \left(\sum_{j=0}^C \frac{\nu_{\alpha j}}{T} (w_j - w_0)\right)^2 \right] $$

- $\nu_{\alpha j}$: Total counts (prior + observed).
- $T$: Total effective trials.

---

## Equivalence to Average Accuracy

**Theorem**: Under a **uniform prior**, the Bayesian ranking is identical to Average Accuracy (Avg@N).

$$ \mu_{\text{Bayes}} \propto \text{Avg@N} $$

- **Why it matters**: Explains why Avg@N is robust.
- **Benefit**: Bayes adds **principled uncertainty** ($\sigma$) without CLT assumptions.
- Valid even for small $N$.

---

## Validation: Biased Coin Simulation

- **Setup**: 11 "LLMs" (biased coins) with known ground truth probabilities.
- **Comparison**: Bayes@N vs Pass@k variants.
- **Metric**: Kendall's $\tau$ rank correlation with Gold Standard.

</div>
<div style="flex: 1;">

![Biased Coins Simulation](/assets/slides/2025-11-21-test-time-scaling-under-budget/three_panel_biased_coins_10k.webp)

</div>
</div>

Note:
  Bayes@N (blue) converges much faster to the true ranking than Pass@k (orange/green).

---

## Ranking with Uncertainty

**Decision Rule**:
Do not declare a winner if **Credible Intervals (CI)** overlap.

- **Z-score**: $z = \frac{|\mu - \mu'|}{\sqrt{\sigma^2 + \sigma'^2}}$
- **Confidence**: $\rho = \frac{1}{2}(1 + \text{erf}(z/\sqrt{2}))$

</div>
<div style="flex: 1;">

![CI Simulation](/assets/slides/2025-11-21-test-time-scaling-under-budget/CI_bayesian_sim.webp)

</div>
</div>

Note:
  Even with N=80, some models are statistically indistinguishable. We need ~200 trials to separate close models (0.608 vs 0.621).

---

## Experiments: Math Reasoning

<div style="font-size: 0.9em;">

**Benchmarks**:
- **AIME 2024** & **AIME 2025**
- **HMMT Feb 2025**
- **BrUMO 2025**

**11 Reasoning Models**:
<div style="display: flex; gap: 0.5rem; justify-content: center; align-items: center; flex-wrap: wrap; margin: 0.5rem 0;">
  <img src="/assets/slides/2025-10-27-psychometric-modeling/qwen.webp" alt="Qwen" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/deepseek.webp" alt="DeepSeek" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/chatgpt.webp" alt="GPT" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/Nvidia.webp" alt="NVIDIA" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/openthinker.webp" alt="OpenThinker" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/gair.webp" alt="GAIR" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/lg.webp" alt="LG" style="height: 55px;" />
  <img src="/assets/slides/2025-10-27-psychometric-modeling/compass.webp" alt="Compass" style="height: 55px;" />
</div>

**Setup**: 80 trials/model, uniform prompting.

</div>

<div style="display: flex; flex-direction: column; align-items: center;">
  <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/four_panel_kendall_boot_10000_top_p.webp" style="max-width: 100%; max-height: 650px; object-fit: contain;" />
  <div style="font-size: 0.8em; margin-top: 0.5rem; opacity: 0.8;">
    Bayes@N (blue) consistently achieves higher rank correlation with the Gold Standard (N=80) faster than Pass@k.
  </div>
</div>

</div>

---

## Convergence Speed

**Convergence@n**: Smallest $N$ where ranking stabilizes to the Gold Standard.

- **Bayes@N**: Converges earlier (peaks to the left).
- **Pass@k**: Long tail, often fails to converge within budget.

</div>
<div style="display: flex; justify-content: center;">

</div>
</div>

---

## Convergence Probability (CDF)

**Cumulative Distribution**: Probability that ranking has converged by trial $N$.

- **Steeper Rise**: Bayes@N (Blue) reaches high probability of convergence much faster.
- **Pass@k**: Slower rise, indicating more trials needed for stable ranking.

</div>
<div style="display: flex; justify-content: center;">

</div>
</div>

---

## Worst-Case Trajectories

Even with 80 trials, there is no guarantee of convergence.

- **AIME '24, BrUMO, HMMT**: Converges within 80 trials.
- **AIME '25**: No method fully converges within budget $\rightarrow$ Need CIs!

</div>
<div style="flex: 1;">

![Worst Case Rankings](/assets/slides/2025-11-21-test-time-scaling-under-budget/combined_rankings_grid.webp)

</div>
</div>

---

## Categorical Evaluation: Methodology

**Beyond Binary**: Rubric-Aware Evaluation.

- **Base Signals**: `has_box`, `is_correct`, `token_ratio`, `prompt_bpt`, `verifier_probs` (A/B/C).
- **Schemata**: Map signals to $C+1$ categories.
  - *Format Aware*: Penalize unboxed correct.
  - *Conf-Wrong Penalty*: Penalize confidently wrong.
  - *Efficiency-Adjusted*: Reward concision.

---

## Categorical Evaluation: Results

- **Rankings Shift**: Depending on what you value (e.g., penalizing verbosity or confidence calibration).
- **Qwen3-30B**: Consistently top-ranked.
- **Middle Tier**: Sensitive to rubric choice (e.g., *Verifier-Only* vs *Format Aware*).

</div>
<div style="flex: 1;">

![Categorical Ranking](/assets/slides/2025-11-21-test-time-scaling-under-budget/Categorical_ranking.webp)

</div>
</div>

---

## Summary of Bayesian Evaluation

| **Evaluation Method** | **Convergence** | **Confidence Interval** | **Prior Knowledge** | **Categorical** |
|:---------------------|:---------------:|:-----------------------:|:-------------------:|:---------------:|
| Pass@k (& variants) | ❌ | ❌ | ❌ | ❌ |
| avg@N | ✅ | ⚠️ Limited (bootstrap) | ❌ | ❌ |
| **Bayes@N** | ✅  | ✅ **Analytic** | ✅ **Yes** | ✅ **Yes** |

</div>

<div>

**Bayes@N**

- Faster convergence (fewer trials)
- Closed-form uncertainty (no bootstrapping)
- Handles categorical rubrics (beyond 0/1)
- Supports priors

</div>

**Code**

- Source: `https://mohsenhariri.github.io/scorio`

- Python Package: `pip install scorio`

- Julia Package: `Pkg.add("Scorio")`

</div>

---

## Bayesian Framework Big Picture & Future Directions

```mermaid
%%{init: { 'logLevel': 'debug', 'theme': 'default', 'flowchart': { 'diagramPadding': 50, 'nodeSpacing': 80, 'rankSpacing': 90 }, 'themeVariables': { 'fontSize': '26px' }, 'config': { 'scale': 2 } }}%%
graph LR
    %% Main Hub
    Root((Bayesian<br/>Framework))

%% Branch 1: Evaluation
    Eval[Evaluation<br/>Metric]

%% Branch 2: Prior
    Prior{Prior<br/>Knowledge}

%% Branch 3: Categorical
    Cat[Categorical<br/>Evaluation]

%% Branch 4: Ranking
    Rank[Ranking]
    BT(Bradley–Terry–MAP)

%% Sub-branches for Prior
    Quant[Quantization]
    PostTrain[Post-Training]

%% Leaves for Post-Training
    SFT(SFT)
    RL(RL)
    Inst(Instruction<br/>Tuned)

%% Connections
    Root ==> Eval
    Root ==> Prior
    Root ==> Cat
    Root ==> Rank
    
    Rank --- BT
    
    Prior --> Quant
    Prior --> PostTrain
    
    PostTrain --- SFT
    PostTrain --- RL
    PostTrain --- Inst

%% Styling — Increased sizes & thickness
    classDef root fill:#2E4053,stroke:#5D6D7E,stroke-width:6px,color:white,font-size:32px,font-weight:bold;
    classDef eval fill:#117A65,stroke:#1ABC9C,stroke-width:3px,color:white,font-size:26px;
    classDef prior fill:#A93226,stroke:#E74C3C,stroke-width:3px,color:white,font-size:26px;
    classDef cat fill:#D68910,stroke:#F39C12,stroke-width:3px,color:white,font-size:26px;
    classDef rank fill:#8E44AD,stroke:#9B59B6,stroke-width:3px,color:white,font-size:26px;
    
    classDef sub fill:#2471A3,stroke:#5499C7,stroke-width:2px,color:white,font-size:22px;
    classDef leaf fill:#EAEDED,stroke:#BDC3C7,stroke-width:2px,color:#2C3E50,font-size:20px;

class Root root;
    class Eval eval;
    class Prior prior;
    class Cat cat;
    class Rank rank;
    class Quant,PostTrain sub;
    class SFT,RL,Inst,BT leaf;

```

---

<h2 style="font-size: 2.2em; margin: 0 0 2rem 0;">Numerical Data Type & Entropy</h2>

<h3 style="font-size: 1.3em; font-weight: normal; opacity: 0.8;"> Lossless Compression of LLMs</h3>

</div>

---

## Part II: Numerical Data Type & Entropy

<div>
<h3 style="font-size: 1.4em; color: var(--r-heading-color); margin: 0;">LLM Compression</h3>
</div>

<div>
<div style="margin-bottom: 0.5rem;"><strong>Paper:</strong></div>
<div style="margin-bottom: 0.5rem;">70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float</div>
<div style="font-style: italic; margin-bottom: 0.5rem; opacity: 0.9;">Tianyi Zhang, Mohsen Hariri, Shaochen Zhong, Vipin Chaudhary, Yang Sui, Xia Hu, Anshumali Shrivastava (2025)</div>
<a href="https://arxiv.org/abs/2504.11651" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://arxiv.org/abs/2504.11651</a>
</div>

<div>
<h3 style="font-size: 1.4em; color: var(--r-heading-color); margin: 0;">Data Type Entropy</h3>
</div>

<div>
<div style="margin-bottom: 0.5rem;"><strong>Posts:</strong></div>

<div style="margin-bottom: 1rem;">
<div>Entropy of bfloat16 During Training: How Optimizers Shape Weight Distributions (2025-11-17)</div>
<a href="https://mohsenhariri.github.io/posts/2025-11-17-bf16-evolution.html" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://mohsenhariri.github.io/posts/2025-11-17-bf16-evolution.html</a>
</div>

<div style="margin-bottom: 1rem;">
<div>Entropy of bfloat16: 8 Bits Are Doing 2.6 Bits of Work (2025-10-28)</div>
<a href="https://mohsenhariri.github.io/posts/2025-10-28-data-type-entropy.html" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://mohsenhariri.github.io/posts/2025-10-28-data-type-entropy.html</a>
</div>

<div style="margin-bottom: 0.5rem;"><strong>Code:</strong></div>
<div>
<a href="https://github.com/mohsenhariri/dtypen" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://github.com/mohsenhariri/dtypen</a>
</div>
</div>

</div>

---

## BFloat16: Structure and Entropy in Pretrained Models

<div style="display: grid; grid-template-columns: 1fr 1fr; gap: 2rem; align-items: center;">

<div style="display: flex; flex-direction: column; gap: 2rem; font-family: sans-serif;">

<div>
<div style="font-weight: bold; margin-bottom: 0.5rem; font-size: 1.2em;">Float16</div>
<div style="display: flex; width: 50%; height: 60px; border: 1px solid #888; box-shadow: 2px 2px 5px rgba(0,0,0,0.1);">
<div style="flex: 1; background: #58d68d; display: flex; align-items: center; justify-content: center; color: black; font-size: 0.9em; border-right: 1px solid rgba(255,255,255,0.3);" title="Sign: 1 bit">S</div>
<div style="flex: 5; background: #ec7063; display: flex; align-items: center; justify-content: center; color: white; font-size: 0.9em; border-right: 1px solid rgba(255,255,255,0.3);" title="Exponent: 5 bits">Exp (5)</div>
<div style="flex: 10; background: #3498db; display: flex; align-items: center; justify-content: center; color: white; font-size: 0.9em;" title="Mantissa: 10 bits">Mantissa (10)</div>
</div>
</div>

<div>
<div style="font-weight: bold; margin-bottom: 0.5rem; font-size: 1.2em;">BFloat16</div>
<div style="display: flex; width: 50%; height: 60px; border: 1px solid #888; box-shadow: 2px 2px 5px rgba(0,0,0,0.1);">
<div style="flex: 1; background: #58d68d; display: flex; align-items: center; justify-content: center; color: black; font-size: 0.9em; border-right: 1px solid rgba(255,255,255,0.3);" title="Sign: 1 bit">S</div>
<div style="flex: 8; background: #ec7063; display: flex; align-items: center; justify-content: center; color: white; font-size: 0.9em; border-right: 1px solid rgba(255,255,255,0.3);" title="Exponent: 8 bits">Exp (8)</div>
<div style="flex: 7; background: #3498db; display: flex; align-items: center; justify-content: center; color: white; font-size: 0.9em;" title="Mantissa: 7 bits">Mantissa (7)</div>
</div>
</div>

<div>
<div style="font-weight: bold; margin-bottom: 0.5rem; font-size: 1.2em;">Float32</div>
<div style="display: flex; width: 100%; height: 60px; border: 1px solid #888; box-shadow: 2px 2px 5px rgba(0,0,0,0.1);">
<div style="flex: 1; background: #58d68d; display: flex; align-items: center; justify-content: center; color: black; font-size: 0.9em; border-right: 1px solid rgba(255,255,255,0.3);" title="Sign: 1 bit">S</div>
<div style="flex: 8; background: #ec7063; display: flex; align-items: center; justify-content: center; color: white; font-size: 0.9em; border-right: 1px solid rgba(255,255,255,0.3);" title="Exponent: 8 bits">Exp (8)</div>
<div style="flex: 23; background: #3498db; display: flex; align-items: center; justify-content: center; color: white; font-size: 0.9em;" title="Mantissa: 23 bits">Mantissa (23)</div>
</div>
</div>
</div>

<div style="width: 100%; display: flex; flex-direction: column; align-items: center;">
<img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-5.webp" style="max-width: 100%; max-height: 650px; object-fit: contain;" />
<div style="margin-top: 1rem; font-size: 0.8em; opacity: 0.8; text-align: center;">
The Shannon entropy of the components (sign, exponent, mantissa) of BFloat16 weights in various LLMs.
</div>
</div>
</div>

---

## Entropy in Pretrained LLMs

---

## Huffman Coding of Exponents

Compressing 8-bit Exponents in LLMs.

**Fixed Code**: 8 bits per value.

**Huffman Code**:
- **Common** (e.g., 60%) $\rightarrow$ `0` (1 bit)
- **Less Common** (20%) $\rightarrow$ `10` (2 bits)
- **Rare** (20%) $\rightarrow$ Long codes...

**Empirical Compression**: $8 \rightarrow 2.6$ bits

</div>

```mermaid
%%{init: {'theme': 'base', 'themeVariables': { 'fontSize': '24px'}}}%%
graph TD
    Root((100%)) -->|0| A[Common: 60%]
    Root -->|1| N1((40%))
    N1 -->|0| B[Less Common: 20%]
    N1 -->|1| N2((20%))
    N2 -->|0| C[Rare: 10%]
    N2 -->|1| D[Rare: 10%]

style A fill:#58d68d,stroke:#333,stroke-width:2px,color:black
    style B fill:#58d68d,stroke:#333,stroke-width:2px,color:black
    style C fill:#58d68d,stroke:#333,stroke-width:2px,color:black
    style D fill:#58d68d,stroke:#333,stroke-width:2px,color:black
    
    style Root fill:#ec7063,stroke:#333,stroke-width:2px,color:white
    style N1 fill:#ec7063,stroke:#333,stroke-width:2px,color:white
    style N2 fill:#ec7063,stroke:#333,stroke-width:2px,color:white
```

</div>

---

## Integration with vLLM

- **Optimization**: Just-In-Time Compilation $\rightarrow$ **Precompiled Kernels**
- **Efficiency**: On-the-fly decompression
- **Implementation**: Custom kernels designed by *Tianyi Zhang*

</div>

**Open Source Implementation**

<a href="https://github.com/mohsenhariri/vllm-df11" style="color: inherit; text-decoration: none; border-bottom: 1px dotted currentColor;">
 github.com/mohsenhariri/vllm-df11
</a>

<a href="https://pypi.org/project/vllm-df11/" style="color: inherit; text-decoration: none; border-bottom: 1px dotted currentColor;">
 pypi.org/project/vllm-df11/
</a>

</div>

---

<h2 style="font-size: 2.2em; margin: 0 0 2rem 0;">Demystifying 2.6 bits of Exponent</h2>

<h3 style="font-size: 1.3em; font-weight: normal; opacity: 0.8;">The Role of Initialization and Optimizers</h3>

</div>

---

## Is 2.6 Bits Universal?

- Neural Networks
- Toy Model: MLP on MNIST
- Trained models (after convergence)

![alt text](/assets/slides/2025-11-21-test-time-scaling-under-budget/image-1.webp)

---

## Trajectories of Entropy During Training

- **X-Axis**: Training Epochs (0-50), **Y-Axis**: Shannon Entropy (Bits).
- **Sign (Left)**: Stays at **1 bit** (balanced).
- **Exponent (Middle)**: Fluctuates around **2.6 bits** (low information).
- **Mantissa (Right)**: Stays near **7 bits** (fully utilized).

</div>

---

## Initialization x Optimizers

- Red dashed line marks **2.6 bits**; shaded region shows the **[2.4, 2.8]** range observed in trained LLMs.
- Lines group by **optimizer** more than by initialization.

</div>

---

## Why 2.6 in LLMs and Diffusion Models?

- **AdamW Consistency**: Four different initializations (Xavier, Kaiming, Orthogonal, Normal), all trained with AdamW.
- **Convergence**: Despite starting from different initial configurations, all four models converge to remarkably similar exponent distributions.
- **Result**: The shapes of the distributions are nearly identical, and all produce entropy around **2.6 bits**.

</div>

</div>

---

<h2 style="font-size: 2.2em; margin: 0 0 2rem 0;"> KV Cache Compression </h2>

<h3 style="font-size: 1.3em; font-weight: normal; opacity: 0.8;"> Spectral Norms of KV Cache</h3>

</div>

---

## Compression & Quantization

<div>
<h3 style="font-size: 1.4em; color: var(--r-heading-color); margin: 0;">1. Quantize What Counts: More For Keys, Less For Values</h3>
</div>

<div>
<div style="margin-bottom: 0.5rem;"><strong>Paper:</strong></div>
<div style="margin-bottom: 0.5rem;">Quantize What Counts: More for Keys, Less for Values</div>
<div style="font-style: italic; margin-bottom: 0.5rem; opacity: 0.9;"> Mohsen Hariri, Alan Luo, Weicong Chen, Shaochen Zhong, Tianyi Zhang, Qifan Wang, Xia Hu, Xiaotian Han, Vipin Chaudhary (2025)</div>
<a href="https://arxiv.org/abs/2502.15075" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://arxiv.org/abs/2502.15075</a>
</div>

<div>
<h3 style="font-size: 1.4em; color: var(--r-heading-color); margin: 0;">2. Norm-Aware KVQuant: Precision Where It Counts</h3>
</div>

<div>

<div style="margin-bottom: 0.5rem;"><strong>Code:</strong></div>
<div>
<a href="https://github.com/mohsenhariri/spectral-kv" target="_blank" rel="noopener noreferrer" style="font-family: monospace; font-size: 0.9em; opacity: 0.8;">https://github.com/mohsenhariri/spectral-kv</a>
</div>
</div>

</div>

---

## Attention Mechanism

For input hidden states $X \in \mathbb{R}^{\text{seq} \times d_{\text{model}}}$:

</div>

$$
\begin{align}
Q &= X W^Q, \quad K = X W^K, \quad V = X W^V \\
\text{Attention}(Q, K, V) &= \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V
\end{align}
$$

</div>

**$W^K$** and **$W^V$** play different architectural roles

- Keys: Shape attention patterns (multiplicative interaction with queries)
- Values: Provide content to be retrieved (linear weighted sum)

</div>

Note:
  This architectural asymmetry is fundamental.
  Keys influence attention scores exponentially via softmax.
  Values are just weighted and summed.

---

## Norms in Key and Value Matrices

<div style="text-align: center; font-size: 0.8em; margin-bottom: 1.5em; opacity: 0.9;">
  <strong>Model:</strong> Llama3 70B &nbsp;•&nbsp; <strong>Dataset:</strong> C4 &nbsp;•&nbsp; <strong>Analysis:</strong> Frobenius and Spectral Norms Across Layers
</div>

<div style="display: flex; justify-content: center; gap: 2rem; align-items: center;">
  <div style="flex: 1;">
    <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-6.webp" style="width: 100%; height: auto; object-fit: contain;" />
    
  </div>
  <div style="flex: 1;">
    <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-7.webp" style="width: 100%; height: auto; object-fit: contain;" />
    
  </div>
</div>

---

## Singular Value Distribution of Key and Value Tensors

---

## Current Approaches: Ad Hoc and Unprincipled

**Existing strategies:**

1. **Fixed-precision**: Same bit-width for keys and values 
   - Ignores their different roles

2. **Hyperparameter tuning**: Grid search over bit allocations 
   - Expensive, model-specific

3. **Heuristic-based**: Use inference-time cache statistics 
   - Data-dependent, no theoretical insight

</div>

**Missing**: A principled, generalizable, theory-driven approach

</div>

---

## Theorem 1: Key-Value Norm Disparity

**Theorem (Key-Value Norm Disparity)**

Let $W^K$ and $W^V$ denote the key and value projection matrices. Then:

$$
\mathbb{E}[\|W^K\|_F] > \mathbb{E}[\|W^V\|_F]
$$

</div>

**Proof intuition**:

1. Start with Xavier initialization → equal expected norms 
2. During training, $W^K$ receives amplified gradients from $W^Q$ 
3. $W^V$ only influences post-attention representations (no multiplication) 
4. This architectural asymmetry causes $W^K$ to grow larger

</div>

Note:
  This is not just an empirical observation—it's provable.
  The dual role of keys causes systematically larger updates.
  Full proof in Appendix.

---

## Theorem 2: Key-Prioritized Quantization

**Theorem (Key-Prioritized Quantization)**

Let $(b_K, b_V)$ denote bit allocations for key and value caches. For any pair with $b_K > b_V$, the expected reconstruction error is strictly lower than the swapped allocation $(b_V, b_K)$, provided that:

$$
\mathbb{E}[\|K\|_F^2] > \mathbb{E}[\|V\|_F^2]
$$

</div>

**Translation**: If keys have higher norms (Theorem 1), then prioritizing key precision minimizes total error

</div>

Note:
  This theorem transforms bit allocation from art to science.
  It's not just "keys seem to need more bits"—it's provable.
  Detailed proof in Appendix (Theorems A1, A2, A3).

---

## Leveraging Norm Disparity in KV Cache

- **Idea**: Since Keys have larger norms and are more sensitive, we should allocate more bits to them.
- **Strategy**: 
  - **Keys**: High precision (e.g., 4-bit)
  - **Values**: Low precision (e.g., 2-bit)
- **Result**: Better accuracy-memory trade-off than uniform quantization.

</div>

**Results: K₄V₂ vs K₂V₄ on GSM8k**

| Model | K₂V₄ | K₄V₂ |
|:------|:----:|:----:|
| 1B    | 0.06 | **0.34** |
| 8B    | 0.55 | **0.75** |
| 14B   | 0.78 | **0.91** |
| 70B   | 0.76 | **0.87** |

</div>

---

## Ablation Studies

**Impact of Group Size**
- Smaller group sizes benefit Keys more than Values.

**Impact of Rotation**
- Rotation (Hadamard) mitigates outliers.
- Key-only rotation provides significant gains.

</div>

</div>

---

## Summary of KV Quantization: Norm-Aware Allocation

<div>
<h4 style="color: var(--r-main-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem; margin-bottom: 1rem;">Theoretical</h4>
<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 0.8rem; padding-left: 1rem; border-left: 2px solid rgba(255,255,255,0.3);">Key projections have larger norms than value tensors.</li>
  <li style="margin-bottom: 0.8rem; padding-left: 1rem; border-left: 2px solid rgba(255,255,255,0.3);">Higher key precision minimizes quantization error.</li>
</ul>
</div>

<div>
<h4 style="color: var(--r-main-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem; margin-bottom: 1rem;">Empirical</h4>
<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 0.8rem; padding-left: 1rem; border-left: 2px solid rgba(255,255,255,0.3);">Key caches incur 1.5–2× larger reconstruction errors.</li>
  <li style="margin-bottom: 0.8rem; padding-left: 1rem; border-left: 2px solid rgba(255,255,255,0.3);">K₄V₂ retains 98.3% accuracy of K₄V₄.</li>
  <li style="margin-bottom: 0.8rem; padding-left: 1rem; border-left: 2px solid rgba(255,255,255,0.3);">Key-only rotation provides most synergistic benefits.</li>
</ul>
</div>

<div>
<h4 style="color: var(--r-main-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem; margin-bottom: 1rem;">Practical</h4>
<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 0.8rem; padding-left: 1rem; border-left: 2px solid rgba(255,255,255,0.3);">Geometry-driven bit allocation is lightweight, generalizable, and orthogonal.</li>
</ul>
</div>

</div>

---

## Acknowledgments

<div style="margin-bottom: 0.5rem;">
  <h4 style="font-size: 1.1em; border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.2rem; margin-bottom: 0.5rem;">Advisor & Committee</h4>
  <div style="display: flex; gap: 2rem; justify-content: center; flex-wrap: wrap;">
    
    <div style="text-align: center;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/profile.webp" style="width: 110px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold;">Vipin Chaudhary, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">Advisor</div>
    </div>
    
    <div style="text-align: center;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-12.webp" style="width: 110px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold;">Sanmukh Kuppannagari, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">Committee</div>
    </div>
    <div style="text-align: center;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-11.webp" style="width: 110px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold;">Michael Lewicki, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">Committee</div>
    </div>
  </div>
</div>

<div style="margin-bottom: 0.5rem;">
  <h4 style="font-size: 1.1em; border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.2rem; margin-bottom: 0.5rem;">Collaborators</h4>
  <div style="display: flex; gap: 1rem; justify-content: center; flex-wrap: wrap;">
    <div style="text-align: center; width: 200px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/mhincz.webp" style="width: 100px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em; line-height: 1.2;">Michael Hinczewski, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">CWRU</div>
    </div>
    <div style="text-align: center; width: 200px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-18.webp" style="width: 100px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em; line-height: 1.2;">Xia Hu, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">Rice Univ.</div>
    </div>
    <div style="text-align: center; width: 200px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-20.webp" style="width: 100px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em; line-height: 1.2;">Lauren Calandruccio, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">CWRU</div>
    </div>
    <div style="text-align: center; width: 200px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-19.webp" style="width: 100px; height: 100px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em; line-height: 1.2;">Anshumali Shrivastava, Ph.D.</div>
      <div style="opacity: 0.7; font-size: 0.9em;">Rice Univ.</div>
    </div>
  </div>
</div>

<div style="margin-bottom: 0.5rem;">
  
  <div style="display: flex; gap: 1.5rem; justify-content: center; flex-wrap: wrap;">
    
      <div style="text-align: center; width: 120px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-14.webp" style="width: 80px; height: 80px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em;">Shaochen Zhong</div>
      <div style="opacity: 0.7; font-size: 0.8em;">Rice Univ.</div>
    </div>
    <div style="text-align: center; width: 120px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-13.webp" style="width: 80px; height: 80px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em;">Tianyi Zhang</div>
      <div style="opacity: 0.7; font-size: 0.8em;">Rice Univ.</div>
    </div>
    
    <div style="text-align: center; width: 120px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-15.webp" style="width: 80px; height: 80px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em;">Weicong Chen</div>
      <div style="opacity: 0.7; font-size: 0.8em;">CWRU</div>
    </div>
    <div style="text-align: center; width: 120px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-16.webp" style="width: 80px; height: 80px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em;">Amirhossein Samandar</div>
      <div style="opacity: 0.7; font-size: 0.8em;">CWRU</div>
    </div>
    <div style="text-align: center; width: 120px;">
      <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/image-17.webp" style="width: 80px; height: 80px; border-radius: 50%; object-fit: cover; margin-bottom: 0.2rem;">
      <div style="font-weight: bold; font-size: 0.9em;">Andrew Seohwan Yu</div>
      <div style="opacity: 0.7; font-size: 0.8em;">CWRU</div>
    </div>
  </div>
</div>

<div style="display: flex; align-items: center; justify-content: center; gap: 2rem; margin-top: 2rem; padding-top: 1rem; border-top: 1px solid rgba(255,255,255,0.1);">
  <div style="display: flex; align-items: center; gap: 1rem;">
    <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/NSF_Official_logo_Med_Res_600ppi.webp" style="height: 60px; width: auto;">
    <div style="text-align: left; font-size: 0.8em; opacity: 0.8;">
      <div>NSF Awards</div>
      <div>2117439, 2112606, 2320952</div>
    </div>
  </div>
  <div style="height: 40px; width: 1px; background: rgba(255,255,255,0.2);"></div>
  <div style="display: flex; align-items: center; gap: 1rem;">
    <img src="/assets/slides/2025-11-21-test-time-scaling-under-budget/cwru_logo.webp" style="height: 50px; width: auto;">
    <div style="font-size: 0.9em; opacity: 0.9; text-align: left;">
      High Performance Computing (HPC)<br>Center at CWRU
    </div>
  </div>
</div>

</div>

---

## Research Contributions: Compression & Test-Time Scaling (2025)

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Compression & Quantization</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>Quantize What Counts: More for Keys, Less for Values</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Mohsen Hariri, Alan Luo, Weicong Chen, Shaochen Zhong, Tianyi Zhang, Qifan Wang, Xia Hu, Xiaotian Han, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Under Review: ACL (October Cycle)</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Precision Where It Counts: Norm-Aware KVQuant</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Mohsen Hariri, Weicong Chen, Sreehari Sankar, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress • Target: ACL (January Cycle)</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>70% Size, 100% Accuracy: Lossless LLM Compression (DFloat11)</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Tianyi Zhang, Mohsen Hariri, Shaochen Zhong, Vipin Chaudhary, Yang Sui, Xia Hu, Anshumali Shrivastava</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">NeurIPS 2025 • Collaboration with Rice University</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Future Directions</strong>
    <div style="opacity: 0.7; margin-left: 1rem;">DF11 in KV cache</div>
  </li>
</ul>
</div>

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Reasoning LLMs & Test-Time Scaling</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>Don't Pass@k: A Bayesian Framework for LLM Evaluation</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Mohsen Hariri, Amirhossein Samandar, Michael Hinczewski, Vipin Chaudhary</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Under Review: ICLR • Collaboration with Physics Department</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Ranking without Averaging: A Bradley-Terry-Bayesian Framework</strong>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Practices for Evaluating Reasoning Models</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Reassessing Reasoning Evaluation in LLMs: Metrics, Ranking, and Reproducibility</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Work in Progress</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Next Steps</strong>
    <div style="opacity: 0.7; margin-left: 1rem;">Rubric-based evaluation (extended categorical evaluation)</div>
    <div style="opacity: 0.7; margin-left: 1rem;">Prior (role of post-training and compression in LM evaluation)</div>
  </li>
</ul>
</div>

</div>

---

## Research Contributions: Applications & Biomedical (2024-25)

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Applications</h4>

<div>
<h4 style="color: var(--r-heading-color); border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 0.5rem;">Biomedical</h4>

<ul style="list-style-type: none; padding: 0; margin: 0;">
  <li style="margin-bottom: 1rem;">
    <strong>Novel adaptation of video segmentation to 3D MRI: efficient zero-shot knee segmentation with SAM2</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">Andrew Yu, Mohsen Hariri, Xuecen Zhang, Mingrui Yang, Vipin Chaudhary, Xiaojuan Li</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">SPIE Medical Imaging 2024</div>
  </li>
  <li style="margin-bottom: 1rem;">
    <strong>Radiomics to detect inflammation and fibrosis in Crohn’s disease</strong>
    <div style="opacity: 0.7; margin-left: 1rem; font-size: 0.9em;">P. Chirra, J. Sleiman, N. Gandhi, I. Gordon, M. Hariri, M. Baker, R. Ottichilo, D. Bruining, J. Kurowski, S. Viswanath, F. Rieder</div>
    <div style="opacity: 0.9; margin-left: 1rem; color: var(--r-main-color);">Journal of Crohn's and Colitis, 2025</div>
  </li>
</ul>
</div>

</div>

---

# Thank You!

<h3 style="margin-bottom: 2rem; font-size: 2em;">Questions?</h3>

<div style="font-size: 1.1em; opacity: 0.9; line-height: 2; text-align: center;">
<div><strong>Email:</strong> <a href="mailto:mohsen.hariri@case.edu" style="color: inherit; text-decoration: none; border-bottom: 1px dotted currentColor;">mohsen.hariri@case.edu</a></div>
<div><strong>Website:</strong> <a href="https://mohsenhariri.github.io" target="_blank" rel="noopener noreferrer" style="color: inherit; text-decoration: none; border-bottom: 1px dotted currentColor;">mohsenhariri.github.io</a></div>

</div>

Navigation