Serving Reasoning LLMs Efficiently and Reliably (Slides)

.reveal .slides {
    text-align: left;
  }

.reveal section.conf-slide {
    box-sizing: border-box;
    width: 100%;
    height: 100%;
    padding: 54px 70px 44px;
    color: var(--conf-ink);
    background:
      linear-gradient(135deg, rgba(37, 99, 235, 0.055), rgba(255, 255, 255, 0) 32%),
      linear-gradient(225deg, rgba(15, 118, 110, 0.055), rgba(255, 255, 255, 0) 36%),
      #f8fafc;
    overflow: hidden;
  }

.reveal section.conf-title {
    padding: 64px 82px 54px;
  }

.reveal .conf-slide h1,
  .reveal .conf-slide h2,
  .reveal .conf-slide h3 {
    color: var(--conf-ink);
    letter-spacing: 0;
    text-transform: none;
  }

.reveal .conf-slide h1 {
    font-size: 76px;
    line-height: 1.03;
    margin: 0;
  }

.reveal .conf-slide h2 {
    font-size: 46px;
    line-height: 1.08;
    margin: 0 0 26px;
  }

.reveal .conf-slide h3 {
    font-size: 25px;
    line-height: 1.18;
    margin: 0 0 10px;
  }

.reveal .conf-slide p,
  .reveal .conf-slide li {
    color: var(--conf-muted);
    font-size: 24px;
    line-height: 1.36;
  }

.reveal .conf-slide strong {
    color: var(--conf-ink);
    font-weight: 720;
  }

.reveal .conf-kicker {
    color: var(--conf-teal);
    font-size: 18px;
    font-weight: 760;
    letter-spacing: 0.08em;
    margin-bottom: 14px;
    text-transform: uppercase;
  }

.reveal .conf-subtitle {
    max-width: 950px;
    color: var(--conf-muted);
    font-size: 31px;
    line-height: 1.32;
    margin-top: 26px;
  }

.reveal .conf-title-grid {
    display: grid;
    grid-template-columns: minmax(0, 1100px) 286px;
    gap: 42px;
    align-content: center;
    align-items: center;
    height: 100%;
    max-width: none;
  }

.reveal .conf-title-main {
    min-width: 0;
  }

.reveal .conf-title-qr {
    box-sizing: border-box;
    justify-self: end;
    width: 286px;
    padding: 18px;
    text-align: center;
  }

.reveal .conf-title-qr img {
    display: block;
    width: 238px;
    height: 238px;
    max-width: none;
    max-height: none;
    object-fit: contain;
    margin: 0 auto 13px;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .conf-title-qr strong {
    display: block;
    color: var(--conf-ink);
    font-size: 21px;
    line-height: 1.15;
  }

.reveal .conf-acl-logo {
    display: block;
    width: 300px;
    max-width: none;
    height: auto;
    margin: 0 0 48px;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .conf-meta {
    color: var(--conf-muted);
    font-size: 21px;
    line-height: 1.45;
    margin-top: 24px;
  }

.reveal .conf-presenter {
    display: block;
    color: var(--conf-ink);
    font-size: 33px;
    font-weight: 740;
    line-height: 1.14;
    margin-bottom: 5px;
  }

.reveal .conf-event-line {
    color: var(--conf-teal);
    font-size: 23px;
    font-weight: 760;
    line-height: 1.25;
    margin-top: 18px;
  }

.reveal .conf-logo-row {
    display: flex;
    align-items: center;
    gap: 24px;
    margin-top: 40px;
    flex-wrap: nowrap;
  }

.reveal .conf-logo-badge {
    display: flex;
    align-items: center;
    justify-content: center;
    height: 84px;
    padding: 10px 18px;
    border-radius: 14px;
    background: rgba(255, 255, 255, 0.92);
    box-shadow: 0 18px 46px rgba(15, 23, 42, 0.12);
  }

.reveal .conf-logo-badge.nsf {
    width: 104px;
    height: 92px;
    padding: 7px;
  }

.reveal .conf-logo-badge.scorio {
    width: 250px;
  }

.reveal .conf-logo-row img {
    display: block;
    width: auto;
    max-width: none;
    height: 100%;
    max-height: none;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-panel {
    border: 1px solid rgba(15, 23, 42, 0.14);
    border-radius: 8px;
    background: var(--conf-panel);
    box-shadow: 0 18px 46px rgba(15, 23, 42, 0.08);
  }

.reveal .conf-title-card {
    padding: 24px;
  }

.reveal .conf-title-card img {
    width: 100%;
    height: 370px;
    object-fit: contain;
    display: block;
  }

.reveal .conf-title-metric {
    display: grid;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    gap: 12px;
    margin-top: 20px;
  }

.reveal .conf-title-metric p,
  .reveal .conf-metric p {
    margin: 0;
  }

.reveal .conf-title-metric strong,
  .reveal .conf-metric strong {
    display: block;
    color: var(--conf-blue);
    font-size: 34px;
    line-height: 1;
  }

.reveal .conf-title-metric span,
  .reveal .conf-metric span {
    display: block;
    color: var(--conf-muted);
    font-size: 16px;
    line-height: 1.22;
    margin-top: 7px;
  }

.reveal .conf-chip-row {
    display: flex;
    flex-wrap: wrap;
    gap: 10px;
    margin-top: 18px;
  }

.reveal .conf-chip {
    display: inline-flex;
    align-items: center;
    border: 1px solid rgba(15, 23, 42, 0.14);
    border-radius: 999px;
    color: var(--conf-muted);
    background: rgba(255, 255, 255, 0.74);
    font-size: 16px;
    font-weight: 720;
    line-height: 1;
    padding: 9px 13px;
  }

.reveal .conf-chip.blue {
    color: var(--conf-blue);
    background: var(--conf-blue-soft);
    border-color: rgba(37, 99, 235, 0.24);
  }

.reveal .conf-chip.teal {
    color: var(--conf-teal);
    background: var(--conf-teal-soft);
    border-color: rgba(15, 118, 110, 0.24);
  }

.reveal .conf-chip.amber {
    color: var(--conf-amber);
    background: var(--conf-amber-soft);
    border-color: rgba(180, 83, 9, 0.24);
  }

.reveal .conf-grid {
    display: grid;
    gap: 18px;
  }

.reveal .conf-grid.two {
    grid-template-columns: repeat(2, minmax(0, 1fr));
  }

.reveal .conf-grid.three {
    grid-template-columns: repeat(3, minmax(0, 1fr));
  }

.reveal .conf-grid.four {
    grid-template-columns: repeat(4, minmax(0, 1fr));
  }

.reveal .conf-card {
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.78);
    padding: 20px 22px;
    min-height: 138px;
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-card p {
    margin: 0;
    font-size: 20px;
  }

.reveal .conf-num {
    color: var(--conf-blue);
    font-size: 18px;
    font-weight: 780;
    margin-bottom: 12px;
  }

.reveal .conf-callout {
    border-left: 5px solid var(--conf-blue);
    border-radius: 8px;
    background: var(--conf-blue-soft);
    padding: 18px 22px;
    color: var(--conf-ink);
    font-size: 25px;
    line-height: 1.34;
  }

.reveal .conf-callout.teal {
    border-left-color: var(--conf-teal);
    background: var(--conf-teal-soft);
  }

.reveal .conf-callout.amber {
    border-left-color: var(--conf-amber);
    background: var(--conf-amber-soft);
  }

.reveal .conf-callout.rose {
    border-left-color: var(--conf-rose);
    background: var(--conf-rose-soft);
  }

.reveal .conf-big-question {
    margin-top: 24px;
    padding: 26px 30px;
    color: var(--conf-ink);
    font-size: 35px;
    line-height: 1.2;
  }

.reveal .conf-equation {
    display: flex;
    align-items: center;
    justify-content: center;
    box-sizing: border-box;
    min-width: 0;
    min-height: 112px;
    max-width: 100%;
    overflow-x: auto;
    border: 1px solid rgba(15, 23, 42, 0.12);
    border-radius: 8px;
    background: rgba(15, 23, 42, 0.04);
    color: var(--conf-ink);
    font-size: 31px;
    line-height: 1.25;
    padding: 18px 22px;
  }

/* Keep display math inside its panel instead of bleeding over the next
     column. Wide formulas still read best one size down: add `conf-equation sm`
     or give the equation a full-width row (see slide-patterns.md). */
  .reveal .conf-equation .katex { max-width: 100%; }

.reveal .conf-equation.sm {
    font-size: 24px;
  }

.reveal .conf-figure {
    border: 1px solid rgba(15, 23, 42, 0.12);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.84);
    padding: 16px;
  }

.reveal .conf-figure img {
    display: block;
    width: 100%;
    height: 100%;
    object-fit: contain;
  }

.reveal .conf-figure figcaption {
    color: var(--conf-muted);
    font-size: 16px;
    line-height: 1.25;
    margin-top: 10px;
  }

.reveal .conf-fig-short {
    height: 430px;
  }

.reveal .conf-fig-mid {
    height: 540px;
  }

.reveal .conf-fig-wide {
    height: 465px;
  }

.reveal .conf-fig-tall {
    height: 610px;
  }

.reveal .conf-split {
    display: grid;
    grid-template-columns: minmax(0, 0.98fr) minmax(420px, 0.82fr);
    gap: 30px;
    align-items: stretch;
  }

.reveal .conf-split.reverse {
    grid-template-columns: minmax(420px, 0.76fr) minmax(0, 1fr);
  }

.reveal .conf-metric {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 14px;
    margin-top: 18px;
  }

.reveal .conf-metric .conf-card {
    min-height: 104px;
  }

.reveal .conf-mini-table,
  .reveal .conf-table {
    width: 100%;
    border-collapse: collapse;
  }

.reveal .conf-mini-table {
    font-size: 18px;
  }

.reveal .conf-table {
    font-size: 20px;
  }

.reveal .conf-mini-table th,
  .reveal .conf-mini-table td,
  .reveal .conf-table th,
  .reveal .conf-table td {
    border-bottom: 1px solid rgba(15, 23, 42, 0.12);
    padding: 11px 10px;
    text-align: right;
    color: var(--conf-muted);
  }

.reveal .conf-mini-table th:first-child,
  .reveal .conf-mini-table td:first-child,
  .reveal .conf-table th:first-child,
  .reveal .conf-table td:first-child {
    text-align: left;
  }

.reveal .conf-mini-table th,
  .reveal .conf-table th {
    color: var(--conf-ink);
    font-weight: 760;
  }

.reveal .conf-win {
    color: var(--conf-green) !important;
    font-weight: 760;
    background: rgba(21, 128, 61, 0.08);
  }

.reveal .conf-loss {
    color: var(--conf-rose) !important;
  }

.reveal .conf-flow {
    display: grid;
    grid-template-columns: repeat(4, minmax(0, 1fr));
    gap: 12px;
    margin-top: 18px;
  }

.reveal .conf-flow .conf-card {
    min-height: 148px;
  }

.reveal .conf-flow h3 {
    font-size: 23px;
  }

.reveal .conf-list {
    margin: 0;
    padding-left: 1.1em;
  }

.reveal .conf-list li {
    margin: 0 0 12px;
  }

.reveal .conf-footer-note {
    position: absolute;
    left: 70px;
    right: 70px;
    bottom: 28px;
    color: rgba(71, 85, 105, 0.78);
    font-size: 16px;
  }

.reveal .conf-takeaway-header {
    display: flex;
    align-items: center;
    gap: 24px;
    margin: 0 0 34px;
  }

.reveal .conf-takeaway-icon {
    display: block;
    width: 104px;
    height: 104px;
    max-width: none;
    max-height: none;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-takeaway-title {
    color: var(--conf-ink);
    font-size: 62px;
    line-height: 0.98;
    margin: 0;
  }

.reveal .conf-takeaway-subtitle {
    color: var(--conf-teal);
    font-size: 28px;
    font-weight: 760;
    line-height: 1.12;
    margin-top: 12px;
  }

.reveal .conf-takeaway-grid {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 22px;
  }

.reveal .conf-takeaway-card {
    position: relative;
    min-height: 250px;
    padding: 30px 30px 28px;
    overflow: hidden;
    border-top: 5px solid var(--conf-teal);
    background:
      linear-gradient(180deg, rgba(15, 118, 110, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
    box-shadow: 0 18px 42px rgba(15, 23, 42, 0.09);
  }

.reveal .conf-takeaway-card:nth-child(2) {
    border-top-color: var(--conf-blue);
    background:
      linear-gradient(180deg, rgba(37, 99, 235, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
  }

.reveal .conf-takeaway-card:nth-child(3) {
    border-top-color: var(--conf-amber);
    background:
      linear-gradient(180deg, rgba(180, 83, 9, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
  }

.reveal .conf-takeaway-num {
    display: inline-grid;
    place-items: center;
    width: 42px;
    height: 42px;
    margin-bottom: 20px;
    border-radius: 50%;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 22px;
    font-weight: 820;
  }

.reveal .conf-takeaway-card:nth-child(2) .conf-takeaway-num {
    background: var(--conf-blue-soft);
    color: var(--conf-blue);
  }

.reveal .conf-takeaway-card:nth-child(3) .conf-takeaway-num {
    background: var(--conf-amber-soft);
    color: var(--conf-amber);
  }

.reveal .conf-takeaway-card h3 {
    font-size: 30px;
    line-height: 1.08;
    min-height: 42px;
    margin-bottom: 16px;
  }

.reveal .conf-takeaway-card p {
    font-size: 25px;
    line-height: 1.34;
    min-height: 128px;
  }

.reveal .conf-closing-layout {
    display: grid;
    grid-template-rows: auto minmax(0, 1fr) auto;
    gap: 26px;
    align-items: stretch;
    height: 100%;
  }

.reveal .conf-closing-main {
    display: grid;
    justify-items: center;
    align-content: start;
    text-align: center;
    padding-top: 8px;
  }

.reveal .conf-closing-title {
    font-size: 72px;
    line-height: 1.03;
    margin: 0 0 18px;
  }

.reveal .conf-closing-main .conf-subtitle {
    max-width: none;
    margin-top: 0;
  }

.reveal .conf-closing-authors {
    color: var(--conf-muted);
    font-size: 22px;
    line-height: 1.25;
    margin-top: 12px;
  }

.reveal .conf-qr-grid {
    display: grid;
    grid-template-columns: repeat(4, 260px);
    justify-content: center;
    gap: 24px;
    width: min(1120px, 100%);
    margin: 32px auto 0;
  }

.reveal .conf-qr-item {
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.82);
    padding: 18px;
    text-align: center;
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-qr-item img {
    display: block;
    width: 210px;
    height: 210px;
    max-width: none;
    max-height: none;
    object-fit: contain;
    margin: 0 auto 12px;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .conf-qr-item strong {
    display: block;
    color: var(--conf-ink);
    font-size: 20px;
    line-height: 1.1;
  }

.reveal .conf-contact-line {
    max-width: 1220px;
    margin: 18px auto 0;
    color: var(--conf-muted);
    font-size: 20px;
    line-height: 1.25;
  }

.reveal .conf-contact-email {
    color: var(--conf-blue);
    font-weight: 780;
    white-space: nowrap;
  }

.reveal .conf-ack-stack {
    display: grid;
    grid-template-columns: 450px 760px;
    justify-content: center;
    align-items: center;
    gap: 34px;
    width: 100%;
    margin: 0 auto;
    padding: 0;
  }

.reveal .conf-ack-item {
    display: grid;
    grid-template-columns: auto minmax(0, 1fr);
    gap: 22px;
    align-items: center;
    border: 0;
    border-radius: 0;
    background: transparent;
    padding: 0;
    box-shadow: none;
  }

.reveal .conf-ack-cwru {
    grid-template-columns: 290px minmax(0, 1fr);
  }

.reveal .conf-ack-item img {
    display: block;
    width: auto;
    max-width: none;
    max-height: none;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-ack-nsf img {
    width: 84px;
    height: 84px;
  }

.reveal .conf-ack-cwru img {
    width: 270px;
    height: auto;
  }

.reveal .conf-ack-item h3 {
    font-size: 26px;
    line-height: 1.08;
    margin: 0 0 8px;
    white-space: nowrap;
  }

.reveal .conf-ack-item p {
    font-size: 22px;
    line-height: 1.18;
    margin: 0;
    white-space: nowrap;
  }

.reveal .slides section.conf-slide .conf-acl-logo {
    width: 300px;
    max-width: none;
    height: auto;
    margin: 0 0 48px;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .slides section.conf-slide .conf-logo-row img {
    width: auto;
    max-width: none;
    height: 64px;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .slides section.conf-slide .conf-logo-badge.nsf img {
    width: 76px;
    height: 76px;
  }

.reveal .slides section.conf-slide .conf-logo-badge.scorio img {
    width: 214px;
    height: auto;
  }

.reveal .slides section.conf-slide .conf-qr-item img {
    width: 210px;
    height: 210px;
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0 auto 12px;
  }

.reveal .slides section.conf-slide .conf-title-qr img {
    width: 238px;
    height: 238px;
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0 auto 13px;
  }

.reveal .slides section.conf-slide .conf-ack-item img {
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .slides section.conf-slide .conf-ack-nsf img {
    width: 84px;
    height: 84px;
  }

.reveal .slides section.conf-slide .conf-ack-cwru img {
    width: 270px;
    height: auto;
  }

.reveal .conf-title-thesis {
    max-width: 1080px;
    color: var(--conf-muted);
    font-size: 29px;
    line-height: 1.28;
    margin-top: 28px;
  }

.reveal .conf-slide .conf-lead {
    max-width: 1210px;
    color: var(--conf-muted);
    font-size: 30px;
    line-height: 1.28;
    margin: -8px 0 28px;
  }

.reveal .conf-slide .conf-small {
    color: var(--conf-muted);
    font-size: 18px;
    line-height: 1.3;
  }

.reveal .conf-stat-row {
    display: grid;
    grid-template-columns: repeat(4, minmax(0, 1fr));
    gap: 14px;
    margin-top: 20px;
  }

.reveal .conf-stat {
    border: 1px solid rgba(15, 23, 42, 0.12);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.78);
    padding: 18px 18px 16px;
    min-height: 112px;
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-stat strong {
    display: block;
    color: var(--conf-blue);
    font-size: 43px;
    line-height: 1;
  }

.reveal .conf-stat span {
    display: block;
    color: var(--conf-muted);
    font-size: 17px;
    line-height: 1.22;
    margin-top: 9px;
  }

.reveal .conf-slide .conf-table.compact {
    font-size: 17px;
  }

.reveal .conf-slide .conf-table.compact th,
  .reveal .conf-slide .conf-table.compact td {
    padding: 8px 8px;
  }

.reveal .conf-slide .conf-table.wide-text td,
  .reveal .conf-slide .conf-table.wide-text th {
    text-align: left;
  }

.reveal .conf-slide .conf-table.wide-text td:last-child,
  .reveal .conf-slide .conf-table.wide-text th:last-child {
    text-align: right;
  }

.reveal .conf-slide .conf-table .conf-method-cell {
    color: var(--conf-ink);
    font-weight: 720;
  }

.reveal .conf-two-thirds {
    display: grid;
    grid-template-columns: minmax(0, 1.05fr) minmax(420px, 0.72fr);
    gap: 28px;
    align-items: stretch;
  }

.reveal .conf-figure-grid {
    display: grid;
    grid-template-columns: minmax(0, 1fr) minmax(390px, 0.52fr);
    gap: 24px;
    align-items: stretch;
  }

.reveal .conf-figure-grid.reverse {
    grid-template-columns: minmax(390px, 0.52fr) minmax(0, 1fr);
  }

.reveal .conf-figure-grid .conf-figure {
    height: 590px;
  }

.reveal .conf-figure-grid .conf-figure.short {
    height: 500px;
  }

.reveal .conf-stack {
    display: grid;
    gap: 16px;
    align-content: start;
  }

.reveal .conf-target-card {
    min-height: 210px;
  }

.reveal .conf-target-card .conf-equation {
    min-height: 82px;
    margin-bottom: 16px;
    font-size: 27px;
  }

.reveal .conf-method-map {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 18px;
    margin-top: 20px;
  }

.reveal .conf-method-map .conf-card {
    min-height: 255px;
  }

.reveal .conf-method-map .conf-equation {
    min-height: 76px;
    font-size: 23px;
    margin-bottom: 14px;
  }

.reveal .conf-rule-grid {
    display: grid;
    grid-template-columns: repeat(5, minmax(0, 1fr));
    gap: 13px;
    margin-top: 26px;
  }

.reveal .conf-rule-grid .conf-card {
    min-height: 190px;
    padding: 18px 18px;
  }

.reveal .conf-rule-grid h3 {
    font-size: 22px;
  }

.reveal .conf-code {
    box-sizing: border-box;
    max-width: 100%;
    overflow-x: auto;
    border: 1px solid rgba(148, 163, 184, 0.2);
    border-radius: 12px;
    background: linear-gradient(180deg, #111c33, #0b1424);
    color: #dbe3f1;
    font-family: ui-monospace, "SF Mono", "SFMono-Regular", "Menlo", "JetBrains Mono", "Cascadia Code", "Consolas", monospace;
    font-size: 18px;
    line-height: 1.5;
    letter-spacing: -0.01em;
    margin: 0;
    padding: 20px 22px;
    white-space: pre;
    tab-size: 2;
    -webkit-font-smoothing: antialiased;
    text-rendering: optimizeLegibility;
    box-shadow: 0 16px 34px rgba(8, 15, 30, 0.4), inset 0 1px 0 rgba(255, 255, 255, 0.05);
  }

.reveal .conf-code .cm {
    color: #6b7a92;
    font-style: italic;
  }

.reveal .conf-code .kw {
    color: #7dd3fc;
  }

.reveal .conf-code .fn {
    color: #86efac;
  }

.reveal .conf-code .arg {
    color: #fcd34d;
  }

.reveal .conf-code .str {
    color: #fca5a5;
  }

.reveal .conf-code .num {
    color: #c4b5fd;
  }

.reveal .conf-qrs-small {
    display: grid;
    grid-template-columns: repeat(2, 180px);
    gap: 18px;
    justify-content: start;
    margin-top: 16px;
  }

.reveal .conf-qrs-small .conf-qr-item {
    padding: 14px;
  }

.reveal .conf-qrs-small .conf-qr-item img {
    width: 144px;
    height: 144px;
  }

.reveal .conf-scorio-header {
    display: grid;
    grid-template-columns: 250px minmax(0, 1fr);
    gap: 24px;
    align-items: center;
    margin-bottom: 18px;
  }

.reveal .conf-scorio-logo-panel {
    display: flex;
    align-items: center;
    justify-content: flex-start;
    height: 104px;
  }

.reveal .conf-scorio-logo-panel img {
    display: block;
    width: 202px;
    height: auto;
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-scorio-header h2 {
    font-size: 40px;
    line-height: 1.08;
    margin: 0;
  }

.reveal .conf-scorio-heading {
    display: grid;
    gap: 9px;
    align-content: center;
  }

.reveal .conf-scorio-docs {
    justify-self: start;
    display: inline-flex;
    align-items: center;
    gap: 7px;
    border: 1px solid rgba(15, 118, 110, 0.3);
    border-radius: 999px;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 18px;
    font-weight: 780;
    line-height: 1;
    padding: 9px 15px;
    text-decoration: none;
  }

.reveal .conf-scorio-docs:hover {
    background: rgba(15, 118, 110, 0.17);
  }

.reveal .conf-scorio-app {
    border: 1px solid rgba(15, 118, 110, 0.22);
    border-radius: 999px;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 17px;
    font-weight: 780;
    line-height: 1;
    padding: 11px 15px;
    text-align: center;
  }

.reveal .conf-scorio-layout {
    display: grid;
    grid-template-columns: minmax(0, 0.78fr) minmax(0, 1.22fr);
    gap: 18px;
    align-items: stretch;
  }

.reveal .conf-scorio-panel,
  .reveal .conf-api-board {
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.8);
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-scorio-panel {
    padding: 18px;
  }

.reveal .conf-scorio-panel h3,
  .reveal .conf-api-board h3 {
    font-size: 23px;
    line-height: 1.08;
    margin: 0 0 12px;
  }

.reveal .conf-scorio-code {
    font-size: 15px;
    line-height: 1.5;
    padding: 16px 18px;
  }

.reveal .conf-scorio-panel .conf-scorio-code {
    font-size: 14px;
    line-height: 1.55;
  }

.reveal .conf-scorio-note {
    color: var(--conf-muted);
    font-size: 18px;
    line-height: 1.26;
    margin-top: 14px;
  }

.reveal .conf-scorio-note strong {
    color: var(--conf-ink);
  }

.reveal .conf-scorio-use {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 8px;
    margin-top: 14px;
  }

.reveal .conf-scorio-use .conf-scorio-app {
    display: grid;
    place-items: center;
    min-height: 52px;
    border: 1px solid rgba(15, 118, 110, 0.22);
    border-radius: 999px;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 15px;
    font-weight: 780;
    line-height: 1.08;
    padding: 10px 12px;
    text-align: center;
  }

.reveal .conf-api-board {
    padding: 16px 18px 14px;
  }

.reveal .conf-api-board h3 {
    display: flex;
    align-items: baseline;
    justify-content: space-between;
    gap: 16px;
  }

.reveal .conf-api-board h3 span {
    color: var(--conf-muted);
    font-size: 15px;
    font-weight: 680;
  }

.reveal .conf-api-group {
    padding: 8px 0;
    border-top: 1px solid rgba(15, 23, 42, 0.11);
  }

.reveal .conf-api-head {
    display: flex;
    align-items: baseline;
    gap: 12px;
    margin-bottom: 4px;
  }

.reveal .conf-api-head code {
    color: var(--conf-ink);
    font-size: 16px;
    font-weight: 820;
  }

.reveal .conf-api-head span {
    color: var(--conf-teal);
    font-size: 14px;
    font-weight: 760;
  }

.reveal .conf-api-columns {
    display: grid;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    column-gap: 16px;
    row-gap: 2px;
  }

.reveal .conf-slide .conf-api-line {
    color: var(--conf-muted);
    font-size: 13px;
    line-height: 1.22;
    margin: 0 0 5px;
  }

.reveal .conf-slide .conf-api-line strong {
    display: inline;
    color: var(--conf-ink);
    font-size: 13px;
    font-weight: 800;
  }

.reveal .conf-slide .conf-api-line code {
    display: inline-block;
    border-radius: 5px;
    background: rgba(15, 23, 42, 0.055);
    color: var(--conf-ink);
    font-size: 11.5px;
    line-height: 1.18;
    margin: 0 2px 3px 0;
    padding: 2px 5px;
  }

.reveal .conf-slide .conf-fit-list li {
    font-size: 21px;
    line-height: 1.28;
    margin-bottom: 10px;
  }

.reveal .conf-slide .conf-tight-list li {
    font-size: 19px;
    line-height: 1.24;
    margin-bottom: 8px;
  }

.reveal .conf-banner {
    display: grid;
    grid-template-columns: auto minmax(0, 1fr);
    gap: 18px;
    align-items: center;
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.78);
    padding: 18px 22px;
    margin-top: 20px;
  }

.reveal .conf-banner-number {
    color: var(--conf-blue);
    font-size: 54px;
    font-weight: 780;
    line-height: 1;
  }

.reveal .conf-banner p {
    margin: 0;
    font-size: 23px;
    line-height: 1.26;
  }

.reveal .conf-label {
    display: inline-flex;
    align-items: center;
    border-radius: 999px;
    background: rgba(15, 23, 42, 0.08);
    color: var(--conf-ink);
    font-size: 15px;
    font-weight: 760;
    line-height: 1;
    padding: 8px 10px;
    text-transform: uppercase;
  }

.reveal .conf-figure img.no-stretch {
    height: auto;
    max-height: 100%;
  }

/* ---- kvq component classes (reused KV-cache slides) ---- */
  .reveal .conf-quanto-layout { align-items: start; }
  .reveal .conf-metric.conf-quanto-metrics {
    grid-template-columns: minmax(0, 1fr);
    grid-template-rows: repeat(3, minmax(0, 1fr));
    gap: 10px;
    height: 246px;
    margin-top: 0;
  }
  .reveal .conf-metric.conf-quanto-metrics .conf-card {
    display: grid; align-content: center; min-height: 0; padding: 10px 20px;
  }
  .reveal .conf-metric.conf-quanto-metrics strong { font-size: 28px; line-height: 1.12; }
  .reveal .conf-metric.conf-quanto-metrics p > span { font-size: 15px; line-height: 1.15; margin-top: 4px; }

.reveal .conf-memory-layout {
    display: grid;
    grid-template-columns: minmax(0, 0.98fr) minmax(420px, 0.82fr);
    gap: 24px 30px; align-items: stretch;
  }
  .reveal .conf-memory-context { display: grid; gap: 14px; }
  .reveal .conf-need-box {
    grid-column: 1 / -1; min-height: 134px;
    border-left: 5px solid var(--conf-teal); background: var(--conf-teal-soft); padding: 24px 30px;
  }
  .reveal .conf-need-box p { font-size: 28px; }

.reveal .conf-accuracy-table {
    table-layout: fixed; overflow: hidden; border-collapse: separate; border-spacing: 0;
    background: rgba(255, 255, 255, 0.9); box-shadow: 0 18px 38px rgba(15, 23, 42, 0.09);
  }
  .reveal .conf-accuracy-table th, .reveal .conf-accuracy-table td {
    padding: 16px 20px; border-bottom: 1px solid rgba(15, 23, 42, 0.1); font-size: 21px;
  }
  .reveal .conf-accuracy-table th {
    background: linear-gradient(180deg, rgba(15, 23, 42, 0.055), rgba(15, 23, 42, 0.015)); font-size: 20px;
  }
  .reveal .conf-accuracy-table th:first-child, .reveal .conf-accuracy-table td:first-child { width: 42%; }
  .reveal .conf-accuracy-table td:first-child { color: var(--conf-ink); font-weight: 720; }
  .reveal .conf-accuracy-table th:not(:first-child), .reveal .conf-accuracy-table td:not(:first-child) {
    width: 29%; font-variant-numeric: tabular-nums;
  }
  .reveal .conf-accuracy-table tbody tr:nth-child(even) td:not(.conf-win) { background: rgba(15, 23, 42, 0.025); }
  .reveal .conf-accuracy-table tbody tr:last-child td { border-bottom: 0; }
  .reveal .conf-accuracy-table .conf-win { background: rgba(21, 128, 61, 0.13); color: var(--conf-green) !important; font-weight: 820; }

/* ---- merged-deck closing layout (two children: main + acks) ---- */
  .reveal .conf-closing-layout { grid-template-rows: auto auto; align-content: center; gap: 44px; height: 100%; }

/* ---- per-paper section dividers (paper headline screenshot) ---- */
  .reveal .conf-paper-divider {
    display: grid;
    grid-template-rows: auto auto auto auto;
    align-content: center;
    gap: 22px;
    height: 100%;
  }
  .reveal .conf-paper-divider .conf-kicker { margin: 0; }
  .reveal .conf-paper-divider h2 { margin: 0; }
  .reveal .conf-paper-shot {
    box-sizing: border-box;
    display: grid;
    place-items: center;
    margin: 0;
    padding: 24px 32px;
    background: #ffffff;
    border: 1px solid rgba(15, 23, 42, 0.14);
    border-radius: 8px;
    box-shadow: 0 18px 46px rgba(15, 23, 42, 0.08);
  }
  .reveal .conf-paper-shot img {
    display: block;
    width: auto;
    max-width: 100%;
    height: auto;
    max-height: 432px;
    margin: 0 auto;
    border-radius: 0;
    box-shadow: none;
  }
  .reveal .conf-paper-foot {
    margin: 0;
    color: var(--conf-muted);
    font-size: 22px;
    line-height: 1.32;
  }
  .reveal .conf-paper-foot strong { color: var(--conf-ink); }

/* venue name caption under each logo on the title slide */
  .reveal .conf-venue-row { align-items: flex-start; }
  .reveal .conf-venue-item {
    display: flex;
    flex-direction: column;
    align-items: center;
    gap: 12px;
  }
  .reveal .conf-venue-name {
    color: var(--conf-muted);
    font-size: 18px;
    font-weight: 720;
    letter-spacing: 0.03em;
  }

/* venue chip (logo + name) instead of a kicker on each paper divider */
  .reveal .conf-paper-venue {
    display: flex;
    align-items: center;
    gap: 16px;
  }
  .reveal .conf-paper-venue .conf-venue-chip {
    display: flex;
    align-items: center;
    justify-content: center;
    height: 56px;
    padding: 8px 16px;
    border-radius: 12px;
    background: rgba(255, 255, 255, 0.92);
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.1);
  }
  .reveal .conf-paper-venue .conf-venue-chip img {
    display: block;
    width: auto;
    height: 38px;
    max-width: none;
    max-height: none;
    margin: 0;
    border-radius: 0;
    box-shadow: none;
    object-fit: contain;
  }
  .reveal .conf-paper-venue-name {
    color: var(--conf-teal);
    font-size: 20px;
    font-weight: 760;
    letter-spacing: 0.06em;
    text-transform: uppercase;
  }

/* four-up takeaways (one card per paper) */
  .reveal .conf-takeaway-grid.quad { grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 18px; }
  .reveal .conf-takeaway-grid.quad .conf-takeaway-card { min-height: 322px; padding: 26px 24px 24px; }
  .reveal .conf-takeaway-grid.quad .conf-takeaway-card h3 { font-size: 24px; line-height: 1.12; min-height: 58px; margin-bottom: 12px; }
  .reveal .conf-takeaway-grid.quad .conf-takeaway-card p { font-size: 19px; line-height: 1.34; min-height: 0; }
  .reveal .conf-takeaway-grid.quad .conf-takeaway-num { width: 38px; height: 38px; margin-bottom: 16px; font-size: 20px; }
  .reveal .conf-takeaway-card:nth-child(4) {
    border-top-color: var(--conf-violet);
    background:
      linear-gradient(180deg, rgba(124, 58, 237, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
  }
  .reveal .conf-takeaway-card:nth-child(4) .conf-takeaway-num {
    background: var(--conf-violet-soft);
    color: var(--conf-violet);
  }

/* ---- animation-first manim video slides (full-bleed, dark) ---- */
  .reveal section.manim-video-slide {
    box-sizing: border-box;
    width: 100%;
    height: 100%;
    padding: 0;
    background: #0B1020;
    overflow: hidden;
  }
  .reveal .manim-video-slide video {
    position: absolute;
    inset: 0;
    width: 100%;
    height: 100%;
    object-fit: contain;
    background: #0B1020;
    display: block;
  }
  .reveal .manim-video-slide .manim-cap {
    position: absolute;
    left: 46px;
    bottom: 42px;
    z-index: 4;
    display: flex;
    flex-direction: column;
    gap: 7px;
    max-width: 66%;
    padding: 15px 22px;
    border-left: 4px solid #22d3ee;
    border-radius: 12px;
    background: rgba(6, 10, 22, 0.62);
    box-shadow: 0 14px 34px rgba(0, 0, 0, 0.4);
    -webkit-backdrop-filter: blur(5px);
    backdrop-filter: blur(5px);
  }
  .reveal .manim-video-slide .manim-cap-kicker {
    color: #22d3ee;
    font-size: 17px;
    font-weight: 760;
    letter-spacing: 0.09em;
    text-transform: uppercase;
  }
  .reveal .manim-video-slide .manim-cap-text {
    color: #f9fafb;
    font-size: 28px;
    line-height: 1.12;
    font-weight: 640;
  }

/* per-animation playback progress bar (a pacing cue for the presenter) */
  .reveal .manim-video-slide .manim-progress {
    position: absolute;
    left: 0;
    right: 0;
    bottom: 0;
    height: 7px;
    background: rgba(148, 163, 184, 0.22);
    z-index: 5;
  }
  .reveal .manim-video-slide .manim-progress-fill {
    height: 100%;
    width: 0%;
    background: linear-gradient(90deg, #22d3ee, #0ea5e9);
    box-shadow: 0 0 12px rgba(34, 211, 238, 0.55);
    transition: width 0.12s linear;
  }
  .reveal .manim-video-slide .manim-progress-time {
    position: absolute;
    left: 50%;
    transform: translateX(-50%);
    bottom: 20px;
    z-index: 5;
    padding: 6px 12px;
    border-radius: 8px;
    background: rgba(6, 10, 22, 0.6);
    color: #cbd5e1;
    font-family: ui-monospace, "SF Mono", "SFMono-Regular", Menlo, Consolas, monospace;
    font-size: 19px;
    font-weight: 600;
    line-height: 1;
    letter-spacing: 0.02em;
    font-variant-numeric: tabular-nums;
    white-space: nowrap;
  }
  /* paused cue: amber bar + readout while the presenter holds an animation */
  .reveal .manim-video-slide.manim-paused .manim-progress-fill {
    background: linear-gradient(90deg, #fbbf24, #f59e0b);
    box-shadow: 0 0 12px rgba(251, 191, 36, 0.5);
  }
  .reveal .manim-video-slide.manim-paused .manim-progress-time {
    color: #fbbf24;
  }
  .reveal .manim-video-slide.manim-paused .manim-progress-time::before {
    content: "paused \00B7 ";
  }
</style>

<div class="conf-title-grid">
  <div class="conf-title-main">
    <h1>Serving Reasoning LLMs Efficiently and Reliably</h1>
    <div class="conf-logo-row conf-venue-row">
      <div class="conf-venue-item">
        <span class="conf-logo-badge"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/neurips.svg" alt="NeurIPS 2025" /></span>
        <span class="conf-venue-name">NeurIPS 2025</span>
      </div>
      <div class="conf-venue-item">
        <span class="conf-logo-badge"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/iclr.svg" alt="ICLR 2026" /></span>
        <span class="conf-venue-name">ICLR 2026</span>
      </div>
      <div class="conf-venue-item">
        <span class="conf-logo-badge"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/acl.svg" alt="ACL 2026" /></span>
        <span class="conf-venue-name">ACL 2026</span>
      </div>
    </div>
    <div class="conf-meta">
      <span class="conf-presenter">Vipin Chaudhary</span>
      Case Western Reserve University
    </div>
    <div class="conf-logo-row">
      <span class="conf-logo-badge"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/cwru.svg" alt="Case Western Reserve University" /></span>
      <span class="conf-logo-badge nsf"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/nsf.svg" alt="National Science Foundation" /></span>
    </div>
  </div>
  <aside class="conf-title-qr conf-panel" aria-label="Interactive slides QR code">
    <img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/slide_qr.svg" alt="QR code for the interactive slides" />
    <strong>Scan for interactive slides</strong>
  </aside>
</div>

Note:
  Same story as the main talk, told in pictures. Reasoning models moved the cost of intelligence from training to inference, and these four papers make that shift both cheap to run and rigorous to measure. Almost every slide from here is a short animation; I'll narrate over them. Two papers go after the memory bill, two after the measurement bill.

---

<h2>Reasoning models think out loud &mdash; and thinking is expensive</h2>
<p class="conf-lead">A reasoning model doesn't answer in one shot. It writes a long chain of thought, and we sample it many times per question.</p>

<div class="conf-flow">
  <div class="conf-card"><div class="conf-num">01</div><h3>Think longer</h3><p>One answer becomes a long chain of thought &mdash; 10&ndash;100&times; more tokens per query.</p></div>
  <div class="conf-card"><div class="conf-num">02</div><h3>Sample more</h3><p>Pass@k, self-consistency, best-of-N &mdash; many tries per question, not one.</p></div>
  <div class="conf-card"><div class="conf-num">03</div><h3>Inference dominates</h3><p>The cost and the memory now live at serving time, not training time.</p></div>
  <div class="conf-card"><div class="conf-num">04</div><h3>Two bills arrive</h3><p>Running it costs <strong>memory</strong>. Trusting it costs <strong>measurement</strong>.</p></div>
</div>

<div class="conf-callout teal" style="margin-top: 26px;">Test-time scaling buys accuracy with inference compute &mdash; and the bill comes due twice.</div>

Note:
  The shift everyone is living through: a reasoning model writes a long chain of thought, so tokens per query jump ten to a hundred times, and then we sample it many times per question. Training used to be the expensive part; now the cost and the memory live at serving time. That hands us two bills, and both get worse as the sampling budget grows — running the model costs memory, and trusting its score costs measurement.

---

<div class="conf-kicker">The plan</div>
<h2>Lossless where you must, calibrated where you can't</h2>
<p class="conf-lead">Test-time scaling sends two bills: one to run the model, one to trust the numbers.</p>
<div class="conf-grid two">
  <div class="conf-card">
    <h3>The memory bill &mdash; run it cheaply</h3>
    <ul class="conf-list conf-fit-list">
      <li><strong>DFloat11</strong> &mdash; losslessly compress the weights <span class="conf-small">(NeurIPS 2025)</span></li>
      <li><strong>More keys, less values</strong> &mdash; quantize the KV cache by where the information is <span class="conf-small">(ACL 2026)</span></li>
    </ul>
  </div>
  <div class="conf-card">
    <h3>The measurement bill &mdash; trust the numbers</h3>
    <ul class="conf-list conf-fit-list">
      <li><strong>Don't Pass@k</strong> &mdash; a Bayesian score with error bars <span class="conf-small">(ICLR 2026)</span></li>
      <li><strong>Ranking under test-time scaling</strong> &mdash; which leaderboard to trust at a given budget <span class="conf-small">(ACL 2026)</span></li>
    </ul>
  </div>
</div>

<div class="conf-callout teal" style="margin-top: 26px;">One idea throughout: spend precision where the information is &mdash; then report the uncertainty you can't remove.</div>

Note:
  Here's the whole talk on one slide. Two papers make reasoning models cheaper to run — DFloat11 losslessly compresses the weights, and the KV-cache work spends a tight bit budget where the information actually is. Two make them trustworthy to measure — Don't Pass@k turns a score into a posterior with error bars, and the ranking paper tells you which leaderboard to trust at a given budget.

---

<div class="conf-kicker">Part 1 of 2</div>
<h2>The memory bill: serve it cheaply</h2>
<p class="conf-lead">At serving time, two things fill the GPU. Both shrink once you stop spending equal precision everywhere.</p>

<div class="conf-grid two">
  <div class="conf-card">
    <div class="conf-num">WEIGHTS</div>
    <h3>Static, and mostly redundant</h3>
    <p>The model's weights are fixed, but stored with bits that are nearly always wasted. <strong>DFloat11</strong> reclaims them &mdash; losslessly.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">KV CACHE</div>
    <h3>Grows with every token</h3>
    <p>Long thinking fills a per-token memory. <strong>More keys, less values</strong> spends its bit budget where the error hurts most.</p>
  </div>
</div>

Note:
  Part one is the memory bill. Two things sit on the GPU while you serve: the weights, which are static but stored wastefully, and the KV cache, which grows with every token of thinking. We'll take them in that order — first the weights with DFloat11, then the cache.

---

<div class="conf-paper-divider">
  <div class="conf-paper-venue">
    <span class="conf-venue-chip"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/neurips.svg" alt="NeurIPS 2025" /></span>
    <span class="conf-paper-venue-name">NeurIPS 2025</span>
  </div>
  <h2>First, the weights: 30% smaller, every output identical</h2>
  <figure class="conf-paper-shot">
    <img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/header_df11.webp" alt="Paper headline: 70% Size, 100% Accuracy — Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float. Tianyi Zhang, Mohsen Hariri, Shaochen Zhong, Vipin Chaudhary, Yang Sui, Xia Hu, Anshumali Shrivastava." />
  </figure>
  <p class="conf-paper-foot">Bit-for-bit identical &mdash; safe even where any deviation is disqualifying.</p>
</div>

Note:
  The first paper, at NeurIPS, attacks the static cost — the weights. The title is the whole pitch: seventy percent of the size, a hundred percent of the accuracy. About a third smaller, and not a single output bit changes — which is what makes it usable where an answer that merely differs from the original is a non-starter.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/df11-01-memory-wall.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/df11-01-memory-wall.mp4" aria-label="The model that won't fit"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Memory wall</span><span class="manim-cap-text">The model that won't fit</span></div>

Note:
  An 810&nbsp;GB model slides over eight 80&nbsp;GB GPUs and overflows. The two usual escapes — buy more machines, or shrink it and lose quality — both dim out. The question: can we make it smaller without changing any output?

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/df11-02-wasted-bits.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/df11-02-wasted-bits.mp4" aria-label="Five bits quietly wasted"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Wasted bits</span><span class="manim-cap-text">Five bits quietly wasted</span></div>

Note:
  Zoom into one 16-bit number. Two of its three parts use their full range, but the middle part — the exponent — almost always lands on the same few values. Most of those bits are wasted, the same way in every weight.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/df11-03-huffman-packing.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/df11-03-huffman-packing.mp4" aria-label="Short codes for common values"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Smart packing</span><span class="manim-cap-text">Short codes for common values</span></div>

Note:
  The fix is classic compression: give the common values short codes and rare values long codes, pack everything into a smaller stream, and decode one back perfectly to prove nothing is lost. Each number drops from 16 bits to about 11 — like zipping a file, not resaving a photo.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/df11-04-lossless-and-fast.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/df11-04-lossless-and-fast.mp4" aria-label="Identical answers, far faster"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Proof it works</span><span class="manim-cap-text">Identical answers, far faster</span></div>

Note:
  Side by side, the scores snap to equals signs — identical accuracy, pixel-identical images — while the lossy alternative drifts. And a custom GPU decoder keeps it fast, many times quicker than the usual offload-to-CPU fallback, freeing memory for much longer outputs.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/df11-05-it-fits.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/df11-05-it-fits.mp4" aria-label="Now it fits, losslessly"></video>
<div class="manim-cap"><span class="manim-cap-kicker">It fits</span><span class="manim-cap-text">Now it fits &mdash; losslessly</span></div>

Note:
  Back to the opening overflow: the 810&nbsp;GB block shrinks to 551&nbsp;GB and now fits the eight-GPU node with room to spare. The red "does not fit" flips to green. Same answers, a third less memory, and 5.7–14.9&times; longer generation.

---

<div class="conf-kicker">Now the moving part</div>
<h2>Weights are fixed &mdash; but every token of thinking adds memory</h2>
<p class="conf-lead">DFloat11 shrank the static weights. The other half of the bill grows as the model thinks: the <strong>KV cache</strong>.</p>

<div class="conf-callout amber" style="margin-top: 24px;">Same trick, harder problem: there's a fixed memory budget &mdash; so which part of the cache should get the precision?</div>

Note:
  DFloat11 handled the static cost. But long reasoning has a second, growing cost: the KV cache — a working memory the model writes for every token. We can compress it too, but under a fixed budget we have to choose what to protect. That choice is the next paper.

---

<div class="conf-paper-divider">
  <div class="conf-paper-venue">
    <span class="conf-venue-chip"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/acl.svg" alt="ACL 2026" /></span>
    <span class="conf-paper-venue-name">ACL 2026</span>
  </div>
  <h2>The KV cache: spend bits where the error hurts</h2>
  <figure class="conf-paper-shot">
    <img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/header_kvq.webp" alt="Paper headline: Quantize What Counts — More for Keys, Less for Values. Mohsen Hariri, Alan Luo, Weicong Chen, Tianyi Zhang, Qifan Wang, Xiaotian Han, Vipin Chaudhary." />
  </figure>
  <p class="conf-paper-foot">Keys and values do different jobs &mdash; so they shouldn't get the same precision.</p>
</div>

Note:
  The second paper, at ACL, is about that cache. Its claim is in the title: more for keys, less for values. Keys and values play different roles, and it turns out they tolerate compression very differently — so splitting the budget evenly is the wrong default.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-01-tokens-fill-cache.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-01-tokens-fill-cache.mp4" aria-label="Every token leaves something behind"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The snowball</span><span class="manim-cap-text">Every token leaves something behind</span></div>

Note:
  Here's where the cache comes from: every new token drops one key and one value into two growing stacks. The model re-reads them on every step, so they have to stay in GPU memory.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-02-context-wall.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-02-context-wall.mp4" aria-label="Long context hits a memory wall"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The wall</span><span class="manim-cap-text">Long context hits a memory wall</span></div>

Note:
  The longer the context, the taller those stacks — and the meter climbs toward a wall. Long chains of thought are exactly the case where the KV cache, not the weights, becomes the bottleneck.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-03-ruler.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-03-ruler.mp4" aria-label="Quantizing is measuring with a ruler"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The ruler</span><span class="manim-cap-text">Quantizing is measuring with a ruler</span></div>

Note:
  Compressing the cache means storing each number with fewer bits — like reading a value off a ruler with coarser tick marks. Fewer bits, coarser ruler, more rounding error.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-04-unequal-error.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-04-unequal-error.mp4" aria-label="Same grid, unequal error"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Unequal damage</span><span class="manim-cap-text">Same grid, unequal error</span></div>

Note:
  Now the key insight, made visual: keys vary over a much bigger range than values. Put both on the same coarse ruler and the bigger signal — the keys — picks up a much larger rounding error. Equal bits do not mean equal damage.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-05-k2v4-bad.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-05-k2v4-bad.mp4" aria-label="Spend on values, keys stay noisy"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Bad split</span><span class="manim-cap-text">Spend on values, keys stay noisy</span></div>

Note:
  Fixed six-bit budget. The tempting split — two bits for keys, four for values — under-protects exactly the part that's most sensitive, so most of the distortion lands on the keys.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-06-k4v2-good.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-06-k4v2-good.mp4" aria-label="More for keys, less for values"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Better split</span><span class="manim-cap-text">More for keys, less for values</span></div>

Note:
  Flip it — four bits for keys, two for values, same total budget. The dominant key error shrinks a lot while the value error rises only a little. That asymmetry is the whole paper.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-07-accuracy.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-07-accuracy.mp4" aria-label="K4V2 nearly matches full precision"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The result</span><span class="manim-cap-text">K4V2 nearly matches full precision</span></div>

Note:
  On real models, the key-heavy split wins: K4V2 beats the even split and nearly matches full precision across model families — exactly as the geometry predicted.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/kv-08-memory-reduction.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/kv-08-memory-reduction.mp4" aria-label="25% less memory, about 98% accuracy"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The payoff</span><span class="manim-cap-text">~25% less cache, ~98% accuracy</span></div>

Note:
  The payoff: about a quarter less KV-cache memory while keeping roughly 98% of accuracy — purely by allocating bits by where the error actually lands.

---

<div class="conf-kicker">Part 1 &mdash; the takeaway</div>
<h2>Spend bits where the information is</h2>

<div class="conf-grid two">
  <div class="conf-card">
    <div class="conf-num">DFloat11</div>
    <h3>Shrink weights, change nothing</h3>
    <p>Compress the near-empty exponent: ~30% smaller, <strong>bit-identical</strong> outputs &mdash; and 405B fits one node.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">More keys, less values</div>
    <h3>Protect the sensitive half</h3>
    <p>Give the KV budget to keys. <strong>K4V2</strong> keeps ~98% accuracy at ~25% less cache.</p>
  </div>
</div>

<div class="conf-callout teal" style="margin-top: 26px;">Cheaper to run &mdash; with no quality lost where it matters. Now: can we trust the scores?</div>

Note:
  So part one, in one line: spend precision where the information is. Losslessly on the weights, and key-heavy on the cache. The model is now cheaper to serve without giving up quality. Which raises the second question — when we score these models, can we trust the leaderboard?

---

<div class="conf-kicker">Part 2 of 2</div>
<h2>The measurement bill: trust the numbers</h2>
<p class="conf-lead">A reasoning model gives a different answer each time. So a score from a few samples is itself a random number &mdash; and leaderboards forget that.</p>

<div class="conf-grid two">
  <div class="conf-card">
    <div class="conf-num">ONE MODEL</div>
    <h3>Score it honestly</h3>
    <p><strong>Don't Pass@k</strong> turns a point score into a posterior with a credible interval &mdash; and a rule for when a gap is real.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">MANY MODELS</div>
    <h3>Rank them fairly</h3>
    <p><strong>Ranking under test-time scaling</strong> asks which ranking method to trust at a given sampling budget.</p>
  </div>
</div>

Note:
  Part two is the measurement bill. Because the model is random, a score built from a handful of samples is noisy — and the usual metrics report it as if it were exact. First we fix the score for one model, then we fix the ranking across many models.

---

<div class="conf-paper-divider">
  <div class="conf-paper-venue">
    <span class="conf-venue-chip"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/iclr.svg" alt="ICLR 2026" /></span>
    <span class="conf-paper-venue-name">ICLR 2026</span>
  </div>
  <h2>Don't Pass@k: report a score with an error bar</h2>
  <figure class="conf-paper-shot">
    <img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/header_bayes.webp" alt="Paper headline: Don't Pass@k — A Bayesian Framework for Large Language Model Evaluation. Mohsen Hariri, Amirhossein Samandar, Michael Hinczewski, Vipin Chaudhary." />
  </figure>
  <p class="conf-paper-foot">Same ranking as accuracy &mdash; but now you know when a gap is real.</p>
</div>

Note:
  The third paper, at ICLR, is about scoring one model. The pitch: stop reporting a bare Pass@k number, and report a Bayesian score with a credible interval instead — so a difference between two models can be called real or just sampling noise.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/bayes-01-flip.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/bayes-01-flip.mp4" aria-label="Re-run it and first place flips"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The flip</span><span class="manim-cap-text">Re-run it and first place flips</span></div>

Note:
  A model answers the same hard question over and over — some hits, some misses. Collapse that into one score, build a small leaderboard, then change the random seed: the top two places swap. Same models, same test, different luck.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/bayes-02-what-breaks.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/bayes-02-what-breaks.mp4" aria-label="Four ways Pass@k misleads"></video>
<div class="manim-cap"><span class="manim-cap-kicker">What breaks</span><span class="manim-cap-text">Four ways Pass@k misleads</span></div>

Note:
  Four concrete failures of a bare Pass@k: the number wobbles when you resample; it has no error bar; rankings reshuffle on a small benchmark; and when two scores are close, there's no principled way to call a winner.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/bayes-03-bayes-fix.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/bayes-03-bayes-fix.mp4" aria-label="Same ranking, plus an error bar"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The fix</span><span class="manim-cap-text">Same ranking, plus an error bar</span></div>

Note:
  The fix in one picture: start from "no information," fold in the right/wrong counts, and a flat curve sharpens into a peak. The peak is the score, the width is the error bar. Sorted against plain accuracy, it gives the exact same order — but now with honest uncertainty.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/bayes-04-convergence.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/bayes-04-convergence.mp4" aria-label="Finds the true ranking faster"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The race</span><span class="manim-cap-text">Finds the true ranking faster</span></div>

Note:
  Using biased coins as stand-in models with a known true order, the Bayesian estimate climbs to the correct ranking faster than the Pass family — reaching agreement in roughly 44 trials where the alternative needs about 70.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/bayes-05-intervals.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/bayes-05-intervals.mp4" aria-label="Overlapping bars mean no winner"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Honest calls</span><span class="manim-cap-text">Overlapping bars? No winner</span></div>

Note:
  And the decision rule the error bars buy you: when two credible intervals overlap, don't declare a winner. That single rule collapses a noisy 20-way leaderboard into a few honest tied groups.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/bayes-06-close.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/bayes-06-close.mp4" aria-label="Don't Pass@k. Use Bayes@N"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Takeaway</span><span class="manim-cap-text">Don't Pass@k &mdash; use Bayes@N</span></div>

Note:
  The one-line takeaway: don't report Pass@k, report Bayes@N — a score with a credible interval, from the same trial data you already collected.

---

<div class="conf-kicker">From one model to many</div>
<h2>An error bar fixes one score &mdash; ranking many models is harder</h2>
<p class="conf-lead">Now line up twenty reasoning models and ask who's best. With each model random, the leaderboard itself becomes a moving target.</p>

<div class="conf-callout amber" style="margin-top: 24px;">Which ranking can you trust &mdash; and how much does it depend on your sampling budget?</div>

Note:
  Bayes@N fixes the score for a single model. But a leaderboard compares many models at once, each of them random — so the whole ranking wobbles. The last paper asks which ranking method to trust, and how that answer changes with your trial budget.

---

<div class="conf-paper-divider">
  <div class="conf-paper-venue">
    <span class="conf-venue-chip"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/acl.svg" alt="ACL 2026" /></span>
    <span class="conf-paper-venue-name">ACL 2026</span>
  </div>
  <h2>Ranking reasoning LLMs under test-time scaling</h2>
  <figure class="conf-paper-shot">
    <img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/header_rank.webp" alt="Paper headline: Ranking Reasoning LLMs under Test-Time Scaling. Mohsen Hariri, Michael Hinczewski, Jing Ma, Vipin Chaudhary." />
  </figure>
  <p class="conf-paper-foot">Rank the distribution, not a single sample &mdash; and say how shaky the ranking is.</p>
</div>

Note:
  The fourth paper, also at ACL, is about ranking many models under repeated sampling. It treats all those tries as one big dataset, and gives a recipe: pick a clear target to rank toward, measure how stable the ranking is, and disclose any shortcut that bends it.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-01-single-trial.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-01-single-trial.mp4" aria-label="One try, green or red"></video>
<div class="manim-cap"><span class="manim-cap-kicker">One draw</span><span class="manim-cap-text">One try: green or red</span></div>

Note:
  Twenty models each answer one hard question once — a single fragile column of green checks and red crosses. That's the evidence a one-shot leaderboard is built on.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-02-resamples.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-02-resamples.mp4" aria-label="Sample again, ranks shuffle"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The flip</span><span class="manim-cap-text">Sample again, ranks shuffle</span></div>

Note:
  Re-sample a few times and the top of the board visibly shuffles — near-ties swap, first place moves. So: when one try is this noisy, what should a rank even mean?

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-03-tensor.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-03-tensor.mp4" aria-label="Every try in one data block"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The data block</span><span class="manim-cap-text">Every try in one data block</span></div>

Note:
  The fix starts by keeping everything: a 3D block whose axes are models, questions, and repeated attempts. In the paper that's 20 models × 30 questions × 80 tries across four math contests — 192,000 runs of evidence instead of one.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-04-pairwise.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-04-pairwise.mp4" aria-label="Head-to-head matchups"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Different lenses</span><span class="manim-cap-text">Head-to-head matchups</span></div>

Note:
  There are many ways to read that block. One lens just averages each model's score; another, shown here, compares models head-to-head question by question, like a tournament. Different lenses, same data — and they don't always agree.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-05-consensus.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-05-consensus.mp4" aria-label="At full budget, methods agree"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Consensus</span><span class="manim-cap-text">At full budget, methods agree</span></div>

Note:
  Good news first: when you use the full budget of 80 tries, almost every ranking method lines up with the accuracy target. With enough data, the choice of method barely matters.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-06-one-trial.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-06-one-trial.mp4" aria-label="Squeeze it to one try"></video>
<div class="manim-cap"><span class="manim-cap-kicker">Stress test</span><span class="manim-cap-text">Squeeze it to one try</span></div>

Note:
  Now the stress test: collapse the thick 80-try block down to a single try. The uncertainty bands blow up — this is the low-budget regime most people actually evaluate in.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-07-near-ties.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-07-near-ties.mp4" aria-label="Near-ties flip, big gaps hold"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The wobble</span><span class="manim-cap-text">Near-ties flip, big gaps hold</span></div>

Note:
  And what wobbles is specific: nearly-equal models flip order under the tiniest nudge, while models that are far apart stay locked. The ranking is trustworthy exactly where the gaps are wide.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-08-alignment.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-08-alignment.mp4" aria-label="Narrower isn't always righter"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The catch</span><span class="manim-cap-text">Narrower isn't always righter</span></div>

Note:
  A tempting shortcut at low budget is to add a confident "greedy" answer as a prior. It does steady the ranking — but watch the catch: if the prior disagrees with the sampled behavior, it pulls the ranking confidently in the wrong direction. Narrower is not the same as righter.

---

<video data-autoplay muted playsinline preload="auto" poster="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/posters/rank-09-design-rule.webp" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/videos/rank-09-design-rule.mp4" aria-label="State the target, measure stability, report priors"></video>
<div class="manim-cap"><span class="manim-cap-kicker">The rule</span><span class="manim-cap-text">Target, stability, priors</span></div>

Note:
  So the recipe, and the takeaway of the whole paper: state the target you're ranking toward, measure how stable the ranking is at your budget, and disclose any prior or auxiliary signal — because it can quietly change what "best" means.

---

<div class="conf-kicker">Part 2 &mdash; the takeaway</div>
<h2>Rank the distribution, report the uncertainty</h2>

<div class="conf-grid two">
  <div class="conf-card">
    <div class="conf-num">Don't Pass@k</div>
    <h3>Score with a posterior</h3>
    <p>Bayes@N replaces a point score with a credible interval &mdash; and a rule for when a gap is real, not noise.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">Ranking under TTS</div>
    <h3>Match the method to the budget</h3>
    <p>At full budget rankings agree; at one try the method you trust depends on the target you declare.</p>
  </div>
</div>

Note:
  Part two in one line: rank the distribution, not a single sample, and always report the uncertainty. Bayes@N gives a single model an honest error bar; the ranking paper tells you which leaderboard to trust at your budget. Together with part one, the model is cheap to run and the scores are honest to read.

---

<h2>Both methods are released as installable packages</h2>

<div class="conf-grid two">
  <div class="conf-card">
    <div class="conf-num">DFloat11</div>
    <h3>vllm-df11: DFloat11 weights in vLLM</h3>
    <pre class="conf-code conf-scorio-code" data-no-copy><span class="kw">pip install</span> vllm-df11
<span class="kw">import</span> os
os.environ[<span class="str">"VLLM_PLUGINS"</span>] = <span class="str">"df11"</span>
<span class="kw">from</span> vllm.plugins <span class="kw">import</span> load_general_plugins
<span class="fn">load_general_plugins</span>()
<span class="kw">from</span> vllm <span class="kw">import</span> LLM, SamplingParams
llm = <span class="fn">LLM</span>(
    <span class="arg">model</span>=<span class="str">"/path/to/llama-3.1-8b-it-df11"</span>,
    <span class="arg">load_format</span>=<span class="str">"df11"</span>,
    <span class="arg">dtype</span>=<span class="str">"bfloat16"</span>,
)
outputs = llm.<span class="fn">generate</span>(prompts, sampling_params)</pre>
  </div>
  <div class="conf-card">
    <div class="conf-num">More Keys, Less Values</div>
    <h3>kvq: non-uniform KV-cache quantization</h3>
    <pre class="conf-code conf-scorio-code" data-no-copy><span class="kw">pip install</span> kvq
<span class="kw">from</span> kvq <span class="kw">import</span> KVQConfig, KVQ
config = <span class="fn">KVQConfig</span>(
    <span class="arg">budget</span>=<span class="num">4</span>,
    <span class="arg">model</span>=<span class="str">"meta-llama/Llama-3.1-8B-Instruct"</span>,
    <span class="arg">residual_length</span>=<span class="num">32</span>,
    <span class="arg">group_size</span>={<span class="str">"k"</span>: <span class="num">64</span>, <span class="str">"v"</span>: <span class="num">64</span>},
    <span class="arg">axis</span>={<span class="str">"k"</span>: <span class="num">0</span>, <span class="str">"v"</span>: <span class="num">0</span>},
)
kv_cache = <span class="fn">KVQ</span>(config)
outputs = model.<span class="fn">generate</span>(
    **inputs,
    <span class="arg">past_key_values</span>=kv_cache,
    <span class="arg">use_cache</span>=<span class="kw">True</span>,
)</pre>
  </div>
</div>

Note:
  And both of these methods ship. vllm-df11 registers DFloat11 as a vLLM plugin — set the plugin variable, point load_format at a DFloat11 checkpoint, and vLLM decompresses the weights losslessly during inference, with the rest of your serving stack untouched. kvq gives you a config and a drop-in KVQ cache object that replaces the standard past-key-values and applies the key-heavy bit allocation from the paper. In both cases the surrounding inference code doesn't change — you opt into the memory savings with a couple of lines.

---

<div class="conf-scorio-header">
  <div class="conf-scorio-logo-panel">
    <img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/scorio.svg" alt="Scorio" />
  </div>
  <div class="conf-scorio-heading">
    <h2>Scorio: a Bayesian evaluation and ranking toolkit</h2>
    <a class="conf-scorio-docs" href="https://mohsenhariri.github.io/scorio/">Documentation: mohsenhariri.github.io/scorio ↗</a>
  </div>
</div>

<div class="conf-scorio-layout">
  <div class="conf-scorio-panel">
    <pre class="conf-code conf-scorio-code" data-no-copy><span class="kw">import</span> numpy <span class="kw">as</span> np
<span class="kw">from</span> scorio <span class="kw">import</span> eval, rank
<span class="cm"># Eval (Don't Pass@k): score an M×N matrix</span>
R = np.<span class="fn">array</span>([[0, 1, 1, 0, 1],
              [1, 1, 0, 1, 1]])
mu, sd, lo, hi = eval.<span class="fn">bayes_ci</span>(R, <span class="arg">confidence</span>=<span class="num">0.95</span>)
p,  sd, lo, hi = eval.<span class="fn">pass_at_k_ci</span>(R, <span class="arg">k</span>=<span class="num">1</span>)
<span class="cm"># Rank L models from an L×M×N tensor</span>
S = np.random.<span class="fn">randint</span>(0, 2, (20, 30, 8))
bayes_ranks = rank.<span class="fn">bayes</span>(S)
bt_ranks    = rank.<span class="fn">bradley_terry</span>(S)
graph_ranks = rank.<span class="fn">pagerank</span>(S)</pre>
    <div class="conf-scorio-note">
      <strong>eval</strong> scores a results matrix and returns a mean with a credible interval; <strong>rank</strong> orders \(L\) models from a response tensor. Both take the trial data already collected for Pass@\(k\).
    </div>
    <div class="conf-scorio-use" aria-label="Modules">
      <span class="conf-scorio-app">Evaluation</span>
      <span class="conf-scorio-app">Ranking</span>
      <span class="conf-scorio-app">Credible intervals</span>
    </div>
  </div>
  <div class="conf-api-board">
    <h3>Public API map <span>scorio.eval / scorio.rank / scorio.utils</span></h3>
    <div class="conf-api-group">
      <div class="conf-api-head"><code>scorio.eval</code><span>metrics and uncertainty</span></div>
      <div class="conf-api-columns">
        <p class="conf-api-line"><strong>Bayes/avg</strong> <code>bayes</code><code>bayes_ci</code><code>avg</code><code>avg_ci</code></p>
        <p class="conf-api-line"><strong>Pass/AUC/majority/max</strong> <code>pass_at_k</code><code>pass_hat_k</code><code>pass_at_k_ci</code><code>pass_hat_k_ci</code><code>auc_at_k</code><code>auc_at_k_ci</code><code>maj_at_k</code><code>maj_at_k_ci</code><code>max_at_k</code><code>max_at_k_ci</code></p>
        <p class="conf-api-line"><strong>Generalized pass</strong> <code>g_pass_at_k</code><code>g_pass_at_k_tau</code><code>mg_pass_at_k</code><code>g_pass_at_k_ci</code><code>g_pass_at_k_tau_ci</code><code>mg_pass_at_k_ci</code></p>
        <p class="conf-api-line"><strong>GeoSpectrum</strong> <code>geom_at_k</code><code>geom_at_k_ci</code><code>geom_ds_at_k</code><code>geom_ds_at_k_ci</code><code>geo_spectrum_at_k</code><code>geo_spectrum_at_k_ci</code><code>geo_spectrum_star_at_k</code><code>geo_spectrum_star_at_k_ci</code><code>threshold_spectrum_at_k</code><code>threshold_spectrum_at_k_ci</code></p>
      </div>
    </div>
    <div class="conf-api-group">
      <div class="conf-api-head"><code>scorio.rank</code><span>model ranking families</span></div>
      <div class="conf-api-columns">
        <p class="conf-api-line"><strong>Priors</strong> <code>Prior</code><code>EmpiricalPrior</code><code>GaussianPrior</code><code>LaplacePrior</code><code>CauchyPrior</code><code>UniformPrior</code><code>CustomPrior</code></p>
        <p class="conf-api-line"><strong>Eval rankers</strong> <code>avg</code><code>bayes</code><code>pass_at_k</code><code>pass_hat_k</code><code>g_pass_at_k_tau</code><code>mg_pass_at_k</code></p>
        <p class="conf-api-line"><strong>Point/graph/structure</strong> <code>inverse_difficulty</code><code>rank_centrality</code><code>hodge_rank</code><code>serial_rank</code><code>pagerank</code><code>spectral</code><code>alpharank</code><code>nash</code></p>
        <p class="conf-api-line"><strong>Pairwise/probabilistic</strong> <code>elo</code><code>trueskill</code><code>glicko</code><code>bradley_terry</code><code>bradley_terry_map</code><code>bradley_terry_davidson</code><code>bradley_terry_davidson_map</code><code>rao_kupper</code><code>rao_kupper_map</code></p>
        <p class="conf-api-line"><strong>Bayesian/IRT</strong> <code>thompson</code><code>bayesian_mcmc</code><code>rasch</code><code>rasch_map</code><code>rasch_mml</code><code>rasch_mml_credible</code><code>rasch_2pl</code><code>rasch_2pl_map</code><code>rasch_3pl</code><code>rasch_3pl_map</code><code>dynamic_irt</code></p>
        <p class="conf-api-line"><strong>Voting/Luce</strong> <code>borda</code><code>copeland</code><code>win_rate</code><code>minimax</code><code>schulze</code><code>ranked_pairs</code><code>kemeny_young</code><code>nanson</code><code>baldwin</code><code>majority_judgment</code><code>plackett_luce</code><code>plackett_luce_map</code><code>davidson_luce</code><code>davidson_luce_map</code><code>bradley_terry_luce</code><code>bradley_terry_luce_map</code></p>
      </div>
    </div>
    <div class="conf-api-group">
      <div class="conf-api-head"><code>scorio.utils</code><span>ranking helpers</span></div>
      <p class="conf-api-line"><code>rank_scores</code><code>compare_rankings</code><code>ranking_hash</code><code>unhash_ranking</code><code>lehmer_hash</code><code>lehmer_unhash</code></p>
    </div>
  </div>
</div>

Note:
  The evaluation half ships as one package too, called Scorio. The eval module is the Don't Pass@k scoring — hand it your M-by-N results matrix and it returns a mean with a credible interval. The rank module is the ranking paper — hand it the L-by-M-by-N tensor and it gives you any of the ranking families we compared, from Bayes to Bradley-Terry to PageRank. The input is exactly the trial data you already collected for pass@k, so adopting it costs nothing extra. The board on the right is just the public API — the full menu of estimators and ranking methods.

---

<div class="conf-takeaway-header">
  <img class="conf-takeaway-icon" src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/takeaway.webp" alt="" aria-hidden="true" />
  <div>
    <h2 class="conf-takeaway-title">Takeaways</h2>
  </div>
</div>

<div class="conf-takeaway-grid quad">
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">1</div>
    <h3>DFloat11: shrink weights, change nothing</h3>
    <p>Huffman-code BFloat16's near-empty exponent: ~30% smaller, bit-identical outputs. Fits 405B on one node and buys 5.7&ndash;14.9&times; longer generation.</p>
  </div>
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">2</div>
    <h3>More keys, less values</h3>
    <p>Key weights dominate values in norm, so give the KV budget to keys. K4V2 keeps ~98% accuracy at ~25% less cache; K2V4 falls behind.</p>
  </div>
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">3</div>
    <h3>Don't Pass@k &mdash; score with a posterior</h3>
    <p>Bayes@N replaces point Pass@k with credible intervals &mdash; and a rule for when a gap is real, not sampling noise.</p>
  </div>
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">4</div>
    <h3>Rank the distribution, not one sample</h3>
    <p>The benchmark is a response tensor. At full budget rankings agree; at N=1 the method you trust depends on the target.</p>
  </div>
</div>

Note:
  Four things to walk out with, one per paper. One: compress the weights losslessly. Two: under a fixed KV budget, give the bits to the keys. Three: report a posterior with a credible interval, not a point Pass@k. Four: rank the whole distribution and match the method to your budget. Underneath all four: spend precision where the information is, and quantify the error you can't remove.

---

<div class="conf-closing-layout">
<div class="conf-closing-main">
<h1 class="conf-closing-title">Thank you! Questions?</h1>
<div class="conf-subtitle">Serving reasoning LLMs efficiently and reliably.</div>
<div class="conf-qr-grid" aria-label="Project links">
<div class="conf-qr-item"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/qr_df11.svg" alt="QR code for the DFloat11 paper" /><strong>DFloat11</strong></div>
<div class="conf-qr-item"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/qr_kvq.svg" alt="QR code for the More Keys, Less Values paper" /><strong>More keys, less values</strong></div>
<div class="conf-qr-item"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/qr_bayes.svg" alt="QR code for the Don't Pass@k paper" /><strong>Don't Pass@k</strong></div>
<div class="conf-qr-item"><img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/qr_rank.svg" alt="QR code for the Ranking under Test-Time Scaling paper" /><strong>Ranking &amp; Scorio</strong></div>
</div>
</div>
<div class="conf-ack-stack" aria-label="Acknowledgments">
<div class="conf-ack-item conf-ack-nsf">
<img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/nsf.svg" alt="National Science Foundation" />
<div><h3>NSF Awards</h3><p>2117439 &middot; 2112606 &middot; 2320952</p></div>
</div>
<div class="conf-ack-item conf-ack-cwru">
<img src="/assets/slides/2026-06-19-serving-reasoning-llms-efficiently-and-reliably/cwru_hpc_logo.webp" alt="Case Western Reserve University" />
<div><h3>High Performance Computing (HPC)</h3><p>Center at CWRU</p></div>
</div>
</div>
</div>

Note:
  Thank you. Each QR code goes to one of the four papers, and the code for the evaluation half is the Scorio library. Happy to take questions — and especially happy to talk about collaborations.

Navigation