Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation (Slides)

.reveal .slides {
    text-align: left;
  }

.reveal section.conf-slide {
    box-sizing: border-box;
    width: 100%;
    height: 100%;
    padding: 54px 70px 44px;
    color: var(--conf-ink);
    background:
      linear-gradient(135deg, rgba(37, 99, 235, 0.055), rgba(255, 255, 255, 0) 32%),
      linear-gradient(225deg, rgba(15, 118, 110, 0.055), rgba(255, 255, 255, 0) 36%),
      #f8fafc;
    overflow: hidden;
  }

.reveal section.conf-title {
    padding: 64px 82px 54px;
  }

.reveal .conf-slide h1,
  .reveal .conf-slide h2,
  .reveal .conf-slide h3 {
    color: var(--conf-ink);
    letter-spacing: 0;
    text-transform: none;
  }

.reveal .conf-slide h1 {
    font-size: 76px;
    line-height: 1.03;
    margin: 0;
  }

.reveal .conf-slide h2 {
    font-size: 46px;
    line-height: 1.08;
    margin: 0 0 26px;
  }

.reveal .conf-slide h3 {
    font-size: 25px;
    line-height: 1.18;
    margin: 0 0 10px;
  }

.reveal .conf-slide p,
  .reveal .conf-slide li {
    color: var(--conf-muted);
    font-size: 24px;
    line-height: 1.36;
  }

.reveal .conf-slide strong {
    color: var(--conf-ink);
    font-weight: 720;
  }

.reveal .conf-kicker {
    color: var(--conf-teal);
    font-size: 18px;
    font-weight: 760;
    letter-spacing: 0.08em;
    margin-bottom: 14px;
    text-transform: uppercase;
  }

.reveal .conf-subtitle {
    max-width: 950px;
    color: var(--conf-muted);
    font-size: 31px;
    line-height: 1.32;
    margin-top: 26px;
  }

.reveal .conf-title-grid {
    display: grid;
    grid-template-columns: minmax(0, 1100px) 286px;
    gap: 42px;
    align-content: center;
    align-items: center;
    height: 100%;
    max-width: none;
  }

.reveal .conf-title-main {
    min-width: 0;
  }

.reveal .conf-title-qr {
    box-sizing: border-box;
    justify-self: end;
    width: 286px;
    padding: 18px;
    text-align: center;
  }

.reveal .conf-title-qr img {
    display: block;
    width: 238px;
    height: 238px;
    max-width: none;
    max-height: none;
    object-fit: contain;
    margin: 0 auto 13px;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .conf-title-qr strong {
    display: block;
    color: var(--conf-ink);
    font-size: 21px;
    line-height: 1.15;
  }

.reveal .conf-acl-logo {
    display: block;
    width: 300px;
    max-width: none;
    height: auto;
    margin: 0 0 48px;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .conf-meta {
    color: var(--conf-muted);
    font-size: 21px;
    line-height: 1.45;
    margin-top: 24px;
  }

.reveal .conf-event-line {
    color: var(--conf-teal);
    font-size: 23px;
    font-weight: 760;
    line-height: 1.25;
    margin-top: 18px;
  }

.reveal .conf-logo-row {
    display: flex;
    align-items: center;
    gap: 24px;
    margin-top: 40px;
    flex-wrap: nowrap;
  }

.reveal .conf-logo-badge {
    display: flex;
    align-items: center;
    justify-content: center;
    height: 84px;
    padding: 10px 18px;
    border-radius: 14px;
    background: rgba(255, 255, 255, 0.92);
    box-shadow: 0 18px 46px rgba(15, 23, 42, 0.12);
  }

.reveal .conf-logo-badge.nsf {
    width: 104px;
    height: 92px;
    padding: 7px;
  }

.reveal .conf-logo-badge.scorio {
    width: 250px;
  }

.reveal .conf-logo-row img {
    display: block;
    width: auto;
    max-width: none;
    height: 100%;
    max-height: none;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-panel {
    border: 1px solid rgba(15, 23, 42, 0.14);
    border-radius: 8px;
    background: var(--conf-panel);
    box-shadow: 0 18px 46px rgba(15, 23, 42, 0.08);
  }

.reveal .conf-title-card {
    padding: 24px;
  }

.reveal .conf-title-card img {
    width: 100%;
    height: 370px;
    object-fit: contain;
    display: block;
  }

.reveal .conf-title-metric {
    display: grid;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    gap: 12px;
    margin-top: 20px;
  }

.reveal .conf-title-metric p,
  .reveal .conf-metric p {
    margin: 0;
  }

.reveal .conf-title-metric strong,
  .reveal .conf-metric strong {
    display: block;
    color: var(--conf-blue);
    font-size: 34px;
    line-height: 1;
  }

.reveal .conf-title-metric span,
  .reveal .conf-metric span {
    display: block;
    color: var(--conf-muted);
    font-size: 16px;
    line-height: 1.22;
    margin-top: 7px;
  }

.reveal .conf-chip-row {
    display: flex;
    flex-wrap: wrap;
    gap: 10px;
    margin-top: 18px;
  }

.reveal .conf-chip {
    display: inline-flex;
    align-items: center;
    border: 1px solid rgba(15, 23, 42, 0.14);
    border-radius: 999px;
    color: var(--conf-muted);
    background: rgba(255, 255, 255, 0.74);
    font-size: 16px;
    font-weight: 720;
    line-height: 1;
    padding: 9px 13px;
  }

.reveal .conf-chip.blue {
    color: var(--conf-blue);
    background: var(--conf-blue-soft);
    border-color: rgba(37, 99, 235, 0.24);
  }

.reveal .conf-chip.teal {
    color: var(--conf-teal);
    background: var(--conf-teal-soft);
    border-color: rgba(15, 118, 110, 0.24);
  }

.reveal .conf-chip.amber {
    color: var(--conf-amber);
    background: var(--conf-amber-soft);
    border-color: rgba(180, 83, 9, 0.24);
  }

.reveal .conf-grid {
    display: grid;
    gap: 18px;
  }

.reveal .conf-grid.two {
    grid-template-columns: repeat(2, minmax(0, 1fr));
  }

.reveal .conf-grid.three {
    grid-template-columns: repeat(3, minmax(0, 1fr));
  }

.reveal .conf-grid.four {
    grid-template-columns: repeat(4, minmax(0, 1fr));
  }

.reveal .conf-card {
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.78);
    padding: 20px 22px;
    min-height: 138px;
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-card p {
    margin: 0;
    font-size: 20px;
  }

.reveal .conf-num {
    color: var(--conf-blue);
    font-size: 18px;
    font-weight: 780;
    margin-bottom: 12px;
  }

.reveal .conf-callout {
    border-left: 5px solid var(--conf-blue);
    border-radius: 8px;
    background: var(--conf-blue-soft);
    padding: 18px 22px;
    color: var(--conf-ink);
    font-size: 25px;
    line-height: 1.34;
  }

.reveal .conf-callout.teal {
    border-left-color: var(--conf-teal);
    background: var(--conf-teal-soft);
  }

.reveal .conf-callout.amber {
    border-left-color: var(--conf-amber);
    background: var(--conf-amber-soft);
  }

.reveal .conf-callout.rose {
    border-left-color: var(--conf-rose);
    background: var(--conf-rose-soft);
  }

.reveal .conf-big-question {
    margin-top: 24px;
    padding: 26px 30px;
    color: var(--conf-ink);
    font-size: 35px;
    line-height: 1.2;
  }

.reveal .conf-equation {
    display: flex;
    align-items: center;
    justify-content: center;
    box-sizing: border-box;
    min-width: 0;
    min-height: 112px;
    max-width: 100%;
    overflow-x: auto;
    border: 1px solid rgba(15, 23, 42, 0.12);
    border-radius: 8px;
    background: rgba(15, 23, 42, 0.04);
    color: var(--conf-ink);
    font-size: 31px;
    line-height: 1.25;
    padding: 18px 22px;
  }

.reveal .conf-equation .katex { max-width: 100%; }

.reveal .conf-equation.sm {
    font-size: 24px;
  }

.reveal .conf-figure {
    border: 1px solid rgba(15, 23, 42, 0.12);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.84);
    padding: 16px;
  }

.reveal .conf-figure img {
    display: block;
    width: 100%;
    height: 100%;
    object-fit: contain;
  }

.reveal .conf-figure figcaption {
    color: var(--conf-muted);
    font-size: 16px;
    line-height: 1.25;
    margin-top: 10px;
  }

.reveal .conf-fig-short {
    height: 430px;
  }

.reveal .conf-fig-mid {
    height: 540px;
  }

.reveal .conf-fig-wide {
    height: 465px;
  }

.reveal .conf-fig-tall {
    height: 610px;
  }

.reveal .conf-split {
    display: grid;
    grid-template-columns: minmax(0, 0.98fr) minmax(420px, 0.82fr);
    gap: 30px;
    align-items: stretch;
  }

.reveal .conf-split.reverse {
    grid-template-columns: minmax(420px, 0.76fr) minmax(0, 1fr);
  }

.reveal .conf-metric {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 14px;
    margin-top: 18px;
  }

.reveal .conf-metric .conf-card {
    min-height: 104px;
  }

.reveal .conf-mini-table,
  .reveal .conf-table {
    width: 100%;
    border-collapse: collapse;
  }

.reveal .conf-mini-table {
    font-size: 18px;
  }

.reveal .conf-table {
    font-size: 20px;
  }

.reveal .conf-mini-table th,
  .reveal .conf-mini-table td,
  .reveal .conf-table th,
  .reveal .conf-table td {
    border-bottom: 1px solid rgba(15, 23, 42, 0.12);
    padding: 11px 10px;
    text-align: right;
    color: var(--conf-muted);
  }

.reveal .conf-mini-table th:first-child,
  .reveal .conf-mini-table td:first-child,
  .reveal .conf-table th:first-child,
  .reveal .conf-table td:first-child {
    text-align: left;
  }

.reveal .conf-mini-table th,
  .reveal .conf-table th {
    color: var(--conf-ink);
    font-weight: 760;
  }

.reveal .conf-win {
    color: var(--conf-green) !important;
    font-weight: 760;
    background: rgba(21, 128, 61, 0.08);
  }

.reveal .conf-loss {
    color: var(--conf-rose) !important;
  }

.reveal .conf-flow {
    display: grid;
    grid-template-columns: repeat(4, minmax(0, 1fr));
    gap: 12px;
    margin-top: 18px;
  }

.reveal .conf-flow .conf-card {
    min-height: 148px;
  }

.reveal .conf-flow h3 {
    font-size: 23px;
  }

.reveal .conf-list {
    margin: 0;
    padding-left: 1.1em;
  }

.reveal .conf-list li {
    margin: 0 0 12px;
  }

.reveal .conf-footer-note {
    position: absolute;
    left: 70px;
    right: 70px;
    bottom: 28px;
    color: rgba(71, 85, 105, 0.78);
    font-size: 16px;
  }

.reveal .conf-takeaway-header {
    display: flex;
    align-items: center;
    gap: 24px;
    margin: 0 0 34px;
  }

.reveal .conf-takeaway-icon {
    display: block;
    width: 104px;
    height: 104px;
    max-width: none;
    max-height: none;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-takeaway-title {
    color: var(--conf-ink);
    font-size: 62px;
    line-height: 0.98;
    margin: 0;
  }

.reveal .conf-takeaway-subtitle {
    color: var(--conf-teal);
    font-size: 28px;
    font-weight: 760;
    line-height: 1.12;
    margin-top: 12px;
  }

.reveal .conf-takeaway-grid {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 22px;
  }

.reveal .conf-takeaway-card {
    position: relative;
    min-height: 250px;
    padding: 30px 30px 28px;
    overflow: hidden;
    border-top: 5px solid var(--conf-teal);
    background:
      linear-gradient(180deg, rgba(15, 118, 110, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
    box-shadow: 0 18px 42px rgba(15, 23, 42, 0.09);
  }

.reveal .conf-takeaway-card:nth-child(2) {
    border-top-color: var(--conf-blue);
    background:
      linear-gradient(180deg, rgba(37, 99, 235, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
  }

.reveal .conf-takeaway-card:nth-child(3) {
    border-top-color: var(--conf-amber);
    background:
      linear-gradient(180deg, rgba(180, 83, 9, 0.08), rgba(255, 255, 255, 0.82) 42%),
      rgba(255, 255, 255, 0.84);
  }

.reveal .conf-takeaway-num {
    display: inline-grid;
    place-items: center;
    width: 42px;
    height: 42px;
    margin-bottom: 20px;
    border-radius: 50%;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 22px;
    font-weight: 820;
  }

.reveal .conf-takeaway-card:nth-child(2) .conf-takeaway-num {
    background: var(--conf-blue-soft);
    color: var(--conf-blue);
  }

.reveal .conf-takeaway-card:nth-child(3) .conf-takeaway-num {
    background: var(--conf-amber-soft);
    color: var(--conf-amber);
  }

.reveal .conf-takeaway-card h3 {
    font-size: 30px;
    line-height: 1.08;
    min-height: 42px;
    margin-bottom: 16px;
  }

.reveal .conf-takeaway-card p {
    font-size: 25px;
    line-height: 1.34;
    min-height: 128px;
  }

.reveal .conf-closing-layout {
    display: grid;
    grid-template-rows: auto minmax(0, 1fr) auto;
    gap: 26px;
    align-items: stretch;
    height: 100%;
  }

.reveal .conf-closing-main {
    display: grid;
    justify-items: center;
    align-content: start;
    text-align: center;
    padding-top: 8px;
  }

.reveal .conf-closing-title {
    font-size: 72px;
    line-height: 1.03;
    margin: 0 0 18px;
  }

.reveal .conf-closing-main .conf-subtitle {
    max-width: none;
    margin-top: 0;
  }

.reveal .conf-closing-authors {
    color: var(--conf-muted);
    font-size: 22px;
    line-height: 1.25;
    margin-top: 12px;
  }

.reveal .conf-qr-grid {
    display: grid;
    grid-template-columns: repeat(4, 260px);
    justify-content: center;
    gap: 24px;
    width: min(1120px, 100%);
    margin: 32px auto 0;
  }

.reveal .conf-qr-item {
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.82);
    padding: 18px;
    text-align: center;
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-qr-item img {
    display: block;
    width: 210px;
    height: 210px;
    max-width: none;
    max-height: none;
    object-fit: contain;
    margin: 0 auto 12px;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .conf-qr-item strong {
    display: block;
    color: var(--conf-ink);
    font-size: 20px;
    line-height: 1.1;
  }

.reveal .conf-contact-line {
    max-width: 1220px;
    margin: 18px auto 0;
    color: var(--conf-muted);
    font-size: 20px;
    line-height: 1.25;
  }

.reveal .conf-contact-email {
    color: var(--conf-blue);
    font-weight: 780;
    white-space: nowrap;
  }

.reveal .conf-ack-stack {
    display: grid;
    grid-template-columns: 450px 760px;
    justify-content: center;
    align-items: center;
    gap: 34px;
    width: 100%;
    margin: 0 auto;
    padding: 0;
  }

.reveal .conf-ack-item {
    display: grid;
    grid-template-columns: auto minmax(0, 1fr);
    gap: 22px;
    align-items: center;
    border: 0;
    border-radius: 0;
    background: transparent;
    padding: 0;
    box-shadow: none;
  }

.reveal .conf-ack-cwru {
    grid-template-columns: 290px minmax(0, 1fr);
  }

.reveal .conf-ack-item img {
    display: block;
    width: auto;
    max-width: none;
    max-height: none;
    object-fit: contain;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-ack-nsf img {
    width: 84px;
    height: 84px;
  }

.reveal .conf-ack-cwru img {
    width: 270px;
    height: auto;
  }

.reveal .conf-ack-item h3 {
    font-size: 26px;
    line-height: 1.08;
    margin: 0 0 8px;
    white-space: nowrap;
  }

.reveal .conf-ack-item p {
    font-size: 22px;
    line-height: 1.18;
    margin: 0;
    white-space: nowrap;
  }

.reveal .slides section.conf-slide .conf-acl-logo {
    width: 300px;
    max-width: none;
    height: auto;
    margin: 0 0 48px;
    border-radius: 0;
    box-shadow: none;
  }

.reveal .slides section.conf-slide .conf-logo-row img {
    width: auto;
    max-width: none;
    height: 64px;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .slides section.conf-slide .conf-logo-badge.nsf img {
    width: 76px;
    height: 76px;
  }

.reveal .slides section.conf-slide .conf-logo-badge.scorio img {
    width: 214px;
    height: auto;
  }

.reveal .slides section.conf-slide .conf-qr-item img {
    width: 210px;
    height: 210px;
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0 auto 12px;
  }

.reveal .slides section.conf-slide .conf-title-qr img {
    width: 238px;
    height: 238px;
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0 auto 13px;
  }

.reveal .slides section.conf-slide .conf-ack-item img {
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .slides section.conf-slide .conf-ack-nsf img {
    width: 84px;
    height: 84px;
  }

.reveal .slides section.conf-slide .conf-ack-cwru img {
    width: 270px;
    height: auto;
  }

.reveal .conf-title-thesis {
    max-width: 1080px;
    color: var(--conf-muted);
    font-size: 29px;
    line-height: 1.28;
    margin-top: 28px;
  }

.reveal .conf-slide .conf-lead {
    max-width: 1210px;
    color: var(--conf-muted);
    font-size: 30px;
    line-height: 1.28;
    margin: -8px 0 28px;
  }

.reveal .conf-slide .conf-small {
    color: var(--conf-muted);
    font-size: 18px;
    line-height: 1.3;
  }

.reveal .conf-stat-row {
    display: grid;
    grid-template-columns: repeat(4, minmax(0, 1fr));
    gap: 14px;
    margin-top: 20px;
  }

.reveal .conf-stat {
    border: 1px solid rgba(15, 23, 42, 0.12);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.78);
    padding: 18px 18px 16px;
    min-height: 112px;
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-stat strong {
    display: block;
    color: var(--conf-blue);
    font-size: 43px;
    line-height: 1;
  }

.reveal .conf-stat span {
    display: block;
    color: var(--conf-muted);
    font-size: 17px;
    line-height: 1.22;
    margin-top: 9px;
  }

.reveal .conf-slide .conf-table.compact {
    font-size: 17px;
  }

.reveal .conf-slide .conf-table.compact th,
  .reveal .conf-slide .conf-table.compact td {
    padding: 8px 8px;
  }

.reveal .conf-slide .conf-table.wide-text td,
  .reveal .conf-slide .conf-table.wide-text th {
    text-align: left;
  }

.reveal .conf-slide .conf-table.wide-text td:last-child,
  .reveal .conf-slide .conf-table.wide-text th:last-child {
    text-align: right;
  }

.reveal .conf-slide .conf-table .conf-method-cell {
    color: var(--conf-ink);
    font-weight: 720;
  }

.reveal .conf-two-thirds {
    display: grid;
    grid-template-columns: minmax(0, 1.05fr) minmax(420px, 0.72fr);
    gap: 28px;
    align-items: stretch;
  }

.reveal .conf-figure-grid {
    display: grid;
    grid-template-columns: minmax(0, 1fr) minmax(390px, 0.52fr);
    gap: 24px;
    align-items: stretch;
  }

.reveal .conf-figure-grid.reverse {
    grid-template-columns: minmax(390px, 0.52fr) minmax(0, 1fr);
  }

.reveal .conf-figure-grid .conf-figure {
    height: 590px;
  }

.reveal .conf-figure-grid .conf-figure.short {
    height: 500px;
  }

.reveal .conf-stack {
    display: grid;
    gap: 16px;
    align-content: start;
  }

.reveal .conf-target-card {
    min-height: 210px;
  }

.reveal .conf-target-card .conf-equation {
    min-height: 82px;
    margin-bottom: 16px;
    font-size: 27px;
  }

.reveal .conf-method-map {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 18px;
    margin-top: 20px;
  }

.reveal .conf-method-map .conf-card {
    min-height: 255px;
  }

.reveal .conf-method-map .conf-equation {
    min-height: 76px;
    font-size: 23px;
    margin-bottom: 14px;
  }

.reveal .conf-rule-grid {
    display: grid;
    grid-template-columns: repeat(5, minmax(0, 1fr));
    gap: 13px;
    margin-top: 26px;
  }

.reveal .conf-rule-grid .conf-card {
    min-height: 190px;
    padding: 18px 18px;
  }

.reveal .conf-rule-grid h3 {
    font-size: 22px;
  }

.reveal .conf-code {
    box-sizing: border-box;
    max-width: 100%;
    overflow-x: auto;
    border: 1px solid rgba(15, 23, 42, 0.14);
    border-radius: 8px;
    background: #111827;
    color: #e5e7eb;
    font-size: 20px;
    line-height: 1.42;
    margin: 0;
    padding: 24px 26px;
    white-space: pre;
  }

.reveal .conf-code .kw {
    color: #67e8f9;
  }

.reveal .conf-code .fn {
    color: #a7f3d0;
  }

.reveal .conf-code .arg {
    color: #fbbf24;
  }

.reveal .conf-qrs-small {
    display: grid;
    grid-template-columns: repeat(2, 180px);
    gap: 18px;
    justify-content: start;
    margin-top: 16px;
  }

.reveal .conf-qrs-small .conf-qr-item {
    padding: 14px;
  }

.reveal .conf-qrs-small .conf-qr-item img {
    width: 144px;
    height: 144px;
  }

.reveal .conf-scorio-header {
    display: grid;
    grid-template-columns: 250px minmax(0, 1fr);
    gap: 24px;
    align-items: center;
    margin-bottom: 18px;
  }

.reveal .conf-scorio-logo-panel {
    display: flex;
    align-items: center;
    justify-content: flex-start;
    height: 104px;
  }

.reveal .conf-scorio-logo-panel img {
    display: block;
    width: 202px;
    height: auto;
    max-width: none;
    max-height: none;
    border-radius: 0;
    box-shadow: none;
    margin: 0;
  }

.reveal .conf-scorio-header h2 {
    font-size: 40px;
    line-height: 1.08;
    margin: 0;
  }

.reveal .conf-scorio-app {
    border: 1px solid rgba(15, 118, 110, 0.22);
    border-radius: 999px;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 17px;
    font-weight: 780;
    line-height: 1;
    padding: 11px 15px;
    text-align: center;
  }

.reveal .conf-scorio-layout {
    display: grid;
    grid-template-columns: minmax(0, 0.66fr) minmax(0, 1.34fr);
    gap: 18px;
    align-items: stretch;
  }

.reveal .conf-scorio-panel,
  .reveal .conf-api-board {
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.8);
    box-shadow: 0 12px 28px rgba(15, 23, 42, 0.055);
  }

.reveal .conf-scorio-panel {
    padding: 18px;
  }

.reveal .conf-scorio-panel h3,
  .reveal .conf-api-board h3 {
    font-size: 23px;
    line-height: 1.08;
    margin: 0 0 12px;
  }

.reveal .conf-scorio-code {
    font-size: 15px;
    line-height: 1.34;
    padding: 16px 18px;
  }

.reveal .conf-scorio-note {
    color: var(--conf-muted);
    font-size: 18px;
    line-height: 1.26;
    margin-top: 14px;
  }

.reveal .conf-scorio-note strong {
    color: var(--conf-ink);
  }

.reveal .conf-scorio-use {
    display: grid;
    grid-template-columns: repeat(3, minmax(0, 1fr));
    gap: 8px;
    margin-top: 14px;
  }

.reveal .conf-scorio-use .conf-scorio-app {
    display: grid;
    place-items: center;
    min-height: 52px;
    border: 1px solid rgba(15, 118, 110, 0.22);
    border-radius: 999px;
    background: var(--conf-teal-soft);
    color: var(--conf-teal);
    font-size: 15px;
    font-weight: 780;
    line-height: 1.08;
    padding: 10px 12px;
    text-align: center;
  }

.reveal .conf-api-board {
    padding: 16px 18px 14px;
  }

.reveal .conf-api-board h3 {
    display: flex;
    align-items: baseline;
    justify-content: space-between;
    gap: 16px;
  }

.reveal .conf-api-board h3 span {
    color: var(--conf-muted);
    font-size: 15px;
    font-weight: 680;
  }

.reveal .conf-api-group {
    padding: 8px 0;
    border-top: 1px solid rgba(15, 23, 42, 0.11);
  }

.reveal .conf-api-head {
    display: flex;
    align-items: baseline;
    gap: 12px;
    margin-bottom: 4px;
  }

.reveal .conf-api-head code {
    color: var(--conf-ink);
    font-size: 16px;
    font-weight: 820;
  }

.reveal .conf-api-head span {
    color: var(--conf-teal);
    font-size: 14px;
    font-weight: 760;
  }

.reveal .conf-api-columns {
    display: grid;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    column-gap: 16px;
    row-gap: 2px;
  }

.reveal .conf-slide .conf-api-line {
    color: var(--conf-muted);
    font-size: 13px;
    line-height: 1.22;
    margin: 0 0 5px;
  }

.reveal .conf-slide .conf-api-line strong {
    display: inline;
    color: var(--conf-ink);
    font-size: 13px;
    font-weight: 800;
  }

.reveal .conf-slide .conf-api-line code {
    display: inline-block;
    border-radius: 5px;
    background: rgba(15, 23, 42, 0.055);
    color: var(--conf-ink);
    font-size: 11.5px;
    line-height: 1.18;
    margin: 0 2px 3px 0;
    padding: 2px 5px;
  }

.reveal .conf-slide .conf-fit-list li {
    font-size: 21px;
    line-height: 1.28;
    margin-bottom: 10px;
  }

.reveal .conf-slide .conf-tight-list li {
    font-size: 19px;
    line-height: 1.24;
    margin-bottom: 8px;
  }

.reveal .conf-banner {
    display: grid;
    grid-template-columns: auto minmax(0, 1fr);
    gap: 18px;
    align-items: center;
    border: 1px solid rgba(15, 23, 42, 0.13);
    border-radius: 8px;
    background: rgba(255, 255, 255, 0.78);
    padding: 18px 22px;
    margin-top: 20px;
  }

.reveal .conf-banner-number {
    color: var(--conf-blue);
    font-size: 54px;
    font-weight: 780;
    line-height: 1;
  }

.reveal .conf-banner p {
    margin: 0;
    font-size: 23px;
    line-height: 1.26;
  }

.reveal .conf-label {
    display: inline-flex;
    align-items: center;
    border-radius: 999px;
    background: rgba(15, 23, 42, 0.08);
    color: var(--conf-ink);
    font-size: 15px;
    font-weight: 760;
    line-height: 1;
    padding: 8px 10px;
    text-transform: uppercase;
  }

.reveal .conf-figure img.no-stretch {
    height: auto;
    max-height: 100%;
  }

/* deck-specific additions: text venue mark (no ICLR logo asset exists) */
  .reveal .conf-venue-mark {
    display: inline-flex;
    align-items: baseline;
    gap: 16px;
    margin: 0 0 36px;
    line-height: 1;
  }

.reveal .conf-venue-mark b {
    font-size: 58px;
    font-weight: 840;
    letter-spacing: 0.04em;
    color: var(--conf-ink);
  }

.reveal .conf-venue-mark i {
    font-style: normal;
    font-size: 58px;
    font-weight: 840;
    letter-spacing: 0.04em;
    color: var(--conf-blue);
  }

.reveal .conf-venue-mark small {
    align-self: center;
    font-size: 17px;
    font-weight: 760;
    letter-spacing: 0.16em;
    text-transform: uppercase;
    color: var(--conf-muted);
  }
</style>

<div class="conf-title-grid">
  <div class="conf-title-main">
    <img class="conf-acl-logo" style="width: 148px;" src="/assets/slides/2026-01-25-bayes-at-n/iclr.svg" alt="ICLR 2026" />
    <h1>Don't Pass@<i>k</i>:<br>A Bayesian Framework for Large Language Model Evaluation</h1>
    <div class="conf-event-line"> Pavilion 3 | Apr 25, 2026 | 2:15 PM – 4:45 PM  </div>
    <div class="conf-meta">
      <strong>Mohsen Hariri</strong>, Amirhossein Samandar, Michael Hinczewski, Vipin Chaudhary
    </div>
    <div class="conf-logo-row">
      <span class="conf-logo-badge"><img src="/assets/slides/2026-01-25-bayes-at-n/cwru.svg" alt="Case Western Reserve University" /></span>
      <span class="conf-logo-badge scorio"><img src="/assets/slides/2026-01-25-bayes-at-n/scorio.svg" alt="Scorio" /></span>
      <span class="conf-logo-badge nsf"><img src="/assets/slides/2026-01-25-bayes-at-n/nsf.svg" alt="National Science Foundation" /></span>
    </div>
  </div>
  <aside class="conf-title-qr conf-panel" aria-label="Interactive poster QR code">
    <img src="/assets/slides/2026-01-25-bayes-at-n/qr_slide.svg" alt="QR code for the interactive poster" />
    <strong>Scan for interactive slides</strong>
  </aside>
</div>

Note:
  The one message: how we score sampled LLM outputs decides which model looks best, and Pass@k is the wrong object to score.
  We replace it with a posterior over each model's success probability, which gives stable rankings and an explicit rule for when a gap is real.

---

<div class="conf-kicker">Motivation</div>
<h2>Pass@<i>k</i> ranks the wrong quantity</h2>
<p class="conf-lead">
  Pass@\(k\) estimates the chance of at least one success in \(k\) tries. That is not a model's underlying success probability, and as a ranking signal it is fragile.
</p>

<div class="conf-grid three">
  <div class="conf-card">
    <div class="conf-num">01</div>
    <h3>Unstable at low budget</h3>
    <p>On small or costly benchmarks (tens of AIME problems), rankings swing with decoding and sampling noise.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">02</div>
    <h3>No closed-form uncertainty</h3>
    <p>Variance needs bootstrapping; there is no transparent rule for whether a gap is meaningful or noise.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">03</div>
    <h3>Binary only</h3>
    <p>It does not extend to partial credit, refusals, or rubric-defined outcomes without ad hoc aggregation.</p>
  </div>
</div>

<div class="conf-callout" style="margin-top: 28px;">
  Treat evaluation as statistical inference: estimate the success probability and report how sure you are.
</div>

Note:
  Start from the pain. Pass@k answers "can the model do this in k attempts," which is a coverage question, not a quality estimate.
  When budgets are small, that estimator has high variance, so leaderboards reshuffle for reasons that have nothing to do with the models.

---

<div class="conf-kicker">Overview</div>
<h2>One posterior fixes four recurring problems</h2>

<div class="conf-grid four">
  <div class="conf-card">
    <div class="conf-num">1</div>
    <h3>Convergence</h3>
    <p>Reach a stable ranking with far fewer trials than Pass@\(k\) and its variants.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">2</div>
    <h3>Credible intervals</h3>
    <p>Do not call a winner when intervals overlap; spend more trials only when needed.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">3</div>
    <h3>Prior knowledge</h3>
    <p>Reuse evidence from related tasks or model versions to sharpen small-\(N\) rankings.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">4</div>
    <h3>Categorical scoring</h3>
    <p>Unify 0/1 with partial credit, refusals, and rubric-defined outcomes.</p>
  </div>
</div>

<div class="conf-banner">
  <div class="conf-banner-number">4</div>
  <p>datasets (AIME'24, AIME'25, HMMT'25, BrUMO'25) and 11 reasoning LLMs are evaluated on the same dense repeated-trial evidence, up to \(N=80\) trials per question.</p>
</div>

Note:
  These are not four separate methods. A single Bayesian estimator gives all four, because it hands you a full posterior instead of one point score.

---

<div class="conf-kicker">Method</div>
<h2>Bayes@<i>N</i> turns scoring into posterior inference</h2>

<div class="conf-rule-grid">
  <div class="conf-card">
    <div class="conf-num">1</div>
    <h3>Sample</h3>
    <p>Run \(N\) attempts per question.</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">2</div>
    <h3>Score</h3>
    <p>Label each attempt: binary or a rubric category \(0,\dots,C\).</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">3</div>
    <h3>Tally</h3>
    <p>Count category hits \(n_{\alpha k}\) per question \(\alpha\).</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">4</div>
    <h3>Posterior</h3>
    <p>Form a Dirichlet posterior over outcome probabilities \(\boldsymbol{\pi}_\alpha\).</p>
  </div>
  <div class="conf-card">
    <div class="conf-num">5</div>
    <h3>Report</h3>
    <p>Output mean \(\mu\), uncertainty \(\sigma\), and credible intervals.</p>
  </div>
</div>

<div class="conf-callout teal" style="margin-top: 26px;">
  The unit of evidence is the results matrix \( R \in \{0,\dots,C\}^{M\times N} \): \(M\) questions, \(N\) repeated trials, one rubric label per cell.
</div>

Note:
  None of these five steps needs a new decoder or a new benchmark. It is a scoring layer on top of the trials you already run.

---

<div class="conf-kicker">Estimator</div>
<h2>A Dirichlet posterior gives a mean and exact uncertainty</h2>

<div class="conf-grid two">
  <div class="conf-card">
    <h3>Per-question posterior</h3>
    <div class="conf-equation sm">
      \[ \boldsymbol{\pi}_\alpha \mid R_\alpha \sim \mathrm{Dir}\!\left(n_{\alpha 0}+n^{0}_{\alpha 0},\,\dots,\,n_{\alpha C}+n^{0}_{\alpha C}\right) \]
    </div>
    <p>Counts \(n_{\alpha k}\) come from the data; \(n^{0}_{\alpha k}\) is optional prior evidence.</p>
  </div>
  <div class="conf-card">
    <h3>Weighted rubric metric</h3>
    <div class="conf-equation sm">
      \[ \bar{\pi} = \frac{1}{M}\sum_{\alpha=1}^{M}\sum_{k=0}^{C} w_k\,\pi_{\alpha k} \]
    </div>
    <p>The weights \(w_k\) name the metric: \(w=(0,1)\) is accuracy; graded rubrics use any \(w\).</p>
  </div>
</div>

<div class="conf-callout amber" style="margin-top: 26px;">
  Both the posterior mean \(\mu(R)\) and its uncertainty \(\sigma(R)\) have closed forms (Algorithm 1). No Monte Carlo, no bootstrap.
</div>

Note:
  Modeling outcomes as categorical with a Dirichlet prior is what buys the generality. Binary 0/1 is just C equals 1.
  Because the posterior mean and variance are analytic, you can update them online and stop early.

---

<div class="conf-kicker">Adoption</div>
<h2>Same ranking as average accuracy, plus a decision rule</h2>

<div class="conf-two-thirds">
  <div class="conf-stack">
    <div class="conf-callout teal">
      Under a uniform prior in the binary case, Bayes@<i>N</i> is order-equivalent to avg@<i>N</i> &equiv; Pass@1. You lose nothing and gain calibrated uncertainty.
    </div>
    <div class="conf-grid two">
      <div class="conf-card">
        <h3>Decide on overlap</h3>
        <p>If two models' credible intervals overlap, do not declare a winner.</p>
      </div>
      <div class="conf-card">
        <h3>Pairwise confidence</h3>
        <p>\(z=1.645\) gives 95% confidence in the ordering of a pair.</p>
      </div>
    </div>
  </div>
  <div class="conf-panel conf-title-card">
    <h3>The rule, exactly</h3>
    <div class="conf-equation">
      \[ z = \frac{\lvert \mu-\mu' \rvert}{\sqrt{\sigma^2+\sigma'^2}} \]
    </div>
    <div class="conf-equation" style="margin-top: 14px;">
      \[ \rho = \tfrac{1}{2}\left(1 + \operatorname{erf}\left(z/\sqrt{2}\right)\right) \]
    </div>
    <p class="conf-small" style="margin-top: 12px;">\(\rho\) is the probability the \(\mu\)-based ordering of the pair is correct.</p>
  </div>
</div>

Note:
  People trust average accuracy, and under a uniform prior Bayes@N reproduces its exact ordering, so the switch costs nothing.
  What you gain is the z-score rule: an analytic test for whether a gap is real.

---

<div class="conf-kicker">Result · convergence</div>
<h2>Bayes@<i>N</i> reaches the stable ranking with fewer trials</h2>

<div class="conf-split">
  <figure class="conf-figure conf-fig-mid">
    <img src="/assets/slides/2026-01-25-bayes-at-n/convergence.webp" alt="Convergence@n probability mass functions comparing Bayes@N against Pass@2/4/8 across four math benchmarks, with example ranking traces" />
    <figcaption>Convergence@\(n\): the smallest trial budget after which the ranking matches the \(N=80\) gold standard and stays put.</figcaption>
  </figure>
  <div class="conf-stack">
    <div class="conf-card">
      <h3>Pass@<i>k</i> can fail to converge</h3>
      <p>On AIME'24 and AIME'25, the Pass family often never settles within the budget; Bayes@\(N\) does.</p>
    </div>
    <div class="conf-card">
      <h3>And converges slower when it does</h3>
      <p>On HMMT'25 and BrUMO'25, mean convergence@\(n\) is \(\approx 44.2\) and \(\approx 27.1\) for Bayes@\(N\) versus \(\approx 69.5\) and \(\approx 48.5\) for Pass@\(k\).</p>
    </div>
    <div class="conf-callout">
      Same trials, same data. Only the scoring rule changes.
    </div>
  </div>
</div>

Note:
  Convergence@n is the practical question: how many samples before the leaderboard stops moving.
  The Pass family either never stabilizes or needs far more trials; the Bayesian estimator stabilizes early.

---

<div class="conf-kicker">Result · benchmarks</div>
<h2>Faster agreement with the gold standard across datasets</h2>

<div class="conf-split reverse">
  <div class="conf-stack">
    <div class="conf-stat-row" style="grid-template-columns: repeat(2, minmax(0,1fr));">
      <div class="conf-stat"><strong>&gt;0.90</strong><span>Kendall &tau; by N = 10, all four datasets</span></div>
      <div class="conf-stat"><strong>&asymp;1.0</strong><span>by N &asymp; 80 on three of four</span></div>
    </div>
    <div class="conf-card">
      <h3>Measured against avg@80</h3>
      <p>The gold standard is Bayes@80 \(\equiv\) avg@80; \(\tau_b\) handles ties without inflating agreement.</p>
    </div>
    <div class="conf-card">
      <h3>AIME'25 stays unresolved</h3>
      <p>It plateaus near \(\tau\approx 0.95\) at \(N=80\): extensive ties mean even the gold standard needs interval-aware reporting.</p>
    </div>
  </div>
  <figure class="conf-figure conf-fig-tall">
    <img src="/assets/slides/2026-01-25-bayes-at-n/benchmarks.webp" alt="Kendall tau versus number of trials for Bayes@N and Pass@k variants on AIME'25, AIME'24, HMMT'25, and BrUMO'25" />
    <figcaption>Average Kendall \(\tau\) to the gold-standard ranking versus trial budget \(N\), over \(10^4\) bootstrap resamples.</figcaption>
  </figure>
</div>

Note:
  Across four real math-reasoning benchmarks the Bayesian curve sits above Pass@k everywhere and rises faster.
  AIME'25 is the honest caveat: it does not fully resolve at 80 trials, which is exactly why you want intervals rather than a hard ranking.

---

<div class="conf-kicker">Result · uncertainty</div>
<h2>Close models need far more than 80 trials to separate</h2>

<div class="conf-split">
  <figure class="conf-figure conf-fig-mid">
    <img src="/assets/slides/2026-01-25-bayes-at-n/decision-rule.webp" alt="Probability and z-score of correctly ranking two close models as a function of the number of trials" />
    <figcaption>Probability and \(z\)-score of correctly ordering two near-tied models (\(\bar\pi = 0.608\) vs \(0.6213\)) versus \(N\).</figcaption>
  </figure>
  <div class="conf-stack">
    <div class="conf-metric" style="grid-template-columns: repeat(3, minmax(0,1fr));">
      <div class="conf-card"><p><strong>83.7%</strong><span>correct order at N = 80</span></p></div>
      <div class="conf-card"><p><strong>94.7%</strong><span>N = 199 &middot; z = 1.645</span></p></div>
      <div class="conf-card"><p><strong>96.9%</strong><span>N = 285 &middot; z = 1.96</span></p></div>
    </div>
    <div class="conf-callout amber">
      A typical \(N=80\) evaluation is not enough to confidently separate two models that differ by \(\approx 1.3\) accuracy points.
    </div>
    <div class="conf-card">
      <h3>The interval makes this visible</h3>
      <p>Without a credible interval, the point estimate silently picks a winner the data does not support.</p>
    </div>
  </div>
</div>

Note:
  The number to remember: two models a point apart need roughly 200 trials to separate at 95% confidence.
  The credible interval makes that visible, where a bare Pass@k or average number hides it.

---

<div class="conf-kicker">Leaderboards</div>
<h2>The bigger the leaderboard, the more intervals matter</h2>

<div class="conf-split reverse">
  <div class="conf-stack">
    <div class="conf-card">
      <h3>Brittleness scales with model count</h3>
      <p>Mean convergence@\(n\) worsens as the compared pool grows from 5 to 15 to 20 models.</p>
    </div>
    <div class="conf-card">
      <h3>Subsets stop stabilizing</h3>
      <p>Without credible intervals, many model subsets fail to settle within 80 trials across all four datasets.</p>
    </div>
    <div class="conf-callout teal">
      Posterior intervals give a transparent stopping rule exactly when strict point rankings are no longer trustworthy. Qwen3-30B-A3B-Thinking and Qwen3-4B-Thinking top every dataset, separable at the 95% level.
    </div>
  </div>
  <figure class="conf-figure conf-fig-tall">
    <img src="/assets/slides/2026-01-25-bayes-at-n/leaderboard.webp" alt="Heatmap of mean convergence across datasets and model subsets as the leaderboard widens" />
    <figcaption>Mean convergence@\(n\) per dataset as the leaderboard widens; darker cells take more trials to stabilize.</figcaption>
  </figure>
</div>

Note:
  Real leaderboards have many entries, and adding entries makes strict orderings less stable, not more.
  The interval-aware protocol is what keeps a large leaderboard honest: it refuses to rank what it cannot yet distinguish.

---

<div class="conf-kicker">Priors</div>
<h2>Prior evidence helps at small budgets, in moderation</h2>

<div class="conf-split">
  <figure class="conf-figure conf-fig-mid">
    <img src="/assets/slides/2026-01-25-bayes-at-n/prior-transfer.webp" alt="Kendall tau versus number of trials for a uniform prior and non-uniform priors with different amounts of prior data D" />
    <figcaption>Mean \(\tau\) for updated models under a uniform prior (\(D=0\)) versus non-uniform priors built from \(D\) trials of a related model.</figcaption>
  </figure>
  <div class="conf-stack">
    <div class="conf-card">
      <h3>Plug priors in through \(R^0\)</h3>
      <p>Results from an older or related model enter as pseudo-counts \(n^{0}_{\alpha k}\), with no change to the estimator.</p>
    </div>
    <div class="conf-card">
      <h3>Small \(N\), small \(D\) wins</h3>
      <p>For \(D\le 4\), priors lift \(\tau\) at low trial counts; \(D=8\) or \(16\) over-emphasizes the prior and can hurt at larger \(N\).</p>
    </div>
    <div class="conf-callout">
      Across 50k stochastic updates, the ranking changes for \(\approx 86\%\) of cases (mean \(\tau \approx 0.88\)) — so priors must be used judiciously.
    </div>
  </div>
</div>

Note:
  Priors are a real lever but a double-edged one. A little prior data from a related model sharpens small-budget rankings.
  Too much prior weight drags the estimate toward the old model, so the benefit reverses as you collect more real trials.

---

<div class="conf-kicker">Rubrics</div>
<h2>The same machinery scores beyond right-or-wrong</h2>

<div class="conf-split reverse">
  <div class="conf-stack">
    <div class="conf-card">
      <h3>Outcomes are categories</h3>
      <p>Correct, partial credit, format errors, refusals, verifier signals, efficiency penalties — all live in one Dirichlet posterior.</p>
    </div>
    <div class="conf-card">
      <h3>Weights encode the goal</h3>
      <p>Different \(w\) vectors express different evaluation priorities while the uncertainty stays explicit and comparable.</p>
    </div>
    <div class="conf-callout teal">
      Across categorical schemes, Qwen3-Thinking stays first; the rubric choice mainly reshuffles the middle of the pack.
    </div>
  </div>
  <figure class="conf-figure conf-fig-tall">
    <img src="/assets/slides/2026-01-25-bayes-at-n/categorical.webp" alt="Ranking traces of 11 models across several categorical scoring schemes" />
    <figcaption>Model rank across categorical schemes; top models are stable while mid-pack order depends on the rubric.</figcaption>
  </figure>
</div>

Note:
  Binary scoring throws away most of what a grader sees. Modeling outcomes as categories keeps partial credit, refusals, and verifier signals on the same probabilistic footing.
  Reassuringly, the rubric mostly changes the middle of the leaderboard, not the top.

---

<h2>Scorio is the drop-in implementation</h2>

<div class="conf-grid two">
  <div class="conf-card">
    <h3>Two calls, same interface</h3>
    <pre class="conf-code" style="font-size: 18px;"><span class="kw">pip install</span> scorio

<span class="kw">from</span> scorio <span class="kw">import</span> eval

mu, sigma, lo, hi = eval.<span class="fn">bayes_ci</span>(R, <span class="arg">confidence</span>=0.95)
p,  sigma, lo, hi = eval.<span class="fn">pass_at_k_ci</span>(R, <span class="arg">k</span>=1)</pre>
  </div>
  <div class="conf-stack">
    <div class="conf-card">
      <h3>Takes the matrix you already have</h3>
      <p>Feed the M&times;N results matrix R; get back a mean, an uncertainty, and a credible interval.</p>
    </div>
    <div class="conf-card">
      <h3>Bayes and Pass side by side</h3>
      <p>Both estimators share one API, so you can compare orderings and intervals without rewriting your harness.</p>
    </div>
    <div class="conf-chip-row">
      <span class="conf-chip teal">analytic CIs</span>
      <span class="conf-chip blue">online updates</span>
      <span class="conf-chip amber">rubric-aware</span>
    </div>
  </div>
</div>

Note:
  The framework ships as a small library so adoption is a one-line install and a one-line call.
  Same input you already build for Pass@k, richer output.

---

<div class="conf-takeaway-header">
  <img class="conf-takeaway-icon" src="/assets/slides/2026-01-25-bayes-at-n/takeaway.webp" alt="" aria-hidden="true" />
  <div>
    <h2 class="conf-takeaway-title">Takeaways</h2>
    <div class="conf-takeaway-subtitle">Replace Pass@k with Bayes@N plus credible intervals</div>
  </div>
</div>

<div class="conf-takeaway-grid">
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">1</div>
    <h3>Estimate the right thing</h3>
    <p>Bayes@\(N\) targets the underlying success probability and converges to a stable ranking with fewer trials.</p>
  </div>
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">2</div>
    <h3>Decide with intervals</h3>
    <p>Do not call a winner on overlapping credible intervals; separating close models can take \(\approx 200\) trials.</p>
  </div>
  <div class="conf-card conf-takeaway-card">
    <div class="conf-takeaway-num">3</div>
    <h3>One framework, more reach</h3>
    <p>Categorical rubrics and prior evidence drop into the same closed-form posterior.</p>
  </div>
</div>

Note:
  If the audience keeps one sentence: stop ranking on Pass@k point estimates, report Bayes@N with credible intervals, and only call a winner when the intervals separate.

---

<div class="conf-closing-layout">
<div class="conf-closing-main">
<h1 class="conf-closing-title">Thank you! Questions?</h1>
<div class="conf-subtitle">Don't Pass@<i>k</i>: A Bayesian Framework for Large Language Model Evaluation</div>
<div class="conf-qr-grid" aria-label="Project links">
<div class="conf-qr-item">
<img src="/assets/slides/2026-01-25-bayes-at-n/qr_poster.svg" alt="QR code for the interactive poster" />
<strong>Poster</strong>
</div>
<div class="conf-qr-item">
<img src="/assets/slides/2026-01-25-bayes-at-n/qr_github.svg" alt="QR code for the GitHub repository" />
<strong>GitHub</strong>
</div>
<div class="conf-qr-item">
<img src="/assets/slides/2026-01-25-bayes-at-n/qr_arxiv.svg" alt="QR code for the arXiv paper" />
<strong>arXiv</strong>
</div>
<div class="conf-qr-item">
<img src="/assets/slides/2026-01-25-bayes-at-n/qr_docs.svg" alt="QR code for the documentation" />
<strong>Docs</strong>
</div>
</div>
<div class="conf-contact-line">Code and docs at <span class="conf-contact-email">github.com/mohsenhariri/scorio</span> · reach me at <span class="conf-contact-email">mohsen.hariri@case.edu</span>, especially to collaborate</div>
</div>
<div class="conf-ack-stack" aria-label="Acknowledgments">
<div class="conf-ack-item conf-ack-nsf">
<img src="/assets/slides/2026-01-25-bayes-at-n/nsf.svg" alt="National Science Foundation" />
<div>
<h3>NSF Awards</h3>
<p>2117439, 2320952</p>
</div>
</div>
<div class="conf-ack-item conf-ack-cwru">
<img src="/assets/slides/2026-01-25-bayes-at-n/cwru_hpc_logo.webp" alt="Case Western Reserve University" />
<div>
<h3>High Performance Computing (HPC)</h3>
<p>Center at CWRU</p>
</div>
</div>
</div>
</div>

Note:
  Close on the links: poster, code, paper, and docs all point at Scorio.
  Acknowledge NSF awards 2117439 and 2320952 and the High Performance Computing Center at CWRU.

Navigation