Aggregation (Scorio.Aggregate)
Test-time scaling utilities live under Scorio.Aggregate (also available as Scorio.Agg and Scorio.agg). Candidate pools are vectors for one question or M × N matrices for a batch. To match Python exactly, returned indices are 0-based and -1 is the no-valid-candidate sentinel.
Scorio.Aggregate — Module
Answer aggregation and selection for test-time scaling.
Scorio.Aggregate is the Julia port of Python's scorio.aggregate package. It contains per-trace confidence signals, process-reward reductions, fixed-pool selection and voting rules, and online stopping rules.
Candidate pools use N-element vectors for one question or M × N matrices for batches. Returned candidate and token indices deliberately match Python's 0-based convention; -1 denotes that no valid candidate exists.
Confidence signals
Scorio.Aggregate.mean_logprob — Function
Mean chosen-token log-probability (higher is more confident).
Scorio.Aggregate.sequence_logprob — Function
Total chosen-token sequence log-likelihood.
Scorio.Aggregate.perplexity — Function
Sequence perplexity, exp(-mean_logprob(logprobs)).
Scorio.Aggregate.picsar — Function
picsar(logprobs; answer_start=nothing, normalize_reasoning=false)PiCSAR reasoning-plus-answer log-likelihood. answer_start is the number of tokens before the answer span, matching the Python split index.
Scorio.Aggregate.self_certainty — Function
Top-k KL-from-uniform self-certainty, reduced across tokens.
Scorio.Aggregate.token_entropy — Function
Top-k Shannon entropy in nats, reduced across tokens.
Scorio.Aggregate.varentropy — Function
Top-k varentropy, reduced across tokens.
Scorio.Aggregate.max_softmax_probability — Function
Maximum raw softmax probability, reduced across tokens.
Scorio.Aggregate.logprob_margin — Function
Top-one/top-two log-probability (or probability) margin.
Scorio.Aggregate.token_confidence — Function
DeepConf per-token confidence: negative mean raw top-k log-probability.
Scorio.Aggregate.deepconf_confidence — Function
DeepConf trace confidence using mean, tail, or group reductions.
Process rewards
Scorio.Aggregate.prm_aggregate — Function
prm_aggregate(step_scores; method="last") -> Float64Reduce one trace's finite per-step process-reward scores using "last", "min", "mean", "prod", or "max".
Selection and voting
majority_of_the_bests is also available under the short method alias mob.
Scorio.Aggregate.best_of_n — Function
Best-of-N: select the valid candidate with the largest score.
Scorio.Aggregate.majority_of_the_bests — Function
majority_of_the_bests(answers, scores; m=nothing, ...)Exact mode of the Best-of-N answer under size-m bootstrap resampling. The default is floor(sqrt(n)) over valid candidates.
Scorio.Aggregate.best_of_majority — Function
Frequency-gated reward selection (Best-of-Majority).
Scorio.Aggregate.majority_vote — Function
Plain majority vote with ties broken by earliest appearance.
Scorio.Aggregate.weighted_majority_vote — Function
Raw-score weighted majority vote using group sums or means.
Scorio.Aggregate.softmax_weighted_vote — Function
Temperature-softmax-weighted majority vote (CISC).
Scorio.Aggregate.rank_weighted_vote — Function
Rank/Borda-weighted vote, invariant to monotone score transforms.
Scorio.Aggregate.logit_weighted_vote — Function
Threshold-shifted log-odds or linear weighted majority vote.
Scorio.Aggregate.filtered_vote — Function
Vote after retaining only the top-scoring fraction or count.
Scorio.Aggregate.KDEVoteCalibration — Type
Immutable fitted state for non-parametric KDE weighted voting.
Scorio.Aggregate.fit_kde_vote_calibration — Function
Fit class-conditional Gaussian KDEs and a quantile-binned calibrator.
Scorio.Aggregate.kde_weighted_vote — Function
Select answers using a fitted non-parametric KDE vote calibration.
Scorio.Aggregate.cges_vote — Function
cges_vote(answers, scores; allow_other=false, return_index=false, return_score=false)Select the answer having the largest CGES posterior score. Candidate indices follow Python's 0-based convention and use -1 for CGES_OTHER or an empty row.
Scorio.Aggregate.cges_stop — Function
cges_stop(answers, scores; threshold=0.95, include_other=false,
min_samples=1, return_prob=false)Stop one sampling stream once the largest checked CGES posterior reaches the requested threshold.
Online stopping
Scorio.Aggregate.adaptive_consistency_stop — Function
adaptive_consistency_stop(answers; threshold=0.95, return_prob=false)Stop candidate sampling when the Beta posterior probability that the leading answer beats the runner-up reaches threshold.
Scorio.Aggregate.adaptive_consistency_dirichlet_stop — Function
Full observed-support Dirichlet Adaptive-Consistency stopping rule.
Scorio.Aggregate.adaptive_consistency_crp_stop — Function
Finite-horizon CRP Adaptive-Consistency Monte Carlo stopping rule.
Scorio.Aggregate.esc_stop — Function
Early-Stopping Self-Consistency: true for a nonempty unanimous valid window.
Scorio.Aggregate.deepconf_stop_threshold — Function
DeepConf warmup threshold: the (1 - keep) linear quantile.
Scorio.Aggregate.deepconf_online_stop — Function
deepconf_online_stop(topk_logprobs, threshold; window=2048)Return Python's 0-based end-token index for the first below-threshold confidence window, or nothing if generation runs to completion.