Struct Outcome 

pub(crate) struct Outcome {
    pub(crate) primitive: &'static str,
    pub(crate) answer: Option<Answer>,
    pub(crate) faithful: bool,
    pub(crate) correct: bool,
    pub(crate) logprobs: BTreeMap<String, f64>,
    pub(crate) correct_label: String,
    pub(crate) latency: Duration,
    pub(crate) truncated: bool,
    pub(crate) picked: Option<String>,
}

One model’s result on one case.

Fields

primitive: &'static str

answer: Option<Answer>

None when the model produced no usable label at all.

faithful: bool

Whether the most likely first token was one of the offered labels. An answer can exist without this: the engine reads any label in the top 20 and renormalises, so a model that opens with ** or The still gets an answer, and a confident-looking one.

correct: bool

logprobs: BTreeMap<String, f64>

Logprobs per label, for refitting the calibration temperature afterwards.

correct_label: String

latency: Duration

truncated: bool

picked: Option<String>

The label the model actually picked, for agreement against the reference.