Over time
Triangles and the "short" tag mark runs that are not full RigMark cards: fewer samples and fewer tests. Their commits may not be on main yet. Compare them with each other, not with full cards. Squares mark full-length answers: much longer than the 64-token short runs, so they run slower and are not a drop.
Drafting on Drafting off Thinking on Failed or partial run Short run, not a full card Full-length answers
A benchmark run is one fixed set of chat requests sent to a running Vade server. Speeds are in tokens per second (tok/s) and are medians. Prose, Code and Structured each use one chat at a time. 8 chats is the total speed of eight chats at once. Cold 8K is the time to the first token after a new 8,192-token prompt. 32K first token is the time to the first token for the first question of a 32K-token chat.
FP4-Local and FP8 are the 4-bit and 8-bit forms of the model weights. FP4-Local fits on one Spark. FP8 needs two or more Sparks. Drafting means that the model's own prediction head guesses a few tokens ahead. The full model then checks each guess. The page shows only runs of Vade from its main branch, on our own Sparks.
Exact compares the token IDs of a run with the drafting-off run of the same engine and setup. It shows match or differ. It shows n/a when there is nothing to compare. Open a row to see why. Times are US Central time, and each run shows its own zone (CDT or CST). A dash means that the run did not measure that number.