← Back to dashboard

Performance Benchmarks

About the readiness labels

Quick editorial read on how practical each capability is for office-style workflows today — not a warranty of field or jobsite performance.

Automation Building — SWE-bench

Useful with review

AI can help build and troubleshoot the software glue behind construction workflows.

Tested bySWE-bench / SWE-bench VerifiedSWE‑Bench Pro (multi-language public set)HumanEvalTerminal-Bench
What each test measures
SWE-bench / SWE-bench Verified
Issue-fix tasks under a fixed scaffold: produce a change that passes automated checks — a stand-in for patching automations and internal tools, not general software theory.
SWE‑Bench Pro (multi-language public set)
Broader multi-language fix tasks than the Verified scaffold — useful extra signal for mixed-language automation stacks.
HumanEval
Small coding tasks with unit checks — analogous to short scripts in an automation or integration layer.
Terminal-Bench
CLI-style tasks in a controlled agent environment.

It can help with…

  • Email-to-log automation
  • RFI intake workflows
  • Bid tracker updates
  • Submittal tracking tools
  • Dashboard fixes
  • Excel cleanup scripts

Document Review — DocVQA / Long Context

Useful with review

AI can help search, summarize, and extract from large project documents.

Tested byDocVQA / document visual QALong-context / needle-in-a-haystack style tests
What each test measures
DocVQA / document visual QA
Answers questions about document images — forms, invoices, typed pages — not only raw text extracts.
Long-context / needle-in-a-haystack style tests
Whether the model can find or reason about a fact buried in a very long input window; vendor protocols differ.

It can help with…

  • Spec lookup
  • Submittal checks
  • Contract clause search
  • Pay app backup review
  • Change order backup review
  • Meeting minute extraction

Visual Review — MMMU

Useful with review

AI can help interpret visual information when text and images are mixed together.

Tested byMMMU / MMMU-Pro
What each test measures
MMMU / MMMU-Pro
University-level multimodal problems spanning charts, figures, and diagrams alongside text prompts.

It can help with…

  • Plan snippet review
  • Marked-up PDF review
  • Product data sheet comparison
  • Installation diagram review
  • Schedule chart interpretation
  • Site photo sorting

Technical Reasoning — GPQA

Strong assist

AI can help reason through technical conflicts before a team escalates them.

Tested byGPQA Diamond
What each test measures
GPQA Diamond
Difficult multiple-choice science questions filtered for quality; Diamond subset is the most trusted slice.

It can help with…

  • Pre-RFI analysis
  • Scope gap checks
  • Product comparisons
  • Technical submittal pre-review
  • Spec conflict summaries
  • Material substitution review

Math + Quantities — AIME / MATH

Useful with review

AI can help check structured calculations when the inputs are clear.

Tested byAIME / competition math suitesGSM8K (grade school math word problems)
What each test measures
AIME / competition math suites
Timed competition-style problems requiring multi-step symbolic reasoning.
GSM8K (grade school math word problems)
Word problems requiring ordered arithmetic and light reasoning — easier than AIME but diagnostic.

It can help with…

  • Unit conversions
  • Production rate checks
  • Quantity takeoff logic
  • Schedule duration math
  • Cost breakdown checks
  • Alternate comparison

Workflow Execution — OSWorld / τ²-bench

Limited / narrow use

AI can help move information through office workflows when tools and approvals are controlled.

Tested byτ²-bench (policy + tools)OSWorld
What each test measures
τ²-bench (policy + tools)
Multi-turn tool scenarios (e.g. retail/telecom service desks) with policy constraints.
OSWorld
Sandboxed GUI tasks requiring pixel-level pointing and clicks.

It can help with…

  • Email triage
  • RFI log updates
  • Follow-up drafts
  • Approval routing
  • Form filling
  • Meeting action item tracking