Benchmarking
Benchmarks compare the same numeric program as OpenCL, Spire-on-CPU, Breeze, and a plain while
loop. These numbers describe one machine, not a portable promise.
When This Actually Pays Off
OpenCL wins when three things line up: a large array (below ~10^6 elements, launch overhead
dominates and the JVM wins outright), heavy math (cheap formulas like a*b+c can be up to 11x
slower on the GPU), and enough chained work per element (a single FMA isn't enough to hide a
transfer; tens of chained steps are). Hit all three and the win is an order of magnitude or more —
up to ~195x for heavy, ~688x for the 4x-heavier veryHeavy. Miss them and OpenCL can lose
outright. This is a compute accelerator for compute-bound work, not a blanket "GPU makes everything
faster" claim — see the tables below for exactly where the line sits.
Generated from bench/results/2026-08-15-apple-m3-max.json, 2026-08-24-170344.json. Scores are us/op, lower is better. +/- is JMH's 99.9% confidence half-width.
Benchmark Results
Elementwise, 10^7 floats
Purpose: cheapest possible math, so transfer/launch cost is not hidden by compute. Shows the worst case for OpenCL.
Tested formula: elementwise(a, b, c) = a*b + c.
| row | us/op | vs Spire |
|---|---|---|
| OpenCL, resident arrays + mapped output | 1085 +/- 95.0 | 1.4x faster |
| OpenCL, resident arrays | 1923 +/- 78.3 | 1.3x slower |
| Spire on JVM | 1523 +/- 23.5 | 1.0 |
| plain while loop | 1570 +/- 38.4 | - |
| Breeze DenseVector | 4250 +/- 99.5 | 2.8x slower |
| OpenCL, packed params | 10372 +/- 303 | 6.8x slower |
| OpenCL, packed params + interleave | 17033 +/- 482 | 11.2x slower |
Resident arrays avoid per-launch input upload. Mapped output removes the readback copy on this unified-memory machine.
Heavy math, 10^7 floats
Purpose: expensive math (sin/exp/sqrt), same data size as elementwise. Shows the library's actual sweet spot.
Tested formula: heavy(x, a) = sqrt(exp(sin(x*a))^2 + sin(x*a)^2).
| row | us/op | vs Spire |
|---|---|---|
| OpenCL, resident arrays | 3573 +/- 804 | 30.8x faster |
| OpenCL, packed params | 11413 +/- 467 | 9.6x faster |
| Spire on JVM | 110101 +/- 620 | 1.0 |
| plain while loop | 110843 +/- 450 | 1.0x slower |
| Breeze scalar fallback | 153269 +/- 1801 | 1.4x slower |
The GPU wins clearly once the expression has enough arithmetic to amortize launch and transfer.
Generator, 10^7 floats
Purpose: same chain/heavy math, but no array upload at all — isolates pure compute cost from
transfer cost.
Tested formulas: chain(x, a, b, depth) = acc = x; repeat depth times: acc = acc*a + b (depth 8
here) and heavy(x, a) = sqrt(exp(sin(x*a))^2 + sin(x*a)^2).
| row | chain depth 8 | sin/exp/sqrt | chain vs Spire | sin/exp/sqrt vs Spire |
|---|---|---|---|---|
| OpenCL | 1752 +/- 122 | 2076 +/- 158 | 46.0x faster | 194.8x faster |
| plain while loop | 21975 +/- 147 | 404511 +/- 1682 | 3.7x faster | 1.0x slower |
| Spire on JVM | 80627 +/- 5592 | 404413 +/- 1557 | 1.0 | 1.0 |
| Breeze | 100750 +/- 1478 | n/a | 1.2x slower | n/a |
Generator kernels derive values from Expr.Index; there is no input array upload.
Generator, veryHeavy (sin/exp/sqrt composed 4x)
heavy's body run 4 times over (Bench.VeryHeavyDepth), same traffic as heavy, 4x the
arithmetic. Single-fork quick run, not the 3-fork config the rest of this page uses.
Purpose: check if the OpenCL win keeps growing when compute gets heavier still, at the same array size.
Tested formula: veryHeavy(x, a) = heavy(x, a) composed 4x (sin/exp/sqrt repeated 4 times).
| size | OpenCL | plain while loop | Spire on JVM | OpenCL vs Spire |
|---|---|---|---|---|
| 10000 | 171 +/- 84.1 | 1037 +/- 44.9 | 1044 +/- 6.50 | 6.1x faster |
| 1000000 | 364 +/- 58.2 | 105429 +/- 3870 | 105696 +/- 2846 | 290.2x faster |
| 10000000 | 2012 +/- 397 | 1383969 +/- 49238 | 1392328 +/- 26425 | 691.9x faster |
Device time barely moves between heavy and veryHeavy; CPU time scales with the added
arithmetic, so the win grows with arithmetic intensity rather than staying fixed.
Depth sweep, 10^6 floats
Purpose: find how many chained multiply-adds it takes before OpenCL beats the JVM.
Tested formula: chain(x, a, b, depth) = acc = x; repeat depth times: acc = acc*a + b.
| depth | OpenCL | plain | Spire | OpenCL vs Spire |
|---|---|---|---|---|
| 1 | 346 +/- 4.30 | 815 +/- 19.6 | 2559 +/- 49.0 | 7.4x faster |
| 8 | 355 +/- 5.35 | 2362 +/- 175 | 7474 +/- 47.0 | 21.0x faster |
| 32 | 387 +/- 10.4 | 11371 +/- 55.8 | 19931 +/- 1296 | 51.5x faster |
| 128 | 383 +/- 2.56 | 109889 +/- 1379 | 131254 +/- 1199 | 342.3x faster |
The OpenCL row is nearly flat: transfer plus launch dominates, while extra multiply-add levels are cheap on the device.
Host-side phase split, 10^7 floats
Purpose: see where OpenCL time actually goes — upload, kernel launch, or readback.
Per-launch cost broken into uploading parameters to the device, the kernel launch itself, and
reading the result back. readback is not device compute time: the upload and launch enqueue
asynchronously, so almost all device time is absorbed there as the blocking wait.
| row | upload us/op | launch us/op | readback us/op |
|---|---|---|---|
| OpenCL, resident arrays | 0.44 | 9.41 | 1907 |
| OpenCL, resident arrays + mapped output | 0.13 | 9.10 | 1058 |
| OpenCL, packed params | 5008 | 8.10 | 5336 |
| OpenCL, packed params + interleave | 10574 | 14.8 | 6414 |
| Generator, chain depth 8 | 1.31 | 10.3 | 1790 |
What Each Row Actually Runs
Every contender executes the same two formulas, defined once in Programs.scala and reified for
each backend so the comparison is not GPU-code-vs-hand-tuned-CPU-code:
chain(x, a, b, depth)—acc = x; repeat depth times: acc = acc * a + b. Atdepth = 1this isa*x + b(saxpy's inner expression). Memory-bound: FMA only, no transcendentals.depthis swept so time-vs-depth gives a line whose intercept is transfer + launch latency and whose slope is pure arithmetic throughput.heavy(x, a)—sqrt(exp(sin(x*a))^2 + sin(x*a)^2). Compute-bound: same array traffic aschainat depth 1, but sin/exp/sqrt dominate instead of memory. On the GPU these are hardware transcendental units; on the JVM they'reStrictMathcalls.elementwise(a, b, c)—a*b + cover three input arrays, one output.
Encodings (how the array reaches the kernel)
- generator (encoding A) — value is a function of
Expr.Indexonly; no array is uploaded at all, just two scalar uniforms. Best case for the library:size = Non the GPU's fast dimension. - packed params (encoding B,
elementwise*rows) — this library has no native "array argument" node, so a,b,c are packed element-major into one buffer and shipped as batch parameters along dimension 1, while dimension 0 (the fast-varying axis the generated code expects) is a single work-item.openclPackedstarts from data already interleaved (upload only);openclPackingincludes the interleave step itself — the number a caller with three separate arrays actually pays. - device inputs (encoding C,
openclInput*rows) — sameelementwise/heavyarithmetic, but arrays live in device memory viaExpr.Input/writeInput, uploaded once and reused, withsize = Non the fast dimension like encoding A. Pairing B and C numbers is exactly the cost of the missing array-argument node. - mapped output — device inputs, plus
renderBatchMappedUnsafeto skip the device→host readback copy; result stays in a mapped buffer instead of a fresh host array.
The JVM/CPU rows
- Spire —
Programs.chain/heavyinstantiated atFloatthrough Spire'sField/Trig/NRoottypeclasses — the exact same source the OpenCL kernel is reified from, just run on JVM primitives instead of compiled to a kernel. - plain while loop — the same expression hand-transcribed as a raw
whileloop overArray[Float], no typeclass dispatch. This is the floor: it's what tells you Spire's abstraction overhead (if any) actually costs on the JVM, since without it a ratio between the other rows has no scale. - Breeze —
DenseVector[Float]ops (a *:* b + c,tabulate). Forheavy, Breeze has noDenseVector[Float]sin/exp/sqrt (onlyDouble), so that row falls back to a per-element scalar map rather than silently promoting precision — reported as the fallback it is, not a fair vectorised number.
AgreementSpec runs all of these on the same inputs and checks they agree numerically, so "same
formula, different backend" is enforced, not just asserted in a comment.
Why OpenCL is sometimes slower, not always much faster
- Fixed cost per launch. Every OpenCL call pays kernel-launch latency plus (for encodings A/B)
a data transfer, before a single FLOP runs. At small
sizethis fixed cost dominates and awhileloop that never leaves the CPU cache wins outright — seeDepthSweepBench's intercept (transfer + launch) vs. slope (arithmetic) framing. chainis memory-bound, not compute-bound. One FMA per depth level is nowhere near enough arithmetic to hide a PCIe upload/download; the bottleneck is bytes moved, and a modern CPU's memory bandwidth for a single float array isn't dramatically worse than the bus transfer to a discrete/integrated GPU.- Encoding B is deliberately the bad shape. Packed params put the array on the slow batch dimension with dimension 0 (the one the kernel is optimized for) at width 1 — the numbers are there to show what NOT having an array-argument node costs, not to flatter the library.
heavyis where the GPU should win, since transcendentals are real hardware units there andStrictMathcalls on the JVM. On the recorded Apple M3 Max run it does, decisively: the generator-encodingheavyrow is ~195x faster than the plain while loop at 10^7 elements, and even encoding B (packed params, the deliberately bad shape) is still ~9.7x faster. The case where OpenCL is not much faster is the cheap, memory-boundelementwiseworkload (a*b+c) compared against the fast JVM rows (Spire/plain) — see the "vs Spire" column in the Elementwise table above.
Benchmarking Notes
Run correctness first. Timings are meaningless if the implementations do not agree:
./mill bench.test
Then run JMH:
./mill bench.runJmh GeneratorBench
./mill bench.runJmh -f 1 -wi 2 -i 3 GeneratorBench
Record and compare history:
./mill bench.record -f 1 -wi 2 -i 3 GeneratorBench
./mill bench.history
./mill bench.compare
Reports are written to bench/results/<timestamp>.json.
Small arrays mostly measure launch latency. Larger arrays show transfer cost and device throughput:
| row | shape |
|---|---|
| generator | values derived from Expr.Index; no input array upload |
| packed params | host arrays packed and uploaded every launch |
| device inputs | arrays uploaded with writeInput, then reused across launches |
| mapped output | device inputs plus renderBatchMappedUnsafe to avoid the readback copy |
Use ./mill bench.report on a recorded run for rates. Use ./mill bench.precision to check OpenCL
math accuracy on the current device.