Benchmarking

Benchmarks compare the same numeric program as OpenCL, Spire-on-CPU, Breeze, and a plain while loop. These numbers describe one machine, not a portable promise.

When This Actually Pays Off

OpenCL wins when three things line up: a large array (below ~10^6 elements, launch overhead dominates and the JVM wins outright), heavy math (cheap formulas like a*b+c can be up to 11x slower on the GPU), and enough chained work per element (a single FMA isn't enough to hide a transfer; tens of chained steps are). Hit all three and the win is an order of magnitude or more — up to ~195x for heavy, ~688x for the 4x-heavier veryHeavy. Miss them and OpenCL can lose outright. This is a compute accelerator for compute-bound work, not a blanket "GPU makes everything faster" claim — see the tables below for exactly where the line sits.

Generated from bench/results/2026-08-15-apple-m3-max.json, 2026-08-24-170344.json. Scores are us/op, lower is better. +/- is JMH's 99.9% confidence half-width.

Benchmark Results

Elementwise, 10^7 floats

Purpose: cheapest possible math, so transfer/launch cost is not hidden by compute. Shows the worst case for OpenCL.

Tested formula: elementwise(a, b, c) = a*b + c.

row us/op vs Spire
OpenCL, resident arrays + mapped output 1085 +/- 95.0 1.4x faster
OpenCL, resident arrays 1923 +/- 78.3 1.3x slower
Spire on JVM 1523 +/- 23.5 1.0
plain while loop 1570 +/- 38.4 -
Breeze DenseVector 4250 +/- 99.5 2.8x slower
OpenCL, packed params 10372 +/- 303 6.8x slower
OpenCL, packed params + interleave 17033 +/- 482 11.2x slower

Resident arrays avoid per-launch input upload. Mapped output removes the readback copy on this unified-memory machine.

Heavy math, 10^7 floats

Purpose: expensive math (sin/exp/sqrt), same data size as elementwise. Shows the library's actual sweet spot.

Tested formula: heavy(x, a) = sqrt(exp(sin(x*a))^2 + sin(x*a)^2).

row us/op vs Spire
OpenCL, resident arrays 3573 +/- 804 30.8x faster
OpenCL, packed params 11413 +/- 467 9.6x faster
Spire on JVM 110101 +/- 620 1.0
plain while loop 110843 +/- 450 1.0x slower
Breeze scalar fallback 153269 +/- 1801 1.4x slower

The GPU wins clearly once the expression has enough arithmetic to amortize launch and transfer.

Generator, 10^7 floats

Purpose: same chain/heavy math, but no array upload at all — isolates pure compute cost from transfer cost.

Tested formulas: chain(x, a, b, depth) = acc = x; repeat depth times: acc = acc*a + b (depth 8 here) and heavy(x, a) = sqrt(exp(sin(x*a))^2 + sin(x*a)^2).

row chain depth 8 sin/exp/sqrt chain vs Spire sin/exp/sqrt vs Spire
OpenCL 1752 +/- 122 2076 +/- 158 46.0x faster 194.8x faster
plain while loop 21975 +/- 147 404511 +/- 1682 3.7x faster 1.0x slower
Spire on JVM 80627 +/- 5592 404413 +/- 1557 1.0 1.0
Breeze 100750 +/- 1478 n/a 1.2x slower n/a

Generator kernels derive values from Expr.Index; there is no input array upload.

Generator, veryHeavy (sin/exp/sqrt composed 4x)

heavy's body run 4 times over (Bench.VeryHeavyDepth), same traffic as heavy, 4x the arithmetic. Single-fork quick run, not the 3-fork config the rest of this page uses.

Purpose: check if the OpenCL win keeps growing when compute gets heavier still, at the same array size.

Tested formula: veryHeavy(x, a) = heavy(x, a) composed 4x (sin/exp/sqrt repeated 4 times).

size OpenCL plain while loop Spire on JVM OpenCL vs Spire
10000 171 +/- 84.1 1037 +/- 44.9 1044 +/- 6.50 6.1x faster
1000000 364 +/- 58.2 105429 +/- 3870 105696 +/- 2846 290.2x faster
10000000 2012 +/- 397 1383969 +/- 49238 1392328 +/- 26425 691.9x faster

Device time barely moves between heavy and veryHeavy; CPU time scales with the added arithmetic, so the win grows with arithmetic intensity rather than staying fixed.

Depth sweep, 10^6 floats

Purpose: find how many chained multiply-adds it takes before OpenCL beats the JVM.

Tested formula: chain(x, a, b, depth) = acc = x; repeat depth times: acc = acc*a + b.

depth OpenCL plain Spire OpenCL vs Spire
1 346 +/- 4.30 815 +/- 19.6 2559 +/- 49.0 7.4x faster
8 355 +/- 5.35 2362 +/- 175 7474 +/- 47.0 21.0x faster
32 387 +/- 10.4 11371 +/- 55.8 19931 +/- 1296 51.5x faster
128 383 +/- 2.56 109889 +/- 1379 131254 +/- 1199 342.3x faster

The OpenCL row is nearly flat: transfer plus launch dominates, while extra multiply-add levels are cheap on the device.

Host-side phase split, 10^7 floats

Purpose: see where OpenCL time actually goes — upload, kernel launch, or readback.

Per-launch cost broken into uploading parameters to the device, the kernel launch itself, and reading the result back. readback is not device compute time: the upload and launch enqueue asynchronously, so almost all device time is absorbed there as the blocking wait.

row upload us/op launch us/op readback us/op
OpenCL, resident arrays 0.44 9.41 1907
OpenCL, resident arrays + mapped output 0.13 9.10 1058
OpenCL, packed params 5008 8.10 5336
OpenCL, packed params + interleave 10574 14.8 6414
Generator, chain depth 8 1.31 10.3 1790

What Each Row Actually Runs

Every contender executes the same two formulas, defined once in Programs.scala and reified for each backend so the comparison is not GPU-code-vs-hand-tuned-CPU-code:

Encodings (how the array reaches the kernel)

The JVM/CPU rows

AgreementSpec runs all of these on the same inputs and checks they agree numerically, so "same formula, different backend" is enforced, not just asserted in a comment.

Why OpenCL is sometimes slower, not always much faster

Benchmarking Notes

Run correctness first. Timings are meaningless if the implementations do not agree:

./mill bench.test

Then run JMH:

./mill bench.runJmh GeneratorBench
./mill bench.runJmh -f 1 -wi 2 -i 3 GeneratorBench

Record and compare history:

./mill bench.record -f 1 -wi 2 -i 3 GeneratorBench
./mill bench.history
./mill bench.compare

Reports are written to bench/results/<timestamp>.json.

Small arrays mostly measure launch latency. Larger arrays show transfer cost and device throughput:

row shape
generator values derived from Expr.Index; no input array upload
packed params host arrays packed and uploaded every launch
device inputs arrays uploaded with writeInput, then reused across launches
mapped output device inputs plus renderBatchMappedUnsafe to avoid the readback copy

Use ./mill bench.report on a recorded run for rates. Use ./mill bench.precision to check OpenCL math accuracy on the current device.