Every headline number on this site comes from a controlled benchmark. You can run the load-bearing ones yourself in a few minutes with the licensed client and a fresh random problem. Each script below is self-contained: paste it, set your key, run it.
Three results, three different axes. Knowing which axis a result lives on is what makes it credible — and what stops an apples-to-oranges comparison from looking like a loss. We name the axis for each one.
Axis: measurements per update step. The runtime makes one update from a single measurement; a finite-difference gradient needs n+1. Tracking quality stays flat as the channel count grows. This is structural — it holds at every size.
copy# REPRODUCTION 1: regulation measurement-efficiency vs finite-difference gradient.
# Claim: SWC tracks at 1 measurement/round where finite-difference needs n+1; tracking
# quality stays flat as n grows while the gradient baseline saturates.
import math, random
from swc import swc_regulate, SWCOptimizer
def make_plant(n, seed):
rng = random.Random(seed)
gains = [rng.uniform(0.6, 1.4) for _ in range(n)] # per-channel response gain
def measure(x): # nonlinear, drifting plant
return [min(1.0, max(0.0, math.sin(x[i]*math.pi/2*gains[i]))) for i in range(n)]
return measure
def swc_rms(n, key, endpoint, rounds=80, seed=0):
target = [0.3 + 0.4*((i % 5)/4.0) for i in range(n)]
measure = make_plant(n, seed)
x, _hist = swc_regulate(measure, [0.5]*n, target, key, rounds=rounds, endpoint=endpoint)
fin = measure(x)
return math.sqrt(sum((fin[i]-target[i])**2 for i in range(n))/n)
if __name__ == "__main__":
import sys
KEY = sys.argv[1] if len(sys.argv) > 1 else "EVAL-..."
EP = sys.argv[2] if len(sys.argv) > 2 else "https://trueloopcompute.com"
print(f"{'n':>6} {'SWC tracking RMS':>18} {'measurements/update':>22}")
for n in [8, 32, 128, 512]:
rms = swc_rms(n, KEY, EP, seed=1)
print(f"{n:>6} {rms:>18.4f} {'1 vs %d (finite-diff)'%(n+1):>22}")
print("\nExpected: RMS stays low and roughly flat as n grows; SWC uses ONE measurement")
print("per update where a finite-difference gradient needs n+1.")You should see the tracking RMS stay low and roughly flat from n=8 to n=512, with the one-vs-(n+1) measurement ratio widening as n grows.
Axis: best solution before a fixed wall-clock deadline. Under a tight deadline that does not grow with n, a sequential classical solver does fewer useful steps as the problem grows, while the runtime's parallel rounds hold. It overtakes past a crossover (~n=512) and the gap widens. This rests on one premise, stated in the cost model: a configuration measurement is parallel and constant-cost in n.
copy# REPRODUCTION 2: optimization under a fixed wall-clock deadline (the crossover).
# Claim: under a tight deadline that does NOT grow with n, classical sequential search
# starves as n grows while the runtime's constant-cost parallel rounds hold -- so the
# runtime overtakes past a crossover (~512) and the margin widens. Premise (explicit):
# a configuration measurement is parallel, constant-cost in n.
import math, random
from swc import SWCOptimizer
T_MEAS = 0.05 # ms per parallel configuration measurement (constant in n) -- the premise
T_OP = 0.001 # ms per digital scalar op
def make_qubo(n, density, coupling, seed):
rng = random.Random(seed); Q = {}; adj = {i: [] for i in range(n)}
for i in range(n):
Q[(i, i)] = rng.uniform(-2, 2)
for j in range(i + 1, n):
if rng.random() < density:
w = coupling * rng.uniform(-1, 1)
Q[(i, j)] = w; adj[i].append((j, w)); adj[j].append((i, w))
return Q, adj
def energy(x, Q):
e = 0.0
for (i, j), w in Q.items():
e += w*x[i]*x[j] if i != j else w*x[i]
return e
def swc_best(n, Q, key, endpoint, deadline_ms, ns=8, seed=7):
random.seed(seed)
opt = SWCOptimizer(license_key=key, n=n, mode="optimization", endpoint=endpoint)
theta = opt.start([math.pi/2]*n); best = 1e9; t = 0.0
while t < deadline_ms:
samp = [[1.0 if random.random() < 0.5*(1+math.cos(c)) else 0.0 for c in theta] for _ in range(ns)]
es = [energy(b, Q) for b in samp]
bi = min(range(ns), key=lambda k: es[k]); best = min(best, es[bi])
t += ns * T_MEAS
phat = [sum(b[i] for b in samp)/ns for i in range(n)]
theta = opt.step(phat, score=-es[bi])
opt.end(); return best
def sa_best(n, Q, adj, deadline_ms, seed=7):
rng = random.Random(seed*7+3)
x = [1.0 if rng.random() < 0.5 else 0.0 for _ in range(n)]
e = energy(x, Q); best = e; t = 0.0
while t < deadline_ms:
i = rng.randrange(n); xi = 1-x[i]; d = Q.get((i, i), 0)*(xi-x[i])
for j, w in adj[i]: d += w*(xi-x[i])*x[j]
if d < 0 or rng.random() < math.exp(-d/1.001):
x[i] = xi; e += d; best = min(best, e)
t += (1 + len(adj[i])) * T_OP
return best
if __name__ == "__main__":
import sys
KEY = sys.argv[1] if len(sys.argv) > 1 else "EVAL-..."
EP = sys.argv[2] if len(sys.argv) > 2 else "https://trueloopcompute.com"
print(f"{'n':>6} {'SWC e/n':>9} {'SA e/n':>9} {'winner':>7}")
for n in [64, 256, 512, 1024]:
deadline = 2.0
sw=[]; sa=[]
for s in range(3):
Q, adj = make_qubo(n, 0.05, 0.3, s)
sw.append(swc_best(n, Q, KEY, EP, deadline, seed=s)/n)
sa.append(sa_best(n, Q, adj, deadline, seed=s)/n)
swm=sum(sw)/3; sam=sum(sa)/3
print(f"{n:>6} {swm:>9.4f} {sam:>9.4f} {('SWC' if swm<sam else 'SA'):>7}")
print("\nExpected: SA wins small n; the runtime overtakes around n~512 and the gap widens.")Expected shape: simulated annealing wins at small n; the runtime overtakes around n=512 and the margin widens with dimension. Tighten the deadline and the crossover moves to smaller problems. Your exact crossover shifts with the T_MEAS/T_OP ratio you assume.
Axis: total measurements to a quality bar. The opt-in smart estimator reaches a usable solution in about n1.2 total measurements on structured problems, where the default elite estimator scales near n2. This compares the runtime against itself.
Unlike the two above, a clean one-shot script understates this result: a fair exponent needs a fixed hard quality bar, fresh non-overlapping seeds, and a log-log fit with bootstrap confidence intervals — otherwise an easy bar is hit in the first round and the estimators look identical. We hold ourselves to that bar. The validated protocol: banded QUBOs (bandwidth 2 and 4), 10 seeds, quality bars swept 0.6–0.9, paired bootstrap 95% CIs (10k resamples), sizes to n=512. Measured exponents: n1.18 [1.16, 1.21] light, n1.21 [1.16, 1.27] coupled, degrading to n1.50 [1.36, 1.65] on dense all-to-all — all sub-quadratic, all against the elite law's ~n2. To run it yourself, switch estimator between "elite" and "smart" in script 2's optimizer and measure rounds-to-bar across sizes with those controls in place.
Honest scope. All three are simulation results with the measurement-cost premise stated; hardware validation is the next step. None of them claims to beat simulated annealing on equal-measurement-budget solution quality at small n — that is a different axis, and there a classical solver is the right tool. We say so on the baselines page.
Install the client from your welcome email, set KEY to your license key, and pass your endpoint. Questions about a result that does not reproduce on your hardware? Email matthew@trueloopcompute.com — a number that does not hold up is something we want to know about.