Measurements / 2026-10
Numbers, and their edges.
Only HouyiDB's own single-machine measurements: 1×/2× connectivity, a contiguous-vector filter cache, and streaming plan/apply. No combined score, no external-system data.
Read the method before the numbers.
All figures come from the same class of machine and process: each dataset is loaded once, then ef_search is swept with --skip-load; the data directory is cleared and the server restarted before every load. Different tables come from different dates or code paths, so they are A/B comparisons within one scenario, never a combined score.
Bars are normalized to the maximum value within their own figure and serve only relative comparison inside that figure; exact values are printed at the end of each bar or in the data table.
- Machine
- Apple Silicon (arm64)
8 cores · 16GB · macOS · AC - Version
- v0.2.0-beta.5
candidate builds - Shared parameters
- k=10 · m=24
ef_construction=400 · COSINE - Concurrency
- 1 / 8 / 16 · 30s each
load_concurrency=8
Build tuning uses PARALLEL_BATCH_BUILD=1, MAX_BUILD_BATCH=8192, and SEED_SERIAL_POINTS=256. Full parameters, fairness rules, and the result-record template are in VDBBENCH_PROTOCOL.en.md ↗.
1× vs 2×: speed traded for recall.
The default 2× prioritizes recall and compatibility with existing snapshots; 1× halves level-0 connectivity for higher QPS. Both datasets below are A/B comparisons inside their own scenario.
OpenAI 1536D / 50K — QPS
Same dataset, same-day builds; ef64 has conc 8 / 16, while ef128 / ef256 show the QPS recorded for this scenario. Bars are 2×M then 1×M.
Same-scenario notes: ef64 recall is 0.9999 for both; ef64 p99 goes from 9.6ms to 6.0ms; load drops from 81.2s (insert 42.6 + optimize 38.6) to 57.2s (38.9 + 18.3).
Show data table
| Metric | 2× | 1× |
|---|---|---|
| ef64 QPS · c8 | 426.1 | 686.0 |
| ef64 QPS · c16 | 400.3 | 680.3 |
| ef128 QPS | 403.9 | 663.5 |
| ef256 QPS | 411.8 | 652.6 |
| ef64 recall | 0.9999 | 0.9999 |
| ef64 p99 | 9.6ms | 6.0ms |
| Load duration | 81.2s | 57.2s |
Cohere 768D / 1M — QPS and recall
Same 1M dataset scenario; 2× was built on 2026-10-21 and 1× on 2026-10-22. ef128 / ef256 each have conc 8 / 16.
The trade-off: ef128 / ef256 recall is 0.9971 (2×) versus 0.9883 (1×), about 0.9pp lower; load goes from 1653.7s (1465.1 + 188.6) to 1012s (825 + 187).
Show data table
| Metric | 2× | 1× |
|---|---|---|
| ef128 QPS · c8 | 610 | 894 |
| ef128 QPS · c16 | 614 | 864 |
| ef256 QPS · c8 | 608 | 833 |
| ef256 QPS · c16 | 627 | 839 |
| ef128 / ef256 recall | 0.9971 | 0.9883 |
| Load duration | 1653.7s | 1012s |
Filter cache: random access becomes sequential SIMD.
In the Cohere 1M, 1%-labels filter scenario, HouyiDB lazily builds a contiguous vector cache per label when a filter hits many rows (Cosine vectors are pre-normalized), so the query degrades to a sequential SIMD scan plus top-k. Recall stays at 1.0000 on both paths.
Label filter — Cohere 1M (1% labels)
Old path is 2× (2026-10-22); new path is 1× plus the contiguous-vector cache (2026-10-23 rerun).
Run variance: the first run reached ef128 1361/1365 and ef256 1321/1386 QPS; the 10-23 rerun was affected by background load and measured 1161.9/1176.4 at conc 8. Both runs exceed the old path in the same implementation. The cache is invalidated and rebuilt in batches after writes; labels above 256MB fall back to the old path.
Show data table
| Metric | Old 2× | 1× + cache |
|---|---|---|
| ef128 QPS · c8 | 697 | 1161.9 |
| ef128 QPS · c16 | 698 | 907.3 |
| ef256 QPS · c8 | 688 | 1176.4 |
| ef256 QPS · c16 | 747 | 784.2 |
| ef128 / ef256 recall | 1.0000 | 1.0000 |
| Load duration | 1624.1s | 1301.6s |
Streaming: 104 seconds down to 3.9.
Graph merge defaults to "plan under a read lock, commit under a short write lock": planning runs concurrently inside the HNSW read lock, while commit holds the write lock for only 3–7ms per batch (measured), so searches can read the graph during construction. Scenario: Cohere 100K, insert_rate=500/s, k=100.
Streaming checkpoint arrival time
Arrival time (seconds) at the 50% / 80% / 100% / 110% checkpoints. Actual writes take 200.1s for all three paths, so a shorter bar means the checkpoint arrives earlier.
Same-scenario notes: optimize 0.07s → 0.01s; mature recall (100% / 110%): old .988 / .988, new 1× .9876 / .9876, new 2× .996 / .996. 100% QPS (c10): 1221 → 1355 (new 1×) / 887 (new 2×).
Show data table
| Metric | Old 1× | New 1× | New 2× |
|---|---|---|---|
| Checkpoint 50% | 100.8s | 101.0s | 101.0s |
| Checkpoint 80% | 206.3s | 164.4s | 165.3s |
| Checkpoint 100% | 211.2s | 201.0s | 201.0s |
| Checkpoint 110% | 294.0s | 282.5s | 284.5s |
| Mature recall 100% / 110% | .988 / .988 | .9876 / .9876 | .996 / .996 |
| 100% QPS · c10 | 1221 | 1355 | 887 |
| 110% QPS · c10 | 1175 | 1353 | 894 |
| optimize | 0.07s | 0.01s | 0.01s |
50% stage — total time for 1,000 serial queries
Old-path total 104s is drawn as 100%. The label at the end of each bar is "total time / p99".
Ratios: the new paths take roughly 1/17 to 1/27 of the old total time, and p99 is roughly 1/69 to 1/82. Bars are linear, so the new bars are physically tiny; read the labels, not the width.
The new path costs about 11% more wall-clock time for a full bulk build (Cohere 1M: 1170s vs 1055s); if load time is critical, set HOUYI_HNSW_PLAN_APPLY=0 to fall back. The final gate build was rerun on 2026-10-24: checkpoints 100.95 / 164.69 / 200.90 / 283.25s, recall .5148 / .8307 / .988 / .988, serial p99 22 / 24.9 / 4.9 / 4.7ms, and 100/110% QPS 999/1160 and 980/1214, consistent with the table within run variance.
Reproduce, and what not to do.
Full cases, parameters, and the result-record template are in VDBBENCH_PROTOCOL.en.md ↗. The shortest path to the HouyiDB-side self-test gate is below.
export VDB_ROOT=/path/to/VectorDBBench
python3 benchmarks/vectordb_bench_houyidb/install.py --vdb-root "$VDB_ROOT"
# Start houyi-server, then run
VDB_ROOT="$VDB_ROOT" ./benchmarks/vectordb_bench_houyidb/run_bench.sh
# 1× / old graph-merge path: set before starting the server
# HOUYI_HNSW_LAYER0_MULT=1 / HOUYI_HNSW_PLAN_APPLY=0- No total scoreTables come from different dates or code paths; each table is its own A/B.
- No mixed scopeDo not compare numbers across machines, modes (1×/2×), ef_search, concurrency, or interface paths.
- No snapshot flipDo not switch 1×/2× on an existing graph snapshot; confirm snapshot / WAL integrity or accept a full replay.
- Not a benchmarkThese are local engineering measurements from one machine and one run, not results from a lab or any external environment.
Data source: docs/benchmarks/houyidb-performance-2026-10.en.md ↗. Parameters and tuning knobs are in the server manual ↗.