Lane

Lifecycle

Opening, closing, deployment boundaries, and what waiting for the disk costs.

Illustration for DBBench, an open benchmark of database engines

Results

Tasks and results.

These are the costs that nobody benchmarks and everybody pays. A session table opens a database, does one thing and closes it, for each kind of data the engine can hold, embedded and served. It is the cold measurement in its purest form.

A second table separates the two costs of running a database in another process: turning the answer into a wire format, and crossing a process boundary. A third deployment in between, a server inside the same process, tells them apart.

The durability table runs every timed write twice, once with the commit returning without waiting for the disk and once with the commit waiting for the log to be flushed and synced. It prints both with the ratio. Some engines have no such setting, and the table names them instead of comparing them against a setting they do not have.

A session for each kind of data, embedded and served. The embedded rows also carry what starting the process costs before the first database call, because an open measured inside an already-running process is not what someone launching a script pays.

Session cost, open to close synthetic, one structure per row
Engine	Mode	Size	JVM start ms ↓	first open ms ↓	cold process ms ↓	open and close ms ↓	one query ms ↓	one write ms ↓	write, then query ms ↓	peak memory GiB ↓
Dense vectors	embedded	10k	182.1	222.7	500.2	14.3	26.7	24.4	33.3	1.12
Dense vectors	server	10k	—	—	—	16.3	34.3	27.8	38.5	1.37
Documents	embedded	10k	177.8	185.5	483.3	12.3	11.1	20.8	22.0	0.36
Documents	server	10k	—	—	—	14.8	15.1	24.0	28.6	0.40
Documents, ten indexes	embedded	10k	179.8	195.6	471.0	18.7	17.5	48.7	48.9	0.66
Documents, ten indexes	server	10k	—	—	—	20.4	23.5	53.2	56.4	0.88
Empty database	embedded	10k	179.1	184.3	455.1	11.3	11.6	20.3	21.1	0.26
Empty database	server	10k	—	—	—	13.1	14.9	22.4	23.8	0.30
Graph	embedded	10k	177.8	186.2	475.9	13.7	15.2	21.1	23.3	2.11
Graph	server	10k	—	—	—	15.2	18.6	30.2	26.7	2.01
Sparse vectors	embedded	10k	183.7	195.1	468.5	12.9	11.9	32.5	29.8	0.98
Sparse vectors	server	10k	—	—	—	14.9	15.7	33.6	33.7	1.15
Time series	embedded	10k	180.2	198.4	472.9	18.2	17.0	27.9	28.7	2.08
Time series	server	10k	—	—	—	21.2	21.0	29.3	32.9	2.01
Dense vectors (SurrealDB)	embedded	10k	—	79.1	207.3	54.7	59.4	53.6	61.2	re-run
Documents (SurrealDB)	embedded	10k	—	29.8	158.8	8.47	11.8	9.75	14.0	re-run
Documents, ten indexes (SurrealDB)	embedded	10k	—	129.7	259.5	114.0	116.4	116.4	117.7	re-run
Empty database (SurrealDB)	embedded	10k	—	22.9	150.7	2.60	2.55	4.08	3.96	re-run
Graph (SurrealDB)	embedded	10k	—	255.7	384.3	205.9	218.1	279.1	213.8	re-run
Time series (SurrealDB)	embedded	10k	—	34.1	163.0	9.82	13.8	10.9	14.2	re-run
Dense vectors	embedded	100k	186.2	230.7	507.6	22.4	103.5	32.6	111.1	2.88
Dense vectors	server	100k	—	—	—	17.4	107.9	34.9	117.7	4.16
Documents	embedded	100k	179.9	180.6	449.4	11.9	11.4	19.5	22.2	0.88
Documents	server	100k	—	—	—	14.0	15.8	27.8	27.9	0.94
Documents, ten indexes	embedded	100k	177.8	193.4	465.4	18.7	17.9	50.1	49.1	2.09
Documents, ten indexes	server	100k	—	—	—	20.7	21.6	53.6	57.0	2.06
Empty database	embedded	100k	180.3	184.1	454.7	12.5	11.8	20.9	21.5	0.25
Empty database	server	100k	—	—	—	12.4	13.9	22.9	25.5	0.30
Graph	embedded	100k	180.3	181.7	453.5	13.8	15.0	24.2	22.0	2.76
Graph	server	100k	—	—	—	16.1	17.9	30.1	27.9	2.75
Sparse vectors	embedded	100k	179.4	195.7	464.5	12.6	12.2	29.8	27.5	2.74
Sparse vectors	server	100k	—	—	—	14.1	16.1	35.1	33.6	4.11
Time series	embedded	100k	181.6	197.1	472.9	17.9	16.8	24.9	29.2	2.70
Time series	server	100k	—	—	—	20.4	21.0	32.7	30.1	2.69
Dense vectors (SurrealDB)	embedded	100k	—	666.9	796.8	730.3	621.6	623.4	632.8	re-run
Documents (SurrealDB)	embedded	100k	—	106.4	237.5	83.0	110.0	82.9	110.7	re-run
Documents, ten indexes (SurrealDB)	embedded	100k	—	1,033	1,162	1,247	1,251	1,257	1,249	re-run
Empty database (SurrealDB)	embedded	100k	—	20.7	149.3	2.55	2.42	3.86	4.10	re-run
Graph (SurrealDB)	embedded	100k	—	2,544	2,676	3,156	3,169	3,160	3,166	re-run
Time series (SurrealDB)	embedded	100k	—	124.9	253.9	118.0	170.2	120.6	171.0	re-run
Dense vectors	embedded	1M	178.6	209.1	482.6	98.0	853.1	98.1	872.3	12.2
Dense vectors	server	1M	—	—	—	16.2	897.3	103.9	902.0	12.5
Documents	embedded	1M	178.2	171.8	436.8	12.1	11.5	22.4	22.7	4.19
Documents	server	1M	—	—	—	14.0	13.8	21.5	30.1	4.22
Documents, ten indexes	embedded	1M	178.8	194.9	484.4	30.3	30.3	64.5	64.4	10.4
Documents, ten indexes	server	1M	—	—	—	35.4	37.3	68.3	69.1	10.3
Empty database	embedded	1M	186.9	175.2	447.4	12.1	12.1	20.7	21.0	0.47
Empty database	server	1M	—	—	—	13.1	14.5	24.5	24.7	0.52
Graph	embedded	1M	179.9	172.1	441.5	12.7	14.8	23.2	22.8	10.5
Graph	server	1M	—	—	—	15.6	17.0	27.8	28.2	10.5
Sparse vectors	embedded	1M	180.3	182.7	451.0	11.8	12.1	29.7	28.6	10.5
Sparse vectors	server	1M	—	—	—	13.3	14.8	30.1	28.9	10.5
Time series	embedded	1M	180.9	190.7	469.6	18.6	17.0	29.6	27.0	10.1
Time series	server	1M	—	—	—	21.4	21.0	33.0	27.7	10.2
Dense vectors (SurrealDB)	embedded	1M	>8h	>8h	>8h	>8h	>8h	>8h	>8h	—
Dense vectors	embedded	10M	179.3	206.9	480.4	1,412	18k	1,420	18k	24.7
Dense vectors	server	10M	—	—	—	15.2	18k	1,489	18k	24.7
Documents	embedded	10M	178.8	171.9	437.0	11.0	10.7	21.1	21.4	15.3
Documents	server	10M	—	—	—	13.9	13.8	27.2	28.1	15.3
Documents, ten indexes	embedded	10M	180.1	201.9	494.2	24.7	26.9	64.6	63.5	22.7
Documents, ten indexes	server	10M	—	—	—	28.4	32.4	65.8	62.9	22.1
Empty database	embedded	10M	179.4	174.1	441.7	11.3	11.4	21.4	20.8	0.60
Empty database	server	10M	—	—	—	14.1	14.8	26.7	24.9	0.68
Graph	embedded	10M	180.1	172.4	441.5	12.7	14.5	20.8	23.0	19.1
Graph	server	10M	—	—	—	14.9	16.8	28.9	24.0	19.1
Sparse vectors	embedded	10M	179.5	179.8	449.2	11.9	12.0	29.8	28.3	19.1
Sparse vectors	server	10M	—	—	—	13.0	14.8	33.8	27.5	19.1
Time series	embedded	10M	180.2	193.3	472.1	18.6	17.4	28.0	28.1	15.2
Time series	server	10M	—	—	—	21.0	21.0	26.5	31.2	15.3
Documents, ten indexes (SurrealDB)	embedded	10M	OOM	OOM	OOM	OOM	OOM	OOM	OOM	—
Graph (SurrealDB)	embedded	10M	OOM	OOM	OOM	OOM	OOM	OOM	OOM	—

↑ higher is better, ↓ lower is better; the best value in each column, within one size, is bold

Measured rows: benchmarks/experiments/results/runs_paper_oct.csv

The SESSION is open + action + close. Reporting open and close alone hides work triggered between them.
Cold start is measured in a fresh subprocess and reported beside every session number, because a millisecond open inside a process that takes half a second to reach its first database call is not a millisecond to whoever launched it.
A clean close should be O(what was written), not O(what is stored): write nothing and closing should cost the same at 10k documents and 10M.
Server rows have no JVM start, first open, or cold process: the server is already running when the probe connects, so those three columns describe the embedded process only. The session columns are measured for both.
Graph with the analytical view is withheld: its query grew from a bounded set of seeds to an unbounded 2-hop, so it stays off this table until the next measurement bounds the query again.
SurrealDB's peak memory on this table is withheld, and its 10M dense-vector cell is not shown: our build fed SurrealDB from one in-memory Python list while the ArcadeDB arm streamed, so its memory reading was partly ours, and the 10M vector build was killed for our allocation, not SurrealDB's. Its session timings ran after the build and stand. Both are re-measured in the next measurement, with the input streamed.
An engine with no row for a situation cannot build it, and its adapter says so rather than leaving the row blank; the declared row keeps the statement the adapter tried and the engine's own answer.
SurrealDB (embedded) has no row for the graph with the analytical view (no graph analytical view) or sparse vectors (no sparse-vector index type).
The dense-vector rows' one-query and write-then-query sessions, embedded and server alike, carry a defect in the engine build measured here, ArcadeDB issue #8852 (https://github.com/ArcadeData/arcadedb/issues/8852): the first vector search after a database is reopened re-reads every vector's record on one thread, so from a million vectors up those sessions are mostly that re-read, and ArcadeDB 26.10.1 fixes it.
The rows each session reads back are hashed, and for documents, documents with ten indexes, the graph, and time series every embedded engine that builds the situation must return the same hash as ArcadeDB embedded, or nothing is published. Only ArcadeDB (embedded) builds sparse vectors, so there is no second answer to compare. Not compared: the empty database, which reads nothing back; and dense vectors, whose read goes through an approximate index and is checked by recall on the dense vector table instead. The server rows are compared only with other server rows, because a server session runs a different set of modes and its record counts differ by construction; ArcadeDB (server) is the only engine served here, so they are compared with nothing.
ArcadeDB's embedded dense-vector rows read the index's statistics just before close, to record whether a rebuild ran. The read is outside the timers, but on this engine build it raises the cost of the close that follows it, so that row's open and close figure is an upper bound for a session that touches nothing.
SurrealDB rows have no JVM start: the SurrealDB core is a compiled extension that the Python import loads, so there is no runtime to start apart from the import. Their first open is the first open with the SDK already imported, and their cold process is interpreter start, import, open, and close, the same span the ArcadeDB embedded rows time. SurrealDB's time-series row is a plain table of timestamped records, the footing its time-series rows elsewhere on this page run on, because the engine has no time-series type.
SurrealDB (embedded) at 1M: the cell for dense vectors exceeded its 8 hour budget, the same budget every engine on this table had, on its first attempt and was not retried; there is no row.
SurrealDB (embedded) at 10M: the cell for documents with ten indexes reached the 28g memory envelope (peak 28,672 MiB) and was killed by the kernel, the same envelope every engine on this table had; it did not run out of time, and there is no row.
SurrealDB (embedded) at 10M: the cell for the graph reached the 28g memory envelope (peak 28,672 MiB) and was killed by the kernel, the same envelope every engine on this table had; it did not run out of time, and there is no row.
In the cells: `>8h` means the cell ran past its tier's cap and was not retried. `OOM` means killed at the cell's memory envelope rather than running out of time. `re-run` means the cell was measured before a fix to our harness and stays off the table until it is measured again; the note says what was wrong. A dash is an operation the engine cannot express, never a slow one.

One projection answered at a range of result sizes by three deployments: in process, an HTTP server inside the same process, and a separate container. Read a row across to see which of the two costs grows with the answer.

What the client/server split costs 200,000-document projection, one engine, three deployments
Engine	Size	in-process ms ↓	in-process server, HTTP ms ↓	separate container, HTTP ms ↓
1 documents	1	1.20	2.54	2.73
10 documents	10	0.94	1.81	2.02
100 documents	100	1.83	2.92	3.69
1,000 documents	1,000	3.10	4.70	4.36
10,000 documents	10,000	19.4	22.9	22.6
100,000 documents	100,000	198.9	207.9	258.1

↑ higher is better, ↓ lower is better; the best value in each column, within one size, is bold

Measured rows: benchmarks/experiments/results/e4decomp_417314c18

Every number is milliseconds. One engine build (arcadedb 26.10.1-dev · 417314c18) in all three deployments, 15 repetitions after 3 warmup, identical cpuset 0-11, memory cap 8g and heap 6g.
All three deployments turn the answer into Python objects the same way, so the difference is how the database was deployed and not how we read the result.
The separate container runs on the same machine, talking over the local network interface. It says what running the database beside your program costs, and says nothing about a database on another machine across a real network.

Every timed write at both durability settings, with the ratio, and the single-record read beneath the document writes as a control. The Size column names the operation rather than a corpus size, because the lanes that time a write do not write the same thing.

What waiting for the disk costs Every timed write, run twice, once at each durability setting
Engine	Mode	Size	no wait ms ↓	waits for the disk ms ↓	cost of waiting
ArcadeDB	embedded	TPC-C new-order, one transaction (documents)	0.46	8.21	18.0x
ArcadeDB	server	TPC-C new-order, one transaction (documents)	1.26	9.91	7.86x
ArangoDB	server	TPC-C new-order, one transaction (documents)	2.56	13.1	5.12x
DuckDB	embedded	TPC-C new-order, one transaction (documents)	—	10.7	—
MongoDB	server	TPC-C new-order, one transaction (documents)	0.57	6.83	11.9x
PostgreSQL	server	TPC-C new-order, one transaction (documents)	0.088	2.84	32.3x
SQLite	embedded	TPC-C new-order, one transaction (documents)	0.012	2.54	211.8x
SurrealDB	embedded	TPC-C new-order, one transaction (documents)	0.16	8.77	54.1x
SurrealDB	server	TPC-C new-order, one transaction (documents)	—	10.1	—
ArcadeDB	embedded	TPC-C payment, one transaction (documents)	0.19	8.51	45.6x
ArcadeDB	server	TPC-C payment, one transaction (documents)	0.88	10.5	11.9x
ArangoDB	server	TPC-C payment, one transaction (documents)	2.57	12.9	5.02x
DuckDB	embedded	TPC-C payment, one transaction (documents)	—	10.5	—
MongoDB	server	TPC-C payment, one transaction (documents)	0.45	6.76	15.0x
PostgreSQL	server	TPC-C payment, one transaction (documents)	0.083	3.06	36.9x
SQLite	embedded	TPC-C payment, one transaction (documents)	0.010	2.54	254.0x
SurrealDB	embedded	TPC-C payment, one transaction (documents)	0.14	8.64	60.4x
SurrealDB	server	TPC-C payment, one transaction (documents)	—	9.88	—
ArcadeDB	embedded	one record inserted (documents)	0.10	7.98	80.2x
ArcadeDB	server	one record inserted (documents)	0.70	10.2	14.6x
ArangoDB	server	one record inserted (documents)	0.51	10.1	19.9x
DuckDB	embedded	one record inserted (documents)	—	9.10	—
MongoDB	server	one record inserted (documents)	0.080	4.19	52.4x
PostgreSQL	server	one record inserted (documents)	0.043	3.53	83.1x
SQLite	embedded	one record inserted (documents)	0.0060	2.53	421.3x
SurrealDB	embedded	one record inserted (documents)	0.080	8.31	103.9x
SurrealDB	server	one record inserted (documents)	—	9.20	—
ArcadeDB	embedded	one record updated (documents)	0.084	8.00	95.3x
ArcadeDB	server	one record updated (documents)	0.64	10.2	15.8x
ArangoDB	server	one record updated (documents)	0.53	10.2	19.2x
DuckDB	embedded	one record updated (documents)	—	8.95	—
MongoDB	server	one record updated (documents)	0.11	4.33	38.0x
PostgreSQL	server	one record updated (documents)	0.045	3.55	79.0x
SQLite	embedded	one record updated (documents)	0.0055	2.51	456.6x
SurrealDB	embedded	one record updated (documents)	0.077	8.38	108.8x
SurrealDB	server	one record updated (documents)	—	9.17	—
ArcadeDB	embedded	one record deleted (documents)	0.18	8.55	46.2x
ArcadeDB	server	one record deleted (documents)	0.79	10.6	13.4x
ArangoDB	server	one record deleted (documents)	0.51	10.1	19.8x
DuckDB	embedded	one record deleted (documents)	—	8.95	—
MongoDB	server	one record deleted (documents)	0.11	4.29	40.1x
PostgreSQL	server	one record deleted (documents)	0.039	3.34	85.6x
SQLite	embedded	one record deleted (documents)	0.0050	2.52	503.8x
SurrealDB	embedded	one record deleted (documents)	0.12	8.37	72.8x
SurrealDB	server	one record deleted (documents)	—	10.7	—
ArcadeDB	embedded	one record read (documents) -- the control: a read commits nothing	0.031	0.027	0.87x
ArcadeDB	server	one record read (documents) -- the control: a read commits nothing	0.61	0.65	1.07x
ArangoDB	server	one record read (documents) -- the control: a read commits nothing	0.47	0.46	0.97x
DuckDB	embedded	one record read (documents) -- the control: a read commits nothing	—	0.16	—
MongoDB	server	one record read (documents) -- the control: a read commits nothing	0.092	0.087	0.95x
PostgreSQL	server	one record read (documents) -- the control: a read commits nothing	0.040	0.043	1.07x
SQLite	embedded	one record read (documents) -- the control: a read commits nothing	0.0020	0.0035	1.75x
SurrealDB	embedded	one record read (documents) -- the control: a read commits nothing	0.049	0.049	1.00x
SurrealDB	server	one record read (documents) -- the control: a read commits nothing	—	0.14	—
ArcadeDB	embedded	one person and one edge inserted (graph)	0.51	8.98	17.7x
ArcadeDB	server	one person and one edge inserted (graph)	1.28	11.2	8.78x
ArangoDB	server	one person and one edge inserted (graph)	0.88	10.5	12.0x
DuckPGQ	embedded	one person and one edge inserted (graph)	—	10.4	—
FalkorDB	server	one person and one edge inserted (graph)	0.17	8.83	53.2x
LadybugDB	embedded	one person and one edge inserted (graph)	—	re-run	—
Memgraph	server	one person and one edge inserted (graph)	0.15	9.31	61.0x
MongoDB	server	one person and one edge inserted (graph)	0.23	5.57	24.7x
Neo4j	server	one person and one edge inserted (graph)	—	7.36	—
SurrealDB	embedded	one person and one edge inserted (graph)	0.19	16.7	88.1x
SurrealDB	server	one person and one edge inserted (graph)	—	17.4	—
ArcadeDB	embedded	one person's property updated (graph)	0.20	8.55	42.4x
ArcadeDB	server	one person's property updated (graph)	0.84	10.5	12.5x
ArangoDB	server	one person's property updated (graph)	0.71	10.4	14.6x
DuckPGQ	embedded	one person's property updated (graph)	—	8.96	—
FalkorDB	server	one person's property updated (graph)	0.11	8.68	76.4x
LadybugDB	embedded	one person's property updated (graph)	—	re-run	—
Memgraph	server	one person's property updated (graph)	0.14	9.15	64.2x
MongoDB	server	one person's property updated (graph)	0.086	4.05	47.1x
Neo4j	server	one person's property updated (graph)	—	5.98	—
SurrealDB	embedded	one person's property updated (graph)	0.087	8.50	97.2x
SurrealDB	server	one person's property updated (graph)	—	9.12	—
ArcadeDB	embedded	one person and its edges deleted (graph)	0.48	9.30	19.6x
ArcadeDB	server	one person and its edges deleted (graph)	1.08	11.5	10.7x
ArangoDB	server	one person and its edges deleted (graph)	0.90	10.7	11.9x
DuckPGQ	embedded	one person and its edges deleted (graph)	—	10.5	—
FalkorDB	server	one person and its edges deleted (graph)	0.39	9.49	24.1x
LadybugDB	embedded	one person and its edges deleted (graph)	—	re-run	—
Memgraph	server	one person and its edges deleted (graph)	0.14	9.13	64.5x
MongoDB	server	one person and its edges deleted (graph)	0.32	5.82	18.4x
Neo4j	server	one person and its edges deleted (graph)	—	6.14	—
SurrealDB	embedded	one person and its edges deleted (graph)	0.17	9.38	54.2x
SurrealDB	server	one person and its edges deleted (graph)	—	10.5	—
ArcadeDB (one transaction)	embedded	one transaction across three models (cross-model, 50k products)	1.71	1.70	0.99x
ArcadeDB (one transaction)	server	one transaction across three models (cross-model, 50k products)	7.52	17.0	2.26x
ArangoDB	server	one transaction across three models (cross-model, 50k products)	re-run	re-run	re-run
MongoDB	server	one transaction across three models (cross-model, 50k products)	5.68	10.1	1.77x
Neo4j (vector index)	server	one transaction across three models (cross-model, 50k products)	—	re-run	—
PostgreSQL + pgvector + AGE	server	one transaction across three models (cross-model, 50k products)	4.93	8.45	1.71x
Qdrant + Neo4j (no shared transaction)	server	one transaction across three models (cross-model, 50k products)	—	re-run	—
SurrealDB	embedded	one transaction across three models (cross-model, 50k products)	7.88	16.1	2.04x
SurrealDB	server	one transaction across three models (cross-model, 50k products)	—	16.0	—
ArcadeDB (one transaction)	embedded	one transaction across three models (cross-model, 500k products)	2.15	2.16	1.00x
ArcadeDB (one transaction)	server	one transaction across three models (cross-model, 500k products)	7.92	18.1	2.29x
ArangoDB	server	one transaction across three models (cross-model, 500k products)	re-run	re-run	re-run
MongoDB	server	one transaction across three models (cross-model, 500k products)	6.14	9.58	1.56x
Neo4j (vector index)	server	one transaction across three models (cross-model, 500k products)	—	re-run	—
PostgreSQL + pgvector + AGE	server	one transaction across three models (cross-model, 500k products)	13.1	18.4	1.41x
Qdrant + Neo4j (no shared transaction)	server	one transaction across three models (cross-model, 500k products)	—	re-run	—
SurrealDB	server	one transaction across three models (cross-model, 500k products)	—	15.4	—

↑ higher is better, ↓ lower is better; the best value in each column, within one size, is bold

Each row is one engine running ONE operation twice: once where a commit returns without waiting for the disk, which is the setting every other table on this page reports, and once where the commit waits for the log to be flushed and synced. Nothing else about the cell changes.
Every timed write on the page is here: the six document operations, the three graph writes, and the cross-model transaction. Read down the operations for one engine rather than across the engines for one operation, because what the setting costs is a property of the engine's commit and the rest of the page already compares the engines.
The read is the control, and it is the row that makes the rest readable: it runs in both cells like everything else and commits nothing, so it is what the writes are moving against.
The Size column names the operation rather than a corpus size, and for the cross-model transaction the catalog size as well: the three lanes that time a write do not write the same thing, so each engine is compared only against the engines running its own operation.
The setting is a property of the cell, not a second measurement inside one: it lives on the database or on the server for most of these engines, so each pair of numbers is two runs.
DuckDB, DuckPGQ, and Neo4j have no setting to relax: DuckDB was traced to a sync at every commit, and DuckPGQ runs on DuckDB; Neo4j forces its log at commit as it documents, and offers no setting that changes it. So each prints one number, in the column for a commit that waits. Reading it against the other column would be reading a choice the engine does not offer.
SurrealDB (server) ran at its default in both runs, a sync to disk at every commit: the server log of every run reads "Sync mode: every transaction commit", and a trace counts one sync per commit. So the one number printed is in the waiting column. It has a setting that does not wait, which our harness missed, and the next measurement runs both.
In the runs that wait for the disk, ArcadeDB was set to sync its write-ahead log's data and metadata at every commit (`txWalFlush=2`, an fsync). The next measurement uses `txWalFlush=1` instead, a data-only sync (fdatasync) like SQLite's FULL setting, which ArcadeDB's documentation recommends for production and says recovers everything the full sync does.
ArcadeDB (one transaction) shows almost no cost of waiting because of a defect in the engine build measured here, ArcadeDB issue #8129 (https://github.com/ArcadeData/arcadedb/issues/8129): after the vector index is built, later commits in the same session skip the write-ahead log, so there is nothing to wait for, and ArcadeDB 26.10.1 fixes it.
SurrealDB (embedded) and SurrealDB (server) send the graph insert's person and its edge as two statements with no transaction around them, so each commits on its own: the run that waits for the disk waits twice for each insert, and the cost of waiting on that row counts both. The next measurement wraps the two in one transaction.
ArangoDB, LadybugDB, Neo4j (vector index), and Qdrant + Neo4j (no shared transaction) have cells marked `re-run` here because the same writes are withheld on the tables they come from, and the note under each of those tables says why.
The ratio is what the strict setting costs on that engine for that operation. The document and graph operations write the same few records whatever the corpus holds, so each row pools every corpus size its lane ran it at; the cross-model transaction searches and expands over the whole catalog, so it has one row per catalog size and its ratio holds at that size. It is not a claim about any other write. A large ratio is not a slow engine: it is an engine whose relaxed path was fast, measured against a flush that costs what a flush costs.
The document operations here are timed from the first one after the data is loaded, so their cells include each engine's warm-up. On ArcadeDB (embedded) and ArcadeDB (server), which run on a JVM, that warm-up includes the JVM compiling the statement path. The next measurement gives every engine an untimed warm-up on separate keys first, and this table prints their warm median.
In the cells: `re-run` means the cell was measured before a fix to our harness and stays off the table until it is measured again; the note says what was wrong. A dash is an operation the engine cannot express, never a slow one.

The rules every result follows are in the methodology. The settings for each engine are on the engines page. If one is wrong, tell us.

DBBench

The other lanes.

Tables: Documents and structured query language (SQL) analytics.
Graphs: Interactive and analytical queries on a social network graph.
Vectors: Dense and sparse vector search, with recall beside every latency.
Time series: Monitoring-style queries over timestamped readings.
Cross-model: One operation across a vector, a graph edge, and a document.

Resources

Rules and code.

All DBBench lanesOpen MethodologyOpen Engine settingsOpen Report a wrong settingOpen