Lane
Cross-model
One operation across a vector, a graph edge, and a document.
Results
Tasks and results.
Most lanes compare engines that do one thing. This lane asks what a single engine buys when one operation touches more than one data model. The comparators are other engines that hold several models and, beside them, a composed stack: separate stores with application code between them.
The retrieval path is a vector search, a one-hop expansion from what it found, and a projection of the documents behind those. A second path restricts the vector search to the neighbourhood of a starting node. Both report recall against a brute-force answer over the same candidate set.
The crash table is a correctness result, not a speed one. The same operation is interrupted part-way through, many times, and afterwards the store is asked what it holds. It has no latency column on purpose.
The composed write, the retrieval path, and the graph-filtered vector search, with recall for the two read paths. The write is a transaction on the engines that have one and a sequence of calls on the stack that does not.
| Engine | Mode | Size | transaction p50 ms ↓ | transaction p99 ms ↓ | retrieval p50 ms ↓ | graph-filtered search p50 ms ↓ | retrieval recall@10 | filtered recall@10 | ingest+index vertices+edges/s ↑ | ingest+index total s ↓ | peak memory GiB ↓ | disk GiB ↓ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ArcadeDB (one transaction) | embedded | 50k products | 1.71 | 3.19 | 2.62 | 6.81 | 0.77 | 0.069 | 21k | 9.50 | 2.41 | 0.038 |
| ArcadeDB (one transaction) | server | 50k products | 7.52 | 9.87 | 5.29 | 9.74 | 0.79 | 0.069 | 15k | 13.6 | 4.09 | 0.098 |
| ArangoDB | server | 50k products | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run |
| MongoDB | server | 50k products | 5.68 | 9.07 | 7.62 | 6.70 | 0.84 | 1 | 6,993 | 28.6 | 1.18 | 0.070 |
| Neo4j (vector index) | server | 50k products | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run |
| PostgreSQL + pgvector + AGE | server | 50k products | 4.93 | 7.32 | 2.41 | 1.25 | 0.78 | 1 | 24k | 8.23 | 0.12 | 0.16 |
| Qdrant + Neo4j (no shared transaction) | server | 50k products | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run |
| SurrealDB | embedded | 50k products | 7.88 | 8.87 | 9.30 | re-run | 0.70 | 1 | 620 | 322.7 | 0.84 | 0.24 |
| SurrealDB | server | 50k products | re-run | re-run | 5.17 | re-run | 0.73 | 1 | 16k | 12.2 | 0.90 | 0.47 |
| ArcadeDB (one transaction) | embedded | 500k products | 2.15 | 4.41 | 6.46 | 9.33 | 0.42 | 0.013 | 17k | 117.4 | 14.1 | 0.39 |
| ArcadeDB (one transaction) | server | 500k products | 7.92 | 10.3 | 8.47 | 11.9 | 0.42 | 0.012 | 11k | 190.2 | 11.6 | 0.42 |
| ArangoDB | server | 500k products | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run |
| MongoDB | server | 500k products | 6.14 | 7.75 | 8.70 | 7.09 | 0.69 | 1 | 35k | 57.6 | 3.92 | 0.85 |
| Neo4j (vector index) | server | 500k products | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run |
| PostgreSQL + pgvector + AGE | server | 500k products | 13.1 | 15.2 | 6.11 | 1.37 | 0.41 | 1 | 18k | 113.6 | 0.67 | 1.51 |
| Qdrant + Neo4j (no shared transaction) | server | 500k products | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run | re-run |
| SurrealDB | embedded | 500k products | >2h | >2h | >2h | >2h | >2h | >2h | >2h | >2h | >2h | >2h |
| SurrealDB | server | 500k products | re-run | re-run | re-run | re-run | re-run | 1 | 15k | 137.4 | 6.49 | 0.69 |
↑ higher is better, ↓ lower is better; the best value in each column, within one size, is bold
Measured rows: benchmarks/experiments/results/runs_paper_oct.csv
Exact builds measured
- arcadedb 26.10.1-dev · 417314c18 (in-process)
- arcadedb 26.10.1-dev · 417314c18
arcadedb-c25:417314c18 - ArangoDB
arangodb@563cb2c07af0… - mongodb 8.2.12 + MongoDb Search Community Version 1.70.4
dbbench:mongo-search - Neo4j (vector index), Qdrant + Neo4j (no shared transaction)
neo4j@e702d6b535d9… - PostgreSQL 18.6 + pgvector 0.8.6 + age 1.8.0
dbbench:pg-age - surrealdb 2.3.10
- surrealdb 3.2.4
surrealdb/surrealdb@6a5002363ff5…
- The engines do not all run the same SHAPE of filtered search, and it shows in the recall column as much as the time one. ArangoDB, MongoDB, Neo4j (vector index), PostgreSQL + pgvector + AGE, Qdrant + Neo4j (no shared transaction), SurrealDB (embedded), and SurrealDB (server) restrict the candidate set first and rank what is left by distance, so they return that set's true nearest matches. ArcadeDB (one transaction) and ArcadeDB (server, one transaction) instead take an over-large global top-k and keep whichever candidates happen to fall inside it: more work, and a partial answer, because a global list of that size usually does not contain enough of the candidates. Read both columns for those arms as what this query shape costs rather than as what the engine can do.
- The transaction is the update, and every engine here commits it all or nothing. ArcadeDB (one transaction), ArcadeDB (server, one transaction), Neo4j (vector index), and PostgreSQL + pgvector + AGE also run the vector search and the hop inside it; ArangoDB, MongoDB, SurrealDB (embedded), and SurrealDB (server) run those two reads first and wrap only the update (MongoDB cannot do otherwise: it refuses a vector search inside a multi-document transaction, and its rows record the refusal). One client runs at a time, so the answers are the same either way. Qdrant + Neo4j (no shared transaction) has no transaction spanning its two systems, which is what that row exists to show.
- Disk is what the workload left on disk, in GiB: the engine's writable layer plus its volumes after the cell, minus the same engine's empty footprint.
- ArangoDB's row is marked `re-run`: its rows on this table were measured before the index on the product id that the protocol builds for it existed, so its reads scanned every product. They are withheld until it is measured with the index.
- SurrealDB's graph-filtered search is marked `re-run`: our query scanned every product instead of reading the candidates by record id, the access path every other engine on this table was given. It is being re-measured with the fix.
- SurrealDB (server) retrieval at the 500k tier is marked `re-run`: the server was still building its vector index in the background when these queries ran. It is being re-measured with the build waiting for the index to catch up.
- Qdrant + Neo4j's row is marked `re-run`. Its vector half ran in the Qdrant client's in-memory local mode, a pure-Python reimplementation rather than the Qdrant server, so every time, recall, and disk value it produced described that reimplementation. It is being re-run against the Qdrant server the vector table uses. Its all-or-nothing result on the atomicity table below does not depend on this.
- Neo4j's row is marked `re-run`: its vector index ran at Neo4j's default quantization, binary with a three-fold search expansion, which our index definition did not override, so its vector search was not the unquantized search the other engines ran. The next measurement sets the quantization explicitly. Its all-or-nothing result on the atomicity table does not depend on the index.
- SurrealDB (server)'s transaction cells are marked `re-run`: the server ran at its default, a sync to disk at every commit, although it has a setting that does not wait, and every other engine on this table that has such a setting ran at it. Our harness missed that setting, and the next measurement runs the transaction at it.
- There is no cold column on this table, and the reason is recorded on the rows themselves: no cold/warm split: each operation runs against an already-built, already-warm database by construction
- Durability is matched at the relaxed end: on every engine that has the setting, a commit returns without waiting for the disk and the log is flushed by the engine's own background policy, which is ArcadeDB's engine default and a documented production mode for each of the others. Each row records what it ran as `durability`, and the setting was read out of the engine rather than assumed on every engine except ArcadeDB (server, one transaction) and SurrealDB (server). ArcadeDB (server, one transaction) is given the setting when the server starts, and the row records it, because the server offers no way to read it back. SurrealDB (server)'s rows record that no setting was found, and its server log shows what it ran: its default in both runs, a sync to disk at every commit (the log reads "Sync mode: every transaction commit" in every run), although it has a setting that does not wait; the next measurement runs both. ArcadeDB (one transaction) loaded its catalog with the write-ahead log off, which the bulk-load helper it used does by default, so its ingest figures had less durability than the relaxed setting; its timed operations ran with the log on. The next measurement loads with the log on.
- Atomic means all or nothing: the whole update happens, or none of it does, with no state in between that anyone can observe. One engine can promise that across a vector, a graph edge, and a document because they share a transaction. Qdrant and Neo4j cannot promise it to each other, because nothing spans the two.
- So the interesting result here is not the speed. It is what a crash halfway through leaves behind, and the atomicity table below shows it: for each engine, how many interrupted writes left the store holding some but not all of the write after it was reopened. That is what this comparison exists to show.
- SurrealDB embedded runs on the SDK's SurrealKV store on disk and SurrealDB server on RocksDB, and each has its own disk reading.
- ArcadeDB (one transaction) and ArcadeDB (server, one transaction) run on a JVM, and for them this column is close to the heap they were given rather than the memory the work needed: ArcadeDB (server, one transaction)'s 4.09 GiB here is about 68 per cent of its 6g heap. Every engine on a table gets the same memory envelope; how much of it a JVM takes is a property of the runtime, so read these cells against each other rather than against an engine that manages its own memory.
- Each repetition runs the transaction 300 times; the p50 and p99 are over those.
- Indexes on this table are matched by effect rather than by rule: ArcadeDB (one transaction), ArcadeDB (server, one transaction), MongoDB, PostgreSQL + pgvector + AGE, SurrealDB (embedded), and SurrealDB (server) build one. Where building one was a choice, the engine was timed with it and without it and the faster configuration kept; where an engine reaches these queries through its own primary key or record id, that is its equivalent. This table has no index column: the time an index took to build is inside the ingest+index total.
- SurrealDB (embedded) at 500k products: the transaction cell exceeded its 2 hour budget, the same budget every engine on this table had, on its first attempt and was not retried; there is no row. It was still in the index build when the budget ran out.
- In the cells: `>2h` means the cell ran past its tier's cap and was not retried. `re-run` means the cell was measured before a fix to our harness and stays off the table until it is measured again; the note says what was wrong. A dash is an operation the engine cannot express, never a slow one.
- ingest+index total s is one timer around loading the vertices and edges and creating the vector index. Ingest paths: ArcadeDB embedded loads with the Python package's graph_batch (5,000 records per commit, vertices then edges) and then CREATE INDEX ... LSM_VECTOR; served sends CREATE VERTEX and CREATE EDGE batches as sqlscript over HTTP, then the same CREATE INDEX; SurrealDB embedded inserts through its Python SDK into the SDK's SurrealKV store on disk, and SurrealDB server through the same SDK over WebSocket onto RocksDB; Neo4j and the composed stack load the graph with UNWIND over bolt, and the composed stack upserts its vectors into Qdrant; PostgreSQL + pgvector + AGE loads the products with COPY under a pgvector HNSW index and creates the graph with UNWIND inside Cypher; ArangoDB import_bulk; MongoDB insert_many.
Interrupted composed operations, and how many of them left the store holding half of one. This table has no latency column on purpose: it is a correctness result, not a speed one.
| Engine | Mode | Size | trials | torn results ↓ |
|---|---|---|---|---|
| ArcadeDB (one transaction) | embedded | 50k products | 40 | 0 |
| ArcadeDB (one transaction) | server | 50k products | 40 | 0 |
| ArangoDB | server | 50k products | 40 | 0 |
| MongoDB | server | 50k products | 40 | 0 |
| Neo4j (vector index) | server | 50k products | 40 | 0 |
| PostgreSQL + pgvector + AGE | server | 50k products | 40 | 0 |
| Qdrant + Neo4j (no shared transaction) | server | 50k products | 40 | 40 |
| SurrealDB | embedded | 50k products | 40 | 0 |
| SurrealDB | server | 50k products | 40 | 0 |
| ArcadeDB (one transaction) | embedded | 500k products | 40 | 0 |
| ArcadeDB (one transaction) | server | 500k products | 40 | 0 |
| ArangoDB | server | 500k products | 40 | 0 |
| MongoDB | server | 500k products | 40 | 0 |
| Neo4j (vector index) | server | 500k products | 40 | 0 |
| PostgreSQL + pgvector + AGE | server | 500k products | 40 | 0 |
| Qdrant + Neo4j (no shared transaction) | server | 500k products | 40 | 40 |
| SurrealDB | embedded | 500k products | >2h | >2h |
| SurrealDB | server | 500k products | 40 | 0 |
↑ higher is better, ↓ lower is better; the best value in each column, within one size, is bold
Measured rows: benchmarks/experiments/results/runs_paper_oct.csv
Exact builds measured
- arcadedb 26.10.1-dev · 417314c18 (in-process)
- arcadedb 26.10.1-dev · 417314c18
arcadedb-c25:417314c18 - arangodb 3.12.11
arangodb@563cb2c07af0… - mongodb 8.2.12 + MongoDb Search Community Version 1.70.4
dbbench:mongo-search - neo4j 2026.08.1
neo4j@e702d6b535d9… - PostgreSQL 18.6 + pgvector 0.8.6 + age 1.8.0
dbbench:pg-age - qdrant-client 1.19.1 (local mode) + neo4j 2026.08.1
neo4j@e702d6b535d9… - surrealdb 2.3.10
- surrealdb 3.2.4
surrealdb/surrealdb@6a5002363ff5…
- A trial writes the three products, kills the process between them, reopens, and checks whether every product is present or none. Torn means some but not all.
- SurrealDB (embedded) at 500k products: the atomicity cell exceeded its 2 hour budget, the same budget every engine on this table had, on its first attempt and was not retried; there is no row. It was still in the index build when the budget ran out.
- In the cells: `>2h` means the cell ran past its tier's cap and was not retried. A dash is an operation the engine cannot express, never a slow one.
Which multi-model engines have rows on which table, read from the published tables. Each cell says whether the engine was measured there, was left off with a reason the table itself prints, or was not run on that workload at all.
| Engine | Mode | Size | Document OLTP | Document OLAP | Graph OLTP | Sparse vector search | Dense vector search | Cross-model transaction | Cross-model transaction: what survives a crash | What waiting for the disk costs | Session cost, open to close | What the client/server split costs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ArcadeDB | embedded, server | every size above | measured | measured | measured | measured | measured | measured | measured | measured | measured | measured |
| ArangoDB | server | every size above | measured | measured | measured | declared, cannot express | measured | declared, withdrawn | measured | measured | no arm | no arm |
| MongoDB | server | every size above | measured | measured | measured | declared, cannot express | measured | measured | measured | measured | no arm | no arm |
| SurrealDB | embedded, server | every size above | measured | measured | measured | declared, cannot express | measured | measured | measured | measured | measured | no arm |
Measured rows: benchmarks/experiments/results/runs_paper_oct.csv
- This table is derived from the ten tables above and carries no measurement of its own. Each column is one of those tables. measured means the engine has at least one row on that table, counting every arm of one engine as one, so both ArcadeDB deployments are ArcadeDB and both SurrealDB modes are SurrealDB; declared means the table has no row for the engine and says why beneath itself, in the words printed there (cannot express, the engine's own language cannot ask the query; withdrawn, the row was measured before a fix to our harness and came down until it is measured again); no arm means the engine was not run on that workload, with neither a row nor a declared reason. The Mode column lists the deployments the engine has rows for anywhere on the page.
The rules every result follows are in the methodology. The settings for each engine are on the engines page. If one is wrong, tell us.
DBBench
The other lanes.
- Tables: Documents and structured query language (SQL) analytics.
- Graphs: Interactive and analytical queries on a social network graph.
- Vectors: Dense and sparse vector search, with recall beside every latency.
- Time series: Monitoring-style queries over timestamped readings.
- Lifecycle: Opening, closing, deployment boundaries, and what waiting for the disk costs.
Resources