online:
- user: admin@localhost
- password: secret itself claims
In this article, we'll compare TrailBase's performance to a few amazing, and certainly more weathered alternatives such as SupaBase, PocketBase, and vanilla SQLite.
Disclaimer
Generally benchmarks are tricky, both to do well and to interpret.
Benchmarks never show how fast something can theoretically go but merely how fast the author managed to make it go. Micro-benchmarks, especially, offer only a key-hole insights, which may be biased and may not apply to your workload.
Performance also doesn't exist in a vacuum. If something is super fast but doesn't do what you need it to do, performance is an illusive luxury. Doing less makes it naturally easier to go fast, which is not a bad thing, however means that comparing a specific aspect of a highly specialized solution to a more general one may be misleading, unfair or irrelevant for you.
We tried our hardest to give all contenders the best chance1 2 and were initially surprised by the observed performance gap ourselves. We suspect that given how quick SQLite itself is for simple queries, even small overheads weigh heavily. If you have any suggestions on how to make anyone go faster, make it more apples-to-apples or generally see any issues,
The graph shows the overall time it takes to insert 100k messages into a mock chat-room table setup. The less time it takes, the better.
Unsurprisingly, in-process vanilla SQLite is the quickest 3. All other setups add additional table look-ups for authorization, IPC overhead4, and layers of features on top. Think of the vanilla SQLite point as an upper bound on how fast one could go or the cost for adopting any of the other systems.
The measurement suggests that for this specific setup TrailBase can insert 100k records almost 70 times faster than Payload2, 9 to 16 times faster than SupaBase5, and roughly 6 to 7 times faster than PocketBase 1.
Total time of inserting a large batch of data tells only part of the story. Let's have a quick look at resource consumption to get an intuition for provisioning or footprint requirements, i.e. what kind of machine one would need:
TrailBase & PocketBase Utilization
including a dozen separate services providing various functionality:
SupaBase Memory Usage
Looking at the CPU usage, one can see it jump up to roughly 9 cores (the benchmark ran on a machine with 8 physical cores and 16 threads: 7840U). Most of the CPUs seem to be consumed by supabase-rest, the API frontend, with Postgres itself hovering at only about 0.7 cores. Also, supabase-analytics definitely seems to be in use.
Latency and Read Performance
Let's take a closer look at latency distributions. To keep things manageable we'll focus on PocketBase and TrailBase, which are architecturally simpler and more comparable.
For TrailBase, reads were on average 3.5 and insertions 6 times faster. That latter is in line with the throughput results we've seen above.
for both, PocketBase and TrailBase. This is meant as a proxy for a computationally heavy workload to primarily benchmark the performance of the underlying JavaScript engines: for TrailBase. In other words, the impact of any overhead within PocketBase or TrailBase is diminished by the time it takes to compute fibonacci(N) for sufficiently large N.
We found that for N=40, V8 (TrailBase) is around 40 times faster than goja (PocketBase):
execution model, one would expect similar behavior for a conservative JIT threshold in combination with a global interpreter lock 🤷. However, even after using all cores completing the benchmark takes significantly longer.
With the addition of V8 to TrailBase, we've experienced a significant increase in the memory baseline dominating the overall footprint. In this setup, TrailBase consumes roughly 4 times more memory than PocketBase. If memory footprint is a major concern for you, constraining the number of V8 threads will be an effective remedy (--js-runtime-threads).
Final Words
We're very happy to confirm that TrailBase's APIs and JS/ES6/TS runtime are quick. The significant performance gap we observed, especially for the APIs, might just be a consequence of how much even small overheads matter given how quick SQLite itself is.
With the numbers fresh off the press, prudence is of the essence and ultimately nothing beats benchmarking your own specific setup and workloads. In any case, we hope this was at least somewhat insightful. Let us know if you see anything that can or should be improved. The benchmarks are available on to be many times faster than popular options like Strapi or Directus. We were using a v3 pre-release, as recommended, also using the SQLite/drizzle database adapter marked as beta. We manually turned on WAL mode and filed an issue with payload, otherwise stock payload was ~210x times slower. ↩
Our setup with drizzle and node.js is certainly not the fastest possible. For example, we could drop down to SQLite in C or another low-level language with less FFI overhead. That said, drizzle is a great popular choice which mostly serves as a point-of-reference and sanity check. ↩
The actual magnitude on IPC overhead will depend on the communication cost. For the benchmarks at hand we're using a loopback network device. ↩
The SupaBase benchmark setup skips row-level access checks. Technically, this is in its favor from a performance standpoint, however looking at the overall load on its constituents with PG being only a sliver, it probably would not make much of an overall difference nor would PG17's vectorization, which has been released since the benchmarks were run. That said, these claims deserve re-validation. ↩
We're unsure as to what causes these 1-core swings. Runtime-effects, such as garbage collection, may have an effect, however we would have expected these to show on shorter time-scales. This could also indicate a contention or thrashing issue 🤷. ↩
SOCIAL SHARE CARD GENERATOR