Today we are releasing pgrust v0.4-beta. As the name suggests, this is a beta release rather than the final v0.4 release, and we are sharing it now to show you what’s coming in v0.4 while we finish stabilizing the remaining pieces.

If v0.3 was about smashing bugs and proving out our Postgres compatibility, v0.4-beta is about pushing the ceiling on what a modern database engine can do. This release introduces Gravity (our new query engine that runs as much as 3× faster than ClickHouse), adds full MVCC support to pgrcolumnar, and enables pgrcolumnar to sit directly on top of row stores. It also eliminates manual memory tuning for work_mem and buffer caches, and brings native statistics to jsonb.

We are also changing how we build pgrust. Starting today, we are moving away from big monthly code drops and switching to continuous development out in the open.

Here is a look at what is new in v0.4-beta.

The Gravity Query Engine: As Much As 3× Faster Than ClickHouse, 100% Postgres

In v0.2, our query engine edged out ClickHouse on ClickBench by 18.5%. For v0.4-beta, we went back to the drawing board and built Gravity, a brand-new query execution engine that is as much as 3× faster than ClickHouse.

To get that kind of speedup, we rewrote our JIT compiler from the ground up to maximize performance. Previously, our JIT only targeted ARM64. With Gravity, we have introduced a new low-level Lisp-based intermediate representation (IR). This Lisp IR made it easy to port Gravity across multiple platforms, enabling us to add JIT support for x86, ARM, and WASM. Underneath that IR, we heavily optimized the generated assembly code to squeeze every bit of performance we can.

On top of better performance, Gravity supports all PostgreSQL queries and types natively. You get standard Postgres semantics running at as much as 3× the speed of a dedicated analytical engine. Gravity also works on top of the traditional PostgreSQL row store. Queries over more than a few hundred rows on the row store can be up to 5x faster.

pgrcolumnar: Full MVCC, Fast Writes, and Columnar Indexes on Row Stores

Columnar storage in Postgres (whether through extensions or external engines) has historically come with frustrating tradeoffs: slow point updates, append-only limitations, missing index support, or having to migrate your tables entirely out of standard row storage.

In v0.4-beta, pgrcolumnar removes those limitations:

  • Adds full MVCC support: In practice, this means you get full transactional support on columnar tables. You can run concurrent reads and writes inside transactions with standard Postgres isolation levels, rollbacks, and snapshot visibility, just like you would on a regular heap table.
  • New columnar index: You can now create a columnar index directly on top of an existing row store table. This gives you columnar-level analytical speed without forcing you to convert your underlying table away from a traditional row store.
  • Fast insertions, deletions, and updates: pgrcolumnar now supports fast, high-throughput insertions, deletions, and updates directly over columnar tables without degrading write performance as data changes.
  • Supports all traditional Postgres indexes: Every standard Postgres index type (such as B-tree, GIN, and GiST) now works out of the box on pgrcolumnar tables.

Automatic Memory Management: Dynamic work_mem and Buffer Cache

One of the longest-running operational headaches in PostgreSQL is tuning memory. If you set work_mem too low, complex queries spill to disk needlessly and run slowly. If you set it too high, a spike in concurrent sorts or hash joins can exhaust system memory and trigger the OOM killer. Meanwhile, shared_buffers sits at a static allocation regardless of what else the database is doing.

Because pgrust uses a single-process, thread-per-connection architecture we can construct a single unified view of all of memory. We eliminate the pain of tuning memory with:

  • Automatic work_mem: Query operators (like hash tables and sorts) will now automatically take more memory as long as it is available on the system. Only when free memory is running low will pgrust select a query and spill its memory to disk.
  • Automatic buffer cache sizing: Instead of locking memory behind a fixed configuration parameter picked at startup time, the buffer cache now automatically resizes itself based on your idle memory, expanding when memory is free and yielding it back when queries need working memory.

Native jsonb Statistics

In standard PostgreSQL, the query planner does not maintain deep statistics on the keys and values inside jsonb columns. Because the planner has to guess at selectivities, queries over jsonb frequently end up with terrible row estimates and slow execution plans.

pgrust v0.4-beta adds native statistics support for jsonb. Because the planner now has accurate stats on the data inside your jsonb columns, it can pick dramatically better join orders and index scans. Depending on your workload, your jsonb queries can get orders of magnitude faster out of the box.

Developing Out in the Open

Up to this point, we have done most of our development privately and published our work in large monthly releases like v0.2 and v0.3.

Starting today, we are doing all of our development out in the open. Instead of holding changes back for big monthly drops, we are pushing every feature, optimization, and bug fix directly into the public repository as soon as it is ready.

Try It Out

Because v0.4-beta is a beta release, please remember that pgrust is still experimental and not ready to be the primary database for data you cannot afford to lose. Keep standard PostgreSQL as your source of truth, and spin up pgrust on analytical replicas or test workloads to see what it can do.

Known limitation: memory. pgrust's analytics memory manager is new. On small machines or very large queries, some analytical queries spill intermediate results to disk and run slower, and a few may be refused rather than risk running the server out of memory. We're actively working on it; if you hit it, please open an issue with the query.

You can check out the code, try out v0.4-beta, and follow our daily commits on GitHub at github.com/malisper/pgrust. If you run into any bugs or find a query where pgrust diverges from Postgres, please open an issue.

Thanks to our early users, including Kanu, PGX and Folio, for building with pgrust.