Fork of akopytov/sysbench

sysbench Firebird Edition

Run the standard OLTP benchmarks used for MySQL and PostgreSQL against Firebird 3, 4, 5 and 6 — through the Firebird OO API, not the legacy ISC layer.

Driver
OO API · fb_c_api.h
Bulk loading
IBatch
Workloads
10 OLTP scripts
Lua scripts
Byte-identical to upstream

Building it

Requirements

sysbench's own dependencies:

# Debian/Ubuntu
apt -y install make automake libtool pkg-config libaio-dev
apt -y install libmysqlclient-dev libssl-dev   # MySQL/MariaDB support
apt -y install libpq-dev                       # PostgreSQL support

# RHEL/CentOS
yum -y install make automake libtool pkgconfig libaio-devel
yum -y install mariadb-devel openssl-devel     # MySQL/MariaDB support
yum -y install postgresql-devel                # PostgreSQL support

On top of those, the driver needs Firebird 5 or newer headers. fb_c_api.h first shipped in FB5; FB3 and FB4 install only the C++ interface headers, so they cannot be built against — though the binary still connects to FB3 and FB4 servers at runtime.

Distribution packages are generally too old: Debian 13 ships firebird-dev 4.0.5, and most distributions are still on FB3 or FB4. Either install the official tarball from firebirdsql.org, which includes include/firebird/, or build Firebird from source — the latter is also what you want if you intend to benchmark several versions side by side.

libfbclient.so is not linked at build time. The driver loads it with dlopen at startup, so only headers are needed to compile.

Build and install

./autogen.sh
./configure --with-mysql --with-pgsql --with-firebird=/opt/firebird
make -j$(nproc)
sudo make install        # optional — ./src/sysbench runs from the build tree

# check what was built
./src/sysbench --version
./src/sysbench --help | grep -A5 "firebird options"

--with-firebird takes the installation prefix; --with-firebird-includes and --with-firebird-libs set the paths explicitly if they live somewhere unusual. Add --without-mysql --without-pgsql for a Firebird-only build.

Run a benchmark

./src/sysbench ./src/lua/oltp_read_write.lua \
  --db-driver=firebird \
  --firebird-db=localhost:/var/lib/firebird/sbtest.fdb \
  --firebird-user=SYSDBA --firebird-password=masterkey \
  --tables=4 --table-size=10000 prepare

./src/sysbench ./src/lua/oltp_read_write.lua \
  --db-driver=firebird \
  --firebird-db=localhost:/var/lib/firebird/sbtest.fdb \
  --firebird-user=SYSDBA --firebird-password=masterkey \
  --tables=4 --table-size=10000 --threads=4 --time=10 run

Driver options

  • --firebird-db — connection string, e.g. localhost/3055:/path/sbtest.fdb
  • --firebird-user, --firebird-password
  • --firebird-client — which libfbclient.so to dlopen, so one binary can drive several Firebird versions

Supported versions

The client library is loaded at runtime and the driver checks its IUtil vtable before using it. That check is about the client, not the server — an FB3 server benchmarks fine when driven by an FB5 client.

VersionAs serverAs client libraryBatch API
Firebird 6.0Tested (trunk)Worksyes
Firebird 5.0Reference platformWorksyes
Firebird 4.0TestedWorks — cannot be built againstyes
Firebird 3.0TestedRefused at startup (vtable v2)server-side, no

The Batch API is a Firebird 4+ server feature, so bulk loading against FB3 falls back to single-row inserts — visible in the bulk_insert row below. For Firebird 3 and 2.5 clients, the legacy ISC driver lives on the firebird-isc branch.

Benchmarks

Four tables of 10,000 rows, four threads, ten seconds per benchmark. All six engines on the same disk, buffer caches matched at 128 MB, and the Firebird database on disk rather than tmpfs.

Transactions per second, 2026-10-14. Higher is better.
Benchmark FB 3.0.14 FB 4.0.7 FB 5.0.4 FB 6.0.0 MariaDB 11.8 PostgreSQL 17

Firebird trails on every workload here: reads roughly 3–4× behind, writes an order of magnitude. The cursor open/close round trips in Firebird's wire protocol are visible on the read benchmarks and are not something the driver can remove.

A result we had to withdraw

The benchmark write-ups in this repository reported Firebird winning oltp_insert by about 4× — 3,687 TPS against roughly 900 — and stated that the figure did not come from batching. Both halves were wrong. The numbers stood from May until October 2026.

  • Published
    3,687
    TPS — “leads by ~4×”
  • Measured
    153
    TPS — same workload, fixed driver
  • MariaDB
    1,149
    TPS — ~7.5× faster

oltp_insert.lua sends one INSERT per event through query(). The driver matched that statement shape and diverted it into the OO API batch path, which buffers 1,000 rows per round trip and commits them with commitRetaining. MariaDB and PostgreSQL committed every statement and paid an fsync each time. The gap was the measurement, not the engine.

The driver now batches only while a bulk load is genuinely in progress, which is how prepare and bulk_insert.lua arrive. Data loading is unchanged: preparing 4 × 10,000 rows still takes 5.0 seconds, and bulk_insert still measures about 40,000 TPS.

Single runs are not reliable here

Running the identical suite twice, back to back, moved Firebird's write benchmarks by factors of 0.37–1.42× while MariaDB and PostgreSQL stayed within 3%. One cause is background garbage collection: Firebird 5's first pass ran shortly after a million-row table was dropped, and its reads were depressed roughly 2× until a second pass.

Every figure here is the second of two consecutive runs. The write rows should not be read as version rankings.

Keeping it fair

  • Firebird data on disk, never tmpfs — same physical disk and mount options as the other engines.
  • Buffer caches matched at 128 MB across all engines.
  • Per-statement autocommit when no explicit BEGIN was sent, so each statement pays its own fsync, as on MySQL and PostgreSQL.
  • Batching confined to the bulk-load API, where every engine that has one is free to use it.
  • Raw sysbench output committed alongside the results, each file recording its own connection string and client library.

Corrections so far have come from Firebird core developers reviewing the numbers. The tmpfs imbalance, the unmatched caches and the long-transaction inserts were all found that way.

Contribute your results

Every number on this page comes from one machine: Debian 13 under WSL2 on a single-socket AMD desktop. That is enough to compare engines against each other on identical hardware, and not enough to say how Firebird behaves on bare metal, on more cores, on NVMe, or on a different filesystem. Results from other hardware are welcome, and they are the most useful thing anyone can send.

Run the suite

# one suite per engine; each writes benchmark_<driver>_<timestamp>.txt
TABLES=4 TABLE_SIZE=10000 THREADS=4 TIME=10 \
  FIREBIRD_DB=localhost:/var/lib/firebird/sbtest.fdb \
  ./run_benchmarks.sh firebird

./run_benchmarks.sh mysql
./run_benchmarks.sh pgsql

# run each engine twice and send both — the second is the one to quote

Before you measure

  • Put the Firebird database on disk, not under /tmp if that is tmpfs on your system.
  • Match the buffer caches: DefaultDbCachePages = 16384 at an 8 KB page size is the 128 MB used here, and give MariaDB and PostgreSQL the same.
  • Keep the machine otherwise idle. A compile running alongside a benchmark is visible in the numbers.
  • Run each engine twice. Firebird's write benchmarks moved by factors between repeats on this hardware, so a single pass is not evidence.

What to send

The raw benchmark_*.txt files, unedited. Each one records its own connection string and resolved client library in the header, so the engine and port are already in the file. Add a note of the CPU, the disk, the filesystem, and the Firebird, MariaDB and PostgreSQL versions you ran against.

Post them as a comment on the results thread, which already holds the runs above — no need to open a separate issue. By email works too. Pull requests adding files under results/<date>-<your-handle>/ are equally welcome, as is disagreement with the numbers here: every correction so far came from someone pointing out that a comparison was not fair.