Pricing

September Release Notes – TerrariumDB

In this month’s update, we’re sharing the release notes for TerrariumDB v8.0. This version introduces significant changes across storage, SQL, security, and observability, along with a range of improvements and fixes. In the article, you’ll find a breakdown of what’s new, what’s improved, and what to keep in mind for rollout and compatibility.

TerrariumDB v8.0 introduces a new Storage Core Format (SCF4) with pluggable storage engines and execution-path isolation, alongside major SQL additions — DELETE, WITH, multi-JOINs, and TRUNCATE ALL.

Security is strengthened with TLS-secured gRPC by default, and observability is upgraded across metrics and logging. Operationally, v8.0 reduces ingest service CPU utylization, and speedup backups. Under the hood, you’ll see faster, more predictable query execution via refined thread-pooling and scheduler work, plus numerous fixes that improve SQL correctness, stabilize aggregator ↔ worker flows, and ensure cleaner shutdown behavior.

What’s new

  • Pluggable storage engines — a stable Storage Engine API that decouples query/execution from persistence, letting the database load a single, process-wide engine implementation (e.g., row, columnar, LSM, in-memory, cloud-backed) at startup and swap/upgrade it without changing application code.
  • Parallel interpreter unification — merged GROUP BY and ORDER BY interpreters to improve parallel processing.
  • SQL: DELETE statement — native row removal in the SQL layer.
  • Segment index lookup endpoint — returns the matched segment index (or null) for a client through the aggregator/worker path.
  • SQL: WITH (CTE) support — richer query composition.
  • SQL: Multi-JOINs — support for multiple JOIN operations in a single query.
  • Rebalance tooling — shard rebalance script to assist controlled data movement across the cluster and aggregator.
  • gRPC TLS by default — single-file certificate chains, TLS 1.2+ (1.3 preferred), SAN validation.
  • Client-Segment Index Lookup - a new endpoint in analytics layer returns the matched segment index per client to speed real-time routing decisions.

Improvements

  • SCF4 engine support — new on-disk format with zero SCF2 conversion during reads; foundation for future type-specific optimizations.
  • SCF4 compression strategies — evaluated per-segment and whole-column approaches to cut storage and IO.
  • SQL thread-pool isolation — dedicated pool on workers for SQL layer (JOIN, GROUP BY, ORDER BY, etc.).
  • Dedicated ingestion pool (analytics layer) — separates ingestion from other SQL work to protect latency under write pressure.
  • TRUNCATE ALL TABLES — management command for bulk table truncation within a database.
  • Observability: SQL logging — richer fields (start/end, duration, parameters, rows, user/host, status) and worker execution times; reduced footprint for non-tracked query types.
  • Operational metrics — added user/host to MYSQL_ACTIVE_CONNECTIONS and new metrics for the aggregator/controller.
  • Security & auth UX — password-cache invalidation across gateways on ALTER USER; test suites for user/role and privilege surfaces; SHOW ROLE EDGES.
  • ConcurrencyRAND() rewritten to be multi-thread friendly.
  • Client compatibility — added default_storage_engine system variable required by sql client connectivity.

Fixes

  • Aggregator stability — safer error-data handling; prevent disconnections when coordination service session not initialized; cancel streaming RPCs before shutdown.
  • Time-zone performance — resolved slowdown caused by a custom date/time runtime layer.
  • SQL correctness
    • proper deserialization for GROUP BY;
    • fix * schema expansion; CAST … AS CHAR now copies by byte size;
    • proper escaping for patterns like \[abc] in LIKE;
    • whitespace stripped from SQL logs;
  • Build & robustness — Debian Trixie build fixed; guard against excessively deep JSON in serialized queries with a clear exception instead of generic “Unknown statement.”.

Compatibility & rollout notes

  • Storage format (SCF4). SCF4 becomes the primary format and removes on-read SCF2 conversions. Expect fewer physical files per dataset, faster backups, and a better path for future per-type optimizations. Plan migrations and backup policies accordingly.
  • Thread-pool separation. New pools for SQL and ingestion isolate workloads; validate CPU reservations/limits and update SLOs/alerts to reflect the new pools.
  • Secure transport. Enforce TLS for all gRPC listeners/clients using single-file PEM bundles (server and client trust). Verify SANs and cipher policies before enabling cluster-wide.

Further reading & benchmarks

SCF4 Storage Format

We delivered a brand-new storage layer with redesigned columnar files, custom vector encodings, and optimized dictionaries. This reduced disk usage by 20%, RAM by 31%, and lowered CPU load by ~30%, while also cutting query latencies significantly (up to 80% faster p50 under mixed workloads).

Smarter Column Encoding

Columns now leverage specialized chunk types for repeated, sorted, and boolean values (dictionary, dChunks, vChunks). These optimizations enable more compact storage and faster access, paving the way for further file-level improvements.

Ingest Threading Architecture

A new execution model of thread pool architecture for intestion services, combined with SCF4, improved CPU efficiency. CPU reservations on ingestion service dropped by ~45%, raising utilization efficiency from 57% → 66% and directly reducing cluster operating costs.

Use Cases

We’ve expanded our use case catalog with new, real-world scenarios! 🚀 Built on insights from our customers and powered by the latest features in our platform, these use cases are designed to help you unlock even more value.

September Release Notes – TerrariumDB

In this month’s update, we’re sharing the release notes for TerrariumDB v8.0. This version introduces significant changes across storage, SQL, security, and observability, along with a range of improvements and fixes. In the article, you’ll find a breakdown of what’s new, what’s improved, and what to keep in mind for rollout and compatibility.

TerrariumDB v8.0 introduces a new Storage Core Format (SCF4) with pluggable storage engines and execution-path isolation, alongside major SQL additions — DELETE, WITH, multi-JOINs, and TRUNCATE ALL.

Security is strengthened with TLS-secured gRPC by default, and observability is upgraded across metrics and logging. Operationally, v8.0 reduces ingest service CPU utylization, and speedup backups. Under the hood, you’ll see faster, more predictable query execution via refined thread-pooling and scheduler work, plus numerous fixes that improve SQL correctness, stabilize aggregator ↔ worker flows, and ensure cleaner shutdown behavior.

What’s new

  • Pluggable storage engines — a stable Storage Engine API that decouples query/execution from persistence, letting the database load a single, process-wide engine implementation (e.g., row, columnar, LSM, in-memory, cloud-backed) at startup and swap/upgrade it without changing application code.
  • Parallel interpreter unification — merged GROUP BY and ORDER BY interpreters to improve parallel processing.
  • SQL: DELETE statement — native row removal in the SQL layer.
  • Segment index lookup endpoint — returns the matched segment index (or null) for a client through the aggregator/worker path.
  • SQL: WITH (CTE) support — richer query composition.
  • SQL: Multi-JOINs — support for multiple JOIN operations in a single query.
  • Rebalance tooling — shard rebalance script to assist controlled data movement across the cluster and aggregator.
  • gRPC TLS by default — single-file certificate chains, TLS 1.2+ (1.3 preferred), SAN validation.
  • Client-Segment Index Lookup - a new endpoint in analytics layer returns the matched segment index per client to speed real-time routing decisions.

Improvements

  • SCF4 engine support — new on-disk format with zero SCF2 conversion during reads; foundation for future type-specific optimizations.
  • SCF4 compression strategies — evaluated per-segment and whole-column approaches to cut storage and IO.
  • SQL thread-pool isolation — dedicated pool on workers for SQL layer (JOIN, GROUP BY, ORDER BY, etc.).
  • Dedicated ingestion pool (analytics layer) — separates ingestion from other SQL work to protect latency under write pressure.
  • TRUNCATE ALL TABLES — management command for bulk table truncation within a database.
  • Observability: SQL logging — richer fields (start/end, duration, parameters, rows, user/host, status) and worker execution times; reduced footprint for non-tracked query types.
  • Operational metrics — added user/host to MYSQL_ACTIVE_CONNECTIONS and new metrics for the aggregator/controller.
  • Security & auth UX — password-cache invalidation across gateways on ALTER USER; test suites for user/role and privilege surfaces; SHOW ROLE EDGES.
  • ConcurrencyRAND() rewritten to be multi-thread friendly.
  • Client compatibility — added default_storage_engine system variable required by sql client connectivity.

Fixes

  • Aggregator stability — safer error-data handling; prevent disconnections when coordination service session not initialized; cancel streaming RPCs before shutdown.
  • Time-zone performance — resolved slowdown caused by a custom date/time runtime layer.
  • SQL correctness
    • proper deserialization for GROUP BY;
    • fix * schema expansion; CAST … AS CHAR now copies by byte size;
    • proper escaping for patterns like \[abc] in LIKE;
    • whitespace stripped from SQL logs;
  • Build & robustness — Debian Trixie build fixed; guard against excessively deep JSON in serialized queries with a clear exception instead of generic “Unknown statement.”.

Compatibility & rollout notes

  • Storage format (SCF4). SCF4 becomes the primary format and removes on-read SCF2 conversions. Expect fewer physical files per dataset, faster backups, and a better path for future per-type optimizations. Plan migrations and backup policies accordingly.
  • Thread-pool separation. New pools for SQL and ingestion isolate workloads; validate CPU reservations/limits and update SLOs/alerts to reflect the new pools.
  • Secure transport. Enforce TLS for all gRPC listeners/clients using single-file PEM bundles (server and client trust). Verify SANs and cipher policies before enabling cluster-wide.

Further reading & benchmarks

SCF4 Storage Format

We delivered a brand-new storage layer with redesigned columnar files, custom vector encodings, and optimized dictionaries. This reduced disk usage by 20%, RAM by 31%, and lowered CPU load by ~30%, while also cutting query latencies significantly (up to 80% faster p50 under mixed workloads).

Smarter Column Encoding

Columns now leverage specialized chunk types for repeated, sorted, and boolean values (dictionary, dChunks, vChunks). These optimizations enable more compact storage and faster access, paving the way for further file-level improvements.

Ingest Threading Architecture

A new execution model of thread pool architecture for intestion services, combined with SCF4, improved CPU efficiency. CPU reservations on ingestion service dropped by ~45%, raising utilization efficiency from 57% → 66% and directly reducing cluster operating costs.

Product Newsletter
Stay up to date with the latest Synerise product updates, new features, and practical insights — delivered straight to your inbox. Just subscribe to our weekly communication!
Read about our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Key benefits

Explore the core advantages of this feature and discover the value it brings to your daily work with Synerise.
No items found.

Use Cases

Explore real-life use cases that demonstrate how to apply this feature in practice through inspiring, ready-to-use scenarios that solve real challenges.
No items found.

Share your feedback with us!

Want to share your thoughts or have any questions about this article?
Leave us a message — we’d love to hear your opinion!
Synerise is the controller of your personal data processed for the purpose of fulfilling your request specified in this form. You may withdraw your consent at any time by contacting us. For more information on how we process your personal data and what rights you have, see our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.