# Storage Stack Options

> ## Flash Media Access Options for Real-Time Databases

> From a database management perspective, there are three main ways to access flash memory:

> 1. Using low-level, vendor-specific media drivers,
> 2. Leveraging third-party FTLs (Flash Translation Layers) or block-device interfaces, and
> 3. Employing flash-aware or general-purpose file systems.

> Each option presents trade-offs in terms of performance, complexity, and latency control.

> Real-time systems in general — and eXtremeDB/rt in particular — must offer predictable, bounded access latency. Third-party FTLs and file systems, however, introduce significant variability. FTLs are typically geared toward extending flash lifespan, and in doing so, introduce latency through internal operations such as concurrent writes, garbage collection, wear leveling, read scrubbing (error correction via ECC), and block erase cycles.

> File systems add further unpredictability with read caching, write buffering, and additional I/O to maintain layout metadata. They can also incur allocation delays and lock contention due to competing file system tenants. These behaviors complicate efforts to ensure worst-case latency bounds, which are essential for deterministic transaction processing in real-time environments.

> Direct access to flash through low-level interfaces offers the greatest control. Unlike general-purpose file systems, eXtremeDB/rt does not require file system-style metadata hierarchies. Instead, the database kernel organizes data using an internal, storage-specific layout via integrated page managers, designed for random access patterns and other database-specific workload characteristics. This makes it practical for the kernel to manage wear leveling, bad block remapping, and garbage collection internally. Media-specific delays such as block erasure can also be accounted for in transaction scheduling.

> Furthermore, to further optimize performance, eXtremeDB/rt integrates flash management operations directly with transaction processing, taking advantage of the fact that both rely on mapping logical pages to physical pages. This integration is only possible when the kernel accesses flash media directly through drivers supplied by device manufacturers or operating system vendors.

> As a practical compromise, accessing flash through a block-device interface—typically exposed by a third-party FTL—reduces integration complexity while still offering more stable performance than full file systems. This approach provides a consistent logical block abstraction and internalizes many low-level flash behaviors. While not perfectly deterministic, it is often sufficient for soft real-time workloads.

> ## eXtremeDB/rt Flash Storage Integration

> eXtremeDB/rt supports all three flash access strategies through a modular architecture known as the storage stack, which is responsible for storing, organizing, and retrieving data efficiently while ensuring reliability and transactional consistency.

> ## Direct Flash Access (Transactional FTL):

> For hard real-time systems or architectures that lack an FTL or file system, eXtremeDB/rt provides a Transactional Flash Translation Layer. This component interfaces directly with low-level media drivers and yields the highest degree of control and determinism. Note: The required driver may or may not be included in the eXtremeDB/rt distribution.

> ## Block Device Access (Mapped File Interface):

> For systems that include a third-party FTL and expose a block device interface, but do not use a file system, the storage stack supports a Copy-on-Write Mapped File transaction module. This option balances deterministic behavior and development effort and is a strong fit for systems requiring moderate latency guarantees.

> ## File System Access (File System Wrappers):

> For architectures with an integrated file system, eXtremeDB/rt can operate over it using file system wrappers. These components abstract file system differences and present a unified interface to the database kernel. Transactions are still handled by the Mapped File component, but instead of accessing a block device, the underlying interface is the file system. Wrappers are available for a wide range of commercial and custom file systems.

> Database Kernel Virtual Disk Module - Provides virtualized storage access to the database kernel using abstract read, write commit, and rollback services. This module is implemented in the mcovtdsk library.
> Transactional File
> Option
> TFTL Option
> Transactional Mapped File - Implements Copy-on-Write (CoW)-based transaction processing and maintains logical-to-physical address transaction for persistent storage operations.
> Implemented in the mcoxfile library.
> Transactional Flash Translation Layer (TFTL) interfaces directly with low-level media drivers and implements real-time databases transactions by integrating transaction management with flash-specific operations enabling predictable access.
> Implemented in the mcoxflash library.
> File System Wrappers provide a unified interfice for file-level access across various file systems and block devices, standardizing operations like open, read, write, and close.
> Implemented in the target/sal/fsys directory.
> Low-level, OS-dependent flash media device drivers provide the direct interface to the underlying flash hardware and are specific to the target platform. Drivers may or may not be included in the eXtremeDB/rt distribution, depending on the supported hardware and integration requirements
> File system and external Flash Translation Layers (FTLs) that provide block-level access to raw flash devices while managing flash-specific tasks such as wear leveling and garbage collection.
> These are external modules
> FLASH MEDIA

> ## Mapped File (xfile)

> eXtremeDB/rt guarantees ACID compliance in persistent databases using a Copy-on-Write (CoW) mechanism. This approach ensures consistency and integrity during transactions while optimizing memory and disk management. Instead of modifying database pages directly, database indexes (e.g., B-Trees) reference logical offsets rather than physical addresses. These logical page numbers act as indices in a page map, which translates them into actual physical locations in storage.

> ## Database File Structure

> eXtremeDB/rt doesn’t rely on a file system. But for simplicity, we’ll use the term "database file" when talking about where the database stores its data. The only thing required here is a block device—a type of storage that reads and writes data in fixed-size blocks. You can access these blocks in any order, but you have to read or write the whole block at once. Examples include hard drives, SSDs, flash storage, and even eNVM devices.

> The database file is split into three parts:

> - Two page maps – One current (used for active transactions) and one shadow (stores the last consistent state).
> - One data segment – Stores actual database records and structures.

> Inside the data segment, information is stored in fixed-size pages, which make up a page pool. The database kernel can allocate pages from this pool as needed:

> - Some pages store actual database content (like objects and indexes).
> - Some are used for transaction management to track changes.

> The eXtremeDB/rt kernel doesn’t access data directly. Instead, it uses virtual addresses that map to physical locations. A virtual address consists of a virtual page number and an offset within the page. The page map is what translates virtual addresses into physical locations inside the database file.The index in the page map represents a logical page number and the value stored in that index tells you where the physical page is. The physical address of the data element then can be easily calculated as follows:

>         physical_address = (physical_page_number * page_size) + logical_address % page_size;

> When the database kernel is running, it keeps track of two sets of pages—one that represents the last saved (consistent) state and another that holds any changes made by the current transaction but not yet committed. These are managed using two-page maps.

> The first one, called the "current transaction map," tracks all pages that have been modified during the current transaction. The second, known as the "shadow map," holds the last stable version of the database. To keep things simple, we assume that only one transaction at a time is allowed to make changes, though multiple read-only transactions can still happen simultaneously. Under this setup, only a single "write" transaction works with the "current" map at any given time.

> At the start of a transaction, before anything changes, both maps are identical. As soon as a page is modified, a copy of that page is created, and its new location is recorded in the "current" page map. From that moment on, until the transaction is either committed or rolled back, any access to that page goes through the updated version in the current map.

> ## CoW-based transaction algorithm: first time page update

> ### Writing to logical page 2

> 1. The second element of the current map is equal to the second element of the shadow map. Both point to the "physical" page 6
> 2. Locate a free page in the file (physical page). It happens to be a "physical" page 3.
> 3. Copy the content of the second page there
> 4. Change the current page map second element content from 6 to 3

> ## CoW-based transaction algorithm: subsequent page update

> ### Writing to logical page 2

> 1. The second element of the current map is not equal to the second element of the shadow map. The point to the "physical" pages 3 and 6
> 2. The page has already been modified and the copy has already been created in the current thransactions
> 3. Modify the content of the physical page 6 again