# Modular implementation

> In embedded systems there is a vast variety of storage and flash devices—ranging from NOR/NAND raw chips to managed eMMC, SD cards, and RTOS-provided filesystems. Each device type comes with its own constraints: page sizes, erase-before-write rules, wear characteristics, latency profiles, and error correction schemes.

> eXtremeDB/rt addresses this diversity with a persistent storage stack. Persistence is implemented in the mcovtdsk module (`target/mcovtdsk/`), which provides a uniform abstraction layer and communicates with storage back-ends via `struct mco_file_t` (see `include/mco.h`).

> By isolating device-specific details behind `mco_file_t` and `mco_xflash_media_t`, the database core remains unchanged regardless of the storage technology underneath. This design:

> - Supports diverse hardware without modifications to the kernel.
> - Simplifies integration with new storage technologies.
> - Provides flexibility to migrate between managed and raw flash depending on cost, performance, and reliability.

> This modularity ensures the same database kernel can operate consistently across a wide spectrum of embedded hardware configurations.

> ### Managed Flash

> For managed flash devices, the underlying module is mcoxfile (target/mcovtdsk/mcoxfile/). The important point is that mcoxfile supports access through either a filesystem or a block device. To achieve this, it relies on filesystem wrappers located in `target/sal/fsys/mcof*`. Regardless of whether a file system or a block device is used, the interface between mcovtdsk and mcoxfile is always mco_file_t.

> ### Raw (Unmanaged) Flash

> Raw flash media is handled by the mcoxflash module (/target/mcovtdsk/mcoxflash/). This module implements a software Flash Translation Layer (FTL) and serves as a bridge between mcovtdsk and a flash-specific interface that includes:

> - erase-before-write operations
> - bad block management
> - wear leveling
> - low-level page/block addressing

> In this case, mcoxflash communicates with the underlying unmanaged flash driver (or a flash simulator via `struct mco_xflash_media_t` (also defined in `include/mco.h`). Detailed descriptions of the fields and callbacks in this structure are included in the source comments.

> Using a flash simulator in early stages for development and testing is often the best practice in embedded development, as it allows teams to test persistence behavior, validate wear-leveling and bad block handling logic before committing to hardware runs.

> ### Modular Implementation Overview

> - mcovtdsk always interacts with storage through mco_file_t, ensuring a consistent interface.
> - mcoxfile adapts mco_file_t to managed flash, supporting file system or block device access.
> - mcoxflash adapts mco_file_t to raw flash, using mco_xflash_media_t to drive erase/ program/read primitives.

> This separation of persistence logic from media-specific details is the key to portability and long-term maintainability in embedded systems.

> ## Key API Definitions

> ### mco_file_t

> `is_reentrant`
> Non-zero if the file supports concurrent access by multiple threads (i.e., `pread`/`pwrite` are thread-safe).
> `is_transactional`
> Indicates whether the file implements transactional semantics (e.g., supports commit/rollback).
> `pwrite()`
> Writes data to a specified offset in the file. Must be reentrant if `is_reentrant` is true. Returns number of bytes written or -1 on error.
> `pread`
> Reads data from a specified offset in the file. Must be reentrant if `is_reentrant` is true. Returns number of bytes read or -1 on error.
> `write()`
> Writes data to the current file position. Returns number of bytes written or -1 on error.
> `seek()`
> Sets the current file position for subsequent `read`/`write` operations. Returns 0 on success, -1 on error.
> `truncate()`
> Truncates the file to the specified size. Returns 0 on success, -1 on error.
> `read()`
> Reads data from the current file position. Returns number of bytes read or -1 on error.
> `rawread()`
> Reads raw (unprocessed) data from a specified offset—bypassing decryption and decompression. Used only when `MCO_CFG_DISK_WARMUP` is enabled. Must be reentrant if `is_reentrant` is true. Returns number of bytes read or -1 on error.
> `flush()`
> Flushes OS file buffers to persistent storage. Returns 0 on success, -1 on error.
> `size()`
> Retrieves the current file size. Returns 0 on success, -1 on error.
> `close()`
> Closes the file. Returns 0 on success, -1 on error.
> `advise()`
> Provides a hint to the file system about future access patterns (e.g., whether a region will be needed soon). Returns 0 on success, -1 on error.
> `punch_hole()`
> Deallocates (deallocates or “punches a hole” in) a region of the file starting at `pos` for `size` bytes. Returns 0 on success, -1 on error.
> `compact()`
> Performs file compaction (shrinking) — supported only for transactional files when `MCO_DB_INMEM_PAGE_MAP` is enabled. Compaction occurs at transaction commit. Returns 0 on success, -1 on error.
> `commit()`
> Commits pending changes (transactional file systems only). Returns 0 on success, -1 on error.
> `rollback()`
> Rolls back pending changes (transactional file systems only). Returns 0 on success, -1 on error.

> ### mcoxflash (Raw Flash with software FTL),

> #### struct mco_xflash_media

> `geometry`
> Flash device geometry (page size, pages per block, total blocks).
> `is_block_bad()`
> Checks whether the specified erase block is marked as bad. Returns non-zero if bad.
> `flag_block_bad()`
> Marks the specified erase block as bad (e.g., due to a write/erase failure).
> `block_erase()`
> Erases the specified block. Returns `MCO_S_OK` on success or `MCO_E_DISK_BAD_BLOCK` if the block is faulty.
> `page_prog()`
> Programs (writes) data to the specified page. Returns `MCO_S_OK` on success or `MCO_E_DISK_BAD_BLOCK` on failure.
> `is_page_erased()`
> Checks whether the specified page has been erased (i.e., contains only 0xFF bytes). Returns non-zero if erased.
> `page_read()`
> Reads a portion of data from the specified page at a given offset. Returns `MCO_S_OK` on success or `MCO_E_DISK_CRC_MISMATCH` if an ECC error is detected.
> `page_copy()`
> Copies the contents of one page to another (used during wear leveling or garbage collection). Returns `MCO_S_OK` on success or an error code on failure.
> `media_sync()`
> Flushes any pending data to persistent storage (e.g., forces write-back cache to commit). Returns `MCO_S_OK` on success.
> `media_close()`
> Closes the flash media device and releases associated resources.

> #### struct mco_xflash_geometry_t

> `page_size`
> Page size in bytes.
> `pages_per_block`
> Number of pages within a single erase block.
> `n_blocks`
> Total number of erase blocks in the flash device.

> Please refer to the eXtremeDB/rt headers (the definitions of `mco_file_t` and `mco_xflash_media_t` in `include/mco.h`) for verbose declarations.

> The xflash approach is illustrated with the flash-crud sample and the chapter "Using Flash Simulator"