net.metanotion.io.block

## Class BlockFile

- java.lang.Object
- - net.metanotion.io.block.BlockFile

- All Implemented Interfaces:
    Closeable, AutoCloseable

      public class BlockFile
      extends Object
      implements Closeable

  Block-based file storage system with indexed access.

  Provides paged file storage with skiplist indexes for efficient key-value data retrieval. Supports multiple named indexes within a single file.

  On-disk format:

          Magic number (6 bytes)
          Version major/minor (2 bytes)
          file length (long)
          free list start (unsigned int)
          is mounted (unsigned short) 0 = no, 1 = yes
          span size (unsigned short)
          block size (unsigned int)

  Metaindex skiplist is on page 2. Pages are 1 KB and are numbered starting from 1.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `RandomAccessInterface` | `file` The underlying file interface for this block file
    `Log` | `log` Logger instance for this BlockFile
    `static int` | `MAGIC_CONT` Magic number for continuation pages ("CONT")
    `static int` | `METAINDEX_PAGE` Page number for the metaindex
    `static long` | `OFFSET_MOUNTED` File offset where the mounted flag is stored
    `static int` | `PAGESIZE` Page size in bytes for block file pages
    `int` | `spanSize` Default span size for new skiplists

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `BlockFile(File f, boolean init)` Create a BlockFile from a File with optional initialization.
    `BlockFile(RandomAccessFile raf)` Create a BlockFile from a RandomAccessFile.
    `BlockFile(RandomAccessFile raf, boolean init)` Create a BlockFile from a RandomAccessFile with optional initialization.
    `BlockFile(RandomAccessInterface rai)` Use this constructor with a readonly RAI for a readonly blockfile.
    `BlockFile(RandomAccessInterface rai, boolean init)` Create a BlockFile from a RandomAccessInterface with optional initialization.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `int` | `allocPage()` Allocate a new page from the free list or extend the file.
    `boolean` | `bfck(boolean fix)` Run an integrity check on the blockfile and all the skiplists in it.
    `void` | `close()` Closes all open skiplists and then the blockfile itself.
    `void` | `closeIndex(String name)` Close a skiplist if it is open.
    `void` | `delIndex(String name)` Delete a skiplist if it exists.
    `void` | `freePage(int page)` Add the page to the free list.
    `<K extends Comparable<? super K>,V> BSkipList<K,V>` | `getIndex(String name, Serializer<K> key, Serializer<V> val)` Open a skiplist if it exists.
    `static void` | `main(String[] args)` Run an integrity check on the blockfile and all the skiplists in it.
    `<K extends Comparable<? super K>,V> BSkipList<K,V>` | `makeIndex(String name, Serializer<K> key, Serializer<V> val)` Create and open a new skiplist if it does not exist.
    `static void` | `pageSeek(RandomAccessInterface file, int page)` Go to any page but the superblock.
    `int` | `readMultiPageData(byte[] arr, int page, int[] curPageOff, int[] nextPage)` Read bytes from multiple pages.
    `<K extends Comparable<? super K>,V> void` | `reformatIndex(String name, Serializer<K> oldKey, Serializer<V> oldVal, Serializer<K> newKey, Serializer<V> newVal)` Reformat a skiplist with new Serializers if it exists.
    `int` | `skipMultiPageBytes(int length, int page, int[] curPageOff, int[] nextPage)` Skip length bytes across multiple pages.
    `boolean` | `wasMounted()` Check if the file was locked when opened.
    `int` | `writeMultiPageData(byte[] data, int page, int[] curPageOff, int[] nextPage)` Write bytes to multiple pages.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### file

          public final RandomAccessInterface file

      The underlying file interface for this block file

    - #### log

          public final Log log

      Logger instance for this BlockFile

    - #### MAGIC_CONT

          public static final int MAGIC_CONT

      Magic number for continuation pages ("CONT")
      See Also:
        Constant Field Values

    - #### METAINDEX_PAGE

          public static final int METAINDEX_PAGE

      Page number for the metaindex
      See Also:
        Constant Field Values

    - #### OFFSET_MOUNTED

          public static final long OFFSET_MOUNTED

      File offset where the mounted flag is stored
      See Also:
        Constant Field Values

    - #### PAGESIZE

          public static final int PAGESIZE

      Page size in bytes for block file pages
      See Also:
        Constant Field Values

    - #### spanSize

          public int spanSize

      Default span size for new skiplists

  - ### Constructor Detail

    - #### BlockFile

          public BlockFile(File f,
                           boolean init)
                    throws IOException

      Create a BlockFile from a File with optional initialization. File must be writable.
      Parameters:
        `f` - the File to use
        `init` - if true, initialize the file with a new block structure
      Throws:
        `IOException` - if an I/O error occurs

    - #### BlockFile

          public BlockFile(RandomAccessFile raf)
                    throws IOException

      Create a BlockFile from a RandomAccessFile. RAF must be writable.
      Parameters:
        `raf` - the RandomAccessFile to use
      Throws:
        `IOException` - if an I/O error occurs

    - #### BlockFile

          public BlockFile(RandomAccessFile raf,
                           boolean init)
                    throws IOException

      Create a BlockFile from a RandomAccessFile with optional initialization. RAF must be writable.
      Parameters:
        `raf` - the RandomAccessFile to use
        `init` - if true, initialize the file with a new block structure
      Throws:
        `IOException` - if an I/O error occurs

    - #### BlockFile

          public BlockFile(RandomAccessInterface rai)
                    throws IOException

      Use this constructor with a readonly RAI for a readonly blockfile.
      Parameters:
        `rai` - the RandomAccessInterface to use
      Throws:
        `IOException` - if an I/O error occurs

    - #### BlockFile

          public BlockFile(RandomAccessInterface rai,
                           boolean init)
                    throws IOException

      Create a BlockFile from a RandomAccessInterface with optional initialization. Use this constructor with a readonly RAI and init = false for a readonly blockfile.
      Parameters:
        `rai` - the RandomAccessInterface to use
        `init` - if true, initialize the file with a new block structure
      Throws:
        `IOException` - if an I/O error occurs

  - ### Method Detail

    - #### allocPage

          public int allocPage()
                        throws IOException

      Allocate a new page from the free list or extend the file.
      Returns:
        the page number of the newly allocated page
      Throws:
        `IOException` - if an I/O error occurs

    - #### bfck

          public boolean bfck(boolean fix)

      Run an integrity check on the blockfile and all the skiplists in it.
      Parameters:
        `fix` - if true, attempt to fix any corruption found
      Returns:
        true if the levels were modified.

    - #### close

          public void close()
                     throws IOException

      Closes all open skiplists and then the blockfile itself. Note (I2P) Does NOT close the RAF / RAI.
      Specified by:
        `close` in interface `Closeable`
      Specified by:
        `close` in interface `AutoCloseable`
      Throws:
        `IOException`

    - #### closeIndex

          public void closeIndex(String name)

      Close a skiplist if it is open.
      Parameters:
        `name` - the name of the skiplist index to close

    - #### delIndex

          public void delIndex(String name)
                        throws IOException

      Delete a skiplist if it exists. Must be open. Throws IOException if exists but is closed. Broken before 0.9.26.
      Parameters:
        `name` - the name of the skiplist index to delete
      Throws:
        `IOException` - if it is closed.

    - #### freePage

          public void freePage(int page)

      Add the page to the free list. The file is never shrunk. TODO: Reclaim free pages at end of file, or even do a full compaction. Does not throw exceptions; logs on failure.
      Parameters:
        `page` - the page number to free

    - #### getIndex

          public <K extends Comparable<? super K>,V> BSkipList<K,V> getIndex(String name,
                                                                             Serializer<K> key,
                                                                             Serializer<V> val)
                                                                      throws IOException

      Open a skiplist if it exists. Returns null if the skiplist does not exist. Empty skiplists are not preserved after close. If the file is writable, this runs an integrity check and repair on first open.
      Type Parameters:
        `K` - the type of keys in the skiplist
        `V` - the type of values in the skiplist
      Parameters:
        `name` - the name of the skiplist index
        `key` - the Serializer for keys
        `val` - the Serializer for values
      Returns:
        the BSkipList, or null if not found
      Throws:
        `IOException` - if an I/O error occurs

    - #### main

          public static void main(String[] args)

      Run an integrity check on the blockfile and all the skiplists in it. WARNING: This only works on skiplists using UTF8StringBytes as a key serializer, unless the exception has been coded in bfck below. Will CORRUPT other skiplists.
      Parameters:
        `args` - command line arguments (file path)

    - #### makeIndex

          public <K extends Comparable<? super K>,V> BSkipList<K,V> makeIndex(String name,
                                                                              Serializer<K> key,
                                                                              Serializer<V> val)
                                                                       throws IOException

      Create and open a new skiplist if it does not exist. Throws IOException if it already exists.
      Type Parameters:
        `K` - the type of keys in the skiplist
        `V` - the type of values in the skiplist
      Parameters:
        `name` - the name of the skiplist index
        `key` - the Serializer for keys
        `val` - the Serializer for values
      Returns:
        the newly created BSkipList
      Throws:
        `IOException` - if already exists or other errors

    - #### pageSeek

          public static void pageSeek(RandomAccessInterface file,
                                      int page)
                               throws IOException

      Go to any page but the superblock. Page 1 is the superblock, must use file.seek(0) to get there.
      Parameters:
        `file` - the RandomAccessInterface to seek in
        `page` - the page number (>= 2)
      Throws:
        `IOException` - if page is invalid

    - #### readMultiPageData

          public int readMultiPageData(byte[] arr,
                                       int page,
                                       int[] curPageOff,
                                       int[] nextPage)
                                throws IOException

      Read bytes from multiple pages.
      Parameters:
        `arr` - fill this array fully with data
        `page` - current page
        `curPageOff` - in (current) and out (new) parameter at index 0
        `nextPage` - in (current) and out (new) parameter at index 0
      Returns:
        current page
      Throws:
        `IOException` - if an I/O error occurs or not enough pages to read

    - #### reformatIndex

          public <K extends Comparable<? super K>,V> void reformatIndex(String name,
                                                                        Serializer<K> oldKey,
                                                                        Serializer<V> oldVal,
                                                                        Serializer<K> newKey,
                                                                        Serializer<V> newVal)
                                                                 throws IOException

      Reformat a skiplist with new Serializers if it exists. The skiplist must be closed. Throws IOException if the skiplist is open. The skiplist will remain closed after completion.
      Type Parameters:
        `K` - the type of keys in the skiplist
        `V` - the type of values in the skiplist
      Parameters:
        `name` - the name of the skiplist index
        `oldKey` - the current Serializer for keys
        `oldVal` - the current Serializer for values
        `newKey` - the new Serializer for keys
        `newVal` - the new Serializer for values
      Throws:
        `IOException` - if it is open or on errors

    - #### skipMultiPageBytes

          public int skipMultiPageBytes(int length,
                                        int page,
                                        int[] curPageOff,
                                        int[] nextPage)
                                 throws IOException

      Skip length bytes across multiple pages. The same as readMultiPageData() without returning a result
      Parameters:
        `length` - number of bytes to skip
        `page` - current page
        `curPageOff` - in (current) and out (new) parameter at index 0
        `nextPage` - in (current) and out (new) parameter at index 0
      Returns:
        current page
      Throws:
        `IOException` - if an I/O error occurs or not enough pages to skip

    - #### wasMounted

          public boolean wasMounted()

      Check if the file was locked when opened.
      Returns:
        true if the file was mounted

    - #### writeMultiPageData

          public int writeMultiPageData(byte[] data,
                                        int page,
                                        int[] curPageOff,
                                        int[] nextPage)
                                 throws IOException

      Write bytes to multiple pages. This will allocate additional continuation pages as necessary.
      Parameters:
        `data` - data to write
        `page` - current page
        `curPageOff` - in (current) and out (new) parameter at index 0
        `nextPage` - in (current) and out (new) parameter at index 0
      Returns:
        current page
      Throws:
        `IOException` - if an I/O error occurs