org.klomp.snark

## Class Storage

- java.lang.Object
- - org.klomp.snark.Storage

- All Implemented Interfaces:
    Closeable, AutoCloseable

      public class Storage
      extends Object
      implements Closeable

  Storage manages torrent data files on disk for I2PSnark.

  This class handles all aspects of storing and retrieving torrent pieces:

  - Creating and managing files/directories for torrent data
  - Reading and writing piece data to disk
  - Tracking downloaded pieces via BitField
  - Validating downloaded pieces against piece hashes
  - File pre-allocation for sparse file support
  - Priority-based file selection for partial downloads
  - File checking and rechecking for data integrity

  Storage implements Closeable and should be properly closed after use. The storage must be checked via `check()` before reading or writing pieces.

  Thread safety: This class uses synchronized methods and concurrent collections for thread-safe access to file handles and metadata.

  See Also:
    `MetaInfo`, `BitField`, `StorageListener`

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static boolean` | `DEFAULT_PREALLOCATE_FILES` Default value for file pre-allocation (true).
    `static int` | `MAX_PIECE_SIZE` Maximum permitted piece size.
    `static int` | `MAX_PIECES` The maximum number of pieces allowed in a single torrent.
    `static long` | `MAX_TOTAL_SIZE` Maximum total torrent size (MAX_PIECE_SIZE * MAX_PIECES).
    `static int` | `PRIORITY_NORMAL` Default priority value for files.
    `static int` | `PRIORITY_SKIP` Priority value indicating a file should be skipped during download.
    `static String` | `PROP_PREALLOCATE_FILES` Configuration key for enabling file pre-allocation.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Storage(I2PSnarkUtil util, File baseFile, MetaInfo metainfo, StorageListener listener, boolean preserveFileNames)` Creates a new storage based on the supplied MetaInfo.
    `Storage(I2PSnarkUtil util, File baseFile, String announce, List<List<String>> announce_list, String created_by, boolean privateTorrent, List<String> url_list, String comment, StorageListener listener, List<TorrentCreateFilter> filters)` Creates a storage from the existing file or directory.
    `Storage(I2PSnarkUtil util, File baseFile, String announce, List<List<String>> announce_list, String created_by, boolean privateTorrent, StorageListener listener, List<TorrentCreateFilter> filters)` Creates a storage from the existing file or directory.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `check()` Creates (and/or checks) all files from the metainfo file list.
    `void` | `check(long savedTime, BitField savedBitField)` Creates (and/or checks) all files from the metainfo file list.
    `void` | `cleanRAFs()` Close unused RAFs - call periodically
    `(package private) void` | `clearChanged()` Clear the storage changed variable
    `void` | `close()` Closes the Storage and makes sure that all RandomAccessFiles are closed.
    `boolean` | `complete()` Whether or not this storage contains all pieces if the MetaInfo.
    `static String` | `filterName(String name)` Removes 'suspicious' characters from the given file name.
    `long` | `getActivity()` Get the last activity timestamp.
    `File` | `getBase()` The base file or directory.
    `String` | `getBaseName()` The base file or directory name of the data, as specified in the .torrent file, but filtered to remove illegal characters.
    `BitField` | `getBitField()` The BitField that tells which pieces this storage contains.
    `double` | `getCheckingProgress()` If checking is in progress, return completion 0.0 ...
    `SortedSet<File>` | `getDirectories()` Includes the base for a multi-file torrent.
    `List<String>` | `getExcludedFiles(File base)`
    `int` | `getFileCount()` Does not include directories.
    `int[]` | `getFilePriorities()` Get the file priorities array.
    `List<File>` | `getFiles()` Does not include directories.
    `boolean` | `getInOrder()`
    `MetaInfo` | `getMetaInfo()` Returns the MetaInfo associated with this Storage.
    `ByteArray` | `getPiece(int piece, int off, int len)` Returns a byte array containing a portion of the requested piece or null if the storage doesn't contain the piece yet.
    `int[]` | `getPiecePriorities()` Call setPriority() for all changed files first, then call this.
    `boolean` | `getPreserveFileNames()`
    `int` | `getPriority(int fileIndex)`
    `long` | `getSkippedLength()` Call setPriority() for all changed files first, then call this.
    `int` | `indexOf(File file)` Get index to pass to remaining(), getPriority(), setPriority()
    `boolean` | `isAllocating()` Disk allocation (ballooning) in progress.
    `boolean` | `isChanged()` Has the storage changed since instantiation?
    `boolean` | `isChecking()` File checking in progress.
    `static void` | `main(String[] args)` Create a metainfo.
    `int` | `needed()` How many pieces are still missing from this storage.
    `boolean` | `putPiece(PartialPiece pp)` Put the piece in the Storage if it is correct.
    `boolean` | `recheck()` Blocking.
    `long[]` | `remaining()` For efficiency, calculate remaining bytes for all files at once
    `long[][]` | `remaining2()` For efficiency, calculate remaining bytes for all files at once.
    `void` | `reopen()` Doesn't really reopen the file descriptors for a restart.
    `void` | `setActivity(long time)` Set the activity timestamp.
    `(package private) void` | `setFilePriorities(int[] p)` Set the file priorities array.
    `void` | `setInOrder(boolean yes)` Enable or disable in-order download mode.
    `void` | `setPriority(int fileIndex, int pri)` Must call Snark.updatePiecePriorities() (which calls getPiecePriorities()) after calling this.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_PREALLOCATE_FILES

          public static final boolean DEFAULT_PREALLOCATE_FILES

      Default value for file pre-allocation (true).
      See Also:
        Constant Field Values

    - #### MAX_PIECE_SIZE

          public static final int MAX_PIECE_SIZE

      Maximum permitted piece size. Torrents with pieces larger than this will be rejected.
      See Also:
        Constant Field Values

    - #### MAX_PIECES

          public static final int MAX_PIECES

      The maximum number of pieces allowed in a single torrent.
      See Also:
        Constant Field Values

    - #### MAX_TOTAL_SIZE

          public static final long MAX_TOTAL_SIZE

      Maximum total torrent size (MAX_PIECE_SIZE * MAX_PIECES).
      See Also:
        Constant Field Values

    - #### PRIORITY_NORMAL

          public static final int PRIORITY_NORMAL

      Default priority value for files.
      See Also:
        Constant Field Values

    - #### PRIORITY_SKIP

          public static final int PRIORITY_SKIP

      Priority value indicating a file should be skipped during download.
      See Also:
        Constant Field Values

    - #### PROP_PREALLOCATE_FILES

          public static final String PROP_PREALLOCATE_FILES

      Configuration key for enabling file pre-allocation.
      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### Storage

          public Storage(I2PSnarkUtil util,
                         File baseFile,
                         MetaInfo metainfo,
                         StorageListener listener,
                         boolean preserveFileNames)

      Creates a new storage based on the supplied MetaInfo.

      Does not check storage. Caller MUST call check(), which will try to create and/or check all needed files in the MetaInfo.

      Parameters:
        `baseFile` - the torrent data file or dir
        `preserveFileNames` - if true, do not remap names to a 'safe' charset

    - #### Storage

          public Storage(I2PSnarkUtil util,
                         File baseFile,
                         String announce,
                         List<List<String>> announce_list,
                         String created_by,
                         boolean privateTorrent,
                         List<String> url_list,
                         String comment,
                         StorageListener listener,
                         List<TorrentCreateFilter> filters)
                  throws IOException

      Creates a storage from the existing file or directory. Creates an in-memory metainfo but does not save it to a file, caller must do that.

      Creates the metainfo, this may take a LONG time. BLOCKING.

      Parameters:
        `announce` - may be null
        `listener` - may be null
        `created_by` - may be null
        `url_list` - may be null
        `comment` - may be null
      Throws:
        `IOException` - when creating and/or checking files fails.

    - #### Storage

          public Storage(I2PSnarkUtil util,
                         File baseFile,
                         String announce,
                         List<List<String>> announce_list,
                         String created_by,
                         boolean privateTorrent,
                         StorageListener listener,
                         List<TorrentCreateFilter> filters)
                  throws IOException

      Creates a storage from the existing file or directory. Creates an in-memory metainfo but does not save it to a file, caller must do that.

      Creates the metainfo, this may take a LONG time. BLOCKING.

      Parameters:
        `announce` - may be null
        `listener` - may be null
        `created_by` - may be null
      Throws:
        `IOException` - when creating and/or checking files fails.

  - ### Method Detail

    - #### check

          public void check()
                     throws IOException

      Creates (and/or checks) all files from the metainfo file list. Only call this once, and only after the constructor with the metainfo. Use recheck() to check again later.
      Throws:
        `IllegalStateException` - if called more than once
        `IOException`

    - #### check

          public void check(long savedTime,
                            BitField savedBitField)
                     throws IOException

      Creates (and/or checks) all files from the metainfo file list. Use a saved bitfield and timestamp from a config file. Only call this once, and only after the constructor with the metainfo. Use recheck() to check again later.
      Throws:
        `IllegalStateException` - if called more than once
        `IOException`

    - #### cleanRAFs

          public void cleanRAFs()

      Close unused RAFs - call periodically

    - #### clearChanged

          void clearChanged()

      Clear the storage changed variable

    - #### close

          public void close()
                     throws IOException

      Closes the Storage and makes sure that all RandomAccessFiles are closed. The Storage is unusable after this.
      Specified by:
        `close` in interface `Closeable`
      Specified by:
        `close` in interface `AutoCloseable`
      Throws:
        `IOException`

    - #### complete

          public boolean complete()

      Whether or not this storage contains all pieces if the MetaInfo.

    - #### filterName

          public static String filterName(String name)

      Removes 'suspicious' characters from the given file name. http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx Then replace chars not supported in the charset.

      This is called frequently and it can be pretty slow so cache the result.

      TODO: If multiple files in the same torrent map to the same filter name, the whole torrent will blow up. Check at torrent creation?

    - #### getActivity

          public long getActivity()

      Get the last activity timestamp.
      Returns:
        timestamp in milliseconds since epoch, or 0 if never set

    - #### getBase

          public File getBase()

      The base file or directory.
      Returns:
        the File

    - #### getBaseName

          public String getBaseName()

      The base file or directory name of the data, as specified in the .torrent file, but filtered to remove illegal characters. This is where the data actually is, relative to the snark base dir.

    - #### getBitField

          public BitField getBitField()

      The BitField that tells which pieces this storage contains. Do not change this since this is the current state of the storage.

    - #### getCheckingProgress

          public double getCheckingProgress()

      If checking is in progress, return completion 0.0 ... 1.0, else return 1.0.

    - #### getDirectories

          public SortedSet<File> getDirectories()

      Includes the base for a multi-file torrent. Sorted bottom-up for easy deletion. Slow. Use for deletion only.
      Returns:
        a new Set or null for a single-file torrent

    - #### getExcludedFiles

          public List<String> getExcludedFiles(File base)

    - #### getFileCount

          public int getFileCount()

      Does not include directories.

    - #### getFilePriorities

          public int[] getFilePriorities()

      Get the file priorities array.
      Returns:
        null on error, if complete, or if only one file

    - #### getFiles

          public List<File> getFiles()

      Does not include directories. Unsorted.
      Returns:
        a new List

    - #### getInOrder

          public boolean getInOrder()

      Returns:
        whether in-order download mode is enabled

    - #### getMetaInfo

          public MetaInfo getMetaInfo()

      Returns the MetaInfo associated with this Storage.

    - #### getPiece

          public ByteArray getPiece(int piece,
                                    int off,
                                    int len)
                             throws IOException

      Returns a byte array containing a portion of the requested piece or null if the storage doesn't contain the piece yet.
      Throws:
        `IOException`

    - #### getPiecePriorities

          public int[] getPiecePriorities()

      Call setPriority() for all changed files first, then call this. Set the piece priority to the highest priority of all files spanning the piece. Caller must pass array to the PeerCoordinator.
      Returns:
        null on error, if complete, or if only one file and inOrder not set.

    - #### getPreserveFileNames

          public boolean getPreserveFileNames()

      Returns:
        true if original file names are being preserved, false if filtered

    - #### getPriority

          public int getPriority(int fileIndex)

      Parameters:
        `fileIndex` - as obtained from indexOf

    - #### getSkippedLength

          public long getSkippedLength()

      Call setPriority() for all changed files first, then call this. The length of all the pieces that are not yet downloaded, and are set to skipped. This is not the same as the total of all skipped files, since pieces may span multiple files.
      Returns:
        0 on error, if complete, or if only one file

    - #### indexOf

          public int indexOf(File file)

      Get index to pass to remaining(), getPriority(), setPriority()
      Parameters:
        `file` - non-canonical path (non-directory)
      Returns:
        internal index of file; -1 if unknown file

    - #### isAllocating

          public boolean isAllocating()

      Disk allocation (ballooning) in progress. Always false on Windows.

    - #### isChanged

          public boolean isChanged()

      Has the storage changed since instantiation?

    - #### isChecking

          public boolean isChecking()

      File checking in progress.

    - #### main

          public static void main(String[] args)

      Create a metainfo. Used in the installer build process; do not comment out.

    - #### needed

          public int needed()

      How many pieces are still missing from this storage.

    - #### putPiece

          public boolean putPiece(PartialPiece pp)
                           throws IOException

      Put the piece in the Storage if it is correct. Warning - takes a LONG time if complete as it does the recheck here. TODO thread the recheck?
      Returns:
        true if the piece was correct (sha metainfo hash matches), otherwise false.
      Throws:
        `IOException` - when some storage related error occurs.

    - #### recheck

          public boolean recheck()
                          throws IOException

      Blocking. Holds lock. Recommend running only when stopped. Caller should thread. Calls listener.setWantedPieces() on completion if anything changed.
      Returns:
        true if anything changed, false otherwise
      Throws:
        `IOException`

    - #### remaining

          public long[] remaining()

      For efficiency, calculate remaining bytes for all files at once
      Returns:
        number of bytes remaining for each file, use indexOf() to get index for a file

    - #### remaining2

          public long[][] remaining2()

      For efficiency, calculate remaining bytes for all files at once. Remaining bytes is rv[0]. Preview bytes is rv[1].
      Returns:
        number of bytes remaining and number of bytes available for a preview for each file, use indexOf() to get index for a file

    - #### reopen

          public void reopen()
                      throws IOException

      Doesn't really reopen the file descriptors for a restart. Just does an existence check but no length check or data reverification
      Throws:
        `IOException` - on fail

    - #### setActivity

          public void setActivity(long time)

      Set the activity timestamp.
      Parameters:
        `time` - timestamp in milliseconds since epoch

    - #### setFilePriorities

          void setFilePriorities(int[] p)

      Set the file priorities array. Only call this when stopped, but after check()
      Parameters:
        `p` - may be null

    - #### setInOrder

          public void setInOrder(boolean yes)

      Enable or disable in-order download mode. When enabled, pieces within each file are prioritized sequentially. Must call setFilePriorities() BEFORE this method.
      Parameters:
        `yes` - true to enable in-order mode, false to disable

    - #### setPriority

          public void setPriority(int fileIndex,
                                  int pri)

      Must call Snark.updatePiecePriorities() (which calls getPiecePriorities()) after calling this.
      Parameters:
        `fileIndex` - as obtained from indexOf
        `pri` - default 0; <0 to disable