Skip to content

Sewer56/dxt-lossless-transform

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dxt-lossless-transform

Crates.io Docs.rs CI

[Note: Currently in development. We're far from done. BC1-BC3 mostly done, BC7 barely started]

About

This crates provide fast lossless transforms for DDS files that improve their compression ratio once compressed by external compressors. These transforms reduce the size of the files, once compressed, and improve load speed, by making the data faster to decompress.

Project Layout

Tip

The project is organized into several main directories under /src/:

Project Structure

  • /src/core/ - Houses the unstable API implementations (low-level crates without API stability guarantees)
  • /src/api/ - Houses the stable API (recommended for external users)
  • /src/extensions/ - Contains extensions that build on top of the stable API
  • /src/tools/ - Contains miscellaneous programs and utilities

Click on a project to navigate to its description.

Stable API (/src/api/)

Tip

These provide stable, backwards-compatible interfaces. Recommended for external users.

Extensions (/src/extensions/)

File Format Support

Estimator Libraries

Tip

These libraries help you determine the best transform settings for each algorithm.

Speeds are based on a single core on a Ryzen 9950X3D CPU, and are approximate values.

  • ZStandard (zstd): Uses ZStandard compression to determine best transform settings.

    • Package: dxt-lossless-transform-zstd
    • Speed (1 Thread): ~1060MiB/s (level 1)
    • Accuracy: 79.2% (level 1) (vs ZStandard)
    • Recommended for zstd compression level 4 and above.
    • Level 1 is recommended for a 'balanced' profile; other levels deliver increasingly diminishing returns.
  • Lossless Transform Utils (ltu): Fast estimator for generic LZ-based compression algorithms.

    • Package: dxt-lossless-transform-ltu
    • Speed: ~2565MiB/s single-threaded.
    • Accuracy: ~74.4% for high compression (vs ZStandard)
    • Recommended for zstd compression levels 1-3.
    • Optimized for speed while maintaining reasonable accuracy; suited for real-time compression scenarios.

For more information, refer to my blog post.

Core Implementations (/src/core/) - Unstable API

Warning

These crates have the concrete implementations but no stable API. Use the stable API crates instead.

They are low level crates optimized for maximum performance with frequent breaking changes.

Tools (/src/tools/)

  • CLI tools and debugging utilities
  • Development and testing applications
  • Example implementations and benchmarks

Fuzzing

This project contains a fuzzing crate that can be used to test some of the code against other implementations.

# You may need to install nightly compiler first:
# https://rust-fuzz.github.io/book/cargo-fuzz/setup.html
# Install cargo-fuzz
cargo install cargo-fuzz

# Run the fuzz target
CARGO_PROFILE_RELEASE_LTO=false cargo fuzz run bc1_decode

To find the targets, look at the fuzz folder, Cargo.toml.

Usage

This crate is not yet released.

Testing (CLI)

This project contains a CLI tool that can be used to test the transforms.

./dxt-lossless-transform-cli transform --input textures --output textures-transformed

You can either:

  1. Build and run the CLI:

    cargo build --release --features "debug-bc1"
    ./target/release/dxt-lossless-transform-cli transform --input textures --output textures-transformed
  2. Run directly via Cargo:

    cargo run --release --bin dxt-lossless-transform-cli --features "debug-bc1" transform --input textures --output textures-transformed

Developer Manual

For step-by-step development guidance, see the Developer Manual.

Contributing

We welcome contributions! See the Contributing Guide for details.

License

Licensed under GPL v3 (with Reloaded FAQ).

About

[WIP] Fast lossless transforms for DDS files that improve their compression ratio.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 3

  •  
  •  
  •  

Languages