The DICOM collector API in DicomObjectCollector can be used for reading DICOM objects in contiguous chunks, thus being able to collect meta-data earlier and reducing memory footprint by not having to keep the full data set in memory.

Summary

• [parser] add LazyDataSetReader::peek and LazyDataSetReader::new_with_ts
• [parser] change Debug impl for LazyDataToken so that it works for any data source
• [object] add collector module and DicomCollector abstraction
  • allows users to fetch whole DICOM data elements in chunks, and also enables reading pixel data fragments one by one
  • pub use DicomCollector and DicomCollectorOptions at crate root
• Adjust documentation a bit

Known caveats

• While each pixel data fragment can be retrieved independently, the API does not provide a way to read portions of said fragments, which could be a problem in memory-tight platforms working with large fragments.
  • There's always the theoretical limit of 2^32-2 bytes (~ 4GiB), though other DICOM tools out there cannot even handle more than 2^31-1 bytes (~ 2GiB).
• It cannot read multiple chunks from the same nested data set (sequence). I suspect this should only become a problem in very, very large structured reports.
• read_next_fragment allows consumers to retrieve the basic offset table as plain bytes, despite being the fragment reserved for the basic offset table. Users who are interested in the offset table should call read_basic_offset_table before calling read_next_fragment, because it is not saved by the collector.
  • If this behavior becomes confusing to users, it might need to be reconsidered in some way.
• The plan is to use this API on DICOM object implementations, but this work hasn't been done yet.