Skip to content

[Impeller] Support rendering compressed texture formats / add capability to detect support. #148443

Description

@jonahwilliams

I think that we could fairly easily add support for compressed image formats to Impeller, at least on the Vulkan and Metal backends.

Vulkan: https://docs.vulkan.org/spec/latest/appendices/compressedtex.html
iOS: https://developer.apple.com/documentation/metal/mtlpixelformat/mtlpixelformatastc_6x6_hdr?language=objc

I'm not sure what the actual level of device support is on Vulkan and would need to do some research. For iOS, there is a handy feature set table here: https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf . We could look at the intersection of minimum supported formats as a starting point.

These formats specify how pixel data is compressed, but don't include header information on the format/size like commonly used network image formats like PNG/JPEG. This metadata would need to be provided by the application. I'm not sure if there are other standards here like ASTC in PNG or if the expectation is that the application level logic will handle transport compression/bundling.

For example, a low-level dart:ui API for decompressing.

import 'dart:ui' as uil
[
void main() {
  Uint8List data = getBytes(...);
  if (!ui.Codec.checkFormatSupport(PixelFormatASTC_4x4_sRGB)) { //Probably a different name
    return;
  }
  var ui.image = await ui.decodeCompressedTexture(data, width: 100, height: 100, format: VK_FORMAT_BC1_RGB_UNORM_BLOCK);
 

Then some higher level Image provider API:

Image.network(
  ...
  compressedFormat: CompressedFormat(PixelFormatASTC_4x4_sRGB, width: 100, height: 100)
 )

For testing/evaluating, there is an open source ASTC compressor developed by ARM available at https://github.com/ARM-software/astc-encoder.

Some caveats:

  • We can't support these in the Skia backend as the formats are unsupported as far as I know, unless we wrote separate import code and treated them as external textures (which I'd rather not do).
  • We currently resize images using Skia's software codecs which don't support these formats, so resizing would fail until we switch to GPU based resizing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issues that are less important to the Flutter projectc: new featureNothing broken; request for a new capabilitye: impellerImpeller rendering backend issues and features requeststeam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions