Skip to content

FragmentShader setImageSampler doesn't support FilterQuality #133944

@pshc

Description

@pshc

Is there an existing issue for this?

Steps to reproduce

  1. Generate a ui.Image in code using Uint8List, ImageDescriptor.raw, codec, etc.
  2. Create a FragmentShader that accepts a uniform sampler2D
  3. Pass the image into the shader using setImageSampler
  4. Render the shader, reading directly from the sampler

Expected results

I would expect to have a choice of FilterQuality for the texture, but I can't find any such interface. Scaled-up texture should have a smooth, interpolated appearance.

Actual results

Image is pixelated (nearest neighbor)

Code sample

Code sample
import 'dart:typed_data';
import 'dart:ui' as ui;
import 'package:flutter/material.dart';

void main() async {
  final program = await ui.FragmentProgram.fromAsset('assets/texture.frag');
  final image = await makeImage();
  runApp(MyApp(image, program));
}

class MyApp extends StatelessWidget {
  final ui.FragmentProgram program;
  final ui.Image image;
  const MyApp(this.image, this.program, {super.key});

  @override
  Widget build(BuildContext context) {
    const title = 'Texture Demo';
    return MaterialApp(
      title: title,
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: Scaffold(
        appBar: AppBar(
          backgroundColor: Theme.of(context).colorScheme.inversePrimary,
          title: const Text(title),
        ),
        body: CustomPaint(
          size: Size.infinite,
          painter: TexturePainter(image, program.fragmentShader()),
        ),
      ),
    );
  }
}

class TexturePainter extends CustomPainter {
  final ui.Image image;
  final ui.FragmentShader shader;
  TexturePainter(this.image, this.shader);

  @override
  void paint(Canvas canvas, Size size) {
    debugPrint('picture of size $size');
    shader.setFloat(0, size.width);
    shader.setFloat(1, size.height);
    shader.setImageSampler(0, image);
    canvas.drawRect(Offset.zero & size, Paint()..shader = shader);
  }

  @override
  bool shouldRepaint(TexturePainter oldDelegate) =>
      shader != oldDelegate.shader;
}

Future<ui.Image> makeImage() async {
  const width = 64;
  const height = 64;
  const bytesPerPixel = 4;
  final buffer = Uint8List(width * height * bytesPerPixel);
  for (int y = 0; y < height; y++) {
    for (int x = 0; x < height; x++) {
      final offset = ((y * width) + x) * bytesPerPixel;
      final parity = (x % 2) ^ (y % 2) != 0;
      buffer[offset + 1] = parity ? 255 : 0;
      buffer[offset + 2] = parity ? 200 : 0;
      buffer[offset + 3] = 255;
    }
  }
  final immutable = await ui.ImmutableBuffer.fromUint8List(buffer);
  final descriptor = ui.ImageDescriptor.raw(
    immutable,
    width: width,
    height: height,
    pixelFormat: ui.PixelFormat.rgba8888,
  );
  final codec = await descriptor.instantiateCodec(
    targetWidth: width,
    targetHeight: height,
  );
  final frameInfo = await codec.getNextFrame();
  return frameInfo.image;
}

pubspec.yaml:

  shaders:
    - assets/texture.frag

texture.frag:

#version 460 core
#include <flutter/runtime_effect.glsl>

uniform vec2 uSize;
uniform sampler2D uSampler;
out vec4 fragColor;

void main() {
    vec2 uv = FlutterFragCoord() / uSize;
    fragColor = texture(uSampler, uv);
}

Screenshots or Video

Screenshots / Video demonstration Screenshot 2023-09-03 at 11 11 22 AM

Logs

Logs
% flutter run --verbose -d macos
[   +1 ms] executing: [/Users/pc/fvm/versions/3.10.2/] git -c log.showSignature=false log -n 1 --pretty=format:%H
[  +19 ms] Exit code 0 from: git -c log.showSignature=false log -n 1 --pretty=format:%H
[        ] 9cd3d0d9ff05768afa249e036acc66e8abe93bff
[        ] executing: [/Users/pc/fvm/versions/3.10.2/] git tag --points-at 9cd3d0d9ff05768afa249e036acc66e8abe93bff
[  +89 ms] Exit code 0 from: git tag --points-at 9cd3d0d9ff05768afa249e036acc66e8abe93bff
[        ] 3.10.2
[   +1 ms] executing: [/Users/pc/fvm/versions/3.10.2/] git rev-parse --abbrev-ref HEAD
[   +8 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] stable
[        ] executing: sw_vers -productName
[  +10 ms] Exit code 0 from: sw_vers -productName
[        ] macOS
[        ] executing: sw_vers -productVersion
[   +8 ms] Exit code 0 from: sw_vers -productVersion
[        ] 13.5.1
[        ] executing: sw_vers -buildVersion
[  +11 ms] Exit code 0 from: sw_vers -buildVersion
[        ] 22G90
[        ] executing: uname -m
[   +5 ms] Exit code 0 from: uname -m
[        ] arm64
[   +7 ms] executing: sysctl hw.optional.arm64
[   +6 ms] Exit code 0 from: sysctl hw.optional.arm64
[        ] hw.optional.arm64: 1
[  +11 ms] Unable to locate an Android SDK.
[   +1 ms] executing: [/Users/pc/fvm/versions/3.10.2/] git rev-parse --abbrev-ref --symbolic @{upstream}
[   +8 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{upstream}
[        ] origin/stable
[        ] executing: [/Users/pc/fvm/versions/3.10.2/] git ls-remote --get-url origin
[   +7 ms] Exit code 0 from: git ls-remote --get-url origin
[        ] https://github.com/flutter/flutter.git
[  +41 ms] executing: sysctl hw.optional.arm64
[   +6 ms] Exit code 0 from: sysctl hw.optional.arm64
[        ] hw.optional.arm64: 1
[        ] executing: /usr/bin/arch -arm64e xcrun xcodebuild -version
[ +185 ms] Exit code 0 from: /usr/bin/arch -arm64e xcrun xcodebuild -version
[        ] Xcode 14.3.1
           Build version 14E300c
[   +1 ms] executing: /usr/bin/arch -arm64e xcrun xcdevice list --timeout 5
...

Flutter Doctor output

Doctor output
[✓] Flutter (Channel stable, 3.10.2, on macOS 13.5.1 22G90 darwin-arm64, locale en-CA)
    • Flutter version 3.10.2 on channel stable at /Users/pc/fvm/versions/3.10.2
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 9cd3d0d9ff (3 months ago), 2023-05-23 20:57:28 -0700
    • Engine revision 90fa3ae28f
    • Dart version 3.0.2
    • DevTools version 2.23.1

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, please use
      `flutter config --android-sdk` to update to that location.


[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.12.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).

[✓] VS Code (version 1.81.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.70.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 13.5.1 22G90 darwin-arm64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 116.0.5845.140

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

Metadata

Metadata

Assignees

Labels

P2Important issues not at the top of the work listengineflutter/engine related. See also e: labels.found in release: 3.13Found to occur in 3.13found in release: 3.14Found to occur in 3.14has reproducible stepsThe issue has been confirmed reproducible and is ready to work onr: fixedIssue is closed as already fixed in a newer versionteam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions