What

• Part of Refactor Tensor and Image archetypes #6844
• Closes Encode images as a byte blob + pixel format + resolution; not as a tensor #6386

Breaking changes

• Image(…).compress(…) is now ImageEncoded.compres(…)
• ImageEncoded has very different constructor compared to 0.17, and only supports chroma-downsampled images

TODO

• Fix how enums are handled in Python (I don't understand it). In particular, this code is broken when given a string as a color model:

  rerun/rerun_py/rerun_sdk/rerun/archetypes/image_encoded_ext.py

  Lines 150 to 157 in 8974966

  	color_model = ColorModel(color_model)
  	if color_model == ColorModel.L:
  	mode = "L"
  	elif color_model == ColorModel.RGB:
  	mode = "RGB"
  	else:
  	# TODO(#2340): BGR support!
  	raise ValueError(f"Cannot JPEG compress an image of type {color_model}")

• Fix this test:

  rerun/rerun_py/tests/unit/test_image.py

  Lines 16 to 45 in 4f12d7f

  	IMAGE_INPUTS: list[Any] = [
  	RANDOM_IMAGE_SOURCE,
  	RANDOM_IMAGE_SOURCE,
  	]
  	# 0 = shape
  	# 1 = buffer
  	CHECK_FIELDS: list[list[int]] = [
  	[0, 1],
  	[1],
  	]
  	def tensor_data_expected() -> Any:
  	return TensorDataBatch(IMAGE_INPUTS[0])
  	def compare_images(left: Any, right: Any, check_fields: list[int]) -> None:
  	for field in check_fields:
  	assert left.as_arrow_array().storage.field(field) == right.as_arrow_array().storage.field(field)
  	def test_image() -> None:
  	expected = tensor_data_expected()
  	for input, check_fields in zip(IMAGE_INPUTS, CHECK_FIELDS):
  	arch = rr.Image(input)
  	compare_images(arch.data, expected, check_fields)

• Test detect_and_track_objects
• Test a bunch of other examples
• [ ] Log a warning if color_model is missing, so that we can in the future migrate to having BGR be the default
  • BGR should never be the default. This has haunted opencv for over a decade. We need not inherit this trauma.
• [ ] Consider re-implementing image.compress(…) (with a forward call to ImageEncoded.compress
  • TODO: I don't understand why this was moved in the first place. We should move it back, but it can happen in a follow-on PR.

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)
• I have tested the web demo (if applicable):
  • Using examples from latest main build: rerun.io/viewer
  • Using full set of examples from nightly build: rerun.io/viewer
• The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
• If applicable, add a new check to the release checklist!
• If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide:
  • Capture all the changes to the image APIs in migration guide. #7031

• PR Build Summary
• Recent benchmark results
• Wasm size tracking

To run all checks from main, comment on the PR with @rerun-bot full-check.