Skip to content

tests/test_indexing.py fails on Linux Intel x86 (openSUSE i586 and Debian i386) #3310

@lpechacek

Description

@lpechacek

In openSUSE Build Service the Rasterio tests fail for i586 architecture. As far as I can tell, there are test failures visible in the Debian package build logs on i386 (Debian_rasterio_1.4.3-1+b2_i386.log.gz) as well but do not trigger a package build failure. The issue has been analyzed on openSUSE and the Numpy part was discussed in https://bugzilla.suse.com/show_bug.cgi?id=1236116 . The root cause likely lies in how Rasterio uses Numpy but the very details of the issue are not yet understood.

Expected behavior and actual behavior.

In openSUSE Build Service the package fails to build because a few of its its self-tests fail. The build log showing the failures is at
https://build.opensuse.org/package/live_build_log/Application:Geo/python-rasterio/openSUSE_Tumbleweed/i586 until we disable the failing build. I'm attaching a gzipped build log here for a later reference: OBS_i586_rasterio_build_log.txt.gz

We've invested most effort into the analysis of the the following instance:

[  344s] =================================== FAILURES ===================================
[  344s] __________________________________ test_index __________________________________
[  344s] 
[  344s]     def test_index():
[  344s]         with rasterio.open('tests/data/RGB.byte.tif') as src:
[  344s]             left, bottom, right, top = src.bounds
[  344s] >           assert src.index(left, top) == (0, 0)
[  344s] E           AssertionError: assert (-1, -1) == (0, 0)
[  344s] E             
[  344s] E             At index 0 diff: -1 != 0
[  344s] E             
[  344s] E             Full diff:
[  344s] E               (
[  344s] E             -     0,
[  344s] E             ?     ^...
[  344s] E             
[  344s] E             ...Full output truncated (7 lines hidden), use '-vv' to show
[  344s] 
[  344s] tests/test_indexing.py:24: AssertionError
[  344s] ------------------------------ Captured log call -------------------------------
[  344s] DEBUG    rasterio._env:env.py:338 GDAL data files are available at built-in paths.
[  344s] DEBUG    rasterio._env:env.py:338 PROJ data files are available at built-in paths.
[  344s] DEBUG    rasterio._env:env.py:338 Started GDALEnv: self=<rasterio._env.GDALEnv object at 0xeaf60f28>.
[  344s] DEBUG    rasterio._env:env.py:374 Stopped GDALEnv <rasterio._env.GDALEnv object at 0xeaf60f28>.
[  344s] DEBUG    rasterio._env:env.py:338 GDAL data files are available at built-in paths.
[  344s] DEBUG    rasterio._env:env.py:338 PROJ data files are available at built-in paths.
[  344s] DEBUG    rasterio._env:env.py:338 Started GDALEnv: self=<rasterio._env.GDALEnv object at 0xeaf609d0>.
[  344s] DEBUG    rasterio._env:env.py:374 Stopped GDALEnv <rasterio._env.GDALEnv object at 0xeaf609d0>.

The issue boils down to index() returning [-1, -1] for the top-left corner of the image. The negative numbers come from the fact that the inverse transformation in rowcol() yields numbers that are about one epsilon below zero and the default floor operations rounds it down to -1.

So how come that on x86 the inverse comes out an epsilon below zero while on other platforms it hits exact zero? We don't know exactly, although some clues have been offered in the above referenced openSUSE bug report.

My guess is that the precision issue comes from mixing Python-native and Numpy calculations. AffineTransformer._transform() which ultimately performs the calculation involved in this case first tasks Affine to calculate the transform inverse with Python native floating-point engine and then feeds the inverse matrix numbers into Numpy for the point transformation.

Somewhat unsurprisingly, the following implementation of AffineTransformer._transform() makes the affected tests pass:

    def _transform(self, xs, ys, zs, transform_direction):
        if transform_direction is TransformDirection.forward:
            t = self._transformer
        elif transform_direction is TransformDirection.reverse:
            t = ~self._transformer
        return t * (xs, ys)

Steps to reproduce the problem.

  1. Install openSUSE Tumbleweed for x86 in a virtual machine
  2. Download Rasterio 1.4.3 archive from GitHub
  3. Compile Rasterio
  4. Install packages required for testing (pytest and modules required by the test scripts)
  5. Run pytest-3.11 tests/test_indexing.py - two tests fail

Environment Information

rasterio info:
rasterio: 1.4.3
GDAL: 3.10.1
PROJ: 9.5.1
GEOS: 3.13.0
PROJ DATA: /home/user/.local/share/proj:/usr/share/proj
GDAL DATA: None

System:
python: 3.11.11 (main, Dec 04 2024, 21:44:34) [GCC]
executable: /usr/bin/python3
machine: Linux-6.12.9-1-pae-i686-with-glibc2.40

Python deps:
affine: 2.4.0
attrs: 24.3.0
certifi: 2024.08.30
click: 8.1.8
cligj: 0.7.2
cython: 3.0.11
numpy: 2.1.3
click-plugins: None
setuptools: 75.6.0

Installation Method

Package build for the distribution. The RPM build recipe is in https://build.opensuse.org/projects/Application:Geo/packages/python-rasterio/files/python-rasterio.spec?expand=1 and the build steps are visible in the above package build log file.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions