Skip to content

skyview logging and error handling#756

Merged
pgbrodrick merged 23 commits into
isofit:devfrom
brentwilder:skyview-error-handling
Oct 31, 2025
Merged

skyview logging and error handling#756
pgbrodrick merged 23 commits into
isofit:devfrom
brentwilder:skyview-error-handling

Conversation

@brentwilder

@brentwilder brentwilder commented Sep 9, 2025

Copy link
Copy Markdown
Contributor

Main goal is to have better logging and/or resource management when using Ray for computing horizons. There was a case shared with me where Ray failed to find enough memory/tmp storage for a 1m DSM. Trying to come up with a solution that balances I/O and Memory.

@brentwilder

brentwilder commented Sep 17, 2025

Copy link
Copy Markdown
Contributor Author

Summary of changes:

  • fixing current error of forgetting np.radians() on slope if OBS data given
  • catching/logging of input errors
  • intermediate horizon angles saved to disk, lowering memory usage for fine resolution images. These temp files are removed by default but can be retained.
  • new shadow_mask method, which is just an application of horizon method for specific solar geometry.
  • Cleaned up artifacts in horizon computations (figs of azimuth=85deg (East) below). Downside of using 1d interpolation in this way is that it happens twice, once to skew DEM, and then once again to to unskew the resulting horizon angles. However, this blurring in itself may be better than the original method of whole shifts..
  • Per Evan's comment the ENVI-writing better matches what ISOFIT does / already has defined

using topo-calc's skew() methods with integer shift:
Screenshot 2025-09-22 at 13 49 45

updated, sub-pixel linear interpolations during skew (and de-skewing):
Screenshot 2025-09-22 at 16 44 00

@brentwilder brentwilder marked this pull request as ready for review September 22, 2025 18:37
@brentwilder brentwilder marked this pull request as draft September 22, 2025 18:45
@brentwilder brentwilder marked this pull request as ready for review September 22, 2025 22:39
@brentwilder brentwilder force-pushed the skyview-error-handling branch from c441aff to ba4f2ee Compare September 29, 2025 13:41
@brentwilder

Copy link
Copy Markdown
Contributor Author

Also, addresses #774 by allowing Pathnames to be called without supplying optional arguments (including skyview)

@pgbrodrick

Copy link
Copy Markdown
Collaborator

This looks good....can we pin to zarr > 3.0 (which from conversations are what @brentwilder is uesing)? @jammont may want to revise that with #787 if that moves forward, but my preference would be to stick to zarr 3 if possible.

@pgbrodrick pgbrodrick merged commit a9d6b77 into isofit:dev Oct 31, 2025
16 of 17 checks passed
@jammont

jammont commented Nov 5, 2025

Copy link
Copy Markdown
Collaborator

Don't think we can switch to zarr>3 due to us still supporting python 3.10. Using uv, I tried pinning it:

Creating virtual environment at: .venv
  x No solution found when resolving dependencies for split (markers: python_full_version == '3.10.*'):
  `-> Because only the following versions of zarr are available:
          zarr<=3.0.0
          zarr==3.0.1
          zarr==3.0.2
          zarr==3.0.3
          zarr==3.0.4
          zarr==3.0.5
          zarr==3.0.6
          zarr==3.0.7
          zarr==3.0.8
          zarr==3.0.9
          zarr==3.0.10
          zarr==3.1.0
          zarr==3.1.1
          zarr==3.1.2
          zarr==3.1.3
      and zarr>=3.0.0,<=3.0.2 was yanked (reason: zarr.load() deletes data. See https://github.com/zarr-developers/zarr-python/issues/3061 for more information), we
      can conclude that zarr>=3.0.0,<=3.0.2 cannot be used.
      And because zarr==3.0.3 was yanked (reason: broke a certain indexing incantation with our last release. see #2849. This can lead to data corruption) and
      zarr>=3.0.4,<=3.0.7 was yanked (reason: zarr.load() deletes data. See https://github.com/zarr-developers/zarr-python/issues/3061 for more information), we can
      conclude that zarr>=3.0.4,<=3.0.7 cannot be used. (1)

      Because the requested Python version (>=3.10) does not satisfy Python>=3.11 and zarr>=3.0.8 depends on Python>=3.11, we can conclude that zarr>=3.0.8 cannot
      be used.
      And because we know from (1) that zarr>=3.0.4,<=3.0.7 cannot be used, we can conclude that zarr>=3.0.0 cannot be used.
      And because your project depends on zarr>=3 and your project requires isofit[dev], we can conclude that your project's requirements are unsatisfiable.

      hint: The `requires-python` value (>=3.10) includes Python versions that are not supported by your dependencies (e.g., zarr>=3.0.8 only supports >=3.11).
      Consider using a more restrictive `requires-python` value (like >=3.11).
  help: If you want to add the package regardless of the failed resolution, provide the `--frozen` flag to skip locking and syncing.

@MariaRocco MariaRocco mentioned this pull request Jan 12, 2026
@brentwilder brentwilder deleted the skyview-error-handling branch March 9, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants