Skip to content

objects.colors: add save_cpt_file()#1423

Merged
yunjunz merged 10 commits intoinsarlab:mainfrom
yunjunz:bugfix
Oct 11, 2025
Merged

objects.colors: add save_cpt_file()#1423
yunjunz merged 10 commits intoinsarlab:mainfrom
yunjunz:bugfix

Conversation

@yunjunz
Copy link
Member

@yunjunz yunjunz commented Oct 11, 2025

Description of proposed changes

This PR adds objects.colors.save_cpt_file() and fixes a few bugs:

  • objects.colors:

    • add save_cpt_file() to save an matplotlib colormap object into an GMT-compatible CPT file
    • move the following two functions out of the ColormapExt object for easier maintainence and re-use:
      • read_cpt_file()
      • cmap_map()
  • docs:

    • api/colormaps: update link of cpt-city
    • QGIS: simplify the org
  • circleci: use blobless checkout

  • bugfixes:

    • readfile.read(): check if input box is within the data range
    • utils.transect_yx(): remove distance_unit for simple return; otherwise, it will return an error while concatenating multiple segments.

Reminders

  • Pass Pre-commit check (green)
  • Pass Codacy code review (green)
  • Pass Circle CI test (green)
  • Make sure that your code follows our style. Use the other functions/files as a basis.
  • If modifying functionality, describe changes to function behavior and arguments in a comment below the function declaration.
  • If adding new functionality, add a detailed description to the documentation and/or an example.

+ objects.colors:
   - add `save_cpt_file()` to save an matplotlib colormap object into an GMT-compatible CPT file
   - move the following two functions out of the ColormapExt object for easier maintainence and re-use:
      - read_cpt_file()
      - cmap_map()

+ docs/api/colormaps: update link of cpt-city
otherwise, it will return error while concatenating multiple segments.
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Oct 11, 2025

Reviewer's Guide

This PR implements a new save_cpt_file function for exporting Matplotlib colormaps to GMT-compatible CPT files, refactors existing colormap utilities into top-level functions, and fixes data-range and transect output issues, accompanied by minor documentation and CI configuration tweaks.

Class diagram for refactored colormap utilities in objects.colors

classDiagram
    class ColormapExt {
        +get_single_colormap(cmap_name, cmap_lut)
        +get_cpt_colormap(cmap_name, cmap_lut)
    }
    read_cpt_file : function
    save_cpt_file : function
    cmap_map : function
    ColormapExt --> read_cpt_file : uses
    ColormapExt --> cmap_map : uses
    %% read_cpt_file and cmap_map moved out of ColormapExt
    %% save_cpt_file added as new top-level function
Loading

Class diagram for readfile.read() box validation bugfix

classDiagram
    class readfile {
    }
    read : function
    read : checks box within data range
    readfile --> read : contains
    %% read() now raises ValueError if box is out of bounds
Loading

File-Level Changes

Change Details Files
Add save_cpt_file functionality
  • introduce save_cpt_file to write colormap samples to CPT format
  • sample colors evenly between vmin/vmax and write RGB entries
  • append B/F/N directives for background, foreground, NaN
src/mintpy/objects/colors.py
Refactor colormap utilities
  • move read_cpt_file and cmap_map out of ColormapExt as module-level functions
  • update get_cpt_colormap to call external read_cpt_file
  • clean up commented brightness/dark code and adjust imports
src/mintpy/objects/colors.py
Enforce bounds check in readfile.read
  • add validation that provided box lies within data width/length
  • raise ValueError on out-of-range box coordinates
src/mintpy/utils/readfile.py
Remove distance_unit in transect_yx output
  • comment out dist_unit assignment to avoid downstream concatenation errors
  • remove distance_unit field from returned transect dict
src/mintpy/utils/utils.py
Update documentation links and QGIS install instructions
  • correct cpt-city URLs in api/colormaps.md
  • simplify plugin install steps and add reference link in docs/QGIS.md
docs/api/colormaps.md
docs/QGIS.md
Enable blobless checkout in CircleCI
  • modify .circleci/config.yml to use blobless checkout method
.circleci/config.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `src/mintpy/objects/colors.py:308-313` </location>
<code_context>
-        # list of string --> x/r/g/b
-        x, r, g, b = [], [], [], []
-        colorModel = "RGB"
-        for line in lines:
-            ls = re.split(' |\t|\n|/', line)
+################################## Utility Functions ############################################

-            # skip empty lines
-            if not ls:
-                continue
+def read_cpt_file(cpt_file, cmap_lut=256, print_msg=False):
</code_context>

<issue_to_address>
**suggestion:** The check for empty lines may not work as intended.

Lines containing only whitespace or tabs may not be skipped, as splitting them can yield non-empty lists. Use 'if not line.strip()' to reliably skip such lines before splitting.

```suggestion
        for line in lines:
            # skip lines containing only whitespace or tabs
            if not line.strip():
                continue
            ls = re.split(' |\t|\n|/', line)
```
</issue_to_address>

### Comment 2
<location> `src/mintpy/objects/colors.py:322-328` </location>
<code_context>
+        lines = f.readlines()

-            # convert color name (in GMT cpt file sometimes) to rgb values
-            if not isnumber(ls[1]):
-                ls0 = list(ls) + [0,0]
-                ls0[1:4] = [i*255. for i in to_rgb(ls[1])]
-                ls0[4:] = ls[2:]
-                ls = list(ls0)
-
-            if not isnumber(ls[5]):
-                ls0 = list(ls) + [0,0]
-                ls0[5:8] = [i*255. for i in to_rgb(ls[5])]
</code_context>

<issue_to_address>
**issue:** Potential index error if CPT file lines are malformed.

Add a length check for 'ls' before accessing ls[1] and ls[5] to prevent IndexError on malformed lines.
</issue_to_address>

### Comment 3
<location> `src/mintpy/objects/colors.py:368` </location>
<code_context>
-
-        # x/r/g/b --> colorDict
-        red, blue, green = [], [], []
-        xNorm = (x - x[0]) / (x[-1] - x[0])
-        for i in range(len(x)):
-            red.append((xNorm[i], r[i], r[i]))
</code_context>

<issue_to_address>
**issue (bug_risk):** Division by zero risk if x[0] == x[-1].

Add a condition to handle cases where x[0] == x[-1] to prevent division by zero when the CPT file has only one color entry.
</issue_to_address>

### Comment 4
<location> `src/mintpy/objects/colors.py:390-391` </location>
<code_context>
+    return colormap
+
+
+def save_cpt_file(colormap, cpt_file, cmap_lut=256, vmin=0, vmax=1, print_msg=True):
+    """Save matplotlib colormap object into GMT-compatible CPT file.
+
</code_context>

<issue_to_address>
**suggestion (bug_risk):** Consider validating vmin and vmax values.

A check for vmin < vmax will help avoid silent failures when generating the array.

```suggestion
def save_cpt_file(colormap, cpt_file, cmap_lut=256, vmin=0, vmax=1, print_msg=True):
    """Save matplotlib colormap object into GMT-compatible CPT file.

    if vmin >= vmax:
        raise ValueError(f"Invalid vmin/vmax: vmin ({vmin}) must be less than vmax ({vmax}).")
```
</issue_to_address>

### Comment 5
<location> `src/mintpy/objects/colors.py:417-418` </location>
<code_context>
+            # Normalize to [0, 1] and get RGBA
+            c1 = np.array(colormap(norm(values[i]))[:3]) * 255
+            c2 = np.array(colormap(norm(values[i + 1]))[:3]) * 255
+            f.write(f"{values[i]:.6f} {int(c1[0])} {int(c1[1])} {int(c1[2])} "
+                    f"{values[i+1]:.6f} {int(c2[0])} {int(c2[1])} {int(c2[2])}\n")
+
+        # Optional: NaN color
</code_context>

<issue_to_address>
**suggestion:** Consider rounding color values before casting to int.

Rounding before casting avoids truncation and ensures color values are more accurate.

```suggestion
            f.write(f"{values[i]:.6f} {int(round(c1[0]))} {int(round(c1[1]))} {int(round(c1[2]))} "
                    f"{values[i+1]:.6f} {int(round(c2[0]))} {int(round(c2[1]))} {int(round(c2[2]))}\n")
```
</issue_to_address>

### Comment 6
<location> `src/mintpy/utils/readfile.py:356-358` </location>
<code_context>
-                if ls[-1] == "HSV":
-                    colorModel = "HSV"
-                    continue
-                else:
-                    continue
+    if not os.path.isfile(cpt_file):
</code_context>

<issue_to_address>
**question:** Box boundary check does not handle box[2] == width or box[3] == length.

Please confirm whether box[2] and box[3] should be strictly less than width and length, or if equality is acceptable for your use case.
</issue_to_address>

### Comment 7
<location> `src/mintpy/objects/colors.py:286` </location>
<code_context>
-            # skip empty lines
-            if not ls:
-                continue
+def read_cpt_file(cpt_file, cmap_lut=256, print_msg=False):
+    """Read *.cpt file into matplotlib colormap object.
+    Modified from Scipy Cookbook originally written by James Boyle.
</code_context>

<issue_to_address>
**issue (complexity):** Consider refactoring the parsing logic in read_cpt_file and cmap_map into small, focused helper functions to improve code clarity and maintainability.

The flat‐out parsing in both `read_cpt_file` and `cmap_map` is still huge and deeply nested. You can keep 100% of the functionality but dramatically reduce cognitive load by pulling all of that “string → numeric” work out into small, focused helpers. For example:

```python
# colors.py

def _is_data_line(line):
    """True if this line should become a CPT row (not header, not B/F/N)."""
    stripped = line.strip()
    return (
        stripped
        and not stripped.startswith('#')
        and stripped.split()[0] not in ("B", "F", "N")
    )

def _to_numeric(token):
    """Turn either a number or a matplotlib‐name into a float in [0..255]."""
    try:
        return float(token)
    except ValueError:
        # e.g. “red” → (1.0,0,0) → (255,0,0)
        rgb = np.array(to_rgb(token)) * 255
        return rgb  # maybe return array here and handle below

def _parse_row(tokens):
    """
    Given a cleaned list of tokens (after split+filter), 
    returns (x0, r0, g0, b0, x1, r1, g1, b1) all as floats.
    """
    # ensure we have enough slots
    if not isnumber(tokens[1]):
        tokens = tokens[:1] + list(to_rgb(tokens[1])) + tokens[2:]
    if not isnumber(tokens[5]):
        tokens = tokens[:5] + list(to_rgb(tokens[5])) + tokens[6:]
    return tuple(map(float, tokens[:8]))
```

Then your big `read_cpt_file` collapses to:

```python
def read_cpt_file(cpt_file, cmap_lut=256, print_msg=False):
    if not os.path.isfile(cpt_file):
        raise FileNotFoundError(f"file {cpt_file} not found")
    if print_msg:
        print(f"Reading CPT → {cpt_file}")
    rows = []
    with open(cpt_file) as f:
        for line in f:
            if _is_data_line(line):
                tokens = re.split(r'\s+|/', line.strip())
                tokens = [t for t in tokens if t]
                rows.append(_parse_row(tokens))
    # Now rows is a list of 8‐tuples, you can easily unzip:
    x0, r0, g0, b0, x1, r1, g1, b1 = map(np.array, zip(*rows))
    # … convert HSV→RGB if needed, normalize by 255, build color_dict …
    # (rest of your existing logic)
```

Same idea for `cmap_map`: pull out “compute the LUT” and “rebuild the segment‐dict” into separate helpers. This shrinks each function to a straight-line sequence:

1. load / filter lines  
2. parse one line  
3. postprocess arrays  
4. build and return the colormap  

Now each helper is <20 LOC, the overall flow is obvious, and you haven’t lost a byte of behavior.
</issue_to_address>

### Comment 8
<location> `src/mintpy/objects/colors.py:322` </location>
<code_context>
def read_cpt_file(cpt_file, cmap_lut=256, print_msg=False):
    """Read *.cpt file into matplotlib colormap object.
    Modified from Scipy Cookbook originally written by James Boyle.
    Link: http://scipy-cookbook.readthedocs.io/items/Matplotlib_Loading_a_colormap_dynamically.html

    Parameters: cpt_file - str, path to the GMT-compatible CPT file
                cmap_lut - int, number of color steps
    Returns:    colormap - matplotlib colormap object
    """

    if not os.path.isfile(cpt_file):
        raise FileNotFoundError(f"file {cpt_file} not found")

    # read file into list of strings
    if print_msg:
        print(f'read CPT file: {cpt_file}')
    with open(cpt_file) as f:
        lines = f.readlines()

    # list of string --> x/r/g/b
    x, r, g, b = [], [], [], []
    colorModel = "RGB"
    for line in lines:
        ls = re.split(' |\t|\n|/', line)

        # skip empty lines
        if not ls:
            continue

        # remove empty element
        ls = [i for i in ls if i]

        # parse header info
        if line[0] == "#":
            if ls[-1] == "HSV":
                colorModel = "HSV"
                continue
            else:
                continue

        # skip BFN info
        if ls[0] in ["B", "F", "N"]:
            continue

        # convert color name (in GMT cpt file sometimes) to rgb values
        if not isnumber(ls[1]):
            ls0 = list(ls) + [0,0]
            ls0[1:4] = [i*255. for i in to_rgb(ls[1])]
            ls0[4:] = ls[2:]
            ls = list(ls0)
        if not isnumber(ls[5]):
            ls0 = list(ls) + [0,0]
            ls0[5:8] = [i*255. for i in to_rgb(ls[5])]
            ls = list(ls0)

        # convert str to float
        ls = [float(i) for i in ls]

        # parse color vectors
        x.append(ls[0])
        r.append(ls[1])
        g.append(ls[2])
        b.append(ls[3])

        # save last row
        xtemp = ls[4]
        rtemp = ls[5]
        gtemp = ls[6]
        btemp = ls[7]

    x.append(xtemp)
    r.append(rtemp)
    g.append(gtemp)
    b.append(btemp)
    x = np.array(x, np.float32)
    r = np.array(r, np.float32)
    g = np.array(g, np.float32)
    b = np.array(b, np.float32)

    if colorModel == "HSV":
        # convert HSV to RGB
        for i in range(r.shape[0]):
            r[i], g[i], b[i] = colorsys.hsv_to_rgb(r[i]/360., g[i], b[i])
    elif colorModel == "RGB":
        r /= 255.
        g /= 255.
        b /= 255.

    # x/r/g/b --> color_dict
    red, blue, green = [], [], []
    xNorm = (x - x[0]) / (x[-1] - x[0])
    for i in range(len(x)):
        red.append((xNorm[i], r[i], r[i]))
        green.append((xNorm[i], g[i], g[i]))
        blue.append((xNorm[i], b[i], b[i]))

    # return colormap
    cmap_name = os.path.splitext(os.path.basename(cpt_file))[0]
    color_dict = {"red":tuple(red), "green":tuple(green), "blue":tuple(blue)}
    colormap = LinearSegmentedColormap(cmap_name, color_dict, N=cmap_lut)

    return colormap

</code_context>

<issue_to_address>
**issue (code-quality):** We've found these issues:

- Hoist repeated code outside conditional statement ([`hoist-statement-from-if`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/hoist-statement-from-if/))
- Simplify conditional into switch-like form ([`switch`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/switch/))
- Inline variable that is immediately returned ([`inline-immediately-returned-variable`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/inline-immediately-returned-variable/))
</issue_to_address>

### Comment 9
<location> `src/mintpy/objects/colors.py:438` </location>
<code_context>
def cmap_map(function, cmap):
    """ Applies function (which should operate on vectors of shape 3: [r, g, b]), on colormap cmap.
    This routine will break any discontinuous points in a colormap.
    Link: http://scipy-cookbook.readthedocs.io/items/Matplotlib_ColormapTransformations.html

    Examples:
        colormap = self.cmap_map(lambda x: x/2 + 0.5, colormap)  # brighten colormap
        colormap = self.cmap_map(lambda x: x*0.75, colormap)     # darken   colormap
    """
    cdict = cmap._segmentdata
    step_dict = {}

    # First get the list of points where the segments start or end
    for key in ('red', 'green', 'blue'):
        step_dict[key] = list(map(lambda x: x[0], cdict[key]))
    step_list = sum(step_dict.values(), [])
    step_list = np.array(list(set(step_list)))

    # Then compute the LUT, and apply the function to the LUT
    reduced_cmap = lambda step : np.array(cmap(step)[0:3])
    old_LUT = np.array(list(map(reduced_cmap, step_list)))
    new_LUT = np.array(list(map(function, old_LUT)))

    # Now try to make a minimal segment definition of the new LUT
    cdict = {}
    for i, key in enumerate(['red','green','blue']):
        this_cdict = {}
        for j, step in enumerate(step_list):
            if step in step_dict[key]:
                this_cdict[step] = new_LUT[j, i]
            elif new_LUT[j,i] != old_LUT[j, i]:
                this_cdict[step] = new_LUT[j, i]
        colorvector = list(map(lambda x: x + (x[1], ), this_cdict.items()))
        colorvector.sort()
        cdict[key] = colorvector

    return LinearSegmentedColormap('colormap',cdict,1024)

</code_context>

<issue_to_address>
**issue (code-quality):** We've found these issues:

- Convert for loop into dictionary comprehension ([`dict-comprehension`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/dict-comprehension/))
- Replace a[0:x] with a[:x] and a[x:len(a)] with a[x:] ([`remove-redundant-slice-index`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/remove-redundant-slice-index/))
- Remove an unnecessary list construction call prior to sorting ([`skip-sorted-list-construction`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/skip-sorted-list-construction/))
</issue_to_address>

### Comment 10
<location> `src/mintpy/utils/readfile.py:356-359` </location>
<code_context>
def read(fname, box=None, datasetName=None, print_msg=True, xstep=1, ystep=1, data_type=None,
         no_data_values=None):
    """Read one dataset and its attributes from input file.

    Parameters: fname          - str, path of file to read
                datasetName    - str or list of str, slice names
                box            - 4-tuple of int area to read, defined in (x0, y0, x1, y1) in pixel coordinate
                x/ystep        - int, number of pixels to pick/multilook for each output pixel
                data_type      - numpy data type, e.g. np.float32, np.bool_, etc. Change the output data type
                no_data_values - list of 2 numbers, change the no-data-value in the output
    Returns:    data           - 2/3/4D matrix in numpy.array format, return None if failed
                atr            - dictionary, attributes of data, return None if failed
    Examples:
        from mintpy.utils import readfile
        data, atr = readfile.read('velocity.h5')
        data, atr = readfile.read('timeseries.h5')
        data, atr = readfile.read('timeseries.h5', datasetName='timeseries-20161020')
        data, atr = readfile.read('ifgramStack.h5', datasetName='unwrapPhase')
        data, atr = readfile.read('ifgramStack.h5', datasetName='unwrapPhase-20161020_20161026')
        data, atr = readfile.read('ifgramStack.h5', datasetName='coherence', box=(100,1100, 500, 2500))
        data, atr = readfile.read('geometryRadar.h5', datasetName='height')
        data, atr = readfile.read('geometryRadar.h5', datasetName='bperp')
        data, atr = readfile.read('100120-110214.unw', box=(100,1100, 500, 2500))
    """
    fname = os.fspath(fname)  # Convert from possible pathlib.Path
    # metadata
    dsname4atr = None   #used to determine UNIT
    if isinstance(datasetName, list):
        dsname4atr = datasetName[0].split('-')[0]
    elif isinstance(datasetName, str):
        dsname4atr = datasetName.split('-')[0]
    atr = read_attribute(fname, datasetName=dsname4atr)

    # box
    length, width = int(atr['LENGTH']), int(atr['WIDTH'])
    if not box:
        box = (0, 0, width, length)
    else:
        # check if input box is within the data range
        if box[0] < 0 or box[1] < 0 or box[2] > width or box[3] > length:
            raise ValueError(f'Input box {tuple(box)} is NOT within the data size range (0, 0, {width}, {length})!')

    # read data
    kwargs = dict(
        datasetName=datasetName,
        box=box,
        xstep=xstep,
        ystep=ystep,
    )

    fext = os.path.splitext(os.path.basename(fname))[1].lower()
    if fext in ['.h5', '.he5']:
        data = read_hdf5_file(fname, print_msg=print_msg, **kwargs)

    else:
        data, atr = read_binary_file(fname, **kwargs)

    # customized output data type
    if data_type is not None and data_type != data.dtype:
        if print_msg:
            print(f'convert numpy array from {data.dtype} to {data_type}')
        data = np.array(data, dtype=data_type)

    # convert no-data-value
    if isinstance(no_data_values, list):
        if print_msg:
            print(f'convert no-data-value from {no_data_values[0]} to {no_data_values[1]}')
        data[data == no_data_values[0]] = no_data_values[1]

    return data, atr

</code_context>

<issue_to_address>
**suggestion (code-quality):** Merge else clause's nested if statement into elif ([`merge-else-if-into-elif`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/merge-else-if-into-elif/))

```suggestion
    elif box[0] < 0 or box[1] < 0 or box[2] > width or box[3] > length:
        raise ValueError(f'Input box {tuple(box)} is NOT within the data size range (0, 0, {width}, {length})!')
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

yunjunz and others added 5 commits October 11, 2025 21:52
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
@yunjunz yunjunz merged commit a738e54 into insarlab:main Oct 11, 2025
8 checks passed
@yunjunz yunjunz deleted the bugfix branch October 11, 2025 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant