Skip to content

CI: Fix the release workflows (GHA and travis CI)#1253

Merged
jorisvandenbossche merged 9 commits intoshapely:mainfrom
caspervdw:casper-pygeos012
Apr 27, 2022
Merged

CI: Fix the release workflows (GHA and travis CI)#1253
jorisvandenbossche merged 9 commits intoshapely:mainfrom
caspervdw:casper-pygeos012

Conversation

@caspervdw
Copy link
Copy Markdown
Collaborator

@caspervdw caspervdw commented Dec 9, 2021

The goal is here to make the wheel build and travis CIs pass.

Some open ends:

  • The tests are still outside of the package, which makes running them after building a wheel more complex. I prefer moving them in a separate folder. Maybe we should have tests_shapely and tests_pygeos submodules for now?
  • Travis seems to be out of credits. @sgillies Do you know if we are entitled to OSS credits?
  • We need a PyPI token to be generated, encrypted and configured in the .travis.yml
  • We need another PyPI token set up as pypi_password "secret" in Github Actions (see docs)
  • Appveyor needs to be switched off.

BTW, I don't have manage privileges on this repo, is that correct @sgillies? I'd like to edit the ticket in #962 to reference this PR.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 81.131% when pulling ba46a29 on caspervdw:casper-pygeos012 into a0f8a31 on Toblerity:main.

@jorisvandenbossche
Copy link
Copy Markdown
Member

Regarding the secrets and manage access, I would maybe directly / first move to the shapely org, and fix it there (otherwise we will need to do it twice), see #1204

@jorisvandenbossche
Copy link
Copy Markdown
Member

The tests are still outside of the package, which makes running them after building a wheel more complex. I prefer moving them in a separate folder. Maybe we should have tests_shapely and tests_pygeos submodules for now?

My plan was to progressively move the tests from /tests into /shapely/tests (eg #1257 as a start). Personally I wouldn't care too much about the /tests here for now and just run /shapely/tests in the packaging CI (I suppose we won't make an actual release before the tests are actually moved. Or otherwise we can already move them in bulk into a subdirectory of /shapely/tests (I would avoid creating tests_pygeos / tests_shapely, as that would unnecessarily move once more the tests that are already in /shapely/tests)

@jorisvandenbossche
Copy link
Copy Markdown
Member

@caspervdw I think you should have the privileges now to continue this PR?

@caspervdw
Copy link
Copy Markdown
Collaborator Author

Yes that’s ok, only Travis is still showing the message “ Builds have been temporarily disabled for public repositories due to a negative credit balance”

I am not sure how to proceed here. Did the negative credit balance get carried along with the organisation change of tobleriy to here? @sgillies do you have an idea on how to get Travis running again? Maybe we can reach out to Travis?

@jorisvandenbossche
Copy link
Copy Markdown
Member

Did the negative credit balance get carried along with the organisation change of tobleriy to here?

This organization was already used for the shapely-wheels repo, so I suppose the credits have been used there

@jorisvandenbossche
Copy link
Copy Markdown
Member

I emailed Travis' support for requesting OSS credits.

In the meantime, I think we can already get this PR merged?

@caspervdw
Copy link
Copy Markdown
Collaborator Author

Sure we can get this merged right away, however the PyPI tokens still need to be configured.

@sgillies Could you generate two tokens for the shapely repo? Or grant me rights (username: caspervdw) at PyPI to do so myself?

@jorisvandenbossche
Copy link
Copy Markdown
Member

BTW, I got reply from Travis and we should now have free credits (at least for some time ..)
I see already some things running at https://app.travis-ci.com/github/shapely/shapely. But in order to limit our use of credits, we should maybe disable Travis by default, and only have it run on tags or on specific demand? (to test before a release) But not at every commit on master, or certainly not at every commit on PRs

@jorisvandenbossche
Copy link
Copy Markdown
Member

But so it was running of this PR, so that is useful: https://app.travis-ci.com/github/shapely/shapely/jobs/561140861

@jorisvandenbossche
Copy link
Copy Markdown
Member

jorisvandenbossche commented Feb 24, 2022

There seems to be one test failure on Python 3.8 for the aarch64 builds:

=================================== FAILURES ===================================
____________________________ test_from_invalid_dim _____________________________
    def test_from_invalid_dim():
        # TODO(shapely-2.0) better error message?
        # pytest.raises(ValueError, match="at least 2 coordinate tuples|at least 2 coordinates"):
        with pytest.raises(shapely.GEOSException):
            LineString([(1, 2)])
    
        with pytest.raises(
            ValueError, match="Input operand 0 does not have enough dimensions"
        ):
>           LineString([(1, 2, 3), (4, 5)])
/tmp/tmp.HukXYlwDK9/venv/lib/python3.8/site-packages/shapely/tests/geometry/test_linestring.py:112: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/tmp.HukXYlwDK9/venv/lib/python3.8/site-packages/shapely/geometry/linestring.py:65: in __new__
    geom = shapely.linestrings(coordinates)
/tmp/tmp.HukXYlwDK9/venv/lib/python3.8/site-packages/shapely/decorators.py:80: in wrapped
    return func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
coords = [(1, 2, 3), (4, 5)], y = None, z = None, indices = None, out = None
kwargs = {}
    @multithreading_enabled
    def linestrings(coords, y=None, z=None, indices=None, out=None, **kwargs):
        """Create an array of linestrings.
    
        This function will raise an exception if a linestring contains less than
        two points.
    
        Parameters
        ----------
        coords : array_like
            An array of lists of coordinate tuples (2- or 3-dimensional) or, if ``y``
            is provided, an array of lists of x coordinates
        y : array_like, optional
        z : array_like, optional
        indices : array_like, optional
            Indices into the target array where input coordinates belong. If
            provided, the coords should be 2D with shape (N, 2) or (N, 3) and
            indices should be an array of shape (N,) with integers in increasing
            order. Missing indices result in a ValueError unless ``out`` is
            provided, in which case the original value in ``out`` is kept.
        out : ndarray, optional
            An array (with dtype object) to output the geometries into.
        **kwargs
            For other keyword-only arguments, see the
            `NumPy ufunc docs <https://numpy.org/doc/stable/reference/ufuncs.html#ufuncs-kwargs>`_.
            Ignored if ``indices`` is provided.
    
        Examples
        --------
        >>> linestrings([[[0, 1], [4, 5]], [[2, 3], [5, 6]]]).tolist()
        [<pygeos.Geometry LINESTRING (0 1, 4 5)>, <pygeos.Geometry LINESTRING (2 3, 5 6)>]
        >>> linestrings([[0, 1], [4, 5], [2, 3], [5, 6], [7, 8]], indices=[0, 0, 1, 1, 1]).tolist()
        [<pygeos.Geometry LINESTRING (0 1, 4 5)>, <pygeos.Geometry LINESTRING (2 3, 5 6, 7 8)>]
    
        Notes
        -----
        - Usage of the ``y`` and ``z`` arguments will prevents lazy evaluation in ``dask``.
          Instead provide the coordinates as a ``(..., 2)`` or ``(..., 3)`` array using only ``coords``.
        """
        coords = _xyz_to_coords(coords, y, z)
        if indices is None:
>           return lib.linestrings(coords, out=out, **kwargs)
E           TypeError: ufunc 'linestrings' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Since it was passing on older python versions, it might be dependent on the numpy version?
And it is also only the type of the error that is being catched that is different.

@coveralls
Copy link
Copy Markdown

coveralls commented Feb 24, 2022

Pull Request Test Coverage Report for Build 1895047080

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 84.009%

Totals Coverage Status
Change from base Build 1831557710: 0.0%
Covered Lines: 2196
Relevant Lines: 2614

💛 - Coveralls

- master
- main
- travis
- /^\d+\.\d+(\.\d+)?(-\S*)?$/
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://docs.travis-ci.com/user/conditions-v1, we should be able to add something like if: tag IS present for having it only run on tags (on main)

@jorisvandenbossche jorisvandenbossche changed the title WIP: Fix the release and travis CI CI: Fix the release GHA and travis CI Apr 26, 2022
@jorisvandenbossche jorisvandenbossche changed the title CI: Fix the release GHA and travis CI CI: Fix the release workflows (GHA and travis CI) Apr 26, 2022
@jorisvandenbossche
Copy link
Copy Markdown
Member

For Travis, the wheel build job is passing, but the other ones are still failing (see https://app.travis-ci.com/github/shapely/shapely/builds/249895523). Since this PR is focusing on fixing the release workflow, I am going to leave those other builds for a follow-up.

@jorisvandenbossche
Copy link
Copy Markdown
Member

The proper tokens are still missing here for PyPI uploads, but let's also do that in a follow-up PR. I am going to merge this, to have the release workflow working again (and #1373 can be updated)

@jorisvandenbossche jorisvandenbossche merged commit f947ff7 into shapely:main Apr 27, 2022
@jorisvandenbossche jorisvandenbossche mentioned this pull request May 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants