Skip to content

Python unstable 2022-01-13#154969

Merged
mweinelt merged 432 commits intostagingfrom
python-unstable
Jan 23, 2022
Merged

Python unstable 2022-01-13#154969
mweinelt merged 432 commits intostagingfrom
python-unstable

Conversation

@jonringer
Copy link
Copy Markdown
Contributor

@jonringer jonringer commented Jan 14, 2022

Motivation for this change

It's that time again, not going to immediately trigger eval since the recent CVE master merges caused a lot of queue builds.

https://hydra.nixos.org/jobset/nixpkgs/python-unstable

Previous python-unstable: #149301

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 14, 2022
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jan 14, 2022
@dotlambda dotlambda changed the title Python unstable 2022-01-013 Python unstable 2022-01-13 Jan 14, 2022
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Jan 14, 2022
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jan 14, 2022
@ofborg ofborg bot added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jan 16, 2022
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 17, 2022
@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 17, 2022
@FRidh
Copy link
Copy Markdown
Member

FRidh commented Jan 17, 2022

Just did a force push after rebasing on staging.

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 17, 2022
@cpcloud cpcloud mentioned this pull request Jan 17, 2022
13 tasks
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 23, 2022
@mweinelt mweinelt merged commit 645ddf9 into staging Jan 23, 2022
@jonringer
Copy link
Copy Markdown
Contributor Author

Thanks @mweinelt @dotlambda @fabaff for the help :)

@SuperSandro2000
Copy link
Copy Markdown
Member

@jonringer can you protect the branch so that it does not get deleted by accident?

@trofi
Copy link
Copy Markdown
Contributor

trofi commented Jan 23, 2022

Bisec says c84b744 python3Packages.pyparsing: 2.4.7 -> 3.0.6 broke spice build:

$ nix build -f. spice -L
...
spice> spice-common| Message: Checking for python module six
spice> spice-common| Message: Checking for python module pyparsing
spice> subprojects/spice-common/meson.build:137:8: ERROR: Problem encountered: Python module pyparsing not found
spice> A full log can be found at /build/spice-0.15.0/build/meson-logs/meson-log.txt

@mweinelt
Copy link
Copy Markdown
Member

Fixed in #156305

@trofi
Copy link
Copy Markdown
Contributor

trofi commented Jan 23, 2022

Bisect says c4f55d1 python3Packages.fonttools: 4.26.2 -> 4.28.5 broke python3Packages.afdko tests in staging:

$ nix build -f . python3Packages.afdko
...
python3.9-afdko> tests/ttxn_test.py FFFFFFFFFFFFF                                         [ 73%]
...
python3.9-afdko> =================================== FAILURES ===================================
python3.9-afdko> ___________ test_dump[args0-SourceSansPro-Black_subset.otf-OTF.ttx] ____________
python3.9-afdko> args = [], font_filename = 'SourceSansPro-Black_subset.otf'
python3.9-afdko> exp_filename = 'OTF.ttx'
python3.9-afdko>     @pytest.mark.parametrize('args, font_filename, exp_filename', [
python3.9-afdko>         ([], OTF_FONT, 'OTF.ttx'),
python3.9-afdko>         ([], TTF_FONT, 'TTF.ttx'),
python3.9-afdko>         (['nh'], OTF_FONT, 'OTF_no_hints.ttx'),
python3.9-afdko>         (['tGPOS'], OTF_FONT, 'OTF_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], OTF_FONT, 'OTF_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS'], OTF2_FONT, 'OTF2_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], OTF2_FONT, 'OTF2_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS'], OTF3_FONT, 'OTF3_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], OTF3_FONT, 'OTF3_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS'], TTF2_FONT, 'TTF2_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB'], TTF2_FONT, 'TTF2_GSUB_only.ttx'),
python3.9-afdko>         (['tGPOS', 'se'], TTF3_FONT, 'TTF3_GPOS_only.ttx'),
python3.9-afdko>         (['tGSUB', 'se'], TTF3_FONT, 'TTF3_GSUB_only.ttx'),
python3.9-afdko>     ])
python3.9-afdko>     def test_dump(args, font_filename, exp_filename):
python3.9-afdko>         save_path = tempfile.mkstemp()[1]
python3.9-afdko> >       runner(['-t', TOOL, '-f', font_filename, '-o',
python3.9-afdko>                 f'o{save_path}'] + args)
python3.9-afdko> tests/ttxn_test.py:39:
python3.9-afdko> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python3.9-afdko> tests/runner.py:259: in main
python3.9-afdko>     return run_tool(opts)
python3.9-afdko> tests/runner.py:107: in run_tool
python3.9-afdko>     return subprocess.check_call(args, timeout=TIMEOUT)
python3.9-afdko> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python3.9-afdko> popenargs = (['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf'],)
python3.9-afdko> kwargs = {'timeout': 240}, retcode = 1
python3.9-afdko> cmd = ['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']
python3.9-afdko>     def check_call(*popenargs, **kwargs):
python3.9-afdko>         """Run command with arguments.  Wait for command to complete.  If
python3.9-afdko>         the exit code was zero then return, otherwise raise
python3.9-afdko>         CalledProcessError.  The CalledProcessError object will have the
python3.9-afdko>         return code in the returncode attribute.
python3.9-afdko>         The arguments are the same as for the call function.  Example:
python3.9-afdko>         check_call(["ls", "-l"])
python3.9-afdko>         """
python3.9-afdko>         retcode = call(*popenargs, **kwargs)
python3.9-afdko>         if retcode:
python3.9-afdko>             cmd = kwargs.get("args")
python3.9-afdko>             if cmd is None:
python3.9-afdko>                 cmd = popenargs[0]
python3.9-afdko> >           raise CalledProcessError(retcode, cmd)
python3.9-afdko> E           subprocess.CalledProcessError: Command '['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']' returned non-zero exit status 1.
python3.9-afdko> /nix/store/hxgsg4nn1w47lrbzb20l0b2kzhgi6fmy-python3-3.9.10/lib/python3.9/subprocess.py:373: CalledProcessError
python3.9-afdko> ----------------------------- Captured stderr call -----------------------------
python3.9-afdko> Dumping "/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf" to "/build/tmp1ws_e4mg"...
python3.9-afdko> Traceback (most recent call last):
python3.9-afdko>   File "/nix/store/gis1kmb2lk16plkfxsz6jfdhlvl25pxq-python3.9-afdko-3.7.1/bin/.ttxn-wrapped", line 9, in <module>
python3.9-afdko>     sys.exit(main())
python3.9-afdko>   File "/nix/store/gis1kmb2lk16plkfxsz6jfdhlvl25pxq-python3.9-afdko-3.7.1/lib/python3.9/site-packages/afdko/ttxn.py", line 2324, in main
python3.9-afdko>     ttnDump(input_file, output, options, showExtensionFlag,
python3.9-afdko>   File "/nix/store/7688i22zni4pkznasabcc0n7hcsixscv-python3.9-fonttools-4.28.5/lib/python3.9/site-packages/fontTools/misc/loggingTools.py", line 372, in wrapper
python3.9-afdko>     return func(*args, **kwds)
python3.9-afdko>   File "/nix/store/gis1kmb2lk16plkfxsz6jfdhlvl25pxq-python3.9-afdko-3.7.1/lib/python3.9/site-packages/afdko/ttxn.py", line 2120, in ttnDump
python3.9-afdko>     ttf = TTXNTTFont(input_file, 0, allowVID=options.allowVID,
python3.9-afdko> AttributeError: 'Options' object has no attribute 'allowVID'
python3.9-afdko> ERROR: Command '['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']' returned non-zero exit status 1.
python3.9-afdko> ------------------------------ Captured log call -------------------------------
python3.9-afdko> ERROR    runner:runner.py:112 Command '['ttxn', '-o/build/tmp1ws_e4mg', '/build/afdko-3.7.1/tests/ttxn_data/input/SourceSansPro-Black_subset.otf']' returned non-zero exit status 1.
...

@jonringer
Copy link
Copy Markdown
Contributor Author

@jonringer can you protect the branch so that it does not get deleted by accident?

@SuperSandro2000 I cannot. I think you have to be admin.

cc @domenkozar

@c0bw3b c0bw3b mentioned this pull request Jan 24, 2022
13 tasks
@mweinelt mweinelt deleted the python-unstable branch January 25, 2022 11:10
@mweinelt
Copy link
Copy Markdown
Member

mweinelt commented Jan 25, 2022

Poked on matrix regarding branch protection. Was asked to rename the branch to python-updates so we are aligned with other langs (cmp. haskell-updates). I didn't mind, I hope noone else does either.

@SuperSandro2000
Copy link
Copy Markdown
Member

Does hydra mind?

@mweinelt
Copy link
Copy Markdown
Member

mweinelt commented Jan 25, 2022

Was already updated with that change in mind. The jobset is also now visible on the project page again.

https://hydra.nixos.org/jobset/nixpkgs/python-unstable#tabs-configuration

@smunix
Copy link
Copy Markdown
Contributor

smunix commented Feb 11, 2022

This update has broken kitty #155717

❯ sudo nixos-rebuild switch                                                                                                                                                                                                                                                  [99/370]
building the system configuration...                                                                                                                                                                                                                                                 
warning: Git tree '/etc/nixos' is dirty                        
error: builder for '/nix/store/dhah3nazhf161y7rb77770khvmsxs8xh-kitty-0.24.2.drv' failed with exit code 2;                                                                                                                                                                           
       last 10 log lines:                                    
       >     for result in self.emit(name, *args, allowed_exceptions=allowed_exceptions):                                                 
       >   File "/nix/store/13z9v2893iapkj12gk47q04w2197gysq-python3.9-sphinx-4.4.0/lib/python3.9/site-packages/sphinx/events.py", line 109, in emit
       >     raise ExtensionError(__("Handler %r for event %r threw an exception") %                                                      
       > sphinx.errors.ExtensionError: Handler <function _html_page_context at 0x7ffff3218c10> for event 'html-page-context' threw an exception (exception: [Errno 2] No such file or directory: '/nix/store/pppq4jc77qw9qv4rqghqa32rjdm314hq-python3.9-furo-2022.1.2/lib/python3.9/s
ite-packages/furo/theme/furo/static/styles/furo.css')                                                                                     
       >                                         
       > Extension error (furo):                    
       > Handler <function _html_page_context at 0x7ffff3218c10> for event 'html-page-context' threw an exception (exception: [Errno 2] No such file or directory: '/nix/store/pppq4jc77qw9qv4rqghqa32rjdm314hq-python3.9-furo-2022.1.2/lib/python3.9/site-packages/furo/theme/furo/s
tatic/styles/furo.css')                    
       > make[1]: *** [Makefile:24: html] Error 2                                                                                                                                                                                                                                    
       > make[1]: Leaving directory '/build/source/docs'                                                                                                                                                                                                                             

@mweinelt
Copy link
Copy Markdown
Member

"This" is pretty vague for a package set update. The build of kitty might need to pin an older version of python3Packages.furo. Maybe there is a fix for this update upstream already.

@smunix
Copy link
Copy Markdown
Contributor

smunix commented Feb 11, 2022

Agreed, the fix needs to be made in kitty indeed. I'll discuss this with Kitty's maintainers.

@Luflosi
Copy link
Copy Markdown
Contributor

Luflosi commented Feb 11, 2022

This is not a kitty problem. furo is broken in nixpkgs because it depends on webpack to build some critical files, which is itself broken.

@Luflosi
Copy link
Copy Markdown
Contributor

Luflosi commented Feb 11, 2022

I created #159255, which fixes the problem.

@06kellyjac
Copy link
Copy Markdown
Member

the tomlkit update broke remarshal which is used by pkgs.formats (fixed in #159074)

https://github.com/NixOS/nixpkgs/blob/master/pkgs/pkgs-lib/formats.nix

Were there any tests to catch this? since pkgs.formats is used in quite a lot of modules and is a pain to fix without waiting for fixes to hit unstable
If not maybe we need more tests around "pkgs-lib"?

@mweinelt
Copy link
Copy Markdown
Member

Python-unstable does not run NixOS tests, but maybe we should given that the test runner is also written in python.

@06kellyjac
Copy link
Copy Markdown
Member

Yeah, at the very least we should probably run some of the super core tests e.g pkgs-lib and those meta tests for the nixos testsuite itself (which I think are still in a PR but I cant find them right now)

@jonringer
Copy link
Copy Markdown
Contributor Author

jonringer commented Feb 11, 2022

Solved by #159322

@smunix
Copy link
Copy Markdown
Contributor

smunix commented Feb 12, 2022

I created #159255, which fixes the problem.

thanks @Luflosi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.