Skip to content

Convert format and % strings to f-strings#8958

Merged
bsipocz merged 1 commit into
astropy:masterfrom
taldcroft:f-strings
Jul 11, 2019
Merged

Convert format and % strings to f-strings#8958
bsipocz merged 1 commit into
astropy:masterfrom
taldcroft:f-strings

Conversation

@taldcroft

@taldcroft taldcroft commented Jul 4, 2019

Copy link
Copy Markdown
Member

This used a hacked version of pyupgrade at e477316 to convert format and % strings to modern formatting, including f-strings where possible. The hack bit was to essentially de-select all upgrades except for string formatting ones.

Not included are multiline statements for reasons discussed below.

@taldcroft

Copy link
Copy Markdown
Member Author

And of course I jumped the gun because I see #8955 not yet merged.

@bsipocz

bsipocz commented Jul 4, 2019

Copy link
Copy Markdown
Member

And of course I jumped the gun because I see #8955 not yet merged.

@taldcroft - You're welcome to merge it, or approve so I can do it. Dropping python 3.5 for 4.0 was agreed a while ago, and it's also in the APE.

Comment thread astropy/__init__.py
if sp.returncode != 0:
raise OSError('Running setup.py build_ext --inplace failed '
'with error code {0}: try rerunning this command '
'with error code {}: try rerunning this command '

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific logic why the latter 3 cases keep the .format() syntax while the first one is converted, or does pyupgrade simply not handle multi-line statements?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed it looks like pyupgrade doesn't do multi-line statements. Grr.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's intentionally timid -- it doesn't want to make your lines longer which might anger flake8

@taldcroft

Copy link
Copy Markdown
Member Author

About merging, presumably it needs to pass all the CI tests first. It does pass locally on my laptop.

@dhomeier

dhomeier commented Jul 4, 2019

Copy link
Copy Markdown
Contributor

As long as the 3.5 test setup is included, CircleCI will never make it to the 3.6 tests. But the Travis failures all seem to be due to the np.broadcast_arrays DeprecationWarnings from numpy 1.18.

@dhomeier

dhomeier commented Jul 4, 2019

Copy link
Copy Markdown
Contributor

Tested with Python 3.6 + 3.8b1 on my Mac as well.

@asottile

asottile commented Jul 5, 2019

Copy link
Copy Markdown

the crash in astropy/utils/console.py was fixed in asottile/pyupgrade#173 fwiw -- just haven't made a release yet

Here's the diff if you want to apply it:

$ git diff | cat
diff --git a/astropy/utils/console.py b/astropy/utils/console.py
index a8040906d..c89da247b 100644
--- a/astropy/utils/console.py
+++ b/astropy/utils/console.py
@@ -256,7 +256,7 @@ def _color_text(text, color):
         return text
 
     color_code = color_mapping.get(color, '0;39')
-    return '\033[{0}m{1}\033[0m'.format(color_code, text)
+    return f'\033[{color_code}m{text}\033[0m'
 
 
 def _decode_preferred_encoding(s):
@@ -425,12 +425,12 @@ def human_time(seconds):
     seconds = int(seconds)
 
     if seconds < 60:
-        return '   {0:2d}s'.format(seconds)
+        return f'   {seconds:2d}s'
     for i in range(len(units) - 1):
         unit1, limit1 = units[i]
         unit2, limit2 = units[i + 1]
         if seconds >= limit1:
-            return '{0:2d}{1}{2:2d}{3}'.format(
+            return '{:2d}{}{:2d}{}'.format(
                 seconds // limit1, unit1,
                 (seconds % limit1) // limit2, unit2)
     return '  ~inf'
@@ -482,10 +482,10 @@ def human_file_size(size):
         str_value = str_value[:2]
     else:
         str_value = str_value[:3]
-    return "{0:>3s}{1}".format(str_value, suffix)
+    return f"{str_value:>3s}{suffix}"
 
 
-class _mapfunc(object):
+class _mapfunc:
     """
     A function wrapper to support ProgressBar.map().
     """
@@ -648,10 +648,10 @@ class ProgressBar:
         else:
             t = ((time.time() - self._start_time) * (1.0 - frac)) / frac
             prefix = ' ETA '
-        write(' {0:>4s}/{1:>4s}'.format(
+        write(' {:>4s}/{:>4s}'.format(
             human_file_size(value),
             self._human_total))
-        write(' ({:>6.2%})'.format(frac))
+        write(f' ({frac:>6.2%})')
         write(prefix)
         if t is not None:
             write(human_time(t))
@@ -685,7 +685,7 @@ class ProgressBar:
         # Calculate percent completion, and update progress bar
         frac = (value/self._total)
         self._widget.value = frac * 100
-        self._widget.description = ' ({:>6.2%})'.format(frac)
+        self._widget.description = f' ({frac:>6.2%})'
 
     def _silent_update(self, value=None):
         pass

# Figure out what the current language is set to.
return locale.getlocale(locale.LC_TIME)

class LocaleTime(object):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to omit the extern files from the update.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extern should definitely be left alone for any updates, as we try to simply copy paste their released versions as is for to have a good reference as ease of maintenance.

Comment thread astropy/constants/constant.py Outdated
# The wrapper applies to so many of the __ methods that it's easier to
# just exclude the ones it doesn't apply to
exclude = set(['__new__', '__array_finalize__', '__array_wrap__',
exclude = {'__new__', '__array_finalize__', '__array_wrap__',

@MSeifert04 MSeifert04 Jul 7, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I like these kind of cleanups I would prefer if the indentation (for the following lines) would be changed appropriately.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is a bit of a microbug / nice-to-have in pyupgrade -- I haven't gotten around to fixing it because I use autopep8 / this calling construct which avoids 14-space / arbitrary-space indents

@taldcroft

Copy link
Copy Markdown
Member Author

I re-did this from scratch using a hacked version of pyupgrade at e477316 to convert format and % strings to modern formatting, including f-strings where possible. The hack bit was to essentially de-select all upgrades except for string formatting ones. That removes (at least from initial quick inspection) all the issues with indentation, and also makes the scope of this PR more contained.

@taldcroft

Copy link
Copy Markdown
Member Author

For the record:

diff --git a/pyupgrade.py b/pyupgrade.py
index f048475..960c966 100644
--- a/pyupgrade.py
+++ b/pyupgrade.py
@@ -411,7 +411,7 @@ def _fix_py2_compatible(contents_text):
         return contents_text
     visitor = Py2CompatibleVisitor()
     visitor.visit(ast_obj)
-    if not any((
+    if True or not any((
             visitor.dicts,
             visitor.sets,
             visitor.set_empty_literals,
@@ -1515,7 +1515,7 @@ def _fix_py3_plus(contents_text):
     visitor = FindPy3Plus()
     visitor.visit(ast_obj)
 
-    if not any((
+    if True or not any((
             visitor.bases_to_remove,
             visitor.encode_calls,
             visitor.if_py2_blocks,

@bsipocz

bsipocz commented Jul 8, 2019

Copy link
Copy Markdown
Member

@taldcroft - can the tool understand the narrative docs too? if yes, could you run in on the code snippets in there, either as part of this PR or a separate one?

@taldcroft

Copy link
Copy Markdown
Member Author

@bsipocz - I'd rather leave the narrative docs as a separate PR. 255 files is already big enough, and some custom code would be needed for the narrative docs as the tool is expecting only parsable Python code.

@pllim

pllim commented Jul 9, 2019

Copy link
Copy Markdown
Member

There is already conflict. I'd say, either we get this in soon, or break it into smaller pieces by submodule.

@asottile

asottile commented Jul 9, 2019

Copy link
Copy Markdown

From experience you'll frustrate contributors less by landing it all in one PR (especially if patches tend to span multiple modules) as they'll only have to deal with conflicts once

@bsipocz

bsipocz commented Jul 10, 2019

Copy link
Copy Markdown
Member

I'm happy to hit merge once this is rebased and agree that now that it's done, we don't need to split up to multiple PRs (e.g. I'm happy to help sort out other PRs that get conflicts due to this).

@taldcroft

Copy link
Copy Markdown
Member Author

OK, this is now re-processed using current master.

@bsipocz bsipocz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've skimmed through the diff again, and it mostly looks OK. Once the two things I suggest are removed, I'm happy to merge.

Comment thread astropy/extern/_strptime.py Outdated
return ''
regex = '|'.join(re_escape(stuff) for stuff in to_convert)
regex = '(?P<%s>%s' % (directive, regex)
regex = f'(?P<{directive}>{regex}'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove changes to stuff in extern

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh, made a small mistake in my grepping that let this through, will fix.

Comment thread astropy/modeling/core.py

if param.default is param.empty:
raise ModelDefinitionError(
'The bounding_box method for {0} is not correctly '

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit worried how painful these changes will be for #8769, maybe skip this file.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

@taldcroft

Copy link
Copy Markdown
Member Author

Once more... and in the interest of reproducibility here is what I typed (laying bare my unsophisticated approach):

cd ..
find astropy/astropy -type f -name '*.py' | 
  grep -v astropy/astropy/extern/  | 
  grep -v astropy/astropy/modeling/core.py | 
  xargs python ~/git/pyupgrade/pyupgrade.py --py36-plus

@bsipocz bsipocz added the zzz 💤 merge-when-ci-passes Do not use: We have auto-merge option now. label Jul 11, 2019
@bsipocz

bsipocz commented Jul 11, 2019

Copy link
Copy Markdown
Member

OK, well, I take the blame if this messes something up due to the merging, but based on the 5 rockets, I go ahead and merge now.

@bsipocz bsipocz merged commit 3274189 into astropy:master Jul 11, 2019
@dhomeier

dhomeier commented Aug 9, 2019

Copy link
Copy Markdown
Contributor

@taldcroft - should the recommendation in codeguide.rst be updated accordingly to put

for tea="spam" f"{tea:s}"

in first place?

@taldcroft taldcroft deleted the f-strings branch October 1, 2019 20:45
astrofrog pushed a commit to astropy/pytest-astropy-header that referenced this pull request Oct 7, 2019
Convert format and % strings to  f-strings
@pllim pllim mentioned this pull request Oct 19, 2020
@pllim pllim mentioned this pull request Oct 29, 2020
ntessore pushed a commit to glass-dev/cosmology that referenced this pull request Dec 7, 2020
Convert format and % strings to  f-strings
ntessore pushed a commit to glass-dev/cosmology that referenced this pull request Dec 8, 2020
Convert format and % strings to  f-strings
@saimn saimn mentioned this pull request Jul 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants