Skip to content

[core] Use PicoCli and unify PMD usage under a single main#4059

Merged
oowekyala merged 128 commits into
pmd:pmd/7.0.xfrom
jsotuyod:picocli
Nov 23, 2022
Merged

[core] Use PicoCli and unify PMD usage under a single main#4059
oowekyala merged 128 commits into
pmd:pmd/7.0.xfrom
jsotuyod:picocli

Conversation

@jsotuyod

@jsotuyod jsotuyod commented Jul 22, 2022

Copy link
Copy Markdown
Member

Describe the PR

Migrating away from JCommander. Use PicoCli to provide a unified PMD CLI usage a-la-git. Make the most of PicoCli by shipping bash / zsh autocompletion support with the PMD distribution.

Take this chance to homogenize usage across all utilities. These changes need to be reflected on PMD 6 through a separate PR deprecating options /adding the new aliases to allow people to accommodate.

  • CPD no longer uses --filelist in favor of --file-list (as PMD already did)
  • CPD no longer uses --failOnViolation nor -failOnViolation in favor of --fail-on-violation (as PMD already did)
  • CPD no longer uses --files in favor of --dir / -d (as PMD already did)
  • CPD now also allows the short flag -e as equivalent to --encoding (as PMD and ast-dump already did)
  • CPD and PMD's --fail-on-violation is now negatable. The user no longer has to provide a boolean value, but can directly state --fail-on-violation / --no-fail-on-violation to set the intended behavior. The default keeps being --fail-on-violation.
  • PMD's --no-progress is now negatable, so the user can set either --progress / --no-progress to set the desired behavior. The default keeps being --progress.
  • CPD, PMD, ast-dump and the designer can enable verbose mode through any of the flags --verbose, -v, --debug or -D. The usage of -V is shifted to be an equivalent of --version (which is consistent with most Linux commands).
  • We swapped "cpdgui" with "cpd-gui" for consistency with "ast-dump" and to further differentiate both.
  • All commands now follow a single guideline for exit codes:
    • 0 for successful execution with no violations found.
    • 1 for executions that failed with an exception.
    • 2 for CLI usage errors.
    • 4 for successful executions with 1 or more violations found (as long as --no-fail-on-violation is not set). Only applicable for PMD / CPD tasks.

Additionally:

  • PMD's stress flags are removed, the functionality had long been lost and had no real value to users.

TODOs

Related issues

Ready?

  • Added unit tests for fixed bug/feature
  • Passing all unit tests
  • Complete build ./mvnw clean verify passes (checked automatically by github actions)
  • Added (in-code) documentation (if needed)

@jsotuyod jsotuyod added an:enhancement An improvement on existing features / rules in:cli Affects the PMD Command Line Interface labels Jul 22, 2022
@jsotuyod jsotuyod added this to the 7.0.0 milestone Jul 22, 2022
Comment thread pmd-core/src/main/java/net/sourceforge/pmd/cli/internal/commands/PMDCommand.java Outdated
Comment thread pmd-core/pom.xml Outdated
Comment thread pmd-core/src/main/java/net/sourceforge/pmd/cli/internal/ExecutionResult.java Outdated
Comment thread pmd-cli/src/main/java/net/sourceforge/pmd/cli/PMDCLI.java Outdated

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oowekyala I'm considering removing this from the configuration and PmdAnalysis, as it's exclusively intended for CLI usage… however, since instantiating the listener requires to know the final number of files to analyze, this requires some additional changes in PmdAnalysis. Therefore, I'll refrain from doing it in this PR and leave it for a follow up. WDTY?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that this should be refactored. Maybe we should have an own "ProgressListener" interface in pmd-core which the cli module could implement?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look into that after this PR is closed.

Comment thread pmd-core/src/main/java/net/sourceforge/pmd/cpd/CPD.java Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we need a pmd-cli-integration-tests module to test with actual language modules? I would prefer pmd-cli not to depend on language modules, even if it's only a test-scoped dependency. Otherwise pmd-dist could contain those tests I guess

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we were to do this, I'd do it in pmd-dist (as the same would apply to Ant).

However, I don't think this is particularly pressing… the way both CLI and Ant are being setup with separate modules, they are simply interfaces into PMDConfiguration / PmdAnalysis. These modules can be sufficiently tested on their own with the Dummy Languages. They could even be put into independent repositories without issue (like we do for the Eclipse Plugin as another "interface into PMDConfiguration / PmdAnalysis").

The only real integration point for language modules is ensuring the LanguageRegistry is properly being able to detect it (that is, that the service definitions are properly setup), and that can be tested from the language modules themselves, as it only requires accessing pmd-core (and this is something we should start doing…).

The current usages for this BaseCLITest as it stands are:

  • issues with pmd-core itself, irrespective of CLI / Ant, or even the specific language (ie: these tests from pmd-java are actually testing an issue with pmd-core's RuleSetFactory not emitting errors.
  • testing behavior of the LanguageVersionDiscoverer irrespective of the language in use (ie: these tests in pmd-xml).
  • integration tests with very little value add that limit themselves to checking the run didn't blow up.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually have some basic tests for all languages in pmd-dist already, even for Ant (see AntIT and AllRulesIT).

For the JUnit5 migration (#3797), I skipped for now this BaseCLITest class. But maybe, we can change/move the tests, that we don't actually need separate CLI tests in each module, like @jsotuyod proposes. And we can delete this base test class... I'll keep that in mind, when I continue on JUnit5 migration.

@jsotuyod

jsotuyod commented Aug 22, 2022

Copy link
Copy Markdown
Member Author

I'm also considering allowing positional parameters to be equivalent to using -d. That is, allowing:

pmd run -R ruleset.xml -f text -- src/main/java src/test/java

to be equivalent to:

pmd run -R ruleset.xml -f text -d src/main/java -d src/test/java

I'd do this in addition to the current usage of -d to minimize disruptions, this is merely a quality of life improvement.

I'd also like to change the usage of --fail-on-violation. Currently the default is to fail with violations, so actually changing this value requires using --fail-on-violation false… This is not very friendly. I'm considering of turning this flag into a negatable option, where the user can either use --fail-on-violation / --no-fail-on-violation to explicitly set the expected behavior without extra values. The default would remain to true.

Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread docs/pages/pmd/userdocs/cli_reference.md Outdated
Comment thread docs/pages/pmd/userdocs/cli_reference.md Outdated
Comment thread docs/pages/pmd/userdocs/cli_reference.md Outdated

@adangel adangel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jsotuyod ,
I've tested the batch script, I ran it partly under Windows.
My reference is https://ss64.com/nt/ - and you always learn some new weird behavior, e.g. variable expansion in if blocks... 😄

Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
Comment thread pmd-dist/src/main/resources/scripts/pmd.bat Outdated
@jsotuyod

Copy link
Copy Markdown
Member Author

@adangel thanks for the love with that bat script! I did learn quite a few things, and I'm happy we get to offer Windows users the same experience we do for MacOS and Linux.

@jsotuyod jsotuyod marked this pull request as ready for review September 19, 2022 20:10
@jsotuyod

Copy link
Copy Markdown
Member Author

I think this is finally ready for review 😄

@ghost

ghost commented Sep 20, 2022

Copy link
Copy Markdown
2 Messages
📖 Compared to pmd/7.0.x:
This changeset changes 23 violations,
introduces 16 new violations, 0 new errors and 0 new configuration errors,
removes 18 violations, 0 errors and 0 configuration errors.
Full report
📖 Compared to master:
This changeset changes 0 violations,
introduces 695504 new violations, 22 new errors and 0 new configuration errors,
removes 822667 violations, 17 errors and 7 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 36 violations,
introduces 27 new violations, 0 new errors and 0 new configuration errors,
removes 28 violations, 0 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 693922 new violations, 23 new errors and 0 new configuration errors,
removes 821465 violations, 39 errors and 7 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 25 violations,
introduces 25 new violations, 0 new errors and 0 new configuration errors,
removes 22 violations, 0 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 693926 new violations, 23 new errors and 0 new configuration errors,
removes 821465 violations, 39 errors and 7 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 29 violations,
introduces 25 new violations, 0 new errors and 0 new configuration errors,
removes 21 violations, 0 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 692825 new violations, 23 new errors and 0 new configuration errors,
removes 820392 violations, 39 errors and 7 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 692821 violations, 23 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 820392 violations, 39 errors and 7 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 819853 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report
Compared to pmd/7.0.x:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 699653 violations, 22 errors and 0 configuration errors.
Full report
Compared to master:
This changeset changes 0 violations,
introduces 0 new violations, 0 new errors and 0 new configuration errors,
removes 826978 violations, 39 errors and 6 configuration errors.
Full report

Generated by 🚫 Danger

Comment on lines +529 to +531
* @deprecated Use {@link #setInputPathList(List)}
*/
@Deprecated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, if we merge #4161 first into master(pmd6), then master->pmd/7.0.x and update this PR, these changes should disappear.

And then we can think about, removing the deprecated methods already (if we ourselves don't use them in pmd7 anymore).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread Gemfile.lock Outdated
@jsotuyod

jsotuyod commented Oct 17, 2022

Copy link
Copy Markdown
Member Author

I covered everything now, hopefully pmdtester will run as expected and this should be ready. We may want to wait until we merge #4161 and I'll merge back master into PMD 7 / remove deprecated methods from PMDConfiguration.

@adangel

adangel commented Oct 18, 2022

Copy link
Copy Markdown
Member

I covered everything now, hopefully pmdtester will run as expected and this should be ready. We may want to wait until we merge #4161 and I'll merge back master into PMD 7 / remove deprecated methods from PMDConfiguration.

Thanks!

Yes, pmdtester seems to be working now: #4059 (comment)
It has only the "usual" changes in LawOfDemeter (@oowekyala thinks the problem here is that the rule depends on the visiting order of nodes in the rule chain which is not stable...)

I hope to release a new pmdtester version this Thursday. And yes, I think it's better we wait until #4161 is merged into master.

 - This includes a PR I submitted to support suggestions of multi-word entries
@lmove

lmove commented Nov 18, 2022

Copy link
Copy Markdown

Hi! Our tool BreakBot found that this pull request introduces 15 breaking changes and deprecations and they appear to impact 7 of your clients. You can find the full BreakBot report in our fork repository: report for PR#4059.

We hope this information is valuable to you, and apologize otherwise. If you're willing to help, we would kindly ask for your help to fill in a 5-minutes survey about the report. Your feedback will help us improve the tool and provide a better experience for users in the future!

@adangel

adangel commented Nov 23, 2022

Copy link
Copy Markdown
Member

Hi @jsotuyod, did you see @oowekyala's merge at jsotuyod#4 ? Is it ok, if he goes ahead and merges this PR or do you think there is anything missing?

@jsotuyod

Copy link
Copy Markdown
Member Author

@adangel sorry, I've been out for the last week. It's been merged now!

@oowekyala oowekyala merged commit 2f6ec40 into pmd:pmd/7.0.x Nov 23, 2022
@oowekyala

Copy link
Copy Markdown
Member

Merged, thanks @jsotuyod!

@jsotuyod jsotuyod deleted the picocli branch November 23, 2022 21:06
oowekyala added a commit to oowekyala/pmd that referenced this pull request Nov 26, 2022
Remove some deprecated things left over by pmd#4059
@adangel adangel mentioned this pull request Jan 23, 2023
55 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

an:enhancement An improvement on existing features / rules in:cli Affects the PMD Command Line Interface

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cli] Split off CLI implementation into a pmd-cli submodule [core] Consolidate PMD CLI into a single command

4 participants