Skip to content

Exclude files or directories in cpplint.py#238

Closed
ahcorde wants to merge 18 commits intoament:masterfrom
ahcorde:ahcorde/cpplint/exclude
Closed

Exclude files or directories in cpplint.py#238
ahcorde wants to merge 18 commits intoament:masterfrom
ahcorde:ahcorde/cpplint/exclude

Conversation

@ahcorde
Copy link
Copy Markdown
Contributor

@ahcorde ahcorde commented May 11, 2020

This PR is related to this other #234

ament_cmake_cpplint includes all the files in the package, with this option we can exclude some files or directories.

Signed-off-by: ahcorde ahcorde@gmail.com

Signed-off-by: ahcorde <ahcorde@gmail.com>
@ahcorde ahcorde added the enhancement New feature or request label May 11, 2020
@ahcorde ahcorde requested a review from dirk-thomas May 11, 2020 07:54
@ahcorde ahcorde self-assigned this May 11, 2020
@ahcorde ahcorde force-pushed the ahcorde/cpplint/exclude branch from 0f7eaa1 to d00f4b6 Compare May 12, 2020 07:22
Signed-off-by: ahcorde <ahcorde@gmail.com>
@ahcorde ahcorde force-pushed the ahcorde/cpplint/exclude branch from d00f4b6 to b9f07a6 Compare May 12, 2020 07:23
Signed-off-by: ahcorde <ahcorde@gmail.com>
Signed-off-by: ahcorde <ahcorde@gmail.com>
@ahcorde ahcorde requested a review from dirk-thomas May 21, 2020 17:35
Copy link
Copy Markdown
Contributor

@dirk-thomas dirk-thomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exclude logic needs to be fixed as pointed out multiple times.

ahcorde added 3 commits June 10, 2020 13:09
Signed-off-by: ahcorde <ahcorde@gmail.com>
Signed-off-by: ahcorde <ahcorde@gmail.com>
Signed-off-by: ahcorde <ahcorde@gmail.com>
@dirk-thomas dirk-thomas dismissed their stale review June 12, 2020 19:07

The exclude logic has been fixed

@ahcorde ahcorde requested a review from dirk-thomas June 15, 2020 17:48
files_to_avoid.append(f)
files_check = list(set(files) ^ set(files_to_avoid))

if(len(files_check) != 0):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check us only necessary since the previous check from line 123 isn't effective since the exclude logic is applied afterwards. How about moving the logic into get_file_groups() - which I think would be the most efficient approach since it even avoid crawling / collecting files under an excluded directory. I would expect that would also avoid the need to check all parents.

Nitpick: no parenthesis around Python conditions. The Pythonic way to write this condition would be: if files_check.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic moved to get_file_groups() 3b9b453

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exclude logic can already be applied to the dirnames variable to filter them out early (without the need to recurse into them). That will also simplify the check later which doesn't have to consider parents.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to dirnames 3b9b453

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see where the code filters dirnames based on the excludes?

Signed-off-by: ahcorde <ahcorde@gmail.com>
@ahcorde ahcorde requested a review from dirk-thomas June 16, 2020 12:05
ahcorde added 2 commits June 17, 2020 12:25
Signed-off-by: ahcorde <ahcorde@gmail.com>
Signed-off-by: ahcorde <ahcorde@gmail.com>
Signed-off-by: ahcorde <ahcorde@gmail.com>
@ahcorde ahcorde requested a review from dirk-thomas June 23, 2020 07:43
continue
# ignore folder starting with . or _
dirnames[:] = [d for d in dirnames if d[0] not in ['.', '_']]
dirnames[:] = [d for d in dirnames if d not in excludes]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dirnames only contains the names of the subdirectories of dirpath. As such the condition not in excludes is incorrect again - for the same reasons as the previous comments.

ahcorde added 2 commits July 9, 2020 12:03
Signed-off-by: ahcorde <ahcorde@gmail.com>
Signed-off-by: ahcorde <ahcorde@gmail.com>
@ahcorde ahcorde requested a review from dirk-thomas July 10, 2020 11:06
if ext in ('.%s' % e for e in extensions):
append_file_to_group(groups,
os.path.join(dirpath, filename))
if filename not in excludes:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition still doesn't make sense.

Please write a unit test which tests a variety of cases to validate the implementation against.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

humm, why not? the excludes argument might contains a filename

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think an exclude of bar should match a file foo/bar. Instead the exclude should state foo/bar explicitly. Otherwise how do I exclude the file bar but not the file foo/bar?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should I need to include the test?

Copy link
Copy Markdown
Contributor

@dirk-thomas dirk-thomas Jul 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't have to include the test in this PR but as mentioned before writing one will help you to check the logic. Especially the part to consider an example file system structure and excludes and what the filtered result is expected to be.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dirk-thomas I created a gist with a test: https://gist.github.com/ahcorde/bcc558aedad8560cfa46c4b5de366f32

This is the last commit 6db3ae3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how to use the gist. get_file_groups() seems to always return an empty dict?

Just from reading over the logic I would expect that a file include/test/cache_unittest.cpp would be incorrectly skipped even though only test/cache_unittest.cpp is specified in the excludes.

Also I still don't see why is_file_or_directory_to_exclude() should needs any logic like check_path_parents().

Signed-off-by: ahcorde <ahcorde@gmail.com>
@JWhitleyWork
Copy link
Copy Markdown

@ahcorde Any update on this?

@mm318
Copy link
Copy Markdown
Member

mm318 commented Feb 14, 2021

I agree with Dirk on #238 (comment)

I don't think an exclude of bar should match a file foo/bar. Instead the exclude should state foo/bar explicitly. Otherwise how do I exclude the file bar but not the file foo/bar?

If the desire is to match bar at any level, then the expression should be something like **/bar (basically glob rules). Python already has a glob library as part of its standard libraries (https://docs.python.org/2.7/library/glob.html), we can just leverage that.

@mm318
Copy link
Copy Markdown
Member

mm318 commented Feb 14, 2021

The question now is what's the behavior already built into ament_cppcheck at

cmd.extend(['--suppress=*:', exclude])

as matching that would be more important than implementing glob rules.

EDIT: The current behavior of ament_cppcheck --exclude is wrong (see #234 (comment)), so ament_cpplint should not mimic its behavior.

@audrow
Copy link
Copy Markdown
Contributor

audrow commented May 27, 2021

@ahcorde, what do you think we should do here? Should we close this PR?

@ahcorde
Copy link
Copy Markdown
Contributor Author

ahcorde commented May 27, 2021

yes, we can close this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants