Skip to content

Excluded files impact the performance of swiftlint #5018

@sebmarchand

Description

@sebmarchand

New Issue Checklist

Describe the bug

The number of files present in the working tree of swiftlint has a huge impact on its performance even if most of these files are ignored. Here's a concrete example/repro:

  • Create a temporary directory with just one file to lint:
mkdir /tmp/swiftlint_tests && cd /tmp/swiftlint_tests
touch test.swift
  • Use hyperfine to measure the execution time of swiftlint:
$ hyperfine --warmup 1 'swiftlint'
➜  swiftlint_tests hyperfine --warmup 1 'swiftlint'
  Time (mean ± σ):      61.4 ms ±   0.9 ms    [User: 52.2 ms, System: 7.1 ms]
  Range (min … max):    59.9 ms …  65.4 ms    46 runs
  • Create a subdirectory with a lot of swift files in it, create a linter config to exclude these files:
mkdir .build
for i in {1..10000}
do
  touch .build/$i.swift
done

echo "excluded:\n  - '**/.build'" > .swiftlint.yml
  • Make sure that these ignored files are actually ignored:
$ swiftlint
Linting Swift files in current working directory
Linting 'test.swift' (1/1)
Done linting! Found 0 violations, 0 serious in 1 file.
  • Benchmark swiftlint again:
➜  swiftlint_tests hyperfine --warmup 1 'swiftlint'
  Time (mean ± σ):     378.7 ms ±   6.1 ms    [User: 229.2 ms, System: 377.6 ms]
  Range (min … max):   368.2 ms … 389.6 ms    10 runs

The execution time is 6x what it is when these ignored files don't exist. Using the --use-alternative-excluding flag brings this to 5x but it'd still be nice to completely ignore these file. Not using a glob pattern in the config improves things by another 1x.

It looks like the code is traversing the entire file tree in a few places (e.g. in the glob implementation), we could maybe make it use a recursive approach and stop when it encounters a subdirectory that is ignored (e.g. in my example it'd not look at the content of the .build directory at all).

Environment

  • SwiftLint version (run swiftlint version to be sure)?
    From source

  • Installation method used (Homebrew, CocoaPods, building from source, etc)?
    Source

  • Paste your configuration file:
    See example

  • Are you using nested configurations?
    No

  • Which Xcode version are you using (check xcodebuild -version)?

$ xcodebuild -version
Xcode 14.3
Build version 14E222b

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementIdeas for improvements of existing features and rules.integrationIssues related to integration of SwiftLint into toolchains.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions