ARM64 docker images by bdovaz · Pull Request #1553 · oxsecurity/megalinter

bdovaz · 2022-06-23T15:16:08Z

Edit: 05/01/2022

I have been working a bit on this PR and thinking....

I understand that sooner or later it will get to the point where all linters are amd64/arm64 compatible either because those linters start to be arm64 compatible or because development of them has been abandoned and they end up being removed from Megalinter.

But as of today as we know, this is not the case and we don't know how long it will take.....

Support of runners on arm64

Windows: actions/runner-images#6175

Linux: actions/runner-images#5631

macOS: github/roadmap#528

As you can see, the "closest" is macOS for Q3 of 2023. For the rest there is no ETA.

Change from docker build to docker buildx

A first step in cross-platform image generation.

Already merged in #2199 and working for amd64.

Supported platform descriptors configuration

With the help of @echoix is added on this PR and is that for each descriptor now allows to configure the supported platforms:

supported_platforms:
  platform:
    - linux/amd64
    - linux/arm64

In some linters it is set to more platforms besides amd64 and arm64 because linter itself supports it but the reality is that initially at least we should focus on arm64 only.

This opens the door to:

Being able to document which descriptors work on which platforms.
Create a Dockerfile (or several) based on those values.

Point 2 leads us to...

Decide how to generate the Dockerfile(s).

There are 2 options that at least I know of:

Enter conditions in the RUN command.
- To do this we would have to do something like this: https://stackoverflow.com/a/60518444/699056. It means that whoever makes a change in a descriptor that supports the 2 platforms has to take it into account when making modifications on the dockerfile section.
- There is the problem that there are dependencies that are pip,apk,.... In those it is not possible to introduce a logic as with the RUN method and it would have to be solved in another way (I guess in the build.py script?).
Generate a Dockerfile (amd64) and another one Dockerfile.aarch64 (arm64)
- The build_dockerfile function of the build.py script should be refactored.
- It loses sense the PR of docker buildx that is already merge since being different Dockerfile files, the build would be done in separate commands.

Additional blocks

~~Update to .NET 6 / 7 #1680~~

Additional blocks (external)

misspell: I couldn't get misspell to work as it doesn't target arm (apart from the fact that the repo seems abandoned, last commit march 2018): https://github.com/client9/misspell
Powershell: Please provide docker hub tags for ARM64 PowerShell/PowerShell-Docker#520 (comment)
rakudo-pkg: ARM support? nxadm/rakudo-pkg#96
Chromium doesn't work in arm so I had to set PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true

There may be more...

Testing

I have executed it in the following way (from repo root with Raspberry Pi 4 with Raspberry Pi OS 64 bits):

docker buildx build --platform linux/arm64 -f flavors/ci_light/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/cupcake/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/documentation/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/dotnet/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/go/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/java/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/javascript/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/php/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/python/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/ruby/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/rust/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/salesforce/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/security/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/swift/Dockerfile .
docker buildx build --platform linux/arm64 -f flavors/terraform/Dockerfile .

echoix · 2022-06-23T17:20:46Z

Thanks for the first draft!
What I understood from working with this project the last time is that the Dockerfiles are generated by script, so if there are changes to make in the Dockerfiles, we must change the .megalinter-descriptor.yml like megalinter/descriptors/kubernetes.megalinter-descriptor.yml. Then run the bash build.sh script and see if the Dockerfiles produced match what is expected.

Since you have found the changes needed, a part of the job is done!

With my secondary testing a couple months ago, that didn't finish nor end up with a pull request (after I passed around my raspberry pi 4 wasn't running in a 64 bit OS), I learned that it was possible with the buildx to cross build on my windows amd64 computer, which was quite faster with more cores and way more memory to build simultaneously the aarch64 and amd64. I didn't manage to compile up to the end though, and hadn't had a chance to look at the next problems (solving a problem one layer at the time).

The problem that I didn't have a good idea to fix, that remains, is how to change the .megalinter-descriptor.yml schema to account for the multitude of different conventions of getting and including tools for many architectures. Some might need to be in another format maybe (installed in amd64 vs Docker in aarch64). Sometimes it was a different url scheme, and not everyone names the arm 64 bit architecture the same way. Maybe one of the new active collaborators have some ideas architecture-wise?

nvuillam · 2022-06-24T11:33:36Z

Thanks for this POC !

That looks nice, but as Dockerfiles are not written but generated (from build.py) , such enhancement must be done with an evolution of the architecture.

I think that having a Dockerfile_arm at the root and Dockerfile_arm files in flavors could be a good way to implement that.
And to generate such ARM-specific dockerfiles, I think that adding a new property install_arm in YML descriptor would be the way, and when generating dockerfile:

If in the linter_descriptor there is an install_arm property, use it
else use install property

bdovaz · 2022-06-24T15:00:17Z

@nvuillam I've been analyzing it and from what I understand we have to:

megalinter/descriptors/*: add the "install_arm" field where it differs from "normal" and add whatever changes.
megalinter/descriptors/schemas/megalinter-descriptor.jsonschema.json: add the "install_arm" field.
.automation/build.py: somehow we have to make that script generate the Dockerfile and Dockerfile_arm files (root and flavors) in the same execution no? And in the "normal" case ignore the "install_arm" field but in the case of ARM we will have to search if the "install_arm" field exists and if not use the default "install" field.

echoix · 2022-06-24T16:01:38Z

For

megalinter/descriptors/*: add the "install_arm" field where it differs from "normal" and add whatever changes.
and

And in the "normal" case ignore the "install_arm" field but in the case of ARM we will have to search if the "install_arm" field exists and if not use the default "install" field.

I think we need to account that not every tool can or could be supported on first try, so if we could have a way to "check" (like in a checkbox) which tools to include in this arm flavor. This way, a compatibility table could be created showing which tools are ported.

nvuillam · 2022-06-24T16:06:48Z

@echoix we could add a property arm-compliant on linter descriptors :)

echoix · 2022-06-24T16:14:39Z

@nvuillam Shouldn't we be not short-sighted and inverse the logic a bit, to say we comply with these platforms, instead of considering a dichotomy amd64/aarch64(arm64)? Especially in docker architectures, I'm pretty sure some tools (flavors) like python-only could use armv7l, which is still the default Raspbian OS (most Raspberry Pis can run in 64 bit mode, but when there was no >4Gb option, there wasn't any advantage to use 64bit).

echoix · 2022-06-24T16:15:29Z

@bdovaz I could be there for help with this PR if you need :)

- Temporarily disable validators

bdovaz · 2022-06-24T16:22:43Z

@echoix I'll appreciate that.

Specifically:

Adjust the descriptor schema to add the 2 fields because I have no experience creating schemas:
- arm_compliant: I would make it optional and set it to true by default because there is only one descriptor that doesn't work (dot-env).
- install_arm: Optional also with a fallback to install if it doesn't exist.
Make the settings of the "build.py" file because I have tried but I don't understand well how the Dockerfiles generation system works and which part is automatic and which is manual.
- I have disabled validation temporarily until the json schema changes are done.
- I have tried to create the Dockerfile_arm file of each flavor "in theory" so that it is generated as the original but I have not managed to generate more than the header.

I also see that the main Dockerfile does not use descriptors no? Because just running the original build.py mashes the changes I made and changes are lost.

echoix · 2022-06-24T17:16:13Z

For sure @bdovaz, I'm looking into that this afternoon :)

Kurt-von-Laven · 2022-06-24T17:26:12Z

I agree with @echoix that it may be better long-term to have a supported_architectures (or arches) list. I wonder how many of the linters we use have an asdf plugin with multi-architecture support. I know many of the tools we use have an asdf plugin (or, more often, a package manager with an asdf plugin).

echoix · 2022-06-24T17:28:59Z

Could I have a little recap on asdf, or how it is used or planned to be used in Megalinter? I receive most of the issues of this repo by email, but I still don't understand it in relation to this project.

I'm glad you think the same way as I thought, as I have an idea of how Megalinter could make use of buildx and multi-arch build efficiently if we know how which platform each tool supports.

Kurt-von-Laven · 2022-06-25T09:19:45Z

buildx does seem like a good way to go with the caveat that I always optimistically hope there is some approach that involves less work ha ha ha. Is this the same multiarch you are referring to? #870 contains some of my thoughts on asdf's relevance to MegaLinter. asdf is capable of installing all of these tools. It is basically a version manager manager, but it can manage versions of most anything. asdf has expressed interest in supporting multiple architectures in the core if necessary (c.f., asdf-vm/asdf#62), but I don't know which of the plugins presently do. One of the challenges MegaLinter inherently faces is managing many dependencies from many different package ecosystems plus some with bespoke installation methods. It will simplify maintenance if we can eventually install everything that has an asdf plugin (or, better yet, a package manager that has an asdf plugin) through asdf, but the first step would be to simply install one package manager this way. I suppose one way of marrying the ideas would be to use your buildx approach to improve or create asdf plugins since then MegaLinter could leverage that uniformity to install everything with as much multi-architecture support as the plugin (and underlying tool) allows. This might also be a good way to get more hands on deck since the asdf community is presently much larger than MegaLinter's, and I assume many people who don't use MegaLinter share your woes getting things working on various architectures. If there is some tool other than asdf that is already better suited to your goal, I imagine someone there will educate us, particularly since it brings together a diverse cross-section of people using a huge variety of toolchains.

echoix · 2022-06-25T11:08:47Z

Is this the same multiarch you are referring to?

Nope, the multi arch in Docker, where there an image's manifest contains different images for many cpu architectures. https://docs.docker.com/desktop/multi-arch/

bdovaz · 2022-06-27T16:33:25Z

For sure @bdovaz, I'm looking into that this afternoon :)

Have you finally started working on it?

Likewise you would have to clearly define how you want to make the changes.

As far as I have understood it would be to put a "supported_architectures" to be able to distinguish the installation commands and that for the future it is better designed.

My doubt is at what level would it be? Below "install", above "install"? I also have doubts with the name "amd64" and "arm64", opinions?

Example 1:

linters:
  - linter_name: ansible-lint
    install:
      supported_architectures:
        amd64:
          pip:
            - ansible-lint==6.0.2
        arm64:
          pip:
            - ansible-lint==6.0.2

Example 2:

linters:
  - linter_name: ansible-lint
    supported_architectures:
      install:
          amd64:
            pip:
              - ansible-lint==6.0.2
          arm64:
            pip:
              - ansible-lint==6.0.2

echoix · 2022-07-03T13:14:01Z

(Sorry, I started this message on Monday but never had the opportunity to finish it)

For the name of the architecture used, it's sure that we will be using the names used by Docker, it greatly simplifies writing multi-architecture Dockerfiles.
Ideally, the same Dockerfile can be used for multiple architectures. That way, buildx build --target can be used directly for all architectures (like many multi-arch image https://docs.docker.com/desktop/multi-arch/)

I drafted out scenarios (without committing anything yet), but couldn't find a way that would be a breaking change. The most common problem is when there is a URL that doesn't use the same architecture names.

One of the scenarios that wouldn't be as breaking, would be that we keep the install property with their pip, docker, apk, etc children fields, but add some architecture-specific overrides.
So in this case, supported_architectures, could have the architectures, and any overrides in the same format as install.
One simple override I think could be used to make architecture-independent Dockerfiles is having a mapping of the architecture name, for mapping the shellcheck architecture URL that uses aarch64. Otherwise, we could always use inline shell case/esac construct, in the original RUN command.

A scenario that would be breaking, is explicitly having all instructions for each platform, but doesn't help reuse layer caching.

In the same line of thought, common multi-arch Dockerfile patterns use multi-stage Dockerfiles to be able to execute some build in the builder with one architecture, and for the final layer(s), have them in the target instruction set, so this speeds up building.

…tall overrides.

from https://github.com/docker-library/bashbrew/blob/v0.1.2/architecture/oci-platform.go#L14-L27

bdovaz · 2023-01-16T19:53:07Z

@bdovaz it seems there is a workaround -> docker/buildx#59 (comment)

@echoix can you help me to fix it?

There are several lines here where we should make that change but I don't have much knowledge of bash and I don't know how to change it so that it doesn't affect the following:

In upload-docker.sh:

https://github.com/bdovaz/megalinter/blob/feature/arm-docker-image/.automation/upload-docker.sh#L319

In several workflows like this:

https://github.com/bdovaz/megalinter/blob/feature/arm-docker-image/.github/workflows/auto-update-linters.yml#L61

bdovaz · 2023-01-16T20:00:55Z

@echoix @nvuillam and about the merge issue, if one of the 2 of you is able to do it, the better because my knowledge of git operations/commands is not very advanced either.

echoix · 2023-01-16T20:26:09Z

There are several lines here where we should make that change but I don't have much knowledge of bash and I don't know how to change it so that it doesn't affect the following:

In upload-docker.sh:

https://github.com/bdovaz/megalinter/blob/feature/arm-docker-image/.automation/upload-docker.sh#L319

The 2>&1 is a output redirection that sends the output to somewhere. Program output can be stdout and stderr. It's possible to redirect and output to a file. 2 is a file descriptor for stderr. 1 is a file descriptor for stdout. It's also possible to redirect an output to another stream, instead of a file. And that's what is here. Stderr is redirected to stdout. https://stackoverflow.com/a/818265

I was thinking about this, one day we could let go of bash specific syntax. Is there still a use case where the building of the images of megalinter isn't done on GitHub actions? If we aren't able to understand it well, well, we aren't maintaining it.

Maybe a quick search to see if the github action logs already include both outputs.
However, the important shell syntax is what follows, https://github.com/bdovaz/megalinter/blob/ea67c7777ed2e0e4e7f8642b0c6d29e0c9936017/.automation/upload-docker.sh#L324

The $? is a special variable that is equal to the exit status of the last executed command. I don't really like relying on bashisms. If someone doesn't know about it, it's really hard to follow and debug.
https://stackoverflow.com/a/7248048

In several workflows like this:

https://github.com/bdovaz/megalinter/blob/feature/arm-docker-image/.github/workflows/auto-update-linters.yml#L61

Here I'm not sure what construct you want to know, is it the pipe character (|)?

Here, that line calls the command normally, and at the end, the pipe sends the stream to something else as an argument, (the read line thing), which is a loop that takes each line outputted ($line being an argument), and instead runs echo, to print out a message to the console, and that message is the current date and time with that format, a pipe character (it's inside a string, it's not the operator), and adds the contents of the variable $line. In short, it prepends a time stamp to each line output. Since GitHub action logs can show the time stamps (maybe it wasn't possible before), the rest of the line after load . can either be kept, or simply removed without impact.

bdovaz · 2023-01-16T20:29:57Z

@echoix are you clear about the change to make without breaking the commands below in the 2 cases I have put? In case you want to make help me with a PR in my fork or if you can tell me what to change it to exactly.

Thanks!

From this change could already be done the merge issue that you were talking about in which as I said, I prefer that you give me a hand because of my limited knowledge in git.

echoix · 2023-01-16T20:35:45Z

For this line: https://github.com/bdovaz/megalinter/blob/ea67c7777ed2e0e4e7f8642b0c6d29e0c9936017/.github/workflows/auto-update-linters.yml#L61
It's non breaking to change to

        run: docker buildx build --platform linux/amd64,linux/arm64 --build-arg "BUILD_DATE=${BUILD_DATE}" --build-arg "BUILD_REVISION=auto_update_${GITHUB_SHA}" --build-arg "BUILD_VERSION=auto_update_${GITHUB_SHA}" --no-cache -t oxsecurity/megalinter:auto_update_${GITHUB_SHA} --load .

That is, removing | while read line ; do echo "$(date +'%H:%M:%S')| $line"; done; , as it is purely aesthetic.

The other one, I didn't really specify a change. It's answering what it is. I don't like it, but it might be deep for a first shot.

echoix · 2023-01-16T20:51:28Z

@bdovaz

From this change could already be done the merge issue that you were talking about in which as I said, I prefer that you give me a hand because of my limited knowledge in git.

Oh I think I understand. You mean the suggestion to target a branch inside the oxsecurity/megalinter repo, and continue working on it there?

I'll try to take a look this evening, it's still a workday in my timezone and gotta go back ;)

nvuillam · 2023-01-16T22:34:33Z

Something i thought about (maybe for a later PR... or this one ? ) is to get rid of our code building docker, as it seems that there is a github action from docker that does that very well using buildx, and the neighbours of Super-Linter are using it

https://github.com/github/super-linter/blob/bb170af75d406d562ae56771946053adeed41ca6/.github/workflows/cd.yml#L59

https://github.com/docker/build-push-action

nvuillam · 2023-01-16T22:36:15Z

Another thing that they do well is that when releasing, they don't build again from scratch like we do, but they take the beta images and retag them

echoix · 2023-01-16T22:40:27Z

Same thing for Login Docker and Setup QEMU. We can have multiple login docker for multiple registries, and the post action to logout with chain themselves. I tried it yesterday on another repo:
https://github.com/echoix/editorconfig-checker/blob/66b890c81c54c282068e5b1591049cb68eeb75ad/.github/workflows/release.yml#L29-L45

      - # Add support for more platforms with QEMU (optional)
        # https://github.com/docker/setup-qemu-action
        name: Set up QEMU
        uses: docker/setup-qemu-action@v2
      - name: Docker Setup Buildx
        uses: docker/setup-buildx-action@v2
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Login to GitHub Container Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

Kurt-von-Laven · 2023-01-17T01:19:19Z

Maybe a quick search to see if the github action logs already include both outputs.

They do, yes, but maybe the intention of redirecting stderr to stdout was intended to guarantee stable output order (c.f., isaacs/github#1981) or something else?

I don't really like relying on bashisms. If someone doesn't know about it, it's really hard to follow and debug.

So true, and Bashisms are also unusually difficult to Google.

Something i thought about (maybe for a later PR... or this one ? ) is to get rid of our code building docker, as it seems that there is a github action from docker that does that very well using buildx, and the neighbours of Super-Linter are using it
https://github.com/docker/build-push-action

Good point! Yes, using build-push-action is the best practice I'm aware of here.

echoix · 2023-01-17T01:53:06Z

@bdovaz I've created that dev branch at https://github.com/oxsecurity/megalinter/tree/dev/multi-platform-images. I let you create a new Draft pull request from that branch to continue discussions there, and to have CI there, and it won't be a fork.
We will announce in the PR the location of the new discussion once done.

Relates to #1553

…mpatible linters (#2273) * Build flavors workflow * Add arm_compliant field to env descriptor * Empty Dockerfile_arm * build.py WIP - Temporarily disable validators * Add linter supported_architectures with CPU architecture-specific install overrides. * Apply formatting of json schema * Use proper Docker architectures names from https://github.com/docker-library/bashbrew/blob/v0.1.2/architecture/oci-platform.go#L14-L27 * Change to supported_platforms to match the built-in build variable TARGETPLATFORM * Add riscv into cspell ignored words Found in megalinter-descriptor.jsonschema.json in supported_platforms * Remove changes * WIP * WIP * Update megalinter-descriptor.jsonschema for supported platforms at the descriptor level. * Update examples to remove parent property from being included * Add supported_platforms to bash descriptor Relates to megalinter/megalinter#1553 * First iteration on multi-platform support in descriptors * Add cspell exclusions for TARGETPLATFORM, aarch64, and armv6 * Update jsonschema * Add TARGETPLATFORM arg to shellcheck * Add conditions to kubeval and misspell * Powershell cross platform support * Revert build.py changes * Add linux/arm64 target * Add arm64 arch where possible to descriptors * Fix linters * [MegaLinter] Apply linters fixes * qemu fixes * qemu fixes * Fix buildx command * Remove --platform from run * Split Run Test Cases step by platform * Fix linters * Add supported platforms to all_linters.md * Run update.sh --doc * Run build.sh --doc * Add additional platforms * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * Run build.sh --doc * [MegaLinter] Apply linters fixes * Squashed commit of the following: commit bd884e7d0e08b286f532877b8a8194aa3c30797b Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 17 21:24:41 2023 +0200 [automation] Auto-update linters version, help and documentation (#2817) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 06cbbbe9f671673b2ae444221973461314fe586d Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jul 16 23:09:36 2023 +0200 Stealth project enhancements & fixes (#2816) * Output snippet * Use cyclonedx format by default for trivy SBOM * logger * No flavor suggestion * sbom * SBOM output in SARIF for Trivy + sbom property * fix * [MegaLinter] Apply linters fixes * Catch issue when sarif-fmt nnot reachable * is_sbom management * [MegaLinter] Apply linters fixes * Media + geekflare article * Add outputJson when possible * json output * linter fixes * Generate Worker docker images * Fix Dockerfile-worker * Beta & release jobs for workers * test case --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 9661b3b82467ec353058a596c627f53356b61758 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jul 15 21:35:55 2023 +0200 [automation] Auto-update linters version, help and documentation (#2814) commit 7b8041bc4b6f3297610ff5b83f74aa8e0e6fbfe7 Author: nvuillam <nicolas.vuillamy@ox.security> Date: Sat Jul 15 16:14:14 2023 +0200 Fix json schema commit d533c775706cb115213b83cfedfa9aef1d45a3f1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jul 15 16:08:01 2023 +0200 [automation] Auto-update linters version, help and documentation (#2813) * [automation] Auto-update linters version, help and documentation * Disable trufflehog for internal linters --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> commit d491b4e1537d870c4226e79983caba007017d929 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jul 15 16:05:25 2023 +0200 Add trufflehog secret scanner (#2808) * Fix grype issue * Trufflehog * Truffle infos * [automation] Auto-update linters version, help and documentation * Install trufflehog with docker image * cspell * Downgrade grype * mega-liter-runner ml config * SARIF not available for truffleehog * fix test cases * [MegaLinter] Apply linters fixes * cspell * cli lint mode = project * [MegaLinter] Apply linters fixes * Trufflehog arguments * cli help arg name * trufflehog test case * trufflehog args * build --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit f84e76f30c3e764d6ff671d8e20bb6490b7c3fe1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jul 15 11:19:12 2023 +0200 Generate SBOM from Trivy (#2810) * Fix grype issue * [automation] Auto-update linters version, help and documentation * Downgrade grype * mega-liter-runner ml config * fix test cases * [MegaLinter] Apply linters fixes * trivy sbom * Trivy SBOM * trivy sbom call * trivy SBOM test classes * sbom cyclonedx * Fix call to trivy sbom * JSON output * jscpd * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 2e245cd8a1ce384e23cabc5e24c376539cb1abb1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jul 15 02:07:11 2023 +0200 [automation] Auto-update linters version, help and documentation (#2812) commit 07a6dc3f20a0856845bcd0b9652d9865d7410079 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri Jul 14 20:18:53 2023 +0200 [automation] Auto-update linters version, help and documentation (#2811) commit c046d818e680ded3db21c37f397d201390e576eb Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri Jul 14 13:28:06 2023 +0200 Fix grype issue (#2807) * Fix grype issue * [automation] Auto-update linters version, help and documentation * Downgrade grype * mega-liter-runner ml config * fix test cases * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit ef6107fd3c5fe9fa607c54c3c6da18423f1999cd Author: Sanson Thomas <93382375+ThomasSanson@users.noreply.github.com> Date: Thu Jul 13 23:38:12 2023 +0200 ⬇️ fix(requirements.txt): downgrade pymdown-extensions from 10.0 to 9.11 (#2805) The pymdown-extensions package was downgraded to version 9.11 due to compatibility issues with other packages in the project. The newer version was causing conflicts, hence the need for the downgrade. commit 9088729aa270140baaf3222207a32d5f67aa110f Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Thu Jul 13 09:21:54 2023 +0200 Add grype in all flavors (#2801) commit b7d2238dc686ced7c9ce6d22e567f480bd290693 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Thu Jul 13 07:10:53 2023 +0200 [automation] Auto-update linters version, help and documentation (#2800) commit 6b97d3b5ec76f444d1eeec7da8cd41cb88ded70e Author: Sanson Thomas <93382375+ThomasSanson@users.noreply.github.com> Date: Wed Jul 12 22:33:32 2023 +0200 Fix: Gitpod issues #2792 (#2793) * 🔧 chore(Dockerfile): update NPM_VERSION from 9.7.1 to 9.8.0 The NPM_VERSION variable in the Dockerfile has been updated from 9.7.1 to 9.8.0. This update ensures that the latest version of NPM is used in the Gitpod environment. * 📦 feat(requirements.txt): add redis package for Python development environment The redis package is added to the requirements.txt file to include it as a dependency for the Python development environment. This package is necessary for interacting with Redis, a popular in-memory data store, in the application. * 🔧 chore(requirements.txt): reorder python dependencies in alphabetical order The python dependencies in the requirements.txt file have been reordered in alphabetical order. This change improves readability and makes it easier to locate specific dependencies, which is particularly useful as the list grows. * 🔧 chore(.gitpod.yml): update init task to activate virtual environment after bootstrap The init task in the .gitpod.yml file has been updated to activate the Python virtual environment after the bootstrap process. This change ensures that the correct Python environment is used for subsequent tasks, improving the development setup process in Gitpod. commit 491796d3d301c86892bcc4ffc9db91b5e2e5f397 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed Jul 12 22:30:45 2023 +0200 Add Grype security linter (#2788) * Add Grype security linter * build * Default .grype.yaml + test cases * cli_version_arg_name * Grype test cases & config * cspell * trvy commit a394c6812689813cb66ff4f175f094e9d6883422 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Tue Jul 11 23:28:18 2023 +0200 Stealth updates (#2790) * Increase worker timeout * Remove linters not in flavor before calling reporters * Update dev dockerfile commit b5cf83cbc69c6c11d211d2b80f8e21ef2b7003d3 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Tue Jul 11 21:54:01 2023 +0200 Update dotnetweb to add coffee & react linters (#2787) * Update dotnetweb to add coffee & react linters * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit f6c47c9dabdc05419a6f25848189dde0471ee366 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Tue Jul 11 21:52:58 2023 +0200 [automation] Auto-update linters version, help and documentation (#2789) commit 9c15441a1010ff6205dd5bd73c6462ed12c2112f Author: Peter Dave Hello <hsu@peterdavehello.org> Date: Wed Jul 12 01:27:30 2023 +0800 Remove additional `--update` for apk in Dockerfile (#2619) There is no need to use `--update` with `--no-cache` when using `apk` on Alpine Linux, as using `--no-cache` will fetch the index every time and leave no local cache, so the index will always be the latest without temporary files remain in the image. commit f7df5cf80f6e5b941e009c88f60c2a03259f1537 Author: Sanson Thomas <93382375+ThomasSanson@users.noreply.github.com> Date: Tue Jul 11 19:24:21 2023 +0200 Improvements to Gitpod workspace and addition of Makefile for automation (#2737) * 🆕 chore(Makefile): add Makefile to project The Makefile was added to the project. It includes targets for testing, virtualenv management, and cleaning. The Makefile is intended to simplify the development process by providing a set of commands that can be run to perform common tasks. * 🎉 feat(config.mak): add global Makefile configuration Added a global Makefile configuration file that sets the default goal to 'help', exports all variables, sets the number of cores to use, sets the shell to bash, and sets the shell flags to exit on error, pipefail, and silent. Additionally, added color variables to improve the readability of the output. * 🆕 feat(help.mak): add help command to Makefile The help command is added to the Makefile to provide a list of available commands and their descriptions. The command uses grep and awk to parse the Makefile and display the commands in a formatted way. * 🔍 chore(.gitignore): add .venv to ignore list This commit adds .venv to the ignore list to prevent it from being tracked by git. This is important because virtual environments should not be committed to the repository as they can be easily recreated by running the requirements.txt file. * 🆕 chore(.gitpod.yml): add Gitpod configuration file This commit adds a new file `.gitpod.yml` which contains the configuration for Gitpod. The configuration specifies the Dockerfile to use for the workspace image and a task to run on workspace initialization. The task runs the `make bootstrap-dev` command to set up the development environment. * 🔥 chore(requirements): remove requirements.dev.txt file The requirements.dev.txt file was removed, likely because it was no longer needed or was replaced by a different file. * 🆕 chore(python_version.config): add configuration file for Python version A new configuration file has been added to the project to specify the version of Python to be used. The file is located at `.config/python_version.config` and contains a single line specifying the Python version as `3.10`. * 🎉 feat(Dockerfile): add Dockerfile for Gitpod workspace This commit adds a Dockerfile for a Gitpod workspace. The Dockerfile installs build-time dependencies, Docker, Node.js, npm, and ungit. It also creates a gitpod user with UID 33333. The Dockerfile is split into multiple stages to optimize the build process and reduce the image size. * 🆕 chore(gitpod.mak): add Gitpod make targets for building, linting, and testing This commit adds three new make targets for Gitpod: gitpod-build, gitpod-lint, and gitpod-tests. The gitpod-build target runs a docker build command to build a local image of the postgresql_cluster. The gitpod-lint target runs a hadolint command to lint the Dockerfile. The gitpod-tests target runs the gitpod-build and gitpod-lint targets. These targets are useful for developers who want to test their changes in a Gitpod environment. * 🎉 feat(python.mak): add makefile for python virtual environment management This commit adds a makefile for managing python virtual environments. It includes targets for creating, upgrading, and installing requirements for the virtual environment. It also includes targets for installing linters and running tests. This makefile will help to ensure that the python environment is consistent across different machines and will make it easier to manage dependencies. * 🆕 chore(requirements.txt): add new development dependencies Added new development dependencies to the requirements.txt file. These dependencies include gitpython, jsonschema, jsonpickle, mkdocs, multiprocessing_logging, pychalk, pygithub, python-gitlab, azure-devops, commentjson, pytablewriter, pytest-cov, pytest-timeout, pytest-xdist, pyyaml, regex, requests, terminaltables, webpreview, yq, importlib-metadata, mkdocs-material, pymdown-extensions, mkdocs-glightbox, mdx_truly_sane_lists, beautifulsoup4, giturlparse, json-schema-for-humans, and fastapi. These dependencies are required for the development of the project. * 🚚 chore(Dockerfile): move dev requirements file to a more appropriate location The dev requirements file was moved from the root directory to a more appropriate location in the .config/python/dev directory. This improves the organization of the project and makes it easier to find the dev requirements file. * 🔧 chore(CONTRIBUTING.md): update pip install command to use new requirements file location The pip install command has been updated to use the new location of the requirements file, which is now located in the `.config/python/dev` directory. This change was made to improve the organization of the project files. * 🐛 fix(build-command.yml): update path to requirements.dev.txt to .config/python/dev/requirements.txt The path to the requirements file was incorrect and has been updated to the correct path. This ensures that the correct dependencies are installed during the build process. * 🆕 chore(gitpod.yml): add Gitpod workflow to run tests on push and pull request events This commit adds a new Gitpod workflow to run tests on push and pull request events. The workflow runs on Ubuntu latest and sets up Python 3.10. It installs dependencies and runs Gitpod tests using the `make gitpod-tests` command. The `nick-invision/retry@v2` action is used to retry the command up to 3 times with a timeout of 10 minutes. The `TERM` environment variable is also set to `xterm` to avoid issues with the terminal. * 🔥 chore(.dockerignore): add .venv to ignore list The .venv directory is added to the ignore list to prevent it from being included in the Docker image. This directory contains the virtual environment and its dependencies, which are not required in the Docker image. * 🚀 feat(Dockerfile): add markdown-table-formatter package to the Docker image The markdown-table-formatter package is now installed in the Docker image. This package is useful for formatting markdown tables in a more readable way. * 📝 docs(CONTRIBUTING.md): update contribution guidelines This commit updates the contribution guidelines to provide more detailed instructions on how to contribute to the project. It adds a step-by-step guide on how to fork the repository, make changes, test the changes, and submit a pull request. It also includes instructions on how to use Gitpod for a cloud-based development environment and how to install the necessary dependencies for a desktop environment. Finally, it adds a note to consider sponsoring the maintainer via GitHub. * 🐛 fix(.gitpod.yml): remove "-dev" from "bootstrap-dev" task name The "-dev" suffix was removed from the "bootstrap" task name to match the updated Makefile. * 🚀 feat(Makefile): add nodejs-bootstrap and nodejs-clean targets to Makefile The Makefile now includes two new targets, nodejs-bootstrap and nodejs-clean, which respectively bootstrap and clean the Node.js environment. This is useful for development as it allows for the installation of Node.js dependencies and the ability to clean them up when necessary. * 🚀 chore(package.json): add markdown-table-formatter dependency The markdown-table-formatter package has been added as a dependency to the project. This package will be used to format tables in markdown files. * 🔥 chore(Dockerfile): remove unnecessary whitespace The commit removes an unnecessary whitespace in the Dockerfile. * 🆕 chore(megalinter.mak): add megalinter-test target to run script build.sh This commit adds a new make target called megalinter-test which runs the build.sh script. The script is run in a virtual environment specified by the .venv directory. * 🆕 feat(nodejs.mak): add make targets for cleaning and bootstrapping Node.js dependencies This commit adds two new make targets to the nodejs.mak file. The `nodejs-clean` target removes the node_modules directory, and the `nodejs-bootstrap` target installs the Node.js dependencies using npm. These targets make it easier to manage the Node.js dependencies of the project. * 🐛 fix(CONTRIBUTING.md): update make command names to match their functionality The `make` command names `reinitialization-dev` and `bootstrap-dev` were changed to `reinitialization` and `bootstrap` respectively to better reflect their functionality. * 🔧 chore(Dockerfile): add mega-linter-runner package to Dockerfile The Dockerfile has been updated to include the mega-linter-runner package with version 7.0.0. This package is used to run the Mega-Linter tool, which is a linter that supports multiple programming languages and formats. This change will allow the tool to be used within the Gitpod environment. * 🔥 chore(.gitignore): add megalinter_file_names_cspell.txt to ignore list The megalinter_file_names_cspell.txt file is now being ignored by git. This file is not needed for the project and was causing unnecessary clutter in the repository. * 🔍 chore(.mega-linter.yml): update FILTER_REGEX_EXCLUDE to exclude new directories and files The FILTER_REGEX_EXCLUDE has been updated to exclude the newly added directories and files. The .venv directory has been added to SPELL_MISSPELL_FILTER_REGEX_EXCLUDE to avoid spell checking virtual environment files. * 🚀 feat(megalinter.mak): add new targets to run and test megalinter locally The new targets added are: - megalinter-run: runs megalinter locally using the python flavor and beta release - megalinter-tests: runs all megalinter tests by calling megalinter-run and megalinter-build targets - megalinter-clean: removes all megalinter reports, site and megalinter_file_names_cspell.txt files to clean up the local environment. * 🔧 chore(Makefile): rename megalinter-test to megalinter-tests and add megalinter-clean target The megalinter-test target was renamed to megalinter-tests to improve consistency with the naming conventions. A new target, megalinter-clean, was added to clean the megalinter environment. * 🔨 chore(Dockerfile): update NPM_VERSION to 9.7.1 The NPM_VERSION variable has been updated to 9.7.1 to ensure that the latest version of npm is used in the Gitpod workspace. * 🎨 style(CONTRIBUTING.md): remove unnecessary link reference The link reference for [pulls] was removed as it was not being used in the document. * 🔍 chore(.cspell.json): add "ungit" and "UNGIT" to the dictionary 🔍 chore(.cspell.json): add "dearmor" to the dictionary The words "ungit" and "UNGIT" were added to the dictionary to prevent them from being flagged as spelling errors. Similarly, "dearmor" was added to the dictionary to prevent it from being flagged as a spelling error. * 🔍 chore(.cspell.json): add "addsuffix" to the dictionary The word "addsuffix" was added to the dictionary to prevent it from being flagged as a spelling mistake. * 🐛 fix(nodejs.mak): add sudo to nodejs-clean target to allow deleting node_modules directory with root privileges The node_modules directory is deleted using the 'rm' command in the nodejs-clean target. However, if the user does not have the necessary permissions, the command will fail. Adding 'sudo' to the command allows the user to delete the directory with root privileges. * 🐛 fix(.mega-linter.yml): fix REPOSITORY_TRIVY_ARGUMENTS to skip .automation/test and .venv directories The REPOSITORY_TRIVY_ARGUMENTS configuration was updated to skip the .automation/test directory and the .venv directory. This is to ensure that the linter does not scan the test directory and the virtual environment directory, which can cause unnecessary errors and warnings. * 📈 chore(requirements.txt): update pymdown-extensions to version 10.0 This commit updates the pymdown-extensions package to version 10.0 in the dev requirements file. This update may include new features, bug fixes, or security patches. * 🔧 chore(python_version.config): update Python version to 3.11 The Python version has been updated from 3.10 to 3.11 in the .config/python_version.config file. This update was done to take advantage of the new features and improvements in Python 3.11. * 🔀 chore(gitpod.yml): change default branch name from master to main This commit updates the Gitpod workflow file to use the new default branch name "main" instead of "master". This change is made to improve inclusivity and to avoid any references to slavery. * 🔥 chore(.mega-linter.yml): remove unused SPELL_MISSPELL_FILTER_REGEX_EXCLUDE The SPELL_MISSPELL_FILTER_REGEX_EXCLUDE was not being used and was removed to simplify the configuration file. * 🐛 fix(gitpod.yml): change make bootstrap-dev to make bootstrap to match the correct target The make target was incorrect, it should be "bootstrap" instead of "bootstrap-dev". This commit fixes the target to match the correct one. * 🔧 chore(gitpod.yml): update Python version to 3.11 The Python version used in the Gitpod workflow has been updated from 3.10 to 3.11 to ensure that the latest version of Python is being used. This update ensures that the project is using the latest version of Python and can take advantage of any new features or improvements. * 🆕 chore(.python-version): add Python version 3.11.4 to project This commit adds a new file named .python-version to the project root directory. The file contains the Python version 3.11.4, which is the version of Python that will be used for this project. * 🔥 chore(python_version.config): remove python version configuration file The file .config/python_version.config was removed. It was no longer needed as the project has moved to a different version of Python. * 🐛 fix(Makefile): change python version detection to use .python-version file instead of .config/python_version.config The Makefile now uses the .python-version file to detect the python version instead of the .config/python_version.config file. This change was made to improve consistency with the naming conventions used in the project. * 🔧 chore(Dockerfile): rename python_version.config to .python-version and update reference in Dockerfile The file .config/python_version.config was renamed to .python-version to follow naming conventions. The reference to the file was updated in the Dockerfile to reflect the new name. * 🐛 fix(gitpod.yml): correct typo in job name The job name was corrected from "Run Giptod tests" to "Run Gitpod tests" to fix the typo. * 🔧 chore(python.mak): update python_launcher version to python3.11 The python_launcher variable has been updated to python3.11 to reflect the latest version of Python. This change is necessary to ensure that the correct version of Python is used when running the application. * 🔧 chore(CONTRIBUTING.md): update Python version to 3.11 The Python version required to run the project has been updated from 3.10 to 3.11 to ensure compatibility with the latest version of the language. This change was made to keep the project up-to-date with the latest software versions. * 🐛 fix(CONTRIBUTING.md): update megalinter test command to match the correct target The command to test with megalinter was incorrect and has been updated to `make megalinter-tests` to match the correct target. This will ensure that the code is properly linted and formatted before being merged into the main branch. * 🚀 chore(gitpod.yml): change Python version setup to read from .python-version file The Python version setup has been changed to read from a .python-version file instead of being hardcoded in the workflow file. This allows for easier management of the Python version used in the project. Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com> * 🐛 fix(workflows): use python-version-file instead of python-version to read python version from file The python version is now read from a file named .python-version instead of being hardcoded in the workflow files. This improves consistency and makes it easier to manage the python version across different workflows. Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com> * 🔄 chore(.python-version): add newline at end of file This commit adds a newline at the end of the .python-version file to improve consistency with the project's coding style. * 🐛 fix(Dockerfile): add newline at end of file The commit fixes a minor issue where there was no newline at the end of the Dockerfile. * 🔥 chore(.dockerignore): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔥 chore(gitpod.mak): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔥 chore(help.mak): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔥 chore(megalinter.mak): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔥 chore(nodejs.mak): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔥 chore(python.mak): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔥 chore(Makefile): add newline at end of file The commit fixes a minor issue where there was no newline at the end * 🔧 chore(gitpod.mak): rename docker image tag to megalinter_gitpod:local The docker image tag has been renamed to megalinter_gitpod:local to better reflect the purpose of the image. * 🐛 fix(gitpod.mak): reorder gitpod-tests dependencies to fix build order The gitpod-tests target was not building the application before running the linting command. This commit fixes the build order by moving the gitpod-build target to be a dependency of the gitpod-tests target. * 🔧 chore(.gitignore): ignore Redis dump files and server files Added '**/dump.rdb' and 'server/server-files/' to .gitignore to prevent accidental commit of Redis database dumps and server files. This is important to avoid exposing sensitive data and to keep the repository clean from unnecessary files. --------- Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com> commit c04d8da2e1482bb1110ac6e427d4c4b5937c91c7 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Tue Jul 11 19:10:08 2023 +0200 [automation] Auto-update linters version, help and documentation (#2783) commit 672c723889f3ecab160f2e0d09bc678a862739eb Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 10 19:48:16 2023 +0200 Stealth Upload Files & REDIS global messages (#2776) * Server with redis v0 * +q * x * struggle... * init * print * Docker compose tryouts * v0.0.0 * First working version :) * Alpha workflow * Update flavors Dockerfiles for server * Fix entrypoint * Fixes * Fixes * Fix server * Init RedisReporter * Normalize queues * Manage RedisPubSub * Fix entrypoint * x * Start message ok * Add redis in base image * Fix reporters sorting * Normalize names & test workerpool * Update worker pool * Normalize names * Use security flavor * Build & push megalitner-server on ghcr.io * Push updates * Fix linter.py * Fix case when linter command is a string * Lint fixes * Linter fixes * mypy * Redis reporter v0 * mypy.ini * Factorization * Methods skeletons * mypy * mypy * Free disk space * scopes * Before linting message * Handle zip file * Use fileUploadId * File upload v0 * MegalinterServerException * Upload file * Fix CI triggers for alpha * Default unique pubsub channel by requestId + output called CLI commands * Fix conflicts issues * Linter fixes * Build * QuickFix reporter name --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> commit b90e7cefe4a55caf6b11c15c86fe635763ba9f1f Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 10 17:35:50 2023 +0200 [automation] Auto-update linters version, help and documentation (#2781) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 7ed48a62301e56ca49bb5304353977209bbb1f07 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 10 14:00:35 2023 +0200 Upgrade tflint & build doc (#2752) * Upgrade tflint & build doc * tflint cli_lint_mode as project * Update default tflint aws plugin version * Bump actions/setup-node from 3.6.0 to 3.7.0 Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3.6.0 to 3.7.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v3.6.0...v3.7.0) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * [MegaLinter] Apply linters fixes * gitignore * Call tflint with --recursive * Hide iac-analyzers.dev * add required_version to terraform test files * try fix terraform test files * [MegaLinter] Apply linters fixes * againnnnn * and again >_< * tflint test fix * new tests * tflint failing test cases * build --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> commit b41bcc1d3a2d2577fc8210b417bb77c4e75978ec Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 10 10:47:24 2023 +0200 New flavor DotnetWeb (#2778) * Add dotnetweb flavor descriptive items * Build doc * cspell * ico --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> Co-authored-by: Isalgeon <Isalgeon@users.noreply.github.com> commit 56db111e4dba48c2a78061e1b96edf9c37bdbafa Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 10 05:43:15 2023 +0200 [automation] Auto-update linters version, help and documentation (#2777) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit ba7e6fc948b1ccbaba298245f1dcbee6a40d8cf2 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jul 9 22:22:33 2023 +0200 [automation] Auto-update linters version, help and documentation (#2775) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes * Add libstdc++ for alpine image * Downgrade kubescape * Try again to downgrade kubescape * again * lychee --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> commit bc4542d82550a6d05c0ec15b00884dc3e44210a7 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jul 8 12:38:47 2023 +0200 RedisReporter v0 (#2761) * Server with redis v0 * +q * x * struggle... * init * print * Docker compose tryouts * v0.0.0 * First working version :) * Alpha workflow * Update flavors Dockerfiles for server * Fix entrypoint * Fixes * Fixes * Fix server * Init RedisReporter * Normalize queues * Manage RedisPubSub * Fix entrypoint * x * Start message ok * Add redis in base image * Fix reporters sorting * Normalize names & test workerpool * Update worker pool * Normalize names * Use security flavor * Build & push megalitner-server on ghcr.io * Push updates * Fix linter.py * Fix case when linter command is a string * Lint fixes * Linter fixes * mypy * mypy.ini * mypy * mypy * Free disk space * trvy * changelog --------- Co-authored-by: nvuillam <nicolas.vuillamy@ox.security> commit cf251e17d83e9874df20c3d35af9ebe29ae7b755 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri Jul 7 23:12:07 2023 +0200 [automation] Auto-update linters version, help and documentation (#2774) commit fced0769ac4205e749727d4f3ddde4661299112c Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Thu Jul 6 12:01:40 2023 +0200 [automation] Auto-update linters version, help and documentation (#2772) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 72824276d6787a5eff81ebc52f1901c210f6f18e Author: Kihyeok Kim <96888460+KihyeokK@users.noreply.github.com> Date: Thu Jul 6 05:49:50 2023 -0400 Fix using default hadolint config file (#2763) The default config file is not being used as intended, as file extension names yaml and yml are conflicting. commit 737665c284bee38e6c39fed3ca5c614b2fedc716 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed Jul 5 15:21:22 2023 +0200 [automation] Auto-update linters version, help and documentation (#2771) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 4444486a92c3fa656e76ab028d1f9fb5ce050f3c Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed Jul 5 05:46:19 2023 +0200 [automation] Auto-update linters version, help and documentation (#2769) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 2dc42dc88dd7254787b923fcfb7f940fee8f62fc Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed Jul 5 05:45:43 2023 +0200 Undo downgrade of ansible-lint (#2770) * Undo downgrade of ansible-lint Fixes https://github.com/oxsecurity/megalinter/issues/2768 * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 0e24913a80223185c33629fc51aeb41c8e3c93d6 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 3 09:11:59 2023 +0200 Update doc & explanations about license (#2766) * [automation] Auto-update linters version, help and documentation * Add license explanations * cloudity * cspell * trivy * special thanks * [automation] Auto-update linters version, help and documentation * borja * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * Build * Disable vale to improve perfs --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit adef76f094305f0c987902d4008a20bff252c119 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jul 3 08:48:28 2023 +0200 ENV MYPY_CACHE_DIR=/tmp to avoid messing with Devskim & other linters + no default use of Devskim --ignore-globs (#2767) * [automation] Auto-update linters version, help and documentation * trvy * lychee * [MegaLinter] Apply linters fixes * Fix test cases * ENV MYPY_CACHE_DIR=/tmp to avoid messing with Devskim & other linters Fixes https://github.com/oxsecurity/megalinter/issues/2707 * Remove --ignore-globs by default Fixes https://github.com/oxsecurity/megalinter/issues/1993 * Handle reporter crashes without making all ML crash * Fix dev jobs * Revert "Fix dev jobs" This reverts commit fa1485d22547aa72a6dacbe37f377b2f0f273e0a. * docker image ls * fix buildx issue * Free disk space before build --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 24088b0674c21370cf5f3dca9c27818fa95e62c1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 25 14:23:48 2023 +0200 [automation] Auto-update linters version, help and documentation (#2760) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit b5a1cac6fcfe70fec305fa18315318be69bdf635 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jun 24 21:14:21 2023 +0200 [automation] Auto-update linters version, help and documentation (#2759) commit 6d98c00d864f026d80d5bcab9da01a8f4e119aee Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri Jun 23 22:51:54 2023 +0200 [automation] Auto-update linters version, help and documentation (#2757) * [automation] Auto-update linters version, help and documentation * trvy * [MegaLinter] Apply linters fixes * Fix dependency in mega-linter-runner * Force semver dependency version --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit dae6530e662c526c0509d69a7507a30544897d7d Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed Jun 21 08:34:59 2023 +0200 [automation] Auto-update linters version, help and documentation (#2753) commit 9763770dfbf2f69ba7f729f1fbd70a63d552f99e Author: Josef Andersson <josef.andersson@gmail.com> Date: Mon Jun 19 08:46:45 2023 +0200 feat: add ts-standard linter for ts standard (#2746) Signed-off-by: Josef Andersson <josef.andersson@gmail.com> commit 7558eb33ef3e61c0346541d5c866df485b9b8f19 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 18 23:18:30 2023 +0200 Undowngrade devskim (#2748) * Undowngrade devskim Fixes https://github.com/oxsecurity/megalinter/issues/2745 * Change devskim parameters * arg names * Add new type sarif for counting lines and use it on Devskim * fix * fix * [MegaLinter] Apply linters fixes * Fix sarif extract --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit f8804075673e82d6cfa1ec6c64879f7a98000538 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 18 12:31:38 2023 +0200 Upgrade gitleaks (#2749) commit 6b4b366df151c12784b3655207212b43dae0c018 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 18 09:40:03 2023 +0200 [automation] Auto-update linters version, help and documentation (#2742) * [automation] Auto-update linters version, help and documentation * trvy * lychee config * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 983cd4620e4f31e875a76515ac0c19bd1267bf32 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Wed Jun 14 22:24:27 2023 +0200 Rebuild doc after merging lychee commit f1cd5113bcde149602d087722e735906ce12ccb5 Author: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com> Date: Wed Jun 14 12:13:33 2023 -0700 feat(linter): add new link checker - lychee (#2673) * feat(linter): add lychee * test(lychee): add test * fix(test): update wrong path * chore(cspell): add linter's words to excludes * test(lychee): update test dir config * chore(changelog): add lychee * chore(ignore): add .lycheeignore to the root * chore(cspell): add lycheeignore * chore(lychee): add lychee.toml config file * chore(lychee): update base config * chore(lychee): update base config * chore(lychee): update base config * chore(cspell): add lycheecache * chore(links): update outdated * chore(links): update outdated * chore(schema): add lychee * chore(links): update outdated + config * chore(links): update oudated + config * chore(links): update outdated commit c1c57a06f3bb2315df7a4b81b1a2dbc5eaec5ee1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Tue Jun 13 10:47:00 2023 +0200 [automation] Auto-update linters version, help and documentation (#2741) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit e4ef06b185f56735275cdf6623d95fca96310a49 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jun 12 08:30:36 2023 +0200 [automation] Auto-update linters version, help and documentation (#2740) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit a7b1a3af0f3bd4de4db855969d27e224005665a6 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sun Jun 11 18:49:16 2023 +0200 Release MegaLinter v7.1.0 commit 26f69fa84ed58fbe1fe152a6307696de82b6a0a1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 11 15:57:36 2023 +0200 [automation] Auto-update linters version, help and documentation (#2739) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 4a61e700e136773984c0cc0fda5e054bee820582 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sun Jun 11 12:39:53 2023 +0200 Build doc commit 6e259c1cec7bed0a1703d90a3eaaa9a120c5bd0e Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 11 12:35:41 2023 +0200 Upgrade base image to python:3.11.4-alpine3.17 (#2738) * Bump zricethezav/gitleaks from v8.16.3 to v8.16.4 Bumps zricethezav/gitleaks from v8.16.3 to v8.16.4. --- updated-dependencies: - dependency-name: zricethezav/gitleaks dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Bump python from 3.11.3-alpine3.17 to 3.11.4-alpine3.17 Bumps python from 3.11.3-alpine3.17 to 3.11.4-alpine3.17. --- updated-dependencies: - dependency-name: python dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Upgrade base image to python:3.11.4-alpine3.17 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 4e43f1f3bae832a9744894171f32b89a145fa697 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 11 11:07:57 2023 +0200 [automation] Auto-update linters version, help and documentation (#2731) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit f27a593354d8787fe2b130fe292f87a55c964088 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Thu Jun 8 09:34:37 2023 +0200 [automation] Auto-update linters version, help and documentation (#2728) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 8d59aabba0eb8c132e2b98c834392c85309e3399 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Thu Jun 8 06:48:02 2023 +0200 [automation] Auto-update linters version, help and documentation (#2727) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 3d230f3e3b7a44eef594e706f929599d1f89085a Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed Jun 7 09:17:26 2023 +0200 [automation] Auto-update linters version, help and documentation (#2724) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes * trvy --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit baabfbbb810f79b433774435d11000359ebf3921 Author: Sean Eagan <seaneagan1@gmail.com> Date: Wed Jun 7 02:16:02 2023 -0500 golangci-lint: add capability to autofix (#2700) * golangci-lint: add capability to fix * Change default golangci config * [MegaLinter] Apply linters fixes * Change test case * Fix go auto-fix tests --------- Co-authored-by: nvuillam <nicolas.vuillamy@gmail.com> Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 152ac283fde367be523cf9af79feb82717a61d0b Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon Jun 5 21:51:00 2023 +0200 [automation] Auto-update linters version, help and documentation (#2721) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit c530d0d6bcdf64e596175f36ccc9603a805529f1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun Jun 4 13:37:08 2023 +0200 [automation] Auto-update linters version, help and documentation (#2720) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit ccc931c4522f1a40bde6874e1876ff7e833f71a7 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jun 3 14:32:35 2023 +0200 [automation] Auto-update linters version, help and documentation (#2718) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 72e4483f6dc7afacf57647a3f034263c00b975b1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jun 3 14:32:19 2023 +0200 cljstyle: No config file argument name (#2717) Fixes https://github.com/oxsecurity/megalinter/issues/2716 commit 79ef97c28f73cb4757f181ae187065235a3c6d61 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat Jun 3 11:25:09 2023 +0200 [automation] Auto-update linters version, help and documentation (#2713) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 11371bcb5da39d8f10bdfb265d792705e9107475 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri Jun 2 09:13:14 2023 +0200 Remove downgrade of markdownlint (#2714) commit ca32d6e8778d82911288a776b9be5ddc57953fc8 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri Jun 2 05:52:24 2023 +0200 [automation] Auto-update linters version, help and documentation (#2712) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 6e714d4e0eaa7d374bf20cceb4e8858b5e2d3759 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Thu Jun 1 23:06:47 2023 +0200 [automation] Auto-update linters version, help and documentation (#2710) * [automation] Auto-update linters version, help and documentation * cspell * changelog * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 0d014ffdef3307ca4a4cd1ddf482fc4f05733d7c Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Wed May 31 23:49:55 2023 +0200 Release MegaLinter v7.0.4 commit 1af426df21936e75b154444bd0453869ed4cdf83 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed May 31 22:41:46 2023 +0200 [automation] Auto-update linters version, help and documentation (#2709) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit a10d136c9cbb200c747d68cd77b86378d0d0a64c Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed May 31 21:32:26 2023 +0200 [automation] Auto-update linters version, help and documentation (#2708) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes * Downgrade ansible-lint * [MegaLinter] Apply linters fixes * Merge branch 'improve-tflint-docs' into create-pull-request/patch --------- Co-authored-by: ruzickap <ruzickap@users.noreply.github.com> commit b137b0929dcd895cd8110ca5df046976161182f8 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed May 31 17:11:45 2023 +0200 [automation] Auto-update linters version, help and documentation (#2705) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 60306fec31c8d6520f3ec70161872ecdc1647170 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed May 31 07:09:14 2023 +0200 [automation] Auto-update linters version, help and documentation (#2704) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 5cd8a86b159d3670caabc6d08a7dcf83edf6d7d1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed May 31 00:15:58 2023 +0200 Allow to define _UNSECURED_ENV_VARIABLES for specific linters (#2703) * Manage _UNSECURED_ENV_VARIABLES for linters Fixes https://github.com/oxsecurity/megalinter/issues/2699 * Doc + schema * Build doc & json schema * Doc * typo * Fix typo * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 429032dc3926b61eb22b8fe4fb19effad91c2639 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Mon May 29 21:29:15 2023 +0200 typo commit d392674cc5acf92b0394117c1a8a7bd5016aee75 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Mon May 29 21:28:25 2023 +0200 Use markdown for video commit b7dda6712ce38b415f5411db89ab6d479b1f5b58 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Mon May 29 21:27:03 2023 +0200 Upgrade to v7 video commit 2b1998e443e5757ace627e9af2094d0f1a2f80b4 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Mon May 29 11:14:38 2023 +0200 Release MegaLinter v7.0.3 commit ac577a9cfb72e9f51f70d7df731ed1dd57e7ed9f Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun May 28 22:51:06 2023 +0200 Manage v6 retrocompatiblity with FILTER_REGEX_INCLUDE and FILTER_REGEX_EXCLUDE expression (#2698) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes * TERRAFORM_TFLINT_SECURED_ENV management * [MegaLinter] Apply linters fixes * Fix FILTER_REGEX_EXCLUDE regression Fixes https://github.com/oxsecurity/megalinter/issues/2697 * [MegaLinter] Apply linters fixes * tflint test class * Build doc * Retrocompatiblity of FILTER_REGEX_EXCLUDE with v6 * label --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 8f6a79da3024dfb22b3f320aeb6324ba91d6c587 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun May 28 22:35:05 2023 +0200 [automation] Auto-update linters version, help and documentation + fix tflint secured env (#2696) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes * TERRAFORM_TFLINT_SECURED_ENV management * [MegaLinter] Apply linters fixes * tflint test class * Build doc --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit afae6499671d8d8c61d619982debe4bf9d295dd0 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun May 28 18:25:04 2023 +0200 Secure pre-command & post-command environment variables by default (#2695) * Secure pre-command & post-command environment variables by default * typo * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit bdf1e5ea77fd63403808941994644f97bfdea8f4 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat May 27 21:52:53 2023 +0200 [automation] Auto-update linters version, help and documentation (#2694) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit e99bc276c361c2f68c4976a5f86f7421c2c4d606 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sat May 27 13:49:12 2023 +0200 Release MegaLinter v7.0.2 commit 862fb049894e231ab19c281a853bdd66a6b7a7b6 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat May 27 13:41:31 2023 +0200 Quick fix mega-linter-runner --upgrade (#2693) - Dead link to configuration.md - Regex issue with megalinter-reports commit 8c92ca3cd548dec08716d2151776a7cc06cd4f99 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sat May 27 12:48:21 2023 +0200 Release MegaLinter v7.0.1 commit 4d7aae08422e927e536218141751e93a9bd9174f Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sat May 27 12:30:18 2023 +0200 Fix release workflow commit 02d61243f331ae88676e8c43b7e8a0b79dfb253c Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sat May 27 12:14:39 2023 +0200 Release MegaLinter v7.0.0 commit 32de34d40d7655d5dc167415f881653a25077f31 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat May 27 01:28:38 2023 +0200 Allow SECURED_ENV_VARIABLES to accept regular expressions (#2691) * Allow SECURED_ENV_VARIABLES to accept regular expressions * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit e29d894209de9a8c48e44fd925d94f6e7a62e09c Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat May 27 00:34:16 2023 +0200 Move KICS to REPOSITORY descriptor (#2689) * Move KICS to REPOSITORY descriptor Deprecate TERRAFORM_KICS * Fix json schema * [MegaLinter] Apply linters fixes * test folder * cli_lint_mode * SARIF for KICS * [MegaLinter] Apply linters fixes * Fix sariif arguments * Disable REPOSITORY_KICS for MegaLinter own linting --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 98f73b5c39af3cff94b608d6b53f4da11fe6629a Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri May 26 23:10:43 2023 +0200 mega-linter-runner: Use --platform also for docker run (#2690) commit ad631b85863ba8466e26b2d11121a9d0e7bd6781 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Fri May 26 21:32:32 2023 +0200 Fix permissions commit 86a8394de25a3ed890b402e0b553ceb46e5c0df1 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri May 26 17:33:01 2023 +0200 Improve documentation pages split (#2688) * Split install into different pages * Update build * Build * Config split * Build * typos * [MegaLinter] Apply linters fixes * Manage deprecation in JSON Schema * Mark variables as deprecated * Removed * format * [MegaLinter] Apply linters fixes * Fix generation * formatting * Formatting * Fix links * build doc * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit a1adf83a8ef169568a2b7632b48f518375d33e48 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri May 26 08:49:07 2023 +0200 [automation] Auto-update linters version, help and documentation (#2686) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 849722748d23a75c98fcf8099adaeda579383681 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Wed May 24 05:13:15 2023 +0200 [automation] Auto-update linters version, help and documentation (#2684) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 287895482d751011aefc3138a16da8b03e9f70bc Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Tue May 23 07:33:59 2023 +0200 [automation] Auto-update linters version, help and documentation (#2682) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 4e6ca11e52ed9c1a1e1c5dfbc87b21bdedc43074 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon May 22 22:18:23 2023 +0200 [automation] Auto-update linters version, help and documentation (#2680) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit c9279b4a23bc44a1a6316a5a97c814fd19c2351b Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Mon May 22 12:03:25 2023 +0200 [automation] Auto-update linters version, help and documentation (#2678) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 28721d7b66817b5886dcbfbce2227be703b43ed4 Author: Edouard Choinière <27212526+echoix@users.noreply.github.com> Date: Mon May 22 02:19:54 2023 -0400 Review MegaLinter's own cspell word list for outdated exclusions (#2676) commit 22c7a44ea8226fc646f6752e876335a449e3e18b Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sun May 21 23:03:00 2023 +0200 [automation] Auto-update linters version, help and documentation (#2675) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 85be26f3e4e0a07ecb314593b41c8aa7e37d3bf4 Author: Edouard Choinière <27212526+echoix@users.noreply.github.com> Date: Sun May 21 13:17:40 2023 -0400 Decouple updating docker pull stats from building docs (#2677) * Decouple updating docker pull stats from building docs. Keep updating stats for automated daily updates * Update CHANGELOG.md commit 773fa24b77d5a78cd1732442166b42443d7d48c1 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Sat May 20 13:34:33 2023 +0200 Fix doc workflow to allow it to deploy :) commit f6282f435ea8618b9d7786b8a773246c124768b7 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat May 20 13:12:08 2023 +0200 [automation] Auto-update linters version, help and documentation (#2674) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 457523f9d1c2e5a52dcafe8895788a55d7f0f900 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Sat May 20 08:16:36 2023 +0200 [automation] Auto-update linters version, help and documentation (#2671) * [automation] Auto-update linters version, help and documentation * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 63e74ad1eab40076f7462268c39bd9761a05c850 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri May 19 18:54:37 2023 +0200 Simultaneous regex filtering at descriptor and linter levels (#2669) * Simultaneous regex filtering at decriptor and linter levels Fixes https://github.com/oxsecurity/megalinter/issues/2668 * Add test method * typo * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit 270cd73f1fff0a5f5449fa80731ab56968ef56d8 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri May 19 18:31:12 2023 +0200 Improve tables display (#2670) commit 1092b81ec8c382e66a8dba2c313bc9940d31fd93 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Fri May 19 17:11:48 2023 +0200 Typo commit 81d50b43fe5d02ccd4698c9acde5ca71ae1f9d01 Author: nvuillam <nicolas.vuillamy@gmail.com> Date: Fri May 19 17:08:20 2023 +0200 Fix plugins table commit 6c00dab2bf46ffba0145c358a43252ea88f8d3f7 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com> Date: Fri May 19 16:39:56 2023 +0200 Automate plugin documentation generation (#2667) * Exclude licenses pages from online search results Fixes https://github.com/oxsecurity/megalinter/issues/2664 * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * [MegaLinter] Apply linters fixes * Ignore licenses * Plugins doc v0 * Generate plugins doc * Automate External Plugins table generation using **.automation/plugins.yml** file Fixes https://github.com/oxsecurity/megalinter/issues/2666 * cspell * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <nvuillam@users.noreply.github.com> commit fe51dce97c129129a4350e593743466f477dc6b4 Author: Nicolas Vuillamy <nicolas.vuillamy@gmail.com>…

bdovaz and others added 4 commits June 23, 2022 16:32

Flavor changes

2862a6d

Main Dockerfile changes

7f0df4e

Build flavors workflow

76885a3

Merge branch 'megalinter:main' into feature/arm-docker-image

b61b380

bdovaz mentioned this pull request Jun 23, 2022

Support for ARM64 for docker image #1180

Open

bdovaz changed the title ~~Feature/arm docker image~~ ARM64 docker images Jun 23, 2022

bdovaz added 5 commits June 24, 2022 18:11

Restore flavor Dockerfiles

dca5e3b

Change dotnet version to LTS

0d306e2

Add install_arm field to bash descriptor

c70b003

Add arm_compliant field to env descriptor

454eab3

Empty Dockerfile_arm

6a334e2

build.py WIP

4b7e36b

- Temporarily disable validators

echoix added 3 commits July 4, 2022 00:35

Add linter supported_architectures with CPU architecture-specific ins…

1de041d

…tall overrides.

Apply formatting of json schema

43687be

Use proper Docker architectures names

99cab60

from https://github.com/docker-library/bashbrew/blob/v0.1.2/architecture/oci-platform.go#L14-L27

Fix docker buildx in workflows

29e2ac1

echoix changed the base branch from main to dev/multi-platform-images January 17, 2023 01:45

echoix marked this pull request as ready for review January 17, 2023 01:46

echoix requested a review from nvuillam as a code owner January 17, 2023 01:46

echoix merged commit e223a1a into oxsecurity:dev/multi-platform-images Jan 17, 2023

echoix added enhancement New feature or request docker Pull requests that update Docker code github_actions Pull requests that update Github_actions code labels Jan 17, 2023

bdovaz deleted the feature/arm-docker-image branch January 17, 2023 19:38

bdovaz mentioned this pull request Jan 17, 2023

feat: add support in the build of Docker images for linux/arm64 in compatible linters #2273

Merged

bdovaz pushed a commit that referenced this pull request Feb 26, 2023

Add supported_platforms to bash descriptor

9840062

Relates to #1553

bdovaz pushed a commit that referenced this pull request Mar 10, 2023

Add supported_platforms to bash descriptor

5f54bcc

Relates to #1553

bdovaz pushed a commit that referenced this pull request Mar 26, 2023

Add supported_platforms to bash descriptor

021112b

Relates to #1553

bdovaz pushed a commit that referenced this pull request Mar 26, 2023

Add supported_platforms to bash descriptor

5f6bf1f

Relates to #1553

bdovaz pushed a commit that referenced this pull request Apr 13, 2023

Add supported_platforms to bash descriptor

884d0e0

Relates to #1553

bdovaz pushed a commit that referenced this pull request May 14, 2023

Add supported_platforms to bash descriptor

f1a68c9

Relates to #1553

bdovaz pushed a commit that referenced this pull request May 14, 2023

Add supported_platforms to bash descriptor

40d749e

Relates to #1553

bdovaz pushed a commit that referenced this pull request May 14, 2023

Add supported_platforms to bash descriptor

7cd155f

Relates to #1553

Uh oh!

Conversation

bdovaz commented Jun 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Support of runners on arm64

Change from docker build to docker buildx

Supported platform descriptors configuration

Decide how to generate the Dockerfile(s).

Additional blocks

Additional blocks (external)

Testing

Uh oh!

echoix commented Jun 23, 2022

Uh oh!

nvuillam commented Jun 24, 2022

Uh oh!

bdovaz commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echoix commented Jun 24, 2022

Uh oh!

nvuillam commented Jun 24, 2022

Uh oh!

echoix commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echoix commented Jun 24, 2022

Uh oh!

bdovaz commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echoix commented Jun 24, 2022

Uh oh!

Kurt-von-Laven commented Jun 24, 2022

Uh oh!

echoix commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kurt-von-Laven commented Jun 25, 2022

Uh oh!

echoix commented Jun 25, 2022

Uh oh!

bdovaz commented Jun 27, 2022

Uh oh!

echoix commented Jul 3, 2022

Uh oh!

bdovaz commented Jan 16, 2023

Uh oh!

bdovaz commented Jan 16, 2023

Uh oh!

echoix commented Jan 16, 2023

Uh oh!

bdovaz commented Jan 16, 2023

Uh oh!

echoix commented Jan 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echoix commented Jan 16, 2023

Uh oh!

nvuillam commented Jan 16, 2023

Uh oh!

nvuillam commented Jan 16, 2023

Uh oh!

echoix commented Jan 16, 2023

Uh oh!

Kurt-von-Laven commented Jan 17, 2023

Uh oh!

echoix commented Jan 17, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bdovaz commented Jun 23, 2022 •

edited

Loading

bdovaz commented Jun 24, 2022 •

edited

Loading

echoix commented Jun 24, 2022 •

edited

Loading

bdovaz commented Jun 24, 2022 •

edited

Loading

echoix commented Jun 24, 2022 •

edited

Loading

echoix commented Jan 16, 2023 •

edited

Loading