This repository was archived by the owner on Sep 17, 2024. It is now read-only.
feat: check how many processes of a process are running in the host#1153
Merged
mdelapenya merged 4 commits intoelastic:masterfrom May 10, 2021
Merged
feat: check how many processes of a process are running in the host#1153mdelapenya merged 4 commits intoelastic:masterfrom
mdelapenya merged 4 commits intoelastic:masterfrom
Conversation
This will allow removing the initial bytes when reading outputs from command execution in a container
…ntainer It uses pgrep to get all pids for a process, and then iterates through them to get the runnable status for each pid. If the process must be started in the host, then it will check that the pid is in the S status (to skip zombie processes)
Contributor
Author
|
As commented with @michalpristas online, the Docker image has a default configuration in which it collects metrics but not logs, but monitors everything, hence there are 2 metricbeat processes and 1 filebeat process. |
Contributor
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
adam-stokes
approved these changes
May 10, 2021
mergify bot
pushed a commit
that referenced
this pull request
May 10, 2021
…1153) * fix: use docker's stdcopy to separate stdout from stderr This will allow removing the initial bytes when reading outputs from command execution in a container * feat: support checking the number of occurrences of a process in a container It uses pgrep to get all pids for a process, and then iterates through them to get the runnable status for each pid. If the process must be started in the host, then it will check that the pid is in the S status (to skip zombie processes) * fix: check for only one filebeat instance * fix: check for empty response when listing agent's workdir (cherry picked from commit 78a0d49)
mergify bot
pushed a commit
that referenced
this pull request
May 10, 2021
…1153) * fix: use docker's stdcopy to separate stdout from stderr This will allow removing the initial bytes when reading outputs from command execution in a container * feat: support checking the number of occurrences of a process in a container It uses pgrep to get all pids for a process, and then iterates through them to get the runnable status for each pid. If the process must be started in the host, then it will check that the pid is in the S status (to skip zombie processes) * fix: check for only one filebeat instance * fix: check for empty response when listing agent's workdir (cherry picked from commit 78a0d49)
Contributor
Author
|
@adam-stokes I'm going to send a follow-up PR adding this to the deployer struct, so that we can start decoupling stuff preparing the multiplatform support |
9 tasks
mdelapenya
added a commit
to mdelapenya/e2e-testing
that referenced
this pull request
May 12, 2021
* master: feat: simplify the initialisation of versions (elastic#1159) chore(mergify): delete upstream branches on merge (elastic#1158) chore: abstract process checks to the deployer (elastic#1156) feat: check how many processes of a process are running in the host (elastic#1153)
mdelapenya
added a commit
that referenced
this pull request
May 14, 2021
…are running in the host (#1155) * feat: check how many processes of a process are running in the host (#1153) * fix: use docker's stdcopy to separate stdout from stderr This will allow removing the initial bytes when reading outputs from command execution in a container * feat: support checking the number of occurrences of a process in a container It uses pgrep to get all pids for a process, and then iterates through them to get the runnable status for each pid. If the process must be started in the host, then it will check that the pid is in the S status (to skip zombie processes) * fix: check for only one filebeat instance * fix: check for empty response when listing agent's workdir (cherry picked from commit 78a0d49) * fix: check for 1 filebeat process only * fix: check for 1 metricbeat process only Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya
added a commit
that referenced
this pull request
May 14, 2021
… running in the host (#1154) * feat: check how many processes of a process are running in the host (#1153) * fix: use docker's stdcopy to separate stdout from stderr This will allow removing the initial bytes when reading outputs from command execution in a container * feat: support checking the number of occurrences of a process in a container It uses pgrep to get all pids for a process, and then iterates through them to get the runnable status for each pid. If the process must be started in the host, then it will check that the pid is in the S status (to skip zombie processes) * fix: check for only one filebeat instance * fix: check for empty response when listing agent's workdir (cherry picked from commit 78a0d49) * fix: check for 1 filebeat process only * fix: check for 1 metricbeat process only Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
It creates a new step that abstract the logic to get the number of processes in a desired state, calling this new step in the existing one (with 1 as number of ocurrences).
We had to refactor the existing code to wait for processes, as it only checked that a process existed in the host using
pgrep -n -l, which only checks for the existence of the latest process for a process.With this new approach, we are switching to
pgrep -d , $THE_PROCESS, which separates each pid for the process with the comma delimiter. Having that list of pids for a process we executeps -q $PID -o state --no-headers(linux only) to get the current state of that pid, to discard zombie processes (we detected that the elastic-agent on Centos creates 3 processes, 2 of them in the zombie state cc/ @michalpristas).Finally, we refactored the existing method to get the output of executing a command in a container to avoid receiving extra bytes at the beginning of the output. In the previous code we had a list of bytes to remove, but now we simply use a Docker client helper utility (stdcopy) that splits stdout from stderr in the container, making possible to retrieve just the output. See https://stackoverflow.com/a/57132902 for further information.
Why is it important?
It creates a more comprehensive check about the background processes that the elastic-agent spawns in the host, as we want to make sure the number of processes is correct.
Checklist
make noticein the proper directory)How to test this PR locally
Tests for the backend processes:
Considerations for the stand-alone mode using the Docker image
We noticed that the elastic-agent running in the docker image in stand-alone mode spawns only 1 filebeat process, instead of 2 as using the TAR/DEB/RPM installers. Maybe @michalpristas can help out here.
Related issues