Skip to content

Add system.hostfs setting#673

Merged
ChrsMark merged 2 commits intoelastic:masterfrom
ChrsMark:add_hostfs_setting
Feb 9, 2021
Merged

Add system.hostfs setting#673
ChrsMark merged 2 commits intoelastic:masterfrom
ChrsMark:add_hostfs_setting

Conversation

@ChrsMark
Copy link
Copy Markdown
Member

@ChrsMark ChrsMark commented Feb 8, 2021

What does this PR do?

This PR adds system.hostfs options for system and linux package. This setting was introduced in elastic/beats#23831 in oder to resolve elastic/beats#22915.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.

How to test this PR locally

  1. Update properly the stack manifest file so as to mount the proper directories inside the docker container:
# /home/linux_test_user/.elastic-package/stack/snapshot.yml
      - "0.0.0.0:5601:5601"
  kibana_is_ready:
    image: tianon/true
    depends_on:
      kibana:
        condition: service_healthy
  package-registry:
    build:
      context: .
      dockerfile: Dockerfile.package-registry
    healthcheck:
      test: ["CMD", "curl", "-f", "http://127.0.0.1:8080"]
      retries: 300
      interval: 1s
    ports:
      - "0.0.0.0:8080:8080"
  package-registry_is_ready:
    image: tianon/true
    depends_on:
      package-registry:
        condition: service_healthy
  elastic-agent:
    image: docker.elastic.co/beats/elastic-agent:${STACK_VERSION}
    depends_on:
      elasticsearch:
        condition: service_healthy
      kibana:
        condition: service_healthy
    healthcheck:
      test: "sh -c 'grep \"Agent is starting\" /usr/share/elastic-agent/elastic-agent.log*'"
      retries: 30
      interval: 1s
    environment:
    - "FLEET_ENROLL=1"
    - "FLEET_ENROLL_INSECURE=1"
    - "FLEET_SETUP=1"
    - "KIBANA_HOST=http://kibana:5601"
    volumes:
    - /proc:/hostfs/proc
    - /sys/fs/cgroup:/hostfs/sys/fs/cgroup
    - type: bind
      source: ../tmp/service_logs/
      target: /tmp/service_logs/
  elastic-agent_is_ready:
    image: tianon/true
    depends_on:
      elastic-agent:
        condition: service_healthy
  1. Launch Elastic stack on a linux machine: elastic-package stack up --version 7.12.0-SNAPSHOT -v -d (make sure that the updated package is served by the registry)
  2. From Fleet UI enable System integration and set system.hostfs to /hostfs.
  3. Verify that data-streams work properly for instance data_stream.dataset: system.process filter shows events in Discover page of Kibana.

Relevant Resources:

  1. https://www.elastic.co/guide/en/beats/metricbeat/current/running-on-docker.html#monitoring-host
  2. https://github.com/elastic/beats/blob/master/deploy/kubernetes/metricbeat-kubernetes.yaml#L192-L197

Related issues

Screenshots

Screenshot 2021-02-08 at 4 19 45 PM
Screenshot 2021-02-08 at 4 20 09 PM

@fearful-symmetry could you have a look and share your feedback please?

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark ChrsMark added the Team:Integrations Label for the Integrations team label Feb 8, 2021
@ChrsMark ChrsMark self-assigned this Feb 8, 2021
@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/integrations (Team:Integrations)

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Feb 8, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Branch indexing

  • Start Time: 2021-02-08T22:28:09.945+0000

  • Duration: 8 min 48 sec

  • Commit: 145d20c

Test stats 🧪

Test Results
Failed 0
Passed 203
Skipped 0
Total 203

Trends 🧪

Image of Build Times

Image of Tests

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark
Copy link
Copy Markdown
Member Author

ChrsMark commented Feb 9, 2021

Tested on linux env on GCP, works as expected.

@ChrsMark ChrsMark merged commit b1b35b1 into elastic:master Feb 9, 2021
ycombinator added a commit to ycombinator/integrations that referenced this pull request Feb 9, 2021
ycombinator added a commit that referenced this pull request Feb 9, 2021
* Adding condition

* Revert "Add system.hostfs setting (#673)"

This reverts commit b1b35b1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Team:Integrations Label for the Integrations team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Metricbeat --system.hostfs equivalent to Elastic Agent

4 participants