Skip to content

[System benchmark] Start service in SetUp process to update placeholders in config#3138

Merged
mrodm merged 6 commits intoelastic:mainfrom
mrodm:fix_placeholders_system_benchmark
Dec 16, 2025
Merged

[System benchmark] Start service in SetUp process to update placeholders in config#3138
mrodm merged 6 commits intoelastic:mainfrom
mrodm:fix_placeholders_system_benchmark

Conversation

@mrodm
Copy link
Contributor

@mrodm mrodm commented Dec 4, 2025 •
edited
Loading

Fixes #2905

In order to be able to update the placeholders like Hostname and Port with the expected values, the service must be started in the SetUp method so it can update the variables in the configuration using those placeholders accordingly.

This PR ensures that the service is setup when the configuration is read and therefore policy variables are also set.

Example of error using elastic-package version v0.117.1:

Error: error running package system benchmarks: could not set up benchmark runner: could not create package policy (req {"name":"","description":"","namespace":"ep","policy_id":"a9024eda-09b5-4da0-a4e5-1e485f77c7cf","package":{"name":"sentinel_one","version":"2.0.0"},"inputs":{"sentinel_one-cel":{"enabled":false},"sentinel_one-httpjson":{"enabled":true,"vars":{"api_token":"xxxx","enable_request_tracer":true,"url":"http://:0"},"streams":{"sentinel_one.activity":{"enabled":true,"vars":{"preserve_original_event":true}}}}},"force":true}); API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Package policy is invalid: inputs.httpjson.vars.url: ["Invalid URL format"]"}

where the url is not rendering the variables ("url":"http://:0").

Including the fix in this PR:
URL rendered

Author's Checklist

  • Review system benchmark docs
  • Run system benchmarks and compare with previous implementation.

@mrodm mrodm self-assigned this Dec 4, 2025
@mrodm mrodm changed the title Start service in SetUp process to update placeholders in config [System benchmark] Start service in SetUp process to update placeholders in config Dec 4, 2025
@mrodm
Copy link
Contributor Author

mrodm commented Dec 9, 2025

/test

1 similar comment
@mrodm
Copy link
Contributor Author

mrodm commented Dec 10, 2025

/test

@elasticmachine
Copy link
Collaborator

elasticmachine commented Dec 10, 2025

💚 Build Succeeded

History

cc @mrodm

@mrodm mrodm marked this pull request as ready for review December 12, 2025 11:01
@mrodm mrodm requested a review from a team December 12, 2025 11:01
@mrodm
Copy link
Contributor Author

mrodm commented Dec 12, 2025

@navnit-elastic could you test the changes in this PR to validate it works with the system benchmark you were working on in #2905 ?
Thanks!

@navnit-elastic
Copy link

navnit-elastic commented Dec 15, 2025

@mrodm, I tested the changes in this PR, and everything works as expected. The placeholders are updated correctly. Thank you for working through this!

╭──────────────────────────────────────────────────────────────────────────────╮
│ parameters                                                                   │
├──────────────────────────────────────────┬───────────────────────────────────┤
│ package version                          │                             2.0.0 │
│ input                                    │                          httpjson │
│ vars.url                                 │      http://svc-sentinel_one:8080 │
│ vars.api_token                           │                              xxxx │
│ vars.enable_request_tracer               │                              true │
│ data_stream.name                         │                             alert │
│ data_stream.vars.preserve_original_event │                              true │
│ warmup time period                       │                                2s │
│ benchmark time period                    │                                0s │
│ wait for data timeout                    │                             10m0s │
│ corpora.generator.total_events           │                            100000 │
│ corpora.generator.template.path          │ ./alert-benchmark/template.ndjson │
│ corpora.generator.template.raw           │                                   │
│ corpora.generator.template.type          │                            gotext │
│ corpora.generator.config.path            │      ./alert-benchmark/config.yml │
│ corpora.generator.config.raw             │                             map[] │
│ corpora.generator.fields.path            │      ./alert-benchmark/fields.yml │
│ corpora.generator.fields.raw             │                             map[] │
│ corpora.input_service.name               │                      sentinel_one │
│ corpora.input_service.signal             │                                   │
╰──────────────────────────────────────────┴───────────────────────────────────╯

@mrodm mrodm merged commit 242f879 into elastic:main Dec 16, 2025
3 checks passed
@mrodm mrodm deleted the fix_placeholders_system_benchmark branch December 16, 2025 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jsoriano jsoriano jsoriano approved these changes

@teresaromero teresaromero teresaromero approved these changes

Assignees

@mrodm mrodm

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[system benchmarking] "Hostname" and "Port" placeholders evaluate to empty strings in scenario definition

5 participants

@mrodm @elasticmachine @navnit-elastic @jsoriano @teresaromero