Skip to content

feat(agent): Introduce CLI option to set config URL retry attempts#15377

Merged
srebhan merged 1 commit intoinfluxdata:masterfrom
powersj:fix/8854
May 21, 2024
Merged

feat(agent): Introduce CLI option to set config URL retry attempts#15377
srebhan merged 1 commit intoinfluxdata:masterfrom
powersj:fix/8854

Conversation

@powersj
Copy link
Copy Markdown
Contributor

@powersj powersj commented May 17, 2024

Summary

This introduces a new cli option to allow the user to set the number of retry attempts to something other than 3. It also allows the user to set the attempt count to -1 to infinitely retry.

Checklist

  • No AI generated code was used in this PR

Related issues

fixes: #8854

@powersj
feat(agent): Introduce CLI option to set config URL retry attempts
470da02 
This introduces a new cli option to allow the user to set the number of
retry attempts to something other than 3. It also allows the user to set
the attempt count to -1 to infinitely retry.

fixes: influxdata#8854
@telegraf-tiger telegraf-tiger bot added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label May 17, 2024
@telegraf-tiger
Copy link
Copy Markdown
Contributor

telegraf-tiger bot commented May 17, 2024

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

👍 This pull request doesn't change the Telegraf binary size

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB RPM TAR GZ ZIP
amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip
arm64.deb armel.rpm darwin_arm64.tar.gz windows_arm64.zip
armel.deb armv6hl.rpm freebsd_amd64.tar.gz windows_i386.zip
armhf.deb i386.rpm freebsd_armv7.tar.gz
i386.deb ppc64le.rpm freebsd_i386.tar.gz
mips.deb riscv64.rpm linux_amd64.tar.gz
mipsel.deb s390x.rpm linux_arm64.tar.gz
ppc64el.deb x86_64.rpm linux_armel.tar.gz
riscv64.deb linux_armhf.tar.gz
s390x.deb linux_i386.tar.gz
linux_mips.tar.gz
linux_mipsel.tar.gz
linux_ppc64le.tar.gz
linux_riscv64.tar.gz
linux_s390x.tar.gz

@powersj powersj marked this pull request as ready for review May 20, 2024 14:40
@powersj powersj added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label May 20, 2024
@powersj
Copy link
Copy Markdown
Contributor Author

powersj commented May 20, 2024 •
edited
Loading

Some manual testing results:

Default Case

This is the initial attempt + 3 retires:

❯ ./telegraf --config http://localhost:8080/config.toml
2024-05-20T14:40:49Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:40:49Z I! Using default number of attempts to fetch HTTP config: 3
2024-05-20T14:40:49Z I! Error getting HTTP config (attempt 0 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:40:59Z I! Error getting HTTP config (attempt 1 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:41:09Z I! Error getting HTTP config (attempt 2 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:41:19Z I! Error getting HTTP config (attempt 3 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:41:19Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to 4

More than default 3 retry attempts:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=4
2024-05-20T14:42:11Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:42:11Z I! Error getting HTTP config (attempt 0 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:21Z I! Error getting HTTP config (attempt 1 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:31Z I! Error getting HTTP config (attempt 2 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:41Z I! Error getting HTTP config (attempt 3 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:51Z I! Error getting HTTP config (attempt 4 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:51Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to 2

Less than default 3 retries:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=2
2024-05-20T14:42:58Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:42:58Z I! Error getting HTTP config (attempt 0 of 2): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:08Z I! Error getting HTTP config (attempt 1 of 2): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:18Z I! Error getting HTTP config (attempt 2 of 2): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:18Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to 0

Will use the default 3 retry attempts:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=0
2024-05-20T14:43:40Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:43:40Z I! Using default number of attempts to fetch HTTP config: 3
2024-05-20T14:43:40Z I! Error getting HTTP config (attempt 0 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:50Z I! Error getting HTTP config (attempt 1 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:00Z I! Error getting HTTP config (attempt 2 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:10Z I! Error getting HTTP config (attempt 3 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:10Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to -1

This retries forever:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=-1
2024-05-20T14:44:16Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:44:16Z I! Using unlimited number of attempts to fetch HTTP config
2024-05-20T14:44:16Z I! Error getting HTTP config (attempt 0 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:26Z I! Error getting HTTP config (attempt 1 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:36Z I! Error getting HTTP config (attempt 2 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:46Z I! Error getting HTTP config (attempt 3 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:56Z I! Error getting HTTP config (attempt 4 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:45:06Z I! Error getting HTTP config (attempt 5 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
^C

Set attempts to -2

Anything below -1 is invalid:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=-2
2024-05-20T14:45:21Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:45:21Z E! error loading config file http://localhost:8080/config.toml: invalid number of attempts: -2

Retry works

One positive case, where retry works after I bring server up:

❯ ./telegraf --config http://localhost:8000/config.toml --config-url-retry-attempts=-1
2024-05-20T14:47:04Z I! Loading config: http://localhost:8000/config.toml
2024-05-20T14:47:04Z I! Using unlimited number of attempts to fetch HTTP config
2024-05-20T14:47:04Z I! Error getting HTTP config (attempt 0 of -1): failed to connect to HTTP config server: Get "http://localhost:8000/config.toml": dial tcp 127.0.0.1:8000: connect: connection refused
2024-05-20T14:47:14Z I! Error getting HTTP config (attempt 1 of -1): failed to connect to HTTP config server: Get "http://localhost:8000/config.toml": dial tcp 127.0.0.1:8000: connect: connection refused
2024-05-20T14:47:24Z W! DeprecationWarning: Option "fielddrop" of plugin "inputs.mysql" deprecated since version 1.29.0 and will be removed in 2.0.0: use 'fieldexclude' instead
2024-05-20T14:47:24Z I! Starting Telegraf 1.31.0-cda0f1d9 brought to you by InfluxData the makers of InfluxDB
2024-05-20T14:47:24Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 60 outputs, 6 secret-stores
2024-05-20T14:47:24Z I! Loaded inputs: mysql
2024-05-20T14:47:24Z I! Loaded aggregators: 
2024-05-20T14:47:24Z I! Loaded processors: 
2024-05-20T14:47:24Z I! Loaded secretstores: 
2024-05-20T14:47:24Z I! Loaded outputs: file
2024-05-20T14:47:24Z I! Tags enabled: 
2024-05-20T14:47:24Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:10s
2024-05-20T14:47:24Z D! [agent] Initializing plugins
2024-05-20T14:47:24Z D! [agent] Connecting outputs
2024-05-20T14:47:24Z D! [agent] Attempting connection to [outputs.file]
2024-05-20T14:47:24Z D! [agent] Successfully connected to outputs.file
2024-05-20T14:47:24Z D! [agent] Starting service inputs
^C2024-05-20T14:47:28Z D! [agent] Stopping service inputs
2024-05-20T14:47:28Z D! [agent] Input channel closed
2024-05-20T14:47:28Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-05-20T14:47:28Z D! [outputs.file]  Buffer fullness: 0 / 10000 metrics
2024-05-20T14:47:28Z I! [agent] Stopping running outputs
2024-05-20T14:47:28Z D! [agent] Stopped Successfully

mstrandboge
mstrandboge approved these changes May 20, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@mstrandboge mstrandboge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@mstrandboge mstrandboge removed their assignment May 20, 2024
srebhan
srebhan approved these changes May 21, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@srebhan srebhan merged commit ad59290 into influxdata:master May 21, 2024
@github-actions github-actions bot added this to the v1.31.0 milestone May 21, 2024
pvlltvk pushed a commit to devopsext/telegraf that referenced this pull request May 31, 2024
@powersj @pvlltvk
feat(agent): Introduce CLI option to set config URL retry attempts (i…
3a94fa8 
…nfluxdata#15377)
@BrewTestBot BrewTestBot mentioned this pull request Jun 10, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@srebhan srebhan srebhan approved these changes

@mstrandboge mstrandboge mstrandboge approved these changes

Assignees

@srebhan srebhan

Labels

area/agent feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.

Projects

None yet

Milestone

v1.31.0

Development

Successfully merging this pull request may close these issues.

Configuration capabilities to retry for loading config via URL

3 participants

@powersj @srebhan @mstrandboge