Skip to content

Conversation

@StephenWakely
Copy link
Contributor

@StephenWakely StephenWakely commented Apr 13, 2023

Ref #17115

Currently if you add type information for an event or metadata field and that field is not defined as an Object, Vector panics.

The unfortunate result of this is that if something has gone wrong whilst building up the topology, rather than allowing Vector to report all the errors to allow the problem to be diagnosed, we just get a panic.

This PR changes it so that it produces a warning instead and then continues, which will allow Vector to report better diagnostics around what has gone wrong.


Another issue that cropped up while investigating this is that whilst building up the topology the transforms get compiled several times. The first time it gets compiled, the enrichment tables haven't been loaded yet. This means that every transform that loads an enrichment table will error. Normally this doesn't matter, since by the time the transforms are compiled fully the enrichment tables have been loaded.

However the original bug this is fixing would still cause the panic. Changing it to warnings meant we still received the warnings with perfectly valid configurations. A further change here is for the Remap transform to check if the incoming definition is never and just pass this never definition downstream.

A quick scan of the transform seems to show that remap is the only transform that needs updating with this check.

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@StephenWakely StephenWakely requested a review from a team April 13, 2023 09:46
@netlify
Copy link

netlify bot commented Apr 13, 2023

Deploy Preview for vector-project canceled.

Name Link
🔨 Latest commit 569994a
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/644a8d7ad6da4800083a2234

@netlify
Copy link

netlify bot commented Apr 13, 2023

Deploy Preview for vrl-playground ready!

Name Link
🔨 Latest commit 569994a
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/644a8d7ad20acc00087d96ba
😎 Deploy Preview https://deploy-preview-17140--vrl-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions github-actions bot added the domain: core Anything related to core crates i.e. vector-core, core-common, etc label Apr 13, 2023
@datadog-vectordotdev
Copy link

datadog-vectordotdev bot commented Apr 13, 2023

Datadog Report

Branch report: stephen/schema_event_field_panic
Commit report: f31564a

vector: 0 Failed, 0 New Flaky, 1863 Passed, 0 Skipped, 6m 45.11s Wall Time

@github-actions
Copy link

Regression Detector Results

Run ID: fb9fd36f-5520-4e88-b66d-45e752700efd
Baseline: 8762563
Comparison: 3ba9721
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_to_http_json ingress throughput 238.13KiB/CPU-s 1.74 100.00% 13.36MiB/CPU-s 780.9KiB/CPU-s 10.07KiB/CPU-s 0.0 0.057058 13.6MiB/CPU-s 276.78KiB/CPU-s 3.57KiB/CPU-s 0.0 0.019878 False False
datadog_agent_remap_blackhole ingress throughput 123.09KiB/CPU-s 0.39 100.00% 31.22MiB/CPU-s 1.07MiB/CPU-s 14.16KiB/CPU-s 0.0 0.034339 31.34MiB/CPU-s 939.43KiB/CPU-s 12.12KiB/CPU-s 0.0 0.029271 False False
syslog_humio_logs ingress throughput 19.18KiB/CPU-s 0.21 99.99% 9.04MiB/CPU-s 245.14KiB/CPU-s 3.16KiB/CPU-s 0.0 0.026482 9.06MiB/CPU-s 304.49KiB/CPU-s 3.93KiB/CPU-s 0.0 0.032825 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 7.36KiB/CPU-s 0.20 72.90% 3.63MiB/CPU-s 348.49KiB/CPU-s 4.5KiB/CPU-s 0.0 0.093801 3.64MiB/CPU-s 383.69KiB/CPU-s 4.95KiB/CPU-s 0.0 0.10307 True True
syslog_loki ingress throughput 15.94KiB/CPU-s 0.18 99.84% 8.43MiB/CPU-s 282.22KiB/CPU-s 3.64KiB/CPU-s 0.0 0.032685 8.45MiB/CPU-s 270.49KiB/CPU-s 3.49KiB/CPU-s 0.0 0.031268 False False
syslog_log2metric_splunk_hec_metrics ingress throughput 13.63KiB/CPU-s 0.14 99.13% 9.3MiB/CPU-s 256.59KiB/CPU-s 3.31KiB/CPU-s 0.0 0.026934 9.32MiB/CPU-s 310.25KiB/CPU-s 4.0KiB/CPU-s 0.0 0.032521 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 31.8KiB/CPU-s 0.10 85.41% 32.37MiB/CPU-s 948.61KiB/CPU-s 12.24KiB/CPU-s 0.0 0.028616 32.4MiB/CPU-s 1.37MiB/CPU-s 18.12KiB/CPU-s 0.0 0.042304 False False
http_to_http_noack ingress throughput 10.54KiB/CPU-s 0.08 91.12% 13.61MiB/CPU-s 388.89KiB/CPU-s 5.02KiB/CPU-s 0.0 0.02791 13.62MiB/CPU-s 281.33KiB/CPU-s 3.63KiB/CPU-s 0.0 0.020176 False False
enterprise_http_to_http ingress throughput 4.07KiB/CPU-s 0.03 75.26% 13.62MiB/CPU-s 227.42KiB/CPU-s 2.94KiB/CPU-s 0.0 0.016305 13.62MiB/CPU-s 150.74KiB/CPU-s 1.95KiB/CPU-s 0.0 0.010804 False False
fluent_elasticsearch ingress throughput -374.84B/CPU-s -0.00 49.87% 45.41MiB/CPU-s 30.43KiB/CPU-s 397.8B/CPU-s 0.0 0.000654 45.41MiB/CPU-s 29.87KiB/CPU-s 390.48B/CPU-s 0.0 0.000642 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -730.78B/CPU-s -0.01 8.68% 13.61MiB/CPU-s 362.88KiB/CPU-s 4.68KiB/CPU-s 0.0 0.026028 13.61MiB/CPU-s 354.49KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025427 False False
splunk_hec_indexer_ack_blackhole ingress throughput -4.47KiB/CPU-s -0.03 66.81% 13.62MiB/CPU-s 240.51KiB/CPU-s 3.1KiB/CPU-s 0.0 0.017245 13.61MiB/CPU-s 263.72KiB/CPU-s 3.4KiB/CPU-s 0.0 0.018915 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -3.77KiB/CPU-s -0.03 58.04% 13.62MiB/CPU-s 242.62KiB/CPU-s 3.13KiB/CPU-s 0.0 0.017396 13.61MiB/CPU-s 268.94KiB/CPU-s 3.47KiB/CPU-s 0.0 0.019289 False False
splunk_hec_route_s3 ingress throughput -16.52KiB/CPU-s -0.14 78.36% 11.62MiB/CPU-s 743.04KiB/CPU-s 9.59KiB/CPU-s 0.0 0.062464 11.6MiB/CPU-s 721.75KiB/CPU-s 9.31KiB/CPU-s 0.0 0.060758 False False
datadog_agent_remap_blackhole_acks ingress throughput -130.14KiB/CPU-s -0.40 100.00% 31.41MiB/CPU-s 990.28KiB/CPU-s 12.78KiB/CPU-s 0.0 0.030786 31.28MiB/CPU-s 1002.45KiB/CPU-s 12.94KiB/CPU-s 0.0 0.031291 False False
syslog_splunk_hec_logs ingress throughput -41.81KiB/CPU-s -0.46 100.00% 8.94MiB/CPU-s 350.97KiB/CPU-s 4.53KiB/CPU-s 0.0 0.038345 8.9MiB/CPU-s 419.06KiB/CPU-s 5.41KiB/CPU-s 0.0 0.045993 False False
otlp_http_to_blackhole ingress throughput -12.68KiB/CPU-s -0.77 100.00% 1.6MiB/CPU-s 84.17KiB/CPU-s 1.09KiB/CPU-s 0.0 0.05144 1.59MiB/CPU-s 85.99KiB/CPU-s 1.11KiB/CPU-s 0.0 0.05296 False False
http_to_http_acks ingress throughput -44.36KiB/CPU-s -0.82 61.91% 5.26MiB/CPU-s 2.73MiB/CPU-s 36.1KiB/CPU-s 0.0 0.51903 5.22MiB/CPU-s 2.68MiB/CPU-s 35.48KiB/CPU-s 0.0 0.514294 True False
otlp_grpc_to_blackhole ingress throughput -8.91KiB/CPU-s -0.82 100.00% 1.06MiB/CPU-s 39.01KiB/CPU-s 515.66B/CPU-s 0.0 0.035985 1.05MiB/CPU-s 42.62KiB/CPU-s 563.36B/CPU-s 0.0 0.039639 False False
socket_to_socket_blackhole ingress throughput -121.21KiB/CPU-s -0.85 100.00% 13.85MiB/CPU-s 193.97KiB/CPU-s 2.5KiB/CPU-s 0.0 0.013678 13.73MiB/CPU-s 370.25KiB/CPU-s 4.78KiB/CPU-s 0.0 0.026334 False False
datadog_agent_remap_datadog_logs ingress throughput -300.66KiB/CPU-s -0.90 100.00% 32.8MiB/CPU-s 1.34MiB/CPU-s 17.76KiB/CPU-s 0.0 0.040971 32.5MiB/CPU-s 1.42MiB/CPU-s 18.77KiB/CPU-s 0.0 0.043687 False False
http_text_to_http_json ingress throughput -554.19KiB/CPU-s -2.12 100.00% 25.52MiB/CPU-s 868.98KiB/CPU-s 11.22KiB/CPU-s 0.0 0.033249 24.98MiB/CPU-s 672.44KiB/CPU-s 8.68KiB/CPU-s 0.0 0.026286 False False
syslog_log2metric_humio_metrics ingress throughput -225.17KiB/CPU-s -3.36 100.00% 6.54MiB/CPU-s 183.71KiB/CPU-s 2.37KiB/CPU-s 0.0 0.027422 6.32MiB/CPU-s 286.96KiB/CPU-s 3.7KiB/CPU-s 0.0 0.044324 False False
file_to_blackhole egress throughput -1.13MiB/CPU-s -15.68 100.00% 7.19MiB/CPU-s 4.23MiB/CPU-s 144.81KiB/CPU-s 3.128492 0.587797 6.07MiB/CPU-s 4.71MiB/CPU-s 134.57KiB/CPU-s 0.0 0.77694 True True

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@StephenWakely StephenWakely requested a review from a team April 13, 2023 14:59
@github-actions github-actions bot added the domain: transforms Anything related to Vector's transform components label Apr 13, 2023
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions
Copy link

Regression Detector Results

Run ID: 893ff5fa-5940-4389-80ee-e7b7ad60ce31
Baseline: db39d83
Comparison: 4123f82
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_regex_logs2metric_ddmetrics ingress throughput 93.44KiB/CPU-s 2.54 100.00% 3.6MiB/CPU-s 388.73KiB/CPU-s 5.02KiB/CPU-s 0.0 0.105504 3.69MiB/CPU-s 446.69KiB/CPU-s 5.76KiB/CPU-s 0.0 0.118238 True True
syslog_log2metric_humio_metrics ingress throughput 130.94KiB/CPU-s 2.00 100.00% 6.4MiB/CPU-s 257.61KiB/CPU-s 3.33KiB/CPU-s 0.0 0.039293 6.53MiB/CPU-s 360.08KiB/CPU-s 4.65KiB/CPU-s 0.0 0.053848 False False
otlp_grpc_to_blackhole ingress throughput 19.65KiB/CPU-s 1.90 100.00% 1.01MiB/CPU-s 51.68KiB/CPU-s 682.99B/CPU-s 0.0 0.050044 1.03MiB/CPU-s 50.51KiB/CPU-s 667.57B/CPU-s 0.0 0.048 False False
syslog_splunk_hec_logs ingress throughput 141.48KiB/CPU-s 1.54 100.00% 9.0MiB/CPU-s 321.86KiB/CPU-s 4.15KiB/CPU-s 0.0 0.034921 9.14MiB/CPU-s 186.07KiB/CPU-s 2.4KiB/CPU-s 0.0 0.019882 False False
splunk_hec_route_s3 ingress throughput 160.45KiB/CPU-s 1.33 100.00% 11.81MiB/CPU-s 612.44KiB/CPU-s 7.9KiB/CPU-s 0.0 0.050652 11.96MiB/CPU-s 542.66KiB/CPU-s 7.0KiB/CPU-s 0.0 0.044293 False False
syslog_humio_logs ingress throughput 99.1KiB/CPU-s 1.08 100.00% 8.93MiB/CPU-s 310.64KiB/CPU-s 4.01KiB/CPU-s 0.0 0.033971 9.03MiB/CPU-s 351.62KiB/CPU-s 4.54KiB/CPU-s 0.0 0.03804 False False
datadog_agent_remap_blackhole_acks ingress throughput 298.38KiB/CPU-s 0.96 100.00% 30.4MiB/CPU-s 1.95MiB/CPU-s 25.8KiB/CPU-s 0.0 0.064212 30.69MiB/CPU-s 1.34MiB/CPU-s 17.77KiB/CPU-s 0.0 0.0438 False False
http_to_http_acks ingress throughput 17.01KiB/CPU-s 0.32 26.25% 5.2MiB/CPU-s 2.72MiB/CPU-s 35.97KiB/CPU-s 0.0 0.523215 5.22MiB/CPU-s 2.71MiB/CPU-s 35.79KiB/CPU-s 0.0 0.518815 True False
otlp_http_to_blackhole ingress throughput 2.99KiB/CPU-s 0.19 81.38% 1.51MiB/CPU-s 124.63KiB/CPU-s 1.61KiB/CPU-s 0.0 0.080699 1.51MiB/CPU-s 123.27KiB/CPU-s 1.59KiB/CPU-s 0.0 0.079664 False False
enterprise_http_to_http ingress throughput 6.26KiB/CPU-s 0.04 89.66% 13.62MiB/CPU-s 256.55KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018396 13.62MiB/CPU-s 151.02KiB/CPU-s 1.95KiB/CPU-s 0.0 0.010824 False False
splunk_hec_indexer_ack_blackhole ingress throughput 921.14B/CPU-s 0.01 14.98% 13.61MiB/CPU-s 263.74KiB/CPU-s 3.4KiB/CPU-s 0.0 0.018916 13.62MiB/CPU-s 258.25KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018521 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 406.92B/CPU-s 0.00 4.87% 13.61MiB/CPU-s 352.99KiB/CPU-s 4.55KiB/CPU-s 0.0 0.02532 13.61MiB/CPU-s 359.75KiB/CPU-s 4.64KiB/CPU-s 0.0 0.025804 False False
fluent_elasticsearch ingress throughput 108.08B/CPU-s 0.00 15.37% 45.41MiB/CPU-s 30.15KiB/CPU-s 394.09B/CPU-s 0.0 0.000648 45.41MiB/CPU-s 30.17KiB/CPU-s 394.38B/CPU-s 0.0 0.000649 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -1.78KiB/CPU-s -0.01 29.28% 13.62MiB/CPU-s 255.77KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018342 13.62MiB/CPU-s 263.6KiB/CPU-s 3.4KiB/CPU-s 0.0 0.018906 False False
http_to_http_noack ingress throughput -5.99KiB/CPU-s -0.04 70.01% 13.62MiB/CPU-s 283.07KiB/CPU-s 3.65KiB/CPU-s 0.0 0.0203 13.61MiB/CPU-s 346.54KiB/CPU-s 4.47KiB/CPU-s 0.0 0.024863 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -102.08KiB/CPU-s -0.31 99.95% 31.81MiB/CPU-s 1.6MiB/CPU-s 21.18KiB/CPU-s 0.0 0.050396 31.71MiB/CPU-s 1.56MiB/CPU-s 20.57KiB/CPU-s 0.0 0.0491 False False
socket_to_socket_blackhole ingress throughput -49.9KiB/CPU-s -0.36 100.00% 13.42MiB/CPU-s 266.86KiB/CPU-s 3.44KiB/CPU-s 0.0 0.019418 13.37MiB/CPU-s 255.6KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018666 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -52.65KiB/CPU-s -0.55 100.00% 9.34MiB/CPU-s 292.23KiB/CPU-s 3.77KiB/CPU-s 0.0 0.030568 9.28MiB/CPU-s 353.31KiB/CPU-s 4.56KiB/CPU-s 0.0 0.037161 False False
http_to_http_json ingress throughput -78.65KiB/CPU-s -0.56 100.00% 13.62MiB/CPU-s 226.72KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016254 13.54MiB/CPU-s 323.25KiB/CPU-s 4.17KiB/CPU-s 0.0 0.023305 False False
http_text_to_http_json ingress throughput -377.5KiB/CPU-s -1.48 100.00% 24.97MiB/CPU-s 816.27KiB/CPU-s 10.53KiB/CPU-s 0.0 0.031915 24.61MiB/CPU-s 879.53KiB/CPU-s 11.35KiB/CPU-s 0.0 0.034903 False False
syslog_loki ingress throughput -155.98KiB/CPU-s -1.79 100.00% 8.5MiB/CPU-s 225.49KiB/CPU-s 2.91KiB/CPU-s 0.0 0.025901 8.35MiB/CPU-s 373.31KiB/CPU-s 4.82KiB/CPU-s 0.0 0.043664 False False
file_to_blackhole egress throughput -150.19KiB/CPU-s -2.20 54.63% 6.66MiB/CPU-s 4.32MiB/CPU-s 139.08KiB/CPU-s 4.935834 0.64864 6.51MiB/CPU-s 4.52MiB/CPU-s 144.29KiB/CPU-s 5.43162 0.694161 True True
datadog_agent_remap_blackhole ingress throughput -764.94KiB/CPU-s -2.44 100.00% 30.65MiB/CPU-s 1.54MiB/CPU-s 20.36KiB/CPU-s 0.0 0.050298 29.9MiB/CPU-s 1.72MiB/CPU-s 22.78KiB/CPU-s 0.0 0.057656 False False
datadog_agent_remap_datadog_logs ingress throughput -1.11MiB/CPU-s -3.42 100.00% 32.4MiB/CPU-s 1.34MiB/CPU-s 17.69KiB/CPU-s 0.0 0.0413 31.29MiB/CPU-s 1.64MiB/CPU-s 21.63KiB/CPU-s 0.0 0.052284 False False

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@StephenWakely
Copy link
Contributor Author

We had a discussion and came to the conclusion that this is not the way forward. Essentially we want the following:

  • Panic when something happens that should be impossible.
  • For a bug that is unrecoverable. No point in staying alive. Don't introduce degraded states. Blow up. :explode:
  • If possible, make this impossible leveraging the Rust type system.

Reloading configs is an area that still needs some thought as ideally we don't want reloading a config to result in a crash.

The issues caused by this panic are not in the panic itself, but rather indicative that the design of these traits are not quite correct.

@StephenWakely
Copy link
Contributor Author

I closed the wrong PR... :-(

@StephenWakely StephenWakely reopened this Apr 19, 2023
@github-actions github-actions bot added the domain: topology Anything related to Vector's topology code label Apr 19, 2023
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
This reverts commit 8ed6528.
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
…nitions

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions github-actions bot removed the domain: core Anything related to core crates i.e. vector-core, core-common, etc label Apr 25, 2023
@StephenWakely StephenWakely changed the base branch from master to stephen/return_transform_definitions April 25, 2023 16:19

/// If the VRL returns a value that is not an array (see [`merge_array_definitions`]),
/// or an object, that data is moved into the `message` field.
fn move_fields_into_message(mut definition: schema::Definition) -> schema::Definition {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this function name is totally accurate since we're just moving definitions, not the actual fields?

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@datadog-vectordotdev
Copy link

datadog-vectordotdev bot commented Apr 26, 2023

Datadog Report

Branch report: stephen/schema_event_field_panic
Commit report: 705d899

vector: 1 Failed (1 Known Flaky), 0 New Flaky, 2163 Passed, 0 Skipped, 9m 46.93s Wall Time

❌ Failed Tests (1)

  • sources::aws_s3::integration_tests::handles_errored_status - vector - ❄️ Known flaky - Details

    Expand for error
     Test has failed
    

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Copy link
Member

@fuchsnj fuchsnj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple small suggestions

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Base automatically changed from stephen/return_transform_definitions to master April 27, 2023 10:58
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions
Copy link

Regression Detector Results

Run ID: b3131e1f-c2d4-4de1-bd9c-c0c21d49883a
Baseline: 9031d0f
Comparison: 3e0bb2b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

  1. The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.

  2. Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean % Δ mean % CI confidence
syslog_log2metric_humio_metrics ingress throughput +1.98 [+1.87, +2.09] 100.00%
datadog_agent_remap_datadog_logs_acks ingress throughput +1.93 [+1.81, +2.05] 100.00%
syslog_loki ingress throughput +1.92 [+1.86, +1.98] 100.00%
syslog_log2metric_splunk_hec_metrics ingress throughput +1.88 [+1.81, +1.95] 100.00%
datadog_agent_remap_blackhole_acks ingress throughput +1.14 [+1.08, +1.21] 100.00%
datadog_agent_remap_datadog_logs ingress throughput +1.03 [+0.92, +1.13] 100.00%
datadog_agent_remap_blackhole ingress throughput +1.00 [+0.92, +1.09] 100.00%
splunk_hec_route_s3 ingress throughput +0.91 [+0.76, +1.05] 100.00%
otlp_grpc_to_blackhole ingress throughput +0.89 [+0.78, +1.00] 100.00%
syslog_humio_logs ingress throughput +0.83 [+0.73, +0.93] 100.00%
otlp_http_to_blackhole ingress throughput +0.62 [+0.47, +0.77] 100.00%
http_to_http_acks ingress throughput +0.43 [-0.78, +1.65] 35.24%
http_to_http_json ingress throughput +0.31 [+0.27, +0.35] 100.00%
socket_to_socket_blackhole ingress throughput +0.09 [+0.05, +0.12] 99.54%
enterprise_http_to_http ingress throughput +0.03 [-0.01, +0.07] 68.74%
splunk_hec_to_splunk_hec_logs_acks ingress throughput +0.00 [-0.06, +0.06] 0.33%
splunk_hec_indexer_ack_blackhole ingress throughput -0.00 [-0.04, +0.04] 0.41%
fluent_elasticsearch ingress throughput -0.00 [-0.00, +0.00] 58.58%
splunk_hec_to_splunk_hec_logs_noack ingress throughput -0.01 [-0.06, +0.04] 21.85%
http_to_http_noack ingress throughput -0.01 [-0.07, +0.04] 25.58%
syslog_splunk_hec_logs ingress throughput -0.72 [-0.80, -0.64] 100.00%
file_to_blackhole egress throughput -1.96 [-5.39, +1.47] 53.71%
http_text_to_http_json ingress throughput -2.68 [-2.73, -2.63] 100.00%
syslog_regex_logs2metric_ddmetrics ingress throughput -3.81 [-4.06, -3.57] 100.00%

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions github-actions bot added the domain: core Anything related to core crates i.e. vector-core, core-common, etc label Apr 27, 2023
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions
Copy link

Regression Detector Results

Run ID: a7f2cff6-3c6d-4c8e-8462-4373dd3a3d14
Baseline: 9031d0f
Comparison: 925f413
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

  1. The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.

  2. Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean % Δ mean % CI confidence
datadog_agent_remap_blackhole_acks ingress throughput +3.94 [+3.84, +4.05] 100.00%
file_to_blackhole egress throughput +3.86 [+0.34, +7.38] 83.97%
syslog_splunk_hec_logs ingress throughput +3.62 [+3.55, +3.69] 100.00%
syslog_humio_logs ingress throughput +3.39 [+3.33, +3.45] 100.00%
syslog_loki ingress throughput +2.94 [+2.86, +3.03] 100.00%
syslog_regex_logs2metric_ddmetrics ingress throughput +2.35 [+2.13, +2.58] 100.00%
syslog_log2metric_splunk_hec_metrics ingress throughput +2.07 [+2.00, +2.15] 100.00%
datadog_agent_remap_datadog_logs_acks ingress throughput +0.81 [+0.70, +0.92] 100.00%
datadog_agent_remap_blackhole ingress throughput +0.55 [+0.46, +0.64] 100.00%
http_to_http_noack ingress throughput +0.05 [-0.01, +0.10] 70.60%
splunk_hec_to_splunk_hec_logs_acks ingress throughput +0.03 [-0.03, +0.10] 52.11%
enterprise_http_to_http ingress throughput +0.03 [-0.00, +0.06] 79.87%
splunk_hec_to_splunk_hec_logs_noack ingress throughput +0.00 [-0.04, +0.05] 7.63%
splunk_hec_indexer_ack_blackhole ingress throughput +0.00 [-0.04, +0.05] 5.61%
fluent_elasticsearch ingress throughput -0.00 [-0.00, +0.00] 40.29%
http_to_http_acks ingress throughput -0.01 [-1.25, +1.23] 0.92%
http_to_http_json ingress throughput -0.09 [-0.13, -0.05] 99.71%
otlp_grpc_to_blackhole ingress throughput -0.42 [-0.53, -0.30] 100.00%
datadog_agent_remap_datadog_logs ingress throughput -0.42 [-0.51, -0.33] 100.00%
socket_to_socket_blackhole ingress throughput -2.18 [-2.23, -2.14] 100.00%
syslog_log2metric_humio_metrics ingress throughput -2.28 [-2.39, -2.18] 100.00%
otlp_http_to_blackhole ingress throughput -2.98 [-3.14, -2.82] 100.00%
splunk_hec_route_s3 ingress throughput -3.10 [-3.23, -2.97] 100.00%
http_text_to_http_json ingress throughput -4.32 [-4.39, -4.26] 100.00%

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions
Copy link

Regression Detector Results

Run ID: b263fa0f-0d97-4151-b987-ee9e2e9f9629
Baseline: 9031d0f
Comparison: dced80a
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

  1. The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.

  2. Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean % Δ mean % CI confidence
syslog_log2metric_splunk_hec_metrics ingress throughput +3.53 [+3.44, +3.61] 100.00%
file_to_blackhole egress throughput +2.90 [-0.44, +6.23] 73.37%
syslog_splunk_hec_logs ingress throughput +2.01 [+1.95, +2.07] 100.00%
syslog_loki ingress throughput +1.95 [+1.87, +2.02] 100.00%
syslog_humio_logs ingress throughput +1.83 [+1.77, +1.89] 100.00%
datadog_agent_remap_blackhole ingress throughput +0.65 [+0.57, +0.72] 100.00%
datadog_agent_remap_datadog_logs ingress throughput +0.61 [+0.50, +0.71] 100.00%
datadog_agent_remap_blackhole_acks ingress throughput +0.39 [+0.30, +0.47] 100.00%
syslog_regex_logs2metric_ddmetrics ingress throughput +0.22 [-0.01, +0.46] 77.70%
otlp_grpc_to_blackhole ingress throughput +0.18 [+0.06, +0.29] 95.46%
enterprise_http_to_http ingress throughput +0.03 [-0.00, +0.06] 78.81%
splunk_hec_to_splunk_hec_logs_noack ingress throughput +0.01 [-0.04, +0.05] 19.39%
fluent_elasticsearch ingress throughput +0.00 [-0.00, +0.00] 31.88%
splunk_hec_indexer_ack_blackhole ingress throughput -0.00 [-0.04, +0.04] 0.50%
splunk_hec_to_splunk_hec_logs_acks ingress throughput -0.01 [-0.07, +0.05] 9.64%
http_to_http_noack ingress throughput -0.02 [-0.08, +0.04] 30.48%
datadog_agent_remap_datadog_logs_acks ingress throughput -0.09 [-0.18, -0.00] 80.23%
splunk_hec_route_s3 ingress throughput -0.16 [-0.27, -0.04] 91.92%
http_to_http_json ingress throughput -0.19 [-0.23, -0.14] 100.00%
http_to_http_acks ingress throughput -0.68 [-1.90, +0.54] 52.59%
syslog_log2metric_humio_metrics ingress throughput -1.07 [-1.15, -0.99] 100.00%
socket_to_socket_blackhole ingress throughput -2.32 [-2.36, -2.29] 100.00%
http_text_to_http_json ingress throughput -2.85 [-2.91, -2.79] 100.00%
otlp_http_to_blackhole ingress throughput -3.44 [-3.62, -3.27] 100.00%

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
@github-actions
Copy link

Regression Detector Results

Run ID: 1eac8690-885e-4937-9238-6ef0568d050d
Baseline: 29c34c0
Comparison: 569994a
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

  1. The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.

  2. Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean % Δ mean % CI confidence
datadog_agent_remap_blackhole_acks ingress throughput +4.93 [+4.84, +5.02] 100.00%
file_to_blackhole egress throughput +4.87 [+1.40, +8.35] 92.78%
syslog_humio_logs ingress throughput +3.21 [+3.14, +3.27] 100.00%
datadog_agent_remap_datadog_logs ingress throughput +2.54 [+2.44, +2.64] 100.00%
syslog_log2metric_splunk_hec_metrics ingress throughput +1.64 [+1.56, +1.71] 100.00%
syslog_splunk_hec_logs ingress throughput +1.52 [+1.47, +1.58] 100.00%
syslog_loki ingress throughput +1.18 [+1.11, +1.26] 100.00%
http_text_to_http_json ingress throughput +0.87 [+0.81, +0.94] 100.00%
splunk_hec_route_s3 ingress throughput +0.46 [+0.33, +0.59] 100.00%
http_to_http_json ingress throughput +0.30 [+0.26, +0.35] 100.00%
datadog_agent_remap_blackhole ingress throughput +0.20 [+0.11, +0.30] 99.29%
http_to_http_noack ingress throughput +0.04 [-0.02, +0.11] 63.48%
enterprise_http_to_http ingress throughput +0.04 [+0.00, +0.08] 81.80%
splunk_hec_indexer_ack_blackhole ingress throughput +0.02 [-0.03, +0.06] 36.19%
splunk_hec_to_splunk_hec_logs_acks ingress throughput +0.00 [-0.06, +0.06] 2.86%
fluent_elasticsearch ingress throughput -0.00 [-0.00, +0.00] 45.99%
datadog_agent_remap_datadog_logs_acks ingress throughput -0.01 [-0.11, +0.09] 11.02%
splunk_hec_to_splunk_hec_logs_noack ingress throughput -0.02 [-0.07, +0.02] 43.94%
otlp_grpc_to_blackhole ingress throughput -0.37 [-0.48, -0.25] 100.00%
socket_to_socket_blackhole ingress throughput -0.51 [-0.56, -0.46] 100.00%
syslog_log2metric_humio_metrics ingress throughput -0.61 [-0.70, -0.52] 100.00%
http_to_http_acks ingress throughput -1.03 [-2.22, +0.17] 72.85%
otlp_http_to_blackhole ingress throughput -3.98 [-4.15, -3.81] 100.00%
syslog_regex_logs2metric_ddmetrics ingress throughput -4.17 [-4.41, -3.92] 100.00%

@StephenWakely StephenWakely merged commit 1e43208 into master Apr 27, 2023
@StephenWakely StephenWakely deleted the stephen/schema_event_field_panic branch April 27, 2023 16:07
aholmberg pushed a commit to aholmberg/vector that referenced this pull request Feb 14, 2024
# [1.13.0](answerbook/vector@v1.12.1...v1.13.0) (2023-09-13)

### Bug Fixes

* **appsignal sink**: Add TLS config option (vectordotdev#17122) [198068c](answerbook/vector@198068c) - GitHub
* **buffers**: correctly handle partial writes in reader seek during initialization (vectordotdev#17099) [a791595](answerbook/vector@a791595) - GitHub
* **config**: recurse through schema refs when determining eligibility for unevaluated properties (vectordotdev#17150) [71d1bf6](answerbook/vector@71d1bf6) - GitHub
* **docker_logs source**: Support tcp schema [e1c0c02](answerbook/vector@e1c0c02) - GitHub
* **elasticsearch sink**:  Elasticsearch sink with api_version set to "auto" does not recognize the API version of ES6 as V6 (vectordotdev#17226) (vectordotdev#17227) [9b6ef24](answerbook/vector@9b6ef24) - GitHub
* **gcp_stackdriver_metrics sink**: Call function to regenerate auth token (vectordotdev#17297) [bf7904b](answerbook/vector@bf7904b) - GitHub
* **influxdb_logs**: encode influx line when no tags present (vectordotdev#17029) [c3aa14f](answerbook/vector@c3aa14f) - GitHub
* **reduce transform**: Revert flushing on interval change to `expire_metrics_ms` (vectordotdev#17084) [e86b155](answerbook/vector@e86b155) - GitHub
* **releasing**: Fix globbing of release artifacts for GitHub (vectordotdev#17114) [7fe089c](answerbook/vector@7fe089c) - GitHub
* **schemas**: Dont panic with non object field kinds (vectordotdev#17140) [1e43208](answerbook/vector@1e43208) - GitHub

### Chores

* (syslog source): add source_ip to some syslog tests (vectordotdev#17235) [29c34c0](answerbook/vector@29c34c0) - GitHub
* add note to DEVELOPING.md re panics (vectordotdev#17277) [03e905e](answerbook/vector@03e905e) - GitHub
* Add UX note about encoding of log_schema keys (vectordotdev#17266) [dc6e54c](answerbook/vector@dc6e54c) - GitHub
* **administration**: add `appsignal` to codeowners (vectordotdev#17127) [7b15d19](answerbook/vector@7b15d19) - GitHub
* **buffer**: tidy up some of the module level docs for `disk_v2` (vectordotdev#17093) [edaa612](answerbook/vector@edaa612) - GitHub
* **ci**: bump docker/metadata-action from 4.3.0 to 4.4.0 (vectordotdev#17170) [854d71e](answerbook/vector@854d71e) - GitHub
* **ci**: Disable `appsignal` integration test until CA issues are resolved (vectordotdev#17109) [f3b5d42](answerbook/vector@f3b5d42) - GitHub
* **ci**: Disable scheduled runs of Baseline Timings workflow (vectordotdev#17281) [4335b0a](answerbook/vector@4335b0a) - GitHub
* **ci**: Fix event assertions for `aws_ec2_metadata` transform (vectordotdev#17413) [da36fb6](answerbook/vector@da36fb6) - GitHub
* **ci**: Increase timeout for integration tests (vectordotdev#17326) [e1f125a](answerbook/vector@e1f125a) - GitHub
* **ci**: Increase timeout for integration tests to 30m (vectordotdev#17350) [5d3f619](answerbook/vector@5d3f619) - GitHub
* **ci**: re-enable `appsignal` integration test (vectordotdev#17111) [48fc574](answerbook/vector@48fc574) - GitHub
* **ci**: Remove ci-sweep tasks (vectordotdev#17415) [5c33f99](answerbook/vector@5c33f99) - GitHub
* **ci**: remove unnecessary dep install (vectordotdev#17128) [f56d1ef](answerbook/vector@f56d1ef) - GitHub
* **ci**: Try to fix apt retries (vectordotdev#17393) [6b3db04](answerbook/vector@6b3db04) - GitHub
* **ci**: update unsupported ubuntu version runners (vectordotdev#17113) [e7c4815](answerbook/vector@e7c4815) - GitHub
* **ci**: use python v3.8 in ubuntu 20.04 runner (vectordotdev#17116) [7a40c81](answerbook/vector@7a40c81) - GitHub
* **config**: begin laying out primitives for programmatically querying schema (vectordotdev#17130) [aad8115](answerbook/vector@aad8115) - GitHub
* **config**: emit human-friendly version of enum variant/property names in schema (vectordotdev#17171) [3b38ba8](answerbook/vector@3b38ba8) - GitHub
* **config**: improve config schema output with more precise `unevaluatedProperties` + schema ref flattening (vectordotdev#17026) [2d72f82](answerbook/vector@2d72f82) - GitHub
* **deps**: Add 3rd party license file and CI checks (vectordotdev#17344) [7350e1a](answerbook/vector@7350e1a) - GitHub
* **deps**: bump anyhow from 1.0.70 to 1.0.71 (vectordotdev#17300) [6a5af3b](answerbook/vector@6a5af3b) - GitHub
* **deps**: bump assert_cmd from 2.0.10 to 2.0.11 (vectordotdev#17290) [c4784fd](answerbook/vector@c4784fd) - GitHub
* **deps**: bump async-compression from 0.3.15 to 0.4.0 (vectordotdev#17365) [b9aac47](answerbook/vector@b9aac47) - GitHub
* **deps**: bump async-graphql from 5.0.7 to 5.0.8 (vectordotdev#17357) [05a4f17](answerbook/vector@05a4f17) - GitHub
* **deps**: bump async-graphql-warp from 5.0.7 to 5.0.8 (vectordotdev#17367) [693584e](answerbook/vector@693584e) - GitHub
* **deps**: bump async-stream from 0.3.4 to 0.3.5 (vectordotdev#17076) [c29c817](answerbook/vector@c29c817) - GitHub
* **deps**: bump aws-sigv4 from 0.55.0 to 0.55.1 (vectordotdev#17138) [dbb3f25](answerbook/vector@dbb3f25) - GitHub
* **deps**: bump axum from 0.6.12 to 0.6.18 (vectordotdev#17257) [41ac76e](answerbook/vector@41ac76e) - GitHub
* **deps**: bump cached from 0.42.0 to 0.43.0 (vectordotdev#17118) [f90b3b3](answerbook/vector@f90b3b3) - GitHub
* **deps**: bump chrono-tz from 0.8.1 to 0.8.2 (vectordotdev#17088) [623b838](answerbook/vector@623b838) - GitHub
* **deps**: bump clap_complete from 4.2.0 to 4.2.1 (vectordotdev#17229) [d286d16](answerbook/vector@d286d16) - GitHub
* **deps**: bump clap_complete from 4.2.1 to 4.2.2 (vectordotdev#17359) [565668e](answerbook/vector@565668e) - GitHub
* **deps**: bump clap_complete from 4.2.2 to 4.2.3 (vectordotdev#17383) [111cd07](answerbook/vector@111cd07) - GitHub
* **deps**: bump console-subscriber from 0.1.8 to 0.1.9 (vectordotdev#17358) [97b862c](answerbook/vector@97b862c) - GitHub
* **deps**: bump directories from 5.0.0 to 5.0.1 (vectordotdev#17271) [be69f5f](answerbook/vector@be69f5f) - GitHub
* **deps**: bump dunce from 1.0.3 to 1.0.4 (vectordotdev#17244) [cfc387d](answerbook/vector@cfc387d) - GitHub
* **deps**: bump enumflags2 from 0.7.5 to 0.7.6 (vectordotdev#17079) [cbc17be](answerbook/vector@cbc17be) - GitHub
* **deps**: bump enumflags2 from 0.7.6 to 0.7.7 (vectordotdev#17206) [c80c5eb](answerbook/vector@c80c5eb) - GitHub
* **deps**: bump flate2 from 1.0.25 to 1.0.26 (vectordotdev#17320) [ef13370](answerbook/vector@ef13370) - GitHub
* **deps**: bump getrandom from 0.2.8 to 0.2.9 (vectordotdev#17101) [d53240b](answerbook/vector@d53240b) - GitHub
* **deps**: bump h2 from 0.3.18 to 0.3.19 (vectordotdev#17388) [6088abd](answerbook/vector@6088abd) - GitHub
* **deps**: bump hashlink from 0.8.1 to 0.8.2 (vectordotdev#17419) [01b3cd7](answerbook/vector@01b3cd7) - GitHub
* **deps**: bump hyper from 0.14.25 to 0.14.26 (vectordotdev#17347) [c43dcfd](answerbook/vector@c43dcfd) - GitHub
* **deps**: bump inventory from 0.3.5 to 0.3.6 (vectordotdev#17401) [5b5ad16](answerbook/vector@5b5ad16) - GitHub
* **deps**: bump libc from 0.2.140 to 0.2.141 (vectordotdev#17104) [dd9608a](answerbook/vector@dd9608a) - GitHub
* **deps**: bump libc from 0.2.141 to 0.2.142 (vectordotdev#17273) [bc618a2](answerbook/vector@bc618a2) - GitHub
* **deps**: bump libc from 0.2.142 to 0.2.143 (vectordotdev#17338) [6afe206](answerbook/vector@6afe206) - GitHub
* **deps**: bump libc from 0.2.143 to 0.2.144 (vectordotdev#17346) [99b8dc1](answerbook/vector@99b8dc1) - GitHub
* **deps**: bump memmap2 from 0.5.10 to 0.6.0 (vectordotdev#17355) [dae0c6a](answerbook/vector@dae0c6a) - GitHub
* **deps**: bump memmap2 from 0.6.0 to 0.6.1 (vectordotdev#17364) [58ba741](answerbook/vector@58ba741) - GitHub
* **deps**: bump metrics, metrics-tracing-context, metrics-util (vectordotdev#17336) [9a723e3](answerbook/vector@9a723e3) - GitHub
* **deps**: bump mlua from 0.8.8 to 0.8.9 (vectordotdev#17423) [57f8bd4](answerbook/vector@57f8bd4) - GitHub
* **deps**: bump mock_instant from 0.2.1 to 0.3.0 (vectordotdev#17210) [40c9afc](answerbook/vector@40c9afc) - GitHub
* **deps**: bump mongodb from 2.4.0 to 2.5.0 (vectordotdev#17337) [64f4f69](answerbook/vector@64f4f69) - GitHub
* **deps**: bump nkeys from 0.2.0 to 0.3.0 (vectordotdev#17421) [3320eda](answerbook/vector@3320eda) - GitHub
* **deps**: bump notify from 5.1.0 to 6.0.0 (vectordotdev#17422) [58603b9](answerbook/vector@58603b9) - GitHub
* **deps**: bump num_enum from 0.5.11 to 0.6.0 (vectordotdev#17106) [42f298b](answerbook/vector@42f298b) - GitHub
* **deps**: bump num_enum from 0.6.0 to 0.6.1 (vectordotdev#17272) [f696e7b](answerbook/vector@f696e7b) - GitHub
* **deps**: bump opendal from 0.30.5 to 0.31.0 (vectordotdev#17119) [8762563](answerbook/vector@8762563) - GitHub
* **deps**: bump opendal from 0.31.0 to 0.33.2 (vectordotdev#17286) [3d41931](answerbook/vector@3d41931) - GitHub
* **deps**: bump opendal from 0.33.2 to 0.34.0 (vectordotdev#17354) [ae602da](answerbook/vector@ae602da) - GitHub
* **deps**: bump openssl from 0.10.48 to 0.10.50 (vectordotdev#17087) [9a56ed8](answerbook/vector@9a56ed8) - GitHub
* **deps**: bump openssl from 0.10.50 to 0.10.52 (vectordotdev#17299) [0ecceb3](answerbook/vector@0ecceb3) - GitHub
* **deps**: bump pin-project from 1.0.12 to 1.1.0 (vectordotdev#17385) [e8d3002](answerbook/vector@e8d3002) - GitHub
* **deps**: bump prettydiff from 0.6.2 to 0.6.4 (vectordotdev#17089) [e090610](answerbook/vector@e090610) - GitHub
* **deps**: bump prettydiff from 0.6.2 to 0.6.4 (vectordotdev#17315) [a1ec68d](answerbook/vector@a1ec68d) - GitHub
* **deps**: bump proc-macro2 from 1.0.55 to 1.0.56 (vectordotdev#17103) [6f74523](answerbook/vector@6f74523) - GitHub
* **deps**: bump proc-macro2 from 1.0.56 to 1.0.57 (vectordotdev#17400) [a6e1ae7](answerbook/vector@a6e1ae7) - GitHub
* **deps**: bump prost-build from 0.11.8 to 0.11.9 (vectordotdev#17260) [a88aba4](answerbook/vector@a88aba4) - GitHub
* **deps**: bump quote from 1.0.26 to 1.0.27 (vectordotdev#17348) [f81ff18](answerbook/vector@f81ff18) - GitHub
* **deps**: bump rdkafka from 0.29.0 to 0.30.0 (vectordotdev#17387) [9703188](answerbook/vector@9703188) - GitHub
* **deps**: bump regex from 1.7.3 to 1.8.1 (vectordotdev#17222) [410aa3c](answerbook/vector@410aa3c) - GitHub
* **deps**: bump reqwest from 0.11.16 to 0.11.17 (vectordotdev#17316) [09176ec](answerbook/vector@09176ec) - GitHub
* **deps**: bump security-framework from 2.8.2 to 2.9.0 (vectordotdev#17386) [1287168](answerbook/vector@1287168) - GitHub
* **deps**: bump serde from 1.0.159 to 1.0.160 (vectordotdev#17270) [036ad4a](answerbook/vector@036ad4a) - GitHub
* **deps**: bump serde from 1.0.160 to 1.0.162 (vectordotdev#17317) [79e97a2](answerbook/vector@79e97a2) - GitHub
* **deps**: bump serde from 1.0.162 to 1.0.163 (vectordotdev#17366) [9852c17](answerbook/vector@9852c17) - GitHub
* **deps**: bump serde_json from 1.0.95 to 1.0.96 (vectordotdev#17258) [7570bb3](answerbook/vector@7570bb3) - GitHub
* **deps**: bump serde_with from 2.3.1 to 2.3.2 (vectordotdev#17090) [adbf4d5](answerbook/vector@adbf4d5) - GitHub
* **deps**: bump serde_yaml from 0.9.19 to 0.9.21 (vectordotdev#17120) [d6f2625](answerbook/vector@d6f2625) - GitHub
* **deps**: bump socket2 from 0.4.7 to 0.5.2 (vectordotdev#17121) [db39d83](answerbook/vector@db39d83) - GitHub
* **deps**: bump socket2 from 0.5.2 to 0.5.3 (vectordotdev#17384) [ac51b8a](answerbook/vector@ac51b8a) - GitHub
* **deps**: bump syslog from 6.0.1 to 6.1.0 (vectordotdev#17301) [61e6154](answerbook/vector@61e6154) - GitHub
* **deps**: bump tokio from 1.27.0 to 1.28.0 (vectordotdev#17231) [8067f84](answerbook/vector@8067f84) - GitHub
* **deps**: bump tokio from 1.28.0 to 1.28.1 (vectordotdev#17368) [ae6a51b](answerbook/vector@ae6a51b) - GitHub
* **deps**: bump tokio-stream from 0.1.12 to 0.1.14 (vectordotdev#17339) [80c8247](answerbook/vector@80c8247) - GitHub
* **deps**: bump tokio-tungstenite from 0.18.0 to 0.19.0 (vectordotdev#17404) [ae1dd6e](answerbook/vector@ae1dd6e) - GitHub
* **deps**: bump tonic from 0.8.3 to 0.9.1 (vectordotdev#17077) [eafba69](answerbook/vector@eafba69) - GitHub
* **deps**: bump tonic from 0.9.1 to 0.9.2 (vectordotdev#17221) [aa9cbd0](answerbook/vector@aa9cbd0) - GitHub
* **deps**: bump tonic-build from 0.8.4 to 0.9.2 (vectordotdev#17274) [e0a07c6](answerbook/vector@e0a07c6) - GitHub
* **deps**: bump tracing-subscriber from 0.3.16 to 0.3.17 (vectordotdev#17268) [1406c08](answerbook/vector@1406c08) - GitHub
* **deps**: bump typetag from 0.2.7 to 0.2.8 (vectordotdev#17302) [c8e0e5f](answerbook/vector@c8e0e5f) - GitHub
* **deps**: bump uuid from 1.3.0 to 1.3.1 (vectordotdev#17091) [9cc2f1d](answerbook/vector@9cc2f1d) - GitHub
* **deps**: bump uuid from 1.3.0 to 1.3.2 (vectordotdev#17256) [bc6f7fd](answerbook/vector@bc6f7fd) - GitHub
* **deps**: bump uuid from 1.3.2 to 1.3.3 (vectordotdev#17403) [3a3fe63](answerbook/vector@3a3fe63) - GitHub
* **deps**: bump warp from 0.3.4 to 0.3.5 (vectordotdev#17288) [d8c1f12](answerbook/vector@d8c1f12) - GitHub
* **deps**: bump wasm-bindgen from 0.2.84 to 0.2.85 (vectordotdev#17356) [ea24b4d](answerbook/vector@ea24b4d) - GitHub
* **deps**: bump wasm-bindgen from 0.2.85 to 0.2.86 (vectordotdev#17402) [0518176](answerbook/vector@0518176) - GitHub
* **deps**: bump wiremock from 0.5.17 to 0.5.18 (vectordotdev#17092) [51312aa](answerbook/vector@51312aa) - GitHub
* **deps**: Fix up missing license (vectordotdev#17379) [a2b8903](answerbook/vector@a2b8903) - GitHub
* **deps**: Reset dependencies bumped by a61dea1 (vectordotdev#17100) [887d6d7](answerbook/vector@887d6d7) - GitHub
* **deps**: true up cargo.lock (vectordotdev#17149) [10fce65](answerbook/vector@10fce65) - GitHub
* **deps**: Update h2 (vectordotdev#17189) [a2882f3](answerbook/vector@a2882f3) - GitHub
* **deps**: Upgrade cue to 0.5.0 (vectordotdev#17204) [d396320](answerbook/vector@d396320) - GitHub
* **deps**: Upgrade Debian to bullseye for distroless image (vectordotdev#17160) [c304a8c](answerbook/vector@c304a8c) - GitHub
* **deps**: Upgrade rust to 1.69.0 (vectordotdev#17194) [ef15696](answerbook/vector@ef15696) - GitHub
* **dev**: Add note about generating licenses to CONTRIBUTING.md (vectordotdev#17410) [539f379](answerbook/vector@539f379) - GitHub
* **dev**: ignore `.helix` dir (vectordotdev#17203) [32a935b](answerbook/vector@32a935b) - GitHub
* **dev**: Install the correct `mold` based on CPU architecture (vectordotdev#17248) [4b80c71](answerbook/vector@4b80c71) - GitHub
* **dev**: remove editors from gitignore (vectordotdev#17267) [61c0d76](answerbook/vector@61c0d76) - GitHub
* **docs**: Add Enterprise link and update Support link (vectordotdev#17408) [5184d50](answerbook/vector@5184d50) - GitHub
* **docs**: Add missing 0.28.2 version [38607cd](answerbook/vector@38607cd) - Jesse Szwedko
* **docs**: Clarify `key_field` for `sample` and `throttle` transforms (vectordotdev#17372) [d1e5588](answerbook/vector@d1e5588) - GitHub
* **docs**: Document event type conditions (vectordotdev#17311) [a9c8dc8](answerbook/vector@a9c8dc8) - GitHub
* **docs**: make doc style edits (vectordotdev#17155) [65a8856](answerbook/vector@65a8856) - GitHub
* **docs**: Remove trailing, unmatched quote (vectordotdev#17163) [3c92556](answerbook/vector@3c92556) - GitHub
* **docs**: Remove unneeded `yaml` dependency from website (vectordotdev#17215) [752d424](answerbook/vector@752d424) - GitHub
* **docs**: Update component statuses 2023Q2 (vectordotdev#17362) [22cda94](answerbook/vector@22cda94) - GitHub
* **docs**: update the `v0.28.0` upgrade guide with note about `datadog_logs` sink `hostname` key (vectordotdev#17156) [c169131](answerbook/vector@c169131) - GitHub
* **external docs**: correctly mark some sinks as stateful (vectordotdev#17085) [64d560d](answerbook/vector@64d560d) - GitHub
* **loki sink**: warn on label expansions and collisions (vectordotdev#17052) [f06692b](answerbook/vector@f06692b) - GitHub
* **pulsar**: pulsar-rs bump to v5.1.1 (vectordotdev#17159) [68b54a9](answerbook/vector@68b54a9) - GitHub
* Re-add transform definitions (vectordotdev#17152) [9031d0f](answerbook/vector@9031d0f) - GitHub
* Regen docs for sample and throttle (vectordotdev#17390) [6c57ca0](answerbook/vector@6c57ca0) - GitHub
* **releasing**: Add known issues fixed by 0.29.1 (vectordotdev#17218) [40d543a](answerbook/vector@40d543a) - GitHub
* **releasing**: Bump Vector version to 0.30.0 (vectordotdev#17134) [3834612](answerbook/vector@3834612) - GitHub
* **releasing**: Fix homebrew release script (vectordotdev#17131) [cfbf233](answerbook/vector@cfbf233) - Jesse Szwedko
* **releasing**: Fix release channels (vectordotdev#17133) [58b44e8](answerbook/vector@58b44e8) - Jesse Szwedko
* **releasing**: Prepare v0.28.2 release [a61dea1](answerbook/vector@a61dea1) - Jesse Szwedko
* **releasing**: Prepare v0.29.0 release [4bf6805](answerbook/vector@4bf6805) - Jesse Szwedko
* **releasing**: Prepare v0.30.0 release [38c3f0b](answerbook/vector@38c3f0b) - Jesse Szwedko
* **releasing**: Regenerate Kubernetes manifests for 0.21.2 (vectordotdev#17108) [fd13d64](answerbook/vector@fd13d64) - GitHub
* **releasing**: Regenerate manifests for 0.21.1 chart (vectordotdev#17187) [1f0de6b](answerbook/vector@1f0de6b) - GitHub
* **releasing**: Regenerate manifests for 0.22.0 chart (vectordotdev#17135) [e7ea0a8](answerbook/vector@e7ea0a8) - GitHub
* **releasing**: update patch release template with extra step details [27c3526](answerbook/vector@27c3526) - GitHub
* Remove skaffold from project (vectordotdev#17145) [d245927](answerbook/vector@d245927) - GitHub
* remove transform type coercion (vectordotdev#17411) [b6c7e0a](answerbook/vector@b6c7e0a) - GitHub
* Revert transform definitions (vectordotdev#17146) [05a3f44](answerbook/vector@05a3f44) - GitHub
* **socket source**: Remove deprecated `max_length` setting from `tcp` and `unix` modes. (vectordotdev#17162) [9ecfc8c](answerbook/vector@9ecfc8c) - GitHub
* **syslog source**: remove the remove of source_ip (vectordotdev#17184) [5dff0ed](answerbook/vector@5dff0ed) - GitHub
* **topology**: Transform outputs hash table of OutputId -> Definition (vectordotdev#17059) [1bdb24d](answerbook/vector@1bdb24d) - GitHub
* Upgrade `VRL` to `0.3.0` (vectordotdev#17325) [4911d36](answerbook/vector@4911d36) - GitHub

### Features

* adds 'metric_name' field to internal logs for the tag_cardinality_limit component (vectordotdev#17295) [4317340](answerbook/vector@4317340) - GitHub
* **codecs**: Add full codec support to AWS S3 source/sink (vectordotdev#17098) [d648c86](answerbook/vector@d648c86) - GitHub
* **kubernetes_logs**: use kube-apiserver cache for list requests (vectordotdev#17095) [e46fae7](answerbook/vector@e46fae7) - GitHub
* Merge with upstream v0.30.0 [6b93177](answerbook/vector@6b93177) - GitHub [LOG-17997](https://logdna.atlassian.net/browse/LOG-17997)
* **new sink**: Initial `datadog_events` sink (vectordotdev#7678) [fef3810](answerbook/vector@fef3810) - Jesse Szwedko
* Update VRL library [6ace1e6](answerbook/vector@6ace1e6) - Jorge Bay

### Miscellaneous

* Merge branch 'master' [d4b35bb](answerbook/vector@d4b35bb) - Jorge Bay
* Merge tag 'v0.30.0' into update-upstream [ee2f300](answerbook/vector@ee2f300) - Jorge Bay
* Merge commit '9031d0faa2811187874364e1b5a3305c9ed0c0da' into update-upstream [c12faae](answerbook/vector@c12faae) - Jorge Bay
* Improve tokio::select behavior for kafka source and finalizers (vectordotdev#17380) [d4df21c](answerbook/vector@d4df21c) - GitHub
* Prepare v0.29.1 release [21beed7](answerbook/vector@21beed7) - Kyle Criddle
* Add a quickfix to handle special capitalization cases (vectordotdev#17141) [ba63e21](answerbook/vector@ba63e21) - GitHub
* Adjust doc comment locations (vectordotdev#17154) [730c938](answerbook/vector@730c938) - GitHub
* **amqp sink**: Support AMQ Properties (content-type) in AMQP sink (vectordotdev#17174) [c10d30b](answerbook/vector@c10d30b) - GitHub
* **aws provider**: Let `region` be configured for default authentication (vectordotdev#17414) [c81ad30](answerbook/vector@c81ad30) - GitHub
* **core**: add unit test (ignored) for support for encoding special chars in `ProxyConfig` (vectordotdev#17148) [5247972](answerbook/vector@5247972) - GitHub
* **loki sink**: Fix formatting in labels example (vectordotdev#17396) [f3734e8](answerbook/vector@f3734e8) - GitHub
* **observability**: Log underlying error for unhandled HTTP errors (vectordotdev#17327) [bf8376c](answerbook/vector@bf8376c) - GitHub
* **observability**: Update internal log rate limiting messages (vectordotdev#17394) [1951535](answerbook/vector@1951535) - GitHub
* **pulsar sink**: Refactor to use StreamSink (vectordotdev#14345) [1e97a2f](answerbook/vector@1e97a2f) - GitHub
* **topology**: Add source id to metadata (vectordotdev#17369) [c683999](answerbook/vector@c683999) - GitHub
* **vdev**: Load compose files and inject network block (vectordotdev#17025) [5d88655](answerbook/vector@5d88655) - GitHub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: topology Anything related to Vector's topology code domain: transforms Anything related to Vector's transform components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants