[Endpoint] Sample data generator for endpoint app#58936
Merged
marshallmain merged 31 commits intoelastic:masterfrom Mar 10, 2020
Merged
[Endpoint] Sample data generator for endpoint app#58936marshallmain merged 31 commits intoelastic:masterfrom
marshallmain merged 31 commits intoelastic:masterfrom
Conversation
Contributor
|
Pinging @elastic/endpoint-app-team (Feature:Endpoint) |
Contributor
|
Pinging @elastic/endpoint-response (Team:Endpoint Response) |
dplumlee
reviewed
Mar 3, 2020
| for (let i = 0; i < generations; i++) { | ||
| const newParents: EndpointEvent[] = []; | ||
| parents.forEach(element => { | ||
| // const numChildren = randomN(maxChildrenPerNode); |
Contributor
There was a problem hiding this comment.
you might still be working on stuff, but probably can delete this
Contributor
Author
There was a problem hiding this comment.
oops yeah i need to replace the random generator with a seeded random generator so we can have determinism but also a simple way to get variety
dplumlee
reviewed
Mar 3, 2020
| sid?: string; | ||
| start: number; | ||
| malware_classifier: MalwareClassifierFields; | ||
| malware_classifier?: MalwareClassifierFields; |
Contributor
There was a problem hiding this comment.
just a general question, what was the criteria of making some of these fields optional?
Contributor
Author
There was a problem hiding this comment.
I made as many things optional as I could without breaking the type checker
Contributor
Author
|
@elasticmachine merge upstream |
Contributor
Author
|
@elasticmachine merge upstream |
dplumlee
approved these changes
Mar 10, 2020
Contributor
💛 Build succeeded, but was flaky
Test FailuresKibana Pipeline / kibana-xpack-agent / X-Pack Spaces API Integration Tests -- security_and_spaces.x-pack/test/spaces_api_integration/security_and_spaces/apis/resolve_copy_to_space_conflicts·ts.spaces api with security resolve copy to spaces conflicts user with no access from the default space "before each" hook for "should return 404 when overwriting, with references"Standard OutStack TraceKibana Pipeline / kibana-xpack-agent / X-Pack Spaces API Integration Tests -- security_and_spaces.x-pack/test/spaces_api_integration/security_and_spaces/apis/resolve_copy_to_space_conflicts·ts.spaces api with security resolve copy to spaces conflicts user with no access from the default space "before each" hook for "should return 404 when overwriting, with references"Standard OutStack TraceKibana Pipeline / kibana-xpack-agent / X-Pack Spaces API Integration Tests -- security_and_spaces.x-pack/test/spaces_api_integration/security_and_spaces/apis/resolve_copy_to_space_conflicts·ts.spaces api with security resolve copy to spaces conflicts user with no access from the default space "after each" hook for "should return 404 when overwriting, with references"Standard OutStack TraceHistory
To update your PR or re-run it, just comment with: |
peluja1012
reviewed
Mar 10, 2020
| "dependencies": { | ||
| "react-redux": "^7.1.0" | ||
| "react-redux": "^7.1.0", | ||
| "seedrandom": "^3.0.5" |
Contributor
There was a problem hiding this comment.
This library seems to already be included in Kibana. https://github.com/elastic/kibana/blob/master/package.json#L255
peluja1012
reviewed
Mar 10, 2020
| }, | ||
| }, | ||
| }; | ||
| return generator.generateEndpointMetadata(new Date().getTime()); |
Contributor
There was a problem hiding this comment.
You could just do generator.generateEndpointMetadata() since you added a default date.
peluja1012
reviewed
Mar 10, 2020
| }, | ||
| }, | ||
| }; | ||
| return generator.generateEndpointMetadata(new Date().getTime()); |
Contributor
There was a problem hiding this comment.
You could just do generator.generateEndpointMetadata() since you added a default date.
gmmorris
added a commit
to gmmorris/kibana
that referenced
this pull request
Mar 10, 2020
* master: Add a retry to dashboard test for a sometimes slow async operation (elastic#59600) [Endpoint] Sample data generator for endpoint app (elastic#58936) [Vis Editor] Refactoring metrics axes (elastic#59135) [DOCS] Changed Discover app to Discover (elastic#59769) [Metrics Alerts] Add support for search query and groupBy on alerts (elastic#59388) Enhancement - EUICodeEditor for Visualize JSON (elastic#58679) [ML] Transforms: Data grid fixes. (elastic#59538) [SIEM] Fix and consolidate handling of error responses in the client (elastic#59438) [Maps] convert tooltip classes to typescript (elastic#59589) [ML] Functional tests - re-activate date_nanos test (elastic#59649) Move canvas to use NP Expressions service (elastic#58387) Update misc dependencies (elastic#59542) [Unit Testing] Configure react-testing-library queries to use Kibana's data-test-subj instead of default data-testid (elastic#59445) [Console] Remove unused code (elastic#59554) [Logs / Metrics UI] Link handling / stop page reloads (elastic#58478) Add SavedObject management section registration in core (elastic#59291)
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
This was referenced Mar 11, 2020
marshallmain
added a commit
to marshallmain/kibana
that referenced
this pull request
Mar 12, 2020
* scaffolding and notes.md * add skeleton event generator to kibana * add optional entityID param to generateEvent * add tree generation * add tests * working tests * fix up tests * fix linting * fix event types * make process parent types consistent * make generator match types * move test resolver node out of common types * fix random string generation * fix typecheck errors * remove extraneous stuff * address PR comments * add test for full resolver tree * cleanup * make tests clearer * add seedrandom to endpoint plugin. contains DONOTMERGE example code * remove robs test * start replacing random with seedrandom * use seeded random for uuidv4 * separate out IP randomization * typecheck fixes Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
marshallmain
added a commit
to marshallmain/kibana
that referenced
this pull request
Mar 13, 2020
* scaffolding and notes.md * add skeleton event generator to kibana * add optional entityID param to generateEvent * add tree generation * add tests * working tests * fix up tests * fix linting * fix event types * make process parent types consistent * make generator match types * move test resolver node out of common types * fix random string generation * fix typecheck errors * remove extraneous stuff * address PR comments * add test for full resolver tree * cleanup * make tests clearer * add seedrandom to endpoint plugin. contains DONOTMERGE example code * remove robs test * start replacing random with seedrandom * use seeded random for uuidv4 * separate out IP randomization * typecheck fixes Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Contributor
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
1 similar comment
Contributor
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
marshallmain
added a commit
that referenced
this pull request
Mar 16, 2020
* scaffolding and notes.md * add skeleton event generator to kibana * add optional entityID param to generateEvent * add tree generation * add tests * working tests * fix up tests * fix linting * fix event types * make process parent types consistent * make generator match types * move test resolver node out of common types * fix random string generation * fix typecheck errors * remove extraneous stuff * address PR comments * add test for full resolver tree * cleanup * make tests clearer * add seedrandom to endpoint plugin. contains DONOTMERGE example code * remove robs test * start replacing random with seedrandom * use seeded random for uuidv4 * separate out IP randomization * typecheck fixes Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a generator that provides sample events, alerts, and endpoint metadata documents for use in testing. Documents created using an instance of the generator will have corresponding host and agent information so they can be correlated. Additionally, helper functions
generateResolverTreeandgenerateEventAncestryautomatically create a set of events that have parent-child relationships defined by their entity_ids.generateResolverTreeexpects a root event to be passed in and generates and returns a list of events and alerts that form the tree of children as well as related events for some random child processes.Checklist
Delete any items that are not applicable to this PR.
For maintainers