[github] Add dependabot alerts data stream#3754
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
🌐 Coverage report
|
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
| {{/if}} | ||
|
|
||
| request.body: | ||
| query: |
There was a problem hiding this comment.
Could we store this as a formatted string to make reading it easier? Like
query: |-
query fetchRepoAlerts($org: String!, $repo: String!) {
repository(owner: $org, name: $repo) {
vulnerabilityAlerts(first: 2) {
nodes {
createdAt
dependabotUpdate {
error {
body
errorType
title
}
pullRequest {
createdAt
closed
closedAt
merged
mergedAt
number
url
title
}
}
dependencyScope
dismissReason
dismissedAt
dismisser {
login
url
}
fixReason
fixedAt
number
repository {
description
isInOrganization
isPrivate
name
owner {
login
url
}
url
}
securityAdvisory {
classification
cvss {
score
vectorString
}
cwes(first: 100) {
nodes {
cweId
description
name
}
}
description
ghsaId
identifiers {
type
value
}
origin
permalink
references {
url
}
publishedAt
severity
summary
updatedAt
withdrawnAt
}
securityVulnerability {
firstPatchedVersion {
identifier
}
package {
ecosystem
name
}
severity
updatedAt
vulnerableVersionRange
}
state
vulnerableManifestPath
vulnerableManifestFilename
vulnerableRequirements
}
pageInfo {
hasNextPage
endCursor
}
}
}
}There was a problem hiding this comment.
@andrewkroh Thanks for the suggestion. Unfortunately, the above formatted string doesn't seem to be working. Following is the error message in the agent logs:
"message":"error processing response: template: :1:69: executing \"\" at <.last_response.body.data.repository.vulnerabilityAlerts.pageInfo.endCursor>: map has no entry for key \"data\""
I think the request might be getting an empty response and trying to get cursor info where there is none and logging it.
There was a problem hiding this comment.
I don't think the request body gets marshalled correctly that way, a bit unsure if that is just because of a typo or not
| api_url: http://{{Hostname}}:{{Port}} | ||
| access_token: xxxxxxxxxx | ||
| owner: sample_owner | ||
| repo: sample_repo |
There was a problem hiding this comment.
Would it be possible to construct a query that gets the alerts from all repos in an org? As repos are added within an org, then you will need to keep adding new inputs which isn't sustainable for big orgs.
There was a problem hiding this comment.
This is an incomplete PoC, but it appears that it is possible (I know very little about graphql). So perhaps the integration could function is two modes like org mode or single repo mode. I think org mode will be used the most.
query orgRepoVulnerabilities($org: String!) {
organization(login: $org) {
repositories(first: 100) {
nodes {
name
vulnerabilityAlerts(first: 10) {
nodes {
createdAt
dependabotUpdate {
error {
body
errorType
title
}
pullRequest {
createdAt
closed
closedAt
merged
mergedAt
number
url
title
}
}
}
}
}
pageInfo {
hasNextPage
endCursor
}
}
}
}There was a problem hiding this comment.
This is a great suggestion @andrewkroh. When I met with Github folks they also wanted organisation level alerts implemented. Here is the PR where this enhancement will be implemented(targeting for 8.5): #3935
The reason repository was targeted in the first run was to have some consistency in the implementation as their REST APIs for Github Code Scanning alerts at ORG level wasn't working. So, thought of adding repo endpoint first, then followup with org endpoint and provide choice to the user.
There was a problem hiding this comment.
Yeah the end goal is to offer 2 options.
Organization level as default, and a repo option in advanced options, if repo is set, it will focus on a single repo, as there is no way to filter on repo's at all when using organization/root level API's.
There was a problem hiding this comment.
Code changed to query organization level alerts by default, and if repository (optional) is provided, the integration would only query repository level alerts.
| "name": "jsonwebtoken" | ||
| }, | ||
| "updated_at": "2018-11-30T19:54:28Z", | ||
| "vulnerable_version_range": "\u003c 4.2.2" |
There was a problem hiding this comment.
The \u encoding of < is unfortunate. This is also present in event.original which seems worse, verging on a bug.
ISTM that this should be fixed in httpjson (https://play.golang.com/p/YGhWbtX7jO0)
packages/github/data_stream/dependabot/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
|
Demo of Github Advanced Security (GHAS) Alerts: GHAS.Demo.mp4 |
|
Great demo! Thank you for adding that! Feedback
|
|
Thanks for the feedback @andrewkroh! I will be implementing above suggestions in the subsequent release. |
What does this PR do?
For ingesting Github Advanced Security (GHAS) Dependabot alerts
Checklist
changelog.ymlfile.Screenshots
Demo
Demo of Github Advanced Security (GHAS) Alerts:
GHAS.Demo.mp4