Skip to content

fetch cloud metada - common logic#1453

Closed
videnkz wants to merge 13 commits intoelastic:masterfrom
videnkz:issue-1264-collect-cloud-metadata
Closed

fetch cloud metada - common logic#1453
videnkz wants to merge 13 commits intoelastic:masterfrom
videnkz:issue-1264-collect-cloud-metadata

Conversation

@videnkz
Copy link
Copy Markdown
Contributor

@videnkz videnkz commented Oct 24, 2020 •
edited
Loading

refer #1264
Note: we may have problems if a new version is used in the cloud with old versions of apm-server
Note2: I didn't add cloud.account.name field in serialization, because this field value is not available in the selected providers (aws, gcp, azure).

@videnkz videnkz marked this pull request as draft October 24, 2020 21:16
@ghost
Copy link
Copy Markdown

ghost commented Oct 24, 2020 •
edited by ghost
Loading

❕ Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts

Expand to view the summary

Build stats

  • Build Cause: Branch indexing

  • Reason: The PR is not allowed to run in the CI yet

  • Start Time: 2021-01-12T15:33:56.048+0000

  • Duration: 3 min 50 sec

  • Commit: 717c34c

Steps errors 1

Expand to view the steps failures

Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permission

Log output

Expand to view the last 100 lines of log output 

[2021-01-12T15:36:18.642Z] Using reference repository: /var/lib/jenkins/.git-references/apm-agent-java.git
[2021-01-12T15:36:18.649Z] Fetching upstream changes from https://github.com/elastic/apm-agent-java.git
[2021-01-12T15:36:18.649Z]  > git --version # timeout=10
[2021-01-12T15:36:18.666Z]  > git --version # 'git version 2.17.1'
[2021-01-12T15:36:18.666Z] using GIT_ASKPASS to set credentials GitHub user @apmmachine User + Personal Access Token
[2021-01-12T15:36:18.668Z]  > git fetch --no-tags --progress -- https://github.com/elastic/apm-agent-java.git +refs/heads/*:refs/remotes/origin/* # timeout=15
[2021-01-12T15:36:19.457Z] Pruning obsolete local branches
[2021-01-12T15:36:19.821Z] Merging remotes/origin/master commit 56bae31eb563fbc532729c13a24f01523e382958 into PR head commit 717c34cb47916406830c7e47889fedc6132ab113
[2021-01-12T15:36:20.297Z] Merge succeeded, producing b6eb776db834f6a7015e7ddabd22fd1d64937289
[2021-01-12T15:36:20.297Z] Checking out Revision b6eb776db834f6a7015e7ddabd22fd1d64937289 (PR-1453)
[2021-01-12T15:36:19.422Z]  > git config remote.origin.url https://github.com/elastic/apm-agent-java.git # timeout=10
[2021-01-12T15:36:19.438Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2021-01-12T15:36:19.448Z]  > git config remote.origin.url https://github.com/elastic/apm-agent-java.git # timeout=10
[2021-01-12T15:36:19.461Z] Fetching upstream changes from https://github.com/elastic/apm-agent-java.git
[2021-01-12T15:36:19.461Z] using GIT_ASKPASS to set credentials GitHub user @apmmachine User + Personal Access Token
[2021-01-12T15:36:19.462Z]  > git fetch --no-tags --progress --prune -- https://github.com/elastic/apm-agent-java.git +refs/pull/1453/head:refs/remotes/origin/PR-1453 +refs/heads/master:refs/remotes/origin/master # timeout=15
[2021-01-12T15:36:19.828Z]  > git config core.sparsecheckout # timeout=10
[2021-01-12T15:36:19.836Z]  > git checkout -f 717c34cb47916406830c7e47889fedc6132ab113 # timeout=15
[2021-01-12T15:36:20.051Z]  > git remote # timeout=10
[2021-01-12T15:36:20.062Z]  > git config --get remote.origin.url # timeout=10
[2021-01-12T15:36:20.071Z] using GIT_ASKPASS to set credentials GitHub user @apmmachine User + Personal Access Token
[2021-01-12T15:36:20.072Z]  > git merge 56bae31eb563fbc532729c13a24f01523e382958 # timeout=10
[2021-01-12T15:36:20.289Z]  > git rev-parse HEAD^{commit} # timeout=10
[2021-01-12T15:36:20.299Z]  > git config core.sparsecheckout # timeout=10
[2021-01-12T15:36:20.319Z]  > git checkout -f b6eb776db834f6a7015e7ddabd22fd1d64937289 # timeout=15
[2021-01-12T15:36:24.097Z] Commit message: "Merge commit '56bae31eb563fbc532729c13a24f01523e382958' into HEAD"
[2021-01-12T15:36:24.115Z] First time build. Skipping changelog.
[2021-01-12T15:36:24.100Z]  > git rev-list --no-walk 6a1b4a639d8073cb1f3fc594d820c1e6cbf0c6c8 # timeout=10
[2021-01-12T15:36:25.048Z] Masking supported pattern matches of $JOB_GCS_BUCKET or $NOTIFY_TO
[2021-01-12T15:36:25.094Z] Timeout set to expire in 1 hr 0 min
[2021-01-12T15:36:25.111Z] The timestamps step is unnecessary when timestamps are enabled for all Pipeline builds.
[2021-01-12T15:36:25.369Z] [INFO] Number of builds to be searched 10
[2021-01-12T15:36:25.644Z] [INFO] 'shallow' is forced to be disabled when running on PullRequests
[2021-01-12T15:36:25.661Z] Running in /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1453/src/github.com/elastic/apm-agent-java
[2021-01-12T15:36:25.680Z] [INFO] gitCheckout: Checkout SCM PR-1453 with some customisation.
[2021-01-12T15:36:25.708Z] [INFO] Override default checkout
[2021-01-12T15:36:25.742Z] Sleeping for 10 sec
[2021-01-12T15:36:35.875Z] using credential 2a9602aa-ab9f-4e52-baf3-b71ca88469c7-UserAndToken
[2021-01-12T15:36:36.060Z] Cloning the remote Git repository
[2021-01-12T15:36:36.072Z] Cloning repository https://github.com/elastic/apm-agent-java.git
[2021-01-12T15:36:36.097Z]  > git init /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1453/src/github.com/elastic/apm-agent-java # timeout=10
[2021-01-12T15:36:36.107Z] Using reference repository: /var/lib/jenkins/.git-references/apm-agent-java.git
[2021-01-12T15:36:36.107Z] Fetching upstream changes from https://github.com/elastic/apm-agent-java.git
[2021-01-12T15:36:36.108Z]  > git --version # timeout=10
[2021-01-12T15:36:36.113Z]  > git --version # 'git version 2.17.1'
[2021-01-12T15:36:36.114Z] using GIT_ASKPASS to set credentials GitHub user @apmmachine User + Personal Access Token
[2021-01-12T15:36:36.115Z]  > git fetch --tags --progress -- https://github.com/elastic/apm-agent-java.git +refs/heads/*:refs/remotes/origin/* # timeout=10
[2021-01-12T15:36:36.637Z]  > git config remote.origin.url https://github.com/elastic/apm-agent-java.git # timeout=10
[2021-01-12T15:36:36.643Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2021-01-12T15:36:36.650Z]  > git config remote.origin.url https://github.com/elastic/apm-agent-java.git # timeout=10
[2021-01-12T15:36:36.658Z] Fetching upstream changes from https://github.com/elastic/apm-agent-java.git
[2021-01-12T15:36:36.658Z] using GIT_ASKPASS to set credentials GitHub user @apmmachine User + Personal Access Token
[2021-01-12T15:36:36.659Z]  > git fetch --tags --progress -- https://github.com/elastic/apm-agent-java.git +refs/pull/1453/head:refs/remotes/origin/PR-1453 +refs/heads/master:refs/remotes/origin/master # timeout=10
[2021-01-12T15:36:36.989Z] Checking out Revision 717c34cb47916406830c7e47889fedc6132ab113 (origin/PR-1453)
[2021-01-12T15:36:37.194Z] Commit message: "deleted todo"
[2021-01-12T15:36:37.194Z] First time build. Skipping changelog.
[2021-01-12T15:36:36.982Z]  > git rev-parse origin/PR-1453^{commit} # timeout=10
[2021-01-12T15:36:36.990Z]  > git config core.sparsecheckout # timeout=10
[2021-01-12T15:36:37.003Z]  > git checkout -f 717c34cb47916406830c7e47889fedc6132ab113 # timeout=10
[2021-01-12T15:36:38.395Z] Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[2021-01-12T15:36:38.988Z] + git fetch https://****:****@github.com/elastic/apm-agent-java.git +refs/pull/*/head:refs/remotes/origin/pr/*
[2021-01-12T15:36:39.978Z] Running in /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1453/src/github.com/elastic/apm-agent-java/.git
[2021-01-12T15:36:39.999Z] Archiving artifacts
[2021-01-12T15:36:40.646Z] + git rev-parse HEAD
[2021-01-12T15:36:40.999Z] + git rev-parse HEAD
[2021-01-12T15:36:41.295Z] + git rev-parse origin/pr/1453
[2021-01-12T15:36:41.334Z] [INFO] githubEnv: Found Git Build Cause: pr
[2021-01-12T15:36:41.702Z] Masking supported pattern matches of $GITHUB_TOKEN
[2021-01-12T15:36:42.700Z] [WARN] githubApiCall: The REST API call https://api.github.com/repos/elastic/apm-agent-java/pulls/1453/reviews return 0 elements
[2021-01-12T15:36:42.738Z] [INFO] githubPrCheckApproved: Title: fetch cloud metada - common logic - User: kananindzya - Author Association: CONTRIBUTOR
[2021-01-12T15:36:42.979Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet
[2021-01-12T15:36:42.979Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permissions can do so.)
[2021-01-12T15:36:43.015Z] [INFO] Let's stop build #27. The PR is not allowed to run in the CI yet
[2021-01-12T15:36:43.029Z] Sleeping for 5 sec
[2021-01-12T15:36:44.069Z] Stage "Build" skipped due to earlier failure(s)
[2021-01-12T15:36:44.151Z] Stage "Tests" skipped due to earlier failure(s)
[2021-01-12T15:36:44.197Z] Stage "Unit Tests" skipped due to earlier failure(s)
[2021-01-12T15:36:44.198Z] Stage "Smoke Tests 01" skipped due to earlier failure(s)
[2021-01-12T15:36:44.199Z] Stage "Smoke Tests 02" skipped due to earlier failure(s)
[2021-01-12T15:36:44.199Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-01-12T15:36:44.200Z] Stage "Javadoc" skipped due to earlier failure(s)
[2021-01-12T15:36:44.263Z] Failed in branch Unit Tests
[2021-01-12T15:36:44.264Z] Failed in branch Smoke Tests 01
[2021-01-12T15:36:44.264Z] Failed in branch Smoke Tests 02
[2021-01-12T15:36:44.265Z] Failed in branch Benchmarks
[2021-01-12T15:36:44.266Z] Failed in branch Javadoc
[2021-01-12T15:36:44.321Z] Stage "Integration Tests" skipped due to earlier failure(s)
[2021-01-12T15:36:44.358Z] Stage "AfterRelease" skipped due to earlier failure(s)
[2021-01-12T15:36:44.373Z] Stage "AfterRelease" skipped due to earlier failure(s)
[2021-01-12T15:36:44.674Z] Running on Jenkins in /var/lib/jenkins/workspace/_java_apm-agent-java-mbp_PR-1453
[2021-01-12T15:36:44.800Z] [INFO] getVaultSecret: Getting secrets
[2021-01-12T15:36:45.240Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-01-12T15:36:45.896Z] + chmod 755 generate-build-data.sh
[2021-01-12T15:36:45.896Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1453/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1453/runs/27 ABORTED 169589
[2021-01-12T15:36:46.147Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1453/runs/27/steps/?limit=10000 -o steps-info.json
[2021-01-12T15:36:47.058Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1453/runs/27/tests/?status=FAILED -o tests-errors.json
[2021-01-12T15:36:47.058Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-01-12T15:36:48.401Z] Retry 2/3 exited 22, retrying in 2 seconds...
[2021-01-12T15:36:50.262Z] Retry 3/3 exited 22, no more retries left.
[2021-01-12T15:36:50.262Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-java/apm-agent-java-mbp/PR-1453/runs/27/log/ -o pipeline-log.txt

@videnkz
Copy link
Copy Markdown
Contributor Author

videnkz commented Nov 11, 2020

this work only with apm-server 7.10.0 version:
image

@videnkz videnkz marked this pull request as ready for review November 11, 2020 17:39
@videnkz videnkz changed the title WIP: fetch cloud metada - common logic fetch cloud metada - common logic Nov 11, 2020
@videnkz
Copy link
Copy Markdown
Contributor Author

videnkz commented Nov 12, 2020

Hi @eyalkoren , please review this pull request.
Thanks

@videnkz videnkz force-pushed the issue-1264-collect-cloud-metadata branch from 940b15e to 717c34c Compare November 12, 2020 19:33
@codecov-io
Copy link
Copy Markdown

codecov-io commented Nov 16, 2020

Codecov Report

Merging #1453 (717c34c) into master (fc1a0f4) will decrease coverage by 0.09%.
The diff coverage is 70.74%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1453      +/-   ##
============================================
- Coverage     71.41%   71.32%   -0.10%     
- Complexity     2583     2614      +31     
============================================
  Files           252      254       +2     
  Lines         11221    11489     +268     
  Branches       1460     1502      +42     
============================================
+ Hits           8014     8194     +180     
- Misses         2688     2737      +49     
- Partials        519      558      +39
Impacted Files Coverage Δ Complexity Δ
...src/main/java/co/elastic/apm/agent/impl/Cloud.java 61.29% <61.29%> (ø) 11.00 <11.00> (?)
...stic/apm/agent/impl/payload/CloudProviderInfo.java 69.38% <69.38%> (ø) 16.00 <16.00> (?)
.../apm/agent/report/serialize/DslJsonSerializer.java 89.51% <97.22%> (+0.33%) 193.00 <6.00> (+7.00)
...tic/apm/agent/configuration/CoreConfiguration.java 96.88% <100.00%> (+0.07%) 39.00 <1.00> (+1.00)
.../main/java/co/elastic/apm/agent/impl/MetaData.java 95.45% <100.00%> (+0.45%) 9.00 <2.00> (+1.00)
...iguration/PrefixingConfigurationSourceWrapper.java 50.00% <0.00%> (-16.67%) 3.00% <0.00%> (-1.00%)
...agent/report/serialize/MetricRegistryReporter.java 86.66% <0.00%> (-6.67%) 4.00% <0.00%> (-1.00%)
...ration/source/PropertyFileConfigurationSource.java 71.42% <0.00%> (-5.72%) 10.00% <0.00%> (-1.00%)
...lastic/apm/agent/impl/ElasticApmTracerBuilder.java 73.33% <0.00%> (-4.45%) 20.00% <0.00%> (ø%)
...ent/report/serialize/MetricRegistrySerializer.java 87.60% <0.00%> (-2.48%) 30.00% <0.00%> (-2.00%)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fc1a0f4...717c34c. Read the comment docs.

@eyalkoren eyalkoren self-assigned this Nov 24, 2020
@eyalkoren
Copy link
Copy Markdown
Contributor

eyalkoren commented Nov 24, 2020

@kananindzya don't worry, I didn't forget about it, it just takes some time to get to it...
I will probably do a PR against your PR, suggesting some changes

@eyalkoren eyalkoren mentioned this pull request Dec 20, 2020
Open
@eyalkoren eyalkoren mentioned this pull request Jan 6, 2021
Merged
10 tasks
@videnkz videnkz closed this Jan 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@eyalkoren eyalkoren

Labels

agent-java

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@videnkz @codecov-io @eyalkoren