Update jackson and azure by nik9000 · Pull Request #80160 · elastic/elasticsearch

nik9000 · 2021-11-01T19:54:20Z

This grabs a new version of jackson and snakeyaml to get some bug fixes.
It also updates azure to stay compatible with the new jackson.

Closes #66555
Closes #67214
Closes #80142

nik9000 · 2021-11-01T19:55:40Z

There were other changes the last time we tried to update azure - https://github.com/elastic/elasticsearch/pull/73837/files#diff-0a3347d8c82d7507fa4507d2fde0f5feb4fa718a0d5025a249afbcde5621c555 - but the tests don't fail and I don't know what they were for.

elasticmachine · 2021-11-01T19:56:01Z

Pinging @elastic/es-distributed (Team:Distributed)

elasticmachine · 2021-11-01T19:56:02Z

Pinging @elastic/es-core-infra (Team:Core/Infra)

This grabs a new version of jackson and snakeyaml to get some bug fixes. It also updates azure to stay compatible with the new jackson. Closes elastic#66555 Closes elastic#67214 Closes elastic#80142

fcofdez · 2021-11-02T08:42:51Z

Sadly we are still blocked by this issue Azure/azure-sdk-for-java#22042 I've pinged the team several times but it seems like there hasn't been any progress on that

nik9000 · 2021-11-02T20:43:34Z

There were other changes the last time we tried to update azure - https://github.com/elastic/elasticsearch/pull/73837/files#diff-0a3347d8c82d7507fa4507d2fde0f5feb4fa718a0d5025a249afbcde5621c555 - but the tests don't fail and I don't know what they were for.

Adding these back helped the broken tests.

Sadly we are still blocked by this issue Azure/azure-sdk-for-java#22042 I've pinged the team several times but it seems like there hasn't been any progress on that

@fcofdez Oh! Is there any way we can get just jackson upgraded? I really do want to get us unstuck and I don't know anything about Azure.

nik9000 · 2021-11-02T20:45:54Z

@fcofdez Oh! Is there any way we can get just jackson upgraded? I really do want to get us unstuck and I don't know anything about Azure.

Do we have any tests that'd fail on upgrade that I can trigger btw?

grcevski · 2021-11-02T21:04:46Z

Sadly we are still blocked by this issue Azure/azure-sdk-for-java#22042 I've pinged the team several times but it seems like there hasn't been any progress on that

I know few folks on a related team, I'll reach out.

fcofdez · 2021-11-03T13:07:18Z

Do we have any tests that'd fail on upgrade that I can trigger btw?

The CI job that runs the problematic test is elastic+elasticsearch+{branch}+multijob+third-party-tests-azure-sas but I don't know how to trigger those in a PR.

nik9000 · 2021-11-03T13:22:41Z

** elasticsearch-ci/part-1-windows ** — Build finished.

This failure is the same one we had last time we tried to update:

»  Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "C:\Windows\system32\drivers\etc\hosts" "read")

I really don't know anything about the Azure deployment. I'd like to hand this upgrade off to someone that does. I'm many layers deep from the code I was trying to write in the first place and there are much better experts over here.

fcofdez · 2021-11-03T13:32:39Z

@nik9000 I can take care of this once the bug in the azure SDK is fixed

nik9000 · 2021-11-03T13:33:58Z

@nik9000 I can take care of this once the bug in the azure SDK is fixed

Thanks so much. I'm way out of my depth here. Or, at least, it'd take me a lot of reading to catch up.

weizijun · 2021-12-08T01:58:08Z

hi, @nik9000 , is it necessary to update jackson first? as the badcase in exclude exist in the current version?

nik9000 · 2021-12-08T14:33:08Z

hi, @nik9000 , is it necessary to update jackson first? as the badcase in exclude exist in the current version?

For now at least these are stuck together due to a breaking change in Jackson that breaks azure. It's pretty deep so I can see how the folks that made the change didn't think of it as a version change thing.

I think we're waiting on @ChrisHegarty's contacts at Azure to update to the newer version of jackson.

ChrisHegarty · 2021-12-08T14:58:47Z

I think we're waiting on @ChrisHegarty's contacts at Azure to update to the newer version of jackson.

That would be @grcevski's contact. :-)

fcofdez · 2021-12-08T15:01:14Z

The bug was finally fixed Azure/azure-sdk-for-java#22042 once they release a new version with the fix, I'll upgrade the libraries.

costin · 2021-12-24T13:17:41Z

Fwiw, jackson 2.13.1 was released 6 days ago (at the timing of writing this comment):
https://github.com/FasterXML/jackson-core/releases/tag/jackson-core-2.13.1
https://github.com/FasterXML/jackson-core/milestone/61?closed=1

nik9000 · 2022-01-04T16:03:35Z

Once we can upgrade jackson and jackson released 2.14 we can fix #63842.

nik9000 · 2022-01-04T16:04:59Z

The bug was finally fixed Azure/azure-sdk-for-java#22042 once they release a new version with the fix, I'll upgrade the libraries.

@fcofdez do you know what version we're waiting on? I can't really tell from the linked issues.

fcofdez · 2022-01-04T16:12:13Z

@nik9000 We're waiting for 12.15.0. They released a beta almost a month ago https://github.com/Azure/azure-sdk-for-java/releases/tag/azure-storage-blob_12.15.0-beta.2 I guess the final release should be around the corner.

nik9000 · 2022-01-04T16:44:41Z

@nik9000 We're waiting for 12.15.0. They released a beta almost a month ago https://github.com/Azure/azure-sdk-for-java/releases/tag/azure-storage-blob_12.15.0-beta.2 I guess the final release should be around the corner.

Thanks!

Checks that archive functionality works with source-only repositories. Relates #80160

Checks that archive functionality works with source-only repositories. Relates elastic#80160

add a xcontent filter test that will failed, when random to `filterOnParser`. The test will fixed when jackson upgrade in #80160

nik9000 · 2022-03-09T18:17:55Z

Overtaken by #83884

elasticsearchmachine added the v8.1.0 label Nov 1, 2021

nik9000 added :Core/Infra/Core Core issues without another label :Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs labels Nov 1, 2021

elasticmachine added Team:Distributed Meta label for distributed team. Team:Core/Infra Meta label for core/infra team labels Nov 1, 2021

nik9000 added v8.0.0-beta1 >non-issue labels Nov 1, 2021

nik9000 requested a review from fcofdez November 1, 2021 19:56

nik9000 added the test-windows Trigger CI checks on Windows label Nov 1, 2021

nik9000 added 2 commits November 1, 2021 15:57

Update jackson and azure

2820d0a

This grabs a new version of jackson and snakeyaml to get some bug fixes. It also updates azure to stay compatible with the new jackson. Closes elastic#66555 Closes elastic#67214 Closes elastic#80142

No databind

6b1da9d

nik9000 added 2 commits November 2, 2021 09:35

Finish restore?

94e84c9

Merge branch 'master' into upgrade_jackson_azure

2e23aa3

Add old code

9e4ead8

nik9000 mentioned this pull request Nov 3, 2021

XContentParser with excludes filters emits invalid json in common cases #80142

Open

weizijun mentioned this pull request Dec 12, 2021

Test: add a failed xcontent filter test that will wait for jackson upgrade #81634

Merged

ywelsch mentioned this pull request Jan 4, 2022

Check older source-only repos are supported #82213

Merged

ywelsch added a commit that referenced this pull request Jan 5, 2022

Check older source-only repos are supported (#82213)

bb8578b

Checks that archive functionality works with source-only repositories. Relates #80160

astefan pushed a commit to astefan/elasticsearch that referenced this pull request Jan 7, 2022

Check older source-only repos are supported (elastic#82213)

9a6a117

Checks that archive functionality works with source-only repositories. Relates elastic#80160

astefan pushed a commit to astefan/elasticsearch that referenced this pull request Jan 7, 2022

Check older source-only repos are supported (elastic#82213)

2f41736

Checks that archive functionality works with source-only repositories. Relates elastic#80160

elasticsearchmachine pushed a commit that referenced this pull request Jan 13, 2022

add test (#81634)

d07d27e

add a xcontent filter test that will failed, when random to `filterOnParser`. The test will fixed when jackson upgrade in #80160

mark-vieira added v8.2.0 and removed v8.1.0 labels Feb 2, 2022

weizijun mentioned this pull request Feb 25, 2022

Upgrade Azure SDK #83884

Merged

nik9000 removed v8.0.0-beta1 v8.2.0 labels Mar 9, 2022

nik9000 closed this Mar 9, 2022

DaveCTurner mentioned this pull request Dec 24, 2022

Reinstate AbstractXContentFilteringTestCase#testDotsAndDoubleWildcardInExcludedFieldName #92559

Closed

Conversation

nik9000 commented Nov 1, 2021

Uh oh!

nik9000 commented Nov 1, 2021

Uh oh!

elasticmachine commented Nov 1, 2021

Uh oh!

elasticmachine commented Nov 1, 2021

Uh oh!

fcofdez commented Nov 2, 2021

Uh oh!

nik9000 commented Nov 2, 2021

Uh oh!

nik9000 commented Nov 2, 2021

Uh oh!

grcevski commented Nov 2, 2021

Uh oh!

fcofdez commented Nov 3, 2021

Uh oh!

nik9000 commented Nov 3, 2021

Uh oh!

fcofdez commented Nov 3, 2021

Uh oh!

nik9000 commented Nov 3, 2021

Uh oh!

weizijun commented Dec 8, 2021

Uh oh!

nik9000 commented Dec 8, 2021

Uh oh!

ChrisHegarty commented Dec 8, 2021

Uh oh!

fcofdez commented Dec 8, 2021

Uh oh!

costin commented Dec 24, 2021

Uh oh!

nik9000 commented Jan 4, 2022

Uh oh!

nik9000 commented Jan 4, 2022

Uh oh!

fcofdez commented Jan 4, 2022

Uh oh!

nik9000 commented Jan 4, 2022

Uh oh!

nik9000 commented Mar 9, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants