Skip to content

flatcar-postinst: Handle airgapped self-hosted nebraska instances#39

Merged
jepio merged 1 commit intoflatcar-3815-backportfrom
jepio/3815-fix-self-sync-nebraska
Feb 26, 2024
Merged

flatcar-postinst: Handle airgapped self-hosted nebraska instances#39
jepio merged 1 commit intoflatcar-3815-backportfrom
jepio/3815-fix-self-sync-nebraska

Conversation

@jepio
Copy link
Copy Markdown
Member

@jepio jepio commented Feb 22, 2024

This is a backport of the equivalent of flatcar/scripts#1580 to 3815, without switching to ue-rs.

Tested on the full-response from the airgapped environment (in commit message):

$ cat flatcar-postinst
set -euo pipefail
set -x
umask 0022
OEMID=vmware
sysext_download() {
    ....
}
sysext_download "oem-${OEMID}.gz" "./var/lib/update_engine/oem-${OEMID}.raw" test.xml

Output:

$ bash -x flatcar-postinst
+ set -euo pipefail
+ set -x
+ umask 0022
+ OEMID=vmware
+ sysext_download oem-vmware.gz ./var/lib/update_engine/oem-vmware.raw test2.xml
+ local name=oem-vmware.gz
+ local target=./var/lib/update_engine/oem-vmware.raw
+ local from=test2.xml
+ local base=
+ local entries=
+ local hash=
+ local size=
+ local url=
+ local ret
+ SUCCESS=false
+ set +e
+ set -e
+ '[' test2.xml = release-server ']'
+ '[' test2.xml = bincache-server ']'
++ grep -m 1 -o 'codebase="[^"]*"' test2.xml
++ cut -d '"' -f 2
+ base=https://nebraska-example.org/flatcar/
++ grep -m 1 -o '<package name="[^"]*oem-vmware.gz"[^>]*' test2.xml
+ entries='<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o 'name="[^"]*'
++ cut -d '"' -f 2
+ name=extrafile-amd64-3815.2.0-oem-vmware.gz
+ url=https://nebraska-example.org/flatcar/extrafile-amd64-3815.2.0-oem-vmware.gz
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o 'size="[0-9]*'
++ cut -d '"' -f 2
+ size=1536465
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o -P 'hash="[^"]*'
++ cut -d '"' -f 2
+ hash=llmt48kEE1pZxHbhE86zTczZMmY=
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o -P 'hash_sha256="[^"]*'
++ cut -d '"' -f 2
+ hash_sha256=7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88
+ rm -f ./var/lib/update_engine/oem-vmware.raw.tmp
+ curl -fsSL --retry-delay 1 --retry 60 --retry-connrefused --retry-max-time 60 --connect-timeout 20 -o ./var/lib/update_engine/oem-vmware.raw.tmp https://nebraska-example.org/flatcar/extrafile-amd64-3815.2.0-oem-vmware.gz
curl: (6) Could not resolve host: nebraska-example.org
curl: (6) Could not resolve host: nebraska-example.org
....

@jepio
flatcar-postinst: Handle airgapped self-hosted nebraska instances
4c7f06e 
Right now the stable update from 3602.2.3 -> 3815.2.0 fails for airgapped
environment with self-hosted nebraska instances that sync packages, because the
parsing in flatcar-postinst can't handle the names of OEM sysexts in the
nebraska response. These are called "extrafile-<BOARD>-<VERSION>-oem-<OEM>.gz".
Fix this by matching against "*oem-<OEM>.gz" and extracting the remote filename
from the response.

In main this is fixed in #36, but
that depends on switching to ue-rs.

Here is an example response for testing:

<response protocol="3.0" server="nebraska">
  <daystart elapsed_seconds="0"/>
  <app appid="{e96281a6-d1af-4bde-9a0a-97b76e56dc57}" status="ok">
    <ping status="ok"/>
    <updatecheck status="ok">
      <urls>
        <url codebase="https://nebraska-example.org/flatcar/"/>
      </urls>
      <manifest version="3815.2.0">
        <packages>
          <package name="flatcar-amd64-3815.2.0.gz" hash="JLT5o2wJzffAgHY5GesCWymJHng=" size="458309926" required="true"/>
          <package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/>
        </packages>
    </updatecheck>
  </app>
</response>

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
@jepio jepio mentioned this pull request Feb 22, 2024
Merged
2 tasks
pothos
pothos reviewed Feb 26, 2024
View reviewed changes
pothos
pothos approved these changes Feb 26, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@pothos pothos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@jepio jepio merged commit 71b7cc3 into flatcar-3815-backport Feb 26, 2024
@jepio jepio deleted the jepio/3815-fix-self-sync-nebraska branch February 26, 2024 11:06
jepio added a commit to flatcar/scripts that referenced this pull request Feb 26, 2024
@jepio
coreos-base/update_engine: Bump commit for bugfix
9c7ad4e 
Pulls in flatcar/update_engine#39 which fixes updates
on the stable channel for nodes with self-hosted packages in an environment
where nodes are not allowed access to the upstream Flatcar update server.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
@jepio jepio mentioned this pull request Feb 26, 2024
Merged
2 tasks
@github-actions github-actions bot mentioned this pull request Mar 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pothos pothos pothos approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jepio @pothos