Enable CPI Release golang #290

OBoehrer · 2024-10-18T13:08:21Z

Outcome of this PR

This PR is a follow up on #278, here we provide the complete implementation of the Golang-implemented job 'openstack_cpi_golang'. This job is completely independent, and can be used as an alternative to the job 'openstack_cpi'.

Background

Maintaining the existing Ruby CPI implies sustaining the used openstack-fog and fog core libraries. But that would be very problematic since the communities for these libraries do not seem to be active since some years now.

Fog isn't just a thin wrapper around the OpenStack API. On the contrary, Fog returns rich modules and classes that need to be refreshed, updated, ... . That means, replacing Fog calls by REST calls isn't a trivial task and will finally end up in a more or less complete re-implementation of the CPI.

A Golang based CPI reimplementation would profit from the existence of an actively maintained Golang SDK for OpenStack (see: https://github.com/gophercloud/gophercloud)

Difference

This is not a 1:1 implementation of the Ruby CPI. We try to improve and change the structure as we best see fit. Deprecated APIs are not supported by this CPI, hence we only support the Octavia APIs, Compute APIs, Networking APIs ... and not the neutron/networking endpoints.

Note

This golang implementation is based on the Ruby version as of Mar 22, 2024.

linux-foundation-easycla · 2024-10-18T13:08:36Z

The committers listed above are authorized under a signed CLA.

✅ login: OBoehrer (c54a6b4, 51709e2, 2c207d8, b42d4f3, 826769a)
✅ login: a-hassanin / name: Ahmed Hassanin (8c86592, 72324e4, 1219c28)

fmoehler · 2024-10-21T07:46:54Z

Co-authored by @a-hassanin, @anshrupani, @mvach, @ShilpaChandrashekara, @fmoehler

jobs/openstack_cpi_golang/spec

a-hassanin

These calls seem to be incompatible with Windows:

https://github.com/OBoehrer/bosh-openstack-cpi-release/blob/c54a6b48f5ec00b787ae7b7ab45894ff5820908a/src/openstack_cpi_golang/cpi/image/root_image/root_image_test.go#L15

https://github.com/OBoehrer/bosh-openstack-cpi-release/blob/c54a6b48f5ec00b787ae7b7ab45894ff5820908a/src/openstack_cpi_golang/cpi/methods/create_stemcell.go#L65

See Go / Run tests and Lint on windows-2019

src/openstack_cpi_golang/cpi/volume/volume_service_test.go

a-hassanin

LGTM

aramprice · 2024-10-24T19:07:55Z

I'd recommend adding a .golangci.yml, specifically having a longer timeout: can help with windows linting as well as updating the go.yml workflow to something like cloudfoundry/bosh-warden-cpi-release's go.yml

name: go
on:
  push:
    paths:
    - 'src/openstack_cpi_golang/**'
  pull_request:
    paths:
    - 'src/openstack_cpi_golang/**'
jobs:
  lint_and_test:
    strategy:
      matrix:
        os: [macos-latest, ubuntu-latest]
    name: lint and test
    runs-on: ${{ matrix.os }}
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-go@v5
      with:
        go-version-file: src/openstack_cpi_golang/go.mod
    - uses: golangci/golangci-lint-action@v6
      with:
        working-directory: src/openstack_cpi_golang/
    - run: run-unit-tests
      working-directory: src/openstack_cpi_golang/
    - run: run-integration-tests
      working-directory: src/openstack_cpi_golang/

And adding a dependabot.yml will keep these Github Actions (rather than hard-coded uses of golangci-lint) up to date.

I would like to change myself from reviewer to approver. You can find my contributions so far listed below: # Foundational Infrastructure: VM deployment lifecycle (BOSH) Contributions ### PRs Commented on/Reviewed: - 2024-12-12T22:52:53Z: [Remove `UseIsolatedEnv` from `system.Command`](cloudfoundry/bosh-utils#110) - 2025-01-28T10:33:14Z: [enable ipv6 in boshSysctl](cloudfoundry/bosh-agent#347) - 2025-02-10T07:47:30Z: [Extend bosh tagging for cpi](cloudfoundry/bosh#2603) - 2025-02-26T16:50:01Z: [When removing recursors for Noble usage, we accidentally also removed handlers](cloudfoundry/bosh-dns-release#107) ### Issues that may be relevant: - 2022-06-21T14:28:19Z: [Fix regression: Deployment manifests can contain yaml anchors](cloudfoundry/bosh#2386) - 2023-03-23T12:33:02Z: [Update aws.md](cloudfoundry/docs-bosh#788) - 2023-04-04T18:17:29Z: [Update post-stop.md](cloudfoundry/docs-bosh#789) - 2023-04-13T07:47:12Z: [Update dns.md](cloudfoundry/docs-bosh#790) - 2023-05-04T13:33:37Z: [correct the diskID for device path resolution](cloudfoundry/bosh-agent#309) - 2023-05-04T14:22:24Z: [Device path resolution times out for aws and ali](cloudfoundry/bosh-agent#310) - 2023-07-14T14:46:10Z: [introduce -v flag to show version information](cloudfoundry/bosh-azure-storage-cli#8) - 2023-07-18T13:19:16Z: [auto bump the azure storage cli](cloudfoundry/bosh#2455) - 2023-07-25T09:36:07Z: [update docs for azure storage account](cloudfoundry/docs-bosh#798) - 2023-10-26T10:00:07Z: [append global flags to deploy cmd](cloudfoundry/bosh-cli#633) - 2023-10-26T16:42:17Z: [Update configs.md](cloudfoundry/docs-bosh#806) - 2023-10-31T16:19:11Z: [enable global flags for deploy command](cloudfoundry/bosh-cli#634) - 2023-11-03T09:17:19Z: [bumped dependencies for nats-pure and ruby-prof to the most recent versions](cloudfoundry/bosh#2476) - 2023-11-07T16:55:08Z: [update documentation](cloudfoundry/docs-bosh#810) - 2023-11-09T12:57:03Z: [update global cli flags documentation](cloudfoundry/docs-bosh#811) - 2023-12-12T13:53:01Z: [Include deploy config documentation](cloudfoundry/docs-bosh#816) - 2023-12-13T16:33:11Z: [enable metrics for deploy config](cloudfoundry/bosh#2482) - 2024-01-11T13:37:42Z: [fix typo](cloudfoundry/docs-bosh#818) - 2024-01-11T13:44:44Z: [bosh scp command failing if used according to documentation](cloudfoundry/docs-bosh#819) - 2024-02-14T13:05:36Z: [use octavia endpoint instead of neutron for lb activites](cloudfoundry/bosh-openstack-cpi-release#268) - 2024-05-15T06:59:59Z: [bump fog openstack gem to new version 1.1.1 & enable application credential authentication](cloudfoundry/bosh-openstack-cpi-release#272) - 2024-07-05T15:11:06Z: [bump ruby-prof version from 1.6.3 to 1.7.0](cloudfoundry/bosh#2535) - 2024-08-07T13:25:05Z: [update gosigar version](cloudfoundry/bosh-dns-release#105) - 2024-08-09T07:11:03Z: [fix formatting for configure blobstore for gcp section](cloudfoundry/docs-bosh#848) - 2024-08-13T14:12:56Z: [introduce support for ceph blobstore](cloudfoundry/bosh-s3cli#43) - 2024-08-28T08:27:36Z: [Isolated environment is hard-coded for building bosh-releases](cloudfoundry/bosh-cli#660) - 2024-10-18T13:08:21Z: [Enable CPI Release golang](cloudfoundry/bosh-openstack-cpi-release#290) - 2024-11-11T12:11:36Z: [link should point to main branch and not master branch](cloudfoundry/docs-bosh#853) - 2024-11-13T20:34:24Z: [ensure that cpi uses the correct gem path during runtime](cloudfoundry/bosh-azure-cpi-release#702) - 2024-11-13T20:51:42Z: [revert fastjsonparser](cloudfoundry/bosh-azure-cpi-release#703) - 2024-11-13T21:04:07Z: ["Illegal instruction" error](cloudfoundry/bosh-azure-cpi-release#704) - 2024-11-19T15:33:32Z: [Remove isolated environment from cpi and compiler cmd](cloudfoundry/bosh-cli#673) - 2024-11-21T13:40:36Z: [Update pre-start.sh.erb in harden_ssh for noble stemcell](cloudfoundry/os-conf-release#70) - 2024-11-26T19:30:35Z: [enable dual stack support](cloudfoundry/bosh-aws-cpi-release#172) - 2024-12-05T10:11:36Z: [Metrics missing after upgrading to 280.1.10 ](cloudfoundry/bosh#2593) - 2024-12-19T09:57:45Z: [Disable error messages related to networks/network interface cards](cloudfoundry/bosh-agent#344) - 2024-12-19T10:12:41Z: [sysctl job disables ipv6 implicitly on jammy stemcells](cloudfoundry/os-conf-release#71) - 2025-01-09T15:42:13Z: [Enable dual stack with single nic](cloudfoundry/bosh-aws-cpi-release#174) - 2025-01-15T15:49:28Z: [Update 60-bosh-sysctl.conf](cloudfoundry/bosh-linux-stemcell-builder#407) - 2025-01-28T10:33:14Z: [enable ipv6 in boshSysctl](cloudfoundry/bosh-agent#347) - 2025-02-03T12:30:20Z: [set retry interval to avoid bosh workers sleeping forever in case of error](cloudfoundry/bosh#2599) - 2025-02-07T11:40:47Z: [Update kernel_ipv6.go](cloudfoundry/bosh-agent#348) - 2025-02-10T07:47:30Z: [Extend bosh tagging for cpi](cloudfoundry/bosh#2603) - 2025-02-10T08:48:50Z: [Azure Compute Gallery integration](cloudfoundry/bosh-azure-cpi-release#709) - 2025-02-10T08:50:20Z: [add ipv6 prefix](cloudfoundry/bosh-aws-cpi-release#176) ### Code contributions: - 2023-03-23T12:32:37Z: [Update aws.md](cloudfoundry/docs-bosh@14888a5) - 2023-04-04T18:17:16Z: [Update post-stop.md](cloudfoundry/docs-bosh@f8e4db2) - 2023-04-13T07:47:00Z: [Update dns.md](cloudfoundry/docs-bosh@a51e96f) - 2023-07-14T14:39:55Z: [introduce -v flag to show version information](cloudfoundry/bosh-azure-storage-cli@4fbf437) - 2023-07-18T13:15:27Z: [auto bump the azure storage cli](cloudfoundry/bosh@c3b1cd0) - 2023-07-18T13:19:24Z: [auto bump the azure storage cli](cloudfoundry/bosh@c3b1cd0) - 2023-07-19T07:56:16Z: [introduce -v flag to show version information](cloudfoundry/bosh-azure-storage-cli@4fbf437) - 2023-07-20T09:45:02Z: [add integration test](cloudfoundry/bosh-azure-storage-cli@844083d) - 2023-07-25T09:35:44Z: [update docs for azure storage account](cloudfoundry/docs-bosh@df2d802) - 2023-10-26T16:41:26Z: [Update configs.md](cloudfoundry/docs-bosh@db6957e) - 2023-11-07T16:51:19Z: [update documentation](cloudfoundry/docs-bosh@06ea871) - 2023-11-08T10:13:38Z: [add global flags to bosh deploy command](cloudfoundry/bosh-cli@cfbabbb) - 2023-11-09T12:25:46Z: [add excluding of deployments and tests](cloudfoundry/bosh-cli@55961e4) - 2023-11-09T12:56:53Z: [update global cli flags documentation](cloudfoundry/docs-bosh@2261b7a) - 2023-11-09T15:20:32Z: [add excluding of deployments and tests](cloudfoundry/bosh-cli@55961e4) - 2023-11-23T09:17:38Z: [color the cli output in case deployment flags are used](cloudfoundry/bosh-cli@f80c9e4) - 2023-11-23T09:19:37Z: [rename the search function](cloudfoundry/bosh-cli@d7ae88a) - 2023-11-23T16:00:47Z: [fix issues](cloudfoundry/bosh-cli@958d4aa) - 2023-11-23T16:31:41Z: [fix some more issues](cloudfoundry/bosh-cli@1a8d04d) - 2023-11-30T13:03:33Z: [Update cli-v2.md](cloudfoundry/docs-bosh@2d43295) - 2023-11-30T13:03:54Z: [Update deploy-config.md](cloudfoundry/docs-bosh@a378c26) - 2023-12-12T13:51:48Z: [Include deploy config documentation](cloudfoundry/docs-bosh@4c3a536) - 2023-12-18T14:04:27Z: [Introduce monitoring metrics for deploy config enablement](cloudfoundry/bosh@1874004) - 2023-12-18T16:21:04Z: [Update config_manager_spec.rb](cloudfoundry/bosh@0613612) - 2023-12-18T16:47:44Z: [Update config_manager.rb](cloudfoundry/bosh@21fd8e1) - 2024-01-03T08:26:56Z: [Update config_manager.rb](cloudfoundry/bosh@e66fea9) - 2024-01-11T13:30:21Z: [fix typo](cloudfoundry/docs-bosh@c5535da) - 2024-01-11T13:39:44Z: [We used bosh scp today and faced the issue mentioned here:](cloudfoundry/docs-bosh@6d2df8f) - 2024-05-15T06:57:25Z: [bump fog openstack gem to new version 1.1.1](cloudfoundry/bosh-openstack-cpi-release@367524f) - 2024-05-15T07:58:04Z: [enable application credential authentication](cloudfoundry/bosh-openstack-cpi-release@d12b287) - 2024-05-15T12:37:16Z: [Update cloud.rb](cloudfoundry/bosh-openstack-cpi-release@f4cee7a) - 2024-05-15T12:38:54Z: [Update cloud_spec.rb](cloudfoundry/bosh-openstack-cpi-release@dbdf465) - 2024-07-05T14:40:37Z: [bump ruby-prof version from 1.6.3 to 1.7.0](cloudfoundry/bosh@6365a5d) - 2024-07-09T12:37:17Z: [bump ruby-prof version from 1.6.3 to 1.7.0](cloudfoundry/bosh@6365a5d) - 2024-08-07T13:24:53Z: [update gosigar version](cloudfoundry/bosh-dns-release@c1be1b9) - 2024-08-08T08:33:14Z: [update gosigar version](cloudfoundry/bosh-dns-release@c1be1b9) - 2024-08-09T06:40:52Z: [fix formatting for configure blobstore for gcp section](cloudfoundry/docs-bosh@68d35c4) - 2024-11-11T12:11:17Z: [link should point to main branch and not master branch](cloudfoundry/docs-bosh@e919e42) - 2024-11-12T08:10:05Z: [ensure that cpi uses the correct gem path during runtime](cloudfoundry/bosh-azure-cpi-release@c49f723) - 2024-11-13T20:44:53Z: [revert fastjsonparser](cloudfoundry/bosh-azure-cpi-release@ed1bbcc) - 2024-11-13T20:49:18Z: [revert fastjsonparser](cloudfoundry/bosh-azure-cpi-release@ed1bbcc) - 2024-11-19T15:14:44Z: [Merge branch 'master' into revert-fastjsonparser](cloudfoundry/bosh-azure-cpi-release@f7f412d) - 2024-11-19T15:37:05Z: [Merge branch 'main' into optional_isolated_environment](cloudfoundry/bosh-cli@72c44e6) - 2024-11-19T15:39:06Z: [Update compiler.go](cloudfoundry/bosh-cli@2e81740) - 2024-11-19T19:35:33Z: [fix env_factory](cloudfoundry/bosh-cli@8867168) - 2024-11-19T19:44:10Z: [fix compiler.go](cloudfoundry/bosh-cli@eacc737) - 2024-11-21T10:25:08Z: [fix compiler.go](cloudfoundry/bosh-cli@eacc737) - 2024-11-21T13:40:25Z: [Update pre-start.sh.erb for noble stemcell](cloudfoundry/os-conf-release@2aab03c) - 2024-11-21T16:09:16Z: [add test for compiler.go](cloudfoundry/bosh-cli@8f84216) - 2024-12-18T08:57:31Z: [remove nix definitions](cloudfoundry/bosh-cli@bdf1b53) - 2024-12-19T09:54:34Z: [do not raise error in case more networks than network interface cards are applied](cloudfoundry/bosh-agent@3c63d2b) - 2024-12-30T12:22:59Z: [enable dual stack support on aws](cloudfoundry/bosh-aws-cpi-release@5cbfb80) - 2025-01-02T15:02:59Z: [enable dual stack support on aws](cloudfoundry/bosh-aws-cpi-release@5cbfb80) - 2025-01-08T08:39:04Z: [fix unit tests](cloudfoundry/bosh-agent@4e1db30) - 2025-01-08T12:32:44Z: [create tests](cloudfoundry/bosh-aws-cpi-release@463ff5e) - 2025-01-29T13:02:31Z: [remove isolated env](cloudfoundry/bosh-cli@ba77137) - 2025-01-29T13:11:25Z: [remove isolated env](cloudfoundry/bosh-cli@ba77137) - 2025-01-30T15:47:00Z: [Delete .tool-versions](cloudfoundry/bosh-cli@799e6e5) - 2025-01-31T09:49:58Z: [Update opts.go](cloudfoundry/bosh-cli@8bfb6a6) - 2025-02-03T12:26:38Z: [set retry interval to avoid bosh workers sleeping forever in case of error](cloudfoundry/bosh@23d5899) - 2025-02-06T15:54:40Z: [enable ipv6 in boshSysctl (cloudfoundry#347)](cloudfoundry/bosh-agent@a7d6a6b) - 2025-02-07T11:40:08Z: [Update kernel_ipv6.go](cloudfoundry/bosh-agent@a79e5fc) # Foundational Infrastructure: Ali Cloud VM deployment lifecycle (BOSH) Contributions ### PRs Commented on/Reviewed: ### Issues that may be relevant: - 2023-08-14T11:48:57Z: [Changing Tags from uppercase to lowercase ends in ](cloudfoundry/bosh-alicloud-cpi-release#162) - 2023-11-23T11:24:07Z: [update dependencies](cloudfoundry/bosh-alicloud-cpi-release#165) - 2023-11-28T17:20:05Z: [Update Documentation](cloudfoundry/bosh-alicloud-cpi-release#166) - 2024-11-28T12:56:51Z: [use bash from PATH](cloudfoundry/bosh-alicloud-cpi-release#174) - 2024-11-28T12:57:06Z: [bosh create-env terminates with: "cannot execute: required file not found"](cloudfoundry/bosh-alicloud-cpi-release#175) ### Code contributions: - 2023-11-23T11:23:20Z: [update dependencies](cloudfoundry/bosh-alicloud-cpi-release@7f50084) - 2023-11-28T17:18:16Z: [Update Documentation](cloudfoundry/bosh-alicloud-cpi-release@a72e168)

Enable golang CPI implementation

c54a6b4

OBoehrer force-pushed the enable_golang_cpi branch from 9ed0509 to c54a6b4 Compare October 18, 2024 13:17

beyhan mentioned this pull request Oct 18, 2024

Adding multiqueue support for Virtio to the stemcell #289

Merged

aramprice reviewed Oct 21, 2024

View reviewed changes

jobs/openstack_cpi_golang/spec Outdated Show resolved Hide resolved

Remove Ruby source from spec

72324e4

cloudfoundry deleted a comment from ShilpaChandrashekara Oct 22, 2024

a-hassanin added 2 commits October 22, 2024 15:24

Delete templates cpi-ruby.erb

8c86592

Delete cpi-golang.erb

1219c28

a-hassanin requested changes Oct 22, 2024

View reviewed changes

OBoehrer added 2 commits October 23, 2024 10:45

remove tmp dir specification

51709e2

tmp dir specification add changes

b42d4f3

a-hassanin requested changes Oct 23, 2024

View reviewed changes

src/openstack_cpi_golang/cpi/volume/volume_service_test.go Show resolved Hide resolved

OBoehrer added 2 commits October 23, 2024 15:26

race condition removed

2c207d8

tmp file path win

826769a

a-hassanin approved these changes Oct 23, 2024

View reviewed changes

OBoehrer force-pushed the enable_golang_cpi branch from d810be1 to 826769a Compare October 24, 2024 12:19

rkoster requested review from a team, rajathagasthya and selzoc and removed request for a team October 24, 2024 14:48

aramprice approved these changes Oct 24, 2024

View reviewed changes

jpalermo merged commit e723996 into cloudfoundry:master Oct 31, 2024

a-hassanin mentioned this pull request Nov 19, 2024

Enhance Github Workflows #294

Merged

fmoehler mentioned this pull request Mar 6, 2025

Update foundational-infrastructure.md cloudfoundry/community#1095

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable CPI Release golang #290

Enable CPI Release golang #290

Uh oh!

OBoehrer commented Oct 18, 2024 •

edited

Loading

Uh oh!

linux-foundation-easycla bot commented Oct 18, 2024 •

edited

Loading

Uh oh!

fmoehler commented Oct 21, 2024 •

edited by a-hassanin

Loading

Uh oh!

Uh oh!

a-hassanin left a comment

Uh oh!

Uh oh!

a-hassanin left a comment

Uh oh!

aramprice commented Oct 24, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Enable CPI Release golang #290

Enable CPI Release golang #290

Uh oh!

Conversation

OBoehrer commented Oct 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Outcome of this PR

Background

Difference

Note

Uh oh!

linux-foundation-easycla bot commented Oct 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fmoehler commented Oct 21, 2024 • edited by a-hassanin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

a-hassanin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

a-hassanin left a comment

Choose a reason for hiding this comment

Uh oh!

aramprice commented Oct 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

OBoehrer commented Oct 18, 2024 •

edited

Loading

linux-foundation-easycla bot commented Oct 18, 2024 •

edited

Loading

fmoehler commented Oct 21, 2024 •

edited by a-hassanin

Loading

aramprice commented Oct 24, 2024 •

edited

Loading