Visual Studio Geeks

Exploring GitHub Advanced Security for Azure DevOps

Utkarsh Shigihalli — Sat, 17 Feb 2024 00:00:00 +0000

It has been a few months since GitHub Advanced Security (GHAS) has been made generally available for Azure DevOps. During this time, I’ve engaged with numerous customers eager to implement GHAS within their Azure subscriptions. In this post, I wanted to show you a quick way to set up GHAS within Azure DevOps and explore the features available.

Enabling Advanced Security in Azure DevOps

This is easy, however, you need to be a member of the Project Collection Administrator group. You can verify that from Organization Settings -> Permissions and then the Members tab. You should see your name.

Once you have verified you are a Project Collection Administrator, you are ready to enable GHAS.

You can enable GHAS either individually per repo or across the organisation for all the repositories.

If you want to enable it for all the repositories in your organization, you will need to go to organization settings and then the Repositories section and enable it there.

For this post, I am enabling it only for a single repository. Once you click the Advanced Security flag (1), you will be prompted to show the number of committers you will be billed against (more on billing below). This repository has only me committing to it, so it has correctly identified 1 active committer (2).

Once you click Begin billing GHAS should be enabled.

If your ADO instance does not have a linked and active subscription, you might get the below error.

You will need to select an active Azure subscription under the Billing tab under organization settings in ADO.

Once you select a valid subscription, you will be able to enable GHAS for the repositories.

Exploring GHAS features

Block secrets on push

Once you enable GHAS, by default Block secrets on push feature is enabled too. With this setting enabled, ADO will automatically check any incoming pushes for embedded secrets and reject them automatically. Not only works on CLI, but it works on the web interface too.

For a simple test, below I am trying to commit a file with the GitHub API key, and it was rejected.

Note that at the time of writing this GHAS supports secrets push protection only for certain service providers. Although secrets from the majority of service providers are supported, I was surprised to see GitLab personal access token is not supported yet.

Although not recommended, there is a way to push a secret that has been blocked. To push, you need to have skip-secret-scanning:true in your commit message.

This will allow the secret to be committed, however, it will be caught in the Secret scanning alert (more on that below).

The great thing about GHAS is that clicking on the alert will give you remediation steps too.

Dependency Scanning and Code Scanning

Dependency and Code scanning are additional features of GHAS. Dependency scanning will scan any vulnerabilities in your repo from your open-source dependencies. Code scanning will scan your source code (from supported languages) for vulnerabilities.

Both these tasks are enabled through pipeline tasks. On any GHAS-enabled repo, you will be able to run the pipeline with these tasks and get the status of the repo.

You can create a pipeline and add the tasks for dependency and code scanning.

However, this post has already gotten super long than I intended it to be, I will probably do another post and explore both of those functionalities in detail.

Pricing

GHAS for Azure DevOps is a paid product and is available only for Azure DevOps Services. For Azure DevOps Service linked to an Azure Subscription, this will automatically be visible in your billing for subscription. At the time of writing this, it costs 49$ per active committer per month.

That is it for this post. Thank you for reading. 🎉

Use a single repository for multiple wikis in Azure DevOps

Utkarsh Shigihalli — Thu, 07 Dec 2023 00:00:00 +0000

This is a quick post to share how you can use a single Azure DevOps repository for multiple wikis in Azure DevOps. This approach will help you avoid creating multiple repositories for each wiki and manage them in a single repository.

Create a new repo for wiki

Firs lets create a separate repository for the wiki.

Create a repository

I want to create two wiki for two teams in my project. So I have created subfolders for each team in the repository and have placed a sample markdown files in each folder. The idea is that each team will place their wiki content in their respective folder.

Folder structure

Publish code as a new wiki

Now lets create a new wiki in Azure DevOps. Go to the project settings and click on the Wiki tab. We will publish new code wiki.

Publish code as wiki

Do the same for the second wiki.

Publish code as wiki

This will create two wikis in the project.

Configure the wiki

As you can see that was easy. However, at it stands now, both the wikis are open for edits from all the team members. We need to restrict the access to the wiki content based on the team. We will make use of branch policies to achieve this.

Set the branch policy

Now, when any time changes are made to the wiki under 01-team-infra/* folder and a PR is created, Infra team will automatically be included as reviewers. Similarly, for the 02-team-app/* folder, App team will be included as reviewers.

Pull request showing automatic reviewers added

That is it for this post. You can now use a single Azure DevOps repository for multiple wikis in Azure DevOps. As mentioned above, this approach will help you avoid creating multiple repositories for each wiki and manage them in a single repository.

Hope you enjoyed reading this.

Creating KEDA Scalar in Azure Container Apps using Azure Portal

Utkarsh Shigihalli — Wed, 22 Nov 2023 00:00:00 +0000

In this post, I would like to show you how we added custom scaling with KEDA using the Azure Portal — Thanks to the KEDA scaler, we have a dynamic scaling pool, which automatically scales when there are more jobs in the queue and scales back down when demand reduces.

We run our Azure DevOps Build agents inside a container. This has allowed us to package various tools like kubectl, helm, and terraform are installed and available inside the agent image, which gives us control over versions of the tools we use — as we can execute our continuous integrations with consistent configuration. Also, adding any new tool is just adding installation instructions to the Dockerfile and publishing a new image.

Microsoft has detailed documentation on running Azure DevOps agent inside a container here

Further, we are running our agents as an Azure Container App, which has freed us from maintaining a dedicated AKS cluster and lets us dynamically scale the agents — a new agent per pipeline job with the help of custom scaling rules and KEDA.

Creating an Azure Container App

Creating an Azure Container App can be done in a variety of ways — Terraform or any other IaC code, Azure CLI, or through Portal. We are using Terraform internally, but for the sake of this post, I am showing creating using the portal.

So, search for the service and select Container App.

The first step is the provide the name and select a region and Container Apps environment. I have decided to use the existing environment below.

configuring the container app

The next tab in the wizard is about the container. Our team-specific image is in our internal Azure Container Registry and the wizard fills the drop-downs for easy selection.

The only change I have made here is, changing the CPU and Memory values as needed for the image — Notice I am using Consumption Plan as we know this team does not need high-performance agents.

As conveyed previously, this feature of allocating individual CPU and Memory configurations has great benefits over AKS. For example, we have a separate container app with a dedicated workload profile for running CPU and Memory-intensive jobs.

specify the container

The rest of the wizard is left at defaults as I do not have any bindings or Ingress to configure.

This should get your container app running.

Create a secret to store Azure DevOps PAT

The first step in the container app is to Add a secret and store our Azure DevOps PAT (Personal Access Token). PAT is needed for the Azure DevOps build agent to connect to our Azure DevOps service. Later in the post, we use this PAT to let KEDA authenticate to our Azure DevOps service to monitor the Agent Pool for new jobs.

Add PAT as a secret

Create a new revision

The next step is to edit the container and define the few environment variables that are required for the container (for more on these specific environment variables refer to this documentation).

Notice, for AZP_TOKEN the source is set as Reference a secret as I want value for that to come from the secret defined in the previous step.

Edit container environment variables

Create a custom Scale rule

The next and final step is to define the custom scale rule — which in our case is KEDA.

So in the Scale tab, click +Add and then enter the details below

Rule name: This is the name for the custom scale rule, can be anything.
Type: Custom
Custom rule type: This is defined with the Scaler definition. I am using Azure Pipelines scaler, so this should match type field of the Scaler definition.

Next, we need to add a few metadata values so that KEDA knows which agent pool should it monitor for new jobs. These will come from metadata keys of the scaler definition.

peronalAccessTokenFromEnv: This lets KEDA authenticate with our Azure DevOps service to monitor the Agent Pool. The value is PAT we defined in the secret previously — which has been passed to the container as an environment variable, so we use that environment variable.
organizationURLFromEnv: This is our Azure DevOps organization URL, again we set this as an environment variable in the previous section.
poolID: This is the Azure DevOps pool ID which KEDA should monitor. Refer to the scaler docs on how to get this.

Add metadata for the scaler

That is it for the post. Hope you found it useful.

Conclusion

In this post, I showed how we added custom scaling with KEDA using the Azure Portal. Thanks to the KEDA scaler, we have a dynamic scaling pool, which automatically scales when there are more jobs in the queue and scales back down when demand reduces.

Installing Docker without Docker Desktop on WSL2

Utkarsh Shigihalli — Wed, 04 Oct 2023 00:00:00 +0100

I have been using Docker on WSL2 without Docker Desktop for a while now and recently I had to rebuild my Ubuntu distro. I had to go through installing the Docker Engine once again following a series of steps, so thought I would document it so that I can use it again as a reference.

This post is just that and I hope others will find it useful too.

Prerequisites

Before installing Docker in WSL2, first make sure you are running WSL2. You can do that using wsl --version in your PowerShell which should output as below.

You can also check you are running the Ubuntu version using lsb_release -a command which should output as below

Installing Docker

Now that you confirmed you have the prerequisites, let’s start by installing Docker. Docker documentation provides a convenient script for us to run, which just involves running the script below.

However, you might get the error Updates for this repository will not be applied

A quick StackOverflow search suggested that this is caused due to the clock being out-of-sync between the Windows machine and the Ubuntu distro running in WSL2. To sync the clock, I had to run sudo hwclock --hctosys and restart the WSL using wsl --shutdown` and start WSL again.

The second attempt to run the command should be successful.

Creating docker group and adding the current user

Due to the way the Unix socket works, we need to preface docker command with sudo. To avoid it, Docker recommends creating a group name docker and adding the current user to it. Docker daemon can then create a Unix socket accessible by members of the docker group. So, first, create docker group using sudo groupadd docker then add yourself to that group using the command

sudo usermod -aG docker $USER

Finally, run newgrp docker to activate the changes to the group.

Test running Docker

You can now test the docker commands without sudo. Try running docker run hello-world and it should run correctly.

Ensuring Docker daemon starts when you open WSL

Finally, to ensure you have the docker daemon start when you open WSL, you can run the below commands which will start the docker service as soon as you boot.

That is it, you should be able to run all your docker commands without Docker desktop now.

Trigger a Netlify build every day using GitHub Actions

Utkarsh Shigihalli — Sun, 20 Feb 2022 00:00:00 +0000

I host this blog on Netlify. Often, I end up writing few blog posts on the same day, but not necessarily want all of them published together. Jekyll allows to add future date to the posts, and those posts will not get published until the date set. This lets me write blog posts on the same day, but publish them later based on the date set for the post.

However, this means that when you build the site (locally or on Netlify), posts with future dates will be skipped from the generated site. You will need to run build again on the blog post date to publish the blog posts. One way is to schedule a build on Netlify so that blog posts with future date can be included in the site. Curious how to do this?

Jekyll skipping posts with future date

Create Netlify build hook

Netlify supports build hooks, which lets you trigger new builds and deploys by making a request to a URL.

Configure build hooks

Go to Site Settings and Build Hooks section and click Add Build Hook button.

Add build hook

Give a name and select a branch to trigger the build and click Save. You will see the URL which you can use to trigger the build.

URL of the build hook

Create GitHub Actions workflow

Go to GitHub repo and create a actions yaml file under .github\workflows folder. GitHub Actions supports scheduled jobs, which lets you run a job at a specific time. See the below example. I am using cron expression to run the build every day at 04:00 AM UTC timezone.

name: nightly-netlify-build

on:
  schedule:
    - cron: "1 4 * * *"

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: trigger netlify build
        run: |
          curl -X POST -d '{}' https://api.netlify.com/build_hooks/2eae8d5c79506d0213a38be0e

Note, if you hover over the cron expression, GitHub nicely displays the tooltip explaining the cron expression.

GitHub shows cron expression detail

Save the file and push the changes. That is it, you have created a GitHub Actions workflow which can trigger nightly build of the blog. Go to Netlify and see the Deploys tab. You will see the deploys triggered by the build hook.

Deploys triggered by the build hook

Isn’t that cool? If you liked the post, do share it in your social channels. Thanks for reading.

Policy enforced deployments for your Kubernetes resources

Utkarsh Shigihalli — Sun, 13 Feb 2022 00:00:00 +0000

As your team starts to deploy resources to Kubernetes regularly, it becomes necessary for you as a cluster administrator to maintain good standards and consistency of the Kubernetes resources. Be it, ensuring all the resources have set of labels, or ensuring you only pull images from your enterprise container registry. Gatekeeper is a well known policy enforcement tool using Open Policy Agent (OPA) - which is a opensource, Cloud Native Computing Foundation (CNCF) project.

But did you know you can validate policies on your Kubernetes manifests before you deploy them on to the cluster? In this post, we will see how we can govern our deployments using Conftest and OPA policy agent.

However, Gatekeeper is installed on the cluster and thus ensures no policy is broken at deployment time. This means that any validation of policies happen only when you are trying to deploy resources to cluster. While this ensures that no resource violates the policy, you would like to know about these policies much earlier in your CI/CD pipeline. Doing policy validations much to the left of your deployment pipeline ensures your deployment going smooth when necessary.

This is where Conftest helps. Conftest relies on OPA and policies are written using Rego - thus the policies you write for Gatekeeper will be compatible with Conftest. But more importantly with Conftest, you can validate your local manifests against OPA policies locally and ensure your resources are compliant before you deploy them.

Installation

Installation is really easy if you are on Mac - For other platforms refer to the documentation

brew install conftest

Folder structure

By default, Conftest expects you to maintain your policies under policy folder at the same location as your Kubernetes resources. If you prefer a different path, you will want to pass it using CLI or set environment variable CONFTEST_POLICY.

📂 src
    📂 k8s
        📄 deployment.yml
        📄 service.yml
        📁 policy
            📄 replica.rego
            📄 labels.rego
    📂 app
        📄 main.ts
        📄 package.json

Writing Policies

As mentioned previously, policies are written in Rego. I struggled to write policies initially and constantly went back to documentation. However, once you write couple of policies, you will get a hang of it. Take a look at the simple policy to check every deployment has at least 2 replicas.

package main

deny_replicas[msg] {
    input.kind == "Deployment"                          # check if it is a Deployment
    input.spec.replicas < 2                             # And the replicas are < 2
    msg := "Deployments must have 2 or more replicas"   # show the error message and fail the test
}

input is the complete yaml document from our deployment yaml (see below) and we we are checking if kind is equal to Deployment. If its deployment, we move to next line in the constraint and check if spec.replicas is less that 2.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mynodeapi-dep
  labels:
    app: dep-k8s-nodejs-api
spec:
  replicas: 1
  selector:
    ...

You can write other policies similar to the one above to validate various aspects of Kubernetes resources. Let us see few examples.

Validate resources have recommended labels

This policy validates our resources have the required labels and fail if any labels from required_deployment_labels object are not found.

package main

import data.kubernetes

name = input.metadata.name

required_deployment_labels {
	input.metadata.labels["app.kubernetes.io/name"]
	input.metadata.labels["app.kubernetes.io/instance"]
	input.metadata.labels["app.kubernetes.io/version"]
	input.metadata.labels["app.kubernetes.io/component"]
	input.metadata.labels["app.kubernetes.io/part-of"]
	input.metadata.labels["app.kubernetes.io/managed-by"]
}

violation[msg] {
	input.kind == "Deployment"
	not required_deployment_labels
	msg = "Must include Kubernetes recommended labels: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels"
}

Reference container images only from our enterprise Azure Container Registry

package main

deny[msg] {
    input.kind == "Deployment"
    some i
    image := input.spec.template.spec.containers[i].image
    not startswith(image, "myacr.azurecr.io") # validate images start with endpoint for our container registry
    msg := sprintf("image '%v' comes from untrusted registry", [image])
}

As you can see rules can be very powerful.

Testing

The command to test resources using Conftest is conftest test <PATH>. Since I would like to test all resources under k8s folder, I pass folder path as below.

conftest test ./k8s

Running this you will see the output as below (ignore other errors as I have other policies). The test failed because we have set deny rule if spect.replicas < 2 and in our case our deployment yaml has replicas: 1 (see spec section in the deployment yaml above).

Conftest failing due to policy violation

Using Conftest in GitHub Actions

Making Conftest work in your Continuous Integration (CI) process is simple. For demo purposes, I am using GitHub Actions in my repo here. If you run the tests, you will see the action fails with errors - see the output

My action workflow looks like below.

name: build

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: install conftest
        run: |
          wget https://github.com/open-policy-agent/conftest/releases/download/v0.30.0/conftest_0.30.0_Linux_x86_64.tar.gz
          tar xzf conftest_0.30.0_Linux_x86_64.tar.gz
          sudo mv conftest /usr/local/bin
          rm -rf conftest_0.30.0_Linux_x86_64.tar.gz

      - name: run conftest
        run: |
          conftest test $/k8s

Conclusion

As you can see, Conftest lets you validate and govern your Kubernetes resources efficiently and can easily be integrated with your CI workflows. This lets your team standardise the common practices, go through PR review process before eventually deploying to the cluster. Once deployed to cluster, you can use Gatekeeper to validate as well to full proof your workloads.

Keep your workflow actions up to date using GitHub Dependabot

Utkarsh Shigihalli — Fri, 28 Jan 2022 00:00:00 +0000

GitHub Actions is great in automating your workflows. However, as you start using various actions from GitHub Marketplace in your workflow, it will soon become necessary for you to keep the actions up-to-date. Actions might contain security fixes, bug fixes etc and manually keeping track of updates or updating them when a newer version is available is a lot of hassle. This is where we can use Depndabot, which can help by automatically raising PR’s whenever there is a newer version of action is available used in the workflow. In this post, we will see quick way to keep the actions up-to-date using GitHub Dependabot.

For this post, I am using my Git Config User Profiles repository. I have workflow setup which builds and releases the VS Code extension to VS Marketplace.

Create dependabot.yml file

To setup Dependabot scan, first got to .github folder in your root and create a depndabot.yml file. Then add the following content. This will ensure GitHub Dependabot raise a PR whenever there is a newer version of action is available

version: 2
updates:
  - package-ecosystem: "github-actions" # search for actions - there are other options available
    directory: "/" # search in .github/workflows under root `/`
    schedule:
      interval: "weekly" # check for action update every week

Commit the file

Commit the file created above and wait for few seconds. Based on your workflow, you will see a bunch of PR’s raised.

Dependabot Alerts as PR

If you look at the PR, you will be able to see the change and take a decision whether you want to upgrade the specific action or not. If you decide to accept the change, merge the PR and the changes on the workflow file will be made.

Commit Details

Conclusion

Isn’t it cool? This saves a lot of time, if you have a number of workflows and don’t want to keep checking the latest versions of actions. BTW, not only GitHub actions, you can use the same approach to update npm, docker and many more using various package ecosystems. Do check it out!

Setting up Azure Cosmos DB Emulator on Synology NAS

Utkarsh Shigihalli — Tue, 27 Jul 2021 00:00:00 +0100

Cosmos DB Emulator is great for developing against Azure Cosmos DB in your local environment. If you want to run the emulator on Mac/Linux though, the emulator is now in preview mode at the time of writing this and uses Docker to make it available.

Recently I wanted to setup Cosmos DB Emulator on Mac, but I did not want to set it up such that I keep it running always on my Mac. Instead, I decided to use my Synology NAS to host the emulator directly on my NAS as it is built on Linux and also because it allows the emulator running and available for me all the time.

In this post we will see how to set it up on Synology NAS.

The goal is to run Cosmos DB emulator as a Docker container on Synology.

Enabling SSH on Synology

Since Cosmos DB Emulator image is hosted on Microsoft Registry (mcr.microsoft.com) which does not provide UI/discovery service, Synology’s Docker app, cannot pull images from Microsoft Registry. For this purpose we need to enable SSH on Synology so that we can SSH to Synology and run docker commands directly.

If you have firewall enabled on Synology, you must also Allow port 22.

Pulling the docker image

After enabling SSH on SYnology, you can SSH on to Synology and run the below command to pull the Docker image of Cosmos Emulator.

sudo docker pull mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator

You should see Cosmos DB Emulator image available in Syonology’s Docker app.

Starting the Container

You can now create a container using the docker image you just pulled. Before we start, as per the Microsoft documentation for the Emulator, we need to ensure we have setup few environment variables for the container. They are

AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE = <<SYNOLOGY_IP_ADDRESS>>
AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE = true
AZURE_COSMOS_EMULATOR_PARTITION_COUNT = 10

Finally, you will need to ensure you have mapped these ports for the container. Ensure you have opened these below ports on the Synology’s Firewall as well

For more information on creating container from docker image, see Synology’s documentation.

Install the certificate on your machine

To avoid errors due to certificate, you need to download the self-signed certificate for the emulator and install it on your machine. To download the certificate for the emulator run below command

curl -k https://<<SYNOLOGY_IP_ADDRESS>>:8081/_explorer/emulator.pem > emulatorcert.crt

On my Mac, I had to install it on Keychain Access App and set it as Always Trust.

Once done you should be able to browse the Cosmos Emulator UI using https://<<SYNOLOGY_IP_ADDRESS>>:8081/_explorer/index.html

That is it! Your Cosmos Emulator is now available to use all the time.

Reference

How to publish Helm 3 charts to GitHub Container Registry using GitHub Actions

Utkarsh Shigihalli — Sat, 10 Apr 2021 00:00:00 +0100

I have already written how to publish Helm chart to ACR using Azure DevOps and GitHub actions. But did you know that you can also publish Helm3 charts (or any OCI compliant package) to GitHub Container Registry(GCR)? In this post we will see how to do that.

Enable improved container support

GitHub Container Registry is currently in public beta. So, the first step is to ensure that you have enabled the GitHub Container Registry for your account. If you have GitHub personal account, you can do that from Feature Preview window.

If you have a Enterprise account, you can do that going to Settings page.

GitHub documentation has step-by-step guide of enabling improved container support here

Publishing Helm 3 charts using GitHub Actions

It really takes only couple of steps to publish a Helm chart to GCR using GitHub Actions. Like any other action, you start by creating .github\workflow folder and create an yml file in your repository.

Excluding the name and trigger part, first step in the YAML is to define few necessary variables.

env:
  HELM_EXPERIMENTAL_OCI: 1 #enable OCI support
  HELM_VERSION_TO_INSTALL: 3.5.0 # version of HEL to install
  GCR_IMAGE: ghcr.io/${{ github.repository_owner }}/vote-app

First variable enables the OCI support for the Helm commands we are going to run later in the YAML.

Next variable HELM_VERSION_TO_INSTALL is used later in the workflow to install specific version of the Helm. For this workflow, we need 3.5.0.

Last variable GCR_IMAGE is constructing the chart name for the publication.

The GitHub Container Registry hosts containers at ghcr.io/OWNER/IMAGE-NAME. I get the OWNER of the repository using github.repository_owner from github context.

Next steps are defining the job with steps to download the code (where we have Helm chart) and install the specific Helm tool on the runner/agent.

jobs:
  build:
    name: publish gcr
    runs-on: ubuntu-latest
    environment: prod
    steps:
      - uses: actions/checkout@v2
        name: checkout repo
      
      - name: install helm
        uses: Azure/setup-helm@v1
        with:
          # Version of helm
          version: ${{ env.HELM_VERSION_TO_INSTALL }} # default is latest

The above steps setup the agent machine with the required Helm tool.

Next, we need to run few helm commands to login to GCR (GitHub Container Registry) and finally publish the chart.

We login to GCR using ${{ secrets.GITHUB_TOKEN }}

GitHub Container Registry only recently started supporting GITHUB_TOKEN. Previously you had to create a separate PAT token with specific permissions to GCR.
Next two steps in the workflow will be to save the chart and publish. We do that using helm chart save and help chart push commands as shown below.

- name: login to acr using helm
  run: |
    echo ${{ secrets.GITHUB_TOKEN }} | helm registry login ${{ env.GCR_IMAGE }} --username ${{ github.repository_owner }} --password-stdin

- name: save helm chart to local registry
  run: |
    helm chart save ${{ github.workspace }}/src/azure-vote-helm-chart/ ${{ env.GCR_IMAGE }}:${{ github.sha }}

- name: publish chart to acr
  run: |
    helm chart push ${{ env.GCR_IMAGE }}:${{ github.sha }}

That is it, if all worked successfully for you, you should see the chart in the GCR.

Publish Helm 3 charts to Azure Container Registry (ACR) using GitHub Actions

Utkarsh Shigihalli — Thu, 01 Apr 2021 01:00:00 +0100

In my previous post, we briefly covered how to publish a Helm chart to ACR using Azure DevOps. In this post we will use GitHub actions to build and publish Helm chart to ACR using GitHub Actions. We will also take a sneak peak how GitHub environments work.

Pre-requisites

I am going to assume ACR instance is setup using repository scoped tokens. Since we already covered setting up of ACR this way in the earlier post, I will not include the steps here.

Setting up secrets at GitHub

We would like store Azure Container Registry’s tokens as GitHub repository level secrets. To do that, click on Settings on the repository page and head to Secrets tab. Finally click on New repository secret and add the token name and the password. I have stored token name as ACR_PUSH_USER and token password as ACR_PUSH_TOKEN.

Add repository secrets

Creating the workflow in GitHub Actions

Publish chart to ACR

The first step is to create an yaml file under .github\workflows folder and setup a basic structure. The first things (see the yaml below) are defining name for the action, currently set to trigger via manual trigger using workflow_dispatch and define few environment variables which we are going to use later in the action.

name: ci

on: 
  workflow_dispatch:

env:
  HELM_EXPERIMENTAL_OCI: 1
  HELM_VERSION_TO_INSTALL: 3.5.0
  ACR_NAME: acrdemoutkarsh
  ACR_REPO_NAME: helmdemo/vote-app

The first environment variable conveys to ACR that we are going to publish a OCI package. Next couple of variables just define version of Helm we need on the runner, our ACR name to which we are going to publish this chart and finally to the repository we are publishing this chart to (used in below sections).

Installing Helm 3 on the agent

Now that we have all the variables defined, we need add jobs and steps to build our workflow to publish charts to ACR. We then need to install Helm tool on the agent before we can run the Helm commands. We do that using yaml below.

jobs:
  build:
    name: publish acr
    runs-on: ubuntu-latest
    environment: prod
    steps:
      - uses: actions/checkout@v2
        name: checkout repo
      
      - name: install helm
        uses: Azure/setup-helm@v1
        with:
          version: ${{ env.HELM_VERSION_TO_INSTALL }}# default is latest

As you can see, we have one job named build (which will be displayed as publish acr - see screenshot below) which runs on ubuntu-latest agent. We also are targeting our deployment to an environment prod. Environments in GitHub are cool because you can have approvers, additional protection rules for environments and environment specific secrets. In the screenshot below, notice how the flow is waiting for review.

Next, we checkout the repository and using setup-helm task from Azure repo we install the specific version (3.5.0) of Helm.

Next, we need to login to ACR registry using Helm tool.

- name: login to acr using helm
  run: |
    echo $ | helm registry login $.azurecr.io --username $ --password-stdin 

Save and push the chart to ACR

Next we need to save the chart directory to local cache and publish it to ACR.

- name: save helm chart to local registry
  run: |
    helm chart save $/src/azure-vote-helm-chart/ $.azurecr.io/$:latest
      
- name: publish chart to acr
  run: |
    helm chart push $.azurecr.io/$:latest

Run the workflow, and you will see output as below.

Go to ACR and you will see char correctly published to helmdemo/vote-app repository as declared in the env section above.

Conclusion

In this post, you saw how easily we can deploy a OCI package (helm3 chart) to ACR using GitHub actions. We also saw how GitHub environments help you approve changes to the environment. Hope you enjoyed reading this post.

Visual Studio Geeks

Exploring GitHub Advanced Security for Azure DevOps

Enabling Advanced Security in Azure DevOps

Exploring GHAS features

Block secrets on push

Dependency Scanning and Code Scanning

Pricing

Use a single repository for multiple wikis in Azure DevOps

Create a new repo for wiki

Publish code as a new wiki

Configure the wiki

Creating KEDA Scalar in Azure Container Apps using Azure Portal

Creating an Azure Container App

Create a secret to store Azure DevOps PAT

Create a new revision

Create a custom Scale rule

Conclusion

Installing Docker without Docker Desktop on WSL2

Prerequisites

Installing Docker

Creating docker group and adding the current user

Test running Docker

Ensuring Docker daemon starts when you open WSL

Trigger a Netlify build every day using GitHub Actions

Create Netlify build hook

Create GitHub Actions workflow

Policy enforced deployments for your Kubernetes resources

Installation

Folder structure

Writing Policies

Validate resources have recommended labels

Reference container images only from our enterprise Azure Container Registry

Testing

Using Conftest in GitHub Actions

Conclusion

Keep your workflow actions up to date using GitHub Dependabot

Create dependabot.yml file

Commit the file

Conclusion

Setting up Azure Cosmos DB Emulator on Synology NAS

Enabling SSH on Synology

Pulling the docker image

Starting the Container

Install the certificate on your machine

Reference

How to publish Helm 3 charts to GitHub Container Registry using GitHub Actions

Enable improved container support

Publishing Helm 3 charts using GitHub Actions

Publish Helm 3 charts to Azure Container Registry (ACR) using GitHub Actions

Pre-requisites

Setting up secrets at GitHub

Creating the workflow in GitHub Actions

Publish chart to ACR

Installing Helm 3 on the agent

Login to the ACR using Helm

Save and push the chart to ACR

Conclusion