Skip to content

Conversation

@DiamondJoseph
Copy link
Contributor

@DiamondJoseph DiamondJoseph commented Apr 10, 2025

Closes #234

Adds the option to create a -debug image when publishing a container from a project managed by the copier template and documentation on how to use the debug image to debug within the DLS cluster infrastructure.

I have made a test repository with Helm configuration and an example service which can be fixed internally to the cluster. I have a PR with commits that I made inside a devcontainer inside the cluster to demonstrate how it can be used.

@DiamondJoseph DiamondJoseph marked this pull request as ready for review April 10, 2025 16:14
gilesknap
gilesknap previously approved these changes Apr 14, 2025
@gilesknap
Copy link
Contributor

gilesknap commented Apr 14, 2025

Overall this is a neat solution - great work.

  • I would like to review the mounting of your home directory - not sure we need that?
  • should copier template include a helm chart? then we can supply full an example of how to use the sidecar - the addition to the docs seems to assume a particular helm values.yaml

Other than that my comments were minor.

@DiamondJoseph DiamondJoseph changed the title Publish debug container image and account-sync sidecar feat: Publish debug container image and account-sync sidecarf Apr 14, 2025
@DiamondJoseph DiamondJoseph changed the title feat: Publish debug container image and account-sync sidecarf feat: Publish debug container image and account-sync sidecar Apr 14, 2025
Copy link

@olliesilvester olliesilvester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this seems like a good solution to do something quite complicated. A few comments

  • As we had a few mysterious issues when trying to get it to work for me, I don't think we can trust the steps in the docs to reliably "just work" yet.
  • The details of how this works is pretty confusing for a kube noob, although conceptually "swapping out one container for another and attaching vscode" does make sense
  • I agree with @gilesknap 's comment about adding a full helm chart example in the copier template
  • To prove this is robust, we should do an iterative process of choosing someone in daq/controls and asking them to try and get this working using only the docs, then adjusting the docs with whatever they were missing

@gilesknap
Copy link
Contributor

FYI I'm testing this PR against PandaBlocks-ioc. If anyone else wants to try it see the debug container at https://github.com/gilesknap/PandABlocks-ioc/pkgs/container/pandablocks-ioc/406412377?tag=1.0.0b1-debug

@gilesknap
Copy link
Contributor

FYI I'm testing this PR against PandaBlocks-ioc. If anyone else wants to try it see the debug container at https://github.com/gilesknap/PandABlocks-ioc/pkgs/container/pandablocks-ioc/406412377?tag=1.0.0b1-debug

So this works for me. For a python] IOC I needed to change the values.yaml as below.

Use of root is being discouraged by the cloud team and may be switched off. But using this meant I did not need the sidecar.

My limits needed updating or there was not enough memory for vscode server.

# yaml-language-server: $schema=https://github.com/epics-containers/ec-helm-charts/releases/download/3.4.4/ioc-instance.schema.json#/$defs/service

ioc-instance:
  #image: ghcr.io/pandablocks/pandablocks-ioc:0.11.2
  image: ghcr.io/gilesknap/pandablocks-ioc:1.0.0b1-debug
  iocConfig: /epics/ioc

  # don't make the IOC the entrypoint
  startCommand: sleep
  startArgs: infinity

  # use root for now
  securityContext:
    runAsUser: 0
    runAsGroup: 0

  # generous resources to handle the extra overheads of debugging
  resources:
    limits:
      cpu: 4
      memory: 4Gi
    requests:
      cpu: 1
      memory: 1Gi

@gilesknap
Copy link
Contributor

@DiamondJoseph please can you add a question to the copier template to toggle this feature on/off?

There is a lot of extra stuff in here which is specific to:-

  • containers that will be run in Kubernetes (there are other reasons to make containers e.g. deploy-tools)
  • DLS ldap service

I am personally a customer of this template and would like to make containers of my python projects without these extras.

@DiamondJoseph
Copy link
Contributor Author

@gilesknap isn't it only the Dockerfile change that goes into the created repo? The rest is docs on this repo only, it's not in the template/ or symlinked

@gilesknap
Copy link
Contributor

@DiamondJoseph I still have to build an extra stage in the CI and then publish an extra image to my registry. That feels a little wasteful, especially if I was working in a private repo and therefore have quota to worry about.

Copy link
Contributor

@gilesknap gilesknap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @DiamondJoseph I'm fully on board now!

@DiamondJoseph
Copy link
Contributor Author

I'm going to merge and release this and iterate on the docs as and when I get to try it out with people. If anyone needs advice on debugging inside the cluster please point them my way.

@DiamondJoseph DiamondJoseph merged commit 76c187c into main Jul 1, 2025
7 checks passed
@DiamondJoseph DiamondJoseph deleted the debug-services branch July 1, 2025 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document the standard way to support prod and dev deployments

5 participants