linuxkit icon indicating copy to clipboard operation
linuxkit copied to clipboard

PostgreSQL inside Gitlab container cannot create shared memory

Open mritd opened this issue 3 years ago • 0 comments

Description

PostgreSQL inside Gitlab container cannot create shared memory

Steps to reproduce the issue:

Create an iso with the following configuration files and start:

kernel:
  image: linuxkit/kernel:5.10.104
  cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
  - linuxkit/init:8f1e6a0747acbbb4d7e24dc98f97faa8d1c6cec7
  - linuxkit/runc:f01b88c7033180d50ae43562d72707c6881904e4
  - linuxkit/containerd:de1b18eed76a266baa3092e5c154c84f595e56da
  - linuxkit/ca-certificates:c1c73ef590dffb6a0138cf758fe4a4305c9864f4
onboot:
  - name: sysctl
    image: linuxkit/sysctl:bdc99eeedc224439ff237990ee06e5b992c8c1ae
    readonly: false
    binds:
      - /etc/sysctl.d/90-custom.conf:/etc/sysctl.d/90-custom.conf
  - name: modprobe
    image: linuxkit/modprobe:1b59b4f2ebb877085ea0d8d3a41cf06f64c09a15
    command: ["modprobe", "br_netfilter","ip_vs", "ip_vs_lc", "ip_vs_wlc", "ip_vs_rr", "ip_vs_wrr", "ip_vs_lblc", "ip_vs_lblcr", "ip_vs_dh", "ip_vs_sh", "ip_vs_fo", "ip_vs_nq", "ip_vs_sed"]
  - name: rngd1
    image: linuxkit/rngd:4f85d8de3f6f45973a8c88dc8fba9ec596e5495a
    command: ["/sbin/rngd", "-1"]
  - name: ip
    image: linuxkit/ip:6cc44dd4e18ddb02de01bc4b34b5799971b6a7bf
    binds:
     - /etc/ip:/etc/ip
    command: ["ip", "-b", "/etc/ip/eth0.conf"]
  - name: format
    image: linuxkit/format:7efa07559dd23cb4dbebfd3ab48c50fd33625918
    command: ["/usr/bin/format", "-type", "xfs", "-partition", "gpt", "-label", "DATA", "/dev/sda"]
  # Enable Swap(must be under /var)
  # https://github.com/linuxkit/linuxkit/blob/master/pkg/swap/swap.sh#L64
  - name: mount
    image: linuxkit/mount:c151655f56a3cb0eab6f74873bf1412eb0187c3f
    command: ["/usr/bin/mountie", "-label", "DATA", "/var/external"]
  - name: swap
    image: linuxkit/swap:77305236719ed7ab4be0f3bccc179c583fe7f5ff
    command: ["/swap.sh", "--path", "/var/external/linuxkit.swap", "--size", "2G"]
  - name: mount-data
    image: linuxkit/mount:c151655f56a3cb0eab6f74873bf1412eb0187c3f
    binds.add:
      - /data:/data:rshared,rbind
    command: ["/usr/bin/mountie", "-label", "DATA", "/data"]
  - name: copy-config
    image: linuxkit/alpine:316c3f9d85c21fdd8bc7479e81d290f85bf60eb0
    binds.add:
      - /data:/data
      - /etc/gitlab:/etc/gitlab
    command: ["sh", "-c", "mkdir -p /data/config /data/logs /data/gitlab && cp /etc/gitlab/gitlab.rb /data/config/gitlab.rb"]
services:
  - name: acpid
    image: linuxkit/acpid:c05a368754f6436b326945dc16135ba547568d8d
  - name: rngd
    image: linuxkit/rngd:4f85d8de3f6f45973a8c88dc8fba9ec596e5495a
  - name: open-vm-tools
    image: linuxkit/open-vm-tools:4c3158c7ba27f7ad0ede5d383ca25b57c5588a26
  - name: openntpd
    image: linuxkit/openntpd:d6c36ac367ed26a6eeffd8db78334d9f8041b038
  - name: getty
    image: linuxkit/getty:c9d5afa9a61ac907904090643e946874ff6bf07c
    env:
     - INSECURE=true
    binds.add:
      - /data:/data
  - name: sshd
    image: linuxkit/sshd:4696ba61c3ec091328e1c14857d77e675802342f
    net: host
    binds.add:
      - /root/.ssh:/root/.ssh
      - /data:/data
  - name: gitlab
    image: gitlab/gitlab-ce:15.7.0-ce.0
    net: host
    capabilities:
      - all
    ambient:
      - all
    devices:
      - path: all
        type: a
    binds.add:
      - /etc/gitlab/linuxkit-ssh.conf:/etc/ssh/sshd_config.d/linuxkit-ssh.conf
      - /data/config:/etc/gitlab
      - /data/logs:/var/log/gitlab
      - /data/gitlab:/var/opt/gitlab
files:
  - path: /etc/containerd/runtime-config.toml
    contents: |
      stderr="/var/log/containerd.out.log"
      stdout="/var/log/containerd.out.log"
    mode: "0644"
  - path: /etc/sysctl.d/90-custom.conf
    contents: |
      net.ipv4.ip_forward=1
      net.ipv4.vs.conntrack=1
      net.ipv6.conf.all.forwarding=1
      net.bridge.bridge-nf-call-iptables=1
      net.bridge.bridge-nf-call-ip6tables=1
    mode: "0644"
  - path: /etc/ip/eth0.conf
    contents: |
      address add 172.16.4.40/24 dev eth0
      link set eth0 up
      route add default via 172.16.4.253 dev eth0
    mode: "0644"
  - path: /etc/resolv.conf
    contents: |
      domain node
      nameserver 223.5.5.5
      nameserver 119.29.29.29
    mode: "0644"
  - path: /etc/ntpd.conf
    contents: |
      servers time.windows.com
  - path: /root/.ssh/authorized_keys
    mode: "0600"
    contents: |
      ssh-ed25519 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  - path: /data
    directory: true
    mode: "0755"
  - path: /etc/gitlab/linuxkit-ssh.conf 
    mode: "0644"
    contents: |
      Port 2222
  - path: /etc/gitlab/gitlab.rb
    mode: "0644"
    contents: |
      external_url 'https://git.example.com'
      nginx['listen_port'] = '80'
      nginx['listen_https'] = false

Describe the results you received:

gitlab failed to start, you can see the error log by viewing the log /data/logs/postgresql/current:

FATAL: could not open shared memory segment "/PostgreSQL.1691416535": Permission denied
LOG: database system is shut down

Describe the results you expected:

gitlab started successfully

Additional information you deem important (e.g. issue happens only occasionally):

By looking at the source code, I guess it may be related to #1477

I am running with VMware ESXi, and set up 8 cpu cores and 16G memory; I found the kernel parameters about shm as follows:

kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096

mritd avatar Jan 06 '23 09:01 mritd