Skip to content

services.mysqlBackup: Backup job might break due to garbage collection #29031

Closed
Closed
services.mysqlBackup: Backup job might break due to garbage collection#29031
Labels
0.kind: bugSomething is broken1.severity: blockerThis is preventing another PR or issue from being completed3.skill: sprintableA larger issue which is split into distinct actionable tasks6.topic: nixosIssues or PRs affecting NixOS modules, or package usability issues specific to NixOS
Milestone
17.09
@fadenb

Description

@fadenb
fadenb
opened on Sep 5, 2017

Issue description

Surprise email from cron daemon:

subject: Cron <root@HOSTNAME> mysql /nix/store/vsj6yazxlzrhwhcdvkzp2b1wakz96gpi-mariadb-10.1.25/bin/mysqldump DATABASENAME | /nix/store/c09pc52nm3ii38f6n7qx9k5g81d55w7l-gzip-1.8/bin/gzip -c > /var/backup/mysql/DATABASENAME.gz

execl: couldn't exec `/nix/store/c280fqkd65rrpp5mfarp6cc0v4ns6rlb-bash-4.4-p12/bin/bash'
execl: No such file or directory

The cron job created by the backup service is broken. Several referenced paths do no longer exist:

[root@HOSTNAME:~]# crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/etc/crontab installed on Tue Aug 22 18:01:15 2017)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
SHELL=/nix/store/c280fqkd65rrpp5mfarp6cc0v4ns6rlb-bash-4.4-p12/bin/bash
PATH=/nix/store/qladnhrhvlcxmzv2j2g7wybfj1mmzi7w-system-path/bin:/nix/store/qladnhrhvlcxmzv2j2g7wybfj1mmzi7w-system-path/sbin

NIX_CONF_DIR=/etc/nix
15 01 * * * mysql /nix/store/vsj6yazxlzrhwhcdvkzp2b1wakz96gpi-mariadb-10.1.25/bin/mysqldump  DATABASENAME | /nix/store/c09pc52nm3ii38f6n7qx9k5g81d55w7l-gzip-1.8/bin/gzip -c > /var/backup/mysql/DATABASENAME.gz



[root@HOSTNAME:~]# ls -al /nix/store/vsj6yazxlzrhwhcdvkzp2b1wakz96gpi-mariadb-10.1.25/bin/mysqldump
ls: cannot access '/nix/store/vsj6yazxlzrhwhcdvkzp2b1wakz96gpi-mariadb-10.1.25/bin/mysqldump': No such file or directory

[root@HOSTNAME:~]# ls -al /nix/store/c280fqkd65rrpp5mfarp6cc0v4ns6rlb-bash-4.4-p12/bin/bash
ls: cannot access '/nix/store/c280fqkd65rrpp5mfarp6cc0v4ns6rlb-bash-4.4-p12/bin/bash': No such file or directory

[root@HOSTNAME:~]# ls -al /nix/store/qladnhrhvlcxmzv2j2g7wybfj1mmzi7w-system-path/bin
ls: cannot access '/nix/store/qladnhrhvlcxmzv2j2g7wybfj1mmzi7w-system-path/bin': No such file or directory

Given enough time I will eventually reimplement the service using systemd-timers (as it was already discussed in #24728) but I might not get this done in time for the next release.

edit: It seems to me that services.cron.systemCronJobs it severly broken. Changes are sometimes not detected and cron runs for quite some time with an old version of the crontab (or no crontab at all!)

Steps to reproduce

  • Configure services.mysqlBackup
  • ? (not sure what I did in between)
  • nix-collect-garbage -d
  • Get surprise email from cron telling you the backup failed

Technical details

  • System: 18.03.git.977073d (Impala)
  • Nix version: nix-env (Nix) 1.11.14
  • Nixpkgs version: "17.03.1247.05126bc850"

Metadata

Metadata

Assignees

No one assigned

    Labels

    0.kind: bugSomething is broken1.severity: blockerThis is preventing another PR or issue from being completed3.skill: sprintableA larger issue which is split into distinct actionable tasks6.topic: nixosIssues or PRs affecting NixOS modules, or package usability issues specific to NixOS

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions