stored: implicitly create autochanger from device with count > 1 by florian-at-bareos · Pull Request #2198 · bareos/bareos

florian-at-bareos · 2025-03-03T07:15:49Z

Automatically create Autochanger and devices 0000 - Count if the "Device" resource has set a "Count" directive > 1.

The new behaviour of the "Count" directive in the "Device" resource is the following:

If no autochanger is associated with that device, create one named and add the device.
Multiply the device Count + 1 times, starting from 0000. If devices of that name already exist, discard them.
If the device with serial number 0000 was implicitly created in 3. , set "autoselect" to false.
link all serial devices to the autochanger.
If the original device is linked to the autochanger, remove it.

If devices or the autochanger was not implicitly created, a warning is printed.
If the another autochanger which is not associated with the device has the name ,
no autochanger is created even if no autochanger yet exists for that device and a warning is printed..

Update docs accordingly
Document how to cleanly migrate from what's in config: add virtual file changer example + documentation #2090 to this implementation
make sure config: add virtual file changer example + documentation #2090 is merged first
Add migration instructions to implicit autochanger to release notes

Thank you for contributing to the Bareos Project!

Please check

Short description and the purpose of this PR is present above this paragraph
Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General

Is the PR title usable as CHANGELOG entry?
Purpose of the PR is understood
Commit descriptions are understandable and well formatted
Correct milestone is set

Source code quality

Source code changes are understandable
Variable and function names are meaningful
Code comments are correct (logically and spelling)
Required documentation changes are present and part of the PR

Tests

Decision taken that a test is required (if not, then remove this paragraph)
The choice of the type of test (unit test or systemtest) is reasonable
Testname matches exactly what is being tested
On a fail, output of the test leads quickly to the origin of the fault

florian-at-bareos · 2025-03-14T13:18:51Z

still requires documentation

arogge

I had a quick first look. I found a typo in the docs and the systemtest does not seem to work correctly yet.

docs/manuals/source/Configuration/StorageDaemon.rst

bruno-at-bareos

Generally good, work quite well and being helpful.
Please checks the remarks.

docs/manuals/source/Configuration/StorageDaemon.rst

systemtests/tests/multiplied-device/testrunner

core/src/stored/stored_conf.cc

florian-at-bareos · 2025-04-23T13:02:26Z

maybe it is time to split this testrunner to separate start clean default

You think that is necessary? I personally would just leave it as is as long as there is no second testrunner or the file grows too large.

bruno-at-bareos

Still to be done:

In the documentation, we must also rephrase the :config:option:sd/device/Count option, some of its properties have changed.
adapt the changelog

docs/manuals/source/Configuration/StorageDaemon.rst

systemtests/tests/multiplied-device/testrunner

CHANGELOG.md

bruno-at-bareos

All's good for me, I'll give an approval.
@arogge would you mind to have a recheck for c++ code ?

sebsura

This needs some more time in the oven. I could very reliably create a crash:

Add an implicit autochanger with 4 devices. Set MaxConcurrentJobs to 10 everywhere. Start 10 concurrent jobs. Then the sd crashes.

This is because the sd tries to use the original device resource, which is not setup correctly (it has changer_res set, but not changer_command; see OpenDevice).

core/src/stored/stored_conf.cc

sebsura

We should definitely have a systemtest that actually uses this feature. I imagine the test could check

that the maximum number of running jobs equals the count specified in the resource
that one device was created with autoselect = no
...

This test would also make it easier to reproduce bugs as we would not have to set up the environment from scratch every time :)

Im still getting messages like this when running multiple jobs:

2025-06-05 07:51:23 bareos-sd JobId 5: Using Device "FileStorage" (storage) to write.

So im guessing the issue with the extra device is still not fixed. Could you double check that ?

Interestingly the device does not turn up in status sd.

sebsura · 2025-06-05T05:56:35Z

*show storage
Storage {
  Name = "File"
  Address = "localhost"
  Port = 30363
  Password = "[md5]519001602a0e85b0fe7dbead5d95d710"
  Device = "FileStorage"
  MediaType = "File"
  AutoChanger = Yes
  MaximumConcurrentJobs = 20
}

$ cat etc/bareos/bareos-sd.d/device/FileStorage.conf 
Device {
  Name = FileStorage
  Media Type = File
  Device Type = File
  Archive Device = storage
  LabelMedia = yes;                   # lets Bareos label unlabeled media
  Random Access = yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = yes;
  Description = "File device. A connecting Director must have the same Name and MediaType."
  Count = 4
}

if you want to try and reproduce this.

systemtests/tests/multiplied-device/testrunner-count

systemtests/tests/multiplied-device/testrunner-explicit-autochanger

systemtests/tests/multiplied-device/testrunner-implicit-autochanger

systemtests/tests/multiplied-device/etc/bareos/bareos-sd.d/device/ImplicitAutochanger.conf

core/src/stored/autochanger_resource.cc

I took over the work, so i cannot review it anymore

accounting for review suggestions

and add note to changelog about changed behaviour

where the member changer_command is not set

from stored_conf.cc to the AutochangerResource class. Also add a flag for implicitly created autochangers so they don't get config-dumped.

This was needed when the Count directive reused the original device for a copied device. Now that we changed this behaviour, this code is obsolete.

only its copies should be displayed in the status view

When adding a device with Count > 1 to an AutochangerResource, it is now printed.

The return value is never checked and other "unused" configs also simply return true in this case.

Having some kind of error here would be nice, but I am not sure how to really handle it.

arogge

Looks good to me!

florian-at-bareos added the enhancement label Mar 3, 2025

florian-at-bareos added this to the 25.0.0 milestone Mar 3, 2025

florian-at-bareos self-assigned this Mar 3, 2025

florian-at-bareos requested a review from arogge March 6, 2025 12:02

florian-at-bareos force-pushed the dev/fburger/master/implicitly-create-autochanger branch from c07a2da to cac6065 Compare March 14, 2025 13:18

bruno-at-bareos self-requested a review March 18, 2025 10:45

florian-at-bareos marked this pull request as ready for review March 19, 2025 14:30

arogge requested changes Apr 1, 2025

View reviewed changes

docs/manuals/source/Configuration/StorageDaemon.rst Outdated Show resolved Hide resolved

florian-at-bareos force-pushed the dev/fburger/master/implicitly-create-autochanger branch from b9d5aa4 to 256fa9d Compare April 9, 2025 08:18

bruno-at-bareos requested changes Apr 9, 2025

View reviewed changes

bruno-at-bareos added the requires no backport This will not be backported label Apr 14, 2025

florian-at-bareos force-pushed the dev/fburger/master/implicitly-create-autochanger branch from 256fa9d to 6cb508f Compare April 23, 2025 13:38

florian-at-bareos requested review from arogge and bruno-at-bareos May 2, 2025 07:43

bruno-at-bareos changed the title ~~stored: implicitly create autochanger from device~~ stored: implicitly create autochanger from device with count > 1 May 5, 2025

bruno-at-bareos requested changes May 5, 2025

View reviewed changes

florian-at-bareos requested a review from bruno-at-bareos May 7, 2025 08:25

bruno-at-bareos approved these changes May 8, 2025

View reviewed changes

sebsura requested changes May 14, 2025

View reviewed changes

florian-at-bareos requested a review from sebsura May 19, 2025 09:00

florian-at-bareos force-pushed the dev/fburger/master/implicitly-create-autochanger branch from 3746b81 to 3fc4a38 Compare May 20, 2025 15:17

sebsura requested changes Jun 5, 2025

View reviewed changes

florian-at-bareos force-pushed the dev/fburger/master/implicitly-create-autochanger branch from ab96693 to a3a9053 Compare June 10, 2025 14:29

sebsura force-pushed the dev/fburger/master/implicitly-create-autochanger branch from a3a9053 to db08ef1 Compare September 22, 2025 08:17

sebsura previously requested changes Sep 22, 2025

View reviewed changes

sebsura force-pushed the dev/fburger/master/implicitly-create-autochanger branch from cc4c262 to 5e5d429 Compare October 13, 2025 06:14

florian-at-bareos and others added 22 commits October 31, 2025 08:23

refactor: refactor systemtes multiplied-device

cd51147

accounting for review suggestions

docs: update documentation after bareos#2090 merge

406273e

docs: add migration info to release notes

5dfd6e5

docs: fix warnings and errors in docs

596f603

docs: update Count documentation

5e749a8

and add note to changelog about changed behaviour

docs: apply suggested changes

5ee499a

multiplied-device: improve error messages in test

56006f9

stored: fix bug for implicit autochanger

cda6164

where the member changer_command is not set

store: move implicit autochanger construction

70b10cb

from stored_conf.cc to the AutochangerResource class. Also add a flag for implicitly created autochangers so they don't get config-dumped.

store: check if Count directive is in range

1fdac87

store: move copying devices to DeviceResource

696012f

store: don't longer store DeviceResource base-names

2ebb20f

This was needed when the Count directive reused the original device for a copied device. Now that we changed this behaviour, this code is obsolete.

store: hide count > 1 device in storage status

5986af0

only its copies should be displayed in the status view

stored: fix memory leak

cfc931f

stored: fix PrintConfig for AutochangerResource

1d25750

When adding a device with Count > 1 to an AutochangerResource, it is now printed.

update config-schemas

6b95644

stored: add $ prefix to devices with count > 1

89de812

systemtests: add implicit-autochanger test

3d3e80a

stored: change return to true

fb08cc8

The return value is never checked and other "unused" configs also simply return true in this case.

cmake: add workaround for GCC 13/14, too

1af82bb

sd-volume-limit: remove warning

30c4db8

device-resource: remove failing assert

50d1444

Having some kind of error here would be nice, but I am not sure how to really handle it.

sebsura force-pushed the dev/fburger/master/implicitly-create-autochanger branch from c734249 to 50d1444 Compare October 31, 2025 07:23

sebsura added 2 commits October 31, 2025 10:47

multiplied-device: fix xattr issues on freebsd

17bcd6f

multiplied-device: fix tests sharing restore directory

65a3ebc

sebsura force-pushed the dev/fburger/master/implicitly-create-autochanger branch from 9ddf237 to 65a3ebc Compare October 31, 2025 09:47

arogge approved these changes Nov 3, 2025

View reviewed changes

Update CHANGELOG.md

039c5a7

BareosBot merged commit abe3f01 into bareos:master Nov 3, 2025
1 check was pending

bruno-at-bareos mentioned this pull request Dec 4, 2025

More than 2 concurrent backup jobs against Dedup volume causes bareos-sd segfault. #2470

Open

Conversation

florian-at-bareos commented Mar 3, 2025 • edited by arogge Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Thank you for contributing to the Bareos Project!

Please check

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

General

Source code quality

Tests

Uh oh!

florian-at-bareos commented Mar 14, 2025

Uh oh!

arogge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bruno-at-bareos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

florian-at-bareos commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bruno-at-bareos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bruno-at-bareos left a comment

Choose a reason for hiding this comment

Uh oh!

sebsura left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sebsura left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebsura commented Jun 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arogge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

florian-at-bareos commented Mar 3, 2025 •

edited by arogge

Loading

florian-at-bareos commented Apr 23, 2025 •

edited

Loading

sebsura left a comment •

edited

Loading