Output of restic version
restic 0.18.0 compiled with go1.24.1 on windows/amd64
What backend/service did you use to store the repository?
Azure Blob Storage
Problem description / Steps to reproduce
In Azure Blob Storage, there is 2 ways to upload "blobs" (in the case of BlockBlobs Objects. PageBlobs and AppendBlobs Objects are not considered here):
https://docs.microsoft.com/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs
Restic use exclusively the second method and as Restic's Packs files are by default of 16MiB, for each pack uploaded there is a PutBlock and a PutBlockList transaction, which lead to double the transactions and the fees.
To show the effect, I've uploaded a 5TiB file with --pack-size=4M
From Azure Portal :
In this exemple, the 2M transactions, which are the sum of PutBlock and PutBlockList transactions, have been billed ~13€ (Hot Tier). With PutBlob method, the bill would be half.
Expected behavior
Use of PutBlob API to have only one transaction per Pack file's upload.
Actual behavior
Every Pack files are sent with a PutBlock transaction then a PutBlockList transaction, which leads to unnecessary increase the number of transactions and so the bill.
Do you have any idea what may have caused this?
As Restic's Pack files are far away from the 5000MiB upper limit of the PutBlob API, the code from
https://github.com/restic/restic/blob/master/internal/backend/azure/azure.go
should be reworked to at least use PutBlob API by default.
Here you can find some info from Azure SDK for Go (from line 164 and 203):
https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/storage/azblob/blockblob/client.go
Did restic help you today? Did it make you happy in any way?
Really a good tool. I'm like a kid with a new toy :)
Output of
restic versionrestic 0.18.0 compiled with go1.24.1 on windows/amd64
What backend/service did you use to store the repository?
Azure Blob Storage
Problem description / Steps to reproduce
In Azure Blob Storage, there is 2 ways to upload "blobs" (in the case of BlockBlobs Objects. PageBlobs and AppendBlobs Objects are not considered here):
https://docs.microsoft.com/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs
With PutBlob API => Send a full file in one transaction (limit to 5000MiB)
https://docs.microsoft.com/rest/api/storageservices/put-blob
With PutBlock + PutBlockList API => Send one or multiple blocks (limit to 4000MiB each block) of a file then send the blocklist to validate the file (which could be up to 190TiB). A transaction is made for every PutBlock then one is made for PutBlockLIst
https://docs.microsoft.com/rest/api/storageservices/put-block
https://docs.microsoft.com/rest/api/storageservices/put-block-list
Restic use exclusively the second method and as Restic's Packs files are by default of 16MiB, for each pack uploaded there is a PutBlock and a PutBlockList transaction, which lead to double the transactions and the fees.
To show the effect, I've uploaded a 5TiB file with --pack-size=4M
From Azure Portal :
In this exemple, the 2M transactions, which are the sum of PutBlock and PutBlockList transactions, have been billed ~13€ (Hot Tier). With PutBlob method, the bill would be half.
Expected behavior
Use of PutBlob API to have only one transaction per Pack file's upload.
Actual behavior
Every Pack files are sent with a PutBlock transaction then a PutBlockList transaction, which leads to unnecessary increase the number of transactions and so the bill.
Do you have any idea what may have caused this?
As Restic's Pack files are far away from the 5000MiB upper limit of the PutBlob API, the code from
https://github.com/restic/restic/blob/master/internal/backend/azure/azure.go
should be reworked to at least use PutBlob API by default.
Here you can find some info from Azure SDK for Go (from line 164 and 203):
https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/storage/azblob/blockblob/client.go
Did restic help you today? Did it make you happy in any way?
Really a good tool. I'm like a kid with a new toy :)