Extra MongoDB Tools

This repository provides supplementary tools for MongoDB, supporting both backup and restoration workflows:

mongo-archive – Dumps MongoDB data to disk and uploads it to supported cloud storage services.
mongo-unarchive – Downloads archived dumps from cloud storage and restores them into a live MongoDB database.

🚀 Building the Tools

To build the binaries from source:

Clone the repository:

git clone https://github.com/egose/database-tools
cd database-tools

Install dependencies and build:
```
go mod tidy
make build
```
This will install dependencies and build the binaries into the dist/ directory.

Installation

You can install mongo-archive and mongo-unarchive in two ways:

1. Install via asdf (Recommended)

If you use asdf to manage CLI tools, you can install the mongodb-database-tools plugin and make the CLI available globally.

# Add the mongodb-database-tools plugin (only once)
asdf plugin add mongodb-database-tools

# Install the desired version
asdf install mongodb-database-tools <latest-version>

# Set it as the global version
asdf global mongodb-database-tools <latest-version>
# Or set it locally for a project
asdf local mongodb-database-tools <latest-version>

After installation, you can run:

mongo-archive --version
mongo-unarchive --version

2. Download from GitHub Releases

You can also manually download the prebuilt binaries from the official releases page:

Releases: https://github.com/egose/database-tools/releases

Visit the release page for version .
Download the binary for your operating system and architecture.
Make the binary executable and move it into a directory in your PATH:

chmod +x mongo-archive
chmod +x mongo-unarchive
sudo mv mongo-archive /usr/local/bin/
sudo mv mongo-unarchive /usr/local/bin/

Verify Installation

Run the following commands to confirm the installed version:

mongo-archive --version
mongo-unarchive --version

⚙️ Configuration: CLI Flags & Environment Variables

Both mongo-archive and mongo-unarchive follow the conventions of MongoDB’s native tools (e.g., mongodump, mongorestore), using similar command-line arguments. Configuration values can also be passed via environment variables for convenience or container-based execution.

📦 `mongo-archive`

Functionality

Dumps MongoDB data locally.
Uploads the dump to cloud storage (Azure Blob, AWS S3, or Google Cloud Storage).
Can be run once or as a cron-scheduled job.

Parameters

Flag	Environment Variable	Type	Description
`uri`	`MONGOARCHIVE__URI`	string	MongoDB URI connection string
`db`	`MONGOARCHIVE__DB`	string	Database to use
`collection`	`MONGOARCHIVE__COLLECTION`	string	Collection to use
`host`	`MONGOARCHIVE__HOST`	string	MongoDB host to connect to (for replica sets: `setname/host1,host2`)
`port`	`MONGOARCHIVE__PORT`	string	MongoDB port (can also use `--host hostname:port`)
`ssl`	`MONGOARCHIVE__SSL`	bool	Connect to a mongod or mongos that has SSL enabled
`ssl-ca-file`	`MONGOARCHIVE__SSL_CA_FILE`	string	`.pem` file containing the root certificate chain from the CA
`ssl-pem-key-file`	`MONGOARCHIVE__SSL_PEM_KEY_FILE`	string	`.pem` file containing the certificate and key
`ssl-pem-key-password`	`MONGOARCHIVE__SSL_PEM_KEY_PASSWORD`	string	Password to decrypt the `sslPEMKeyFile`
`ssl-crl-file`	`MONGOARCHIVE__SSL_CRL_FILE`	string	`.pem` file containing the certificate revocation list
`ssl-allow-invalid-certificates`	`MONGOARCHIVE__SSL_ALLOW_INVALID_CERTIFICATES`	bool	Bypass validation for server certificates
`ssl-allow-invalid-hostnames`	`MONGOARCHIVE__SSL_ALLOW_INVALID_HOSTNAMES`	bool	Bypass validation for server hostnames
`ssl-fips-mode`	`MONGOARCHIVE__SSL_FIPS_MODE`	bool	Use FIPS mode of the installed OpenSSL library
`username`	`MONGOARCHIVE__USERNAME`	string	Username for authentication
`password`	`MONGOARCHIVE__PASSWORD`	string	Password for authentication
`authentication-database`	`MONGOARCHIVE__AUTHENTICATION_DATABASE`	string	Database that holds the user's credentials
`authentication-mechanism`	`MONGOARCHIVE__AUTHENTICATION_MECHANISM`	string	Authentication mechanism to use
`gssapi-service-name`	`MONGOARCHIVE__GSSAPI_SERVICE_NAME`	string	Service name for GSSAPI/Kerberos auth (default: `mongodb`)
`gssapi-host-name`	`MONGOARCHIVE__GSSAPI_HOST_NAME`	string	Hostname for GSSAPI/Kerberos auth (default: server address)
`uri-prune`	`MONGOARCHIVE__URI_PRUNE`	bool	Prune MongoDB URI connection string (remove credentials etc.)
`query`	`MONGOARCHIVE__QUERY`	string	Query filter as v2 Extended JSON string
`query-file`	`MONGOARCHIVE__QUERY_FILE`	string	Path to file containing query filter (v2 Extended JSON)
`read-preference`	`MONGOARCHIVE__READ_PREFERENCE`	string	Preference mode (e.g., `nearest`) or preference JSON object
`force-table-scan`	`MONGOARCHIVE__FORCE_TABLE_SCAN`	bool	Force a table scan
`verbose`	`MONGOARCHIVE__VERBOSE`	string	More detailed log output (`-vvvvv` or `--verbose=N`)
`quiet`	`MONGOARCHIVE__QUIET`	bool	Hide all log output
`az-endpoint`	`MONGOARCHIVE__AZ_ENDPOINT`	string	Azure Blob Storage emulator hostname and port
`az-account-name`	`MONGOARCHIVE__AZ_ACCOUNT_NAME`	string	Azure Blob Storage account name
`az-account-key`	`MONGOARCHIVE__AZ_ACCOUNT_KEY`	string	Azure Blob Storage account key
`az-container-name`	`MONGOARCHIVE__AZ_CONTAINER_NAME`	string	Azure Blob Storage container name
`aws-endpoint`	`MONGOARCHIVE__AWS_ENDPOINT`	string	AWS endpoint URL (hostname only or fully qualified URI)
`aws-access-key-id`	`MONGOARCHIVE__AWS_ACCESS_KEY_ID`	string	AWS access key associated with an IAM account
`aws-secret-access-key`	`MONGOARCHIVE__AWS_SECRET_ACCESS_KEY`	string	AWS secret key associated with the access key
`aws-region`	`MONGOARCHIVE__AWS_REGION`	string	AWS region to send requests to
`aws-bucket`	`MONGOARCHIVE__AWS_BUCKET`	string	AWS S3 bucket name
`aws-s3-force-path-style`	`MONGOARCHIVE__AWS_S3_FORCE_PATH_STYLE`	bool	Force path-style S3 addressing instead of virtual-hosted
`gcp-endpoint`	`MONGOARCHIVE__GCP_ENDPOINT`	string	GCP endpoint URL
`gcp-bucket`	`MONGOARCHIVE__GCP_BUCKET`	string	GCP storage bucket name
`gcp-creds-file`	`MONGOARCHIVE__GCP_CREDS_FILE`	string	Path to GCP service account credentials file
`gcp-project-id`	`MONGOARCHIVE__GCP_PROJECT_ID`	string	GCP project ID
`gcp-private-key-id`	`MONGOARCHIVE__GCP_PRIVATE_KEY_ID`	string	GCP private key ID
`gcp-private-key`	`MONGOARCHIVE__GCP_PRIVATE_KEY`	string	GCP private key
`gcp-client-email`	`MONGOARCHIVE__GCP_CLIENT_EMAIL`	string	GCP client email
`gcp-client-id`	`MONGOARCHIVE__GCP_CLIENT_ID`	string	GCP client ID
`local-path`	`MONGOARCHIVE__LOCAL_PATH`	string	Local directory path to store backups
`expiry-days`	`MONGOARCHIVE__EXPIRY_DAYS`	string	Max age (in days) for archives to be retained
`rocketchat-webhook-url`	`MONGOARCHIVE__ROCKETCHAT_WEBHOOK_URL`	string	Rocket.Chat webhook URL
`rocketchat-webhook-prefix`	`MONGOARCHIVE__ROCKETCHAT_WEBHOOK_PREFIX`	string	Prefix for Rocket.Chat webhook messages
`rocketchat-notify-on-failure-only`	`MONGOARCHIVE__ROCKETCHAT_NOTIFY_ON_FAILURE_ONLY`	bool	Send Rocket.Chat notifications only on failure
`cron`	`MONGOARCHIVE__CRON`	bool	Run a cron scheduler and block current execution path
`cron-expression`	`MONGOARCHIVE__CRON_EXPRESSION`	string	Cron schedule expression
`tz`	`MONGOARCHIVE__TZ`	string	User-specified time zone (see GNU `TZ` variable format)
`keep`	`MONGOARCHIVE__KEEP`	bool	Keep data dump after completion
`version`	(no env var)	bool	Show the version and exit

🔄 `mongo-unarchive`

Functionality

Downloads archived MongoDB dumps from supported cloud storage.
Restores the data to a MongoDB database.
Supports applying update operations post-restore using a JSON configuration.

Parameters

Flag	Environment Variable	Type	Description
`verbose`	`MONGOUNARCHIVE__VERBOSE`	string	More detailed log output (`-vvvvv` or `--verbose=N`)
`quiet`	`MONGOUNARCHIVE__QUIET`	bool	Hide all log output
`host`	`MONGOUNARCHIVE__HOST`	string	MongoDB host to connect to (for replica sets: `setname/host1,host2`)
`port`	`MONGOUNARCHIVE__PORT`	string	MongoDB port (can also use `--host hostname:port`)
`ssl`	`MONGOUNARCHIVE__SSL`	bool	Connect to a mongod or mongos that has SSL enabled
`ssl-ca-file`	`MONGOUNARCHIVE__SSL_CA_FILE`	string	`.pem` file containing the root certificate chain from the CA
`ssl-pem-key-file`	`MONGOUNARCHIVE__SSL_PEM_KEY_FILE`	string	`.pem` file containing the certificate and key
`ssl-pem-key-password`	`MONGOUNARCHIVE__SSL_PEM_KEY_PASSWORD`	string	Password to decrypt the `sslPEMKeyFile`
`ssl-crl-file`	`MONGOUNARCHIVE__SSL_CRL_FILE`	string	`.pem` file containing the certificate revocation list
`ssl-allow-invalid-certificates`	`MONGOUNARCHIVE__SSL_ALLOW_INVALID_CERTIFICATES`	bool	Bypass validation for server certificates
`ssl-allow-invalid-hostnames`	`MONGOUNARCHIVE__SSL_ALLOW_INVALID_HOSTNAMES`	bool	Bypass validation for server hostnames
`ssl-fips-mode`	`MONGOUNARCHIVE__SSL_FIPS_MODE`	bool	Use FIPS mode of the installed OpenSSL library
`username`	`MONGOUNARCHIVE__USERNAME`	string	Username for authentication
`password`	`MONGOUNARCHIVE__PASSWORD`	string	Password for authentication
`authentication-database`	`MONGOUNARCHIVE__AUTHENTICATION_DATABASE`	string	Database that holds the user's credentials
`authentication-mechanism`	`MONGOUNARCHIVE__AUTHENTICATION_MECHANISM`	string	Authentication mechanism to use
`gssapi-service-name`	`MONGOUNARCHIVE__GSSAPI_SERVICE_NAME`	string	Service name for GSSAPI/Kerberos auth (default: `mongodb`)
`gssapi-host-name`	`MONGOUNARCHIVE__GSSAPI_HOST_NAME`	string	Hostname for GSSAPI/Kerberos auth (default: server address)
`uri`	`MONGOUNARCHIVE__URI`	string	MongoDB URI connection string
`uri-prune`	`MONGOUNARCHIVE__URI_PRUNE`	bool	Prune MongoDB URI connection string (remove credentials etc.)
`db`	`MONGOUNARCHIVE__DB`	string	Database to use
`collection`	`MONGOUNARCHIVE__COLLECTION`	string	Collection to use
`ns-exclude`	`MONGOUNARCHIVE__NS_EXCLUDE`	string	Exclude matching namespaces
`ns-include`	`MONGOUNARCHIVE__NS_INCLUDE`	string	Include matching namespaces
`ns-from`	`MONGOUNARCHIVE__NS_FROM`	string	Rename matching namespaces (requires matching `ns-to`)
`ns-to`	`MONGOUNARCHIVE__NS_TO`	string	Rename matched namespaces (requires matching `ns-from`)
`drop`	`MONGOUNARCHIVE__DROP`	bool	Drop each collection before import
`dry-run`	`MONGOUNARCHIVE__DRY_RUN`	bool	View summary without importing anything (recommended with verbosity)
`write-concern`	`MONGOUNARCHIVE__WRITE_CONCERN`	string	Write concern options
`no-index-restore`	`MONGOUNARCHIVE__NO_INDEX_RESTORE`	bool	Do not restore indexes
`no-options-restore`	`MONGOUNARCHIVE__NO_OPTIONS_RESTORE`	bool	Do not restore collection options
`keep-index-version`	`MONGOUNARCHIVE__KEEP_INDEX_VERSION`	bool	Do not update index version
`maintain-insertion-order`	`MONGOUNARCHIVE__MAINTAIN_INSERTION_ORDER`	bool	Restore documents in the order they appear in the input source; also enables `--stopOnError` and restricts insertion workers to 1
`num-parallel-collections`	`MONGOUNARCHIVE__NUM_PARALLEL_COLLECTIONS`	string	Number of collections to restore in parallel (default: 4)
`num-insertion-workers-per-collection`	`MONGOUNARCHIVE__NUM_INSERTION_WORKERS_PER_COLLECTION`	string	Number of insert operations to run concurrently per collection (default: 1)
`stop-on-error`	`MONGOUNARCHIVE__STOP_ON_ERROR`	bool	Halt after any insertion error instead of continuing
`bypass-document-validation`	`MONGOUNARCHIVE__BYPASS_DOCUMENT_VALIDATION`	bool	Bypass document validation
`preserve-uuid`	`MONGOUNARCHIVE__PRESERVE_UUID`	bool	Preserve original collection UUIDs (requires `--drop`)
`az-endpoint`	`MONGOUNARCHIVE__AZ_ENDPOINT`	string	Azure Blob Storage emulator hostname and port
`az-account-name`	`MONGOUNARCHIVE__AZ_ACCOUNT_NAME`	string	Azure Blob Storage account name
`az-account-key`	`MONGOUNARCHIVE__AZ_ACCOUNT_KEY`	string	Azure Blob Storage account key
`az-container-name`	`MONGOUNARCHIVE__AZ_CONTAINER_NAME`	string	Azure Blob Storage container name
`aws-endpoint`	`MONGOUNARCHIVE__AWS_ENDPOINT`	string	AWS endpoint URL (hostname only or fully qualified URI)
`aws-access-key-id`	`MONGOUNARCHIVE__AWS_ACCESS_KEY_ID`	string	AWS access key associated with an IAM account
`aws-secret-access-key`	`MONGOUNARCHIVE__AWS_SECRET_ACCESS_KEY`	string	AWS secret key associated with the access key
`aws-region`	`MONGOUNARCHIVE__AWS_REGION`	string	AWS region to send requests to
`aws-bucket`	`MONGOUNARCHIVE__AWS_BUCKET`	string	AWS S3 bucket name
`aws-s3-force-path-style`	`MONGOUNARCHIVE__AWS_S3_FORCE_PATH_STYLE`	bool	Force path-style S3 addressing instead of virtual-hosted
`gcp-endpoint`	`MONGOUNARCHIVE__GCP_ENDPOINT`	string	GCP endpoint URL
`gcp-bucket`	`MONGOUNARCHIVE__GCP_BUCKET`	string	GCP storage bucket name
`gcp-creds-file`	`MONGOUNARCHIVE__GCP_CREDS_FILE`	string	Path to GCP service account credentials file
`gcp-project-id`	`MONGOUNARCHIVE__GCP_PROJECT_ID`	string	GCP project ID
`gcp-private-key-id`	`MONGOUNARCHIVE__GCP_PRIVATE_KEY_ID`	string	GCP private key ID
`gcp-private-key`	`MONGOUNARCHIVE__GCP_PRIVATE_KEY`	string	GCP private key
`gcp-client-email`	`MONGOUNARCHIVE__GCP_CLIENT_EMAIL`	string	GCP client email
`gcp-client-id`	`MONGOUNARCHIVE__GCP_CLIENT_ID`	string	GCP client ID
`local-path`	`MONGOUNARCHIVE__LOCAL_PATH`	string	Local directory path to store backups
`object-name`	`MONGOUNARCHIVE__OBJECT_NAME`	string	Object name of the archived file in the storage (optional)
`dir`	`MONGOUNARCHIVE__DIR`	string	Directory containing the dumped files
`updates`	`MONGOUNARCHIVE__UPDATES`	string	Array of update specifications in JSON string format
`updates-file`	`MONGOUNARCHIVE__UPDATES_FILE`	string	Path to a file containing an array of update specifications
`keep`	`MONGOUNARCHIVE__KEEP`	bool	Keep data dump after completion
`version`	(no env var)	bool	Show the version and exit

🧪 Usage Examples

Dump a Database to Azure Storage

mongo-archive \
  --uri="mongodb://<username>:<password>@cluster0.mongodb.net/" \
  --db=<dbname> \
  --az-account-name=<az_account_name> \
  --az-account-key=<az_account_key> \
  --az-container-name=<az_container_name>

Schedule Regular Backups with Cron

mongo-archive \
  --uri="mongodb://<username>:<password>@cluster0.mongodb.net/" \
  --db=<dbname> \
  --az-account-name=<az_account_name> \
  --az-account-key=<az_account_key> \
  --az-container-name=<az_container_name> \
  --cron \
  --cron-expression="0 * * * *"

Restore from Azure Storage

mongo-unarchive \
  --uri="mongodb://localhost:27017" \
  --db=<dbname> \
  --az-account-name=<az_account_name> \
  --az-account-key=<az_account_key> \
  --az-container-name=<az_container_name>

Restore and Apply Updates

mongo-unarchive \
  --uri="mongodb://localhost:27017" \
  --db=<dbname> \
  --az-account-name=<az_account_name> \
  --az-account-key=<az_account_key> \
  --az-container-name=<az_container_name> \
  --updates-file=/home/nonroot/updates.json

Sample `updates.json`

[
  {
    "collection": "users",
    "filter": {
      "email": { "$exists": true }
    },
    "update": [
      {
        "$set": {
          "email": {
            "$replaceOne": {
              "input": "$email",
              "find": "@",
              "replacement": "_"
            }
          }
        }
      }
    ]
  }
]

🐳 Running with Docker

docker run --rm \
  -v "$(pwd)/tmp:/tmp" \
  -e MONGOARCHIVE__DUMP_PATH=/tmp/datadump \
  ghcr.io/egose/database-tools:latest \
  mongo-archive \
  --uri="mongodb://<username>:<password>@cluster0.mongodb.net/" \
  --db=<dbname> \
  --az-account-name=<az_account_name> \
  --az-account-key=<az_account_key> \
  --az-container-name=<az_container_name> \
  --keep

☁️ Running as a Kubernetes CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: mongo-archive
spec:
  schedule: "0 12 * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      backoffLimit: 3
      template:
        spec:
          restartPolicy: Never
          initContainers:
            - name: backup-permission
              image: alpine:3.18
              command: ["/bin/sh", "-c"]
              args:
                - |
                  rm -rf /tmp/*;
                  adduser -D -u 1000 nonroot;
                  chown nonroot:nonroot /tmp;
              volumeMounts:
                - mountPath: /tmp
                  name: backup-volume
          containers:
            - name: backup-job
              image: ghcr.io/egose/database-tools:<latest-version>
              command: ["/bin/sh", "-c"]
              args:
                - mongo-archive --db=mydb --read-preference=primary --force-table-scan
              env:
                - name: MONGOARCHIVE__URI
                  value: "mongodb+srv://user:password@cluster0.my.mongodb.net"
                - name: MONGOARCHIVE__AZ_ACCOUNT_NAME
                  value: mystorage
                - name: MONGOARCHIVE__AZ_ACCOUNT_KEY
                  value: myaccountkey
                - name: MONGOARCHIVE__AZ_CONTAINER_NAME
                  value: mybackup
              volumeMounts:
                - mountPath: /tmp
                  name: backup-volume
          volumes:
            - name: backup-volume
              persistentVolumeClaim:
                claimName: backup-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: backup-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

🗂️ Backlog

To be documented.

Name		Name	Last commit message	Last commit date
Latest commit History 1,492 Commits
.github		.github
common		common
docs		docs
examples		examples
mongoarchive		mongoarchive
mongounarchive		mongounarchive
notification		notification
sandbox		sandbox
storage		storage
test		test
utils		utils
.conventional-changelog.mjs		.conventional-changelog.mjs
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
.release-it.json		.release-it.json
.tool-versions		.tool-versions
CHANGELOG.md		CHANGELOG.md
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
VERSION		VERSION
commitlint.config.mjs		commitlint.config.mjs
flags.md		flags.md
go.mod		go.mod
go.sum		go.sum
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
renovate.json		renovate.json
requirements.txt		requirements.txt

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

Extra MongoDB Tools

🚀 Building the Tools

Installation

1. Install via asdf (Recommended)

2. Download from GitHub Releases

Verify Installation

⚙️ Configuration: CLI Flags & Environment Variables

📦 `mongo-archive`

Functionality

Parameters

🔄 `mongo-unarchive`

Functionality

Parameters

🧪 Usage Examples

Dump a Database to Azure Storage

Schedule Regular Backups with Cron

Restore from Azure Storage

Restore and Apply Updates

Sample `updates.json`

🐳 Running with Docker

☁️ Running as a Kubernetes CronJob

🗂️ Backlog

About

Uh oh!

Releases 18

Packages

Uh oh!

Uh oh!

Contributors 3

Uh oh!

Languages

License

egose/database-tools

Folders and files

Latest commit

History

Repository files navigation

Extra MongoDB Tools

🚀 Building the Tools

Installation

1. Install via asdf (Recommended)

2. Download from GitHub Releases

Verify Installation

⚙️ Configuration: CLI Flags & Environment Variables

📦 mongo-archive

Functionality

Parameters

🔄 mongo-unarchive

Functionality

Parameters

🧪 Usage Examples

Dump a Database to Azure Storage

Schedule Regular Backups with Cron

Restore from Azure Storage

Restore and Apply Updates

Sample updates.json

🐳 Running with Docker

☁️ Running as a Kubernetes CronJob

🗂️ Backlog

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 18

Packages 0

Uh oh!

Uh oh!

Contributors 3

Uh oh!

Languages

📦 `mongo-archive`

🔄 `mongo-unarchive`

Sample `updates.json`

Packages