Skip to content

glynnbird/cloudantbulkdelete

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

cloudantbulkdelete

A command-line utility that assists in the deletion of many documents from a Cloudant database. The tool expects a Cloudant Query "selector" that defines the slice of data that is to be deleted. The tool can be paired with cloudantimport which will batch the changes into bulk writes to the database.

Installation

You will need to download and install the Go compiler. Clone this repo then:

go build ./cmd/cloudantbulkdelete

The copy the resultant binary cloudantbulkdelete (or cloudantbulkdelete.exe in Windows systems) into your path.

Configuration

cloudantbulkdelete authenticates with your chosen Cloudant service using environment variables as documented here e.g.

CLOUDANT_URL=https://xxxyyy.cloudantnosqldb.appdomain.cloud
CLOUDANT_APIKEY="my_api_key"

Usage

# delete documents where team="red" OR date >= '2020-02-01'
$ cloudantbulkdelete --db users --selector '{"$or":[{"team":{"$eq":"red"}},{"date": {"$gte": "2020-02-01"}}]}'
{"_id":"e15a6a03f75d844a0ac117a3a742f589","_rev":"1-c4f1369224db88c99fa8020c2f177477","_deleted":true}
{"_id":"e15a6a03f75d844a0ac117a3a748a0d0","_rev":"1-c9b0eb03324c3e744b0068e04f36fb52","_deleted":true}
...

The tool outputs the deletion JSON to stdout so that it can be inspected for accuracy. To actually delete the data, install cloudantimport and use the two tools together:

cloudantbulkdelete --db users --selector '{"team":"red"}' | cloudantimport --db users

It is also possible to find the documents to delete from one database and attempt to delete them from another!

cloudantbulkdelete --selector '{"team":"pink"}' --db mydb1 | cloudantimport --db mydb2

How does this work?

A filtered changes feed is set up, using the supplied selector as the filter. Any documents meeting the selector's criteria are turned into JSON objects which when written to Cloudant would delete the documents. The cloudantimport utility already batches and writes data in bulk to Cloudant, so there's no need to copy that code to this tool.

About

Bulk delete documents from a Cloudant database that match a supplied selector

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages