Problem

When adding MongoDB as a data warehouse source, if the connection string's permissions grant read-only access to specific collections rather than the entire database, it throws an authentication error.

MongoDB authentication failed: not authorized on db to execute command { listCollections: 1, cursor: {}, nameOnly: true, ...}

This is important to us because our database contains many collections, and enabling read permissions for the entire database causes the process to time out when linking a new data source. The workaround of setting up a federated database is also not suitable, because it only supports full syncs, as indexes cannot be added to federated databases.

Changes

Passed in authorizedCollections=True when querying collection names.

Relevant docs:

• https://pymongo.readthedocs.io/en/stable/api/pymongo/database.html#pymongo.database.Database.list_collection_names
• https://www.mongodb.com/docs/manual/reference/command/listCollections/

Note that this option does nothing if they already have the necessary permissions.

Optional. A flag, when set to true and used with nameOnly: true, that allows a user without the required privilege (i.e. listCollections action on the database) to run the command when access control is enforced.

When both authorizedCollections and nameOnly options are set to true, the command returns only those collections for which the user has privileges. For example, if a user has find action on specific collections, the command returns only those collections; or, if a user has find or any other action, on the database resource, the command lists all collections in the database.

The default value is false. That is, the user must have listCollections action on the database to run the command.

For a user who has listCollections action on the database, this option has no effect since the user has privileges to list the collections in the database.

When used without nameOnly: true, this option has no effect. That is, the user must have the required privileges to run the command when access control is enforced. Otherwise, the user is unauthorized to run the command.

How did you test this code?

• Spun up PostHog locally
• Created a new database user in MongoDB with specific privileges: read access to a specific database + collection
• Attempted to add a new MongoDB data source using the above database user, which errored out with the message "MongoDB authentication failed..."
• Added the authorizedCollections option
• Successfully added mongodb as a source
• Synced successfully.