Skip to content

Conversation

@MatheusMeloAntiquera
Copy link
Contributor

@MatheusMeloAntiquera MatheusMeloAntiquera commented Feb 11, 2023

This new fix allow support to mongodb driver v5

Closes: #7907

Description of change

Add support for mongodb v5.0.1

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply prettier formatting
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change
  • The new commits follow conventions explained in CONTRIBUTING.md

@MatheusMeloAntiquera MatheusMeloAntiquera marked this pull request as draft February 11, 2023 23:55
@MatheusMeloAntiquera MatheusMeloAntiquera force-pushed the fix-support-to-mongodb-v5 branch 3 times, most recently from d6c1768 to 9e75423 Compare February 12, 2023 20:45
@tgrassl
Copy link
Contributor

tgrassl commented Mar 4, 2023

Hi, are there any updates on this or is there something i could help out with? Would love to get support for the new driver soon 😃

@MatheusMeloAntiquera
Copy link
Contributor Author

MatheusMeloAntiquera commented Mar 5, 2023

Hi, are there any updates on this or is there something i could help out with? Would love to get support for the new driver soon 😃

Hi, I would like some help, thanks.

We need to find how force mongo to stop create the _id field when collection is created, because mongo is always creating now this field : https://www.mongodb.com/docs/manual/core/document/#the-_id-field

So, if there is a entity with an other id name field the mongo will create _id field, like this example:
image

The result is:

image

@tgrassl
Copy link
Contributor

tgrassl commented Mar 5, 2023

Hi, i looked through the code and it seems like this behaviour is related to this issue: #9727

The additional .save() calls always results in an insert because the condition of whether it should be updated never matches. This is because the resulting transformed entities of the findByIds query in SubjectDatabaseEntityLoader don't contain an id property but the subject entity does.

entities = await mongoRepo.findByIds(allIds, findOptions)

Entity from DB: CleanShot 2023-03-05 at 18 47 18@2x
Subject Entity to persist: CleanShot 2023-03-05 at 19 03 25@2x

Because it has no id the following happens:

  1. findByPersistEntityLike fails to match the result to the subject
  2. subject.databaseEntity is skipped
  3. subject.databaseEntityLoaded is set to true
  4. SubjectExecutor only finds a subject to be inserted instead of updated because it has no databaseEntity

this.insertSubjects = new SubjectTopoligicalSorter(
this.insertSubjects,
).sort("insert")
await this.executeInsertOperations()
// console.timeEnd(".insertion");
// recompute update operations since insertion can create updation operations for the
// properties it wasn't able to handle on its own (referenced columns)
this.updateSubjects = this.allSubjects.filter(
(subject) => subject.mustBeUpdated,
)
// execute update operations
// console.time(".updation");
await this.executeUpdateOperations()

get mustBeInserted() {
return this.canBeInserted && !this.databaseEntity
}
/**
* Checks if this subject must be updated into the database.
* Subject can be updated in the database if it is allowed to be updated (explicitly persisted or by cascades)
* and if it does have differentiated columns or relations.
*/
get mustBeUpdated() {
return (
this.canBeUpdated &&
this.identifier &&
(this.databaseEntityLoaded === false ||
(this.databaseEntityLoaded && this.databaseEntity)) &&
// ((this.entity && this.databaseEntity) || (!this.entity && !this.databaseEntity)) &&
this.changeMaps.length > 0
)
}

I think the fix would be to check for the propertyName of the ObjectIdColumn instead of only the databaseNameWithoutPrefixes in the DocumentEntityTransformer transform:

...
        // handle _id property the special way
        if (metadata.objectIdColumn) {
            const { databaseNameWithoutPrefixes, propertyName } =
                metadata.objectIdColumn

            const documentIdWithoutPrefixes =
                document[databaseNameWithoutPrefixes]
            const documentIdWithPropertyName = document[propertyName]

            if (documentIdWithoutPrefixes) {
                entity[propertyName] = documentIdWithoutPrefixes
                hasData = true
            } else if (documentIdWithPropertyName) {
                entity[propertyName] = documentIdWithPropertyName
                hasData = true
            }
        }
...

This now adds the id from the document to the entity and results in an update to the same entity instead of an insert.
I pushed my changes and created a pr for your branch 👍

@MatheusMeloAntiquera
Copy link
Contributor Author

tgrassl

Thank you for your help dude!

@MatheusMeloAntiquera MatheusMeloAntiquera force-pushed the fix-support-to-mongodb-v5 branch from d211e9e to 22b150d Compare March 6, 2023 01:42
@MatheusMeloAntiquera
Copy link
Contributor Author

@pleerock
Can you do the review please?

There is an error Oracle test, but we didn't any changes on Oracle source

@tgrassl
Copy link
Contributor

tgrassl commented Mar 6, 2023

@pleerock I would like to update the typings, so everyone can now use the official typings already included in the mongodb package. This would also finally remove the confusion around ObjectId and ObjectID (e.g. #2238)

As far i can see, i cannot use import { ObjectId } from 'mongodb' inside the following files, because that would require non-mongodb users to install mongodb right?

  • EntityManager.ts
  • Repository.ts
  • BaseEntity.ts
  • EntityId.ts
  • FindOptionsOrder.ts
  • FindOptionsRelations.ts
  • FindOptionsSelect.ts
  • FindOptionsWhere.ts

These all just need the type of the ObjectId. Mongo specific files like MongoEntityManager already import from mongodb, so I guess they are ok.
My idea would be to only copy the ObjectId typings into typeorm but not export it from the package, so these files can use that internal type. Would that be fine for you?

@MatheusMeloAntiquera MatheusMeloAntiquera marked this pull request as ready for review March 13, 2023 19:46
ObjectId,
MongoError,
FilterOperators,
} from "mongodb"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't have direct imports from mongodb

@pleerock
Copy link
Member

pleerock commented Apr 6, 2023

These all just need the type of the ObjectId. Mongo specific files like MongoEntityManager already import from mongodb, so I guess they are ok.

No, we must not have imports from mongodb anywhere, because non-monogdb users will have tsc errors and they'll be forced to install mongodb package even if they don't use it. That's bad.

Matheus Melo Antiquera and others added 4 commits April 7, 2023 18:34
@mptr mptr mentioned this pull request Apr 7, 2023
7 tasks
mptr added 2 commits April 7, 2023 20:52
Instanceof check now no longer references to just the
type but the loaded class ref instead.
This new fix allow support to mongodb driver v5

Closes: typeorm#7907
@mptr
Copy link
Contributor

mptr commented Apr 12, 2023

@MatheusMeloAntiquera how about not submitting the same changes in separate PRs?
I've added you and @tgrassl to my fork. May we close this PR then?
(Also it would be nice to not commit my changes just by copying them on your branch. Instead pull the commits of collaborators on your branch to keep a clean commit history.)

@MatheusMeloAntiquera MatheusMeloAntiquera force-pushed the fix-support-to-mongodb-v5 branch 2 times, most recently from 9d38119 to cf9d7a5 Compare April 13, 2023 01:05
@MatheusMeloAntiquera
Copy link
Contributor Author

@MatheusMeloAntiquera how about not submitting the same changes in separate PRs? I've added you and @tgrassl to my fork. May we close this PR then? (Also it would be nice to not commit my changes just by copying them on your branch. Instead pull the commits of collaborators on your branch to keep a clean commit history.)

I will keep this PR, sorry for copy your code I wanted to resolve this quickly, but you should have create a PR to my branch, not directly to typeorm repository

@MatheusMeloAntiquera MatheusMeloAntiquera force-pushed the fix-support-to-mongodb-v5 branch from cf9d7a5 to 07186d0 Compare April 13, 2023 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for mongodb driver v4

4 participants