Skip to content

feat: selectAndMap, computed/virtual/mapped columns (WIP)#7010

Closed
nebkat wants to merge 2 commits intotypeorm:masterfrom
nebkat:select-and-map
Closed

feat: selectAndMap, computed/virtual/mapped columns (WIP)#7010
nebkat wants to merge 2 commits intotypeorm:masterfrom
nebkat:select-and-map

Conversation

@nebkat
Copy link
Copy Markdown
Contributor

@nebkat nebkat commented Nov 3, 2020

Still WIP, gathering feedback

Fixes #296, #1822, #7008
Alternative to #4703, #6855

Description of change

Universal selectAndMap for computed/virtual/mapped columns with minimal disruption to existing systems and no changes to database drivers.

The column system in TypeORM can generally be broken down into two sides: entity -> database and database -> entity. With computed columns everything in the database -> entity side remains useful, namely the existing RawSqlResultsToEntityTransformer, Driver.prepareHydratedValue and SelectQueryBuilder column aliasing, while the entity -> database side should not be used at all.

With this solution for selectAndMap I have attempted to make the most of the column system and keep a similar pattern for defining computed properties.

1. @Virtual(type: ColumnType, options?: VirtualOptions) decorator

A new decorator is introduced for marking entity properties as being virtual and storing their database type for later reference. It removes all the options from @Column() that are only relevant to database storage. The ColumnMetadataArgs.mode is set to 'computed'.

Normally the type will be inferred from the property type, but for special values such as 'json', 'date', 'enum' and 'hstore' I have left the parameter in. Ideally the available ColumnTypes would be more restrictive given that a 'tinyint' and 'int' are both just numbers once they have been parsed into the result, but there is no harm.

The name "Virtual" is slightly confusing currently since there are already 'virtual' columns from junction tables and table inheritance. I think the current 'virtual' columns should be renamed to 'internal' columns as they are internal to the ORM, while the new mapped properties should be known as virtual. Alternatively 'computed' or 'mapped' could be used, but I would like to see @Computed reserved for putting the select expression directly in the entity in future.

2. addSelectAndMap(selection: string|qb..., mapToProperty: string)

Only the selection expression and mapped property are provided, rather than using an alias, since the main intention is to access the computed value at the property and not have to worry about how exactly that happens.

There might be situations where an alias is also useful if the computed value is being used further in other expressions, but I'm not sure what the best approach is for this. Since the virtual property is still defined as a metadata column it should be possible to use it in where, order, etc using its property name, although I have not yet tested this.

The type of the computed value is determined by the annotated property in the entity, unlike with #6855. I think this is a better approach for maintaining type safety, and avoids all the changes to prepareHydratedValue.

3. EntityMetadata.databaseColumns

A filtered version of columns is added to hold all the non computed columns and only these are used in schema building. This works well, although the naming of all the various column lists is now becoming quite messy, and it feels wrong introducing a new array just because of them.

A possible alternative could be to split up the concept of 'properties' and 'columns' to represent the entity -> database and database -> entity sides. The difficulty is that the new virtual columns mainly care about the property, but they do also need to maintain their databaseName for example, so way to split it isn't exactly obvious. Ideally ColumnMetadata would extend PropertyMetadata, and the appropriate one would be used throughout depending on what is being done.

A list just for virtual would also work, but I think this requires even more changes to the existing code ([...columns, ...virtual], etc).

Sample

@Entity()
export class UserEntity {
    @PrimaryGeneratedColumn()
    id: number;

    @Virtual()
    test: number;
}

const entityManager = connection.createEntityManager();
await entityManager.save(UserEntity, { id: 1 });
const result = await entityManager.createQueryBuilder(UserEntity, 'user')
        .select()
        .addSelectAndMap('5 + 5', 'test')
        .getOne();
SELECT `user`.`id` AS `user_id`, 5 + 5 AS `user_test` FROM `user_entity` `user`
UserEntity { id: 1, test: 10 }

Feedback

Currently only tested with above sample code so might have issues for insert/update/etc, but I think this is a step in the right direction for selectAndMap. Please let me know what you think and send ideas.

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run lint passes with this change
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change
  • The new commits follow conventions explained in CONTRIBUTING.md

@kkoomen
Copy link
Copy Markdown

kkoomen commented Dec 19, 2020

When can this be implemented? We're waiting for this very urgently.

@nathan815
Copy link
Copy Markdown

nathan815 commented Jan 12, 2021

Glad this is being worked on 🙂

Any updates on this? I'd be glad to help with this if you need any help.

@nebkat

@martinsotirov
Copy link
Copy Markdown

I don’t have any preference for this or #6855 as long as there is a solution for that use case. Hopefully soon.

Btw, wouldn’t it make sense to call the decorator for computed properties @Computed(…)?

@nebkat
Copy link
Copy Markdown
Contributor Author

nebkat commented Jan 18, 2021

Wanted to reserve that for providing a query but perhaps one of these could work?

@Computed("varchar", { expression: `CONCAT(text, "-test")` })

// or

@Expression(`CONCAT(text, "-test")`)
@Computed("varchar")

@Asarew
Copy link
Copy Markdown

Asarew commented Apr 14, 2021

@nebkat why did you decide to close this PR?

@nebkat
Copy link
Copy Markdown
Contributor Author

nebkat commented Apr 29, 2021

@Asarew Maintainers have stated that they do not intend on merging the refactoring (#7058) necessary for this change and will instead focus on a new version of TypeORM (https://github.com/typeorm/typeorm/tree/typed-repository).

@nebkat nebkat mentioned this pull request Apr 29, 2021
@Yevhen-Molchaniuk
Copy link
Copy Markdown

I need solution for addSelectAndMap without declaring new attributes in the class, maybe anyone can help me?

@mickaelmesdocteurs
Copy link
Copy Markdown

mickaelmesdocteurs commented May 6, 2021

@Yevhen-Molchaniuk There is no good solution at this point.
The only way is to use getRawOne() or getRawMany() and then map things as you want.

Please correct me if i'm wrong

@vensauro
Copy link
Copy Markdown

vensauro commented May 6, 2021

Asarew Maintainers have stated that they do not intend on merging the refactoring (#7058) necessary for this change and will instead focus on a new version of TypeORM (https://github.com/typeorm/typeorm/tree/typed-repository).

hi @nebkat, you can make an fork of typeorm with your changes?
i am thinking in migrate to mikro-orm, that has more evoluted than typeorm in the actual momento, but maybe your changes can do the work 🤩

@frct1
Copy link
Copy Markdown

frct1 commented May 26, 2021

Hey
This feature actually should be prepared to be in a master branch
Such a basic thing but not available to use , that's a bummer :(

@newerton
Copy link
Copy Markdown

Please, more attention to this type of PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

select additional computed columns

10 participants