Skip to content

feat: expose validate/serialize functions through Request and Reply#3970

Merged
mcollina merged 42 commits intofastify:mainfrom
metcoder95:feat/3897
Jul 14, 2022
Merged

feat: expose validate/serialize functions through Request and Reply#3970
mcollina merged 42 commits intofastify:mainfrom
metcoder95:feat/3897

Conversation

@metcoder95
Copy link
Copy Markdown
Member

@metcoder95 metcoder95 commented Jun 7, 2022

Fixes #3897

Pendings:

  • Cache validate functions
  • Cache serialize functions
  • Support Serializer Compiler setup on-demand
  • Set WeakMaps cache on demand
  • Document each one of the APIs

Checklist

@metcoder95 metcoder95 marked this pull request as ready for review June 10, 2022 14:44
Copy link
Copy Markdown
Member

@jsumners jsumners left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are adding more complexity that is a sibling feature to https://www.fastify.io/docs/latest/Reference/Server/#schemacontroller (i.e. the goal is already possible by a feature already included).

@jsumners jsumners requested a review from Eomm June 10, 2022 14:51
@metcoder95
Copy link
Copy Markdown
Member Author

I think we are adding more complexity that is a sibling feature to fastify.io/docs/latest/Reference/Server/#schemacontroller (i.e. the goal is already possible by a feature already included).

Well, is kinda possible by providing the SchemaController, as you can store the reference somewhere and just decorate the Request with the function you want, but that requires overwriting the SchemaController by providing one by yourself, then in case the developer is not interested in using a custom validation library but the default one provided by Fastify, is not so easy to achieve.

The only reason I didn't go in that way was because of the encapsulation, as one of the easiest ways to do so is just exposing the SchemaController and allowing the implementation to do whatever it wants with it. But stills, as Fastify allows to not only overwrite the controller within an encapsulated context but also at a route level, the number of combinations and cases increases.
So my idea was just to keep everything as it is and intercept the SchemaController to provide a smaller function that validates/serializes on runtime, respecting whatever customization has been by the implementation and any level

Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a unit test?

@metcoder95
Copy link
Copy Markdown
Member Author

Can you please add a unit test?

@mcollina I’ll follow @Eomm suggestion and create a standalone file just for testing the new methods. I’ll use his suggestion for create many tests as I can think off at different context levels

@metcoder95 metcoder95 marked this pull request as draft June 17, 2022 16:01
Copy link
Copy Markdown
Member

@Eomm Eomm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!
I think we are near the perfect API!

EDIT: could you rebase this branch? So the CI should go green 👍🏽

Copy link
Copy Markdown
Member

@Eomm Eomm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

lib/context.js Outdated
this[kRouteByFastify] = isFastify

this[kRequestValidateWeakMap] = new WeakMap()
this[kReplySerializeWeakMap] = new WeakMap()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think creating two WeakMaps for every route is a good idea. Use an LRU that sits in the Fastify instance instead.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, but wouldn't that make the things a little bit more complicated as we'll need to bring a good default for customising the LRU cache size for cache eviction?

I support not building a WeakMap for every route as is possible the implementation is not going to use the serialize/validate feature for that specific route so doesn't have any sense to add it. Maybe we can go with a more conservative approach by lazy loading the WeakMap just if we detect usage, wdyt?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be ok for me, yes.

lib/reply.js Outdated
if (serialize == null) throw new FST_ERR_MISSING_SERIALIZATION_FN(httpStatus)
} else {
// Check if serialize function already compiled
if (this.context[kReplySerializeWeakMap].has(schema)) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using an object as a key will cause bugs: developers could change the object post-fact and it will cause problems. You need some sort of deep-equality-hashing for this.
I use http://npm.im/safe-stable-stringify for these kind of purposes. However generating the hash has a cost too.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can bring a middle ground solution? By dictating in the docs to not alter the original schema object but rather create a new one if there's a need to alter a schema on-the-fly?

Here the overhead of the hash is what worries me the most

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's document it.

@Eomm Eomm added semver-minor Issue or PR that should land as semver minor missing types PR that does not implement TypeScript changes labels Jul 1, 2022
@metcoder95
Copy link
Copy Markdown
Member Author

Sorry for being like a ghost lately. Changed jobs, and laptop so I'm needing some time to setup everything once again. I hope to bring more activity to it by the end of the week 🙂

@Eomm
Copy link
Copy Markdown
Member

Eomm commented Jul 11, 2022

Just one though waiting for the docs:

We could split the caching component into a separate PR.
Doing so we could discuss a bit in a separate issue.
Adding it would improve the feature for sure, but a basic version without cache would be good IMHO.

My doubt comes from the following example: the weakmap won't match and the schema will be always recompiled because the object is always a new one:

    fastify.get('/', (req, reply) => {
      const validate = req.compileValidationSchema({ type: 'object', ...etc })
    })

The user should use this function like this to benefit from the weak map:

    const aschema = { type: 'object', ...etc }
    fastify.get('/', (req, reply) => {
      const validate = req.compileValidationSchema(aschema)
    })

This would ease the merge of this PR too

I hope to see land it soo 😄

@metcoder95
Copy link
Copy Markdown
Member Author

Sure! I think we can go and apply the suggestions from @mcollina and open a separate PR where we can discuss more in big detail an improvement in the shape of the caching, as because of what you already stated, it's easy for the user to implement it in such way that they will never benefit for the cache.

I think as a first iteration just documenting what's the best approach can help to see how everything goes on a first run meanwhile improving that feature specific 🙂

Thankfully I'm free from Tomorrow on so I'll start adding the missed documentation, applying suggestions, and also adding the TS types. Sorry for the delay, has been a rough week.

@metcoder95
Copy link
Copy Markdown
Member Author

Please feel free to have a look, I think is good to go in this state. Just the documentation is missed, I'll add it Tomorrow 🙂

@metcoder95 metcoder95 requested a review from mcollina July 13, 2022 07:36
Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Copy Markdown
Member

CI is failing

@metcoder95
Copy link
Copy Markdown
Member Author

Coverage is low, let me take a look 👍

Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina mcollina merged commit abab588 into fastify:main Jul 14, 2022
@metcoder95 metcoder95 deleted the feat/3897 branch July 14, 2022 11:31
@github-actions
Copy link
Copy Markdown

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

missing types PR that does not implement TypeScript changes semver-minor Issue or PR that should land as semver minor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make available the data validation method with the specified schema and the data serialization method with the specified schema

5 participants