Skip to content

Use ideas from Incremental View Maintenance to know what has changed #7

@mitar

Description

@mitar

Instead of doing a full refresh when any of the sources changes, we could based on what changes in the source compute how does (and if) this influence current results. There is some work for materialized views on how to do that, known as Incremental View Maintenance.

There are two main questions:

  • How to know/store what exactly changed.
  • How to map that to how results change.

Then we can update both our temporary cache table and send those changes to the client. Instead of doing a full refresh. And only if we do not know how to map new inputs to new results, we do a full refresh.

There are some related projects which could help here:

It might be necessary to parse and rewrite the query, for which this could help.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions