Skip to content

Refactor code related to flattenened tables #470

@hermanschaaf

Description

@hermanschaaf

Flattened lists of tables have caused a few bugs now, because once they are returned it's hard to know what kind of list you are working with. Is it a flattened list or a normal list? Calling code that expects one but not the other can cause issues like tables getting traversed twice.

listAndValidateTables is one example of a function that returns a flattened list, and has so far been the main culprit

func (p *SourcePlugin) listAndValidateTables(tables, skipTables []string) (schema.Tables, error) {

Flattened lists of tables are useful, but we should clearly distinguish flattened lists of tables from normal top-level lists. My reading is that we currently only use flattened lists for two things:

  • getting the total number of tables
  • checking if a table is in the top-level list by using GetTopLevel

These two things can both be achieved by a "set" type. If listAndValidateTables returned an abstracted "set" type that hid its internals and exposed only two methods: Size() int and Contains(table) bool, most of the bugs we've had so far would have been avoided. I think we should try to refactor the code to work something along those lines, so that it is easier to reason about and harder to make mistakes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions