-
Notifications
You must be signed in to change notification settings - Fork 26
Refactor code related to flattenened tables #470
Description
Flattened lists of tables have caused a few bugs now, because once they are returned it's hard to know what kind of list you are working with. Is it a flattened list or a normal list? Calling code that expects one but not the other can cause issues like tables getting traversed twice.
listAndValidateTables is one example of a function that returns a flattened list, and has so far been the main culprit
plugin-sdk/plugins/source_validate.go
Line 33 in 7474c90
| func (p *SourcePlugin) listAndValidateTables(tables, skipTables []string) (schema.Tables, error) { |
Flattened lists of tables are useful, but we should clearly distinguish flattened lists of tables from normal top-level lists. My reading is that we currently only use flattened lists for two things:
- getting the total number of tables
- checking if a table is in the top-level list by using
GetTopLevel
These two things can both be achieved by a "set" type. If listAndValidateTables returned an abstracted "set" type that hid its internals and exposed only two methods: Size() int and Contains(table) bool, most of the bugs we've had so far would have been avoided. I think we should try to refactor the code to work something along those lines, so that it is easier to reason about and harder to make mistakes.