Target Use Case

In v9, we are working on the new widgets module, I would like to propose adding a set of advanced data widgets, which allows not only a visual representation but a rich interaction with data & map layers, such as filtering or an automatic data refresh on viewport change.

Consider the following dataset:

[
    {
        "store_id": 445,
        "storetype": "Drugstore",
        "zip": "2111",
        "revenue": 1646773,
        "size_m2": 809,
        "geom": {
            "type": "Point",
            "coordinates": [
                -71.06158,
                42.35157
            ]
        }
    }, 
    // ...
]

A widget for the previous dataset looks like this:

A widget allows:

• Visualize the data of the layer as a chart.
• Filter the layer from the widget.
• Filter other widgets' data.

The widget has two modes: global or viewport. When the viewport is enabled the data of the widget is filtered based on the viewport (so it's updated when the ViewState changes).

Multiple widgets are available, check here the different types of widgets.

It's a very common feature widely used in App Development at CARTO, but there is a strong limitation, they are only available via CARTO for React, so:

• Only React users can use it.
• Only React users who want to use our template architecture.

I feel this would be a great feature to add because:

1. Having widgets natively will be something really valuable for all users.
2. A native integration of widgets will be simpler to use for the final user.
3. CARTO can work on this feature as it will make widgets available on all platforms beyond React.

This map includes a layer with widgets where you can see this functionality in action.

Proposal

This proposal is not simple as it will require deciding on multiple things of the core, so I'm writing down the first ideas, but in this case, I feel we will need a design meeting and move from there.

A new set of widgets could be incorporated into the new widgets module (@deck.gl/widgets):

import { FormulaWidget, CategoryWidget } from '@deck.gl/widgets';

const deckgl = new Deck(
   ...
   layers: [
      new GeoJsonLayer({
        id: 'airports',
        // See comments below
        extensions: [new DataFilterExtension()]
      })
    ]
   widgets: [
      new FormulaWidget ({
          id: '',
          title: '',
          getAggregationValue: (d) => d.properties.revenue,
          aggregationFunction: 'max|min|avg|sum|count',
          layerId: 'airports', 
          filterByViewport: true|false
        }),
        new CategoryWidget ({
          id: '',
          title: '',
          getAggregationValue: (d) => d.properties.revenue 
          getCategoryValue:  (d) => d.properties.storetype
          aggregationFunction: 'max|min|avg|sum|count',
          layerId: 'airports', 
          filterByViewport: true | false
          allowToFilterLayer: true | false
        })
   ]
});

A widget is linked to a layer via the layerIdprop. A widget can filter a layer when allowFilterLayer is set to true. Widgets can filter other widgets if they are associated with the same layer. Check this example to see the expected functionality in action.

Filtering system
As the widgets filter each other, a common state between widgets is required to manage the filtering.

Each widget (except the formula) should be able to filter the layer in the map, in other words, each widget defines a way to filter the layer, and the current implementation of DataFilterExtension doesn't support all the filters required by the widgets. I think it's a great opportunity to expand it as it was commented at #7827 (comment). For example, @felixpalmer added support to filter by category at #7827.

TODO: I will add during the following days a table with the filters each widget requires in the layer.

Gathering data
The initial idea is that the layer will provide the data for the widgets. It should return the data filtered by the viewport (when filterByViewport is true). It's not the rendered data because the widgets are showing the filtered values too.

Tile Layers

Tile layers (like MVT generated with Tippecanoe) sometimes drop features at lower zoom levels to reduce the tile size. For example, a layer of zipcodes at Z=3 doesn't include all the features. Tilers often keep the most prominent features to guarantee a decent size of tiles across multiple zoom levels. If the widgets are calculated with the data of the tiles the result is not always accurate because some features could have been dropped. For those use cases, a backend is required to perform an accurate calculation; or limit the zoom level where the widget is available to guarantee the data is accurate.

For instance, CARTO has two internal modes to calculate widgets: with the data of the tile (for tilesets) and with an external API (for tables and queries).

An option might be to include a prop with an async function to be used in that case.

 new CategoryWidget ({
    ....
    dataFetcher:  async (viewState, props) => {
          const url = `https://${apiBaseUrl}/widgets/formula`;
          const response = await fetch(url);
          return response.json();
        }
  })

Scope
I would keep it at the beginning for the GeoJSON layer and the MVTLayer. We can also reduce the number of widgets to include in v1. We can start with Formula, Category, and Histogram

In general, it's a very large feature. If we decide to go ahead, we will need to split it into smaller pieces.

Should we require a new module for these widgets? Maybe it's too soon yet, but the features required for those data widgets could be more specific. Something to decide as the RFC is taking shape...