Skip to content

feat: Add abstract TileSource#1194

Merged
manzt merged 7 commits intodevelopfrom
manzt/abstract-fetcher
Nov 9, 2024
Merged

feat: Add abstract TileSource#1194
manzt merged 7 commits intodevelopfrom
manzt/abstract-fetcher

Conversation

@manzt
Copy link
Copy Markdown
Member

@manzt manzt commented Feb 16, 2024 •
edited
Loading

Description

What was changed in this pull request?

Extracts a TileSource interface to represent an abstract "source" for HiGlass's DataFetcher. The DataFetcher does a lot of processing of the values returned from the server, so it's challenging to just implement a different source that is not a webserver (e.g., a websocket). I think there is a lot more logic that could be reused to optimize requests, but this seems sufficient for the time being.

Why is it necessary?

This let's us connect DataFetcher to non-HTTP (i.e., non fetch-based) data sources.

Fixes #___

Checklist

  • Set proper GitHub labels (e.g. v1.6+, ignore if you don't know)
  • Unit tests added or updated
  • Documentation added or updated
  • Example(s) added or updated
  • Update schema.json if there are changes to the viewconf JSON structure format
  • Screenshot for visual changes (e.g. new tracks or UI changes)
  • Updated CHANGELOG.md

@manzt
Copy link
Copy Markdown
Member Author

manzt commented Feb 16, 2024 •
edited
Loading

For higlass/higlass-python#144, this would let us do something like:

function JupyterDataFetcher(hgc, dataConfig, options) {
  let jupyterTileSource = {
    getTiles: /* ... */,
    getTilesetInfo: /* ... */,
    registerTileset: /* ... */
  };
  return new hgc.DataFetcher(dataConfig, options, jupyterTileSource);
}

Where jupyterTileSource uses Jupyter comms (websocket) to send/receive data, rather than fetch (HTTP-requests). We could even experiment with sending binary data over a protocol like this in the future (rather than base64 encoding dense tiles).

@manzt
Copy link
Copy Markdown
Member Author

manzt commented Feb 16, 2024

Tbh i don't really know what registerTileset is or does.... I've never implemented in higlass-python or server side implementations of higlass tile sources.

@manzt manzt mentioned this pull request Feb 16, 2024
Merged
@pkerpedjiev
Copy link
Copy Markdown
Member

pkerpedjiev commented Feb 16, 2024

registerTileset was a semi-half-baked way of fetching tilesets that were not on the server. The idea was that the client would call the register_tileset endpoint with a url, the server would return a uuid that the client could use to fetch tiles. It would be as if the client was fetching directly from the url without implementing the http-based tile fetching code.

I think 4DN was using it. Might be possible to deprecate.

@manzt manzt force-pushed the manzt/abstract-fetcher branch from 244d8d0 to b4fa5f5 Compare November 9, 2024 16:52
@manzt manzt merged commit 3f8cd61 into develop Nov 9, 2024
@manzt manzt deleted the manzt/abstract-fetcher branch November 9, 2024 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@etowahadams etowahadams

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@manzt @pkerpedjiev @etowahadams