Meta issue
Child issues are listed below: #7899 (comment)
Background on the problem
This topic is rooted in an IE bug caused by a hard limit on the length of characters IE accepts into its URL bar. We've been able to verify that JavaScript can read from a URL that's over this limit if a user clicks a link (but not if the URL is copy/pasted).
This topic overlaps with additional feedback we've gotten about the difficulty with sharing dashboards/visualizations and updating shared dashboards/visualizations. The problem with sharing these documents is that sharing a URL defaults to sharing a "fork" of the document, instead of sharing a link to the original document. This creates a problem when the document is changed, and other viewers of the document don't see the changes -- e.g. a bookmarked dashboard will be stale.
References
Current system
Visualization state is stored in both the URL and on the server. Users share visualizations by copy/pasting URLs to each other. They access visualizations by bookmarking them. There is some demand for being able to create, edit, and delete visualizations programmatically.
New visualizations
https://localhost:5601/myz/app/kibana#/visualize/create?type=pie&_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))&_a=(filters:!(),linked:!f,query:(query_string:(analyze_wildcard:!t,query:'*')),uiState:(),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count),(enabled:!t,id:'2',params:(field:bytes,ranges:!((from:0,to:100),(from:100,to:600))),schema:segment,type:range)),listeners:(),params:(addLegend:!t,addTooltip:!t,isDonut:!f,shareYAxis:!t),title:'New%20Visualization',type:pie))&indexPattern=logstash-*
Data stored in URL:
- Type
- Index pattern
- Refresh interval
- Time period
- Visualization state
Saved visualizations
https://localhost:5601/myz/app/kibana#/visualize/edit/Pie-chart?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))&_a=(filters:!(),linked:!f,query:(query_string:(analyze_wildcard:!t,query:'*')),uiState:(),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count),(enabled:!t,id:'2',params:(field:bytes,ranges:!((from:0,to:100),(from:100,to:600))),schema:segment,type:range)),listeners:(),params:(addLegend:!t,addTooltip:!t,isDonut:!f,shareYAxis:!t),title:'Pie%20chart',type:pie))
Data stored in URL:
- Name
- Refresh interval
- Time period
- Visualization state
Shared visualizations
You can share a visualization with someone by sending them a link with just the visualization name in it.
Clicking the link will load the app, which appends the interval and visualization state – the same as the saved visualization.
Short URLs
Short URLs are hashes that reference visualizations persisted on the server. This is a very similar user-facing interface and persistence mechanism as the "Save visualization" feature.
https://localhost:5601/myz/app/kibana#/visualize/edit/Pie-chart-2 (saved)
https://localhost:5601/myz/goto/7fc85c9bb03917da6dd608c9a98bec3a (short)
Browser history
Making changes to the visualization pushes a new state to the browser history. Clicking the back and forward buttons allow you to navigation the different states the visualization has gone through as you've edited it. This is essentially an "undo/redo" feature.
Proposed system
The current system uses the URL as the source of truth for the data visualization, and stores the visualization state in memory as a secondary step.
This creates several problems: long URLs become unwieldy to share, super-long URLs cause IE bugs, and URLs that are copy/pasted default to "forking" (where each user has his/her own visualization state), instead of the generally-preferred "sharing" (where all users have the same visualization state).
We can solve these problems by inverting the roles of memory and URL. The single source of truth for the data visualization will live within memory, and the URL will be populated with the visualization state as a secondary, optional step (when forking).
In general, the URL search string will only be used for storing data for functionality such as refresh interval, time period, search, filtering, pagination, etc (ways of adjusting perspective).
As an additional and supplemental improvement, we can also use the visualization ID to reference it in the URL, instead of using its name. This will enable name-editing functionality in the future.
Users can create, edit, and delete visualizations programmatically via ES API calls. They can also programmatically create "forked visualizations" (see below).
Note: the goals of this proposed system can be largely met by the Intermediate system, below.
https://localhost:5601/myz/app/kibana#/visualize/edit/7fc85c9bb03917da6dd608c9a98bec3a?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))
Data stored in URL:
- ID
- Refresh interval
- Time period
New visualizations
New visualizations will not have an ID associated with them until they're saved. For this reason, we'll need to surface information to the user that the visualization needs to be saved before it can be shared (e.g. a notification, banner, or warning message). This information could be surfaced when the user clicks the play button to re-render the visualization.
Note: the visualization can still be forked without being saved.
https://localhost:5601/myz/app/kibana#/visualize/create?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))
Data stored in URL:
- Refresh interval
- Time period
Saved visualizations
https://localhost:5601/myz/app/kibana#/visualize/edit/7fc85c9bb03917da6dd608c9a98bec3a?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))
Data stored in URL:
- ID
- Refresh interval
- Time period
Shared visualizations
This has the same URL format as a saved visualization. Visualizations can be shared by copy/pasting this URL to users.
As a bonus, we can remove the URL shortener, since the URL will already be short. We should leave the server-side routing functionality in place, to preserve backwards-compatibility. The short URL will direct to the old URL format, which will be handled gracefully by the client (see "Migrations", below).
In the future, we can add a polling function to detect when the visualization has changed. This change can either be automatically applied to the visualization (useful for public dashboards) if there are no local unsaved changes, or surface a "conflict" message to the user if there are local unsaved changes.
Forked visualizations
If you want to share the state of your visualization without sharing the original, you can fork it. This essentially duplicates our current URL-sharing functionality (adding index pattern and visualization state to the URL).
The user can click a "Fork" button, to generate a copy/pasteable link in a text field:
https://localhost:5601/myz/app/kibana#/visualize/fork/?type=pie&indexPattern=logstash-*&_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))&_a=(filters:!(),linked:!f,query:(query_string:(analyze_wildcard:!t,query:'*')),uiState:(),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count),(enabled:!t,id:'2',params:(field:bytes,ranges:!((from:0,to:100),(from:100,to:600))),schema:segment,type:range)),listeners:(),params:(addLegend:!t,addTooltip:!t,isDonut:!f,shareYAxis:!t),title:'New%20Visualization',type:pie))
When a user clicks this link, the app will load up, parse the URL, store visualization state in memory, and replace the URL with a format that follows the "new visualization" pattern:
https://localhost:5601/myz/app/kibana#/visualize/create?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-5y,mode:quick,to:now))
At this point the user can share the visualization by saving or forking it.
This supports the use-case of one of our customers:
"The reason we need the human-readable/editable URLs is because we create webpages and emails that contain links to Kibana dashboards with certain filters applied. While it was not very easy to figure out what to change in the URL for the filter value, we were able to figure it and it is now a key part of our offering to users. Without human-readable/editable URLs, I don't think we could use Kibana. We are paying gold level subscription customers btw." - @eamonngryan
Note: This feature will work for IE users as long as they click a forked URL link. If they copy/paste a URL into the browser and the URL exceeds IE's length limit, then the visualization data will be un-parseable. In this scenario, we'll need to surface an error message to the user informing them of the problem and how to solve it (by clicking a link instead of copy/pasting).
Browser history
Other document-centric web apps use the browser history for storing route changes (i.e. navigation changes), and surface undo/redo functionality separately (e.g. with buttons or keyboard shortcuts). We should leverage user expectations by conforming to these patterns. We can add this functionality in the future.
Migration
It's important that the new system still handle the current system's URL format gracefully. There are two migration scenarios.
Saved visualizations
A saved visualization's URL contains either a) the name and visualization state or b) just the name. In the event a user opens a saved visualization, here's what happens:
- The client sends an API request to fetch document (using the visualization's name to look it up). The API returns the visualization state and ID.
- The client stores the visualization state in memory.
- The client reformats the URL to conform to the "Saved visualization" format, above (adding the ID, refresh interval, and time period, and removing the name and visualization state).
This way, the old URL will still load the most recently saved state of the visualization.
Non-saved visualizations
A user can currently create a visualization and bookmark it without saving it. If a user opens this bookmark, here's what happens:
- The client stores the visualization state in memory.
- The client reformats the URL to conform to the "New visualization" format, above (containing just the refresh interval and time period).
Intermediate system
We can work our way towards the Proposed system in steps. As an intermediate step, we can build the Proposed system, but keep using visualization names instead of switching to IDs. We can also preserve existing "undo/redo" functionality using history.pushState.
Meta issue
Child issues are listed below: #7899 (comment)
Background on the problem
This topic is rooted in an IE bug caused by a hard limit on the length of characters IE accepts into its URL bar. We've been able to verify that JavaScript can read from a URL that's over this limit if a user clicks a link (but not if the URL is copy/pasted).
This topic overlaps with additional feedback we've gotten about the difficulty with sharing dashboards/visualizations and updating shared dashboards/visualizations. The problem with sharing these documents is that sharing a URL defaults to sharing a "fork" of the document, instead of sharing a link to the original document. This creates a problem when the document is changed, and other viewers of the document don't see the changes -- e.g. a bookmarked dashboard will be stale.
References
Current system
Visualization state is stored in both the URL and on the server. Users share visualizations by copy/pasting URLs to each other. They access visualizations by bookmarking them. There is some demand for being able to create, edit, and delete visualizations programmatically.
New visualizations
Data stored in URL:
Saved visualizations
Data stored in URL:
Shared visualizations
You can share a visualization with someone by sending them a link with just the visualization name in it.
Clicking the link will load the app, which appends the interval and visualization state – the same as the saved visualization.
Short URLs
Short URLs are hashes that reference visualizations persisted on the server. This is a very similar user-facing interface and persistence mechanism as the "Save visualization" feature.
Browser history
Making changes to the visualization pushes a new state to the browser history. Clicking the back and forward buttons allow you to navigation the different states the visualization has gone through as you've edited it. This is essentially an "undo/redo" feature.
Proposed system
The current system uses the URL as the source of truth for the data visualization, and stores the visualization state in memory as a secondary step.
This creates several problems: long URLs become unwieldy to share, super-long URLs cause IE bugs, and URLs that are copy/pasted default to "forking" (where each user has his/her own visualization state), instead of the generally-preferred "sharing" (where all users have the same visualization state).
We can solve these problems by inverting the roles of memory and URL. The single source of truth for the data visualization will live within memory, and the URL will be populated with the visualization state as a secondary, optional step (when forking).
In general, the URL search string will only be used for storing data for functionality such as refresh interval, time period, search, filtering, pagination, etc (ways of adjusting perspective).
As an additional and supplemental improvement, we can also use the visualization ID to reference it in the URL, instead of using its name. This will enable name-editing functionality in the future.
Users can create, edit, and delete visualizations programmatically via ES API calls. They can also programmatically create "forked visualizations" (see below).
Note: the goals of this proposed system can be largely met by the Intermediate system, below.
Data stored in URL:
New visualizations
New visualizations will not have an ID associated with them until they're saved. For this reason, we'll need to surface information to the user that the visualization needs to be saved before it can be shared (e.g. a notification, banner, or warning message). This information could be surfaced when the user clicks the play button to re-render the visualization.
Note: the visualization can still be forked without being saved.
Data stored in URL:
Saved visualizations
Data stored in URL:
Shared visualizations
This has the same URL format as a saved visualization. Visualizations can be shared by copy/pasting this URL to users.
As a bonus, we can remove the URL shortener, since the URL will already be short. We should leave the server-side routing functionality in place, to preserve backwards-compatibility. The short URL will direct to the old URL format, which will be handled gracefully by the client (see "Migrations", below).
In the future, we can add a polling function to detect when the visualization has changed. This change can either be automatically applied to the visualization (useful for public dashboards) if there are no local unsaved changes, or surface a "conflict" message to the user if there are local unsaved changes.
Forked visualizations
If you want to share the state of your visualization without sharing the original, you can fork it. This essentially duplicates our current URL-sharing functionality (adding index pattern and visualization state to the URL).
The user can click a "Fork" button, to generate a copy/pasteable link in a text field:
When a user clicks this link, the app will load up, parse the URL, store visualization state in memory, and replace the URL with a format that follows the "new visualization" pattern:
At this point the user can share the visualization by saving or forking it.
This supports the use-case of one of our customers:
Note: This feature will work for IE users as long as they click a forked URL link. If they copy/paste a URL into the browser and the URL exceeds IE's length limit, then the visualization data will be un-parseable. In this scenario, we'll need to surface an error message to the user informing them of the problem and how to solve it (by clicking a link instead of copy/pasting).
Browser history
Other document-centric web apps use the browser history for storing route changes (i.e. navigation changes), and surface undo/redo functionality separately (e.g. with buttons or keyboard shortcuts). We should leverage user expectations by conforming to these patterns. We can add this functionality in the future.
Migration
It's important that the new system still handle the current system's URL format gracefully. There are two migration scenarios.
Saved visualizations
A saved visualization's URL contains either a) the name and visualization state or b) just the name. In the event a user opens a saved visualization, here's what happens:
This way, the old URL will still load the most recently saved state of the visualization.
Non-saved visualizations
A user can currently create a visualization and bookmark it without saving it. If a user opens this bookmark, here's what happens:
Intermediate system
We can work our way towards the Proposed system in steps. As an intermediate step, we can build the Proposed system, but keep using visualization names instead of switching to IDs. We can also preserve existing "undo/redo" functionality using history.pushState.