Skip to content

Make the search index load faster #167

Description

@isaisabel

As a user of the ATT&CK Website, I want the search feature to load quickly. The search feature is slow to load because the search index is quite large and must be retrieved over the network whenever the user accesses the site.

As with all optimization, we should measure first before implementing them to ensure that we're actually saving a meaningful amount of time.

There are two general methods we could take to reduce load time. Both could be implemented for maximum time save if they're determined to be worthwhile.

Cache the search index on the user's computer so that it doesn't need to be fetched every time the interface is opened.

The index is probably too large to fit in cookies, but other methods like Local Storage could possibly fit it. We'd use the local copy whenever possible since it wouldn't require transmitting over the network.

We'd need to ensure that the search feature rejects the local version whenever the site updates so that it stays up-to-date. Presumably we could achieve this by generating a hash whenever the site builds, and comparing the cached search json's hash to the expected hash to detect an out of date index.

This method would also require compatibility checks for browsers and/or user settings which might disable the storage method. It would fall-back to retrieving the search json over the network if a localstorage copy was unavailable.

Reduce the filesize of the search index json

We can optimize the load time of the search feature by reducing the filesize of the search index. Currently index.json is pretty-formatted with indentation. By removing the whitespace, we can reduce the size of this file and speed up load times for the users.

Further improvements could be made by one of the following methods:

  • Reducing the length of the keys "id", "title", "path", "content" to a single character each
  • removing keys and making each entry an array of values.

Both of these methods would sacrifice readability of the index. They'd also require restoring the format using javascript. It is quite possible that restoring the format in JS would take longer than the time saved in the GET by reducing the filesize, so measuring performance is especially important here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Points: 88 agile story pointsenhancementNew feature or requestpriority/highHigh priority Issuetimescale/short-termBecomes timescale/imminent in weeks or months

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions