Skip to content

Add Asset event type and capture assets#1239

Draft
Juice10 wants to merge 86 commits intomasterfrom
asset-event
Draft

Add Asset event type and capture assets#1239
Juice10 wants to merge 86 commits intomasterfrom
asset-event

Conversation

@Juice10
Copy link
Member

@Juice10 Juice10 commented Jun 8, 2023

Asset events allow for asynchronous events containing (image or potentially other) assets.
See #860 for more info.

These are async events that get emmited after the original dom mutation events and can be used to inject these assets after the fact in the player or via post processing.

Todo:

  • create asset event for assets that failed to load
  • implement asset in player
  • [ ] add flag to log assets being captured
  • add documentation for recording config
  • [docs] explain how its different to inline images
  • decided whether or not to move the capturing and encoding to a web worker (out of scope)
  • use canvas webgl code to do blob encoding
  • move more types from rrweb-snapshot to @rrweb/types to circumvent circular dependencies
  • add meta data to meta event including what asset origins are going to be cached,
    • also filter hooks in Asset Manager listening to those origins
  • add support for all elements
    • video
    • audio
    • embed
    • source
    • track
    • input#type=image
    • img#srcset
    • svg
    • table#background
  • [ ] Remove inlineImages in favour of assetCapture
  • Forward inlineImages config to assetCapture
  • get it working in yarn live-stream
  • hide src attributes while asset is being awaited
  • work with rrdom
  • add tests for svg elements, also update asset manager to work with setAttributeNS setAttributeNS was needed for xlink:href, but all modern browsers rewrite that to href which is already supported out of the box.
  • add implementation for srcset
  • rename replay/asset/index.ts to replay/asset-manager/index.ts

@changeset-bot
Copy link

changeset-bot bot commented Jun 8, 2023

🦋 Changeset detected

Latest commit: c160f50

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
Name Type
rrweb-snapshot Major
rrweb Major
rrdom Major
@rrweb/types Major
@rrweb/rrweb-plugin-canvas-webrtc-record Major
@rrweb/rrweb-plugin-canvas-webrtc-replay Major
@rrweb/rrweb-plugin-console-record Major
@rrweb/rrweb-plugin-console-replay Major
@rrweb/rrweb-plugin-sequential-id-record Major
@rrweb/rrweb-plugin-sequential-id-replay Major
rrdom-nodejs Major
rrweb-player Major
@rrweb/all Major
@rrweb/replay Major
@rrweb/record Major
@rrweb/packer Major
@rrweb/web-extension Major
rrvideo Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Contributor

@eoghanmurray eoghanmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering whether it's worth taking this commit out and merging in a separate PR?
Fine if not; maybe in future this sort of thing could be merged in earlier while rest of a PR is ongoing.

@Juice10
Copy link
Member Author

Juice10 commented Jan 16, 2024

I'm wondering whether it's worth taking this commit out and merging in a separate PR? Fine if not; maybe in future this sort of thing could be merged in earlier while rest of a PR is ongoing.

If we weren't on an alpha release that type of commit would also require us to do a major version change. Not sure if it makes sense now, it's bound to break some tests and those got fixed later in subsequent commits. But in general I do agree with you, this type of thing would be better in a separate PR

return Promise.all(promises);
}

public reset(config?: captureAssetsParam | undefined): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to reset when a meta event is captured?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for live-mode/addEvent. Once a meta event is triggered its an indication that a page navigation has happened. That tells us that we don't have to keep waiting for new asset events to show up, hence the reset being called.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. But do we need to store the capture config in the meta event, and reload it as this.config = config when a meta event is emitted?

Or we can think the capture config is the same across a session and also call reset when a meta event is emitted, but without storing a copy of the capture params in the meta event?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict of our conversation:

  • I'll move over the captureAssetsParam for the replay from the meta event to the full snapshot.
  • Copy dimensions from meta to full snapshot.

};

this.capturedURLs.add(url);
this.capturingURLs.delete(url);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about move this line and the same line in the error block into a finally block?

@Yuyz0112
Copy link
Member

Just reviewed the last two files: the asset manager observer and the replay side manager, really cool implementation!

Leave some small comments, and we should be able to merge this in the following days.

@eoghanmurray
Copy link
Contributor

Not sure if this is a goal or a non-goal, but should we be collecting all assets for the srcset attribute? Shouldn't we be able to tell which one of the images is the one that will need to be rendered?

Copy link
Contributor

@eoghanmurray eoghanmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename cacheable -> capturable

Copy link
Contributor

@eoghanmurray eoghanmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more cacheable -> capturable

(This is now a commit)

private async preloadAllAssets(timestamp: number): Promise<void[]> {
const promises: Promise<void>[] = [];
for (const event of this.service.state.context.events) {
if (event.timestamp <= timestamp) continue;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we are ignoring old asset events ... is the idea here to get the 'most recent version' of an asset?
Aside: is there any risk that the timestamp of the asset event will be the same as the fullsnapshot timestamp?

@eoghanmurray eoghanmurray force-pushed the asset-event branch 2 times, most recently from 7434552 to a88ff5e Compare March 25, 2024 11:54
Juice10 and others added 23 commits June 26, 2024 10:39
"diff algorithm for rrdom › apply virtual style rules to node › should insert rule at index [0,0] and keep existing rules"
AssetManager.manageAttribute
different types of assets (not just img#src)
Co-authored-by: Eoghan Murray <eoghan@getthere.ie>
…the AssetManager needs to interrogate other attributes to determine how to manage, e.g. <link with href depending on rel="stylesheet"
@eoghanmurray eoghanmurray marked this pull request as draft February 20, 2025 17:12
@eoghanmurray
Copy link
Contributor

eoghanmurray commented Feb 20, 2025

I've made this a draft as there are many changes in #1475 (apart from the addition of stylesheet type assets).

So converted to draft to redirect any review effort to #1475 as we hopefully will merge everything all at once

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants