Skip to content

RUM-5445: Revive Session Replay instrumentation tests#2965

Merged
jonathanmos merged 11 commits into
developfrom
jmoskovich/rum-5445/make-instrumentation-tests-behavioral
Nov 23, 2025
Merged

RUM-5445: Revive Session Replay instrumentation tests#2965
jonathanmos merged 11 commits into
developfrom
jmoskovich/rum-5445/make-instrumentation-tests-behavioral

Conversation

@jonathanmos

@jonathanmos jonathanmos commented Oct 23, 2025

Copy link
Copy Markdown
Member

What does this PR do?

This pr is a proof of concept for reviving the session replay instrumentation tests and adding the missing tests for image, text and touch privacy.
The core change here is to move from a system where we verify the individual fields in the json against "golden files", to one where we verify the existence of whole components in the output wireframe (such as the existence or absence of a placeholder) in a way that is more behavioral testing. This will make the tests device agnostic and considerably less flaky, and we would no longer need to modify the snapshots as we did before.

The test suite takes 6 minutes and 15 seconds to run locally. This seems to be mainly due to the time it takes to parse the jsons, but we could improve on this if we were able to parallelize the tests. The suite would run as part of the test pyramids.

Note: resurrecting these tests exposed a bug in checkbox rendering so in the meantime I've removed checkbox tests. This will be handled in another pr and those tests will be restored.

Motivation

Working instrumentation tests for session replay.

Additional Notes

Anything else we should know when reviewing?

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Oct 23, 2025

Copy link
Copy Markdown

🎯 Code Coverage
Patch Coverage: 100.00%
Total Coverage: 70.93% (-0.34%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 35dcb23 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@codecov-commenter

codecov-commenter commented Oct 23, 2025

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.84%. Comparing base (d493283) to head (35dcb23).
⚠️ Report is 290 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2965      +/-   ##
===========================================
+ Coverage    70.79%   70.84%   +0.05%     
===========================================
  Files          829      829              
  Lines        30384    30383       -1     
  Branches      5184     5184              
===========================================
+ Hits         21510    21524      +14     
+ Misses        7392     7374      -18     
- Partials      1482     1485       +3     
Files with missing lines Coverage Δ
...sionreplay/internal/net/BatchesToSegmentsMapper.kt 73.47% <ø> (ø)
...y/internal/recorder/resources/ImageTypeResolver.kt 100.00% <ø> (ø)

... and 39 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jonathanmos jonathanmos marked this pull request as ready for review November 11, 2025 15:00
@jonathanmos jonathanmos requested review from a team as code owners November 11, 2025 15:00
@jonathanmos jonathanmos marked this pull request as draft November 11, 2025 15:16
@jonathanmos jonathanmos force-pushed the jmoskovich/rum-5445/make-instrumentation-tests-behavioral branch from d1e3d5b to 495db93 Compare November 12, 2025 12:30
@jonathanmos jonathanmos force-pushed the jmoskovich/rum-5445/make-instrumentation-tests-behavioral branch from 495db93 to 7145153 Compare November 12, 2025 12:39
@jonathanmos jonathanmos marked this pull request as ready for review November 12, 2025 13:27

@0xnm 0xnm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

I left some comments, but looks good!

@jonathanmos jonathanmos force-pushed the jmoskovich/rum-5445/make-instrumentation-tests-behavioral branch from 581ade8 to 43fd4c2 Compare November 17, 2025 13:23
@jonathanmos jonathanmos force-pushed the jmoskovich/rum-5445/make-instrumentation-tests-behavioral branch from e30e333 to e21f118 Compare November 18, 2025 15:41
@jonathanmos jonathanmos requested a review from 0xnm November 19, 2025 11:35
0xnm
0xnm previously approved these changes Nov 20, 2025

@0xnm 0xnm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have one more pair of eyes reviewing this PR.

appendLine()

// Record type constants
recordTypes.forEach { (className, value) ->

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normally it is better to use KotlinPoet to generate classes, but we can keep it like that as well

Comment thread features/dd-sdk-android-session-replay/generate_session_replay_models.gradle.kts Outdated
@jonathanmos jonathanmos merged commit cf41629 into develop Nov 23, 2025
27 checks passed
@jonathanmos jonathanmos deleted the jmoskovich/rum-5445/make-instrumentation-tests-behavioral branch November 23, 2025 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants