Skip to content

feat(revenue_recovery): add support for updating additional card info data from csv to redis#9233

Merged
likhinbopanna merged 26 commits intomainfrom
redis_backfill
Sep 18, 2025
Merged

feat(revenue_recovery): add support for updating additional card info data from csv to redis#9233
likhinbopanna merged 26 commits intomainfrom
redis_backfill

Conversation

@AdityaKumaar21
Copy link
Member

@AdityaKumaar21 AdityaKumaar21 commented Sep 1, 2025

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

The format of csv:-
Screenshot 2025-09-02 at 13 57 36
Screenshot 2025-09-02 at 13 57 54
Screenshot 2025-09-02 at 13 58 07

here CustomerReq_ID is the customer token and Token is the payment token which is used as an identifier in Redis for tokens, ("customer:{}:tokens", customer_id) is used to get token and Token is used to find the field which we need to update.
So, we are parsing the csv and getting the valid records and building a request per record RevenueRecoveryBackfillRequest, from this request we are fetching the data like expiry month, year, card type, card issuer, card network, any of these fields are null, we are updating them from the request.

Complexity:-
for each valid record we are doing two Redis call, one for get and another for set.

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

For Revenue recovery service, we are storing data in Redis with format

HGETALL customer:fake_customer123:tokens
1) "fake_token123"
2) "{\"payment_processor_token_details\":{\"payment_processor_token\":\"fake_token123\",\"expiry_month\":\"08\",\"expiry_year\":\"26\",\"card_issuer\":\"Green Dot Bank Dba Bonneville Bank\",\"last_four_digits\":null,\"card_network\":\"Visa\",\"card_type\":\"debit\"},\"inserted_by_attempt_id\":\"12345_att_fake_entry123\",\"error_code\":\"insufficient_funds\",\"daily_retry_history\":{\"2024-05-29\":1},\"scheduled_at\":null}"

in last data migration card_type was populated as null, but for Smart retry logic we need fields like card_type, card_issuer, card_network.
So, inorder to work with Smart retry we need to backfill card_type data from merchant csv.

How did you test it?

cURL:-

curl --location 'http://localhost:8080/recovery/data-backfill' \
--header 'Authorization: api_key =' \
--header 'x-profile-id: pro_PljcGfk8IckJFT1WdFsn' \
--header 'api-key:' \
--form 'file=@"/Users/aditya.c/Downloads/_first_100_rows.csv"'

Router log:-
Screenshot 2025-09-02 at 13 47 22

cURL with query param for updating scheduled at time

curl --location 'http://localhost:8080/v2/recovery/data-backfill?cutoff_time=2025-09-17T10%3A30%3A00' \
--header 'Authorization: admin-api-key=test_admin' \
--header 'x-profile-id: pro_4xOQlxvKwZbLUx8ThJXJ' \
--form 'file=@"/Users/aditya.c/Downloads/sample_merged_data (1).csv"'

Router log:-
Screenshot 2025-09-17 at 17 06 56

Redis-cli log(updating scheduledat to null if cut-off time is greater than scheduled at time):-
Screenshot 2025-09-17 at 17 10 31

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible

@AdityaKumaar21 AdityaKumaar21 requested review from a team as code owners September 1, 2025 20:39
@semanticdiff-com
Copy link

semanticdiff-com bot commented Sep 1, 2025

hyperswitch-bot bot and others added 2 commits September 1, 2025 20:39
Removed commented-out code for updating Redis token and parsing card data.
jarnura
jarnura previously approved these changes Sep 3, 2025
jarnura
jarnura previously approved these changes Sep 9, 2025
@Aprabhat19
Copy link
Contributor

Add support to update all possible values in redis , the ones that we internally compute as well.

@Aprabhat19
Copy link
Contributor

Aprabhat19 commented Sep 10, 2025

Why are we just updating if the fields are None . Why not update if we know some fields were inserted with wrong information. We should support both data_correction as well as Backfill.

}

#[derive(Debug, Clone, Serialize)]
pub enum BackfillError {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

derive error On this and also move it to some other file

srujanchikke
srujanchikke previously approved these changes Sep 16, 2025
srujanchikke
srujanchikke previously approved these changes Sep 17, 2025
card_network: None,
card_issuer: None,
card_issuing_country: None,
daily_retry_history: None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we not allowed to update these fields?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this update is only a fallback when we get an error

@likhinbopanna likhinbopanna added this pull request to the merge queue Sep 18, 2025
Merged via the queue into main with commit d98ffdf Sep 18, 2025
21 of 25 checks passed
@likhinbopanna likhinbopanna deleted the redis_backfill branch September 18, 2025 11:31
serde = { version = "1.0.219", features = ["derive"] }
serde_json = "1.0.140"
strum = { version = "0.26", features = ["derive"] }
tempfile = "3.8"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AdityaKumaar21 Where are we using this dependency in this crate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are using this in /api_models/src/revenue_recovery_data_backfill.rs ->
RevenueRecoveryDataBackfillForm (it handles large files)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(revenue_recovery): add support for data migration from merchant csv to Redis

7 participants