Skip to content

refactor(schema): add a new column for storing large connector transaction IDs#7017

Merged
likhinbopanna merged 21 commits intomainfrom
6316-db-and-dynamic-fields-update
Feb 12, 2025
Merged

refactor(schema): add a new column for storing large connector transaction IDs#7017
likhinbopanna merged 21 commits intomainfrom
6316-db-and-dynamic-fields-update

Conversation

@kashif-m
Copy link
Contributor

@kashif-m kashif-m commented Jan 9, 2025

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

This PR introduces below changes around Worldpay connector

Core changes

  • add new columns (TEXT) for storing the JWE based connector transaction ID
    • payment_attempt - processor_transaction_data
    • refund - processor_transaction_data and processor_refund_data
    • captures - processor_capture_data

Connector integration changes (Worldpay only)

  • make line1 and city non optional in payment request
  • add zip in dynamic required fields for Worldpay cards

QA

  • add cypress test cases for dynamic fields for Worldpay

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

This PR keeps Worldpay's payments integration intact.

How did you test it?

1. Route a payment through Worldpay

cURL

curl --location --request POST 'http://localhost:8080/payments' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header 'api-key: dev_Wc9lMS2KJ6l35ehmlSsR3dwwiOvGZaWLmkk3BqRLRe6vqa1rkiNJ3mVza3U1EdWh' \
    --data-raw '{"amount":100,"currency":"USD","confirm":true,"capture_method":"automatic","capture_on":"2022-09-10T10:11:12Z","customer_id":"cus_JgZZwioWal2CoKgsLXEb","email":"guest@example.com","name":"John Doe","phone":"999999999","profile_id":"pro_NyQDoa1SRAC4rZ5QjDv9","phone_country_code":"+65","description":"Its my first payment request","authentication_type":"no_three_ds","return_url":"https://hyperswitch.io","setup_future_usage":"off_session","customer_acceptance":{"acceptance_type":"online","accepted_at":"1963-05-03T04:07:52.723Z","online":{"ip_address":"127.0.0.1","user_agent":"amet irure esse"}},"connector":["worldpay"],"payment_method":"card","payment_method_data":{"card":{"card_number":"4917610000000000","card_exp_month":"03","card_exp_year":"2030","card_cvc":"737","nick_name":"JD"},"billing":{"email":"guest@example.com","address":{"first_name":"John","last_name":"US"}}},"billing":{"address":{"line1":"1467","line2":"Harrison Street","line3":"Harrison Street","city":"San Fransico","state":"California","zip":"94122","country":"US","first_name":"John","last_name":"US"}},"statement_descriptor_name":"joseph","statement_descriptor_suffix":"JS","metadata":{"udf1":"value1","new_customer":"true","login_date":"2019-09-10T10:11:12Z"},"browser_info":{"ip_address":"129.0.0.1","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36","accept_header":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","language":"en-US","color_depth":32,"screen_height":1117,"screen_width":1728,"time_zone":-330,"java_enabled":true,"java_script_enabled":true}}'

Response

{"payment_id":"pay_wY8cJE6MgvO13EE0vIyx","merchant_id":"merchant_1736497521","status":"succeeded","amount":100,"net_amount":100,"shipping_cost":null,"amount_capturable":0,"amount_received":100,"connector":"worldpay","client_secret":"pay_wY8cJE6MgvO13EE0vIyx_secret_42yawtMDmMbJf15nP8TP","created":"2025-01-10T09:35:51.576Z","currency":"USD","customer_id":"cus_JgZZwioWal2CoKgsLXEb","customer":{"id":"cus_JgZZwioWal2CoKgsLXEb","name":"John Doe","email":"guest@example.com","phone":"999999999","phone_country_code":"+65"},"description":"Its my first payment request","refunds":null,"disputes":null,"mandate_id":null,"mandate_data":null,"setup_future_usage":"off_session","off_session":null,"capture_on":null,"capture_method":"automatic","payment_method":"card","payment_method_data":{"card":{"last4":"0000","card_type":"CREDIT","card_network":"Visa","card_issuer":"BANKPOLSKAKASAOPIEKIS.A.(BANKPEKAOSA)","card_issuing_country":"POLAND","card_isin":"491761","card_extended_bin":null,"card_exp_month":"03","card_exp_year":"2030","card_holder_name":"John US","payment_checks":null,"authentication_data":null},"billing":{"address":{"city":null,"country":null,"line1":null,"line2":null,"line3":null,"zip":null,"state":null,"first_name":"John","last_name":"US"},"phone":null,"email":"guest@example.com"}},"payment_token":null,"shipping":null,"billing":{"address":{"city":"San Fransico","country":"US","line1":"1467","line2":"Harrison Street","line3":"Harrison Street","zip":"94122","state":"California","first_name":"John","last_name":"US"},"phone":null,"email":null},"order_details":null,"email":"guest@example.com","name":"John Doe","phone":"999999999","return_url":"https://hyperswitch.io/","authentication_type":"no_three_ds","statement_descriptor_name":"joseph","statement_descriptor_suffix":"JS","next_action":null,"cancellation_reason":null,"error_code":null,"error_message":null,"unified_code":null,"unified_message":null,"payment_experience":null,"payment_method_type":"credit","connector_label":null,"business_country":null,"business_label":"default","business_sub_label":null,"allowed_payment_method_types":null,"ephemeral_key":{"customer_id":"cus_JgZZwioWal2CoKgsLXEb","created_at":1736501751,"expires":1736505351,"secret":"epk_d47f022a1de246bebe54d6898c408836"},"manual_retry_allowed":false,"connector_transaction_id":"eyJrIjoiazNhYjYzMiIsImxpbmtWZXJzaW9uIjoiNi4wLjAifQ==.sN:g8wd64bwkbrp0md+bPxcanBnk2zLdsIqSa1pR99GGg8fCNQpPLoWNslSzWNPFBM5Tpa8tW7EFI5onKINsgChMHeJVoeH2lrBWCRyjZYT6h+lbqfJa+1BSoKFSY8HLJDIunO0DhjaUi3niPBfv98:VUQcO2wyJ4EGywR:l4ssCpZY8cZeCeMgt1ol:GABqQG6u+T:u3sfHv3ezSOJxioRTixsThPEhzFW4ZZ6mObj3CK0rnFndcM0swvZMqgQwSEj5tBsydfzM4XKX20O6Nme96ha9twqxTSQIfr1rtl9V3q7fw3w5O9UZT4vQi9BMyCcaHkWSD:RbCWCcmiQqa5EPu9d6p6Bt0lCYl:Np4aWZ0pJCEaV4g9uGTAlKDtig0jKl6MU3ArZWtDWm7LKjrd+hQ9RR:Z491q0vLPK+etFBNqgwjiceXMDUvG7lafYHHR5Ak5lI+Sa7YaesgGkXXzgiDTOQzNK9UH+W9rzqFt6t5VcedliH8B9MKI3IKaPyVavYo4YzMmVfRID7seWtakAYMzGq+YEqwioSKkuPXbo=","frm_message":null,"metadata":{"udf1":"value1","login_date":"2019-09-10T10:11:12Z","new_customer":"true"},"connector_metadata":null,"feature_metadata":null,"reference_id":"95f4583f-a8ee-426c-acd8-f475d85b0bef","payment_link":null,"profile_id":"pro_NyQDoa1SRAC4rZ5QjDv9","surcharge_details":null,"attempt_count":1,"merchant_decision":null,"merchant_connector_id":"mca_jJbWUjL9iGOEfQo8Uw8P","incremental_authorization_allowed":null,"authorization_count":null,"incremental_authorizations":null,"external_authentication_details":null,"external_3ds_authentication_attempted":false,"expires_on":"2025-01-10T09:50:51.576Z","fingerprint":null,"browser_info":{"language":"en-US","time_zone":-330,"ip_address":"129.0.0.1","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36","color_depth":32,"java_enabled":true,"screen_width":1728,"accept_header":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","screen_height":1117,"java_script_enabled":true},"payment_method_id":"pm_xeVtmDykRQKg3pvt4EI5","payment_method_status":"active","updated":"2025-01-10T09:35:52.981Z","split_payments":null,"frm_metadata":null,"merchant_order_reference_id":null,"order_tax_amount":null,"connector_mandate_id":"https://try.access.worldpay.com/tokens/eyJrIjoxLCJkIjoiWndnc0NNQjJXNkZFTkl4TUZUZFBKdzhQdm9OS3JjYnFnT0haOThuOVE4RT0ifQ"}
2. Create a payment link without passing in details

cURL

curl --location --request POST 'http://localhost:8080/payments' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header 'api-key: dev_Wc9lMS2KJ6l35ehmlSsR3dwwiOvGZaWLmkk3BqRLRe6vqa1rkiNJ3mVza3U1EdWh' \
    --data-raw '{"authentication_type":"three_ds","customer_id":"cus_JgZZwioWal2CoKgsLXEb","profile_id":"pro_NyQDoa1SRAC4rZ5QjDv9","amount":100,"currency":"USD","payment_link":true,"connector":["worldpay"],"description":"This is my description of why this payment was requested.","capture_method":"automatic","billing":{"address":{"line2":"Harrison Street","line3":"Harrison Street","state":"CA","first_name":"John","last_name":"Doe"},"phone":{"number":"8056594427","country_code":"+91"}},"email":"john.doe@example.co.in","session_expiry":100000,"return_url":"https://example.com","payment_link_config":{"theme":"#0E103D","logo":"https://hyperswitch.io/favicon.ico","seller_name":"Hyperswitch Inc.","enabled_saved_payment_method":true,"hide_card_nickname_field":true,"show_card_form_by_default":true,"transaction_details":[{"key":"Policy Number","value":"938478327648","ui_configuration":{"is_key_bold":false,"is_value_bold":false,"position":2}},{"key":"Business Country","value":"Germany","ui_configuration":{"is_key_bold":false,"is_value_bold":false,"position":1}},{"key":"Product Name","value":"Life Insurance Elite","ui_configuration":{"is_key_bold":false,"is_value_bold":false,"position":3}}]}}'
  • Open payment link
Screenshot 2025-01-10 at 3 08 50 PM
3. Dynamic field tests Screenshot 2025-01-10 at 3 10 29 PM

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible

@kashif-m kashif-m requested review from a team as code owners January 9, 2025 14:03
@semanticdiff-com
Copy link

semanticdiff-com bot commented Jan 9, 2025

Review changes with  SemanticDiff

Changed Files
File Status
  crates/hyperswitch_connectors/src/connectors/worldpay/requests.rs  59% smaller
  crates/hyperswitch_connectors/src/connectors/worldpay/transformers.rs  48% smaller
  crates/diesel_models/src/schema_v2.rs  28% smaller
  crates/router/src/db/capture.rs  18% smaller
  crates/common_utils/src/types.rs  0% smaller
  crates/diesel_models/src/capture.rs  0% smaller
  crates/diesel_models/src/kv.rs  0% smaller
  crates/diesel_models/src/payment_attempt.rs  0% smaller
  crates/diesel_models/src/query/capture.rs  0% smaller
  crates/diesel_models/src/refund.rs  0% smaller
  crates/diesel_models/src/schema.rs  0% smaller
  crates/diesel_models/src/user/sample_data.rs  0% smaller
  crates/hyperswitch_domain_models/src/payments/payment_attempt.rs  0% smaller
  crates/router/src/configs/defaults/payment_connector_required_fields.rs  0% smaller
  crates/router/src/core/payments/operations/payment_response.rs  0% smaller
  crates/router/src/core/payments/transformers.rs  0% smaller
  crates/router/src/core/refunds.rs  0% smaller
  crates/router/src/core/webhooks/incoming.rs  0% smaller
  crates/router/src/db/refund.rs  0% smaller
  crates/router/src/types/storage/payment_attempt.rs  0% smaller
  crates/router/src/utils/user/sample_data.rs  0% smaller
  crates/storage_impl/src/payments/payment_attempt.rs  0% smaller
  cypress-tests/cypress/e2e/configs/Payment/WorldPay.js  0% smaller
  migrations/2025-01-09-135057_add_processor_transaction_data/down.sql Unsupported file format
  migrations/2025-01-09-135057_add_processor_transaction_data/up.sql Unsupported file format
  v2_migrations/2024-08-28-081721_add_v2_columns/up.sql Unsupported file format
  v2_migrations/2025-01-13-081847_drop_v1_columns/down.sql Unsupported file format
  v2_migrations/2025-01-13-081847_drop_v1_columns/up.sql Unsupported file format

@kashif-m kashif-m self-assigned this Jan 9, 2025
@hyperswitch-bot hyperswitch-bot bot added the M-database-changes Metadata: This PR involves database schema changes label Jan 9, 2025
@kashif-m kashif-m added A-payments Area: payments and removed M-database-changes Metadata: This PR involves database schema changes labels Jan 9, 2025
@kashif-m kashif-m requested a review from a team as a code owner January 10, 2025 07:55
@hyperswitch-bot hyperswitch-bot bot added the M-database-changes Metadata: This PR involves database schema changes label Jan 10, 2025
@kashif-m kashif-m linked an issue Jan 10, 2025 that may be closed by this pull request
2 tasks
@kashif-m kashif-m requested a review from a team as a code owner January 10, 2025 09:27
@kashif-m kashif-m requested review from a team as code owners January 13, 2025 08:16
@kashif-m kashif-m changed the title refactor(schema): alter type of connector_transaction_data and connector_refund_data columns to 1024 chars refactor(schema): add a new column for storing large connector transaction IDs Jan 13, 2025
likhinbopanna
likhinbopanna previously approved these changes Feb 10, 2025
jarnura
jarnura previously approved these changes Feb 10, 2025
pixincreate
pixincreate previously approved these changes Feb 10, 2025
apoorvdixit88
apoorvdixit88 previously approved these changes Feb 10, 2025
@likhinbopanna likhinbopanna added this pull request to the merge queue Feb 11, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Feb 11, 2025
@likhinbopanna likhinbopanna added this pull request to the merge queue Feb 12, 2025
Merged via the queue into main with commit fa09db1 Feb 12, 2025
17 of 20 checks passed
@likhinbopanna likhinbopanna deleted the 6316-db-and-dynamic-fields-update branch February 12, 2025 08:49
kashif-m added a commit that referenced this pull request Feb 12, 2025
…ction IDs (#7017)

Co-authored-by: hyperswitch-bot[bot] <148525504+hyperswitch-bot[bot]@users.noreply.github.com>
(cherry picked from commit fa09db1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-payments Area: payments M-database-changes Metadata: This PR involves database schema changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] connector's txn ID exceeding 512 characters are not being stored in DB

6 participants