feat: add primary key not null query to generic filter function by hrithikesh026 · Pull Request #7785 · juspay/hyperswitch

hrithikesh026 · 2025-04-10T11:43:16Z

Type of Change

Description

Since schema for both v1 and v2 can be unified, it paves way for a new bug.
In v1, while querying a record with a column that is common for both v1 and v2, the query may retrieve list of records belonging to both v1 and v2.

Now v1 application will be able to deserialise v1 records in the list without any problems. But v2 records would have few v1 required fields as null. This records will not be deserialized by the v1 application.

For example:
Lets say an orgnization has both v1 and v2 merchants.
Lets do merchant list by organization_id in v1. This will fetch all v1 and v2 merchant_accounts from the DB. Since merchant_id column will not be populated in v2 records, it will cause deserialization failure.

Fix:
In order to fix this problem, for all filter DB queries in application, we will add AND PRIMARY_KEY IS NOT NULL along with existing where clause. This way v2 created records will not be fetched in v1 application.

This will also make sure list endpoint in v2 application won't fail when backfill is not made.

Additional Changes

This PR modifies the API contract
This PR modifies the database schema
This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Create a merchant in v1 (org and profile with be created internally)

curl --location 'http://localhost:8080/accounts' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'api-key: test_admin' \
--data-raw '{
  "merchant_id": "merchant_1744289448",
  "locker_id": "m0010",
  "merchant_name": "NewAge Retailer",
  "merchant_details": {
    "primary_contact_person": "John Test",
    "primary_email": "JohnTest@test.com",
    "primary_phone": "sunt laborum",
    "secondary_contact_person": "John Test2",
    "secondary_email": "JohnTest2@test.com",
    "secondary_phone": "cillum do dolor id",
    "website": "www.example.com",
    "about_business": "Online Retail with a wide selection of organic products for North America",
    "address": {
      "line1": "1467",
      "line2": "Harrison Street",
      "line3": "Harrison Street",
      "city": "San Fransico",
      "state": "California",
      "zip": "94122",
      "country": "US"
    }
  },
  "return_url": "https://google.com/success",
  "webhook_details": {
    "webhook_version": "1.0.1",
    "webhook_username": "ekart_retail",
    "webhook_password": "password_ekart@123",
    "payment_created_enabled": true,
    "payment_succeeded_enabled": true,
    "payment_failed_enabled": true
  },
  "sub_merchants_enabled": false,
  "metadata": {
    "city": "NY",
    "unit": "245"
  }
}'

Response:

{
    "merchant_id": "merchant_1744288162",
    "merchant_name": "NewAge Retailer",
    "return_url": "https://google.com/success",
    "enable_payment_response_hash": true,
    "payment_response_hash_key": "QzOPDGAfZ1NVNXim3qyyiJYjUXheuHY7IXoEQikHpN9iVnRFwM0D80qRqKdZI5l6",
    "redirect_to_merchant_with_http_post": false,
    "merchant_details": {
        "primary_contact_person": "John Test",
        "primary_phone": "sunt laborum",
        "primary_email": "JohnTest@test.com",
        "secondary_contact_person": "John Test2",
        "secondary_phone": "cillum do dolor id",
        "secondary_email": "JohnTest2@test.com",
        "website": "www.example.com",
        "about_business": "Online Retail with a wide selection of organic products for North America",
        "address": {
            "city": "San Fransico",
            "country": "US",
            "line1": "1467",
            "line2": "Harrison Street",
            "line3": "Harrison Street",
            "zip": "94122",
            "state": "California",
            "first_name": null,
            "last_name": null
        }
    },
    "webhook_details": {
        "webhook_version": "1.0.1",
        "webhook_username": "ekart_retail",
        "webhook_password": "password_ekart@123",
        "webhook_url": null,
        "payment_created_enabled": true,
        "payment_succeeded_enabled": true,
        "payment_failed_enabled": true
    },
    "payout_routing_algorithm": null,
    "sub_merchants_enabled": false,
    "parent_merchant_id": null,
    "publishable_key": "pk_dev_0eccdc2b95184c13885ae712f851d1ec",
    "metadata": {
        "city": "NY",
        "unit": "245",
        "compatible_connector": null
    },
    "locker_id": "m0010",
    "primary_business_details": [],
    "frm_routing_algorithm": null,
    "organization_id": "org_6QGW3lXGhurZmJ2rfo8A",
    "is_recon_enabled": false,
    "default_profile": "pro_nOKzSPRoENNiLTMXlEdP",
    "recon_status": "not_requested",
    "pm_collect_link_config": null,
    "product_type": "orchestration"
}

Create a Merchant Account in v2 for the above Org.

curl --location 'localhost:8080/v2/merchant-accounts' \
--header 'x-organization-id: org_6QGW3lXGhurZmJ2rfo8A' \
--header 'Content-Type: application/json' \
--header 'Authorization: admin-api-key=test_admin' \
--data '{
    "merchant_name": "cloth_seller"
}'

Response:

{
    "id": "cloth_seller_5yMFtTAMqUQC4CyQwJD8",
    "merchant_name": "cloth_seller",
    "merchant_details": null,
    "publishable_key": "pk_dev_a9e62e6b0c3a4c1584fd7ae89bab646d",
    "metadata": null,
    "organization_id": "org_6QGW3lXGhurZmJ2rfo8A",
    "recon_status": "not_requested",
    "product_type": "orchestration"
}

Do merchant list for org in V1.

curl --location 'http://localhost:8080/accounts/list?organization_id=org_6QGW3lXGhurZmJ2rfo8A' \
--header 'Accept: application/json' \
--header 'api-key: test_admin'

Response:

[
    {
        "merchant_id": "merchant_1744288162",
        "merchant_name": "NewAge Retailer",
        "return_url": "https://google.com/success",
        "enable_payment_response_hash": true,
        "payment_response_hash_key": "QzOPDGAfZ1NVNXim3qyyiJYjUXheuHY7IXoEQikHpN9iVnRFwM0D80qRqKdZI5l6",
        "redirect_to_merchant_with_http_post": false,
        "merchant_details": {
            "primary_contact_person": "John Test",
            "primary_phone": "sunt laborum",
            "primary_email": "JohnTest@test.com",
            "secondary_contact_person": "John Test2",
            "secondary_phone": "cillum do dolor id",
            "secondary_email": "JohnTest2@test.com",
            "website": "www.example.com",
            "about_business": "Online Retail with a wide selection of organic products for North America",
            "address": {
                "city": "San Fransico",
                "country": "US",
                "line1": "1467",
                "line2": "Harrison Street",
                "line3": "Harrison Street",
                "zip": "94122",
                "state": "California",
                "first_name": null,
                "last_name": null
            }
        },
        "webhook_details": {
            "webhook_version": "1.0.1",
            "webhook_username": "ekart_retail",
            "webhook_password": "password_ekart@123",
            "webhook_url": null,
            "payment_created_enabled": true,
            "payment_succeeded_enabled": true,
            "payment_failed_enabled": true
        },
        "payout_routing_algorithm": null,
        "sub_merchants_enabled": false,
        "parent_merchant_id": null,
        "publishable_key": "pk_dev_0eccdc2b95184c13885ae712f851d1ec",
        "metadata": {
            "city": "NY",
            "unit": "245",
            "compatible_connector": null
        },
        "locker_id": "m0010",
        "primary_business_details": [],
        "frm_routing_algorithm": null,
        "organization_id": "org_6QGW3lXGhurZmJ2rfo8A",
        "is_recon_enabled": false,
        "default_profile": "pro_nOKzSPRoENNiLTMXlEdP",
        "recon_status": "not_requested",
        "pm_collect_link_config": null,
        "product_type": "orchestration"
    }
]

Only v1 merchants are being returned.

Response: (only v1 merchant list should be returned).

But in DB, there are v1 as well as v2 merchants.

DB query has the appropriate IS NOT NULL clause.

Checklist

I formatted the code cargo +nightly fmt --all
I addressed lints thrown by cargo clippy
I reviewed the submitted code
I added unit tests for my changes where possible

semanticdiff-com · 2025-04-10T11:43:19Z

Review changes with

Changed Files

File	Status
crates/diesel_models/src/query/generics.rs	54% smaller
crates/diesel_models/src/query.rs	0% smaller
crates/diesel_models/src/query/utils.rs	0% smaller

jarnura

LGTM

SanchithHegde · 2025-04-11T10:32:41Z

+    schema_v2::payment_attempt::table
+);
+
+// Manual implementations below


Why did we have to write these implementations manually?

…ilter

…acilitapay-pix-pmt * 'main' of github.com:juspay/hyperswitch: (21 commits) refactor(required_fields): move pm required fields to pm crate (#7539) fix(connector): [noon] address `next_action_url` being `null` for cards in 3ds payment (#7832) refactor(middleware): add middleware to record metrics for request count and duration (#7803) chore(version): 2025.04.18.0 chore(postman): update Postman collection files fix(connector): [globalpay] handle edge case where currency comes as empty upon payment decline (#7812) refactor(cypress-v2): change `Authorization` and `payment_methods_enabled` for v2 cypress tests (#7805) fix(connector): [Cybersource] send type selection indicator for co-batch cards (#7828) feat(payment_method): add logic for setup_future_usage downgrade and add filter based on zero mandate config (#7775) refactor(accounts): move dashboard_metadata table to accounts_schema and point v2 to v1 dashboard_metadata (#7793) chore(analytics): opensearch client creation based on config (#7810) ci(postman): update assertion error message for nmi collection (#7765) feat: add primary key not null query to generic filter function (#7785) chore(version): 2025.04.17.0 chore: change payment method files ownership to `hyperswitch-payment-methods` (#7808) feat(vsaas): modify api key auth to support vsaas cases (#7593) ci(cypress): verify mandate id to be `null` if payment id not `succeeded` (#7749) feat(connector): [chargebee] consumes required fields to support transaction monitoring (#7774) ci(configs): remove vault private key from configs (#7825) chore(version): 2025.04.16.0 ...

feat: add primary key not null query to generic filter function

5c3955e

hrithikesh026 added the A-core Area: Core flows label Apr 10, 2025

hrithikesh026 requested a review from jarnura April 10, 2025 11:43

hrithikesh026 self-assigned this Apr 10, 2025

hrithikesh026 requested a review from a team as a code owner April 10, 2025 11:43

hrithikesh026 linked an issue Apr 11, 2025 that may be closed by this pull request

[FEATURE] create accounts schema for accounts tables #7125

Closed

2 tasks

hrithikesh026 mentioned this pull request Apr 11, 2025

[FEATURE] create accounts schema for accounts tables #7125

Closed

2 tasks

chore: remove unwanted trait bounds

5bffb19

hrithikesh026 requested a review from SanchithHegde April 11, 2025 06:29

jarnura previously approved these changes Apr 11, 2025

View reviewed changes

hrithikesh026 requested a review from Narayanbhat166 April 11, 2025 08:26

SanchithHegde reviewed Apr 11, 2025

View reviewed changes

chore: introduce trait CompositeKey

1a121ea

hrithikesh026 dismissed jarnura’s stale review via 1a121ea April 11, 2025 12:42

jarnura approved these changes Apr 13, 2025

View reviewed changes

SanchithHegde approved these changes Apr 15, 2025

View reviewed changes

Merge branch 'main' into add-primary-key-not-null-query-for-generic-f…

64b7953

…ilter

Gnanasundari24 added this pull request to the merge queue Apr 16, 2025

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 16, 2025

Merge branch 'main' into add-primary-key-not-null-query-for-generic-f…

87d7110

…ilter

Gnanasundari24 enabled auto-merge April 17, 2025 04:43

Gnanasundari24 added this pull request to the merge queue Apr 17, 2025

Merged via the queue into main with commit 8738f19 Apr 17, 2025
16 of 20 checks passed

Gnanasundari24 deleted the add-primary-key-not-null-query-for-generic-filter branch April 17, 2025 05:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add primary key not null query to generic filter function#7785

feat: add primary key not null query to generic filter function#7785
Gnanasundari24 merged 5 commits intomainfrom
add-primary-key-not-null-query-for-generic-filter

hrithikesh026 commented Apr 10, 2025 •

edited

Loading

Uh oh!

semanticdiff-com Bot commented Apr 10, 2025 •

edited

Loading

Uh oh!

jarnura left a comment

Uh oh!

SanchithHegde Apr 11, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

hrithikesh026 commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of Change

Description

Additional Changes

Motivation and Context

How did you test it?

Checklist

Uh oh!

semanticdiff-com Bot commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jarnura left a comment

Choose a reason for hiding this comment

Uh oh!

SanchithHegde Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hrithikesh026 commented Apr 10, 2025 •

edited

Loading

semanticdiff-com Bot commented Apr 10, 2025 •

edited

Loading