feat(opensearch): add profile_id and organization_id to /search APIs by tsdk02 · Pull Request #5705 · juspay/hyperswitch

tsdk02 · 2024-08-27T05:25:37Z

Type of Change

Description

Global Search is currently being done through the merchant_id authentication.
Enhancing the global search to search through new hierarchical structure of OrgLevel, MerchantLevel and ProfileLevel.
Structure of the enum used:

pub enum AuthInfo {
    OrgLevel {
        org_id: id_type::OrganizationId,
    },
    MerchantLevel {
        org_id: id_type::OrganizationId,
        merchant_ids: Vec<id_type::MerchantId>,
    },
    ProfileLevel {
        org_id: id_type::OrganizationId,
        merchant_id: id_type::MerchantId,
        profile_ids: Vec<id_type::ProfileId>,
    },
}

Modified the query-building for opensearch/elasticsearch by adding the orgLevel and ProfileLevel queries.
Results will now be returned if any search query is matched, and for all matched queries

Additional Changes

This PR modifies the API contract
This PR modifies the database schema
This PR modifies application configuration/environment variables

Motivation and Context

OrgLevel, MerchantLevel and ProfileLevel provide for enhanced analytics insights while using global search to the users

How did you test it?

Sample Query Structure
Hit the /search API (using query, filters and timeRange).

curl --location 'http://localhost:8080/analytics/v1/search' \
--header 'Accept: */*' \
--header 'Accept-Language: en-US,en;q=0.9' \
--header 'Connection: keep-alive' \
--header 'Content-Type: application/json' \
--header 'Origin: http://localhost:9000' \
--header 'Referer: http://localhost:9000/' \
--header 'Sec-Fetch-Dest: empty' \
--header 'Sec-Fetch-Mode: cors' \
--header 'Sec-Fetch-Site: same-site' \
--header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36' \
--header 'api-key: hyperswitch' \
--header 'authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoiY2M3M2ZiY2EtYTExMS00Nzk5LWI3MDEtN2E4NmU0OWIxMzY1IiwibWVyY2hhbnRfaWQiOiJtZXJjaGFudF8xNzI1MjY3Mzk3Iiwicm9sZV9pZCI6Im9yZ19hZG1pbiIsImV4cCI6MTcyNTUxNzc2Nywib3JnX2lkIjoib3JnX0lpbTRxOHpPdlA3aGp3QTRxTkpoIiwicHJvZmlsZV9pZCI6bnVsbH0.7vZllJaSLEw1EsU4tgXkEcpAjRmjlukk6swzZfa6uU0' \
--header 'sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--data '{
    "query": "usd",
    "filters": {
        "status": [
            "succeeded", "charged", "success"
        ]
    },
    "timeRange": {
        "startTime": "2024-09-02T00:30:00Z",
        "endTime": "2024-09-04T21:45:00Z"
    }
}'

Sample Response:

[
    {
        "count": 2,
        "index": "payment_attempts",
        "hits": [
            {
                "payment_id": "pay_mUtaBhUnlC2b7kTtxTJ0",
                "merchant_id": "merchant_1725267397",
                "attempt_id": "pay_mUtaBhUnlC2b7kTtxTJ0_1",
                "status": "charged",
                "amount": 6540,
                "currency": "USD",
                "save_to_locker": null,
                "connector": "stripe_test",
                "error_message": null,
                "offer_amount": null,
                "surcharge_amount": null,
                "tax_amount": null,
                "payment_method_id": null,
                "payment_method": "card",
                "connector_transaction_id": "pay_XO5KGoaMl5y7R8fM3Xya",
                "capture_method": "automatic",
                "capture_on": 1662804672,
                "confirm": true,
                "authentication_type": "no_three_ds",
                "created_at": 1725339016,
                "modified_at": 1725339017,
                "last_synced": 1725339016,
                "cancellation_reason": null,
                "amount_to_capture": 6540,
                "mandate_id": null,
                "browser_info": null,
                "error_code": null,
                "connector_metadata": null,
                "payment_experience": null,
                "payment_method_type": "credit",
                "payment_method_data": "{\"card\":{\"last4\":\"4242\",\"bank_code\":null,\"card_isin\":\"424242\",\"card_type\":null,\"card_issuer\":null,\"card_network\":null,\"card_exp_year\":\"25\",\"card_exp_month\":\"10\",\"payment_checks\":null,\"card_holder_name\":null,\"card_extended_bin\":null,\"authentication_data\":null,\"card_issuing_country\":null}}",
                "error_reason": null,
                "multiple_capture_count": null,
                "amount_capturable": 0,
                "merchant_connector_id": "mca_zPBTVwk8mQPMvW9rrW8n",
                "net_amount": 6540,
                "unified_code": null,
                "unified_message": null,
                "mandate_data": null,
                "client_source": null,
                "client_version": null,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "card_network": null,
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-03T04:50:16.000000000+00:00"
            },
            {
                "payment_id": "pay_CW0tpqeKk7F6q2TVKffH",
                "merchant_id": "merchant_1725267397",
                "attempt_id": "pay_CW0tpqeKk7F6q2TVKffH_1",
                "status": "charged",
                "amount": 6540,
                "currency": "USD",
                "save_to_locker": null,
                "connector": "stripe_test",
                "error_message": null,
                "offer_amount": null,
                "surcharge_amount": null,
                "tax_amount": null,
                "payment_method_id": null,
                "payment_method": "card",
                "connector_transaction_id": "pay_w5UHXOqOiLqRieOUQL63",
                "capture_method": "automatic",
                "capture_on": 1662804672,
                "confirm": true,
                "authentication_type": "no_three_ds",
                "created_at": 1725283209,
                "modified_at": 1725283210,
                "last_synced": 1725283209,
                "cancellation_reason": null,
                "amount_to_capture": 6540,
                "mandate_id": null,
                "browser_info": null,
                "error_code": null,
                "connector_metadata": null,
                "payment_experience": null,
                "payment_method_type": "credit",
                "payment_method_data": "{\"card\":{\"last4\":\"4242\",\"bank_code\":null,\"card_isin\":\"424242\",\"card_type\":null,\"card_issuer\":null,\"card_network\":null,\"card_exp_year\":\"25\",\"card_exp_month\":\"10\",\"payment_checks\":null,\"card_holder_name\":null,\"card_extended_bin\":null,\"authentication_data\":null,\"card_issuing_country\":null}}",
                "error_reason": null,
                "multiple_capture_count": null,
                "amount_capturable": 0,
                "merchant_connector_id": "mca_zPBTVwk8mQPMvW9rrW8n",
                "net_amount": 6540,
                "unified_code": null,
                "unified_message": null,
                "mandate_data": null,
                "client_source": null,
                "client_version": null,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "card_network": null,
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-02T13:20:09.000000000+00:00"
            }
        ],
        "status": "Success"
    },
    {
        "count": 2,
        "index": "payment_intents",
        "hits": [
            {
                "payment_id": "pay_mUtaBhUnlC2b7kTtxTJ0",
                "merchant_id": "merchant_1725267397",
                "status": "succeeded",
                "amount": 6540,
                "currency": "USD",
                "amount_captured": 6540,
                "customer_id": "StripeCustomer",
                "description": "Its my first payment request",
                "return_url": "https://google.com/",
                "metadata": "{\"data2\":\"camel\",\"login_date\":\"2019-09-10T10:11:12Z\",\"new_customer\":\"true\"}",
                "connector_id": null,
                "statement_descriptor_name": "joseph",
                "statement_descriptor_suffix": "JS",
                "created_at": 1725339016,
                "modified_at": 1725339017,
                "last_synced": 1725339016,
                "setup_future_usage": null,
                "off_session": null,
                "client_secret": "pay_mUtaBhUnlC2b7kTtxTJ0_secret_wiZ5NQnCZ2i9ajdAYyAJ",
                "active_attempt_id": "pay_mUtaBhUnlC2b7kTtxTJ0_1",
                "business_country": null,
                "business_label": "default",
                "attempt_count": 1,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "payment_confirm_source": null,
                "billing_details": null,
                "shipping_details": null,
                "customer_email": "f6cd0753a831880835a0e0d23136ef66318d8303605bd37fa80b4ecf248bd303",
                "feature_metadata": {
                    "redirect_response": null,
                    "search_tags": [
                        "9c202f2fabd12f758a12b449944f5cdaeadf39132680667543033158023bc805",
                        "ffd1c37eb18f3459e3da6f21f739866bce89a334061c248656cb71d83d2375f4"
                    ]
                },
                "merchant_order_reference_id": null,
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-03T04:50:16.000000000+00:00"
            },
            {
                "payment_id": "pay_CW0tpqeKk7F6q2TVKffH",
                "merchant_id": "merchant_1725267397",
                "status": "succeeded",
                "amount": 6540,
                "currency": "USD",
                "amount_captured": 6540,
                "customer_id": "StripeCustomer",
                "description": "Its my first payment request",
                "return_url": "https://google.com/",
                "metadata": "{\"data2\":\"camel\",\"login_date\":\"2019-09-10T10:11:12Z\",\"new_customer\":\"true\"}",
                "connector_id": null,
                "statement_descriptor_name": "joseph",
                "statement_descriptor_suffix": "JS",
                "created_at": 1725283209,
                "modified_at": 1725283210,
                "last_synced": 1725283209,
                "setup_future_usage": null,
                "off_session": null,
                "client_secret": "pay_CW0tpqeKk7F6q2TVKffH_secret_rZIT5OsQzbhkE7rhsUrq",
                "active_attempt_id": "pay_CW0tpqeKk7F6q2TVKffH_1",
                "business_country": null,
                "business_label": "default",
                "attempt_count": 1,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "payment_confirm_source": null,
                "billing_details": null,
                "shipping_details": null,
                "customer_email": "f6cd0753a831880835a0e0d23136ef66318d8303605bd37fa80b4ecf248bd303",
                "feature_metadata": {
                    "redirect_response": null,
                    "search_tags": [
                        "9c202f2fabd12f758a12b449944f5cdaeadf39132680667543033158023bc805",
                        "ffd1c37eb18f3459e3da6f21f739866bce89a334061c248656cb71d83d2375f4"
                    ]
                },
                "merchant_order_reference_id": null,
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-02T13:20:09.000000000+00:00"
            }
        ],
        "status": "Success"
    },
    {
        "count": 1,
        "index": "refunds",
        "hits": [
            {
                "internal_reference_id": "refid_yWIsLDexIn3LvMoEvXds",
                "refund_id": "ref_WBSqj48rx3gaOuz1kUJv",
                "payment_id": "pay_mUtaBhUnlC2b7kTtxTJ0",
                "merchant_id": "merchant_1725267397",
                "connector_transaction_id": "pay_XO5KGoaMl5y7R8fM3Xya",
                "connector": "stripe_test",
                "connector_refund_id": "dummy_ref_DA5VW0KwaiCvnTaeBtcA",
                "external_reference_id": "ref_WBSqj48rx3gaOuz1kUJv",
                "refund_type": "instant_refund",
                "total_amount": 6540,
                "currency": "USD",
                "refund_amount": 600,
                "refund_status": "success",
                "sent_to_gateway": true,
                "refund_error_message": null,
                "refund_arn": "",
                "created_at": 1725339184,
                "modified_at": 1725339185,
                "description": "Customer returned product",
                "attempt_id": "pay_mUtaBhUnlC2b7kTtxTJ0_1",
                "refund_reason": "Customer returned product",
                "refund_error_code": null,
                "profile_id": "pro_GOyb8ecA2QA4pXnJTvAL",
                "organization_id": "org_Iim4q8zOvP7hjwA4qNJh",
                "sign_flag": 1,
                "tenant_id": "public",
                "clickhouse_database": "default",
                "@timestamp": "2024-09-03T04:53:04.000000000+00:00"
            }
        ],
        "status": "Success"
    },
    {
        "count": 0,
        "index": "disputes",
        "hits": [],
        "status": "Failure"
    }
]

Checklist

I formatted the code cargo +nightly fmt --all
I addressed lints thrown by cargo clippy
I reviewed the submitted code
I added unit tests for my changes where possible

semanticdiff-com · 2024-08-27T05:25:40Z

Review changes with SemanticDiff.

Analyzed 3 of 3 files.

Overall, the semantic diff is 17% smaller than the GitHub diff.

	Filename	Status
✔️	crates/router/src/analytics.rs	10.69% smaller
✔️	crates/analytics/src/opensearch.rs	6.1% smaller
✔️	crates/analytics/src/search.rs	43.63% smaller

…auth layer and constructing search_params

…(for the time being)

* 'main' of github.com:juspay/hyperswitch: (51 commits) feat(connector): [DEUTSCHEBANK] Integrate SEPA Payments (#5826) feat(payments_v2): payment intent diesel and domain models changes v2 (#5783) feat(connector): [Fiuu] ADD Wasm Configs (#5874) chore(version): 2024.09.13.0 refactor(core): Update shipping_cost and order_tax_amount to net_amount of payment_attempt (#5844) refactor: return optional request body from build_request_v2 in ConnectorIntegrationV2 trait (#5865) feat(refunds): Refunds aggregate api (#5795) refactor: handle redirections for iframed content (#5591) refactor(payment_links): Update API contract for dynamic transaction details and upgrade UI (#5849) fix(router): add payment_method check in `get_mandate_type` (#5828) fix(connector): [ZSL] compare consr_paid_amt with the total amount for identifying partial payments (#5873) feat(connector): [Novalnet] add Payment flows for cards (#5726) chore(version): 2024.09.12.0 fix(router): return `collect_billing_details_from_wallet_connector` if `always_collect_billing_details_from_wallet_connector ` is false in merchant payment method list (#5854) feat(opensearch): add profile_id and organization_id to /search APIs (#5705) build(deps): bump `sqlx` to `0.8.2` (#5859) refactor: Remove unwanted commented lines (#5851) feat(payments): add support for profile aggregates (#5845) Feat(connector): [Fiuu] Add DuitNow/FPX PaymentMethod (#5841) chore: remove Connectors enum dependency from ConnectorIntegrationV2 trait (#5840) ...

tsdk02 added 3 commits August 27, 2024 10:36

add: search api changes to accept org id, profile id

25ec998

merged main

806f78e

fix: make fmt

f323108

tsdk02 added C-feature Category: Feature request or enhancement A-Analytics labels Aug 27, 2024

tsdk02 self-assigned this Aug 27, 2024

tsdk02 requested review from a team as code owners August 27, 2024 05:25

fixed errors

b590afb

tsdk02 linked an issue Aug 27, 2024 that may be closed by this pull request

feat(opensearch): add profile_id and organization_id to /search APIs #5706

Closed

tsdk02 and others added 10 commits August 27, 2024 11:25

removed print statements

7e31da6

Merged main

4de0c6c

added ProfileId id_type and minor nits

b041207

merged main

e86709f

merged main

550f53d

resolved conflicts and minor nits

993c968

chore: run formatter

e447a9d

merged main

e7ca008

update: /search api for accepting merchantId and organizationId from …

bb9b0ff

…auth layer and constructing search_params

chore: run formatter

c361c47

tsdk02 requested review from a team, ThisIsMani and lsampras September 4, 2024 04:04

ThisIsMani previously approved these changes Sep 4, 2024

View reviewed changes

resolved conflicts

07d47a9

tsdk02 dismissed ThisIsMani’s stale review via 07d47a9 September 11, 2024 09:50

tsdk02 and others added 3 commits September 11, 2024 15:48

update: removed org_id as a filter in merchantLevel and profileLevel …

ec5a71f

…(for the time being)

chore: run formatter

fe31aa5

add: TODO to add the org_id as filter later

3cf94a0

ThisIsMani approved these changes Sep 11, 2024

View reviewed changes

lsampras approved these changes Sep 11, 2024

View reviewed changes

Gnanasundari24 added this pull request to the merge queue Sep 11, 2024

Merged via the queue into main with commit 7eb5354 Sep 11, 2024

Gnanasundari24 deleted the accept-profile-id-org-id-search-api branch September 11, 2024 16:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(opensearch): add profile_id and organization_id to /search APIs#5705

feat(opensearch): add profile_id and organization_id to /search APIs#5705
Gnanasundari24 merged 18 commits intomainfrom
accept-profile-id-org-id-search-api

tsdk02 commented Aug 27, 2024 •

edited

Loading

Uh oh!

semanticdiff-com bot commented Aug 27, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

tsdk02 commented Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of Change

Description

Additional Changes

Motivation and Context

How did you test it?

Checklist

Uh oh!

semanticdiff-com bot commented Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tsdk02 commented Aug 27, 2024 •

edited

Loading

semanticdiff-com bot commented Aug 27, 2024 •

edited

Loading