[IONIX] Migrate to Codeless Connector Framework (CCF) by ItaiMargalit · Pull Request #13524 · Azure/Azure-Sentinel

ItaiMargalit · 2026-01-29T13:40:16Z

Summary

Migrated IONIX connector from HTTP Data Collector API (push model) to Codeless Connector Framework (CCF) with RestApiPoller (pull model)
Automatic daily polling from IONIX API - no need to configure IONIX portal to push data
Simplified setup - users just enter API token and account name
Added query-time deduplication using id_s field to prevent duplicate action items
Data continues to flow to existing CyberpionActionItems_CL table

Changes

mainTemplate.json: Added RestApiPoller data connector with DCE/DCR resources
Workbooks/IONIXOverviewWorkbook.json: Updated KQL queries with summarize arg_max(TimeGenerated, *) by id_s deduplication
Analytic Rules/HighUrgencyActionItems.yaml: Updated query with id_s deduplication, bumped version to 1.0.2
ReleaseNotes.md: Added v4.0.0 changelog

- Replace HTTP Data Collector API (push) with Codeless Connector Framework (pull) - Add RestApiPoller with daily polling (1440 min) from IONIX API - Simplified setup: API token + account name only - Data continues to CyberpionActionItems_CL table

Updated workbook and analytics rule to use summarize arg_max(TimeGenerated, *) by id_s for proper deduplication. This prevents duplicate action items from appearing when the CCF RestApiPoller polls the same open items daily. - Workbook queries now deduplicate by id_s before aggregating - Analytics rule uses id_s deduplication instead of time bucket approach - Historical chart counts distinct id_s per day

ItaiMargalit · 2026-01-29T13:42:07Z

@ItaiMargalit please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]
Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"
Contributor License Agreement

@microsoft-github-policy-service agree company="IONIX"

Removed the unused 'restApiPollerName' variable from mainTemplate.json to fix the "Variables Must Be Referenced" ARM-TTK test failure. Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

v-maheshbh · 2026-02-02T11:56:42Z

Hi @ItaiMargalit

Kindly refer to the below-mentioned solution for the correct folder structure and update the necessary changes.
https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/Cloudflare%20CCF

Thanks!

…nector (v4.0.0) - Add new CCF data connector using Codeless Connector Framework with RestApiPoller - Automatic daily polling from IONIX API with pagination support - DCR transform maps API response fields (including nested title/urgency objects) - Keep old push connector marked as [DEPRECATED] for backward compatibility - Update Solution_IONIX.json with CCF connector file references - Update createUiDefinition.json to reflect 2 data connectors - Add ReleaseNotes entry for v4.0.0

Update stream declaration types and transformKql in IONIX_DCR.json to match the deployed mainTemplate: title/urgency as dynamic for nested JSON objects, asset_type as string, datetime fields as string, and bracket syntax for column references in transformKql.

v-maheshbh · 2026-02-16T09:19:48Z

Hi @ItaiMargalit
The package folder does not contain the ZIP file. Kindly package this solution using V3 tool .
If the current version is 3.0.0, then the next version should be 3.0.1. If additional features or significant changes are included, you may increment it to 3.1.0 accordingly.

https://github.com/Azure/Azure-Sentinel/blob/master/Tools/Create-Azure-Sentinel-Solution/V3/README.md.

Thanks!

- Downgrade version from 4.0.0 to 3.1.0 per reviewer request - Regenerate mainTemplate.json and createUiDefinition.json using V3 packaging tool - Add 3.1.0.zip package and testParameters.json - Fix CCF files: wrap Table in array, remove dependsOn:null from ConnectorDefinition - Fix DCR transform: add missing is_open_b column (queries depend on it) - Fix Solution_IONIX.json: only list ConnectorDefinition (tool auto-discovers rest) - Add DataConnectorCCFVersion, lastPublishDate, Author email - Fix deprecation notice title to match actual CCF connector name - Update ReleaseNotes.md version to 3.1.0

- Fix empty label/text nulls in createUiDefinition.json workbook section - Add default value for workbook1-name parameter - Fix "Sentinel" → "Microsoft Sentinel" branding across all files

v-maheshbh · 2026-02-18T12:06:30Z

Hi @ItaiMargalit

The following error was encountered while deploying the main template: “BadRequestException – properties.contentId is required.
kindly check

Thanks!

The V3 packaging tool (commit e694e4e) blanked out workbookContentId1 and workbookVersion1 variables, causing ARM deployment to fail with "BadRequestException - properties.contentId is required".

v-maheshbh · 2026-02-18T12:42:28Z

Hi @ItaiMargalit

Manual updates to the main template are not recommended. Please verify why these changes were not added during the solution packaging process.

Thanks!

Regenerated mainTemplate.json and createUiDefinition.json using the V3 packaging tool (createSolutionV3.ps1) instead of manual edits. The tool reads the standalone CCF connector files, workbook, and analytic rule from the Solution_IONIX.json manifest and produces the ARM template. Fixes: "Manual updates to the main template are not recommended"

v-maheshbh · 2026-02-24T09:22:10Z

Hi @ItaiMargalit

Kindly attach a screenshot of the CCF data connector showing its connected state for reference and include analytics rule , workbook creation.

Thanks!

ItaiMargalit · 2026-02-25T16:41:42Z

Hi @ItaiMargalit

Kindly attach a screenshot of the CCF data connector showing its connected state for reference and include analytics rule , workbook creation.

Thanks!

Hi @v-maheshbh
Thank you for the review. Here are the requested screenshots from a fresh deployment to a test Sentinel workspace.

CCF Data Connector - Connected State
The CCF connector (IONIX Security Logs (via Codeless Connector Framework)) successfully connects and polls data from the IONIX API using RestApiPoller.

Analytics Rule

(Note: The analytics rule shows its data source as [DEPRECATED] IONIX Security Logs (Push) — this is expected. The requiredDataConnectors metadata field references the original connector ID
(CyberpionSecurityLogs), but the rule's actual KQL query runs against the CyberpionActionItems_CL table directly, which both connectors write to. The rule fires correctly regardless of which
connector ingested the data. This is the same pattern used by other dual-connector solutions (e.g. Auth0).)

Workbook

The workbook queries all use summarize arg_max(TimeGenerated, *) by id_s for deduplication, ensuring correct results even if both connectors are active simultaneously.

Thanks!

Fixes two deploy-time bugs in v3.1.0 (PR Azure#13524). Bug Azure#1 — Paging: Switched pagingType from "NextPageUrl" to "Offset", matching the IONIX API's standard DRF LimitOffsetPagination shape. The old config failed CCF connectivity validation with "Missing NextPageParaName from config" (RES40002) for every customer on 3.1.0. Bug Azure#3 — Table declaration: The original v3.1.1 attempt removed IONIX_Table.json on the theory that Log Analytics would auto-create the custom table on first DCR write. That's not how it works — Sentinel's DCR validator requires the custom output table to already exist, and Connect failed with: InvalidOutputTable: Table for output stream 'Custom-CyberpionActionItems_CL' is not available for destination 'IONIX-Destination'. Restored IONIX_Table.json with the 12-column declaration (same shape as v3.1.0) so the table is created when the solution is installed via Content Hub (which auto-executes the contentTemplates nested mainTemplate). This matches the pattern used by every other CCF solution in the catalog (Auth0, Cloudflare CCF, SAP BTP, CyrenThreatIntelligence — all have the table declared inside the contentTemplates wrapper, none at top level). Package regenerated via Tools/Create-Azure-Sentinel-Solution/V3 (createSolutionV3.ps1) — mainTemplate.json + 3.1.1.zip are canonical V3-tool output. Verified end-to-end on a fresh Sentinel workspace: - Solution installs via Content Hub - CyberpionActionItems_CL table created - CCF connector connects (green), no RES40002 - First poll ingested 64 deduped action items via Offset paging - Workbook + analytics rule template both render Note: customers migrating from the deprecated push connector who already have a populated CyberpionActionItems_CL table with the legacy 15-18 column set will still hit "Invalid output table schema" on this version (same behavior as v3.1.0). Migration handling is tracked separately for v3.1.2 — recommended approach is a new table name for the CCF connector with a union shim during the transition period.

itai.margalit added 2 commits January 29, 2026 12:04

ItaiMargalit requested review from a team as code owners January 29, 2026 13:40

v-atulyadav assigned v-maheshbh Jan 30, 2026

v-atulyadav added the Solution Solution specialty review needed label Jan 30, 2026

Fix ARM-TTK test: Remove unused restApiPollerName variable

05e4744

Removed the unused 'restApiPollerName' variable from mainTemplate.json to fix the "Variables Must Be Referenced" ARM-TTK test failure. Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

itai.margalit added 2 commits February 10, 2026 14:15

ItaiMargalit changed the title ~~[IONIX] Migrate to Codeless Connector Framework (CCF) v4.0.0~~ [IONIX] Migrate to Codeless Connector Framework (CCF) Feb 16, 2026

itai.margalit added 2 commits February 16, 2026 12:03

[IONIX] Fix ARM-TTK and branding validation failures

2c101b6

- Fix empty label/text nulls in createUiDefinition.json workbook section - Add default value for workbook1-name parameter - Fix "Sentinel" → "Microsoft Sentinel" branding across all files

v-maheshbh added the Codeless Connector Framework (CCF) Connector label Feb 18, 2026

[IONIX] Fix deployment error: restore workbook contentId and version

eae1025

The V3 packaging tool (commit e694e4e) blanked out workbookContentId1 and workbookVersion1 variables, causing ARM deployment to fail with "BadRequestException - properties.contentId is required".

v-maheshbh approved these changes Feb 27, 2026

View reviewed changes

v-atulyadav merged commit 4a8fe51 into Azure:master Feb 27, 2026
36 checks passed

ItaiMargalit mentioned this pull request Apr 23, 2026

[IONIX] v3.1.1 — fix CCF paging config and remove table pre-declaration #14122

Open

ItaiMargalit mentioned this pull request Jun 9, 2026

[IONIX] v3.1.1 — fix CCF paging config and remove table pre-declaration #14438

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[IONIX] Migrate to Codeless Connector Framework (CCF) #13524

[IONIX] Migrate to Codeless Connector Framework (CCF) #13524
v-atulyadav merged 9 commits into
Azure:masterfrom
ItaiMargalit:feature/ionix-ccf-migration

ItaiMargalit commented Jan 29, 2026 •

edited

Loading

Uh oh!

ItaiMargalit commented Jan 29, 2026

Uh oh!

v-maheshbh commented Feb 2, 2026

Uh oh!

v-maheshbh commented Feb 16, 2026

Uh oh!

v-maheshbh commented Feb 18, 2026

Uh oh!

v-maheshbh commented Feb 18, 2026

Uh oh!

v-maheshbh commented Feb 24, 2026

Uh oh!

ItaiMargalit commented Feb 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ItaiMargalit commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Uh oh!

ItaiMargalit commented Jan 29, 2026

Uh oh!

v-maheshbh commented Feb 2, 2026

Uh oh!

v-maheshbh commented Feb 16, 2026

Uh oh!

v-maheshbh commented Feb 18, 2026

Uh oh!

v-maheshbh commented Feb 18, 2026

Uh oh!

v-maheshbh commented Feb 24, 2026

Uh oh!

ItaiMargalit commented Feb 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ItaiMargalit commented Jan 29, 2026 •

edited

Loading