Skip to content

fix: Handle NotFound error when syncing Subscriptions with deleted topic (aws_sns_subscriptions)#14771

Merged
kodiakhq[bot] merged 3 commits intocloudquery:mainfrom
AshCorr:ash/AwsSnsSubscriptions
Oct 20, 2023
Merged

fix: Handle NotFound error when syncing Subscriptions with deleted topic (aws_sns_subscriptions)#14771
kodiakhq[bot] merged 3 commits intocloudquery:mainfrom
AshCorr:ash/AwsSnsSubscriptions

Conversation

@AshCorr
Copy link
Copy Markdown
Contributor

@AshCorr AshCorr commented Oct 20, 2023

Summary

AWS allows users to delete a SNS Topic even if it has active subscribers. This causes a scenario where the user is unable to delete the hanging subscribers and apparently requires AWS support to manually intervene, see https://stackoverflow.com/a/64568018/8836650 EDIT: actually it appears you can delete hanging subscriptions now as long as they're confirmed, but its still doesn't seem possible to delete unconfirmed ones.

When CloudQuery encounters a Subscription that is in this state it throws an error:

11:40AM ERR pre resource resolver failed error="operation error SNS: GetSubscriptionAttributes, https response error StatusCode: 404, RequestID: (redacted), NotFound: Topic does not exist" client=(redacted) module=aws-src table=aws_sns_subscriptions

This causes CloudQuery to lose data by not syncing that subscription resource. This PR handles that error and allows users to join their aws_sns_subscriptions and aws_sns_topics table to find all hanging subscriptions in their orgs.

Sorry about the spam of fix PRs 😅

@AshCorr AshCorr changed the title fix: Handle NotFound error when syncing Subscription with deleted t… fix: Handle NotFound error when syncing Subscription with deleted topic (aws_sns_subscriptions) Oct 20, 2023
@AshCorr AshCorr marked this pull request as ready for review October 20, 2023 11:12
@AshCorr AshCorr force-pushed the ash/AwsSnsSubscriptions branch from 5513ecb to a5f7dcc Compare October 20, 2023 11:25
@erezrokah erezrokah added the automerge Automatically merge once required checks pass label Oct 20, 2023
@AshCorr AshCorr changed the title fix: Handle NotFound error when syncing Subscription with deleted topic (aws_sns_subscriptions) fix: Handle NotFound error when syncing Subscriptions with deleted topic (aws_sns_subscriptions) Oct 20, 2023
@kodiakhq kodiakhq bot merged commit 6fcf43d into cloudquery:main Oct 20, 2023
hydratim pushed a commit to hydratim/cloudquery that referenced this pull request Oct 20, 2023
…topic (`aws_sns_subscriptions`) (cloudquery#14771)



#### Summary

AWS allows users to delete a SNS Topic even if it has active subscribers. <del>This causes a scenario where the user is unable to delete the hanging subscribers and apparently requires AWS support to manually intervene, see https://stackoverflow.com/a/64568018/8836650</del> EDIT: actually it appears you can delete hanging subscriptions now as long as they're confirmed, but its still doesn't seem possible to delete unconfirmed ones.

When CloudQuery encounters a Subscription that is in this state it throws an error:

```bash
11:40AM ERR pre resource resolver failed error="operation error SNS: GetSubscriptionAttributes, https response error StatusCode: 404, RequestID: (redacted), NotFound: Topic does not exist" client=(redacted) module=aws-src table=aws_sns_subscriptions
```

This causes CloudQuery to lose data by not syncing that subscription resource. This PR handles that error and allows users to join their `aws_sns_subscriptions` and `aws_sns_topics` table to find all hanging subscriptions in their orgs.

_Sorry about the spam of fix PRs 😅_ 

<!--
Explain what problem this PR addresses
-->

<!--
kodiakhq bot pushed a commit that referenced this pull request Oct 23, 2023
🤖 I have created a release *beep* *boop*
---


## [22.16.0](plugins-source-aws-v22.15.2...plugins-source-aws-v22.16.0) (2023-10-23)


### This Release has the Following Changes to Tables
- Table `aws_efs_filesystems`: column added with name `file_system_policy` and type `utf8`

### Features

* Add `policy` column to `aws_efs_filesystems` table ([#14672](#14672)) ([833b9c2](833b9c2))


### Bug Fixes

* Changed the condition to check for policies in policies table ([#13935](#13935)) ([f136331](f136331))
* **deps:** Update github.com/cloudquery/arrow/go/v14 digest to f46436f ([#14803](#14803)) ([f5248d7](f5248d7))
* **deps:** Update module github.com/cloudquery/codegen to v0.3.10 ([#14773](#14773)) ([98f3e2c](98f3e2c))
* **deps:** Update module github.com/cloudquery/codegen to v0.3.11 ([#14870](#14870)) ([4fa917d](4fa917d))
* Don't call GetFunctionCodeSigningConfig or GetRuntimeManagementConfig on Lambdas packaged as Images (`aws_lambda_functions`) ([#14729](#14729)) ([6fc30d3](6fc30d3))
* Handle `NotFound` error when syncing Subscriptions with deleted topic (`aws_sns_subscriptions`) ([#14771](#14771)) ([6fcf43d](6fcf43d))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plugin/source/aws automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants