Skip to content

apps: Limit the number of deployments listed when polling.#865

Merged
andrewsomething merged 2 commits intomainfrom
asb/issues/864
Aug 16, 2022
Merged

apps: Limit the number of deployments listed when polling.#865
andrewsomething merged 2 commits intomainfrom
asb/issues/864

Conversation

@andrewsomething
Copy link
Copy Markdown
Member

This works around an issue with the App Platform API where the response body can be too large. We are only actually using the first deployment from the list, so we can use the default page size of 20 rather than 200 to prevent this. I've also removed the loop to prevent iterating through all deployments and using up API request quotas we only need the first one.

Fixes: #864

@andrewsomething andrewsomething requested a review from a team August 16, 2022 14:48
@andrewsomething andrewsomething temporarily deployed to Acceptance Tests August 16, 2022 14:50 Inactive
@andrewsomething andrewsomething temporarily deployed to Acceptance Tests August 16, 2022 15:20 Inactive
@andrewsomething andrewsomething merged commit 2cabf2b into main Aug 16, 2022
@andrewsomething andrewsomething deleted the asb/issues/864 branch August 16, 2022 15:20
@ddelange
Copy link
Copy Markdown

ddelange commented Nov 11, 2025

Hi @andrewsomething 👋

we are experiencing 524 errors on /deployments?per_page=20 (support ticket)

we have a lot of deployments in production, mainly autoscaling events. could the provider be refactored to avoid the GET /deployments call altogether?

2025-11-10T20:02:24.092Z [INFO]  provider.terraform-provider-digitalocean_v2.67.0: 2025/11/10 20:02:24 [DEBUG] PUT https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035: timestamp=2025-11-10T20:02:24.091Z
2025-11-10T20:02:29.308Z [INFO]  provider.terraform-provider-digitalocean_v2.67.0: 2025/11/10 20:02:29 [DEBUG] Waiting for app (b7d31d4f-673b-4e9d-b439-a831f24b5035) deployment to become active: timestamp=2025-11-10T20:02:29.307Z
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 00m10s elapsed]
2025-11-10T20:02:39.314Z [INFO]  provider.terraform-provider-digitalocean_v2.67.0: 2025/11/10 20:02:39 [DEBUG] GET https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=20: timestamp=2025-11-10T20:02:39.314Z
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 00m20s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 00m30s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 00m40s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 00m50s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 01m00s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 01m10s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 03m50s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 04m00s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 04m10s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 04m20s elapsed]
2025-11-10T20:14:42.224Z [INFO]  provider.terraform-provider-digitalocean_v2.67.0: 2025/11/10 20:14:42 [DEBUG] GET https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=20 (status: 524): retrying in 2s (3 left): timestamp=2025-11-10T20:14:42.224Z
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 04m30s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 04m40s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 04m50s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 05m00s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 05m10s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 05m20s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 05m30s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 05m40s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 05m50s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 06m00s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 06m10s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 06m20s elapsed]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 06m30s elapsed]
2025-11-10T20:16:49.412Z [INFO]  provider.terraform-provider-digitalocean_v2.67.0: 2025/11/10 20:16:49 [DEBUG] GET https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=20 (status: 524): retrying in 4s (2 left): timestamp=2025-11-10T20:16:49.411Z
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 06m40s elapsed]
[... the pattern repeats until it fails]
digitalocean_app.main: Still modifying... [id=b7d31d4f-673b-4e9d-b439-a831f24b5035, 10m50s elapsed]
Error: -10T20:21:13.134Z [ERROR] vertex "digitalocean_app.main" error: Error trying to read app deployment state: GET https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=20: 524 <!DOCTYPE html>
  <html>
  <head>
      <title>DigitalOcean - Something went wrong!</title>
  ...

@ddelange
Copy link
Copy Markdown

a little debugging on our production:

?per_page=2

$ time curl -X GET \
>   -H "Content-Type: application/json" \
>   -H "Authorization: Bearer ${DIGITALOCEAN_TOKEN}" \
>   "https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=2"
{"meta":{"total":42554},"links":{"pages":{"last":"https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?page=21277\u0026per_page=2","next":"https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?page=2\u0026per_page=2"}},"deployments":[{"id":"477748d6-978c-434a-81ae-5852df088c45","spec":{"name":"production","services":[{"name":...
real	0m39.485s
user	0m0.024s
sys	0m0.020s

?per_page=20&deployment_types=DEPLOY_ON_PUSH

$ time curl -X GET \
>   -H "Content-Type: application/json" \
>   -H "Authorization: Bearer ${DIGITALOCEAN_TOKEN}" \
>   "https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=20&deployment_types=DEPLOY_ON_PUSH"
{"meta":{"total":0}}
real	0m38.998s
user	0m0.023s
sys	0m0.014s

?per_page=2&deployment_types=MANUAL

$ time curl -X GET \
>   -H "Content-Type: application/json" \
>   -H "Authorization: Bearer ${DIGITALOCEAN_TOKEN}" \
>   "https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=2&deployment_types=MANUAL"
{"meta":{"total":862},"links":{"pages":{"last":"https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?deployment_types=MANUAL\u0026page=431\u0026per_page=2","next":"https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?deployment_types=MANUAL\u0026page=2\u0026per_page=2"}},"deployments":[{"id":"d8600cf5-9beb-4ebb-82e6-8cf1034a5ff9","spec":{"name":"production","services":[{"name":...
real	0m39.748s
user	0m0.019s
sys	0m0.019s

?per_page=20&deployment_types=MANUAL

$ time curl -X GET \
>   -H "Content-Type: application/json" \
>   -H "Authorization: Bearer ${DIGITALOCEAN_TOKEN}" \
>   "https://api.digitalocean.com/v2/apps/b7d31d4f-673b-4e9d-b439-a831f24b5035/deployments?per_page=20&deployment_types=MANUAL"
error code: 524
real	2m5.467s
user	0m0.028s
sys	0m0.025s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

terraform apply fails if digitalocean-api response gets to large

3 participants