Skip to content

Dual stack ephemeral IP#9714

Merged
david-crespo merged 10 commits into
mainfrom
dual-stack-ephemeral-ip
Jan 24, 2026
Merged

Dual stack ephemeral IP#9714
david-crespo merged 10 commits into
mainfrom
dual-stack-ephemeral-ip

Conversation

@david-crespo

@david-crespo david-crespo commented Jan 23, 2026

Copy link
Copy Markdown
Contributor

Closes #9665

Had the robots work on this while I got ready for bed.

API schema diff

Notably this did not require changes to the instance create API, only detach.

--- a/2026012200.0.0-13b2b3/spec.json
+++ b/2026012300.0.0-151f11/spec.json
@@ -7,7 +7,7 @@
       "url": "https://oxide.computer",
       "email": "api@oxide.computer"
     },
-    "version": "2026012200.0.0"
+    "version": "2026012300.0.0"
   },
   "paths": {
     "/device/auth": {
@@ -4602,6 +4602,7 @@
           "instances"
         ],
         "summary": "Detach and deallocate ephemeral IP from instance",
+        "description": "When an instance has both IPv4 and IPv6 ephemeral IPs, the `ip_version` query parameter must be specified to identify which IP to detach.",
         "operationId": "instance_ephemeral_ip_detach",
         "parameters": [
           {
@@ -4615,6 +4616,14 @@
           },
           {
             "in": "query",
+            "name": "ip_version",
+            "description": "The IP version of the ephemeral IP to detach.\n\nRequired when the instance has both IPv4 and IPv6 ephemeral IPs. If only one ephemeral IP is attached, this field may be omitted.",
+            "schema": {
+              "$ref": "#/components/schemas/IpVersion"
+            }
+          },
+          {
+            "in": "query",
             "name": "project",
             "description": "Name or ID of the project",
             "schema": {

Comment thread nexus/src/app/instance.rs
}

Ok(())
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would have felt nuts to type all this out by hand but it seems great for the user!

@david-crespo david-crespo force-pushed the dual-stack-ephemeral-ip branch from c7c619c to 05dab02 Compare January 23, 2026 16:11
@david-crespo david-crespo marked this pull request as ready for review January 23, 2026 16:12
@david-crespo david-crespo requested a review from bnaecker January 23, 2026 16:12
Comment thread nexus/src/app/instance.rs Outdated
return Err(Error::invalid_request(format!(
"two ephemeral IPs would both be {pool_version}: \
auto selects default pool, which is {pool_version}"
)));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having to do these lookups here and then again during the actual allocation feels like the worst part of this change. I'm looking into what it would take to avoid this by, e.g., resolving the pool here and passing the result into the saga.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all feels kind of susceptible to TOCTTOUs. I don't see a good way around that, with the way we're structuring the DB tables themselves though.

@david-crespo david-crespo Jan 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had Claude prototype moving the pool resolution out of the sagas (a not-small +400/-300 change), and it had to touch a lot of parts. It would not be easy to be confident the change is correct. I don't think it's worth doing — we do a bunch more queries inside the saga anyway.

@david-crespo

Copy link
Copy Markdown
Contributor Author

Oh also I need to do the whole API schema regen. I left it out so I could use jj diff like a normal person while working on this.

@bnaecker bnaecker left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me. A few vague questions, but shouldn't block integrating this. Thanks for taking this on.

Comment thread nexus/src/app/instance.rs Outdated
Comment thread nexus/src/app/instance.rs Outdated
return Err(Error::invalid_request(format!(
"two ephemeral IPs would both be {pool_version}: \
auto selects default pool, which is {pool_version}"
)));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all feels kind of susceptible to TOCTTOUs. I don't see a good way around that, with the way we're structuring the DB tables themselves though.

`instance_lookup_ephemeral_ip` conflated two concerns: looking up an
ephemeral IP by version (pure data retrieval) and validating that user
requests are unambiguous (error when multiple IPs exist and no version
specified). This validation only makes sense for user-facing operations
like detach, not for internal callers that already know which version
they want.

Changed the function to require an IP version, making it a pure lookup.
Move the "multiple ephemeral IPs" validation into the detach saga, the
only caller that needs it.

better error message for detach with ambiguous version
@david-crespo david-crespo force-pushed the dual-stack-ephemeral-ip branch from 1ee28cf to 2c2c30d Compare January 23, 2026 21:18
),
));
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was initially added to instance_lookup_ephemeral_ip, but that was confusing because that's a lookup function, plus this validation is only relevant when the IP version can be none, which only happens (among calls to instance_lookup_ephemeral_ip) in the detach path. So this logic only needs to happen here. The other paths (instance create and ephemeral IP attach) can also start out without an explicit IP version, but by the time they got to instance_lookup_ephemeral_ip, the IP version has already been resolved.

@david-crespo david-crespo enabled auto-merge (squash) January 23, 2026 21:50
@david-crespo david-crespo merged commit e2491dc into main Jan 24, 2026
16 checks passed
@david-crespo david-crespo deleted the dual-stack-ephemeral-ip branch January 24, 2026 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow instance to have both v4 and v6 ephemeral IP at create time

2 participants