Skip to content

Instances' sled assignments won't change if an instance is stopped and restarted #2315

@gjcolombo

Description

@gjcolombo

The Nexus external API's "instance start" command passes through to Nexus::instance_start_runtime:

let instance = nexus.instance_start(&opctx, &instance_lookup).await?;

/// Make sure the given Instance is running.
pub async fn instance_start(
&self,
opctx: &OpContext,
instance_lookup: &lookup::Instance<'_>,
) -> UpdateResult<db::model::Instance> {
let (.., authz_instance, db_instance) = instance_lookup.fetch().await?;
let requested = InstanceRuntimeStateRequested {
run_state: InstanceStateRequested::Running,
migration_params: None,
};
self.instance_set_runtime(
opctx,
&authz_instance,
&db_instance,
requested,
)
.await?;
self.db_datastore.instance_refetch(opctx, &authz_instance).await
}

If I'm reading things right, this function selects the sled to which to send the runtime state update by looking at the instance record in CRDB without regard for the instance's current state:

let sa = self.instance_sled(&db_instance).await?;
let instance_put_result = sa
.instance_put(
&db_instance.id(),
&sled_agent_client::types::InstanceEnsureBody {
initial: instance_hardware,
target: requested.clone(),
migrate: None,
},
)
.await;

This seems like the right thing to do if the instance is already incarnated on a sled somewhere. But if the instance is stopped and doesn't exist on any sled, this will try to create the instance on the sled on which it most recently ran, which might not have capacity for it (even though some other sled might). This function should distinguish the "instance already incarnated" and "instance stopped" cases and select a new sled in the latter case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    nexusRelated to nexus

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions