cluster_working_state always has two metrics, one with state=working and one state=stopped.
The active state exports the value 1.
If the working state is equal to "stopped", an optional error is provided with an empty string indicating no error.
To make the metrics consistent, an empty value for error is always provided, similarly to a non active state having the value 0.

I'm not sure if this is a conventional way of doing things? Maybe it's better to only report the current one?

cluster_last_update_delta

I suggest to rename this to cluster_last_update_seconds, or cluster_last_update_delta_seconds or cluster_last_update_age_seconds

https://prometheus.io/docs/practices/naming/

I'm not sure if this is a conventional way of doing things? Maybe it's better to only report the current one?

It probably is not a conventional way. It seemed to be more suitable for plotting to me.

Checking with ChatGPT, it seems to confirm my approach:

✅ 1. Is it conventional to represent states like this?

Yes — for multi-state enumerations.

Example (good):
    my_service_state{state="running"} 1
    my_service_state{state="stopped"} 0
    my_service_state{state="error"} 0

⚠️ 2. What about one time series that changes its state label value over time?

    This is **not recommended**.

    Prometheus treats each unique label set as a separate time series.
    If the value of the state label changes (e.g. xy → yz → qq), you are:

    - creating a new time series every time the state changes
    - causing churn, which is bad for performance
    - making long-term trend queries messy
    - losing the ability to track how long each state existed (unless you manually reconstruct it)

Prometheus best practices:
👉 Label values should not have high cardinality or change frequently.

An alternative could be to include the state in the metrics name:

cluster_working_state_working{} 1
cluster_working_state_stopped{error=""} 0

I know ChatGPT could be wrong here so please let me know if you want me to look deeper into the conventions here!

Edit: On a second thought, maybe we shouldn't include the error itself for the same reason. I assume the error, if exists, can be read somewhere else too.

Edit: On a second thought, maybe we shouldn't include the error itself for the same reason. I assume the error, if exists, can be read somewhere else too.

Yeah, that sounds reasonable!

I'm fine with that ChatGPT explanation.

Let's exclude the error message, but just denote the succesful and error states. That would be enough to detect 'an error'. And then a user is responsible for looking into the cluster to see the actual error.