Subtasks:

• Status Service RFC ([skip-ci] Service Status RFC #59621)
• Add basic status service scaffolding, expose CoreStatus API to plugins (Add basic StatusService #60335)
• Add compatibility with legacy plugins statuses
• Expose overall status to plugins (required for Monitoring)
• Add support for plugin derived statuses
• Migrate status API endpoint
• Migrate status page UI Migrate status page app to core #72017
• Display the status page when kibana is loading (see Migrate status page app to core #72017 (comment))
• Have the status_page route return a 503 when kibana is not ready Have the status_page route return a 503 when the kibana status is red #72831
• Add default timeout to custom status Observables Add timeouts and setup enforcement for custom plugins statuses #77965
• Add automatic 503 HTTP error behaviors
• Add unavailableWhen HTTP route utility from RFC
• Fix flaky tests
  • Failing test: Firefox XPack UI Functional Tests.x-pack/test/functional/apps/status_page/status_page·ts - Status page Status Page allows user to navigate without authentication #50448
  • [test-failed]: X-Pack API Integration Tests1.x-pack/test/api_integration/apis/kibana/stats/stats·js - apis kibana stats /api/stats operational stats and usage stats no auth should return 200 and stats for no extended #51491

Original issue content

Status service

from lifecycle rfc:
Core should expose a global mechanism for core services and plugins to signal their status. This is equivalent to the legacy status API kibana.Plugin.status which allowed plugins to set their status to e.g. 'red' or 'green'. The exact design of this API is outside of the scope of this RFC.

What is important, is that there is a global mechanism to signal status changes which Core then makes visible to system administrators in the Kibana logs and the /status HTTP API. Plugins should be able to inspect and subscribe to status changes from any of their dependencies.

This will provide an obvious mechanism for plugins to signal that the conditions which are required for this plugin to operate are not currently present and manual intervention might be required. Status changes can happen in both setup and start lifecycles e.g.:

[setup] a required remote host is down
[start] a remote host which was up during setup, started returning connection timeout errors.

Status API and page

Kibana currently exposes a api/status endpoint and associated status_page which renders this output:

name: config.get('server.name'),
uuid: config.get('server.uuid'),
version: {
  number: config.get('pkg.version').replace(matchSnapshot, ''),
  build_hash: config.get('pkg.buildSha'),
  build_number: config.get('pkg.buildNum'),
  build_snapshot: matchSnapshot.test(config.get('pkg.version'))
},
status: kbnServer.status.toJSON(), // https://github.com/elastic/kibana/blob/2a290a14066d4da2b626bb0b4e4e9d0193853230/src/legacy/server/status/server_status.js#L111
metrics: kbnServer.metrics // https://github.com/elastic/kibana/issues/46563 https://github.com/elastic/kibana/blob/ec481861799ed8dcced9cafd8112e5b26e641c54/src/legacy/server/status/lib/metrics.js#L57-L68

The status page app is rendered as a hiddenUiApp

The api/stats endpoint is currently created in the same legacy plugin. It won't be migrated to Core, but depends on an equivalent to kbnServer.metrics being exposed from Core.