This issue covers all the tasks needed to backport the implementation of ADR-101 from main into v0.38.x.

We will backport this feature into an experimental branch because :

• We have to break existing go API interfaces and v0.37.x is already released
• The pruning mechanism is happening in the background now, instead of synchronously between heights which changes the existing logic.

The ADR will be backported step by step, porting one big feature at a time.

• Create a v0.37.x-experimental branch with all the protections of release branches.
• Port BaseService and VersionService to v0.37.x (PR grpc: Add base gRPC server with version service #818 ): grpc: Add base gRPC server with version service to v0.37.x-experimental #1438
  This step will be the basis for backporting all other gRPC endpoints.
• ADR-101 v0.37.x: Backport block pruning mechanism  #1444
  commits: 04dfaf5, c710bb8 (metrics for pruning), fe644d8
  commit 6fc2c37 depends on the gRPC endpoints so will be done at the end
• Backport indexer pruning mechanism.
  commits: 96f4ab8
• Backport gRPC endpoints:
  • BlockService backport to v0.37.x: commits: 84e2e01, 8006a74 grpc: BlockService backport to v0.37.x-experimental #1441
  • grpc: backport BlockResultsService to v0.37.x-experimental #1442
    Not trivial, due to changes on ABCI interface.
  • PruningService (Block) backport to v0.37.x commits: 69843be
  • Indexer pruning service backport to v0.37.x commits: 4f0d5b4
  • Close method for gRPC client commits 3479642
• Metrics for indexer pruning service monitoring backport to v0.37.x
  commits: 08cf31c
• Backport fixes and optimizations
  commits: 6fc2c37 (depends on the gRPC endpoints so will be done at the end )
• ADR-101 Backport documentation to v0.37.x
  commits: 46a4cea

Fixes related to pruning done in main and on v0.38.x:

• ADR 101: Refactor height check-related logic and tests #1271
• Do not race with pruning to validate headers and validators #1467
• state/pruner: Notify observer only when retain height changes #1490
• e2e:Fix docker port alias #1507
• e2e: Fix flakiness in grpc tests due to pruning #1492

Some of those (especially the e2e related ones) might not be needed in 0.37.x but that needs to be confirmed in case of a backport of ADR 101 to 0.37.x-experimental

Backporting the pruning mechanisms is independent of the gRPC backports and can be done in parallel. The pruning service endpoints depend on the pruning mechanisms being implemented.