Thanks for the release note entry! Nit in phrasing - suggest following the wording in the Order of Changes in the Release Notes Guidelines. Two suggestions follow, choose the one that best describes your work, or write one yourself.

== RELEASE NOTES ==

Iceberg Connector Changes
* Remove in-memory hive metastore cache in Iceberg connector. :pr:`24326`

or

== RELEASE NOTES ==

Iceberg Connector Changes
* Improve Iceberg connector by disabling in-memory hive metastore. :pr:`24326`

Can you help me understand why this is a problem in the Iceberg connector, but not Hive? Does hive have an additional check to ignore the cache TTL on external updates?

Since we store pointers to the metadata files in HMS for Iceberg, having a stale version can corrupt Iceberg tables and violate ACID properties. In contrast, this is not a concern with general Hive tables.

For example, in the case of Iceberg, if a new snapshot is created or a rollback is performed by an external engine, having stale metadata in Presto can lead to issues. Any new operation executed by Presto may corrupt the Iceberg table because Presto will not be operating on the latest version of the table.

I hope this explains the issue.

#24325

@imjalpreet I have a quick question: Even if the external engine is moving forward with the snapshot, it would also change the pointer for metadata location in HMS.
So, in this case, if we need to access the latest snapshot, shouldn't it wait for the cache to expire or until we have an explicit procedure to invalidate the cache like other connectors like Hive?

@agrawalreetika You’re right that when an external engine commits a new snapshot, it updates the metadata pointer in HMS. However, there’s a critical difference between Iceberg and Hive connectors. Iceberg uses a metadata pointer in HMS to track the current snapshot, ensuring strict ACID guarantees. Every query must operate on the latest snapshot to maintain atomicity and consistency. With metastore caching enabled, Presto might use an outdated pointer, querying stale snapshots, and violating Iceberg’s guarantees. This can result in partial data visibility or conflicting and incorrect query results. The example I provided yesterday is one such scenario that can lead to a corrupted table.

Whereas, Hive doesn’t depend on snapshot isolation, so stale cache leads to temporary inconsistencies, not contract violations. Even if operations are performed on the data while the cache is stale, queries will eventually see the updated partitions or data after the cache expires.

@imjalpreet Should we also put a note of this in documentation as well?
https://prestodb.io/docs/current/connector/iceberg.html#metastore-cache

@ZacBlanco Hi Zac, we remember that the perf runs you did earlier this year show Iceberg and Hive performance gap was closed to be within 8%. Was that before or after this PR was merged? We just observed big regression in Iceberg query planning time caused by IcebergHiveMetadata.getTableStatistics() was not able to retrieve table stats from the in memory cache. Did you see that before?