Nimtable Roadmap

1. Optimized Compaction

• (high priority) Embedded Spark compaction
• (high priority) Compaction recommendations
• (mid priority) Compaction service based on iceberg-rs and DataFusion
• (mid priority) Automated compaction scheduling
• (low priority) Overall scheduled compaction jobs dashboard

2. Monitoring & Analytics

• (high priority) Table detail page
• (high priority) Namespace detail page
• (high priority) Catalog-wide dashboard: statistics and metrics
• (high priority) Storage utilization insights
• (mid priority) Storage cost analysis
• (mid priority) Table growth analytics
• (mid priority) Data freshness monitoring
• (mid priority) Partition distribution insights
• (low priority) Table health indicators

3. Caching

• (high priority) Database integration for metadata caching to improve user experience
• (mid priority) Reduced load on catalog services
• (mid priority) Metadata synchronization & cache invalidation
• (low priority) Support different database backend

4. Query Engine Integration

• (mid priority) Embedded Spark Query
• (mid priority) Multi-engine support
• (mid priority) DuckDB integration for lightweight queries
• (mid priority) RisingWave integration
• (mid priority) Trino support for distributed queries
• (low priority) Engine-specific optimizations

5. Meta data Management

• (high priority) show current schema
• (high priority) show current partition
• (high priority) show current snapshot
• (high priority) show table properties
• (high priority) show manifest #70
• (high priority) snapshot history tracking #71
• (high priority) Schema history tracking
• (mid priority) Visual schema diff tool
• (mid priority) Table creation wizard
• (mid priority) Schema management wizard
• (mid priority) Partition management wizard
• (low priority) branch & tag management
• (low priority) Enhanced procedure operations
• (mid priority) Time travel #73

6. Security & Access Control

• (high priority) Configured based login
• (mid priority) User management (RBAC)
• (mid priority) Fine-grained permissions at catalog/namespace/table level
• (low priority) Audit logging for user actions

7. API & integration

• (high priority) REST API (read-only)
• (mid priority) REST API (write)
• (low priority) authentication

8. Data Lineage

• (mid priority) Table and View lineage
• (low priority) column-level lineage

9. Catalog & Warehouse integration

• (high priority) AWS S3
• (high priority) AWS Glue
• (high priority) AWS S3 tables
• (high priority) Rest catalog
• (high priority) JDBC postgres
• (mid priority) JDBC mysql
• (mid priority) Hive
• (mid priority) GCS
• (mid priority) Azure
• (low priority) Storage catalog

10. Deployment

• (high priority) Docker Compose
• (low priority) K8s

11. Catalog Configuration

• (high priority) YAML file based spark compatible catalog configuration
• (high priority) catalog configuration UI #72