Nimtable: Open-source web UI to browse and manage Apache Iceberg tables

原始链接: https://github.com/nimtable/nimtable

Nimtable is a user-friendly platform designed to simplify the management and optimization of Iceberg-based lakehouses. It offers a web interface for exploring tables, running queries, analyzing data distribution, and optimizing storage layouts. It supports multiple catalog types including REST, AWS Glue, AWS S3 Tables, and PostgreSQL (JDBC), seamlessly integrating with S3 and S3-compatible object stores. Nimtable provides features like interactive querying, AI-powered table summarization and exploration assistance, and easy file compaction and snapshot management. It can also act as a standard Iceberg REST Catalog. Configuration can be done via the web UI or a YAML file. Security is addressed by urging users to change the default admin password. Future development focuses on optimized compaction, comprehensive monitoring and analytics, caching, enhanced metadata management, robust security features, data lineage tracking, and deeper integration with AI and various storage backends. Docker deployment offers quick setup.

Here's a concise summary of the Hacker News thread about Nimtable: Nimtable, an open-source web UI for managing Apache Iceberg tables, is gaining attention. A user, tlarkworthy, shared their experience migrating from RDS to Iceberg, noting significant cost savings. While acknowledging challenges like write contention and the need for external vacuums, they highlight the potential of universal compatibility with query engines like DuckDB and ClickHouse for specialized queries over a generic storage layer. This would unlock the true potential of tools like Nimtable. Others noted confusion with the programming language "Nim", and the suggestion of "Icetable" as a more suitable name. Bradhe appreciates the project, citing the lack of basic tools for exploring Iceberg tables and anticipates that table maintenance will need additional support. konsalexee would like an interactive demo. The conversation reflects both excitement about the potential of Iceberg and the ongoing development of its ecosystem.

原文

A lightweight, easy-to-use platform to monitor, optimize, and govern your Iceberg-based lakehouse.

Nimtable helps you easily manage and explore Apache Iceberg catalogs. With a web-based platform designed for clarity and simplicity, Nimtable makes it easy to browse tables, run queries, analyze file distributions, and optimize storage layouts.

🌟 Multi-Catalog Support
Connect to REST Catalog, AWS Glue, AWS S3 Tables, and PostgreSQL (via JDBC).
🗄️ Object Store Integration
Seamlessly work with S3 and S3-compatible stores like Cloudflare R2, Minio, and more.
🔍 Table Exploration
Inspect table schemas, partitions, and snapshots with ease.
⚡ Interactive Querying
Run SQL queries directly from the platform.
🤖 AI Copilot
Get intelligent assistance for Iceberg table exploration.
📄 AI Summary
Automatically generate summaries of your Iceberg tables.
📊 File Distribution Analysis
Visualize how data files are distributed across partitions and snapshots.
🔧 Table Optimization
Run file compaction and manage snapshot expiration.
🔌 REST Catalog Compatibility
Serve as a standard Iceberg REST Catalog, adapting any underlying catalog to a RESTful API.

Nimtable acts as a bridge between users and catalog servers, providing both an interactive web interface and a standard REST Catalog API layer.

The fastest way to get started is using Docker:

cd docker
docker compose up -d

Access the UI at http://localhost:3000.

Username: admin
Password: admin

Managing the Service (Optional)

View logs:
Stop the service:

See HACKING.md for details on how to hack on Nimtable.

Nimtable can be configured in two ways:

Web UI: Easiest for new users - just log in and click "Create Catalog."
YAML Configuration File: Recommended for advanced users or automated deployments.

1. Configuration File Location

By default, Nimtable looks for config.yaml in the working directory.
Docker: Mount your config file to /app/config.yaml inside the container.
See docker/docker-compose.yml for an example of mounting configuration.

2. Minimal Configuration Example

server:
  port: 8182
  host: 0.0.0.0
admin:
  username: admin
  password: admin
database:
  url: jdbc:postgresql://localhost:5432/nimtable_db
  username: nimtable_user
  password: password

Important: Change the default admin password after your first login for security.

You can add catalogs in two ways:

Web UI:
After logging in, click "Create Catalog" and follow the prompts. Catalogs added via the UI are stored in the internal database and do not modify config.yaml.
YAML File:
Pre-configure catalogs by adding them to your config.yaml.
See backend/config.yaml for full examples and templates.

Supported Catalog Types:

REST
AWS Glue
S3 Tables
PostgreSQL (via JDBC)

Each catalog type may require specific fields. Refer to the sample config for details.

4. AWS Credential Configuration

If you use AWS Glue or S3, you can provide credentials in two ways:

🔧 Optimized Compaction: Advanced compaction strategies and scheduling
📊 Monitoring & Analytics: Comprehensive dashboard and insights
💾 Caching: Database integration and metadata caching
⚡ Query Engine Integration: Support for multiple query engines
📋 Metadata Management: Enhanced snapshot, schema and partition management
🔐 Security & Access Control: RBAC and fine-grained permissions
🔌 API & Integration: REST API support and authentication
🔄 Data Lineage: Table and column-level lineage tracking
🤖 Better AI Copilot Support: Enhanced capabilities for AI agent.
🏢 Catalog & Warehouse Integration: Support for various storage backends

For detailed roadmap items and progress tracking, see Roadmap.

We welcome contributions! Please see our Contributing Guide for details.

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.