Overview

Relevant source files

• CHANGES.txt
• LICENSE.txt
• README.md
• pom.xml
• src/main/java/com/treasuredata/client/TDClientBuilder.java
• src/main/java/com/treasuredata/client/model/TDQuery.java
• src/test/java/com/treasuredata/client/Example.java

Purpose and Scope

This document provides a high-level overview of td-client-java, a Java client library for interacting with the Treasure Data REST API. The library enables Java applications to programmatically submit queries, manage databases and tables, execute jobs, import/export data, and monitor job execution on the Treasure Data platform.

This page covers the library's purpose, architecture, and main capabilities. For specific implementation details, refer to:

• Configuration and setup: Getting Started, Configuration
• API operations: Client API
• HTTP communication: HTTP Communication Layer
• Error handling: Error Handling and Resilience
• Data structures: Data Models

Sources: README.md1-14 pom.xml1-10

---

Library Purpose

td-client-java is the official Java client for Treasure Data's cloud-based data platform. It abstracts the complexity of REST API calls into a type-safe, fluent Java API, handling authentication, retries, error handling, and response parsing automatically.

Primary Use Cases:

• Submit and monitor Presto, Trino, Hive, and Pig queries
• Manage databases, tables, and schemas
• Bulk import large datasets via multi-part uploads
• Schedule and execute saved queries
• Export query results to external storage systems
• Retrieve job results in multiple formats (MessagePack, JSON, CSV)

Sources: README.md3-10 Example.java56-346

---

System Requirements

Requirement	Specification
Java Version	1.8 or higher
Build System	Maven 3.x
License	Apache License 2.0
Current Version	1.2.0
Maven Coordinates	com.treasuredata.client:td-client

The library is available as both a regular JAR with transitive dependencies and a standalone "shade" JAR with all dependencies bundled under a relocated package namespace to prevent conflicts.

Sources: pom.xml1-60 README.md12-13 LICENSE.txt1-203

---

High-Level Architecture

The library follows a layered architecture with clear separation of concerns:

Architecture Description

The architecture consists of four primary layers:

1. Public API Layer: TDClient serves as the main entry point, providing ~30 high-level methods. TDClientBuilder handles configuration from multiple sources (environment variables, system properties, td.conf file, programmatic settings).

2. HTTP Communication Layer: TDHttpClient manages all HTTP operations, wrapping OkHttp3. It delegates request preparation to TDHttpRequestHandler, error categorization to TDRequestErrorHandler, and uses pluggable BackOff strategies for retry logic.

3. Infrastructure Layer: Immutable configuration objects (TDClientConfig, ProxyConfig) ensure thread-safety. Retry strategies implement exponential backoff with jitter to prevent thundering herd problems.

4. Dependencies: OkHttp3 handles HTTP communication with connection pooling and timeout management. Jackson handles JSON serialization/deserialization with custom annotations and type adapters.

Sources: TDClient.java1-100 TDClientBuilder.java1-51 pom.xml98-173

---

Core Components

Primary Classes and Their Roles

Class	Package	Role
TDClient	com.treasuredata.client	Main API facade; provides all public methods for database, table, job, and query operations
TDClientBuilder	com.treasuredata.client	Builder for configuring and constructing TDClient instances
TDHttpClient	com.treasuredata.client	Internal HTTP client; manages request/response lifecycle, retries, and error handling
TDClientConfig	com.treasuredata.client	Immutable configuration object containing all client settings
TDApiRequest	com.treasuredata.client	Represents an HTTP API request with method, path, parameters, and body
TDHttpRequestHandler	com.treasuredata.client	Interface for customizing request preparation and response processing
TDRequestErrorHandler	com.treasuredata.client	Categorizes HTTP errors into retryable/non-retryable and resolves appropriate exceptions
BackOff	com.treasuredata.client	Interface for retry strategies; implementations include ExponentialBackOff, FullJitterBackOff

Sources: TDClient.java TDClientBuilder.java1-51

---

Component Interaction: Code Entity View

This diagram shows how key code entities interact during a typical API request:

Key Interactions:

1. Request Initiation: Application calls TDClient methods (e.g., listTables(), submit(), jobStatus())
2. Request Building: TDClient creates TDApiRequest objects with method, path, and parameters
3. Request Execution: TDHttpClient.submitRequest() executes the request using OkHttpClient
4. Request Customization: TDHttpRequestHandler allows pre-request modifications (headers, authentication)
5. Response Processing: Handler's onSuccess() parses JSON responses into model objects using Jackson
6. Error Handling: TDRequestErrorHandler categorizes errors and determines retry behavior
7. Retry Logic: BackOff strategies calculate wait times between retries

Sources: TDClient.java TDHttpClient.java TDHttpRequestHandler.java

---

Key Capabilities

1. Database and Table Management

Operation	Method	Description
Create Database	createDatabase(name)	Creates a new database
List Databases	listDatabases()	Retrieves all databases
Show Database	showDatabase(name)	Gets detailed database info
Delete Database	deleteDatabase(name)	Removes a database
Create Table	createTable(db, table)	Creates a new table
List Tables	listTables(db)	Lists tables in a database
Show Table	showTable(db, table)	Gets table metadata
Delete Table	deleteTable(db, table)	Removes a table
Update Schema	updateTableSchema(db, table, columns)	Modifies table schema

Sources: TDClient.java Example.java99-209

2. Query Execution and Job Management

Operation	Method	Return Type
Submit Query	submit(TDJobRequest)	String (jobId)
Check Status	jobStatus(jobId)	TDJobSummary
Get Job Info	jobInfo(jobId)	TDJob
Get Results	jobResult(jobId, format, handler)	<T> T
Kill Job	kill(jobId)	TDJob

Job Types Supported:

• Presto: TDJobRequest.newPrestoQuery()
• Trino: TDJobRequest.newTrinoQuery()
• Hive: TDJobRequest.newHiveQuery()
• Pig: TDJobRequest.newPigQuery()
• Bulk Load: TDJobRequest.newBulkLoad()

Sources: Example.java121-160 TDJobRequest.java

3. Bulk Import System

The bulk import workflow involves creating a session, uploading data parts, performing the import, and committing:

Key Methods:

• createBulkImportSession(name, database, table): Initialize session
• uploadBulkImportPart(session, partName, file): Upload msgpack.gz data
• performBulkImportSession(session): Start import job
• commitBulkImportSession(session): Finalize import

Sources: Example.java271-296 TDClient.java

4. Saved Query Management

Operation	Method	Description
Save Query	saveQuery(request)	Register a new scheduled query
List Queries	listSavedQueries()	Get all saved queries
Start Query	startSavedQuery(name, time)	Execute a saved query
Update Query	updateSavedQuery(name, request)	Modify saved query
Delete Query	deleteSavedQuery(name)	Remove saved query
Get History	getSavedQueryHistory(name)	Retrieve execution history

Sources: Example.java298-345 TDSavedQuery.java

---

Configuration System

Configuration is loaded from multiple sources with the following precedence (highest to lowest):

1. Properties object passed to TDClientBuilder.setProperties()
2. $HOME/.td/td.conf file
3. System properties (JVM -D flags)
4. Environment variables (TD_API_KEY only)

Essential Configuration Parameters

Parameter	Default	Description
apikey	-	API key for authentication (or use TD_API_KEY env var)
td.client.endpoint	api.treasuredata.com	API endpoint hostname
td.client.port	443 (SSL), 80 (non-SSL)	API port
td.client.usessl	true	Enable SSL/TLS
td.client.retry.limit	7	Maximum retry attempts
td.client.retry.max-interval	60000	Max backoff interval (ms)
td.client.connect-timeout	15000	Connection timeout (ms)
td.client.read-timeout	60000	Read timeout (ms)
td.client.proxy.host	-	Proxy hostname
td.client.proxy.port	-	Proxy port

Configuration Example:

Sources: README.md58-229 TDClientBuilder.java1-51 AbstractTDClientBuilder.java

---

Data Model Overview

The library provides strongly-typed model classes for all Treasure Data entities:

All model classes are immutable and use the Immutables library for type-safe builders. Jackson annotations handle JSON serialization. See Data Models for detailed documentation.

Sources: pom.xml162-173 TDDatabase.java TDTable.java TDJob.java

---

Error Handling Strategy

The library implements sophisticated error handling with automatic retries:

Exception Hierarchy

• TDClientException: Base exception for all client errors
  • TDClientHttpException: HTTP-related errors (includes status code)
    • TDClientHttpUnauthorizedException: 401 errors
    • TDClientHttpNotFoundException: 404 errors
    • TDClientHttpConflictException: 409 errors
    • TDClientHttpTooManyRequestsException: 429 rate limiting

Retry Behavior

Error Type	Status Codes	Retryable	Notes
Server Errors	5xx	Yes	Always retried with backoff
Rate Limiting	429	Yes	Respects Retry-After header
Network Errors	-	Yes	Socket/timeout exceptions
Authorization	401	No	Invalid credentials
Not Found	404	Context-dependent	Retried for read ops, not for create ops
Invalid Input	422	No	Client error, no retry

Backoff Strategies

• ExponentialBackOff: Linear exponential increase
• EqualJitterBackOff: Half exponential + half jitter
• FullJitterBackOff: Full jitter (recommended, default)

Sources: TDClientException.java TDRequestErrorHandler.java BackOff.java

---

Maven Dependencies

The library uses minimal external dependencies with shading to prevent conflicts:

Dependency	Version	Purpose	Shaded
OkHttp3	3.14.9	HTTP client	Yes
Jackson Core	2.14.2	JSON processing	No
Jackson Databind	2.14.2	JSON binding	No
Guava	31.1-jre	Utilities	Yes
SLF4J API	2.0.12	Logging facade	No
Immutables	2.10.1	Code generation (compile-only)	No

Shaded Packages (in td-client-(version)-shade.jar):

• com.google.* → com.treasuredata.client.thirdparty.com.google.*
• okhttp3.* → com.treasuredata.client.thirdparty.okhttp3.*
• okio.* → com.treasuredata.client.thirdparty.okio.*

Sources: pom.xml98-322

---

Quick Start Example

For complete examples including bulk import and saved queries, see Example.java65-346

Sources: README.md94-151 Example.java65-346

---

Next Steps

• Installation and Setup: See Getting Started for detailed installation instructions and initial configuration
• API Reference: See Client API for comprehensive documentation of all TDClient methods
• Configuration Details: See Configuration for advanced configuration options and proxy setup
• Error Handling: See Error Handling and Resilience for exception types and retry strategies
• Development: See Build and Development for information on building from source and contributing

Sources: README.md1-260