Skip to content

Add CloudKit Schema Management APIs (cktool/cktooljs functionality) #135

@leogdion

Description

@leogdion

Overview

Add CloudKit schema management capabilities to MistKit, providing a pure Swift alternative to cktool and cktooljs for automating CloudKit development workflows.

Motivation

Currently, CloudKit schema management requires:

  • macOS + Xcode: cktool (command-line tool bundled with Xcode)
  • Node.js: cktooljs JavaScript libraries
  • Manual: CloudKit Console web interface

A pure Swift implementation in MistKit would enable:

  • Cross-platform schema management (Linux, server-side Swift)
  • Programmatic schema operations in Swift projects
  • Integration with Swift Package Manager workflows
  • Single-language automation for CloudKit applications
  • CI/CD pipelines without Node.js or Xcode dependencies

Current Tools Comparison

cktool (Native CLI)

# Export schema to file
xcrun cktool export-schema \
  --team-id TEAM-ID \
  --container-id CONTAINER \
  --environment development \
  --output-file schema.ckdb

# Import schema from file
xcrun cktool import-schema \
  --team-id TEAM-ID \
  --container-id CONTAINER \
  --environment development \
  --file schema.ckdb

# Reset development to production schema
xcrun cktool reset-schema \
  --team-id TEAM-ID \
  --container-id CONTAINER

cktooljs (JavaScript Library)

// Deploy schema to Sandbox
// Seed databases with test data
// Restore Sandbox to production
// Automated integration test scripts

Proposed MistKit API

Schema Service

@available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)
public struct CloudKitSchemaService {
    let containerIdentifier: String
    let tokenManager: ManagementTokenProvider
    let environment: Environment
    
    // MARK: - Schema Export/Import
    
    /// Export schema definition from CloudKit
    func exportSchema() async throws -> CloudKitSchema
    
    /// Export schema and save to file
    func exportSchema(to url: URL) async throws
    
    /// Import schema definition to CloudKit
    func importSchema(_ schema: CloudKitSchema) async throws
    
    /// Import schema from file
    func importSchema(from url: URL) async throws
    
    // MARK: - Schema Reset
    
    /// Reset development schema to match production
    func resetDevelopmentSchema() async throws
    
    // MARK: - Schema Inspection
    
    /// Get current schema definition
    func getSchema() async throws -> CloudKitSchema
    
    /// Compare schemas between environments
    func compareSchemas(
        source: Environment,
        target: Environment
    ) async throws -> SchemaDiff
    
    // MARK: - Record Type Management
    
    /// Create or update a record type
    func saveRecordType(_ recordType: RecordTypeDefinition) async throws
    
    /// Delete a record type
    func deleteRecordType(_ typeName: String) async throws
    
    /// List all record types
    func listRecordTypes() async throws -> [RecordTypeDefinition]
}

Data Models

/// Complete CloudKit schema definition
public struct CloudKitSchema: Codable {
    public let recordTypes: [RecordTypeDefinition]
    public let securityRoles: [SecurityRole]
    public let subscriptions: [SubscriptionDefinition]?
    public let indexes: [IndexDefinition]?
    
    // File format support (.ckdb compatibility)
    public func write(to url: URL) throws
    public static func read(from url: URL) throws -> CloudKitSchema
}

/// Record type definition
public struct RecordTypeDefinition: Codable {
    public let name: String
    public let fields: [FieldDefinition]
    public let indexes: [IndexDefinition]
}

/// Field definition
public struct FieldDefinition: Codable {
    public let name: String
    public let type: FieldType
    public let isList: Bool
    public let isRequired: Bool
    public let isIndexed: Bool
    public let isUnique: Bool
}

public enum FieldType: String, Codable {
    case string = "STRING"
    case int64 = "INT64"
    case double = "DOUBLE"
    case date = "TIMESTAMP"
    case bytes = "BYTES"
    case reference = "REFERENCE"
    case asset = "ASSET"
    case location = "LOCATION"
    case stringList = "STRING_LIST"
    case int64List = "INT64_LIST"
    case doubleList = "DOUBLE_LIST"
    case dateList = "TIMESTAMP_LIST"
    case referenceList = "REFERENCE_LIST"
    case assetList = "ASSET_LIST"
}

/// Index definition
public struct IndexDefinition: Codable {
    public let fieldName: String
    public let indexType: IndexType
}

public enum IndexType: String, Codable {
    case queryable = "QUERYABLE"
    case sortable = "SORTABLE"
    case unique = "UNIQUE"
}

/// Schema difference report
public struct SchemaDiff {
    public let addedRecordTypes: [String]
    public let removedRecordTypes: [String]
    public let modifiedRecordTypes: [RecordTypeChange]
}

public struct RecordTypeChange {
    public let name: String
    public let addedFields: [String]
    public let removedFields: [String]
    public let modifiedFields: [FieldChange]
}

Management Token Authentication

/// Management token provider (separate from runtime auth)
public protocol ManagementTokenProvider {
    func getManagementToken() async throws -> String
}

/// Simple implementation
public struct StaticManagementToken: ManagementTokenProvider {
    let token: String
    
    public func getManagementToken() async throws -> String {
        return token
    }
}

/// Keychain-backed implementation
public struct KeychainManagementToken: ManagementTokenProvider {
    let keychainKey: String
    
    public func getManagementToken() async throws -> String {
        // Read from keychain
    }
}

Use Cases

1. Development Workflow

let schemaService = try CloudKitSchemaService(
    containerIdentifier: "iCloud.com.example.app",
    tokenManager: StaticManagementToken(token: managementToken),
    environment: .development
)

// Export production schema
let productionService = try CloudKitSchemaService(
    containerIdentifier: "iCloud.com.example.app",
    tokenManager: StaticManagementToken(token: managementToken),
    environment: .production
)

let schema = try await productionService.exportSchema()
try schema.write(to: URL(fileURLWithPath: "schema.json"))

// Import to development
try await schemaService.importSchema(from: URL(fileURLWithPath: "schema.json"))

2. CI/CD Integration

// In integration tests
let schemaService = try CloudKitSchemaService(
    containerIdentifier: containerID,
    tokenManager: EnvironmentManagementToken(),
    environment: .development
)

// Reset to clean state
try await schemaService.resetDevelopmentSchema()

// Import test schema
let testSchema = try CloudKitSchema.read(from: testSchemaURL)
try await schemaService.importSchema(testSchema)

// Run tests...

3. Schema Validation

// Compare development vs production
let diff = try await schemaService.compareSchemas(
    source: .development,
    target: .production
)

if !diff.addedRecordTypes.isEmpty {
    print("New record types in development: \(diff.addedRecordTypes)")
}

4. Programmatic Schema Creation

// Define schema in Swift
let feedType = RecordTypeDefinition(
    name: "PublicFeed",
    fields: [
        FieldDefinition(name: "feedURL", type: .string, isIndexed: true),
        FieldDefinition(name: "title", type: .string),
        FieldDefinition(name: "lastAttempted", type: .date, isIndexed: true),
        FieldDefinition(name: "usageCount", type: .int64)
    ],
    indexes: [
        IndexDefinition(fieldName: "feedURL", indexType: .queryable),
        IndexDefinition(fieldName: "lastAttempted", indexType: .sortable)
    ]
)

try await schemaService.saveRecordType(feedType)

CloudKit Management API Endpoints

Based on reverse engineering cktool/cktooljs, the CloudKit Management API likely provides:

Base URL: https://api.apple-cloudkit.com/database/1/{container}/management/

Endpoints:
- GET    /schema                      # Get schema
- POST   /schema                      # Update schema
- DELETE /schema                      # Reset schema
- GET    /schema/recordTypes          # List record types
- POST   /schema/recordTypes          # Create record type
- PUT    /schema/recordTypes/{name}   # Update record type
- DELETE /schema/recordTypes/{name}   # Delete record type

Authentication: Management Token (different from API Token)

Implementation Phases

Phase 1: Schema Export

  • Management token authentication
  • GET schema endpoint
  • Schema model types
  • File export (.json format)

Phase 2: Schema Import

  • POST schema endpoint
  • Schema validation
  • File import
  • Error handling for schema conflicts

Phase 3: Schema Management

  • Individual record type CRUD
  • Field definitions
  • Index management
  • Schema reset functionality

Phase 4: Advanced Features

  • Schema diffing
  • Migration suggestions
  • .ckdb file format support (compatibility with cktool)
  • Backup/restore workflows
  • Schema versioning

Phase 5: Developer Tools

  • Swift Package Manager plugin
  • Command-line tool (pure Swift alternative to cktool)
  • Xcode integration helpers

Related Issues

  • Depends on understanding CloudKit Management API (may need reverse engineering)
  • Related to #[existing auth issue if any]
  • Complements existing data operation APIs

References

  • .claude/docs/cktool.md - Native tool documentation
  • .claude/docs/cktooljs.md - JavaScript library documentation
  • .claude/docs/webservices.md - CloudKit Web Services API
  • CloudKit Management API (limited public docs)

Benefits

  1. Pure Swift: No Node.js or Xcode dependencies
  2. Cross-platform: Works on Linux, server-side Swift
  3. Type-safe: Leverage Swift's type system for schemas
  4. Integration: Seamless with existing MistKit APIs
  5. Automation: Enable Swift-based CI/CD for CloudKit
  6. Versioning: Schema-as-code in Swift projects

Open Questions

  1. Is the CloudKit Management API publicly documented?
  2. Can we reverse engineer the request/response format from cktool?
  3. Are management tokens different from server-to-server keys?
  4. Should we support .ckdb format for cktool compatibility?
  5. Should this be in MistKit core or a separate MistKitManagement package?

Alternatives Considered

  1. Wrapper around cktool: Requires macOS + Xcode
  2. Wrapper around cktooljs: Requires Node.js runtime
  3. CloudKit Console only: Manual, not automatable
  4. New Swift implementation: ✅ Proposed solution

/cc @brightdigit for thoughts on priority and implementation approach

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions