Skip to content

[Feature] Network Import/Export #171

@zomux

Description

@zomux

[Feature] Network Import/Export

== Overview / Objective / Timeline

Problem: Network administrators cannot easily:

  • Backup their network configuration
  • Share network setups with others
  • Migrate networks between environments
  • Create templates from existing networks

Goal: Enable network import/export via .zip files in the admin dashboard:

  1. Export - Package network configuration into a downloadable .zip file
  2. Import - Upload a .zip file to create/restore a network configuration
  3. UI - Add import/export controls to the admin dashboard

Key Decisions:

Decision Choice Rationale
File Format .zip archive Standard format, supports multiple files, compression
Config Format YAML Consistent with existing network.yaml format
Data Inclusion Config only (no runtime data) Keeps exports small, avoids sensitive data
Import Mode Create new or overwrite Flexibility for different use cases
UI Location Profile/Settings page Where network management already exists
Restart Strategy In-process restart Network.restart() for seamless config reload

Notes
导入.zip文件后 整个网络需要重启。 这个需要实现Network.restart,但不让整个进程重新启动。参考Network Restart Method里面 claude给出的实现方案。

Timeline: 1.5 PD


IMPORTANT: Following content is generated by Claude. Please take it as a reference.

== Functional Requirements

1. Export Network

What Gets Exported:

  • network.yaml - Main network configuration
  • network_profile.yaml - Network profile/discovery info (if configured)
  • mods/ - Mod-specific configuration files (if any)
  • manifest.json - Export metadata (version, timestamp, source)

Export Options:

  • Include/exclude agent group password hashes
  • Include/exclude sensitive mod configurations
  • Add custom notes/description to export

Export Flow:

  1. User clicks "Export Network" in admin dashboard
  2. System packages configuration into .zip
  3. Browser downloads the .zip file
  4. Filename format: {network_name}_{timestamp}.zip

2. Import Network

Import Modes:

  • Create New - Import as a new network (requires unique name)
  • Overwrite - Replace current network configuration (with confirmation)

Validation:

  • Verify .zip structure and required files
  • Validate YAML syntax
  • Check mod compatibility (warn if mods not installed)
  • Validate configuration schema

Import Flow:

  1. User clicks "Import Network" in admin dashboard
  2. File picker opens for .zip selection
  3. System validates the archive
  4. Preview screen shows what will be imported
  5. User confirms import
  6. Network configuration is applied
  7. Network automatically restarts via Network.restart()
  8. UI shows restart progress and success confirmation

3. UI Components

Export Button:

  • Located in Profile/Settings page
  • Dropdown for export options
  • Progress indicator during packaging

Import Button:

  • Located in Profile/Settings page
  • File upload zone (drag & drop supported)
  • Validation feedback
  • Preview modal before import
  • Confirmation dialog for overwrite mode

== API Specifications

Backend Endpoints

Export Network

@router.get("/api/network/export")
async def export_network(
    include_password_hashes: bool = False,
    include_sensitive_config: bool = False,
    notes: Optional[str] = None,
) -> StreamingResponse:
    """
    Export current network configuration as .zip file.

    Returns:
        StreamingResponse with .zip file content
    """

Import Network (Upload)

@router.post("/api/network/import/validate")
async def validate_import(
    file: UploadFile,
) -> ImportValidationResult:
    """
    Validate uploaded .zip file before import.

    Returns:
        ImportValidationResult with validation status, warnings, preview
    """

@router.post("/api/network/import/apply")
async def apply_import(
    file: UploadFile,
    mode: ImportMode,  # "create_new" or "overwrite"
    new_name: Optional[str] = None,  # Required if mode is "create_new"
) -> ImportResult:
    """
    Apply imported network configuration and restart network.

    Returns:
        ImportResult with status and restart result
    """

Data Models

class ImportMode(str, Enum):
    CREATE_NEW = "create_new"
    OVERWRITE = "overwrite"

class ImportValidationResult(BaseModel):
    valid: bool
    errors: List[str]
    warnings: List[str]
    preview: ImportPreview

class ImportPreview(BaseModel):
    network_name: str
    network_mode: str
    mods: List[str]
    agent_groups: List[str]
    transports: List[str]
    export_timestamp: Optional[datetime]
    export_notes: Optional[str]
    source_version: Optional[str]

class ImportResult(BaseModel):
    success: bool
    message: str
    restarted: bool              # True if network was successfully restarted
    restart_error: Optional[str] # Error message if restart failed
    network_name: str

class ExportManifest(BaseModel):
    version: str = "1.0"
    export_timestamp: datetime
    source_network: str
    source_version: str
    openagents_version: str
    notes: Optional[str]
    includes_password_hashes: bool
    includes_sensitive_config: bool

== Zip Archive Structure

network_export.zip
├── manifest.json           # Export metadata
├── network.yaml            # Main network configuration
├── network_profile.yaml    # Network profile (optional)
└── mods/                   # Mod-specific configs (optional)
    ├── messaging.yaml
    ├── forum.yaml
    └── ...

manifest.json Example

{
  "version": "1.0",
  "export_timestamp": "2025-11-29T10:30:00Z",
  "source_network": "MyNetwork",
  "source_version": "1.2.3",
  "openagents_version": "0.5.0",
  "notes": "Production network backup before upgrade",
  "includes_password_hashes": false,
  "includes_sensitive_config": false
}

network.yaml Example

network:
  name: "MyNetwork"
  mode: "centralized"
  node_id: "my-network-1"
  transports:
    - type: "http"
      config:
        port: 8700
  agent_groups:
    default:
      description: "Default agent group"
      # password_hash excluded if includes_password_hashes=false
  mods:
    - name: "openagents.mods.workspace.messaging"
      enabled: true
      config:
        default_channels:
          - name: "general"
            description: "General chat"

== Module Structure

Backend

src/openagents/utils/
├── network_export.py       # Export logic
├── network_import.py       # Import logic and validation
└── ...

src/openagents/api/
├── routes/
│   └── network_management.py  # Add import/export endpoints
└── ...

Frontend

studio/src/
├── pages/
│   └── profile/
│       ├── NetworkImportExport.tsx    # New component
│       └── ProfileMainPage.tsx        # Add import/export section
├── components/
│   └── network/
│       ├── ExportOptionsModal.tsx     # Export options dialog
│       ├── ImportPreviewModal.tsx     # Import preview/confirm dialog
│       └── ImportDropzone.tsx         # File upload component
├── services/
│   └── networkManagementService.ts    # API calls
└── types/
    └── networkManagement.ts           # Type definitions

== Implementation Details

Export Logic

# src/openagents/utils/network_export.py

import zipfile
import yaml
from io import BytesIO
from datetime import datetime

class NetworkExporter:
    def __init__(self, network: Network):
        self.network = network

    def export_to_zip(
        self,
        include_password_hashes: bool = False,
        include_sensitive_config: bool = False,
        notes: Optional[str] = None,
    ) -> BytesIO:
        """Export network configuration to zip archive."""

        buffer = BytesIO()

        with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as zf:
            # Add manifest
            manifest = self._create_manifest(notes, include_password_hashes, include_sensitive_config)
            zf.writestr('manifest.json', json.dumps(manifest, default=str, indent=2))

            # Add network config
            network_config = self._sanitize_config(
                self.network.config,
                include_password_hashes,
                include_sensitive_config,
            )
            zf.writestr('network.yaml', yaml.dump(network_config, default_flow_style=False))

            # Add network profile if exists
            if self.network.profile:
                zf.writestr('network_profile.yaml', yaml.dump(self.network.profile.dict()))

            # Add mod configs
            for mod in self.network.mods:
                if mod.has_exportable_config:
                    zf.writestr(f'mods/{mod.name}.yaml', yaml.dump(mod.export_config()))

        buffer.seek(0)
        return buffer

    def _sanitize_config(self, config: NetworkConfig, include_hashes: bool, include_sensitive: bool) -> dict:
        """Remove sensitive data based on options."""
        data = config.dict()

        if not include_hashes:
            for group in data.get('agent_groups', {}).values():
                group.pop('password_hash', None)

        if not include_sensitive:
            # Remove API keys, secrets, etc. from mod configs
            for mod in data.get('mods', []):
                self._strip_sensitive_fields(mod.get('config', {}))

        return data

Import Logic

# src/openagents/utils/network_import.py

class NetworkImporter:
    REQUIRED_FILES = ['manifest.json', 'network.yaml']

    def validate(self, zip_file: BytesIO) -> ImportValidationResult:
        """Validate zip archive before import."""
        errors = []
        warnings = []

        try:
            with zipfile.ZipFile(zip_file, 'r') as zf:
                # Check required files
                files = zf.namelist()
                for required in self.REQUIRED_FILES:
                    if required not in files:
                        errors.append(f"Missing required file: {required}")

                if errors:
                    return ImportValidationResult(valid=False, errors=errors, warnings=[], preview=None)

                # Parse and validate manifest
                manifest = json.loads(zf.read('manifest.json'))

                # Parse and validate network config
                network_yaml = yaml.safe_load(zf.read('network.yaml'))

                # Check mod compatibility
                for mod in network_yaml.get('network', {}).get('mods', []):
                    if not self._is_mod_available(mod['name']):
                        warnings.append(f"Mod not installed: {mod['name']}")

                # Build preview
                preview = self._build_preview(manifest, network_yaml)

                return ImportValidationResult(
                    valid=True,
                    errors=[],
                    warnings=warnings,
                    preview=preview,
                )

        except Exception as e:
            return ImportValidationResult(
                valid=False,
                errors=[f"Invalid archive: {str(e)}"],
                warnings=[],
                preview=None,
            )

    async def apply(
        self,
        zip_file: BytesIO,
        mode: ImportMode,
        network: Network,
        new_name: Optional[str] = None,
    ) -> ImportResult:
        """Apply imported configuration and restart network."""

        with zipfile.ZipFile(zip_file, 'r') as zf:
            network_yaml = yaml.safe_load(zf.read('network.yaml'))

            if mode == ImportMode.CREATE_NEW:
                network_yaml['network']['name'] = new_name
                network_yaml['network']['node_id'] = f"{new_name.lower().replace(' ', '-')}-1"

            # Write to config file
            config_path = self._get_config_path()
            with open(config_path, 'w') as f:
                yaml.dump(network_yaml, f, default_flow_style=False)

            # Extract mod configs
            for name in zf.namelist():
                if name.startswith('mods/') and name.endswith('.yaml'):
                    mod_config = yaml.safe_load(zf.read(name))
                    self._apply_mod_config(name, mod_config)

        # Parse new config and restart network
        new_config = NetworkConfig(**network_yaml['network'])

        try:
            restart_success = await network.restart(new_config)
            return ImportResult(
                success=True,
                message="Network configuration imported and restarted successfully",
                restarted=restart_success,
                restart_error=None if restart_success else "Restart returned False",
                network_name=network_yaml['network']['name'],
            )
        except Exception as e:
            return ImportResult(
                success=True,  # Config was applied, but restart failed
                message="Configuration imported but restart failed",
                restarted=False,
                restart_error=str(e),
                network_name=network_yaml['network']['name'],
            )

Network Restart Method

# Add to src/openagents/core/network.py

class Network:
    # ... existing methods ...

    async def restart(self, new_config: Optional[NetworkConfig] = None) -> bool:
        """Restart network with optional new configuration.

        Performs a graceful shutdown followed by re-initialization.
        Connected agents will be disconnected and need to reconnect.

        Args:
            new_config: New configuration to apply. If None, reloads from config file.

        Returns:
            bool: True if restart successful
        """
        logger.info(f"Restarting network '{self.network_name}'...")

        # 1. Graceful shutdown - disconnect all agents, stop mods
        shutdown_success = await self.shutdown()
        if not shutdown_success:
            logger.error("Failed to shutdown network during restart")
            return False

        # 2. Apply new config if provided
        if new_config:
            self.config = new_config
            self.network_name = new_config.name
        else:
            # Reload from config file
            self.config = self._load_config_from_file()
            self.network_name = self.config.name

        # 3. Re-initialize with new/reloaded config
        init_success = await self.initialize()
        if not init_success:
            logger.error("Failed to initialize network during restart")
            return False

        logger.info(f"Network '{self.network_name}' restarted successfully")
        return True

    def _load_config_from_file(self) -> NetworkConfig:
        """Reload configuration from the config file."""
        import yaml
        with open(self.config_path, 'r') as f:
            config_data = yaml.safe_load(f)
        return NetworkConfig(**config_data.get('network', config_data))

== Expected Deliverables

Backend:

  • src/openagents/utils/network_export.py
  • src/openagents/utils/network_import.py
  • src/openagents/core/network.py (add restart() method)
  • src/openagents/api/routes/network_management.py (add endpoints)
  • src/openagents/models/network_management.py (data models)

Frontend:

  • studio/src/pages/profile/NetworkImportExport.tsx
  • studio/src/components/network/ExportOptionsModal.tsx
  • studio/src/components/network/ImportPreviewModal.tsx
  • studio/src/components/network/ImportDropzone.tsx
  • studio/src/services/networkManagementService.ts
  • studio/src/types/networkManagement.ts
  • Update ProfileMainPage.tsx to include import/export section

Tests:

  • Test export with various options
  • Test import validation (valid/invalid archives)
  • Test import apply (create new / overwrite)
  • Test sanitization of sensitive data
  • Test mod compatibility warnings
  • Test Network.restart() with new config
  • Test Network.restart() reload from file
  • Test restart failure handling

== Example Usage

Export via UI

  1. Navigate to Profile → Network Settings
  2. Click Export Network button
  3. In the modal:
    • Toggle "Include password hashes" (default: off)
    • Toggle "Include sensitive config" (default: off)
    • Add optional notes
  4. Click Export
  5. Browser downloads MyNetwork_2025-11-29.zip

Import via UI

  1. Navigate to Profile → Network Settings
  2. Click Import Network button
  3. Drag & drop or select .zip file
  4. Review validation results:
    • Green: Valid configuration
    • Yellow: Warnings (e.g., missing mods)
    • Red: Errors (invalid format)
  5. Preview imported configuration
  6. Select mode:
    • Create New: Enter new network name
    • Overwrite: Confirm replacement
  7. Click Import
  8. UI shows "Restarting network..." progress indicator
  9. Network restarts automatically via Network.restart()
  10. Success message: "Network imported and restarted successfully"
    • Or error handling if restart fails

Programmatic Export (Python)

from openagents.utils.network_export import NetworkExporter

exporter = NetworkExporter(network)
zip_buffer = exporter.export_to_zip(
    include_password_hashes=False,
    notes="Backup before v2 migration"
)

with open('network_backup.zip', 'wb') as f:
    f.write(zip_buffer.read())

== Estimates and Records

Workstream

Task Estimate
Backend export logic 0.25 PD
Backend import logic + validation 0.5 PD
API endpoints 0.25 PD
Frontend components 0.5 PD
Integration + testing 0.25 PD
Total 1.75 PD

== Dates

  • PRD Start: November 29, 2025

== Success Criteria

  • Export produces valid .zip with correct structure
  • Sensitive data is excluded when options are off
  • Import validates archive structure and content
  • Import shows clear errors/warnings
  • Import preview displays all key configuration
  • Create new mode works with unique name
  • Overwrite mode requires confirmation
  • UI is intuitive with drag & drop support
  • Network.restart() performs graceful shutdown and re-initialization
  • UI shows restart progress and handles errors gracefully
  • Tests cover all major scenarios

== Security Considerations

  1. Password Hashes - Excluded by default from exports
  2. Sensitive Config - API keys, secrets stripped by default
  3. File Validation - Strict validation of uploaded files
  4. Path Traversal - Prevent zip slip attacks during extraction
  5. Size Limits - Enforce maximum upload size
  6. Overwrite Protection - Require explicit confirmation

== Reference Files

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions