Epic/525 implement fully functioning node registry#545
Conversation
- Implemented a new PublishDialog component for publishing updates to packages. - Allows users to select version bump type (patch, minor, major) and enter release notes. - Validates release notes length before enabling the publish button. - Integrates with existing UI components for a consistent user experience. feat: create usePackageStoreData hook for package purchasing - Added usePackageStoreData hook to manage package purchasing and access requests. - Handles fetching user profile, purchasing packages, and requesting access. - Provides utility functions for formatting prices and managing access states. feat: introduce wasm-package-permission utilities - Created permission helpers for managing package permissions. - Includes functions to check user roles (owner, maintainer, user, buyer) and manage permission levels. feat: extend wasm schema with new package and node definitions - Updated wasm schema to include new interfaces for NodeScores, PinOptions, and Pin. - Enhanced PackageNodeEntry and RegistryEntry with additional fields for better package management. feat: enhance registry state with access request management - Modified IRegistryState interface to include methods for purchasing packages and managing access requests. - Added support for listing and accepting/rejecting access requests. feat: implement version management in WASM client - Updated RegistryClient to handle multiple installed versions of packages. - Introduced methods for installing specific versions and batch installation of packages. feat: add cross-compilation support in WASM engine - Enhanced WasmConfig to support cross-compilation target triples. - Implemented precompilation for both host and cross-compilation targets. feat: improve manifest structure for package nodes - Updated PackageNodeEntry to include optional fields for friendly names, scores, and pin definitions. - Enhanced documentation and metadata handling for nodes. refactor: reorganize installed package structure - Refactored InstalledPackage to manage multiple versions of a package. - Introduced InstalledVersion struct to encapsulate version-specific details.
…eck in API routes
- Added Chain of Density post-processing to enhance information density in summaries. - Introduced EntityTracker to manage and track important entities across summarization chunks. - Developed Hierarchical summarization to leverage document structure for improved summaries. - Implemented Hybrid summarization combining Map-Reduce and Refine strategies for efficiency and coherence. - Created Map-Reduce summarization to process chunks in parallel and reduce them recursively. - Established a modular structure for summarization strategies in the mod.rs file. - Enhanced prompts for various summarization tasks, ensuring clarity and adherence to rules. - Introduced Iterative Refinement summarization for coherent output through sequential processing. - Developed Sliding Window summarization to maintain a fixed-size memory buffer, preventing context pressure.
…ctionality with owned_only filter
- Updated `PackageSummary` struct to include `price` and `visibility` fields with default values. - Added `auth_token` field to `RegistryConfig` for authenticated API requests. - Introduced integration tests for new context functions related to schema, image, DB, and models. - Verified that host-side schema module returns valid JSON schemas and that linker registration succeeds for all new host functions. - Ensured that WAT-based WASM modules can call host functions without panics and that schema data is correctly returned through the WASM ABI.
- Updated `wasm_package_join_queue.rs`, `wasm_package_purchase.rs`, `wasm_package_review.rs`, `wasm_package_user.rs`, `wasm_package_version.rs`, and `widget.rs` to enhance code formatting and organization. - Removed unnecessary whitespace and improved the alignment of attributes for better clarity. - Added missing `impl` blocks for `ActiveModelBehavior` in several entities. - Refactored the `download.rs`, `index.rs`, `prerun_check.rs`, and `server.rs` routes to streamline access control and package retrieval logic. - Enhanced error handling and logging in `jwt.rs`, `execute.rs`, and `wasm_loader.rs` for better debugging and maintainability. - Improved the clarity of comments and documentation throughout the codebase.
…ty, including Sentry integration for error reporting
…ironment variable handling and SQS support; add soft-delete functionality for packages in the registry
…vention to request and response structs
…regardless of sink choice
…ipulation - Introduced `setIframeSrc` node to set the source URL of an iframe element. - Introduced `setIframeSrcdoc` node to set raw HTML content for iframe preview. - Updated `IframeProps` to include optional `srcdoc` field. - Modified `Iframe` component to handle both `src` and `srcdoc` properties. - Updated documentation to reflect changes in iframe component capabilities.
…mproved security and flexibility
…improved write performance
- Updated various ML nodes (Decision Tree, Naive Bayes, SVM, DBSCAN, KMeans, etc.) to use a cached database instance for loading data. - Added calls to `ensure_flushed()` after loading the cached database to ensure data consistency. - Modified database access patterns to improve performance and reliability by reading from the cached database. - Adjusted dataset operations (Sample, Shuffle, Split, Stratified) to ensure proper flushing and access to the source database. - Enhanced metrics nodes (Accuracy, Confusion Matrix, Regression Metrics) to follow the new database loading pattern. - Updated prediction and PCA nodes to utilize the cached database and ensure data is flushed before processing. - Overall improvements to database handling across multiple ML components for better data integrity and performance.
- Updated Cargo.toml to include additional crate types and dependencies for enhanced functionality. - Modified flow-like.toml to streamline package metadata and permissions, including new node definitions. - Refactored lib.rs to implement new nodes: GreetingNode and FileWriterNode, with structured input/output pins and enhanced functionality. - Added tests for new nodes and ensured compliance with linting rules, including checks for duplicate pin names, schema validation, and node descriptions. - Introduced lint.rs to enforce coding standards and best practices across all nodes.
…andling in ECS compiler
- Added a new TypeScript file for dexie-tauri adapter that handles large strings and number arrays by offloading them to Tauri's native filesystem. - Introduced a new tsconfig.json for the frontend package with strict type checking and module resolution settings. - Updated the executor to use a new API endpoint for progress and event callbacks. - Enhanced the model provider to track and accumulate usage statistics for model calls. - Extended the chat default interfaces to include usage statistics for model invocations. - Implemented a new UsageStats component to display aggregated model usage statistics in the UI. - Improved the execution engine to deduplicate events based on event IDs. - Updated the Cargo.toml for the wasm package to enable all cranelift backends for cross-compilation.
|
🚧 Skipped: PR exceeds review size limit. Please split into smaller PRs and re-run. |
|
🚧 Skipped: PR exceeds review size limit. Please split into smaller PRs and re-run. |
There was a problem hiding this comment.
Code Review
This pull request introduces significant updates, including the addition of a WASM compiler service for Kubernetes and AWS, improved storage configuration management, and enhanced UI components for package management and developer tools. I have reviewed the code and identified several issues related to unsafe string slicing, missing timeouts for network requests, potential stack overflows in schema resolution, and code duplication. Please address these concerns to improve the robustness and maintainability of the codebase.
There was a problem hiding this comment.
Pull request overview
Implements a more complete node/package registry experience across the desktop app and backend deployments, adding compiler infrastructure, admin workflows, and updated developer guidance.
Changes:
- Desktop: expanded registry/package UX (explore + details), new admin pages, improved navigation/link handling, and additional settings pages.
- Backend: added Kubernetes + Docker Compose compiler service, AWS compiler/executor targets, secret-store integration, and expanded metrics/monitoring.
- Docs/tooling: added
CLAUDE.md, refined node-creation rules, and updated CI credential decoding.
Reviewed changes
Copilot reviewed 115 out of 1415 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| apps/desktop/components/global-anchor-component.tsx | Improves link/window handling for Tauri, including data-href navigables. |
| apps/desktop/components/auth-provider.tsx | Adjusts Tauri auth invalidation ordering to ensure user info first. |
| apps/desktop/components/app-sidebar.tsx | Updates nav structure and adds server-side profile deletion + new window URL resolution. |
| apps/desktop/components/admin/profile-templates-admin-page.tsx | Adds admin UI for browsing/managing profile templates. |
| apps/desktop/app/widget/page.tsx | Adds widget “Events” editing UI and passes action context into editor. |
| apps/desktop/app/use/page.tsx | Replaces in-app Use page implementation with shared UsePageContent. |
| apps/desktop/app/store/page.tsx | Refactors store page loading/error handling and adds app reviews section. |
| apps/desktop/app/settings/registry/layout.tsx | Renames “Installed” tab text to “Owned”. |
| apps/desktop/app/settings/registry/explore/page.tsx | Adds package detail view selection and authenticated detail fetch support. |
| apps/desktop/app/provider.tsx | Moves PostHog config to environment variables. |
| apps/desktop/app/profile/page.tsx | Adds href support to app cards for consistent navigation. |
| apps/desktop/app/onboarding/page.tsx | Filters out deleted profiles during onboarding fetch. |
| apps/desktop/app/notifications/page.tsx | Adds router-based navigation for notification links. |
| apps/desktop/app/library/packages/publish/page.tsx | Converts manifest payload keys to snake_case and expands resource tier options. |
| apps/desktop/app/library/packages/page.tsx | Initializes registry before queries and adds auth token to update checks. |
| apps/desktop/app/library/config/widgets/page.tsx | Improves empty-state copy/styling and tweaks preview overflow behavior. |
| apps/desktop/app/library/config/visibility-status-switcher.tsx | Adds publication request history card and cache invalidation. |
| apps/desktop/app/library/config/user-storage/page.tsx | Adds user storage file operations (reveal/open-with/list-apps). |
| apps/desktop/app/library/config/templates/page.tsx | Improves templates empty-state UX and copy. |
| apps/desktop/app/library/config/storage/page.tsx | Adds storage file operations (reveal/open-with/list-apps). |
| apps/desktop/app/library/config/runtime-vars/page.tsx | Updates runtime vars layout and header copy/styling. |
| apps/desktop/app/library/config/publication/page.tsx | Adds publication review page backed by API query. |
| apps/desktop/app/library/config/packages/page.tsx | Adds app packages settings page wrapper. |
| apps/desktop/app/library/config/flows/page.tsx | Adds board href generation for flows list navigation. |
| apps/desktop/app/library/config/events/page.tsx | Wires hub context into events settings page. |
| apps/desktop/app/library/config/endpoints/page.tsx | Replaces placeholder with real endpoints settings page. |
| apps/desktop/app/library/config/configuration/page.tsx | Improves configuration empty-state and header UX/copy. |
| apps/desktop/app/library/config/analytics/page.tsx | Replaces placeholder with analytics dashboard. |
| apps/desktop/app/layout.tsx | Adjusts query-cache retention and initializes blob offload. |
| apps/desktop/app/join/page.tsx | Adds retry/backoff + better UX for invite join flow. |
| apps/desktop/app/developer/new/page.tsx | Redirects developer wizard to package registry projects tab. |
| apps/desktop/app/developer/manifest/page.tsx | Expands permission options and removes node list section. |
| apps/desktop/app/debug/streaming/page.tsx | Adds a streaming markdown editor debug page. |
| apps/desktop/app/admin/user/page.tsx | Adds admin “user” page wrapper for profile templates view-only mode. |
| apps/desktop/app/admin/user/edit/page.tsx | Adds admin “user edit” page wrapper for managing templates. |
| apps/desktop/app/admin/solutions/page.tsx | Adds summary cards and improves admin solutions layout. |
| apps/desktop/app/admin/profiles/add/page.tsx | Adds create/edit profile template flow backed by admin APIs. |
| apps/desktop/app/admin/packages/page.tsx | Enhances package registry admin list (visibility/status formatting, layout). |
| apps/desktop/app/admin/governance/requests/page.tsx | Adds governance requests list/detail routing page. |
| apps/desktop/app/admin/governance/page.tsx | Adds governance list/detail routing page. |
| apps/desktop/app/admin/bits/add/llm.tsx | Adds hosted provider params editing and provider options. |
| apps/desktop/.gitignore | Ignores google-services.json for desktop app. |
| apps/desktop/.env.example | Updates staging API URL comment. |
| apps/backend/local/runtime/src/main.rs | Simplifies queued job conversion via ExecutionRequest::try_from(job). |
| apps/backend/local/runtime/Cargo.toml | Adjusts release opt-level for runtime. |
| apps/backend/local/api/src/main.rs | Adds secrets store and Sentry-tracing filtering; passes secret config into state. |
| apps/backend/local/api/Cargo.toml | Enables Sentry feature and adds secrets dependency. |
| apps/backend/kubernetes/sink-trigger/Cargo.toml | Adjusts release opt-level. |
| apps/backend/kubernetes/helm/values.yaml | Adds compiler component configuration section. |
| apps/backend/kubernetes/helm/templates/monitoring/prometheus.yaml | Adds compiler scrape config + recording/alert rules. |
| apps/backend/kubernetes/helm/templates/monitoring/grafana-dashboard-compiler.yaml | Adds compiler Grafana dashboard ConfigMap template. |
| apps/backend/kubernetes/helm/templates/compiler-service.yaml | Adds compiler service (http + metrics ports). |
| apps/backend/kubernetes/helm/templates/compiler-hpa.yaml | Adds optional compiler HPA template. |
| apps/backend/kubernetes/helm/templates/compiler-deployment.yaml | Adds compiler deployment template with env/config. |
| apps/backend/kubernetes/helm/templates/api-deployment.yaml | Wires API env vars for external compiler backend. |
| apps/backend/kubernetes/helm/templates/_helpers.tpl | Adds helper labels for compiler component. |
| apps/backend/kubernetes/executor/Cargo.toml | Adjusts release opt-level. |
| apps/backend/kubernetes/compiler/src/metrics.rs | Adds OTEL tracing + Prometheus metrics setup for compiler. |
| apps/backend/kubernetes/compiler/src/main.rs | Adds compiler binary that serves API + separate metrics listener. |
| apps/backend/kubernetes/compiler/Cargo.toml | Adds new Kubernetes compiler package manifest. |
| apps/backend/kubernetes/api/src/storage.rs | Splits storage into content/meta/cdn store constructors. |
| apps/backend/kubernetes/api/src/main.rs | Switches to CDN store and passes optional secrets config parameter. |
| apps/backend/kubernetes/api/src/config.rs | Adds cdn/meta bucket config fields + env fallback resolution. |
| apps/backend/kubernetes/api/Cargo.toml | Adjusts release opt-level. |
| apps/backend/kubernetes/.env.example | Adds WASM compilation env variables. |
| apps/backend/docker-compose/sink-services/Cargo.toml | Adjusts release opt-level. |
| apps/backend/docker-compose/runtime/src/main.rs | Uses ExecutionRequest::try_from(job) and preserves job_id logging. |
| apps/backend/docker-compose/runtime/Cargo.toml | Adjusts release opt-level. |
| apps/backend/docker-compose/monitoring/prometheus/prometheus.yml | Adds compiler scrape configuration. |
| apps/backend/docker-compose/monitoring/grafana/dashboards/compiler.json | Adds compiler Grafana dashboard. |
| apps/backend/docker-compose/flow-like.config.example.json | Enables wasm_registry in example config. |
| apps/backend/docker-compose/docker-compose.yml | Adds compiler service and wires compilation env vars into API. |
| apps/backend/docker-compose/compiler/Dockerfile | Adds compiler build/docker image recipe. |
| apps/backend/docker-compose/api/src/storage.rs | Splits storage constructors into content/meta/cdn. |
| apps/backend/docker-compose/api/src/main.rs | Uses CDN store and passes optional secrets config parameter. |
| apps/backend/docker-compose/api/Cargo.toml | Adjusts release opt-level. |
| apps/backend/docker-compose/.env.example | Documents compilation backend + compiler settings. |
| apps/backend/aws/media-transformer/Cargo.toml | Adjusts release opt-level. |
| apps/backend/aws/file-tracker/src/main.rs | Switches SeaORM import source and adds entity module. |
| apps/backend/aws/file-tracker/src/event_handler.rs | Moves entities to local module and uses SeaORM crate directly. |
| apps/backend/aws/file-tracker/src/entity.rs | Adds SeaORM entity definitions for app/user tables/enums. |
| apps/backend/aws/file-tracker/Cargo.toml | Removes flow-like-api dependency and adds sea-orm dependency. |
| apps/backend/aws/executor/src/main.rs | Uses current-thread Tokio runtime flavor for Lambda executor. |
| apps/backend/aws/executor/Cargo.toml | Switches to server-execute features and adjusts release opt-level. |
| apps/backend/aws/executor-ecs/src/main.rs | Adds ECS task executor that resolves/executes dispatch payloads. |
| apps/backend/aws/executor-ecs/Cargo.toml | Adds ECS executor package manifest. |
| apps/backend/aws/executor-async/src/main.rs | Uses current-thread Tokio runtime flavor for async executor lambda. |
| apps/backend/aws/executor-async/src/execution.rs | Uses shared DispatchPayload and ExecutionRequest::try_from(payload). |
| apps/backend/aws/executor-async/Cargo.toml | Switches to server-execute features and adjusts release opt-level. |
| apps/backend/aws/event-bridge/src/main.rs | Simplifies handler input type and drops CloudWatchEvent wrapper. |
| apps/backend/aws/event-bridge/Dockerfile | Updates build image + workspace copy strategy. |
| apps/backend/aws/event-bridge/Cargo.toml | Removes aws_lambda_events dependency and adjusts release opt-level. |
| apps/backend/aws/compiler-lambda/src/main.rs | Adds SQS-driven compiler lambda with deadline-aware behavior. |
| apps/backend/aws/compiler-lambda/Cargo.toml | Adds compiler lambda package manifest. |
| apps/backend/aws/compiler-ecs/src/main.rs | Adds ECS compiler task with remote job resolution and diagnostics. |
| apps/backend/aws/compiler-ecs/Cargo.toml | Adds compiler ECS package manifest. |
| apps/backend/aws/api/src/main.rs | Adds secret store (SSM+env), improves Sentry breadcrumb filtering, passes secret config into state. |
| apps/backend/aws/api/Cargo.toml | Enables SQS/Sentry/Secrets features and adjusts release opt-level. |
| Cargo.toml | Adds new workspace members and dependencies (compiler/secrets/etc.) and updates git deps. |
| CLAUDE.md | Adds developer onboarding + coding standards and critical rules. |
| .github/workflows/alpha-release.yml | Decodes Google/Firebase credentials for multiple OS runners. |
| .github/instructions/node-creation.instructions.md | Expands node creation guidance (permissions, pin naming, versioning). |
| .github/instructions/general-coding.instructions.md | Adds explicit guidance against destructive git operations. |
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| BestPractice | 3 medium |
| Security | 1 high |
| CodeStyle | 27 minor |
| Complexity | 69 medium |
🟢 Metrics 9664 complexity . 1928 duplication
Metric Results Complexity 9664 Duplication 1928
TIP This summary will be updated as you push new changes. Give us feedback
…ve embed detection logic and app category formatting
|
🚧 Skipped: PR exceeds review size limit. Please split into smaller PRs and re-run. |
| result = get_type_schema(type_name) | ||
| if result is not None: | ||
| return json.loads(result) | ||
| except Exception: |
| ] | ||
| return AIMessage(content=content, tool_calls=tool_calls) | ||
| return AIMessage(content=content) | ||
| except (json.JSONDecodeError, TypeError): |
|
|
||
| try: | ||
| exec(compile(code, "<flow_code>", "exec"), ns) | ||
| except SystemExit: |
| import types as _types | ||
| if origin is _types.UnionType: | ||
| return True | ||
| except AttributeError: |
…ization only occurs with valid keys
|
🚧 Skipped: PR exceeds review size limit. Please split into smaller PRs and re-run. |
- Added `#[default]` attribute to enum variants in `styles.rs`, removing the need for manual Default implementations. - Updated various enums including `PageSize`, `Orientation`, `TextAlignment`, `ParagraphStyle`, `ShapeType`, `PageNumberPosition`, and `Rotation`. - Improved code readability by using `if let` syntax for conditional checks in multiple files, including `chart_data_agent.rs`, `instantiate_widget.rs`, and others. - Enhanced error handling and logging in `aws.rs` and `sync_node_schema.rs`. - Cleaned up unnecessary code and improved performance in `execution.rs`, `context.rs`, and `node.rs`. - General code style improvements across multiple files for consistency and clarity.
|
🚧 Skipped: PR exceeds review size limit. Please split into smaller PRs and re-run. |
This pull request introduces several important improvements and new documentation for the flow-like project, focusing on enhanced developer guidance, stricter coding standards, and expanded backend capabilities. Notably, it adds a comprehensive developer onboarding guide (
CLAUDE.md), enforces critical node creation and pin naming rules, updates workspace dependencies to include new backend and library packages, and improves workflow automation for credential management.Documentation and Developer Guidance:
CLAUDE.mdonboarding guide covering repository structure, build commands, coding standards, Rust/TypeScript best practices, node creation rules, and code quality requirements.Workspace and Dependency Updates:
copilot-sdkandmarkitdownfor improved protocol support and bug fixes. [1] [2] [3]Backend and Build Improvements: