Feature/520 short human in the loop#523
Conversation
- Introduced interaction types (single choice, multiple choice, form) and their respective schemas. - Implemented interaction request processing in the chat component. - Enhanced chat UI to display interactions alongside messages. - Created a new Interaction component for rendering interaction requests. - Added countdown functionality for pending interactions. - Updated event processor to handle interaction events. - Refactored chat component to utilize new interaction logic. - Added necessary types and interfaces for interactions in the schema.
…andling in chat components
- Introduced a comprehensive styling guide covering theme variables, component styles, spacing, sizing, borders, shadows, responsive design, typography, flexbox utilities, common component patterns, animation classes, z-index, overflow, position, and custom CSS injection. - Emphasized the use of theme variables over hardcoded colors for better dark/light mode support. - Provided examples and best practices for implementing styles in A2UI components.
Summary of ChangesHello @felix-schultz, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a robust human-in-the-loop interaction system, enabling dynamic user engagement within workflows. It establishes the foundational database models, API endpoints with real-time event streaming, and client-side components necessary for both desktop and web applications to facilitate these interactions. The changes also encompass significant dependency updates, authentication enhancements, and general code cleanup, contributing to a more interactive and efficient system. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
| | { type: "interaction"; data: IInteractionRequest; timestamp: number } | ||
| | { type: "interaction-group"; data: IInteractionRequest[]; timestamp: number }; | ||
|
|
||
| function getInteractionCreatedAt(interaction: IInteractionRequest): number { |
| import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "../../ui/select"; | ||
| import { Switch } from "../../ui/switch"; | ||
| import { Textarea } from "../../ui/textarea"; | ||
| import { AlertCircle, CheckCircle2, ChevronDown, ChevronRight, ChevronUp, Clock, Plus, Trash2, XCircle } from "lucide-react"; |
There was a problem hiding this comment.
Code Review
This is a substantial and well-executed pull request that introduces a powerful human-in-the-loop (HITL) interaction system. The changes span the entire stack, from the database schema and backend APIs to the frontend components and execution logic. The new interaction nodes in the catalog are a great addition, and the implementation of both local (polling) and remote (SSE) interaction handling is well-thought-out. The refactoring of the SSE parsing and the DynamicCompletionModel are also significant improvements to code quality and safety. My review includes a few suggestions for improving robustness and maintainability.
| let created_json = serde_json::to_string(&created_payload).unwrap_or_default(); | ||
| yield Ok(Event::default().event("created").data(created_json)); |
There was a problem hiding this comment.
Using unwrap_or_default() on serde_json::to_string can hide potential serialization errors, which would result in an empty string being sent over SSE. While serialization of SseCreatedPayload is unlikely to fail, it's safer to handle the Result explicitly. Logging the error would be helpful for debugging if such an issue ever occurs. This also applies to line 235.
let created_json = match serde_json::to_string(&created_payload) {
Ok(json) => json,
Err(e) => {
tracing::error!("Failed to serialize SseCreatedPayload: {}", e);
String::new()
}
};
yield Ok(Event::default().event("created").data(created_json));| @@ -0,0 +1,1210 @@ | |||
| use flow_like::flow::{ | |||
There was a problem hiding this comment.
This file is over 1200 lines long and contains a lot of distinct logic: schema generation from pins, extensive type coercion functions, node definition, and a large test suite. For better maintainability and readability, consider splitting this file into smaller, more focused modules. For example, you could have schema_builder.rs, coercion.rs, and node.rs within an interaction/form/ directory.
| pub async fn submit_interaction_response(interaction_id: &str, value: Value) -> bool { | ||
| let mut responses = INTERACTION_RESPONSES.lock().await; | ||
| if responses.contains_key(interaction_id) { | ||
| return false; | ||
| } | ||
| responses.insert(interaction_id.to_string(), value); |
There was a problem hiding this comment.
There's a potential race condition here. The check if responses.contains_key(interaction_id) and the subsequent responses.insert(...) are not an atomic operation. If two threads call this function concurrently for the same interaction_id, both could pass the contains_key check before either inserts the value, leading to one response being overwritten. Using the entry API would make this operation atomic and more robust.
| pub async fn submit_interaction_response(interaction_id: &str, value: Value) -> bool { | |
| let mut responses = INTERACTION_RESPONSES.lock().await; | |
| if responses.contains_key(interaction_id) { | |
| return false; | |
| } | |
| responses.insert(interaction_id.to_string(), value); | |
| pub async fn submit_interaction_response(interaction_id: &str, value: Value) -> bool { | |
| let mut responses = INTERACTION_RESPONSES.lock().await; | |
| if let std::collections::hash_map::Entry::Vacant(e) = responses.entry(interaction_id.to_string()) { | |
| e.insert(value); | |
| } else { | |
| // Already responded | |
| return false; | |
| } | |
| drop(responses); |
| const id = `schema-${path.join("-")}`; | ||
| const label = property.title ?? path[path.length - 1] ?? "field"; | ||
| const description = property.description; | ||
| const required = false; |
There was a problem hiding this comment.
The required variable is hardcoded to false inside renderSchemaField. This means that for forms generated from a JSON schema, the UI will not display the required field indicator (asterisk) next to the label, even if the schema specifies that a field is required. The missingRequired logic in FormInteractionForm correctly identifies required fields, but this isn't reflected visually at the field level. The isRequired status should be passed down as a prop to renderSchemaField.
There was a problem hiding this comment.
Pull request overview
This pull request introduces comprehensive support for human-in-the-loop interactions across the FlowLike platform, enabling workflows to pause and request user input through single choice, multiple choice, and form-based interactions. The implementation spans backend infrastructure, desktop/web frontends, and node execution logic.
Changes:
- Added
Interactiondatabase model with SSE-based lifecycle management and JWT-secured response endpoints - Implemented three interaction node types (single choice, multiple choice, form) with JSON Schema-based form generation
- Enhanced chat interface to display and handle interaction requests with countdown timers and response submission
- Added Tauri command for desktop interaction responses and improved SSE parsing for multi-line data fields
Reviewed changes
Copilot reviewed 68 out of 71 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
packages/api/prisma/schema/interaction.prisma |
Defines lean interaction model with status tracking and expiration indices |
packages/api/src/routes/interaction.rs |
SSE streaming endpoint for interaction lifecycle and JWT-secured response handler |
packages/api/src/execution/interaction_jwt.rs |
JWT signing/verification for interaction responders |
packages/catalog/data/src/interaction/form.rs |
Form interaction node with JSON Schema generation from callback function pins |
packages/catalog/data/src/interaction/single_choice.rs |
Single choice interaction with freeform option support |
packages/catalog/data/src/interaction/multiple_choice.rs |
Multiple choice with min/max selection constraints |
packages/ui/components/interfaces/chat-default/interaction.tsx |
React components for rendering and responding to interactions |
packages/ui/components/interfaces/chat-default.tsx |
Integration of interaction state and response handling |
apps/desktop/src-tauri/src/functions/interaction.rs |
Tauri command for submitting interaction responses |
apps/web/lib/web-states/board-state.ts |
Fixed SSE parsing for multi-line data fields |
packages/executor/src/types.rs |
Removed unused token field from execution requests |
Comments suppressed due to low confidence (7)
thirdparty/frontend-builder/claude/flow-like-ui/SKILL.md:1
- Corrected spelling of 'recieve' to 'receive' in flow-like-ui/references/styling-guide.md
packages/ui/hooks/use-theme-gradient.ts:1 - Converting iterables to arrays with
Array.from()is unnecessary when iterating with for-of loops, as they work directly with iterables. The original code was correct.
packages/model-provider/src/llm.rs:1 - Changing
dynamic_modelfrom&selftoselfbreaks the API by consuming theModelConstructor. This prevents reusing the same constructor for multiple models, which is a common use case.
packages/api/src/routes/chat/completions.rs:1 - Switching from
sub()toexecutor_scoped_sub()allows Executor JWTs to invoke LLM endpoints. Verify this is intentional and that executor tokens have appropriate usage tracking/billing, as they bypass normal user auth.
packages/api/src/middleware/jwt.rs:1 - The
tracking_idmethod now accepts Executor JWTs viaexecutor_scoped_sub(). Ensure usage tracking correctly attributes executor-initiated requests, as they may have different billing implications than user-initiated requests.
packages/catalog/data/src/interaction/wait.rs:1 - The function returns an empty string if
execution_cacheis None, which may cause unclear errors downstream. Consider returning a Result with a descriptive error message like "No app_id available in execution context for remote interaction".
packages/catalog/data/src/interaction/form.rs:1 - The warning about multiple callback functions is logged but not visible to users who didn't enable debug logging. Consider adding a clearer indicator in the UI or returning this as part of the validation result.
This pull request introduces support for human-in-the-loop interactions in both the backend and frontend, along with several dependency and configuration updates. The main changes include the addition of a new
Interactionmodel in the Prisma schema, backend and Tauri API wiring for responding to interactions, improvements to SSE parsing in the web frontend, and dependency upgrades and feature enhancements across multiple crates.Interaction model and backend support:
Interactionmodel andInteractionStatusenum to the Prisma schema, representing lean, ephemeral human-in-the-loop interaction records, and created relations toAppandUsermodels. [1] [2] [3]respond_to_interactionininteraction.rsto allow desktop clients to respond to interactions, and registered this command in the Tauri app. [1] [2] [3]Frontend improvements:
WebBoardStateandWebEventStateto properly handle multi-linedata:fields, fixing potential issues with event data integrity. [1] [2]Dependency and configuration updates:
tracing-subscriberto version0.3.22and standardized its usage via workspace features in several crates. [1] [2] [3] [4]lettrecrate to includebuilder,hostname,pool, andsmtp-transport.remoteorlocalfeature toflow-like-catalogdependencies in multiple backend and desktop crates to enable new catalog capabilities. [1] [2] [3] [4] [5] [6]Code cleanup and refactoring:
tokenfields from job processing and execution logic in backend runtimes, streamlining the job payload structure. [1] [2] [3]