-
-
Notifications
You must be signed in to change notification settings - Fork 202
Refactor DevIndex data deployment strategy to prevent huge resource duplication #9290
Copy link
Copy link
Closed
Labels
aibuildBuild system, scripts, and configurationBuild system, scripts, and configurationenhancementNew feature or requestNew feature or request
Description
Problem
The apps/devindex/resources folder contains heavy data files, notably the users.jsonl file which is currently 22MB. When building the project for dist/development and dist/production environments, the build scripts (buildScripts/webpack/... and buildScripts/build/esmodules.mjs) blindly copy everything in the resources directory.
This leads to significant, unnecessary duplication of heavy data files in the deployment artifacts. Furthermore, the backend data factory needs a single source of truth (the dev branch data) to act upon.
Solution
- Move the data files (
*.json,*.jsonl) withinapps/devindex/resourcesinto a dedicateddata/subdirectory (apps/devindex/resources/data). - Update the backend data factory config (
apps/devindex/services/config.mjs) to point to the newdata/paths. - Update the
Contributors.mjsstore proxy URL to fetch data fromNeo.config.basePath + 'apps/devindex/resources/data/users.jsonl'. This ensures thatdistenvironments correctly resolve the path back to the original source folder, enabling them to pull data from the dev environment as the single source of truth. - Modify the resource copying logic in the build scripts (
webpack.config.appworker.mjsfor dev and prod, andesmodules.mjs) to explicitly remove thedatadirectory fromapps/devindex/resourcesafter the initial copy, preventing the heavy files from being deployed todist/.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
aibuildBuild system, scripts, and configurationBuild system, scripts, and configurationenhancementNew feature or requestNew feature or request