Skip to content

Refactor DevIndex data deployment strategy to prevent huge resource duplication #9290

@tobiu

Description

@tobiu

Problem

The apps/devindex/resources folder contains heavy data files, notably the users.jsonl file which is currently 22MB. When building the project for dist/development and dist/production environments, the build scripts (buildScripts/webpack/... and buildScripts/build/esmodules.mjs) blindly copy everything in the resources directory.

This leads to significant, unnecessary duplication of heavy data files in the deployment artifacts. Furthermore, the backend data factory needs a single source of truth (the dev branch data) to act upon.

Solution

  1. Move the data files (*.json, *.jsonl) within apps/devindex/resources into a dedicated data/ subdirectory (apps/devindex/resources/data).
  2. Update the backend data factory config (apps/devindex/services/config.mjs) to point to the new data/ paths.
  3. Update the Contributors.mjs store proxy URL to fetch data from Neo.config.basePath + 'apps/devindex/resources/data/users.jsonl'. This ensures that dist environments correctly resolve the path back to the original source folder, enabling them to pull data from the dev environment as the single source of truth.
  4. Modify the resource copying logic in the build scripts (webpack.config.appworker.mjs for dev and prod, and esmodules.mjs) to explicitly remove the data directory from apps/devindex/resources after the initial copy, preventing the heavy files from being deployed to dist/.

Metadata

Metadata

Assignees

Labels

aibuildBuild system, scripts, and configurationenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions