• 

• 

• 

• 

• GIF

• 

Inspiration

Are autism and neuro-divergence mental deficiencies?

Clinical therapy often treats it like so, but we believe otherwise. The challenges with clinical Applied Behavioural Analysis (ABA) for autism in the status quo:

• Lack of meaningful social engagement with peers.
• Authority, taskmaster-based model for compliance.
• Specialists are uncommon and inaccessible.
• Behavioral improvements are focused on conformity, not individuality.

LEGO-based behavioral therapy is a clinically-studied and gamified approach to social competence therapy. It assigns social roles and encourages interactive gameplay during sessions, to increase intrinsic motivation, foster authentic interaction with peers, and reduce stress/anxiety surrounding therapy. Teaching communication and self-regulation doesn’t have to affect the intrinsic, unique qualities of our kids.

We wanted to find a way to improve and scale this idea, maximizing the creativity and independence of our next generation.

What it does

The generator intakes images (i.e. children’s drawings) and outputs a 3D, layer-by-layer LEGO build with instructions suitable for behavioural exercises. Each instruction set contains separate requirements for a guide, supplier, and builder role in the LEGO construction.

How we built it

Front-end: React.js, Tailwind CSS, NeatJS, Three.js

Back-end: Python (Flask API, Gemini API, Shap-E, Modal)

Challenges we ran into

• Determining a sort algorithm to allocate LEGO blocks for our build (eg. greedy algo, different bounding mechanisms).
• Colorizing our LEGO designs after voxelization.
• Connecting our pipeline: frontend interfaces, backend APIs, and AI model workflows.

Accomplishments that we're proud of

• Developing a brick-placement algorithm leveraging border detection and structural integrity principles.
• Integrating Gemini into our workflow, enabling dynamic AI-generated content.
• Trouble-shooting errors in the “pipeline”, being resilient de-buggers.
• End-to-end system integration, managing API redirects and making generated outputs (files, text from Gemini, etc.) accessible within the application.

What we learned

• Conversion algorithms: color transfer to voxelization and mesh to voxel algorithms.
• Generative 3D model architectures (Shap-E): including latent diffusion models and implicit neural representations.
• How to work with new 3D file formats, including .obj, .ldr, and .stl.

What's next for LegoFIKS

• Expanding build pieces (incl. LEGO Library, custom pieces), fine-tuning structures.
• Generating instructions for single/multi-builder modes with behavioural principles.
• Integrating further clinician insights: creating simultaneous platforms for school/home users and licensed healthcare professionals to work in conjunction.

Built With

• gemini-api
• neatjs
• react.js
• shap-e
• tailwind-css
• three.js-back-end:-python-(flask-api