Skip to content

R1nneC0de/ModelMaestro

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

20 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Agentic Model Training Platform

An autonomous AI system that takes raw data and problem descriptions, then automatically develops, trains, and deploys machine learning models using Google's Gemini and Vertex AI.

🎯 Hackathon Version

This is a simplified version focused on core ML pipeline functionality:

  • βœ… Automated problem analysis
  • βœ… Data processing and labeling
  • βœ… Model selection and training
  • βœ… Real-time progress tracking
  • βœ… Model deployment and testing

πŸ—οΈ Project Structure

.
β”œβ”€β”€ backend/                 # FastAPI backend
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ api/            # API endpoints
β”‚   β”‚   β”œβ”€β”€ core/           # Config and core utilities
β”‚   β”‚   β”œβ”€β”€ schemas/        # Pydantic data models
β”‚   β”‚   β”œβ”€β”€ services/       # Business logic
β”‚   β”‚   β”‚   β”œβ”€β”€ agent/      # AI agent components
β”‚   β”‚   β”‚   └── cloud/      # GCP integrations
β”‚   β”‚   └── utils/          # Helper utilities
β”‚   β”œβ”€β”€ tests/              # Backend tests
β”‚   └── pyproject.toml      # Python dependencies
β”‚
β”œβ”€β”€ frontend/               # React frontend
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ components/     # React components
β”‚   β”‚   β”œβ”€β”€ pages/          # Page components
β”‚   β”‚   β”œβ”€β”€ hooks/          # Custom hooks
β”‚   β”‚   β”œβ”€β”€ services/       # API client
β”‚   β”‚   └── contexts/       # React contexts
β”‚   └── package.json        # Node dependencies
β”‚
β”œβ”€β”€ docs/                   # Documentation
β”‚   └── guides/             # Setup guides
β”‚
└── docker-compose.yml      # Local development setup

πŸš€ Quick Start

Prerequisites

  • Python 3.11+
  • Node.js 18+
  • Docker & Docker Compose
  • Google Cloud account with billing enabled

1. Google Cloud Setup

Follow the detailed guide: docs/guides/google-cloud-setup.md

Quick version:

# Enable required APIs
gcloud services enable aiplatform.googleapis.com storage.googleapis.com

# Create service account and download key
gcloud iam service-accounts create agentic-platform-sa
gcloud iam service-accounts keys create gcp-key.json \
  --iam-account=agentic-platform-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com

# Move key to credentials directory
mkdir -p gcp-credentials
mv gcp-key.json gcp-credentials/

2. Environment Configuration

# Copy example env file
cp .env.example .env

# Edit .env and update:
# - GOOGLE_CLOUD_PROJECT
# - GCS_BUCKET_NAME
# - GEMINI_API_KEY

3. Start Development Environment

# Start all services
docker-compose up

# Backend will be available at http://localhost:8000
# Frontend will be available at http://localhost:3000

4. Install Backend Dependencies

cd backend
poetry install

πŸ› οΈ Development

Backend

cd backend

# Install dependencies
poetry install

# Run development server
poetry run uvicorn app.main:app --reload

# Run tests
poetry run pytest

# Format code
poetry run black .
poetry run ruff check .

Frontend

cd frontend

# Install dependencies
npm install

# Run development server
npm run dev

# Run tests
npm test

# Build for production
npm run build

πŸ“‹ Implementation Status

See .kiro/specs/agentic-model-training-platform/tasks.md for detailed task list.

Completed:

  • βœ… Project infrastructure setup
  • βœ… Docker configurations
  • βœ… Environment configuration
  • βœ… GCS storage structure

In Progress:

  • πŸ”„ GCS storage manager and Pydantic schemas
  • πŸ”„ Core agent components
  • πŸ”„ API endpoints
  • πŸ”„ Frontend UI

🎯 Core Features

  1. Automated Problem Analysis: Upload data and describe your problem - the system analyzes and determines the best ML approach
  2. Smart Data Processing: Automatic data validation, cleaning, and feature engineering
  3. Intelligent Labeling: Zero-shot labeling for unlabeled datasets using Gemini
  4. Model Selection: AI-powered model architecture and hyperparameter recommendations
  5. Automated Training: Hands-off training on Vertex AI with progress monitoring
  6. Real-time Updates: WebSocket-based live progress tracking
  7. Easy Deployment: One-click model deployment with API endpoints
  8. Interactive Testing: Test your models directly in the UI

πŸ”§ Tech Stack

Backend:

  • FastAPI (Python web framework)
  • Google Cloud Storage (Metadata and file storage)
  • Celery + Redis (Task queue)
  • Google Vertex AI (Model training)
  • Google Gemini (AI agent)
  • Pydantic (Data validation)

Frontend:

  • React 18 + TypeScript
  • Material-UI (Components)
  • React Query (Data fetching)
  • React Router (Routing)
  • Vite (Build tool)

Infrastructure:

  • Docker + Docker Compose
  • Google Cloud Storage
  • Google Cloud Logging

πŸ“ API Documentation

Once the backend is running, visit:

πŸ› Troubleshooting

Service Connection Issues

# Check if services are running
docker-compose ps

# View logs
docker-compose logs backend
docker-compose logs redis

GCP Authentication Issues

# Verify credentials file exists
ls -la gcp-credentials/gcp-key.json

# Test authentication
gcloud auth activate-service-account --key-file=gcp-credentials/gcp-key.json

Port Already in Use

# Stop all containers
docker-compose down

# Check what's using the port
lsof -i :8000  # or :3000 for frontend

πŸ“š Additional Resources

🀝 Contributing

This is a hackathon project. Feel free to fork and experiment!

πŸ“„ License

MIT License - feel free to use this for your own projects.

About

Automating end-to-end ML pipeline

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors