Carbon Efficient Computing

Schedule High Performance Computing, but keep it green!

Comment

Inspiration

The University of Virginia’s CY2023 Greenhouse Gas Inventory Report reveals a stark truth: purchased electricity is the leading driver of UVA’s Scope 2 emissions. With UVA striving for carbon neutrality by 2030 and compute-heavy research in AI and Data Science surging, a critical conflict arises. We cannot halt researchers’ innovation, but we must reduce the environmental cost of their computations. Inspired by modern academic research on “Temporal Life Cycle Assessment” (LCA)—particularly the work of Shoaib-ul-Hasan et al., who demonstrated that relying on static, annual carbon averages is misleading and obscures massive opportunities for emissions reductions—we realized that when we compute, matters just as much as how much we compute. Because the power grid’s energy mix (solar vs. natural gas vs. coal) fluctuates drastically throughout the day, a job run at 3:00 PM might emit twice the carbon of the exact same job run at 3:00 AM. Drawing on further 2022 research by Portolani et al. that successfully forecasted these hourly LCA impacts using dynamic grid data, we knew we could proactively predict these “green windows.” We wanted to build a system that passively decarbonizes research clusters by treating time as a key sustainability variable.

What it does

Our project is a Carbon-Aware High-Performance Computing (HPC) Scheduler. It is designed to reduce greenhouse gas emissions by proactively aligning computational workloads with periods of lower carbon intensity on the local energy grid. Instead of treating environmental impact as a static afterthought, our system evaluates emissions by intensity over time. Mathematically, the estimated carbon emissions \( E \) for a job can be modeled as the integral of the power consumption \( P \) multiplied by the grid’s carbon intensity \( I(t) \) over the job’s duration \( d \):

$$E = \int_{t_{start}}^{t_{start} + d} P \cdot I(t) , dt$$

By leveraging this, the scheduling engine dynamically adjusts \( t_{start} \) to minimize \( E \) without sacrificing system throughput. Carbon Estimation: The system assigns jobs a carbon intensity score based on hardware usage and runtime.

Grid Forecasting: It integrates 48-hour-ahead electricity forecast data to identify low-carbon “green windows.”

Intelligent Scheduling: High-impact, flexible jobs are proactively shifted into periods with high renewable energy availability.

Stateful Queueing: The scheduler respects cluster capacity limits. If the optimal window is fully booked, it evaluates and assigns the second-best window, or places the job on a waitlist.

Transparent Accountability: Users receive a dashboard showing the “Baseline vs. Optimized” carbon footprint of their specific computations.

How we built it

We designed our system as an extension to existing HPC scheduling frameworks, with a focus on compatibility with widely used tools such as SLURM. The architecture consists of three decoupled components: Frontend: We built a responsive dashboard using Next.js, React, and Tailwind CSS, hosted live on Vercel. This allows researchers to submit workloads, specify job flexibility/complexity, and view their carbon savings in real-time.

State: We used a Supabase PostgreSQL database to act as the central nervous system. It tracks the state machine of every job (QUEUED, SCHEDULED, RUNNING, COMPLETED) and handles concurrent requests.

Scheduling: We wrote a Python backend worker that acts as the cluster manager. It constantly polls the database, pulls in real hourly grid carbon intensity data, and executes our temporal LCA mathematical models to find the optimal execution window before pushing the schedule back to the cloud.

Challenges we ran into

Interpreting Complex Grid Data: We had to deeply analyze marginal carbon intensity, especially understanding how to mathematically handle negative grid values and how they should influence our scheduling engine’s logic.

Synthetic Workloads: Obtaining realistic workload data is difficult since real HPC job logs are highly private. We had to design synthetic jobs (small, medium, heavy) that accurately reflect real-world researcher behavior.

Stateful Scheduling & Concurrency: Moving from a simple “calculator” to a stateful queue was incredibly difficult. We had to implement logic that locks time windows, shifts jobs to a “second-best” window if capacity is reached, and runs a real-time compressed simulation loop to move jobs from RUNNING to COMPLETED.

Team Constraints: Right before the core development phase, half of our team (2 members) had to drop out due to exams. We had to rapidly adapt, redistribute responsibilities, and focus strictly on the most impactful features to deliver a finished MVP in half the time.

Accomplishments that we’re proud of

We built a mathematically sound, fully functional temporal LCA carbon estimator that moves beyond theoretical sustainability into actual, operational load-shifting. We successfully implemented a real-time simulation engine that handles queueing, concurrency limits, and capacity-aware scheduling. We maintained a clean separation of concerns (decoupled the frontend and backend), allowing us to deploy a live, interactive web app with built-in explainability. We overcame a 50% reduction in team size to deliver a complete, highly polished project that directly addresses our 2030 carbon-neutrality goals.

What we learned

We learned that carbon efficiency is not just about reducing energy usage, but about timing. We gained a deep understanding of marginal carbon intensity and temporal variance. On the engineering side, we learned how to bridge Python data-science logic with modern TypeScript web frameworks using a continuous polling worker and a Supabase bridge. Finally, we learned the value of resilience and aggressive prioritization when facing sudden team constraints during a hackathon.

What’s next for Carbon Efficient Computing

We plan to replace our static datasets with live, real-time APIs (such as WattTime or Electricity Maps) to enable fully dynamic decision-making. We also want to expand our integration to track precise GPU and memory power draws for advanced Machine Learning workloads, and introduce predictive ML models to forecast future carbon intensity.

_Administrative Note for Judges & MLH: Please note that due to academic obligations (exams), two of our originally registered team members had to drop out of the hackathon, and we have officially notified MLH and moved forward with their permission. The University of Virginia’s CY2023 Greenhouse Gas Inventory Report reveals a stark truth: purchased electricity is the leading driver of UVA’s Scope 2 emissions. With UVA striving for carbon neutrality by 2030 and compute-heavy research in AI and Data Science surging, a critical conflict arises. We cannot halt researchers’ innovation, but we must reduce the environmental cost of their computations.

Inspired by modern academic research on “Temporal Life Cycle Assessment” (LCA)—particularly the work of Shoaib-ul-Hasan et al., who demonstrated that relying on static, annual carbon averages is misleading and obscures massive opportunities for emissions reductions—we realized that when we compute, matters just as much as how much we compute. Because the power grid’s energy mix (solar vs. natural gas vs. coal) fluctuates drastically throughout the day, a job run at 3:00 PM might emit twice the carbon of the exact same job run at 3:00 AM. Drawing on further 2022 research by Portolani et al. that successfully forecasted these hourly LCA impacts using dynamic grid data, we knew we could proactively predict these “green windows.” We wanted to build a system that passively decarbonizes research clusters by treating time as a key sustainability variable… Additionally, our GitHub contributor graph shows extra accounts because two members accidentally committed from their school-specific accounts (Devinche is the same as dsingh-umd, and rnshaalr is the same as EnshaalRajput). This project was built entirely by our active, remaining team._

Share this project:

Updates