Energy-Aware Brokering for Scientific Workflows

Authors: Attila Farkas, Krisztián Póra, Andrei Tsaregorodtsev, Mazen Ezzeddine, Jaime Iglesias Blanco, Ignacio Heredia Cachá, Álvaro López García, Sebastian Gallenmüller, Krzysztof Dombek

Digital Research Infrastructures support many types of scientific workloads, from AI model development and interactive cloud environments to high-throughput computing, HPC simulations, and large-scale data transfers. As these workflows are increasingly distributed across multiple sites, their placement can influence not only performance and availability, but also environmental impact.

GreenDIGIT Milestone MS11 defines energy-aware brokering logic for scientific workflows. The aim is to use environmental signals, such as carbon intensity, energy efficiency, power usage effectiveness, water-related impact, and forecast data, to inform workload placement decisions. The approach is designed to complement existing workload management systems rather than replace them.

General Approach

The proposed brokering architecture follows a common pattern. A workflow is submitted, validated, evaluated against available infrastructure capacity, and then passed to an energy-aware brokering service. The broker combines workflow requirements, system state, and environmental indicators to produce either a placement decision or scheduling preferences for the underlying orchestrator.

Figure: The general brokering architecture connects workflow submission, validation, environmental metrics, forecasting, placement decisions, and orchestration.

Brokering across different workflow types

The general model is adapted to several workflow domains:

AI workflows: Integrated with the AI4EOSC platform, where workloads are represented as containerized Nomad jobs. The GreenDirector component converts environmental indicators, such as Green Score, into affinity scores that influence Nomad’s placement decisions while preserving its existing feasibility checks.

Figure: Environmental signals are converted into affinity scores that influence AI workload placement.

Cloud and container workflows: Focused on interactive services such as Jupyter notebooks and RStudio sessions, which require immediate provisioning and uninterrupted execution. The broker evaluates site capacity over the requested execution window and can apply policies such as minimum-carbon placement or weighted balancing between carbon intensity and utilization.
HTC workflows: GreenDIGIT extends DIRAC’s pilot-based scheduling model. Instead of submitting pilots to sites in a random order, the GreenSiteDirector ranks sites based on a Site GreenScore. This score reflects how much useful computation can be delivered per unit of carbon footprint, helping prioritize more environmentally efficient sites.

Figure: Environmental metrics from EIMPS are used to guide pilot submission in DIRAC.

HPC workflows: the milestone focuses on estimating power consumption before execution. Component-level models for CPUs, GPUs, storage, and networking help predict the energy demand of large, distributed workloads.
Network workflows: Concerned with the environmental and operational cost of data movement. Carbon cost, energy cost, load, and timing can be combined to support path selection or time-aware transfer scheduling.

Outlook

MS11 connects brokering with other GreenDIGIT components, including environmental monitoring from Task T6.1, forecasting from T6.2, and Wattnet-based carbon and water indicators from T6.5. This allows sustainability metrics to be used not only for reporting, but also as operational inputs for scheduling decisions.

The next step is validation and deployment in Work Package 7 (WP7), where the proposed brokering strategies will be tested in operational settings. The expected contribution is a practical shift from environmental measurement toward environment-aware operation of Digital Research Infrastructures.