What is Pegasus AI
PegasusAI is the next-generation AI-driven scientific workflow management system developed by a multi-institutional team funded by the U.S. National Science Foundation. Built on the proven foundation of the Pegasus Workflow Management System, PegasusAI introduces artificial intelligence to improve automation, fault detection, performance prediction, and user guidance across the computing continuum—from edge devices to exascale high-performance computing systems.
Key features of PegasusAI:
- Intelligent Resource Planning: Uses machine learning models to predict resource needs and optimize workflow execution.
- Adaptive Workflow Management: Detects anomalies and performance issues in real time, automatically adjusting plans or alerting users.
- Human-in-the-Loop Design: Guides researchers through AI-augmented tools for workflow creation, monitoring, and debugging.
- Scalable Across CI: Supports execution on HPC, cloud, and edge platforms, enabling flexible deployment and broad applicability.
- AI-Ready Data Generation: Provides curated datasets and trained models to advance AI for scientific computing and cyberinfrastructure research.
Pegasus AI within the Workflow Lifecycle
Technical Approach
-
Hierarchical AI Agents
Integrate learning agents at task and workflow levels to support local and global optimization.
-
Hybrid Learning Models
Combine supervised learning (e.g., from historical workflow traces), reinforcement learning (e.g., adaptive decision-making), and online learning (e.g., responding to runtime dynamics).
-
Runtime Monitoring & Feedback Loops
Collect real-time execution metrics and anomalies to update models and guide decision-making during workflow runs.
-
Failure Prediction & Resilience Strategies
Anticipate resource failures, task slowdowns, or system bottlenecks using predictive modeling and trigger intelligent re-planning or retries.
-
Adaptive Scheduling
Improve resource matching and task prioritization based on learned execution profiles and workload characteristics.
-
Workflow-Level Summarization
Provide interpretable insights into why the system took specific actions, supporting human-in-the-loop
decision-making. -
CI-Ready Design
Implement within Pegasus's modular architecture to ensure compatibility with the NSF CI ecosystem.
Access to National CI
ACCESS
ACCESS is a program established and funded by the National Science Foundation to help U.S. -based researchers and educators, with or without supporting grants, to utilize the nation's advanced computing systems and services - at no cost. Single entry point for over 20 compute, cloud, storage, and networking systems.
PATh and OSG Open Science Pool
The PATh Facility, funded by the NSF Office of Advanced Cyberinfrastructure, provides free access to large-scale distributed High Throughput Computing systems for US-based open science projects.
Pegasus AI in Context
What is Pegasus WMS
Pegasus is a system for mapping and executing abstract application workflows over a range of execution environments.
The same abstract workflow can, at different times, be mapped to different execution environments such as ACCESS, PATh, OSG OS Pool, commercial and academic clouds, campus grids, and clusters.
Pegasus can easily scale both the size of the workflow, and the resources that the workflow is distributed over. Pegasus runs workflows ranging from just a few computational tasks up to 1 million.
End-to-end automatic checksumming of workflow data to ensure data integrity.
Stores static and runtime metadata associated with workflow, files and tasks. Accessible via command-line tools and a web-based dashboard.