FAST-HEP overview

Workflows

FAST-HEP workflows start from an workflow.yaml, then compile to normalized workflow and execution-plan files.

        flowchart TD

    subgraph Compile["Compilation and planning"]
        Author["workflow.yaml"]
        Profiles["profiles and registries"]
        Normalised["normalised workflow"]
        Dependency["dependency inference"]
        Plan["execution plan"]

        Author --> Normalised
        Profiles --> Normalised
        Normalised --> Dependency
        Dependency --> Plan
    end

    subgraph Execute["Runtime execution"]
        Runtime["runtime execution"]
        Outputs["artifacts and outputs"]

        Runtime --> Outputs
    end

    Plan --> Runtime

classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef observer fill:#f3f0ff,stroke:#7048e8,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef plan fill:#d9ead3,stroke:#4f7d45,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;

    

Implementation packages contribute functionality through registry profiles. The workshop examples show how flow, carpenter, curator, and render profiles combine in an workflow file.

Packages

The FAST-HEP ecosystem is split into focused packages.

Each package owns a small part of the workflow stack and exposes functionality through public APIs, profiles, registries, or command-line tools.

        flowchart LR

    Flow["<b>fasthep-flow</b><br/>workflow compilation<br/>and orchestration"]

    Curator["<b>fasthep-curator</b><br/>dataset inspection<br/>metadata and provenance"]

    Carpenter["<b>fasthep-carpenter</b><br/>analysis transforms<br/>histograms and cutflows"]

    Render["<b>fasthep-render</b><br/>plots, reports<br/>and rendering"]

    CLI["<b>fasthep-cli</b><br/>command-line interface"]

    Workshop["<b>fasthep-workshop</b><br/>tutorials, examples<br/>and training material"]

    Flow --> Carpenter
    Curator --> Carpenter
    Carpenter --> Render
    CLI --> Flow
    Workshop --> Flow

classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef observer fill:#f3f0ff,stroke:#7048e8,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef plan fill:#d9ead3,stroke:#4f7d45,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;

    

Package	Purpose	Documentation	Git Repo
fasthep-flow	Workflow compilation, planning, runtime orchestration	API docs	FAST-HEP/fasthep-flow
fasthep-carpenter	Analysis transforms, histogramming, awkward/ROOT processing	API docs	FAST-HEP/fasthep-carpenter
fasthep-curator	Dataset inspection, provenance, diagnostics, metadata	API docs	FAST-HEP/fasthep-curator
fasthep-render	Plotting, reports, rendering pipelines	API docs	FAST-HEP/fasthep-render
fasthep-cli	Unified command-line interface	API docs	FAST-HEP/fasthep-cli
fasthep-toolbench	Shared utilities and lightweight tooling	API docs	FAST-HEP/fasthep-toolbench
fasthep-workshop	Tutorials, examples, and training material	—	FAST-HEP/fasthep-workshop
fasthep	Meta package and compatibility bundle	—	FAST-HEP/fasthep
fasthep-dev	Integration workspace and ecosystem validation	—	FAST-HEP/fasthep-dev