๐Ÿ“ฆ PedramNavid / petit

Minimal lightweight task orchestrator in Rust

โ˜… 121 stars โ‘‚ 8 forks ๐Ÿ‘ 121 watching
๐Ÿ“ฅ Clone https://github.com/PedramNavid/petit.git
HTTPS git clone https://github.com/PedramNavid/petit.git
SSH git clone git@github.com:PedramNavid/petit.git
CLI gh repo clone PedramNavid/petit
Pedram Navid Pedram Navid chore: add rustfmt hook for automatic formatting 170dee4 11 days ago ๐Ÿ“ History
๐Ÿ“‚ main View all commits โ†’
๐Ÿ“ .cargo
๐Ÿ“ .claude
๐Ÿ“ .github
๐Ÿ“ benches
๐Ÿ“ dist
๐Ÿ“ examples
๐Ÿ“ migrations
๐Ÿ“ src
๐Ÿ“ tests
๐Ÿ“„ .gitignore
๐Ÿ“„ api.md
๐Ÿ“„ Cargo.lock
๐Ÿ“„ Cargo.toml
๐Ÿ“„ CLAUDE.md
๐Ÿ“„ Makefile
๐Ÿ“„ plan.md
๐Ÿ“„ README.md
๐Ÿ“„ README.md

Petit

A minimal, lightweight task orchestrator with cron-like scheduling, written in Rust.

What .. and why?

I wanted something to orchestrate simple tasks on embedded hardware (Turing PI cluster with RK1 and Raspberry PIs). Low-memory and efficiency were important, but I also wanted something that was usable and friendly.

Importantly, I needed

  • Tasks and Jobs that can form dependencies
  • Simple observability
  • Concurrency
  • YAML configuration
And so I built this.

Should I use this?

No! I built this to see what it would take to build an orchestrator with Claude Code using Opus 4.5. It works for me but please do not use it for you.

Features

  • DAG-based execution โ€” Define task dependencies; independent tasks run in parallel
  • Cron scheduling โ€” Standard cron expressions with timezone support
  • YAML configuration โ€” Define jobs declaratively
  • Retry policies โ€” Configurable retries with fixed delays
  • Conditional execution โ€” Run tasks based on upstream success/failure
  • Cross-job dependencies โ€” Jobs can depend on other jobs
  • Event system โ€” Subscribe to lifecycle events (task started, completed, failed, etc.)
  • HTTP REST API โ€” Optional API server for monitoring and control (enabled by default)
  • Pluggable storage โ€” In-memory (default) or SQLite for persistence
  • Concurrency control โ€” Limit parallel tasks and concurrent job runs

Feature Flags

  • api โ€” Enables HTTP REST API and related CLI flags (--no-api, --api-port, --api-host). Enabled by default.
  • sqlite โ€” Enables SQLite storage backend and --db CLI flag
  • tui โ€” Enables terminal UI dashboard (includes sqlite)

Installation

# Default installation (includes API feature)
cargo install --path .

# With SQLite support
cargo install --path . --features sqlite

# With TUI support
cargo install --path . --features tui

# Minimal installation (no API, no SQLite)
cargo install --path . --no-default-features

Quick Start

There are example jobs in ./examples. You can use those or follow these instructions:

1. Create a job file

# jobs/hello.yaml
id: hello_world
name: Hello World Job

tasks:
  - id: greet
    type: command
    command: echo
    args: ["Hello, World from Petit!"]

2. Run the scheduler

pt run jobs/

3. Or trigger a job manually

pt trigger jobs/ hello_world

4. Run with a database

By default, this runs in-memory with no persistence!

pt trigger jobs/ hello_world --db petit.db

CLI Commands

pt run <jobs-dir>       # Run scheduler with jobs from directory
pt validate <jobs-dir>  # Validate job configurations
pt list <jobs-dir>      # List all jobs
pt trigger <jobs-dir> <job-id>  # Trigger a job manually

Options for run

FlagDescription
-j, --max-jobs <N>Maximum concurrent jobs (default: unlimited)
-t, --max-tasks <N>Maximum concurrent tasks per job (default: 4)
--tick-interval <N>Scheduler tick interval in seconds (default: 1)
--db <PATH>Path to SQLite database file (requires sqlite feature)
--no-apiDisable HTTP API server (requires api feature, enabled by default)
--api-port <PORT>API server port (default: 8565, requires api feature)
--api-host <HOST>API server host (default: 127.0.0.1, requires api feature)
Note: API-related flags (--no-api, --api-port, --api-host) are only available when the api feature is enabled (which is the default). If you compile with --no-default-features, these flags will not be available and no API server will start.

Job Configuration

Jobs are defined in YAML files:

id: data_pipeline
name: Data Pipeline
schedule: "0 0 2 * * *" # 2 AM daily (6-field cron: sec min hour day month weekday)
enabled: true
max_concurrency: 1

config:
  batch_size: 1000
  output_dir: /tmp/data

tasks:
  - id: extract
    type: command
    command: python
    args: ["scripts/extract.py"]
    environment:
      STAGE: extract

  - id: transform
    type: command
    command: python
    args: ["scripts/transform.py"]
    depends_on: [extract]
    condition: all_success
    retry:
      max_attempts: 3
      delay_secs: 10
      condition: always

  - id: notify
    type: command
    command: echo
    args: ["Pipeline done"]
    depends_on: [transform]
    condition: all_done # Runs regardless of upstream success/failure

Task Conditions

ConditionDescription
alwaysRun if dependencies completed (default)
all_successRun only if all upstream tasks succeeded
on_failureRun only if at least one upstream task failed
all_doneRun after dependencies complete, regardless of status

Retry Configuration

retry:
  max_attempts: 3 # Number of retries (0 = no retries)
  delay_secs: 5 # Fixed delay between attempts
  condition: always # always | transient_only | never

Cross-Job Dependencies

id: downstream_job
name: Downstream Job
depends_on:
  - job_id: upstream_job
    condition: last_success # last_success | last_complete | within_window

Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                        CLI                              โ”‚
โ”‚              run | validate | list | trigger            โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                          โ”‚
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                     Scheduler                           โ”‚
โ”‚         โ€ข Cron scheduling    โ€ข Job dependencies         โ”‚
โ”‚         โ€ข Pause/resume       โ€ข Concurrency control      โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                          โ”‚
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    DagExecutor                          โ”‚
โ”‚         โ€ข Topological ordering  โ€ข Parallel execution    โ”‚
โ”‚         โ€ข Task conditions       โ€ข Context propagation   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                          โ”‚
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    TaskExecutor                         โ”‚
โ”‚         โ€ข Retry logic          โ€ข Timeout handling       โ”‚
โ”‚         โ€ข CommandTask          โ€ข Custom Task trait      โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                          โ”‚
          โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
          โ–ผ               โ–ผ               โ–ผ
     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
     โ”‚ Storage โ”‚    โ”‚ EventBus โ”‚    โ”‚  Context  โ”‚
     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Development

# Run tests
cargo test

# Run tests with SQLite
cargo test --features sqlite

# Build release
cargo build --release

# Run with debug logging
RUST_LOG=debug cargo run -- run examples/jobs/

License

MIT