Apache Airflow

HFlow is an open-source SDK from Hebbian Robotics for multimodal data pipelines in robotics and physical AI. It lets teams implement transformations, quality checks, labels, and enrichments as plain Python functions while HFlow handles orchestration, provenance, storage, versioning, and dataset curation. The pipeline accepts supported MCAP episodes and can import LeRobot Dataset v3 repositories. It runs code in-process for development or generates Airflow 3 DAGs for scheduled execution, producing canonical MCAP episodes, artifacts, provenance records, and a Parquet catalog. DuckDB SQL creates version-pinned manifests for curation, while a queryable catalog records metadata and quality evidence without requiring the underlying recordings to be loaded.

CrewAI is an open-source Python framework for building multi-agent workflows. Its Crews use autonomous, role-based agents that collaborate on tasks, while its Flows provide event-driven workflow control combining precise orchestration, individual LLM calls, and Crews. The project also offers CrewAI AMP Suite, a commercial control plane for managed deployment, observability, governance, security, enterprise support, and cloud or on-premise operation. The repository provides high-level abstractions and lower-level APIs, along with documentation, examples, installation guidance, and integrations for connecting agents to language models.

Argo Workflows is an open-source, container-native workflow engine for orchestrating parallel jobs on Kubernetes. It is implemented as a Kubernetes Custom Resource Definition: each workflow step runs as a container, and multi-step processes can be modeled either as ordered steps or as a directed acyclic graph (DAG). It supports machine-learning pipelines, data and batch processing, infrastructure automation, and CI/CD. The project provides workflow templating, artifact and parameter inputs and outputs, loops, conditionals, retries, timeouts, scheduling, suspension and resumption, resource orchestration, garbage collection, event triggers, and Prometheus metrics. It includes a web UI, CLI, REST and gRPC server interfaces, workflow archiving, log viewing, and Java, Go, Python through Hera, and TypeScript through Juno client libraries. Argo Workflows is a graduated Cloud Native Computing Foundation project and can run on Kubernetes clusters across cloud environments.

Apache Kafka is an open-source distributed event-streaming platform developed by the Apache Software Foundation. It is used for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications, and can be deployed on premises, in a cluster, or through a managed service. The official repository contains Java and Scala source code, build tasks, documentation generation, and unit and integration tests.

OpenWorker is an open-source desktop AI coworker that produces finished deliverables from everyday tasks, including code-security reviews with proposed fixes, cloud-configuration audits, incident reports, documents, spreadsheets, drafted messages, and scheduled briefs. It runs on the user's machine through a native desktop app and a local Python agent server built on aisuite, working across local files, the terminal, connected applications, and more than 25 connectors. Specialist coworkers cover security review, cloud posture, incident triage, everyday work, and recurring automations; security review combines deterministic scanners such as Semgrep with model reasoning, then re-scans and diff-reviews proposed fixes before approval. The agent breaks requested outcomes into steps and uses models from providers such as OpenAI, Anthropic, and Google, open-weight providers, or a local Ollama deployment, with the user supplying model access. Consequential actions—including sending messages, changing calendars, and running commands—are approval-gated. Its governance design includes human-only floors for dangerous or irreversible operations, explicitly granted and revocable autonomy rules, circuit-breaker escalation for uncertain or repeatedly denied actions, and an audit trail recording tool calls, approval provenance, and reviewer reasoning; unattended runs cannot self-approve. The project is in open beta and provides downloads for macOS on Apple Silicon and Windows on x64.