Data Engineering
29 items across the graph — tagged with Data Engineering.
From the graph · 29
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Workflow Engine for Kubernetes
Incremental engine for long horizon agents 🌟 Star if you like it!
Graph-Native Infrastructure for Context and Accountable AI Systems
🧙 Build, run, and manage data pipelines for integrating and transforming data.
The Open Source Feature Store for AI/ML
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Maestro: Netflix’s Workflow Orchestrator
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and sca…
A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your developme…
A comprehensive list of 180+ YouTube Channels for Data Science, Data Engineering, Machine Learning, Deep learning, Computer Science, programming, software engin…
Python Streaming DataFrames for Kafka
A comprehensive Python package template to kickstart and standardize your MLOps initiatives and data pipelines.
Open-source agentic data engineering harness for dbt, SQL, and cloud warehouses. 100+ tools, 10 warehouses, AI-powered.
A MCP (Model Context Protocol) server for interacting with dbt.
🧠Mindmap of 🗺️Software Architecture, Software engineering: An Overview of Software Terminologies and Concepts.
AI agent tooling for data engineering workflows.
Zero-config entity resolution that feeds a durable identity layer: resolve messy records from any source into stable golden entities — a Customer 360 with whole…
Resources about data science, machine learning, deep learning, data engineering, and SQL.
Students' material for the course in Artificial Intelligence and Data Engineering at University of Pisa.
mloda.ai - Open Data Access for AI and ML. Plugin-based. Traceable. Framework-agnostic.
Synthetic test data that hits the numbers you declare, exactly. Multi-table with verified foreign-key integrity, deterministic, no model in the data path. Pytho…
Workbench: An easy to use Python API for creating and deploying AWS SageMaker Models
Cloud + Data + AI + Security from zero to hero. 122+ certs across 22 providers, 37 plain-English concepts, 15 hands-on builds, cross-cloud + AI service comparis…
45+ production-ready tutorials on data science, MLOps, and AI tools. All code is executable and adaptable for real projects.
Reverse ETL for the code-first data stack