📦 apache / fluss

Apache Fluss is a streaming storage built for real-time analytics.

★ 1.7k stars ⑂ 486 forks 👁 1.7k watching ⚖️ Apache License 2.0
big-dataflusshacktoberfestlakehousereal-time-analyticsstreaming
fluss
📥 Clone https://github.com/apache/fluss.git
HTTPS git clone https://github.com/apache/fluss.git
SSH git clone git@github.com:apache/fluss.git
CLI gh repo clone apache/fluss
ZIP Desktop VS Code
Yang Wang Yang Wang [kv] Add roaring bitmap aggregate function for aggregation merge engine (#2390) 864bbe6 2 hours ago 📝 History
📂 main View all commits →
📁 .github
📁 .idea
📁 .mvn
📁 docker
📁 fluss-client
📁 fluss-common
📁 fluss-dist
📁 fluss-filesystems
📁 fluss-flink
📁 fluss-jmh
📁 fluss-kafka
📁 fluss-lake
📁 fluss-metrics
📁 fluss-protogen
📁 fluss-rpc
📁 fluss-server
📁 fluss-spark
📁 fluss-test-coverage
📁 fluss-test-utils
📁 helm
📁 tools
📁 website
📄 .asf.yaml
📄 .gitignore
📄 .scalafmt.conf
📄 copyright.txt
📄 DISCLAIMER
📄 LICENSE
📄 mvnw
📄 mvnw.cmd
📄 NOTICE
📄 pom.xml
📄 README.md
📄 README.md

Apache Fluss logo

Documentation | QuickStart | Development

CI License Slack Ask DeepWiki

What is Apache Fluss (Incubating)?

Apache Fluss (Incubating) is a streaming storage built for real-time analytics which can serve as the real-time data layer for Lakehouse architectures.

It bridges the gap between data streaming and data Lakehouse by enabling low-latency, high-throughput data ingestion and processing while seamlessly integrating with popular compute engines like Apache Flink, while Apache Spark, and StarRocks are coming soon.

Fluss (German: river, pronounced /flus/) enables streaming data continuously converging, distributing and flowing into lakes, like a river 🌊

Features

  • Sub-Second Latency: Low-latency streaming reads/writes optimized for real-time applications with Apache Flink.
  • Columnar Stream: 10x improvement in streaming read performance with efficient pushdown projections.
  • Streaming & Lakehouse Unification: Unified data streaming and Lakehouse with low latencies for powerful analytics.
  • Real-Time Updates: Cost-efficient partial updates for large-scale data without expensive join operations.
  • Changelog Generation: Complete changelogs for streaming processors, streamlining analytics workflows.
  • Lookup Queries: Ultra-high QPS for primary key lookups, enabling efficient dimension table serving.

Building

Prerequisites for building Apache Fluss:

  • Unix-like environment (we use Linux, Mac OS X, Cygwin, WSL)
  • Git
  • Maven (we require version >= 3.8.6)
  • Java 11
git clone https://github.com/apache/fluss.git
cd fluss
./mvnw clean package -DskipTests

Apache Fluss is now installed in build-target. The build command uses Maven Wrapper (mvnw) which ensures the correct Maven version is used.

Contributing

Apache Fluss (Incubating) is open-source, and we’d love your help to keep it growing! Join the discussions, open issues if you find a bug or request features, contribute code and documentation, or help us improve the project in any way. All contributions are welcome!

License

Apache Fluss (Incubating) project is licensed under the Apache License 2.0.