📦 apache / fluss

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83<!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<p align="center">
    <picture>
      <source media="(prefers-color-scheme: dark)" srcset="website/static/img/logo/svg/white_color_logo.svg">
      <source media="(prefers-color-scheme: light)" srcset="website/static/img/logo/svg/colored_logo.svg">
      <!-- Fall back to version that works for dark and light mode -->
      <img alt="Apache Fluss logo" src="website/static/img/logo/svg/white_filled.svg">
    </picture>
</p>

<p align="center">
  <a href="https://fluss.apache.org/docs/">Documentation</a> | <a href="https://fluss.apache.org/docs/quickstart/flink/">QuickStart</a> | <a href="https://fluss.apache.org/community/dev/ide-setup/">Development</a>
</p>

<p align="center">
  <a href="https://github.com/apache/fluss/actions/workflows/ci.yaml"><img src="https://github.com/apache/fluss/actions/workflows/ci.yaml/badge.svg?branch=main" alt="CI"></a>
  <a href="https://github.com/apache/fluss/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-4EB1BA.svg" alt="License"></a>
  <a href="https://join.slack.com/t/apache-fluss/shared_invite/zt-33wlna581-QAooAiCmnYboJS8D_JUcYw"><img src="https://img.shields.io/badge/slack-join_chat-brightgreen.svg?logo=slack" alt="Slack"></a>
  <a href="https://deepwiki.com/apache/fluss"><img src="https://deepwiki.com/badge.svg" alt="Ask DeepWiki"></a>
</p>

## What is Apache Fluss (Incubating)?

Apache Fluss (Incubating) is a streaming storage built for real-time analytics which can serve as the real-time data layer for Lakehouse architectures.

It bridges the gap between **data streaming** and **data Lakehouse** by enabling low-latency, high-throughput data ingestion and processing while seamlessly integrating with popular compute engines like **Apache Flink**, while 
Apache Spark, and StarRocks are coming soon.

**Fluss (German: river, pronounced `/flus/`)** enables streaming data continuously converging, distributing and flowing into lakes, like a river 🌊

## Features

- **Sub-Second Latency**: Low-latency streaming reads/writes optimized for real-time applications with Apache Flink.
- **Columnar Stream**: 10x improvement in streaming read performance with efficient pushdown projections.
- **Streaming & Lakehouse Unification**: Unified data streaming and Lakehouse with low latencies for powerful analytics.
- **Real-Time Updates**: Cost-efficient partial updates for large-scale data without expensive join operations.
- **Changelog Generation**: Complete changelogs for streaming processors, streamlining analytics workflows.
- **Lookup Queries**: Ultra-high QPS for primary key lookups, enabling efficient dimension table serving.

## Building

Prerequisites for building Apache Fluss:

- Unix-like environment (we use Linux, Mac OS X, Cygwin, WSL)
- Git
- Maven (we require version >= 3.8.6)
- Java 11

```bash
git clone https://github.com/apache/fluss.git
cd fluss
./mvnw clean package -DskipTests
```

Apache Fluss is now installed in `build-target`. The build command uses Maven Wrapper (`mvnw`) which ensures the correct Maven version is used.

## Contributing

Apache Fluss (Incubating) is open-source, and we’d love your help to keep it growing! Join the [discussions](https://github.com/apache/fluss/discussions),
open [issues](https://github.com/apache/fluss/issues) if you find a bug or request features, contribute code and documentation,
or help us improve the project in any way. All contributions are welcome!

## License

Apache Fluss (Incubating) project is licensed under the [Apache License 2.0](https://github.com/apache/fluss/blob/main/LICENSE).