📦 apache / fluss

📄 deploying-with-helm.md · 409 lines
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409---
title: "Deploying with Helm Charts"
sidebar_position: 5
---

# Deploying with Helm Charts

This page provides instructions for deploying a Fluss cluster on Kubernetes using Helm charts. The chart creates a distributed streaming storage system with CoordinatorServer and TabletServer components.

## Prerequisites

Before installing the Fluss Helm chart, ensure you have:

- Kubernetes
- Helm
- **For Local Development**: Minikube and Docker (see [Local Development with Minikube](#running-fluss-locally-with-minikube))

:::note
A Fluss cluster deployment requires a running ZooKeeper ensemble. To provide flexibility in deployment and enable reuse of existing infrastructure,
the Fluss Helm chart does not include a bundled ZooKeeper cluster. If you don’t already have a ZooKeeper running,
the installation documentation provides instructions for deploying one using Bitnami’s Helm chart.
:::

## Supported Versions

| Component | Minimum Version | Recommended Version |
|-----------|----------------|-------------------|
| [Kubernetes](https://kubernetes.io) | v1.19+ | v1.25+ |
| [Helm](https://helm.sh) | v3.8.0+ | v3.18.6+ |
| [ZooKeeper](https://zookeeper.apache.org) | v3.6+ | v3.8+ |
| [Apache Fluss](https://fluss.apache.org/docs/) (Container Image) | $FLUSS_VERSION$ | $FLUSS_VERSION$ |
| [Minikube](https://minikube.sigs.k8s.io) (Local Development) | v1.25+ | v1.32+ |
| [Docker](https://docs.docker.com/) (Local Development) | v20.10+ | v24.0+ |

## Installation

### Running Fluss locally with Minikube

For local testing and development, you can deploy Fluss on Minikube. This is ideal for development, testing, and learning purposes.

#### Prerequisites

- Docker container runtime
- At least 4GB RAM available for Minikube
- At least 2 CPU cores available

#### Start Minikube

```bash
# Start Minikube with recommended settings for Fluss
minikube start

# Verify cluster is ready
kubectl cluster-info
```

#### Configure Docker Environment (Optional)

To build images directly in Minikube you need to configure the Docker CLI to use Minikube's internal Docker daemon:

```bash
# Configure shell to use Minikube's Docker daemon
eval $(minikube docker-env)
```

To build custom images please refer to [Custom Container Images](#custom-container-images).

### Installing the chart on a cluster

This installation process is generally working for a distributed Kubernetes cluster or a Minikube setup.

### Step 1: Deploy ZooKeeper (Optional if ZooKeeper is existing)

To start Zookeeper use Bitnami’s chart or your own deployment. If you have an existing Zookeeper cluster, you can skip this step. Example with Bitnami’s chart:

```bash
# Add Bitnami repository
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

# Deploy ZooKeeper
helm install zk bitnami/zookeeper
```
### Step 2: Deploy Fluss

#### Install from Helm repo

```bash
helm repo add fluss https://downloads.apache.org/incubator/fluss/helm-chart
helm repo update
helm install helm-repo/fluss
```

#### Install from Local Chart

```bash
helm install fluss ./helm
```

#### Install with Custom Values

You can customize the installation by providing your own `values.yaml` file or setting individual parameters via the `--set` flag. Using a custom values file:

```bash
helm install fluss ./helm -f my-values.yaml
```

Or for example to change the ZooKeeper address via the `--set` flag:

```bash
helm install fluss ./helm \
  --set configurationOverrides.zookeeper.address=<my-zk-cluster>:2181
```

### Cleanup

```bash
# Uninstall Fluss
helm uninstall fluss

# Uninstall ZooKeeper
helm uninstall zk

# Delete PVCs
kubectl delete pvc -l app.kubernetes.io/name=fluss

# Stop Minikube
minikube stop

# Delete Minikube cluster
minikube delete
```

## Architecture Overview

The Fluss Helm chart deploys the following Kubernetes resources:

### Core Components
- **CoordinatorServer**: 1x StatefulSet with Headless Service for cluster coordination
- **TabletServer**: 3x StatefulSet with Headless Service for data storage and processing
- **ConfigMap**: Configuration management for `server.yaml` settings
- **Services**: Headless services providing stable pod DNS names

### Optional Components
- **PersistentVolumes**: Data persistence when `persistence.enabled=true`


### Step 3: Verify Installation

```bash
# Check pod status
kubectl get pods -l app.kubernetes.io/name=fluss

# Check services
kubectl get svc -l app.kubernetes.io/name=fluss

# View logs
kubectl logs -l app.kubernetes.io/component=coordinator
kubectl logs -l app.kubernetes.io/component=tablet
```

## Configuration Parameters

The following table lists the configurable parameters of the Fluss chart and their default values.

### Global Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `nameOverride` | Override the name of the chart | `""` |
| `fullnameOverride` | Override the full name of the resources | `""` |

### Image Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `image.registry` | Container image registry | `""` |
| `image.repository` | Container image repository | `fluss` |
| `image.tag` | Container image tag | `$FLUSS_VERSION$` |
| `image.pullPolicy` | Container image pull policy | `IfNotPresent` |
| `image.pullSecrets` | Container image pull secrets | `[]` |

### Application Configuration

| Parameter | Description | Default |
|-----------|-------------|---------|
| `appConfig.internalPort` | Internal communication port | `9123` |
| `appConfig.externalPort` | External client port | `9124` |

### Fluss Configuration Overrides

| Parameter | Description | Default |
|-----------|-------------|---------|
| `configurationOverrides.default.bucket.number` | Default number of buckets for tables | `3` |
| `configurationOverrides.default.replication.factor` | Default replication factor | `3` |
| `configurationOverrides.zookeeper.path.root` | ZooKeeper root path for Fluss | `/fluss` |
| `configurationOverrides.zookeeper.address` | ZooKeeper ensemble address | `zk-zookeeper.{{ .Release.Namespace }}.svc.cluster.local:2181` |
| `configurationOverrides.remote.data.dir` | Remote data directory for snapshots | `/tmp/fluss/remote-data` |
| `configurationOverrides.data.dir` | Local data directory | `/tmp/fluss/data` |
| `configurationOverrides.internal.listener.name` | Internal listener name | `INTERNAL` |

### Persistence Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `persistence.enabled` | Enable persistent volume claims | `false` |
| `persistence.size` | Persistent volume size | `1Gi` |
| `persistence.storageClass` | Storage class name | `nil` (uses default) |

### Resource Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `resources.coordinatorServer.requests.cpu` | CPU requests for coordinator | Not set |
| `resources.coordinatorServer.requests.memory` | Memory requests for coordinator | Not set |
| `resources.coordinatorServer.limits.cpu` | CPU limits for coordinator | Not set |
| `resources.coordinatorServer.limits.memory` | Memory limits for coordinator | Not set |
| `resources.tabletServer.requests.cpu` | CPU requests for tablet servers | Not set |
| `resources.tabletServer.requests.memory` | Memory requests for tablet servers | Not set |
| `resources.tabletServer.limits.cpu` | CPU limits for tablet servers | Not set |
| `resources.tabletServer.limits.memory` | Memory limits for tablet servers | Not set |


## Advanced Configuration

### Custom ZooKeeper Configuration

For external ZooKeeper clusters:

```yaml
configurationOverrides:
  zookeeper.address: "zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181"
  zookeeper.path.root: "/my-fluss-cluster"
```

### Network Configuration

The chart automatically configures listeners for internal cluster communication and external client access:

- **Internal Port (9123)**: Used for inter-service communication within the cluster
- **External Port (9124)**: Used for client connections

Custom listener configuration:

```yaml
appConfig:
  internalPort: 9123
  externalPort: 9124

configurationOverrides:
  bind.listeners: "INTERNAL://0.0.0.0:9123,CLIENT://0.0.0.0:9124"
  advertised.listeners: "CLIENT://my-cluster.example.com:9124"
```

### Storage Configuration

Configure different storage backends:

```yaml
configurationOverrides:
  data.dir: "/data/fluss"
  remote.data.dir: "s3://my-bucket/fluss-data"
```

## Upgrading

### Upgrade the Chart

```bash
# Upgrade to a newer chart version
helm upgrade fluss ./helm

# Upgrade with new configuration
helm upgrade fluss ./helm -f values-new.yaml
```

### Rolling Updates

The StatefulSets support rolling updates. When you update the configuration, pods will be restarted one by one to maintain availability.

## Custom Container Images

### Building Custom Images

To build and use custom Fluss images:

1. Build the project with Maven:
```bash
mvn clean package -DskipTests
```

2. Build the Docker image:
```bash
# Copy build artifacts
cp -r build-target/* docker/fluss/build-target

# Build image
cd docker
docker build -t my-registry/fluss:custom-tag .
```

3. Use in Helm values:
```yaml
image:
  registry: my-registry
  repository: fluss
  tag: custom-tag
  pullPolicy: Always
```

## Monitoring and Observability

### Health Checks

The chart includes liveness and readiness probes:

```yaml
livenessProbe:
  tcpSocket:
    port: 9124
  initialDelaySeconds: 10
  periodSeconds: 3
  failureThreshold: 100

readinessProbe:
  tcpSocket:
    port: 9124
  initialDelaySeconds: 10
  periodSeconds: 3
  failureThreshold: 100
```

### Logs

Access logs from different components:

```bash
# Coordinator logs
kubectl logs -l app.kubernetes.io/component=coordinator -f

# Tablet server logs
kubectl logs -l app.kubernetes.io/component=tablet -f

# Specific pod logs
kubectl logs coordinator-server-0 -f
kubectl logs tablet-server-0 -f
```

## Troubleshooting

### Common Issues

#### Pod Startup Issues

**Symptoms**: Pods stuck in `Pending` or `CrashLoopBackOff` state

**Solutions**:
```bash
# Check pod events
kubectl describe pod <pod-name>

# Check resource availability
kubectl describe nodes

# Verify ZooKeeper connectivity
kubectl exec -it <fluss-pod> -- nc -zv <zookeeper-host> 2181
```

#### Image Pull Errors

**Symptoms**: `ImagePullBackOff` or `ErrImagePull`

**Solutions**:
- Verify image repository and tag exist
- Check pull secrets configuration
- Ensure network connectivity to registry


#### Connection Issues

**Symptoms**: Clients cannot connect to Fluss cluster

**Solutions**:
```bash
# Check service endpoints
kubectl get endpoints

# Test network connectivity
kubectl exec -it <client-pod> -- nc -zv <fluss-service> 9124

# Verify DNS resolution
kubectl exec -it <client-pod> -- nslookup <fluss-service>
```

### Debug Commands

```bash
# Get all resources
kubectl get all -l app.kubernetes.io/name=fluss

# Check configuration
kubectl get configmap fluss-conf-file -o yaml


# Get detailed pod information
kubectl get pods -o wide -l app.kubernetes.io/name=fluss
```