1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305<!-- omit in toc -->
# AWX on Single Node K3s
An example implementation of AWX on single node K3s using AWX Operator, with easy-to-use simplified configuration with ownership of data and passwords.
- Accessible over HTTPS from remote host
- All data will be stored under `/data`
- Fixed (configurable) passwords for AWX and PostgreSQL
- Fixed (configurable) versions of AWX
**If you want to view the guide for the specific version of AWX Operator, switch the page to the desired tag instead of the `main` branch.**
<!-- omit in toc -->
## Table of Contents
- [Environment](#environment)
- [References](#references)
- [Requirements](#requirements)
- [Deployment Instruction](#deployment-instruction)
- [Prepare CentOS Stream 8 host](#prepare-centos-stream-8-host)
- [Install K3s](#install-k3s)
- [Install AWX Operator](#install-awx-operator)
- [Prepare required files to deploy AWX](#prepare-required-files-to-deploy-awx)
- [Deploy AWX](#deploy-awx)
- [Back up and Restore AWX using AWX Operator](#back-up-and-restore-awx-using-awx-operator)
- [Additional Guides](#additional-guides)
## Environment
- Tested on:
- CentOS Stream 8 (Minimal)
- K3s v1.28.7+k3s1
- Products that will be deployed:
- AWX Operator 2.13.1
- AWX 24.0.0
- PostgreSQL 15
## References
- [K3s - Lightweight Kubernetes](https://docs.k3s.io/)
- [INSTALL.md on ansible/awx](https://github.com/ansible/awx/blob/24.0.0/INSTALL.md) @24.0.0
- [README.md on ansible/awx-operator](https://github.com/ansible/awx-operator/blob/2.13.1/README.md) @2.13.1
## Requirements
- **Computing resources**
- **2 CPUs with x86-64-v2 support**.
- **4 GiB RAM minimum**
- It's recommended to add more CPUs and RAM (like 4 CPUs and 8 GiB RAM or more) to avoid performance issue and job scheduling issue.
- The files in this repository are configured to ignore resource requirements which specified by AWX Operator by default.
- **Storage resources**
- At least **10 GiB for `/var/lib/rancher`** and **10 GiB for `/data`** are safe for fresh install.
- **Both will be grown during lifetime** and **actual consumption highly depends on your environment and your use case**, so you should to pay attention to the consumption and add more capacity if required.
- `/var/lib/rancher` will be created and consumed by K3s and related data like container images and overlayfs.
- `/data` will be created in this guide and used to store AWX-related databases and files.
## Deployment Instruction
### Prepare CentOS Stream 8 host
Disable firewalld and nm-cloud-setup if enabled. This is [recommended by K3s](https://docs.k3s.io/advanced#red-hat-enterprise-linux--centos).
```bash
# Disable firewalld
sudo systemctl disable firewalld --now
# Disable nm-cloud-setup if exists and enabled
sudo systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
sudo reboot
```
Install the required packages to deploy AWX Operator and AWX.
```bash
sudo dnf install -y git curl
```
### Install K3s
Install a specific version of K3s with `--write-kubeconfig-mode 644` to make the config file (`/etc/rancher/k3s/k3s.yaml`) readable by non-root users.
```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.28.7+k3s1 sh -s - --write-kubeconfig-mode 644
```
### Install AWX Operator
> [!NOTE]
> From AWX Operator 2.13.0, Default PostgreSQL version is bumped from 13 to 15. If you have a plan to upgrade existing AWX Operator and AWX, refer to [📝Tips: Upgrade AWX Operator and AWX](tips/upgrade-operator.md) to perform additional tasks to database migration.
Clone this repository and change directory.
If you want to use files suitable for a specific version of AWX Operator, [refer to tags in this repository](https://github.com/kurokobo/awx-on-k3s/tags) and specify the desired tag in `git checkout`. Especially for `0.13.0` or earlier versions of AWX Operator, refer to [📝Tips: Deploy older version of AWX Operator](tips/deploy-older-operator.md).
```bash
cd ~
git clone https://github.com/kurokobo/awx-on-k3s.git
cd awx-on-k3s
git checkout 2.13.1
```
Then invoke `kubectl apply -k operator` to deploy AWX Operator.
```bash
kubectl apply -k operator
```
The AWX Operator will be deployed to the namespace `awx`.
```bash
$ kubectl -n awx get all
NAME READY STATUS RESTARTS AGE
pod/awx-operator-controller-manager-68d787cfbd-kjfg7 2/2 Running 0 16s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/awx-operator-controller-manager-metrics-service ClusterIP 10.43.150.245 <none> 8443/TCP 16s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/awx-operator-controller-manager 1/1 1 1 16s
NAME DESIRED CURRENT READY AGE
replicaset.apps/awx-operator-controller-manager-68d787cfbd 1 1 1 16s
```
### Prepare required files to deploy AWX
Generate a Self-Signed certificate. Note that an IP address can't be specified. If you want to use a certificate from a public ACME CA such as Let's Encrypt or ZeroSSL instead of a Self-Signed certificate, follow the guide on [📁 **Use SSL Certificate from Public ACME CA**](acme) first and come back to this step when done.
```bash
AWX_HOST="awx.example.com"
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -out ./base/tls.crt -keyout ./base/tls.key -subj "/CN=${AWX_HOST}/O=${AWX_HOST}" -addext "subjectAltName = DNS:${AWX_HOST}"
```
Modify `hostname` in `base/awx.yaml`.
```yaml
...
spec:
...
ingress_type: ingress
ingress_hosts:
- hostname: awx.example.com 👈👈👈
tls_secret: awx-secret-tls
...
```
Modify the two `password` entries in `base/kustomization.yaml`. Note that the `password` under `awx-postgres-configuration` should not contain single or double quotes (`'`, `"`) or backslashes (`\`) to avoid any issues during deployment, backup or restoration.
```yaml
...
- name: awx-postgres-configuration
type: Opaque
literals:
- host=awx-postgres-15
- port=5432
- database=awx
- username=awx
- password=Ansible123! 👈👈👈
- type=managed
- name: awx-admin-password
type: Opaque
literals:
- password=Ansible123! 👈👈👈
...
```
Prepare directories for Persistent Volumes defined in `base/pv.yaml`. These directories will be used to store your databases and project files. Note that the size of the PVs and PVCs are specified in some of the files in this repository, but since their backends are `hostPath`, its value is just like a label and there is no actual capacity limitation.
```bash
sudo mkdir -p /data/postgres-15/data
sudo mkdir -p /data/projects
sudo chmod 700 /data/postgres-15/data
sudo chown 26:0 /data/postgres-15/data
sudo chown 1000:0 /data/projects
```
### Deploy AWX
Deploy AWX, this takes few minutes to complete.
```bash
kubectl apply -k base
```
To monitor the progress of the deployment, check the logs of `deployments/awx-operator-controller-manager`:
```bash
kubectl -n awx logs -f deployments/awx-operator-controller-manager
```
If the deployment completes successfully, the logs end with:
```txt
$ kubectl -n awx logs -f deployments/awx-operator-controller-manager
...
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWX, awx/awx) -----
PLAY RECAP *********************************************************************
localhost : ok=90 changed=0 unreachable=0 failed=0 skipped=81 rescued=0 ignored=1
```
The required objects should now have been deployed next to AWX Operator in the `awx` namespace.
```bash
$ kubectl -n awx get awx,all,ingress,secrets
NAME AGE
awx.awx.ansible.com/awx 6m48s
NAME READY STATUS RESTARTS AGE
pod/awx-operator-controller-manager-59b86c6fb-4zz9r 2/2 Running 0 7m22s
pod/awx-postgres-15-0 1/1 Running 0 6m33s
pod/awx-web-549f7fdbc5-htpl9 3/3 Running 0 6m5s
pod/awx-migration-24.0.0-kglht 0/1 Completed 0 4m36s
pod/awx-task-7d4fcdd449-mqkp2 4/4 Running 0 6m4s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/awx-operator-controller-manager-metrics-service ClusterIP 10.43.58.194 <none> 8443/TCP 7m33s
service/awx-postgres-15 ClusterIP None <none> 5432/TCP 6m33s
service/awx-service ClusterIP 10.43.180.226 <none> 80/TCP 6m7s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/awx-operator-controller-manager 1/1 1 1 7m33s
deployment.apps/awx-web 1/1 1 1 6m5s
deployment.apps/awx-task 1/1 1 1 6m4s
NAME DESIRED CURRENT READY AGE
replicaset.apps/awx-operator-controller-manager-59b86c6fb 1 1 1 7m22s
replicaset.apps/awx-web-549f7fdbc5 1 1 1 6m5s
replicaset.apps/awx-task-7d4fcdd449 1 1 1 6m4s
NAME READY AGE
statefulset.apps/awx-postgres-15 1/1 6m33s
NAME COMPLETIONS DURATION AGE
job.batch/awx-migration-24.0.0 1/1 2m4s 4m36s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/awx-ingress traefik awx.example.com 192.168.0.219 80, 443 6m6s
NAME TYPE DATA AGE
secret/redhat-operators-pull-secret Opaque 1 7m33s
secret/awx-admin-password Opaque 1 6m48s
secret/awx-postgres-configuration Opaque 6 6m48s
secret/awx-secret-tls kubernetes.io/tls 2 6m48s
secret/awx-app-credentials Opaque 3 6m9s
secret/awx-secret-key Opaque 1 6m41s
secret/awx-broadcast-websocket Opaque 1 6m38s
secret/awx-receptor-ca kubernetes.io/tls 2 6m14s
secret/awx-receptor-work-signing Opaque 2 6m12s
```
Now your AWX is available at `https://awx.example.com/` or the hostname you specified.
Note that you have to access via the hostname that you specified in `base/awx.yaml`, instead of by IP address, since this guide uses Ingress. So you should configure your DNS or `hosts` file on your client where the browser is running.
At this point, AWX can be accessed via HTTP as well as HTTPS. If you want to force users to use HTTPS, see [📝Tips: Enable HTTP Strict Transport Security (HSTS)](tips/enable-hsts.md).
## Back up and Restore AWX using AWX Operator
The AWX Operator `0.10.0` or later has the ability to back up and restore AWX in easy way.
Refer [📁 **Back up AWX using AWX Operator**](backup) and [📁 **Restore AWX using AWX Operator**](restore) for details.
## Additional Guides
- [📁 **Back up AWX using AWX Operator**](backup)
- The guide to make backup of your AWX using AWX Operator.
- This guide includes not only the way to make backup manually, but also an example simple playbook for Ansible, which can be use with scheduling feature on AWX.
- [📁 **Restore AWX using AWX Operator**](restore)
- The guide to restore your AWX using AWX Operator.
- [📁 **Build and Use your own Execution Environment**](builder)
- The guide to use Ansible Builder to build our own Execution Environment.
- [📁 **Deploy Private Git Repository on Kubernetes**](git)
- The guide to use AWX with SCM. This repository includes the manifests to deploy [Gitea](https://gitea.io/en-us/).
- [📁 **Deploy Private Container Registry on Kubernetes**](registry)
- The guide to use Execution Environments in AWX (AWX-EE).
- If we want to use our own Execution Environment built with Ansible Builder and don't want to push it to the public container registry e.g. Docker Hub, we can deploy a private container registry on K3s.
- [📁 **Integrate AWX with EDA Controller** (Experimental)](rulebooks)
- The guide to deploy and use Event Driven Ansible Controller (EDA Controller) with AWX on K3s.
- **Note that EDA Controller Operator that used in this guide is not a fully supported installation method for EDA Controller.**
- [📁 **Deploy Private Galaxy NG on Kubernetes** (Experimental)](galaxy)
- The guide to deploy our own Galaxy NG instance.
- [📁 **Use SSL Certificate from Public ACME CA**](acme)
- The guide to use a certificate from public ACME CA such as Let's Encrypt or ZeroSSL instead of Self-Signed certificate.
- [📁 **Use Ansible Runner**](runner)
- The guide to use Ansible Runner to run playbook using Execution Environment.
- [📁 **Use Customized Pod Specification for your Execution Environment**](containergroup)
- The guide to use customized Pod of the Execution Environment using **Container Group**.
- [📁 **Tips**](tips)
- [📝Create "Manual" type project](tips/manual-project.md)
- [📝Deploy AWX using external PostgreSQL database](tips/external-db.md)
- [📝Trust custom Certificate Authority](tips/trust-custom-ca.md)
- [📝Expose `/etc/hosts` to Pods on K3s](tips/expose-hosts.md)
- [📝Enable HTTP Strict Transport Security (HSTS)](tips/enable-hsts.md)
- [📝Pass values from Secrets to `extra_settings`](tips/extra-settings.md)
- [📝Use HTTP proxy](tips/use-http-proxy.md)
- [📝Uninstall deployed resources](tips/uninstall.md)
- [📝Deploy older version of AWX Operator](tips/deploy-older-operator.md)
- [📝Upgrade AWX Operator and AWX](tips/upgrade-operator.md)
- [📝Workaround for the rate limit on Docker Hub](tips/dockerhub-rate-limit.md)
- [📝Version Mapping for AWX Operator and AWX](tips/version-mapping.md)
- [📝Use Kerberos authentication to connect to Windows hosts](tips/use-kerberos.md)
- [📝Use Helm or Operator Lifecycle Manager to manage AWX Operator and AWX](tips/alternative-methods.md)
- [📝Troubleshooting Guide](tips/troubleshooting.md)