Bootstrap an Air-Gapped Cluster

Introduction

This guide outlines the steps to bootstrap a Cozystack cluster in an air-gapped environment.

Air-gapped installation means that the cluster has no direct access to the Internet. All necessary resources, such as images and metadata, must be available on the private network.

Configuring Talos Nodes

1. Configure NTP Servers

Accurate time synchronization is critical for the cluster. In your Talos machine configuration, set local NTP servers that are accessible inside your private network:

machine:
  time:
    servers:
      # example values
      - 192.168.0.4
      - 10.10.0.5

Ensure that these NTP servers are reachable from the first Talos node.

2. Configure Container Registry Mirrors

Since the cluster cannot access public container registries, it needs to use their local mirrors. Creating such mirrors is out of the scope of this guide.

Update your machine configuration in the following way, providing the IP addresses and ports of your local mirrors for each registry:

machine:
  registries:
    mirrors:
      docker.io:
        endpoints:
          - http://10.0.0.1:8082
      ghcr.io:
        endpoints:
          - http://10.0.0.1:8083
      gcr.io:
        endpoints:
          - http://10.0.0.1:8084
      registry.k8s.io:
        endpoints:
          - http://10.0.0.1:8085
      quay.io:
        endpoints:
          - http://10.0.0.1:8086
      cr.fluentbit.io:
        endpoints:
          - http://10.0.0.1:8087
      docker-registry3.mariadb.com:
        endpoints:
          - http://10.0.0.1:8088
    config:
      "10.0.0.1:8082":
        tls:
          insecureSkipVerify: true
        auth:
          username: myuser
          password: mypass

Of course, the values for config.[0].auth.* are given as examples, and you need to use real credentials. Make sure your local registry proxies mirror all required images for Talos and Kubernetes components.

3. Add CA Certificate

To use a private Certificate Authority, you need to add its certificate to the nodes.

# talm: nodes=["10.10.10.10"], endpoints=["10.10.10.10"], templates=["templates/controlplane.yaml"]
# THIS FILE IS AUTOGENERATED. PREFER TEMPLATE EDITS OVER MANUAL ONES.
machine:
# ...
# ...
  discovery:
    enabled: false
  etcd:
    advertisedSubnets:
      - 10.4.100.10/24
  allowSchedulingOnControlPlanes: true
---
apiVersion: v1alpha1
kind: TrustedRootsConfig
name: my-enterprise-ca
certificates: |
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----

4. Apply Changes

After you have made the changes above, you can apply the configuration and bootstrap a cluster:

Using Talm

Rebuild the node configuration files and apply them to each node:

talm template -e <ip> -n <ip> -t templates/controlplane.yaml -i > nodes/node1.yaml
talm apply -f nodes/node1.yaml
# repeat for each node

Finally, bootstrap the cluster as usual:

talm bootstrap -f nodes/node1.yaml

Read the Talm configuration guide to learn more.

Using talosctl

Apply the configuration to each node:

talosctl apply -f controlplane.yaml -n <ip> -e <ip> -i

Finally, bootstrap the cluster using one of the nodes:

talosctl bootstrap -n <ip> -e <ip>

Read the talosctl configuration guide to learn more.

5. Configure Registry Mirrors for Tenant Kubernetes

Tenant Kubernetes clusters use Kamaji for the control plane. Control-plane components run as pods on the management-cluster nodes, so they already use the Talos registry mirrors from step 2.

Tenant worker nodes run as separate virtual machines with their own Talos instance, so they are configured through the Kubernetes resource, not through the management nodes’ machine config.

A worker node needs three things reachable without public egress: its OS disk image, the Talos installer, and the container images it pulls. Set them on the Kubernetes resource:

apiVersion: apps.cozystack.io/v1alpha1
kind: Kubernetes
metadata:
  name: my-cluster
spec:
  talos:
    # OS disk image streamed in by CDI over HTTP
    imageFactoryURL: https://factory.example.com
    # installer image for in-guest upgrades: <repo>/<schematicID>:<version>
    installerRepository: registry.example.com/installer
    # machine.registries.mirrors passthrough for worker nodes
    registryMirrors:
      docker.io:
        endpoints:
          - https://registry.example.com/v2/docker.io
        skipFallback: true
      ghcr.io:
        endpoints:
          - https://registry.example.com/v2/ghcr.io
        skipFallback: true
      registry.k8s.io:
        endpoints:
          - https://registry.example.com/v2/registry.k8s.io
        skipFallback: true

• talos.imageFactoryURL (default https://factory.talos.dev) points at a self-hosted Image Factory, a caching mirror, or an internal HTTP file server.
• talos.installerRepository (default factory.talos.dev/installer) is a mirrored OCI registry; it is resolved as <installerRepository>/<schematicID>:<version>.
• talos.registryMirrors has the same shape as the node mirrors in step 2. By default Talos falls back to the upstream registry, so set skipFallback: true per host to enforce a true air-gap.