Mastering Multi-Cluster Kubernetes Management with Octopus: A Comprehensive Tutorial
Modern Kubernetes adoption has evolved far beyond single-cluster deployments. Enterprises now routinely operate tens, sometimes hundreds of Kubernetes clusters across on-premises data centers, public cloud regions, and edge locations. Managing these clusters independently leads to operational fragmentation, inconsistent policies, and duplicated effort. Octopus, an open-source multi-cluster management framework originally contributed by the KubeEdge community and now part of the CNCF landscape, addresses this complexity head-on. Octopus provides a unified control plane that treats all connected clusters as a single virtual cluster, enabling consistent application deployment, resource scheduling, and monitoring across environments. In this tutorial, we will explore Octopus from the ground up: installing the control plane, onboarding clusters, deploying workloads, and applying best practices to keep your multi-cluster environment healthy and efficient. Whether you are a platform engineer responsible for fleet management or a developer seeking to push applications across multiple regions, this guide will equip you with the practical steps needed to harness Octopus effectively.
Before diving into the installation, it is essential to understand Octopus’s architecture and how it differs from other multi-cluster tools like Rancher, Karmada, or Clusternet. Octopus introduces the concept of a “Hub” cluster that acts as the central management point, and multiple “Spoke” clusters that report to the Hub. The communication between Hub and Spokes is highly optimized, using bidirectional streaming (gRPC) to reduce latency and bandwidth usage, which is especially critical for edge deployments with unreliable networks. Octopus also provides a set of custom resource definitions (CRDs) that allow you to define resource templates, distribution policies, and override strategies. Unlike some tools that require proxy agents for every API call, Octopus leverages a lightweight agent on each Spoke to synchronize state and execute commands locally. This design ensures that even if the Hub goes offline, Spoke clusters continue to operate with the last known configuration. In this tutorial, we will deploy Octopus using Helm, connect a local cluster and a remote cluster, deploy a sample application with geo-distribution, and then monitor the results. By the end, you will have a fully functional multi-cluster setup ready for production scenarios.
Step 1: Prerequisites and Installation of Octopus
To follow this tutorial, you need at least two Kubernetes clusters: one will serve as the Hub, and one or more as Spokes. For simplicity, we will use a local Minikube cluster as the Hub and a second Minikube instance (or a cloud-hosted cluster) as the first Spoke. All clusters should run Kubernetes version 1.21 or later, as Octopus relies on features like CustomResourceDefinitions and aggregated API servers. On your local machine, you also need kubectl, helm (version 3 or above), and access to both clusters via kubeconfig contexts. If you are using Minikube, start two profiles: minikube start -p hub and minikube start -p spoke1. Verify that both clusters are accessible by running kubectl config get-contexts. Once your infrastructure is ready, install Octopus on the Hub cluster. The recommended method is via the official Helm chart, which handles all CRDs, the Hub API server, and the controller manager.
Begin by adding the Octopus Helm repository: helm repo add octopus https://kubeedge.github.io/octopus/ and then update it with helm repo update. Now, install Octopus on the Hub cluster. Make sure your current kubectl context points to the Hub cluster (e.g., kubectl config use-context hub). Run the following command: helm install octopus octopus/octopus --namespace octopus-system --create-namespace --set hub.ingress.enabled=true. This installs the Hub components along with an ingress (you may need to adjust the ingress domain based on your environment). The installation process deploys several pods: octopus-hub (the API server), octopus-controller-manager, and octopus-agent-manager. Wait until all pods are in Running state using kubectl get pods -n octopus-system. You should also verify that the CRDs are installed: kubectl get crd | grep octopus. If you see resources like clusters.octopus.io and resourcetemplates.octopus.io, the installation succeeded. The Hub cluster now runs the central control plane, but it is not yet connected to any Spoke clusters. In the next step, we will configure the Hub address and then initiate the join process for the Spoke clusters.
Step 2: Configuring the Hub and Preparing Spoke Clusters
Before Spoke clusters can register with the Hub, you need to generate a join token and expose the Hub’s endpoint to the Spokes. The join token is a signed secret that authorizes a Spoke to connect. On the Hub cluster, use the octopusctl command-line tool (or the kubectl equivalent) to create a token. The Octopus Helm chart also installs the octopusctl binary inside the hub pod. For convenience, you can download the octopusctl binary from the Octopus release page and place it in your PATH. Alternatively, you can use kubectl to create a ClusterJoinToken resource: kubectl apply -f - <kubectl get clusterjointoken mytoken -n octopus-system -o jsonpath='{.status.token}'. The output is a base64-encoded string that you will use on each Spoke.
Next, decide how Spoke clusters will reach the Hub. If your clusters are on the same network (e.g., both Minikube VMs on the same host), you can use the Hub’s cluster IP or the ingress hostname. For production, you should set up a proper DNS record pointing to the Hub ingress. On the Hub cluster, note the ingress address: kubectl get ingress -n octopus-system. If you are using Minikube, enable the ingress addon and use minikube ip -p hub to get the IP, then add a host entry like octopus.hub.local pointing to that IP in your /etc/hosts file. For this tutorial, we will assume the Hub address is octopus.hub.local:443. Now, switch the kubectl context to the first Spoke cluster (kubectl config use-context spoke1). On the Spoke, you need to install the Octopus agent. The agent is a lightweight pod that communicates with the Hub. You can install it using a Helm chart as well, but a simpler method is using a YAML manifest generated by the Hub. On the Hub cluster, run octopusctl get join-script --token mytoken --server https://octopus.hub.local:443 --output spoke-install.yaml. This command creates a YAML manifest that, when applied to a Spoke, will deploy the agent and automatically configure it with the token and Hub endpoint. Transfer this manifest to the machine where you control the Spoke cluster and apply it: kubectl apply -f spoke-install.yaml. After a few seconds, the agent pod appears in the octopus-system namespace on the Spoke. Verify connection by checking the logs: kubectl logs -n octopus-system -l app=octopus-agent. You should see messages like “Connected to Hub successfully”. Now the Spoke is registered and ready to accept workloads from the Hub.
Step 3: Registering and Managing Multiple Spoke Clusters
Repeat the join process for each additional Spoke cluster you want to manage. For the second Spoke (spoke2), generate a new token or reuse the same one if it has not expired. Apply the join script as before. After all Spokes are connected, verify their status from the Hub perspective. Switch the kubectl context back to the Hub and run: kubectl get clusters -n octopus-system. The output shows the Spoke clusters with their health status: Ready, NotReady, or Unknown. A Ready cluster means the agent heartbeat is current and the Spoke can accept deployments. If a cluster shows NotReady, check the Spoke’s agent logs and network connectivity. Octopus also stores cluster metadata such as node count, available resources, and region labels. You can examine details with kubectl describe cluster spoke1 -n octopus-system. The status section includes capacity and allocatable resources per node, which Octopus uses for scheduling decisions.
Managing clusters at scale involves setting labels and annotations on the Cluster objects. For example, you may want to mark a cluster as belonging to a specific region or having GPU capabilities. Use the kubectl label cluster spoke1 region=us-east gpu=true command. These labels become crucial when defining distribution policies in the next step. Octopus also allows you to pause a cluster temporarily. If a Spoke requires maintenance, run kubectl patch cluster spoke1 -p '{"spec":{"paused":true}}'. This stops the Hub from scheduling new deployments to that cluster but does not evict running workloads. When maintenance is complete, set paused back to false. The flexibility of the Cluster CRD ensures that you can treat each Spoke as a first-class citizen while maintaining centralized oversight. You can also delete a Spoke cleanly by running kubectl delete cluster spoke1, which triggers a cleanup of all Octopus-related resources on that Spoke.
| Component | Location | Role |
|---|---|---|
| Octopus Hub | Hub Cluster | Central API server, controller manager, and ingress. Manages CRD state, schedules workloads, and exposes endpoints. |
| Octopus Spoke Agent | Spoke Cluster | Lightweight daemon that syncs state with Hub, executes local reconciliation, and reports health and metrics. |
| Cluster CRD | Hub Cluster | Stores metadata, labels, and connection status of each Spoke cluster. |
| ResourceTemplate CRD | Hub Cluster | Defines the desired resource (e.g., Deployment, Service) as a template with distribution rules. |
Step 4: Deploying Applications Across Clusters Using ResourceTemplates
Octopus introduces the ResourceTemplate CRD as the primary way to deploy workloads across multiple Spoke clusters. A ResourceTemplate contains a full Kubernetes resource manifest (e.g., a Deployment or DaemonSet) combined with a distribution policy that specifies which clusters should receive that resource. The distribution policy can include cluster label selectors, weight-based scheduling, or even geographic constraints. This is where Octopus truly shines: instead of manually applying the same YAML to each cluster, you define it once and let Octopus handle the propagation.
Create a file called myapp-resourcetemplate.yaml with the following content:
apiVersion: octopus.io/v1alpha1
kind: ResourceTemplate
metadata:
name: myapp
namespace: default
spec:
template:
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
labels:
app: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: nginx
image: nginx:1.25
ports:
- containerPort: 80
distribution:
clusters:
matchLabels:
region: us-east
This example creates a Deployment of three nginx replicas, but it will only be applied to clusters labeled with region=us-east. You can also specify matchExpressions for more complex logic, or use removeClusters to exclude specific clusters. Apply the ResourceTemplate on the Hub: kubectl apply -f myapp-resourcetemplate.yaml. After a few seconds, Octopus creates a ResourceBinding object that tracks the state of the Deployment on each matching Spoke. You can check the status: kubectl get resourcebinding -n default. The output shows the expected state, actual state, and any errors. To see the deployment on a Spoke, switch to that Spoke context and run kubectl get deployments -n default. You should see the myapp deployment running with three replicas. If a Spoke does not have enough resources, Octopus will report a scheduling failure in the ResourceBinding status.
Octopus also supports overriding parts of the template per cluster. For example, you might want a larger replica count in a region with more traffic. Add a overrides section inside the distribution policy:
distribution:
clusters:
matchLabels:
region: us-east
overrides:
- clusterName: spoke1
replicas: 5
This tells Octopus to increase the replica count to 5 specifically for spoke1 while keeping 3 for other us-east clusters. Overrides can apply to any field in the template, including container images, environment variables, and resource limits. This capability eliminates the need to maintain separate YAML files for each cluster and reduces configuration drift.
Step 5: Monitoring, Updating, and Scaling Deployments
Once your application is deployed, you need to monitor its health across clusters and update it as needed. Octopus provides a built-in monitoring mechanism through the ResourceBinding status and the Hub’s aggregated API. You can query the aggregated status of all deployments directly from the Hub without having to access each Spoke individually. Run kubectl get resourcebinding myapp -n default -o yaml to see a comprehensive report: the status section lists each Spoke, the current state (Running, Pending, Error), and the last sync time. If a Spoke agent becomes unreachable, Octopus will mark that Spoke’s binding as Unknown and continue to report the last known state.
To update the application, simply modify the ResourceTemplate’s template field. For instance, change the container image from nginx:1.25 to nginx:1.26 and reapply: kubectl apply -f myapp-resourcetemplate.yaml. Octopus detects the diff and propagates the change to all affected Spokes. It uses a rolling update strategy by default, but the Spoke’s native deployment controller handles the actual rollout configuration. If you want to pause updates to all clusters, you can patch the ResourceTemplate: kubectl patch resourcetemplate myapp -p '{"spec":{"suspend":true}}'. This prevents any new changes from being pushed. Resume by setting suspend to false.
Scaling the application across clusters is similarly straightforward. Instead of editing each cluster’s Deployment, you change the replicas field in the template or use overrides. For global scaling, update spec.template.spec.replicas to 5. For targeted scaling, add an override for a specific cluster. Octopus also integrates with HorizontalPodAutoscaler (HPA) on each Spoke. You can deploy an HPA as part of the ResourceTemplate by including it in the same template. Octopus will create the HPA on each Spoke, allowing local autoscaling. However, be aware that Octopus does not federate HPA metrics across clusters; each Spoke makes independent scaling decisions. This is generally acceptable because scaling should be driven by local load.
Tips and Best Practices for Using Octopus in Production
1. Secure Inter-Cluster Communication with TLS and Mutual Authentication. By default, Octopus uses TLS for the Hub’s gRPC endpoint, but you must provide your own certificates in production. The Helm chart allows you to specify existing TLS secrets via --set hub.tls.certSecretName=my-cert. For mutual TLS (mTLS) where the Spoke agent also presents a certificate, enable the agent’s TLS settings. This prevents unauthorized clusters from joining your Hub and ensures that all data in transit is encrypted. Always rotate certificates regularly and consider using a certificate management tool like cert-manager to automate this process. In addition, restrict the ClusterJoinToken expiration to the shortest practical window (e.g., 48 hours) and revoke tokens immediately after use.
2. Use Cluster Labels for Fine-Grained Deployment Policies. One of Octopus’s most powerful features is label-based distribution. Invest time in defining a consistent labeling strategy for your Spoke clusters. Common labels include region, environment (dev/staging/prod), node-type (gpu/cpu), and criticality. Then, in each ResourceTemplate, use matchLabels or matchExpressions to target exactly the clusters that should run the workload. Avoid using a single global ResourceTemplate that applies to all clusters unless the workload is truly global (e.g., monitoring agents). Instead, create separate templates for different tiers or regions. This approach reduces the blast radius of misconfigurations and makes rollbacks easier.
3. Monitor Resource Quotas and Node Capacity to Avoid Over-Scheduling. Octopus uses the capacity information reported by each Spoke agent to make scheduling decisions. However, it does not enforce resource quotas by itself. If you have multiple ResourceTemplates that target the same cluster, it is possible to overcommit resources and cause pod eviction. To prevent this, define ResourceQuota objects on each Spoke cluster or use Octopus’s built-in limitRange overrides. Additionally, monitor the cluster.status.capacity fields regularly to ensure you are not pushing a Spoke beyond its limits. You can set up alerts in Prometheus by scraping the Hub’s metrics endpoint, which exposes metrics like octopus_binding_ready_count and octopus_spoke_heartbeat_missed_count. If a Spoke’s available memory drops below a threshold, you can temporarily pause that cluster in the distribution policy to allow it to catch up.
| Command | Purpose | Example |
|---|---|---|
| octopusctl get join-script | Generate a YAML manifest to install agent on a Spoke | octopusctl get join-script --token mytoken --server https://hub.example.com |
| kubectl get clusters -n octopus-system | List all registered Spoke clusters and their status | kubectl get clusters |
| kubectl label cluster <name> <k=v> | Add or update labels on a Spoke cluster | kubectl label cluster spoke1 region=us-west |
| kubectl describe resourcetemplate <name> | See the full view of a ResourceTemplate and its bindings | kubectl describe resourcetemplate myapp |
| kubectl patch cluster <name> -p '{"spec":{"paused":true}}' | Temporarily suspend scheduling to a cluster | kubectl patch cluster spoke2 -p '{"spec":{"paused":true}}' |
Frequently Asked Questions (FAQ) About Octopus for Kubernetes
Q1: What happens if a Spoke cluster loses connectivity to the Hub?
When a Spoke loses connectivity, the agent on that Spoke stops receiving updates from the Hub, but the existing workloads continue to run normally based on the last known ResourceTemplate state. The Hub marks that Spoke as NotReady and stops scheduling new deployments to it. Once connectivity is restored, the agent resyncs the full state and applies any changes that were queued. Workloads are not automatically scaled down unless the agent detects a persistent conflict. This design ensures resilience in distributed environments, especially edge scenarios with intermittent network links.
Q2: Can I integrate Octopus with existing CI/CD pipelines (e.g., GitOps via ArgoCD)?
Yes. Since Octopus is Kubernetes-native, you can manage ResourceTemplates as GitOps artifacts in a Git repository and use a tool like ArgoCD or Flux to sync the Hub cluster. When ArgoCD sees a change in a ResourceTemplate YAML, it applies it to the Hub, and Octopus propagates it to the Spokes automatically. However, note that ArgoCD’s health checks may not natively understand ResourceTemplate status; you may need to configure custom resource health checks or use Octopus’s aggregated status through the Hub’s API. Alternatively, you can keep the CI/CD pipeline focused on the Hub and let Octopus handle the rest.
Q3: Is Octopus suitable for managing very large clusters (hundreds of nodes per Spoke) and many Spokes (50+) ?
Octopus is designed with scalability in mind. The Hub uses an event-driven architecture and gRPC streaming to minimize overhead. In practice, a single Hub can manage up to 200 Spokes and thousands of nodes, provided the Hub cluster itself has sufficient resources (CPU, memory, and network bandwidth). For extreme scales, you can deploy multiple Hub instances in an active-standby config or use horizontal scaling of the Hub controller. The Spoke agents are lightweight and consume minimal resources (typically less than 100MB RAM per agent). Performance bottlenecks usually arise from the Hub’s etcd store; ensure your Hub cluster uses dedicated etcd nodes with fast SSDs and ample storage.
Q4: How does Octopus handle conflicts when two ResourceTemplates try to manage the same resource on the same Spoke?
Octopus uses a first-writer-wins semantic combined with owner references. If two ResourceTemplates define a Deployment with the same name and namespace on the same Spoke, the one that was created first will be the owner. Any subsequent ResourceTemplate attempting to modify that same resource will fail with a conflict error in its ResourceBinding. The error is visible when describing the ResourceTemplate. To avoid conflicts, adopt a naming convention that prefixes ResourceTemplate names with the application or team name, and use separate namespaces for different applications. Octopus does not enforce global resource uniqueness across all Spokes; it only prevents conflicts on the same Spoke.
Q5: Can I use Octopus to manage Secrets and ConfigMaps across clusters?
Yes, ResourceTemplate supports any Kubernetes resource type, including Secrets, ConfigMaps, Services, and Ingresses. This allows you to distribute configuration and credentials uniformly. However, be careful with Secrets: Octopus transmits them over the gRPC channel that is already TLS-encrypted. For highly sensitive Secrets, consider encrypting the data in the ResourceTemplate YAML itself using a tool like Sealed Secrets or Mozilla SOPS, and then decrypting on the Spoke via a controller. Octopus also supports the use of user-defined override fields to inject cluster-specific configuration (e.g., a different database URL for each region). Simply add the override under the distribution section in the ResourceTemplate.
Conclusion
Octopus offers a powerful, Kubernetes-native approach to multi-cluster management that scales from a handful of development clusters to hundreds of production environments spanning edge, cloud, and on-premises. By following this tutorial, you have installed the Hub and Spoke components, registered clusters, deployed a ResourceTemplate with distribution policies, and learned how to update and monitor applications across clusters. The key takeaways are Octopus’s ability to unify disparate clusters into a single logical control plane while respecting unique cluster configurations through overrides and labels. We also covered best practices for security, labeling, and resource governance that will help you avoid common pitfalls in production. As you expand your multi-cluster strategy, explore Octopus’s advanced features such as federated monitoring, integration with Service Mesh, and custom scheduler algorithms. The project’s active community and growing ecosystem make it a solid foundation for any organization seeking to tame multi-cluster complexity. Start small, build confidence with a few clusters, and then confidently roll out Octopus across your entire fleet.