Getting started with vClusters

vCluster overview

vCluster is an open-source solution that lets teams run virtual Kubernetes clusters inside existing infrastructure. It enables platform engineers to create secure, isolated environments for development, testing, CI/CD, and even production—without the cost and overhead of managing separate physical clusters.

vCluster supports multiple tenancy models, ranging from lightweight namespace-based setups to advanced configurations with private nodes, GPUs, and bare metal. Clusters are defined declaratively, making it easy to provision repeatable environments tailored to specific isolation and performance requirements.

By consolidating workloads onto fewer host clusters, vCluster helps reduce infrastructure sprawl, lower Kubernetes costs, and simplify multi-tenant platform operations.

vCluster components

Control plane

  • Each vCluster creates its own isolated Kubernetes environment in a host cluster.
  • The vCluster instantiation contains it’s own virtual control plane, orchestrating operations within the virtual cluster and facilitating interaction with the underlying host cluster.
  • Contains a virtual control plane deployed as a StatefulSet (default) or Deployment, with:
    • Kubernetes API Server: Manages all API requests.
    • Controller Manager: Maintains desired state of resources.
    • Data Store: A connection to or mount of the data store where the API stores all resources.Defaults to embedded SQLite database but can also use etcd, MySQL, PSG.
    • Syncer: Syncs resources between vCluster and host.
    • Scheduler (optional): By default, uses host scheduler to save on computing resources. Custom schedulers are possible.

Syncer

  • No worker nodes or network in vCluster. Syncer maps pods to the host cluster.
  • Host schedules pods, while Syncer keeps vCluster and host in sync.
  • he Syncer component synchronizes low-level components of the vCluster pod resources (e.g., ConfigMaps, Secrets) are synced.
  • Bi-directional sync:
    • vCluster → host cluster
    • Host cluster → vCluster

More details on Syncer

Host Cluster relationship

Host cluster relationship

Namespaces / Resources within the host cluster

Suppose we have a Kosmos created aks cluster → my-vcluster-aks

  • Example:
kosmos list aks --fleet <FLEET_NAME>
│ CLUSTER │ FLEET │ CREATED AT │ STATUS │
│ CLUSTER_NAME │ FLEET_NAME │ Mar 10, 2025 10:10:22 │ ready │
kubectl config current-context
CLUSTER_NAME

Using CLUSTER_NAME as our host, we created two vClusters in Devspace -> DEVSPACE_NAME

kosmos list vclusters --devspace <DEVSPACE_NAME>
│ NAME │ CLUSTER │ DEVSPACE │ NAMESPACE │ STATUS │ AGE │
│ vcluster-A │ CLUSTER_NAME_CLUSTER_ID │ DEVSPACE_NAME │ loft-p-qe-allclusters-devspace │ Ready │ 2d17h │
│ vcluster-B │ CLUSTER_NAME_CLUSTER_ID │ DEVSPACE_NAME │ loft-p-qe-allclusters-devspace │ Ready │ 2d17h │
kubectl get namespaces

kubectl get all,cm,secret,ing -n loft-qe-allclusters-devspace-v-byoh-vcluster-a

  • Key points:

    • vCluster objects (ingress, services) are mapped to host cluster with naming convention:

      ingress.<name>-x-<namespace>-x-<vcluster-name>
service.<name>-x-<namespace>-x-<vcluster-name>

Communication from the host cluster

  • Host pods can access vCluster services when replicated to host namespaces.
  • Enables inter-workload communication.

Communication within a virtual cluster

  • Pod → Pod: Handled by host’s networking infrastructure.
  • Pod → Service: Resolved by vCluster’s CoreDNS.
  • Pod → Host Cluster Service: Host services can be replicated into vCluster.
  • Pod → Another vCluster Service: Achieved via DNS configurations.

vClustertemplates

  • Created with kosmos create vclustertemplate.
  • YAML CRD format:
apiVersion: storage.kosmos.spcplatform.com/v1
kind: VirtualClusterTemplate
metadata:
  name: <template-name>
spec:
  displayName: <template-name>
  description: <description>
  template:
    accessPoint:
      ingress:
        enabled: <true|false>
    helmRelease:
      chart:
        version: <int>
      values: |
        sync:
          toHost:
            ingresses:
              enabled: <true|false>
          fromHost:
            nodes:
              enabled: <true|false>
            ingressClasses:
              enabled: <true|false>

  • Key values:

    • sync.toHost.ingresses: Sync ingress to host.
    • sync.fromHost.ingressClasses: Sync host ingress classes to vCluster.

  • Defaults: Loft vCluster chart values

vCluster creation

CLI Method 

kosmos create vcluster \
  --cluster <host-cluster-name> \
  --description <description> \
  --devspace <devspace> \
  --display-name <display-name> \
  --name <vcluster-name> \
  --template <template> \
  --version <version> \
  --user <user>

YAML Method 

apiVersion: storage.kosmos.spcplatform.com/v1
kind: VirtualCluster
metadata:
  name: <vcluster-name>
  namespace: <devspace>
spec:
  clusterRef:
    cluster: <host-cluster-name>
    namespace: loft-<devspace>-v-<vcluster-name>
    virtualCluster: <vcluster-name>
  description: <description>
  displayName: <display-name>
  owner:
    user: <kosmos-user>
  sleepModeConfig:
    timezone: PDT#-25200
  templateRef:
    name: <vclustertemplate-name>
    version: <version>

vCluster Interaction

  • Each vCluster has its own kube context.
  • Use kosmos use vcluster <name> to switch to vCluster context.
kosmos list vclusters
kosmos use vcluster byoh-vcluster-a
kubectl config current-context
kubectl get namespaces

vCluster Ingress

  • Host cluster ingress controllers can be shared with vClusters.

  • Benefits:

    • Shared load-balancer on host
    • Simpler DNS

  • Setup:

    • Enable spec.template.accessPoint.ingress.enabled: true
    • Ingress from vCluster → host
    • IngressClass from host → vCluster
    • Host ingress uses DNS A record pointing to public IP of load-balancer.

vCluster DNS

  • Each vCluster deploys CoreDNS internally.
  • Syncer maps service names to host IPs, following Kubernetes DNS naming rules.

In this section

Edit this page on GitHub