Kosmos Managed Observability

Kosmos offers a managed observability solution for metrics, logs, and tracing telemetry data. Each Fleet and DevSpace may opt into the managed observability solution by enabling observability in the spec section of the resource.

Example fleet resource #

apiVersion: storage.kosmos.spcplatform.com/v1
kind: Fleet
metadata:
  name: observability-enabled-fleet
spec:
  displayName: observability-enabled-fleet
  owner:
    user: fleet-owner
  observability: {}
status: {}

When observability is enabled (exists in the spec), Kosmos provisions a Grafana organization with preconfigured data sources and dashboards. If the observability key is later removed, or the Fleet/DevSpace is deleted, the Grafana organization will also be deleted. Re-enabling observability creates a new Grafana organization. It will not restore any previous Grafana organizations.

Members from the Fleet and DevSpace are mapped to the Grafana organization using their respective roles. For example, a Fleet admin becomes an admin in the Grafana organization.

Note: Observability is part of usual onboarding process and most accounts would have it enabled by default in thier account. Please check for bservability: {} tag in your Fleet/DevSpace YAML.

Grafana endpoints #

The Grafana UI endpoint to check you telemetry.

Endpoint: https://metrics.kosmos.spcplatform.com/

Logging in #

On the Grafana login screen, click the Sign in with Kosmos button.

If you have access to multiple Fleets and/or DevSpaces, an organization selector will appear in the top left corner of the Grafana UI. Use this to switch between organizations.

Note: If you are added to new Fleets/DevSpaces while logged in, log out and log back in to see them.

Grafana data sources #

After logging in, navigate to Data sources to view telemetry data sources.

• Loki → log data
• Mimir → metrics data
• Tempo → tracing data

Each data source uses a preconfigured HTTP Authorization header for authentication. Do not modify this value.

Scroll to the bottom and click Save & test. This should succeed.

You may add additional data sources for your applications, but Kosmos will not manage them.

Preconfigured dashboards #

Each Grafana organization is provisioned with a set of Kubernetes dashboards.

Each dashboard includes a cluster drop down at the top. Use this (and other drop downs) to select the correct cluster.

Note: Dashboards will remain empty until clusters are configured to send telemetry data.

Kosmos telemetry helm chart #

You can download the Kosmos Telemetry Helm chart from oci://public-ecr.samsungspc.com/b94496fa/kosmos-telemetry which installs a preconfigured Grafana Alloy instance into a Kubernetes cluster or vcluster.

Clone the repo locally and follow the further instructions.

Note: If you do not have access to this repo, please contact Kosmos PMO @ kosmospmo@joyent.com to request Telemetry helm chart.

Required values #

• observabilityGatewayUrl – URL to send metrics to (see Grafana Endpoints )
• authToken – secret associated with the GrafanaTenant

Note: For Auth token, please contact Kosmos PMO @ kosmospmo@joyent.com

• fleetName – (optional) name of the Kosmos Fleet where the collector is installed
• devspaceName – (optional) name of the Kosmos DevSpace where the collector is installed

Below is an example values.yaml that is used to install the helm chart into a vcluster.

Example values.yaml for a vcluster #

observabilityGatewayUrl: https://metrics.kosmos.spcplatform.com
fleetName:
devspaceName: demo-devspace
authToken: 2e7...a4
k8s-monitoring:
  cluster:
    name: demo
  logs:
    pod_logs:
      gatherMethod: api
  alloy-logs:
    alloy:
      clustering:
        enabled: true
  alloy:
    controller:
      replicas: 1
  prometheus-node-exporter:
    enabled: false
  kube-state-metrics:
    enabled: true
kube-prometheus-stack:
  enabled: false

If you are installing into a fleet, you can install the kube-prometheus-stack to collect more monitoring metrics. Below is an example of what the values.yaml can look like.

Example values.yaml for a fleet #

observabilityGatewayUrl: https://metrics.kosmos.spcplatform.com
fleetName:
authToken: 2e7...a4
k8s-monitoring:
  cluster:
    name: demo
  logs:
    pod_logs:
      gatherMethod: api
  alloy-logs:
    alloy:
      clustering:
        enabled: true
  alloy:
    controller:
      replicas: 1
  prometheus-node-exporter:
    enabled: false
  kube-state-metrics:
    enabled: true
kube-prometheus-stack:
  enabled: true

Install the helm chart #

With the above values.yaml and the vcluster set as the current kube context:

helm upgrade kosmos-telemetry ./kosmos-telemetry/ --values values.yaml

Edit this page on GitHub