Disable the "insight" cluster capability by default on OKD-SCOS ?

[titou10titou10]

IMHO the "insight" cluster capability should be disabled on OKD-SCOS clusters as it is not permit to specify a RH pull-secret on installation and the insight capability is directly linked to the cloud OCP console

Ref:
https://docs.okd.io/latest/installing/overview/cluster-capabilities.html
https://docs.okd.io/latest/installing/overview/cluster-capabilities.html#insights-operator_cluster-capabilities

If you let the insight capability enabled without a RH secret, this alert is raised and stays forever:

InsightsDisabled

Insights operator is disabled. In order to enable Insights and benefit from recommendations specific to your cluster, please follow steps listed in the documentation: https://docs.openshift.com/container-platform/latest/support/remote_health_monitoring/enabling-remote-health-reporting.html

[BeardOverflow]

It is certainly an annoying alert. You could mute the alert, but ideally there should be a way to fine-customize the dashboard and alerts. For example, insights is a bare plugin in the console front-end, but I think it could be a dynamic console-plugin like networking or monitoring.

[titou10titou10]

@BeardOverflow disabling the insight capability on install (As I do..) solves the problem.
As said, as it is not "legal" (?) now to install OKD with a RH linked account and the insight capability is the link between OKD and RH, this capability should not even be possible to activate in OKD IMHO.
Or I miss something?

[Markus-feu]

Hi,
could you tell how you disabled the insights operator on install? I can't find anything within the documentation that gives me a hint on how to do so.

I only see how to enable additional cluster capabilities, and potentially other users face the same issue.

best regards

[BeardOverflow]

@Markus-feu Customize your install-config.yaml. A brief example, for SNO:

apiVersion: v1
baseDomain: domain.tld
compute:
- name: worker
  replicas: 0
controlPlane:
  name: master
  replicas: 1
metadata:
  name: okd
networking:
  networkType: OVNKubernetes
  clusterNetwork:
    - cidr: 10.128.0.0/14
      hostPrefix: 23
  machineNetwork:
    - cidr: 10.0.0.0/16
  serviceNetwork:
    - 172.30.0.0/16
platform:
  none: {}
capabilities:
  baselineCapabilitySet: None
  additionalEnabledCapabilities:
    - CSISnapshot
    - Console
    - ImageRegistry
    - Ingress
    - NodeTuning
    - OperatorLifecycleManager
    - Storage
bootstrapInPlace:
  installationDisk: /dev/sda
pullSecret: '{"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}}'
sshKey: |
  ssh-rsa AAAA... [email protected]

See baselineCapabilitySet and additionalEnabledCapabilities. With baselineCapabilitySet, you configure a predetermined set of capabilities for the current release. With additionalEnabledCapabilities, you can add new capabilities over the set. So, choose None set and add your favourite capabilities as additions.

The above example removes insight, deploymentconfig, baremetal, CloudCredential and marketplace operator. Why?

• Insight capability: take down the telemetry
• Deploymentconfig: safe to remove, deprecated over Deployment
• baremetal/MachineAPI: enables Machine Config Operator, but my platform is proxmox - useless for me (MachineAPI also removed)
• CloudCredential; useless for me, on-premise cluster (CloudControllerManager also removed)
• marketplace: hard to configure on OKD. I prefer install the cluster without marketplace and to install it later using oc patch clusterversion/version --type merge -p '{"spec":{"capabilities":{"additionalEnabledCapabilities":["marketplace"]}}}' and https://github.com/orgs/okd-project/discussions/2095#discussioncomment-11980923 (openshift-sample also removed)

Documentation: https://docs.okd.io/latest/installing/overview/cluster-capabilities.html#viewing-cluster-capabilities_cluster-capabilities
Manifest (by default, all are installed, compare them between releases to see the changes): https://github.com/openshift/installer/blob/release-4.19/data/data/install.openshift.io_installconfigs.yaml#L1305-L1354
Enums: https://github.com/openshift/api/blob/release-4.19/config/v1/types_cluster_version.go#L279-L401

---

A warning about the telemetry:

Insight capabilities routes telemetry for the nodes on the cluster, but telemetry still exists on the console as a console-plugin

oc get configmap console-config -n openshift-console -o yaml # configuration
oc get configmap telemetry-config -n openshift-console-operator -o yaml # api-keys
oc get deployment telemeter-client -n openshift-monitoring -o yaml # pod

To disable it:

Extract it:
oc get configmap console-config -n openshift-console -o yaml > console-config.yaml

Edit console-config.yaml and change TELEMETER_CLIENT_DISABLED to true:
  console-config.yaml: |
    apiVersion: console.openshift.io/v1
    kind: ConsoleConfig
    telemetry:
      TELEMETER_CLIENT_DISABLED: "true"

And apply it:
oc apply -f console-config.yaml

See related discussion: https://github.com/openshift/console/pull/13677/files

[BeardOverflow]

disabling the insight capability on install (As I do..) solves the problem.

Oh, good one! The sad part is that it is not possible to disable it on existing clusters or, even, an upgrade would silently enable it.

I will reference the docs for other users who could able are reading us: https://docs.okd.io/4.17/installing/overview/cluster-capabilities.html#viewing-cluster-capabilities_cluster-capabilities

Also, I found you can create the following ConfigMap to disable alerts related Insights capability (which is better than mute it):

apiVersion: v1
kind: ConfigMap
metadata:
  name: insights-config
  namespace: openshift-insights
data:
  config.yaml: |
    alerting:
       disabled: true

As said, as it is not "legal" (?) now to install OKD with a RH linked account and the insight capability is the link between OKD and RH

I am not the most appropriate person to answer your question about legal issues, but I understand that the main problem is mixing Red Hat operators in OKD:

Can I use a Red Hat pull secret to enable OCP-related content such as operators?
A. No, this would be a violation of Red Hat's Terms of Service.

Source: https://okd.io/docs/project/scos-migration-faq#redhat-pull-secrets

But the following issue partially contradicts, at least partially?, the above statement:

If you do not use the pull secret from the Red Hat Hybrid Cloud Console site:

• Red Hat Operators are not available.
• The Telemetry and Insights operators do not send data to Red Hat.
• Content from the Red Hat Container Catalog registry, such as image streams and Operators, are not available.

Source: #107

Under my knowledge, you must not use Red Hat Operators, but Community Operators so as not to violate the legal terms. In the another side, Insights operator is naturally integrated on OKD ecosystem and, according to the OKD's docs, you can register your Red Hat Hybrid Cloud Console account on it.

The Insights Operator gathers OKD configuration data and sends it to Red Hat. The data is used to produce insights about potential issues that a cluster might be exposed to.

Source: https://docs.okd.io/4.17/support/remote_health_monitoring/about-remote-health-monitoring.html#about-remote-health-monitoring

You can use Insights Advisor to assess and monitor the health of your OKD clusters. Whether you are concerned about individual clusters, or with your whole infrastructure, it is important to be aware of the exposure of your cluster infrastructure to issues that can affect service availability, fault tolerance, performance, or security.

Source: https://docs.okd.io/4.17/support/remote_health_monitoring/using-insights-to-identify-issues-with-your-cluster.html

As a community user, I think we need a strong statement about the scope of the legal terms, well-documented, updated and easy to consult.

[JaimeMagiera]

The product docs are auto-generated from OCP source and as such may have some inaccuracies. We're working to remedy that by sifting through them, but could use more volunteers. If you see something that is questionable, please put in a separate issue labeled as docs. Even better, if you'd like to volunteer for OKD, we could use the help.

The legal terms are this: Don't use a Red Hat pull secret. Any operator that doesn't work without a Red Hat pull secret should be considered non-functional on OKD. If you want to see that operator available on OKD, please put in a separate issue labeled as operator request. Even better, if you'd like to volunteer for OKD, we could use the help.

You can see what operators are being worked on here: OKD Operators

[titou10titou10]

For reference, my owninstall-config.yamlexcerpt about capabilities
I have multiple v4.19 clusters (full, SNO..) on Proxmox baremetal UPI, install via ABI. Not using built-in "build related" things, registry, storage. Works like a charm since 4.18

capabilities:
  baselineCapabilitySet: None
  additionalEnabledCapabilities: 
 #- Build
  - CSISnapshot
 #- CloudControllerManager
 #- CloudCredential
  - Console
 #- DeploymentConfig
 #- ImageRegistry
  - Ingress                    # 4.16+
 #- Insights                   # OKD disconnected from RH...
 #- MachineAPI                 # Mandatory with "baremetal" IPI
  - NodeTuning
  - OperatorLifecycleManager
  - OperatorLifecycleManagerV1 # 4.18+
 #- Storage
 #- baremetal
  - marketplace
 #- openshift-samples