Release notes

Release 2.33.0

2026/04/16

Release notes

• Upgrade Kubernetes 1.33.7
• Support for Traefik Deployment instead of DeamonSet mode
• Support for custom proxy settings per Environment
• Update Traefik and Nginx Ingress controllers

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2025/04/23/kubernetes-v1-33-release/
• Please make sure you have a backup of your data before running an update.
• This version requires Ubuntu 24.04 or newer. Please update all VMs before running Kubernetes upgrade.
• New type of Traefik installation mode is supported - using Deployment. There is an additional flag to control this: when true, Traefik is installed as Deployment. In that case NSX-T LB uses k8s_ingress_default_pool_member_ports. When false, Traefik is DaemonSet (hostPort 80/443) and LB member ports are not set.",

Required steps

• New variable in Environment configuration file config.json:

{
    "_comment": "When true, Traefik is installed as Deployment; NSX-T LB uses k8s_ingress_default_pool_member_ports. When false, Traefik is DaemonSet (hostPort 80/443) and LB member ports are not set.",
    "name": "traefik_ingress_deployment_enabled",
    "value": "false"
},
{
    "_comment": "HTTP proxy URL",
    "name": "http_proxy",
    "value": "((http_proxy))"
},
{
    "_comment": "HTTPS proxy URL",
    "name": "https_proxy",
    "value": "((https_proxy))"
},
{
    "_comment": "Comma-separated list of hosts/CIDRs to bypass proxy (final value adds k8s_master_ips to the list)",
    "name": "no_proxy",
    "value": "((no_proxy))"
}

Release 2.32.0

2025/09/16

Release notes

• Upgrade Kubernetes 1.32.8
• Update certificate handling policy
• Update Keycloak
• Update Velero
• Update Harbor
• Support multiple DNS entries

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2024/12/11/kubernetes-v1-32-release/
• Please make sure you have a backup of your data before running an update.
• There is a new Keycloak version which is a breaking change. Make sure to have your Keycloak backed up.
• By default there is containerd v2 used as a container runtime and there are some important changes: https://github.com/containerd/containerd/releases/tag/v2.0.0 (e.g. docker schema v1 is not supported: https://cloud.google.com/kubernetes-engine/docs/deprecations/migrate-containerd-2#migrate-schema-1)

Required steps

• New variable in Environment configuration file config.json:

{
   "name": "public_dns_ips",
   "value": ["8.8.8.8", "8.8.4.4"]
}

---

Release 2.31.0

2025/05/15

Release notes

• Upgrade Kubernetes 1.31.4
• Change provisioning model from packages to Docker image
• Update certificate handling policy
• Add Keycloak integration with Kibana and Harbor
• Add parallelism flag for Kubernetes terraform execution

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2024/08/13/kubernetes-v1-31-release/
• Please make sure you have a backup of your data before running an update.
• In this release we moved from packages to a Docker image. So the Opscontrol installation process has changed. There is no longer terraform required on the machine. Instead Docker is required. Opscontrol requires configuration which consists of three files:

File name	File description
backend.tf	Terraform backend configuration.
terraform.tfvars	Terraform variables.
storage_config.yml	Cloudboostr storage configuration.

export CB_IMAGE=""                # Unique name of the Docker image (optional)
export CB_REGISTRY=""             # Docker registry from which Docker image should be pulled.
export CB_PROVIDER=""             # Name of the provider to use (aws|azure|vsphere)
export CB_CONFIG_HOST_DIR=""      # Absolute path to Opscontrol configuration directory on the host machine.
export CB_CONFIG_CONTAINER_DIR="" # Absolute path to Opscontrol configuration directory inside Docker containter.

# Run terraform plan
docker run \
  --rm \
  --tty \
  --interactive \
  --platform 'linux/amd64' \
  --name "${CB_IMAGE}" \
  --hostname "${CB_IMAGE}" \
  --mount "type=bind,source=${CB_CONFIG_HOST_DIR},target=${CB_CONFIG_CONTAINER_DIR}" \
  "${CB_REGISTRY}/${CB_IMAGE}:2.31.0" \
  opscontrol-infra-plan "${CB_PROVIDER}" "${CB_CONFIG_CONTAINER_DIR}"

# Run terraform apply
docker run \
  --rm \
  --tty \
  --interactive \
  --platform 'linux/amd64' \
  --name "${CB_IMAGE}" \
  --hostname "${CB_IMAGE}" \
  --mount "type=bind,source=${CB_CONFIG_HOST_DIR},target=${CB_CONFIG_CONTAINER_DIR}" \
  "${CB_REGISTRY}/${CB_IMAGE}:2.31.0" \
  opscontrol-infra-deploy "${CB_PROVIDER}" "${CB_CONFIG_CONTAINER_DIR}"

• There are several changes to Opscontrol tfvars. See below for details.
• Environment deployment process has not changed.
• Environment configuration has been changed and simplified. Instead of several config json files in different directories we placed all configuration in a single config.json file. See below for details.

Required steps

• Removed variables from Opscontrol terraform.tfvars:

    cb_deployments_package_bucket               replaced by docker
    cb_deployments_package_target_cloud         replaced by docker
    cb_deployments_package_version              replaced by docker
    docker_image_repo                           replaced by docker
    use_external_cloud_provider                 always true
    ansible_strategy                            always linear
    translated_snat_ip                          not used
    python_alias                                not used
    vsphere_k8s_username                        not used
    vsphere_k8s_password                        not used
    elk_backup_bucket_name                      renamed to backups_bucket_name
    prometheus_backup_bucket_name               renamed to backups_bucket_name
    elk_deployment_enabled                      renamed to elasticsearch_deployment_enabled
    sensitive_data_vcenter_ca_filename          renamed to sensitive_data_vsphere_ca_filename

• New variables added to Opscontrol terraform.tfvars:

    cloudboostr_image_name
    cloudboostr_image_tag
    docker_registry_url
    docker_registry_username
    docker_registry_password
    docker_registry_proxy_cache_project
    docker_registry_cloudboostr_project
    backups_bucket_name
    sensitive_data_vsphere_ca_filename
    sensitive_data_traefik_crt_chain_filename
    elasticsearch_deployment_enabled
    infrastructure_state_bucket_name (has to match backend.tf configuration)

• New file required for Opscontrol deployment: backend.tf with Terraform backend configuration, e.g.:

terraform {
  backend "s3" {
    bucket     = "terraform-state-bucket"
    key        = "terraform/terraform.tfstate"
    region     = "eu-central-1"
    access_key = "..."
    secret_key = "..."
    encrypt    = true
  }
}

• New Environment configuration file config.json introduced with all pipelines and variables definition in one place:

{
    "pipelines": [
        {
            "name": "deploy_infra",
            "file": "cb-env/ci/pipelines/deploy-pipeline.yml",
            "vars": []
        },
        {
            "name": "destroy_infra",
            "file": "cb-env/ci/pipelines/destroy-pipeline.yml",
            "vars": []
        },
        {
            "name": "deploy_k8s",
            "file": "cb-k8s-deployment/ci/pipelines/deploy-pipeline.yml",
            "vars": []
        },
        {
            "name": "destroy_k8s",
            "file": "cb-k8s-deployment/ci/pipelines/destroy-pipeline.yml",
            "vars": []
        },
        (...)
        {
            "name": "pre_upgrade_checks_k8s",
            "file": "cb-k8s-deployment/ci/pipelines/pre-upgrade-check-pipeline.yml",
            "vars": []
        }
    ],
    "vars": [
        {
            "name": "vsphere_host",
            "vault_var": "/opscontrol/vsphere_host"
        },
        {
            "name": "vsphere_username",
            "vault_var": "/opscontrol/vsphere_username"
        },
        {
            "name": "vsphere_password",
            "vault_var": "/opscontrol/vsphere_password"
        },
        (...)
        {
            "name": "filebeat_cb_annotations",
            "value": "cloudboostr-filebeat"
        }
    ]
}

---

Release 2.9.0

2024/10/15

Release notes

• Upgrade Kubernetes 1.30.4
• Add annotations for Cloudboostr deployed services
• Fixes in the security upgrade pipelines
• [vsphere] Update nsxt and vsphere providers
• [vsphere] Add active monitor path for Kuberentes API Server

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2024/04/17/kubernetes-v1-30-release/
• Please make sure you have a backup of your data before running an update.
• In this release we introduced annotations to services deployed by cloudboostr. There is single annotation added to service with value that can be specified in k8s-deployment.json config file. See details below:

cloudboostr.com/managed-by: "{{ cb_annotation_value }}"

• There is active monitor path added for Kuberentes API Server LoadBalancer next to the Ingress LoadBalancer. This introduced some changes that may affect extensions for terraform, current variables are list of strings with default values:

variable "k8s_lb_ingress_active_monitor_paths" {
  description = "Path of the active monitor K8S LB Pool (Used only by the Policy API)"
  type        = list(string)
  default     = ["/infra/lb-monitor-profiles/default-icmp-lb-monitor"]
}

variable "k8s_lb_api_active_monitor_paths" {
  description = "Path of the active monitor K8S LB Pool (Used only by the Policy API)"
  type        = list(string)
  default     = ["/infra/lb-monitor-profiles/default-icmp-lb-monitor"]
}

Required steps

• New variables in k8s-deployment.json config file:

{"name": "velero_cb_annotation", "value": "cloudboostr"},
{"name": "traefik_ingress_cb_annotation", "value": "cloudboostr"},
{"name": "nginx_ingress_cb_annotation", "value": "cloudboostr"},
{"name": "harbor_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_operator_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_alertmanager_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_thanos_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_grafana_cb_annotation", "value": "cloudboostr"},
{"name": "filebeat_cb_annotations", "value": "cloudboostr"}

---

Release 2.8.0

2024/08/19

Release notes

• Upgrade Kubernetes 1.29.5
• Upgrade Helm charts non-major versions
• Remove deprecated dnssec parameter from Bind9 configuration
• Add custom extenstions for concourse deployment
• Add flag to control automatic TLS secret creation for ingress
• Add smoke tests prefix to configuration

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2023/12/13/kubernetes-v1-29-release/
• Please make sure you have a backup of your data before running an update.
• There is a custom version of calico pinned to v3.27.4 in order to fix below issue: https://github.com/projectcalico/calico/issues/8856

Required steps

• New variables in k8s-deployment.json config file:

{"name": "smoke_test_prefix", "value": "smoke-test-prefix"},
{"name": "ingress_disable_create_tls_secret", "value": "false"},

---

Release 2.7.0

2024/04/02

Release notes

• Upgrade Kubernetes 1.28.6
• Upgrade Helm charts
• Upgrade Velero
• Security improvements (SSH and TLS hardening)
• Add whitelist firewall rules for K8s api

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2023/08/15/kubernetes-v1-28-release/
• Please make sure you have a backup of your data before running an update.

Required steps

• New variable in Opscontrol terraform.tfvars:

variable "control_plane_api_whitelist_in" {
  description = "List of CIDR's from which acccess to k8s_api is allowed"
  type        = list(string)
}

• New variable in k8s-deployment.json config file:

{"name": "k8s_api_whitelist_in", "value": "[0.0.0.0/0]"},

---

Release 2.6.0

2024/01/10

Release notes

• Upgrade Kubernetes 1.27.9
• Upgrade internal Python (3.9) and Ansible (7.6.0) versions
• Add support for trusted CA certificates
• Add support for encrypted private keys
• Disable verbose logging in scripts and Concourse tasks
• Use Velero schedule for periodic backups
• Change default NSX-T provider configuration to avoid hitting API rate limits
• Automate HDD resizing during cluster setup

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2023/04/11/kubernetes-v1-27-release/
• Please make sure you have a backup of your data before running an update.

Required steps

• New variable in common.json config file:

{"name": "nsx_max_retries", "opscontrol_var": "nsx_max_retries"},
{"name": "nsx_retry_min_delay", "opscontrol_var": "nsx_retry_min_delay"},
{"name": "nsx_retry_max_delay", "opscontrol_var": "nsx_retry_max_delay"},

• New variables in k8s-deployment.json config file:

{
    "name": "deploy_k8s",
    "file": "ci/pipelines/deploy-pipeline/deploy-pipeline-vsphere.yml",
-   "vars": []
+   "vars": [
+       {"name": "velero_backup_schedule", "value": "0 1 * * *"},
+       {"name": "velero_backup_ttl", "value": "72h"}
+   ]
},

(...)

{"name": "traefik_ingress_redirect_web_to_websecure", "value": "false"},

• Old variables in k8s-deployment.json config file, that can be removed:

{"name": "kubectl_version", "value": "..."},
{"name": "k8s_version", "value": "..."},

---

Release 2.5.0

2023/06/30

Release notes

• Upgrade Kubernetes 1.26.5
• Upgrade Helm charts for:
• Traefik (23.0.1) and Nginx (4.7.0) ingress controllers
• OpenSearch (2.13.1) and OpenSearch Dashboards (2.11.1)
• Concourse (17.1.1)
• Keycloak (18.4.3)
• Prometheus (46.6.0) and Thanos (12.6.2)
• Fix python issues during Kubernetes installation steps
• Fix Concourse performance issues with worker pods
• Add Velero backup retention period configuration
• Add support for hot add CPU and Memory in vSphere

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2022/12/09/kubernetes-v1-26-release//
• Please make sure you have a backup of your data before running an update.

---

Release 2.4.0

2023/02/01

Release notes

• Kubernetes 1.25.6
• Upgrade internal tools
  • Concourse 7.8.3
  • Keycloak 16.1.1
  • Vault 1.11.6
  • Harbor 2.7.0
  • Prometheus 2.41.0
  • Terraform 1.3.6
  • Traefik Ingress Controller 2.9.6
  • Nginx Ingress Controller 1.5.1
• vSphere: Remove NSX-T Manager API support
• vSphere: Add Gateway Firewall
• Fixes in upgrade pipeline
• Replaced deprecated terraform providers

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2022/08/23/kubernetes-v1-25-release/
• Please make sure you have a backup of your data before running an update.

---

Release 2.3.0

2022/10/07

Release notes

• Kubernetes 1.24
• Support for custom CA certificate
• Support for separate storge configuration for environments
• Support for extensions in git repository
• Replaced deprecated terraform providers

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2022/05/03/kubernetes-1-24-release-announcement/
• Please make sure you have a backup of your data before running an update.
• In order to add custom CA certificate you should place it in sensitive-data bucket and set proper Opscontrol terraform variables sensitive_data_offline_root_ca_key_filename, sensitive_data_offline_root_ca_crt_filename and/or sensitive_data_offline_root_ca_chain_filename. If there is a password set for the key you can pass it via sensitive_data_offline_root_ca_key_password.
• In order to use extensions in git you need to place them in the current Cloudboostr config repository under extensions directory and then set extensions_bucket_name to empty string. Then e.g. extensions_terraform_directory will be used to locate proper directory under extensions/ dir.

---

Release 2.2.0

2022/08/07

Release notes

• Kubernetes 1.23.0
• Update Concourse to 7.8.0
• Update Prometheus/Thanos to 2.38.0/0.26.0
• Update Velero to 1.8.1
• Improve security in metrics components and audit logging
• Bugfixes and improvements

Required steps

• Variables name changed in Opscontrol terraform.tfvars:

telemetry_subnet_cidr             -> control_plane_subnet_cidr
telemetry_router_ip               -> control_plane_router_ip
telemetry_dhcp_server_ip          -> control_plane_dhcp_server_ip
telemetry_dhcp_server_range_start -> control_plane_dhcp_server_range_start
telemetry_dhcp_server_range_end   -> control_plane_dhcp_server_range_end
efk_deployment_enabled            -> elk_deployment_enabled

• Variables that can be removed from Opscontrol terraform.tfvars:

dmz_reserved_ips
dmz_static_ips
mgmt_reserved_ips
telemetry_reserved_ips
telemetry_static_ips

• New variable in common.json config file:

{"name": "elk_deployment_enabled", "opscontrol_var": "elk_deployment_enabled"}

• New variables in k8s-deployment.json config file:

{"name": "delete_k8s_pv_on_destroy", "value": "false"}
{"name": "docker_image_repo", "opscontrol_var": "docker_image_repo"}
{"name": "k8s_packages_ansible_playbook_additional_arguments", "value": ""}
{"name": "filebeat_release_state", "value": "present"}
{"name": "nginx_ingress_release_state", "value": "absent"}
{"name": "traefik_ingress_release_state", "value": "present"}
{"name": "prometheus_release_state", "value": "present"}
{"name": "thanos_release_state", "value": "present"}
{"name": "velero_release_state", "value": "present"}

• Old variables in k8s-deployment.json config file, that can be removed:

{"name": "ingress_additional_files_bucket", "value": "..."}
{"name": "ingress_additional_files", "value": "..."}
{"name": "ingress_type", "value": "..."}

• Move ingress certificate and key from your bucket to Vault:

${ingress_additional_files_bucket}/${ENV_NAME}.k8s.key -> {VAULT_KV_PATH_EXTENSIONS}/${ENV_NAME}/k8s_key
${ingress_additional_files_bucket}/${ENV_NAME}.k8s.crt -> {VAULT_KV_PATH_EXTENSIONS}/${ENV_NAME}/k8s_crt

• Move ingress extensions files to extensions_directory:

${ingress_additional_files_bucket}/nginx-override.yaml -> {EXTENSIONS_BUCKET}/${EXTENSIONS_DIR}/packages/nginx-ingress/values.yml
${ingress_additional_files_bucket}/traefik-override.yaml -> {EXTENSIONS_BUCKET}/${EXTENSIONS_DIR}/packages/traefik-ingress/values.yml

Important notes

• There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2021/12/07/kubernetes-1-23-release-announcement/
• This version requires an update of some tools for OpsControl installation:
  • yq – 2.13.0 (version from pip3 is required: pip3 install yq==2.13.0)
  • jq – 1.6
  • terraform – 1.1.5
• Please make sure you have a backup of your data before running an update.

---

Release 2.1.0

2022/04/11

Release notes

• Upgrade Kubernetes to version 1.22.6
• Use containerd as default container runtime in Kubernetes
• Upgrade Terraform to version 1.1.5
• Upgrade Velero to version 1.8.0 with multibackend and snapshots support
• Remove BOSH fully from Cloudboostr
• Fix OpenSearch service to aggregate logs
• Fix https redirections for OpsControl services
• Improve VMs OS upgrades

Required steps

• New efk_deployment_enabled parameter added to Opscontrol terraform.tfvars file. It allows to disable the ELK (OpenSearch) installation in Opscontrol. Default value is true.
• New velero_snapshot_volumes parameter in k8s-deployment.json which can be used to enable PersistentVolume snapshot feautre in Velero backup service.
• Additional flags that can be set via extensions to control update/migrate process. By default all flags are not set so upgrade and migration run. If both flags are set to “true” nothing will happen.
  • skip_upgrade – to skip update and run just a migration (true/false)
  • disable_containerd_migration – to skip migration and run just an upgrade (true/false)

Important notes

• In this version default container runtime in Kubernetes is changed to containerd. There is still an option to use dockershim and it requires usage of extensions. During upgrade process all pods should be moved from docker to containerd automatically but there might be some unexpected issues.
• There is a new Kubernetes version in this upgrade. There are some APIs removed, see more details here before running upgrade: https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/#api-changes
• After upgrade you may have to manually remove BOSH Director from OpsControl if you have not removed that yet after 2.0.0 upgrade.
• Please make sure you have a backup of your data before running an update.

---

Release 2.0.0

2022/01/24

Release notes

• Replace Credhub with Vault v1.9.2
• Replace UAA with Keycloak v15.0.2
• Replace Opendistro with Opensearch
• Move Prometheus/Grafana to Kubernetes
• Removal BOSH from Environment deployment
• Add multiple users support on jumpbox
• Add concourse extensions and log retention settings
• Add variable with CB version
• vSphere: Add LB active monitor in NSXT Policy API
• AWS: update LoadBalancers in Terraform

Required steps

• In case you have configured Kubernetes with OIDC authentication with UAA it will still work but it is already deprecated and you have to migrate to Keycloak. UAA will be completly removed in Cloudboostr 2.1.0 release
• New ansible_strategy parameter added to Opscontrol terraform.tfvars file. You can check possible values here: https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html
• Removed concourse_ui_certificate_name and grafana_certificate_name from terraform.tfvars
• New users parameter in config.json that can be used to add custom users to all jumpboxes (Opscontrol and all Environments). This is an array of object with two parameters:
  • name – string with username
  • ssh_key – base64 encoded public key. Note: public key should include in a comment valid user email address

# Example of config.json file with users
{
    "envs": [
        {
            "name": "test",
            "backend_type": "aws",
            "config_repo_url": "...",
            "config_repo_branch": "..."
        }
    ],
    "users": [
        {
            "name": "test",
            "ssh_key": "<base64_encoded_public_key>"
        }
    ]
}

• New pipeline available to update jumpbox users in env.json:

(...)
    {
    "name": "update_users",
    "file": "ci/pipelines/update-users.yml",
    "vars": [
        {"name": "timer_interval", "value": "24h"}
    ]
    },
(...)

• Removed bosh_ variables from env.json
• BOSH installation was removed from Environments. In order to fully remove BOSH director you have to delete it manually from Environment

bosh delete-env -n \
  --state /etc/bosh-state/state.json \
  --vars-store /etc/bosh-state/creds.yml \
  ~/configure_jumpbox_bosh_workspace/manifest.yml

• New ansible_strategy parameter in k8s-deployment.json which can be used to modify strategy for kubespray deployment. See above example from terraform.tfvars to see possible values. Note: you can use "opscontrol_var": "ansible_strategy" to reuse value from Opscontrol.

Important notes

• This upgrade is a major upgrade that replaces core components from Cloudboostr: Credhub and UAA with Vault and Keycloak. Please make sure to update to new toolset your custom scripts and extensions before upgrade.
• After changing kube-apiserver (if there is no Kubernetes version change) you have to manually reinit kubeadm from one of the master nodes:

sudo kubeadm init --config /etc/kubernetes/kubeadm-config.yaml phasecontrol-plane all

• Please make sure you have a backup of your data before running an update.
• This version removes completly BOSH from Environment, which affects CloudFoundry deployment. It should change in next Cloudboostr 2.1.0 release.
• If you are going to change from in-tree to external cloud provider in OpsControl you have to manaully migrate the volumes to new CSI or remove them completly and recreate.
• This upgrade does not include new Kubernetes version. It is still v1.21.5 (the same as in Cloudboostr v1.7.0).