Skip to content

Release notes

Release 2.9.0

2024/10/15

Release notes

  • Upgrade Kubernetes 1.30.4
  • Add annotations for Cloudboostr deployed services
  • Fixes in the security upgrade pipelines
  • [vsphere] Update nsxt and vsphere providers
  • [vsphere] Add active monitor path for Kuberentes API Server

Important notes

  • There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2024/04/17/kubernetes-v1-30-release/
  • Please make sure you have a backup of your data before running an update.
  • In this release we introduced annotations to services deployed by cloudboostr. There is single annotation added to service with value that can be specified in k8s-deployment.json config file. See details below:
cloudboostr.com/managed-by: "{{ cb_annotation_value }}"
  • There is active monitor path added for Kuberentes API Server LoadBalancer next to the Ingress LoadBalancer. This introduced some changes that may affect extensions for terraform, current variables are list of strings with default values:
variable "k8s_lb_ingress_active_monitor_paths" {
  description = "Path of the active monitor K8S LB Pool (Used only by the Policy API)"
  type        = list(string)
  default     = ["/infra/lb-monitor-profiles/default-icmp-lb-monitor"]
}

variable "k8s_lb_api_active_monitor_paths" {
  description = "Path of the active monitor K8S LB Pool (Used only by the Policy API)"
  type        = list(string)
  default     = ["/infra/lb-monitor-profiles/default-icmp-lb-monitor"]
}

Required steps

  • New variables in k8s-deployment.json config file:
{"name": "velero_cb_annotation", "value": "cloudboostr"},
{"name": "traefik_ingress_cb_annotation", "value": "cloudboostr"},
{"name": "nginx_ingress_cb_annotation", "value": "cloudboostr"},
{"name": "harbor_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_operator_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_alertmanager_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_thanos_cb_annotation", "value": "cloudboostr"},
{"name": "prometheus_grafana_cb_annotation", "value": "cloudboostr"},
{"name": "filebeat_cb_annotations", "value": "cloudboostr"}

Release 2.8.0

2024/08/19

Release notes

  • Upgrade Kubernetes 1.29.5
  • Upgrade Helm charts non-major versions
  • Remove deprecated dnssec parameter from Bind9 configuration
  • Add custom extenstions for concourse deployment
  • Add flag to control automatic TLS secret creation for ingress
  • Add smoke tests prefix to configuration

Important notes

Required steps

  • New variables in k8s-deployment.json config file:
{"name": "smoke_test_prefix", "value": "smoke-test-prefix"},
{"name": "ingress_disable_create_tls_secret", "value": "false"},

Release 2.7.0

2024/04/02

Release notes

  • Upgrade Kubernetes 1.28.6
  • Upgrade Helm charts
  • Upgrade Velero
  • Security improvements (SSH and TLS hardening)
  • Add whitelist firewall rules for K8s api

Important notes

Required steps

  • New variable in Opscontrol terraform.tfvars:
variable "control_plane_api_whitelist_in" {
  description = "List of CIDR's from which acccess to k8s_api is allowed"
  type        = list(string)
}
  • New variable in k8s-deployment.json config file:
{"name": "k8s_api_whitelist_in", "value": "[0.0.0.0/0]"},

Release 2.6.0

2024/01/10

Release notes

  • Upgrade Kubernetes 1.27.9
  • Upgrade internal Python (3.9) and Ansible (7.6.0) versions
  • Add support for trusted CA certificates
  • Add support for encrypted private keys
  • Disable verbose logging in scripts and Concourse tasks
  • Use Velero schedule for periodic backups
  • Change default NSX-T provider configuration to avoid hitting API rate limits
  • Automate HDD resizing during cluster setup

Important notes

Required steps

  • New variable in common.json config file:
{"name": "nsx_max_retries", "opscontrol_var": "nsx_max_retries"},
{"name": "nsx_retry_min_delay", "opscontrol_var": "nsx_retry_min_delay"},
{"name": "nsx_retry_max_delay", "opscontrol_var": "nsx_retry_max_delay"},
  • New variables in k8s-deployment.json config file:
{
    "name": "deploy_k8s",
    "file": "ci/pipelines/deploy-pipeline/deploy-pipeline-vsphere.yml",
-   "vars": []
+   "vars": [
+       {"name": "velero_backup_schedule", "value": "0 1 * * *"},
+       {"name": "velero_backup_ttl", "value": "72h"}
+   ]
},

(...)

{"name": "traefik_ingress_redirect_web_to_websecure", "value": "false"},
  • Old variables in k8s-deployment.json config file, that can be removed:
{"name": "kubectl_version", "value": "..."},
{"name": "k8s_version", "value": "..."},

Release 2.5.0

2023/06/30

Release notes

  • Upgrade Kubernetes 1.26.5
  • Upgrade Helm charts for:
  • Traefik (23.0.1) and Nginx (4.7.0) ingress controllers
  • OpenSearch (2.13.1) and OpenSearch Dashboards (2.11.1)
  • Concourse (17.1.1)
  • Keycloak (18.4.3)
  • Prometheus (46.6.0) and Thanos (12.6.2)
  • Fix python issues during Kubernetes installation steps
  • Fix Concourse performance issues with worker pods
  • Add Velero backup retention period configuration
  • Add support for hot add CPU and Memory in vSphere

Important notes

Release 2.4.0

2023/02/01

Release notes

  • Kubernetes 1.25.6
  • Upgrade internal tools
    • Concourse 7.8.3
    • Keycloak 16.1.1
    • Vault 1.11.6
    • Harbor 2.7.0
    • Prometheus 2.41.0
    • Terraform 1.3.6
    • Traefik Ingress Controller 2.9.6
    • Nginx Ingress Controller 1.5.1
  • vSphere: Remove NSX-T Manager API support
  • vSphere: Add Gateway Firewall
  • Fixes in upgrade pipeline
  • Replaced deprecated terraform providers

Important notes

Release 2.3.0

2022/10/07

Release notes

  • Kubernetes 1.24
  • Support for custom CA certificate
  • Support for separate storge configuration for environments
  • Support for extensions in git repository
  • Replaced deprecated terraform providers

Important notes

  • There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2022/05/03/kubernetes-1-24-release-announcement/
  • Please make sure you have a backup of your data before running an update.
  • In order to add custom CA certificate you should place it in sensitive-data bucket and set proper Opscontrol terraform variables sensitive_data_offline_root_ca_key_filename, sensitive_data_offline_root_ca_crt_filename and/or sensitive_data_offline_root_ca_chain_filename. If there is a password set for the key you can pass it via sensitive_data_offline_root_ca_key_password.
  • In order to use extensions in git you need to place them in the current Cloudboostr config repository under extensions directory and then set extensions_bucket_name to empty string. Then e.g. extensions_terraform_directory will be used to locate proper directory under extensions/ dir.

Release 2.2.0

2022/08/07

Release notes

  • Kubernetes 1.23.0
  • Update Concourse to 7.8.0
  • Update Prometheus/Thanos to 2.38.0/0.26.0
  • Update Velero to 1.8.1
  • Improve security in metrics components and audit logging
  • Bugfixes and improvements

Required steps

  • Variables name changed in Opscontrol terraform.tfvars:
telemetry_subnet_cidr             -> control_plane_subnet_cidr
telemetry_router_ip               -> control_plane_router_ip
telemetry_dhcp_server_ip          -> control_plane_dhcp_server_ip
telemetry_dhcp_server_range_start -> control_plane_dhcp_server_range_start
telemetry_dhcp_server_range_end   -> control_plane_dhcp_server_range_end
efk_deployment_enabled            -> elk_deployment_enabled
  • Variables that can be removed from Opscontrol terraform.tfvars:
dmz_reserved_ips
dmz_static_ips
mgmt_reserved_ips
telemetry_reserved_ips
telemetry_static_ips
  • New variable in common.json config file:
{"name": "elk_deployment_enabled", "opscontrol_var": "elk_deployment_enabled"}
  • New variables in k8s-deployment.json config file:
{"name": "delete_k8s_pv_on_destroy", "value": "false"}
{"name": "docker_image_repo", "opscontrol_var": "docker_image_repo"}
{"name": "k8s_packages_ansible_playbook_additional_arguments", "value": ""}
{"name": "filebeat_release_state", "value": "present"}
{"name": "nginx_ingress_release_state", "value": "absent"}
{"name": "traefik_ingress_release_state", "value": "present"}
{"name": "prometheus_release_state", "value": "present"}
{"name": "thanos_release_state", "value": "present"}
{"name": "velero_release_state", "value": "present"}
  • Old variables in k8s-deployment.json config file, that can be removed:
{"name": "ingress_additional_files_bucket", "value": "..."}
{"name": "ingress_additional_files", "value": "..."}
{"name": "ingress_type", "value": "..."}
  • Move ingress certificate and key from your bucket to Vault:
${ingress_additional_files_bucket}/${ENV_NAME}.k8s.key -> {VAULT_KV_PATH_EXTENSIONS}/${ENV_NAME}/k8s_key
${ingress_additional_files_bucket}/${ENV_NAME}.k8s.crt -> {VAULT_KV_PATH_EXTENSIONS}/${ENV_NAME}/k8s_crt
  • Move ingress extensions files to extensions_directory:
${ingress_additional_files_bucket}/nginx-override.yaml -> {EXTENSIONS_BUCKET}/${EXTENSIONS_DIR}/packages/nginx-ingress/values.yml
${ingress_additional_files_bucket}/traefik-override.yaml -> {EXTENSIONS_BUCKET}/${EXTENSIONS_DIR}/packages/traefik-ingress/values.yml

Important notes

  • There is a new Kubernetes version in this upgrade. There are some APIs changes, see more details here before running upgrade: https://kubernetes.io/blog/2021/12/07/kubernetes-1-23-release-announcement/
  • This version requires an update of some tools for OpsControl installation:
    • yq – 2.13.0 (version from pip3 is required: pip3 install yq==2.13.0)
    • jq – 1.6
    • terraform – 1.1.5
  • Please make sure you have a backup of your data before running an update.

Release 2.1.0

2022/04/11

Release notes

  • Upgrade Kubernetes to version 1.22.6
  • Use containerd as default container runtime in Kubernetes
  • Upgrade Terraform to version 1.1.5
  • Upgrade Velero to version 1.8.0 with multibackend and snapshots support
  • Remove BOSH fully from Cloudboostr
  • Fix OpenSearch service to aggregate logs
  • Fix https redirections for OpsControl services
  • Improve VMs OS upgrades

Required steps

  • New efk_deployment_enabled parameter added to Opscontrol terraform.tfvars file. It allows to disable the ELK (OpenSearch) installation in Opscontrol. Default value is true.
  • New velero_snapshot_volumes parameter in k8s-deployment.json which can be used to enable PersistentVolume snapshot feautre in Velero backup service.
  • Additional flags that can be set via extensions to control update/migrate process. By default all flags are not set so upgrade and migration run. If both flags are set to “true” nothing will happen.
    • skip_upgrade – to skip update and run just a migration (true/false)
    • disable_containerd_migration – to skip migration and run just an upgrade (true/false)

Important notes

  • In this version default container runtime in Kubernetes is changed to containerd. There is still an option to use dockershim and it requires usage of extensions. During upgrade process all pods should be moved from docker to containerd automatically but there might be some unexpected issues.
  • There is a new Kubernetes version in this upgrade. There are some APIs removed, see more details here before running upgrade: https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/#api-changes
  • After upgrade you may have to manually remove BOSH Director from OpsControl if you have not removed that yet after 2.0.0 upgrade.
  • Please make sure you have a backup of your data before running an update.

Release 2.0.0

2022/01/24

Release notes

  • Replace Credhub with Vault v1.9.2
  • Replace UAA with Keycloak v15.0.2
  • Replace Opendistro with Opensearch
  • Move Prometheus/Grafana to Kubernetes
  • Removal BOSH from Environment deployment
  • Add multiple users support on jumpbox
  • Add concourse extensions and log retention settings
  • Add variable with CB version
  • vSphere: Add LB active monitor in NSXT Policy API
  • AWS: update LoadBalancers in Terraform

Required steps

  • In case you have configured Kubernetes with OIDC authentication with UAA it will still work but it is already deprecated and you have to migrate to Keycloak. UAA will be completly removed in Cloudboostr 2.1.0 release
  • New ansible_strategy parameter added to Opscontrol terraform.tfvars file. You can check possible values here: https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html
  • Removed concourse_ui_certificate_name and grafana_certificate_name from terraform.tfvars
  • New users parameter in config.json that can be used to add custom users to all jumpboxes (Opscontrol and all Environments). This is an array of object with two parameters:
    • name – string with username
    • ssh_key – base64 encoded public key. Note: public key should include in a comment valid user email address
# Example of config.json file with users
{
    "envs": [
        {
            "name": "test",
            "backend_type": "aws",
            "config_repo_url": "...",
            "config_repo_branch": "..."
        }
    ],
    "users": [
        {
            "name": "test",
            "ssh_key": "<base64_encoded_public_key>"
        }
    ]
}
  • New pipeline available to update jumpbox users in env.json:
(...)
    {
    "name": "update_users",
    "file": "ci/pipelines/update-users.yml",
    "vars": [
        {"name": "timer_interval", "value": "24h"}
    ]
    },
(...)
  • Removed bosh_ variables from env.json
  • BOSH installation was removed from Environments. In order to fully remove BOSH director you have to delete it manually from Environment
bosh delete-env -n \
  --state /etc/bosh-state/state.json \
  --vars-store /etc/bosh-state/creds.yml \
  ~/configure_jumpbox_bosh_workspace/manifest.yml
  • New ansible_strategy parameter in k8s-deployment.json which can be used to modify strategy for kubespray deployment. See above example from terraform.tfvars to see possible values. Note: you can use "opscontrol_var": "ansible_strategy" to reuse value from Opscontrol.

Important notes

  • This upgrade is a major upgrade that replaces core components from Cloudboostr: Credhub and UAA with Vault and Keycloak. Please make sure to update to new toolset your custom scripts and extensions before upgrade.
  • After changing kube-apiserver (if there is no Kubernetes version change) you have to manually reinit kubeadm from one of the master nodes:
sudo kubeadm init --config /etc/kubernetes/kubeadm-config.yaml phasecontrol-plane all
  • Please make sure you have a backup of your data before running an update.
  • This version removes completly BOSH from Environment, which affects CloudFoundry deployment. It should change in next Cloudboostr 2.1.0 release.
  • If you are going to change from in-tree to external cloud provider in OpsControl you have to manaully migrate the volumes to new CSI or remove them completly and recreate.
  • This upgrade does not include new Kubernetes version. It is still v1.21.5 (the same as in Cloudboostr v1.7.0).