Skip to content

Opscontrol Installation steps

Prerequisites

To install Cloudboostr, you must meet the following requirements depends on the desired platform:

  • vCenter details:

    • vCenter hostname or IP
    • username and password
    • datacenter and cluster name
    • resource pool name
    • datastore name
    • VM template name

  • NSX-T details:

    • NSX-T hostname or IP
    • username and password
    • overlay transport zone name
    • tier0 router name
    • edge cluster name
  • AWS access details:
    • account ID
    • access key and secret key
    • region name
    • availability zones names
  • Azure access details:
    • subscription ID
    • client ID and secret
    • tenant ID
    • network location (region) and main resource group name
  • Openstack access details:
    • auth url
    • username and password
    • project (tenant) name and id
    • domain name
    • regsion
  • DNS entries:
    • OpsControl base domain configured with OpsControl DNS IP (e.g. "opscontrol.cloudboostr.int")
    • Environment base domain configured with Env DNS IP (e.g. "env1.cloudboostr.int" )

Note

Make sure to create NS entries with accompanying A records

  • Certificates:

    • OpsControl with SANs:
      • *.control-plane.${ops_base_domain}
      • concourse.${ops_base_domain}
      • grafana.${ops_base_domain}
      • kibana.${ops_base_domain}
    • Environment with SANs:
      • *.k8s.${env_base_domain}

  • Environment configuration repository

  • Storage S3 buckets

    • OpsControl terraform state bucket
    • Sensitive data bucket
    • Backups bucket

Prepare local environment

Docker

Install Docker engine on your platform following: https://docs.docker.com/engine/install/.

Pull Cloudboostr image

First pull the cloudboostr image to your machine:

export CB_IMAGE="cloudboostr"   # Unique name of the Docker image
export CB_REGISTRY=""           # Docker registry from which Docker image should be pulled.
export CB_VERSION="2.31.0"      # Cloudboostr version to be installed

docker pull --platform 'linux/amd64' "${CB_REGISTRY}/${CB_IMAGE}:${CB_VERSION}"

Then in order to verify you can execute followiing command to see the version.

docker run "${CB_REGISTRY}/${CB_IMAGE}:${CB_VERSION}" version

Prepare SSH keys

First step is to prepare SSH keys that will be used for jumpbox, k8s cluster and dns connection. Also GIT key is required for terraform and scripts to download the required repositories.

If SSH keys were prepared beforehand or are being created by external tool like Vault skip this and go to the next step.

# Create SSH keys

export EMAIL=[YOUR_EMAIL_HERE]
mkdir keys
ssh-keygen -t rsa -b 4096 -C "$EMAIL" -f ./keys/jumpbox_devops -N '' -m pem
ssh-keygen -t rsa -b 4096 -C "$EMAIL" -f ./keys/k8s_devops -N '' -m pem
ssh-keygen -t rsa -b 4096 -C "$EMAIL" -f ./keys/dns_devops -N '' -m pem
ssh-keygen -t rsa -b 4096 -C "$EMAIL" -f ./keys/git_private_key -N '' -m pem

Prepare resources

  1. Create Resource pool if required
  2. Create VMs template if one does not already exist. You should use OVA cloud-image as a base, eg: https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.ova

Move to the next step

Move to the next step

OpenStack CLIs configuraion

OpenStack tools are configured using environment variables. Depending on the installation and configuration not all the fields are required. Fields listed in the snippet below are most commonly required.

Clean configuration: 

export OS_USERNAME=
export OS_PASSWORD=
export OS_AUTH_URL=
export OS_USER_DOMAIN_NAME=
export OS_PROJECT_DOMAIN_NAME=
export OS_REGION_NAME=
export OS_PROJECT_NAME=
export OS_TENANT_NAME=
export OS_AUTH_VERSION=3
export OS_IDENTITY_API_VERSION=3

OpenStack environment variables

Comprehensive list of OpenStack client environment variables is available in the official documentation: https://docs.openstack.org/python-openstackclient/pike/cli/man/openstack.html.

Create a project for the deployment

If you don't have yet a project for the deployment you should start with configuring it. The minimal quotas are listed on this page.

Create domain:

openstack domain create --description "OpsControl" opscontrol

Create project (tenant):

openstack project create --domain default --description “Cloudboostr” cloudboostr

Create admin user:

openstack user create --domain default --password-prompt admin

Note

If you are not sure if project already exist consult the infrastructure team. Projects configuration require admin privileges.

Upload keys

Then keys should be uploaded to the sensitive data bucket (e.g. sensitive-data.cb). Filenames and container name can be changed in configuration if required.

aws s3 cp ./keys/jumpbox_devops      s3://sensitive-data.cb/jumpbox_devops
aws s3 cp ./keys/jumpbox_devops.pub  s3://sensitive-data.cb/jumpbox_devops.pub
aws s3 cp ./keys/dns_devops          s3://sensitive-data.cb/dns_devops
aws s3 cp ./keys/dns_devops.pub      s3://sensitive-data.cb/dns_devops.pub
aws s3 cp ./keys/k8s_devops          s3://sensitive-data.cb/k8s_devops
aws s3 cp ./keys/k8s_devops.pub      s3://sensitive-data.cb/k8s_devops.pub
aws s3 cp ./keys/git_private_key     s3://sensitive-data.cb/git_private_key

Upload certificates

Then certificates (for OpsControl, RootCA certificate if self signed is used, vSphere and NSX-T CA) should be uploaded to the sensitive-data bucket as well. Filenames and container name can be changed in configuration if required.

aws s3 cp ingress_crt     s3://sensitive-data.cb/ingress_crt
aws s3 cp ingress_key     s3://sensitive-data.cb/ingress_key
aws s3 cp root_ca         s3://sensitive-data.cb/root_ca
aws s3 cp vsphere_ca_crt  s3://sensitive-data.cb/vsphere_ca_crt
aws s3 cp nsxt_ca_crt     s3://sensitive-data.cb/nsxt_ca_crt

Prepare OpsControl configuration

Create 3 files required by OpsControl:

  • terraform.tfvars with basic OpsControl terraform configuration
  • backend.tf with OpsControl terraform state backend configuration
  • storage_config.yml with default storage access credentials

Terraform configuration

At minimum update the required fields using configuration required and optional parameters page.

Example

vsphere_host                                = "<REDACTED>"
vsphere_username                            = "<REDACTED>"
vsphere_password                            = "<REDACTED>"
vsphere_allow_unverified_ssl                = true
vsphere_datacenter                          = "Datacenter"
vsphere_datastore                           = "Datastore"
vsphere_cluster                             = "Cluster"
vsphere_resource_pool                       = "Resources"
vm_hardware_version                         = 18

nsxt_host                                   = "<REDACTED>"
nsxt_username                               = "<REDACTED>"
nsxt_password                               = "<REDACTED>"
nsxt_allow_unverified_ssl                   = true
nsxt_remote_auth                            = false
overlay_transport_zone_name                 = "tz-overlay"
tier0_router_name                           = "T0"
edge_cluster_name                           = "edge-cluster"

env_name                                    = "opscontrol"
opscontrol_base_domain                      = "opscontrol.cloudboostr.int"
network_cidr                                = "10.10.0.0/24"
env_cidrs                                   = ["10.10.1.0/24"]

public_dns_ip                               = "8.8.8.8"
dns_instance_public_ip                      = "10.10.0.150"
dns_instance_private_ip                     = "10.10.0.150"
dns_template_name                           = "Templates/jumpbox-focal-template"

jumpbox_whitelist_ssh_in                    = ["0.0.0.0/0"]
jumpbox_ip                                  = "10.10.0.133"
jumpbox_template_name                       = "Templates/jumpbox-focal-template"

control_plane_api_whitelist_in              = []
control_plane_lb_public_ip                  = "10.10.0.143"
control_plane_template_name                 = "Templates/jumpbox-focal-template"
control_plane_master_ips                    = ["10.10.0.69", "10.10.0.70", "10.10.0.71"]
control_plane_master_cpu                    = 2
control_plane_master_ram                    = 8192
control_plane_worker_ips                    = ["10.10.0.75", "10.10.0.76", "10.10.0.77"]
control_plane_worker_cpu                    = 4
control_plane_worker_ram                    = 16384
control_plane_worker_disk                   = 60
velero_snapshot_volumes                     = false

sensitive_data_storage_container_name       = "sensitive-data.cb"
sensitive_data_vsphere_ca_filename          = "vsphere_ca_crt"
sensitive_data_nsxt_ca_filename             = "nsxt_ca_crt"
sensitive_data_git_private_key_filename     = "git_private_key"
sensitive_data_jumpbox_public_key_filename  = "jumpbox_devops.pub"
sensitive_data_jumpbox_private_key_filename = "jumpbox_devops"
sensitive_data_dns_public_key_filename      = "dns_devops.pub"
sensitive_data_dns_private_key_filename     = "dns_devops"
sensitive_data_k8s_public_key_filename      = "k8s_devops.pub"
sensitive_data_k8s_private_key_filename     = "k8s_devops"
sensitive_data_traefik_crt_filename         = "ingress_crt"
sensitive_data_traefik_crt_chain_filename   = "root_ca"
sensitive_data_traefik_key_filename         = "ingress_key"
sensitive_data_trusted_ca_crt_filenames     = ["vsphere_ca_crt", "nsxt_ca_crt", "root_ca"]

config_repository_url                       = "<REDACTED>"
config_repository_branch                    = "<REDACTED>"
backups_bucket_name                         = "backups.cb"
extensions_bucket_name                      = "extensions.cb"
infrastructure_state_bucket_name            = "infrastructure-state.cb"

aws_access_key = "<REDACTED>"
aws_secret_key = "<REDACTED>"
aws_account_id = "<REDACTED>"
aws_region     = "<REDACTED>"
azs            = "<REDACTED>"
ami_name       = "ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20210907"

env_name                                    = "opscontrol"
opscontrol_base_domain                      = "opscontrol.cloudboostr.int"
hosted_zone_id                              = "<REDACTED>"
network_cidr                                = "10.10.0.0/24"
env_cidrs                                   = ["10.10.1.0/24"]

public_dns_ip                               = "8.8.8.8"
dns_instance_public_ip                      = "10.10.0.150"
dns_instance_private_ip                     = "10.10.0.150"
dns_instance_type                           = "t3.small"

jumpbox_whitelist_ssh_in                    = ["0.0.0.0/0"]
jumpbox_ip                                  = "10.10.0.133"
jumpbox_instance_type                       = "t3.small"

control_plane_api_whitelist_in              = []
control_plane_lb_public_ip                  = "10.10.0.143"
control_plane_master_ips                    = ["10.10.0.69", "10.10.0.70", "10.10.0.71"]
control_plane_master_instance_type          = "t3.medium"
control_plane_worker_ips                    = ["10.10.0.75", "10.10.0.76", "10.10.0.77"]
control_plane_worker_instance_type          = "t3.xlarge"
control_plane_worker_disk                   = 60
velero_snapshot_volumes                     = false

sensitive_data_storage_container_name       = "sensitive-data.cb"
sensitive_data_git_private_key_filename     = "git_private_key"
sensitive_data_jumpbox_public_key_filename  = "jumpbox_devops.pub"
sensitive_data_jumpbox_private_key_filename = "jumpbox_devops"
sensitive_data_dns_public_key_filename      = "dns_devops.pub"
sensitive_data_dns_private_key_filename     = "dns_devops"
sensitive_data_k8s_public_key_filename      = "k8s_devops.pub"
sensitive_data_k8s_private_key_filename     = "k8s_devops"
sensitive_data_traefik_crt_filename         = "ingress_crt"
sensitive_data_traefik_crt_chain_filename   = "root_ca"
sensitive_data_traefik_key_filename         = "ingress_key"
sensitive_data_trusted_ca_crt_filenames     = ["root_ca"]

config_repository_url                       = "<REDACTED>"
config_repository_branch                    = "<REDACTED>"
backups_bucket_name                         = "backups.cb"
extensions_bucket_name                      = "extensions.cb"
infrastructure_state_bucket_name            = "infrastructure-state.cb"

azure_subscription_id       = "<REDACTED>"
azure_client_id             = "<REDACTED>"
azure_client_secret         = "<REDACTED>"
azure_tenant_id             = "<REDACTED>"
network_location            = "East US"
network_resource_group_name = "<REDACTED>"

env_name                                    = "opscontrol"
opscontrol_base_domain                      = "opscontrol.cloudboostr.int"
network_cidr                                = "10.10.0.0/24"
env_cidrs                                   = ["10.10.1.0/24"]

public_dns_ip                               = "8.8.8.8"
dns_instance_public_ip                      = "10.10.0.150"
dns_instance_private_ip                     = "10.10.0.150"
dns_image_reference                         = "0001-com-ubuntu-server-jammy/22_04-lts"

jumpbox_whitelist_ssh_in                    = ["0.0.0.0/0"]
jumpbox_ip                                  = "10.10.0.133"
jumpbox_image_reference                     = "0001-com-ubuntu-server-jammy/22_04-lts"

control_plane_api_whitelist_in              = []
control_plane_lb_public_ip                  = "10.10.0.143"
control_plane_image_reference               = "0001-com-ubuntu-server-jammy/22_04-lts"
control_plane_master_ips                    = ["10.10.0.69", "10.10.0.70", "10.10.0.71"]
control_plane_master_vm_size                = "Standard_A1_v2"
control_plane_worker_ips                    = ["10.10.0.75", "10.10.0.76", "10.10.0.77"]
control_plane_worker_vm_size                = "Standard_A1_v2"
control_plane_worker_disk                   = 60
velero_snapshot_volumes                     = false

sensitive_data_storage_container_name       = "sensitive-data.cb"
sensitive_data_git_private_key_filename     = "git_private_key"
sensitive_data_jumpbox_public_key_filename  = "jumpbox_devops.pub"
sensitive_data_jumpbox_private_key_filename = "jumpbox_devops"
sensitive_data_dns_public_key_filename      = "dns_devops.pub"
sensitive_data_dns_private_key_filename     = "dns_devops"
sensitive_data_k8s_public_key_filename      = "k8s_devops.pub"
sensitive_data_k8s_private_key_filename     = "k8s_devops"
sensitive_data_traefik_crt_filename         = "ingress_crt"
sensitive_data_traefik_crt_chain_filename   = "root_ca"
sensitive_data_traefik_key_filename         = "ingress_key"
sensitive_data_trusted_ca_crt_filenames     = ["root_ca"]

config_repository_url                       = "<REDACTED>"
config_repository_branch                    = "<REDACTED>"
backups_bucket_name                         = "backups.cb"
extensions_bucket_name                      = "extensions.cb"
infrastructure_state_bucket_name            = "infrastructure-state.cb"

auth_url    = "<REDACTED>"
user_name   = "<REDACTED>"
password    = "<REDACTED>"
tenant_name = "<REDACTED>"
tenant_id   = "<REDACTED>"
domain_name = "<REDACTED>"
region      = "<REDACTED>"

env_name                                    = "opscontrol"
opscontrol_base_domain                      = "opscontrol.cloudboostr.int"
network_cidr                                = "10.10.0.0/24"
env_cidrs                                   = ["10.10.1.0/24"]

public_dns_ip                               = "8.8.8.8"
ext_net_name                                = "ext-net"
dns_instance_public_ip                      = "10.10.0.150"
dns_instance_private_ip                     = "10.10.0.150"
dns_image_name                              = "Ubuntu 24.04 Noble Numbat"

jumpbox_whitelist_ssh_in                    = ["0.0.0.0/0"]
jumpbox_ip                                  = "10.10.0.133"
jumpbox_image_name                          = "Ubuntu 24.04 Noble Numbat"

control_plane_api_whitelist_in              = []
control_plane_lb_public_ip                  = "10.10.0.143"
control_plane_image_name                    = "Ubuntu 24.04 Noble Numbat"
control_plane_master_ips                    = ["10.10.0.69", "10.10.0.70", "10.10.0.71"]
control_plane_master_flavor_name            = "2C-8GB"
control_plane_worker_ips                    = ["10.10.0.75", "10.10.0.76", "10.10.0.77"]
control_plane_workerk_flavor_name           = "4C-16GB"
control_plane_worker_disk                   = 60
velero_snapshot_volumes                     = false

sensitive_data_storage_container_name       = "sensitive-data.cb"
sensitive_data_git_private_key_filename     = "git_private_key"
sensitive_data_jumpbox_public_key_filename  = "jumpbox_devops.pub"
sensitive_data_jumpbox_private_key_filename = "jumpbox_devops"
sensitive_data_dns_public_key_filename      = "dns_devops.pub"
sensitive_data_dns_private_key_filename     = "dns_devops"
sensitive_data_k8s_public_key_filename      = "k8s_devops.pub"
sensitive_data_k8s_private_key_filename     = "k8s_devops"
sensitive_data_traefik_crt_filename         = "ingress_crt"
sensitive_data_traefik_crt_chain_filename   = "root_ca"
sensitive_data_traefik_key_filename         = "ingress_key"
sensitive_data_trusted_ca_crt_filenames     = ["root_ca"]

config_repository_url                       = "<REDACTED>"
config_repository_branch                    = "<REDACTED>"
backups_bucket_name                         = "backups.cb"
extensions_bucket_name                      = "extensions.cb"
infrastructure_state_bucket_name            = "infrastructure-state.cb"

Backend configuration

Prepare terraform backend configuration file based on terraform documentation

Example

terraform {
  backend "s3" {
    bucket     = "infrastructure-state.cb"
    key        = "terraform/terraform.tfstate"
    region     = "eu-central-1"
    access_key = "<REDACTED>"
    secret_key = "<REDACTED>"
    encrypt    = true
  }
}

Storage configuration

Create storage configuration file that provides access details for basic storage (e.g. to access sensitive-data bucket) and backup.

Example 

---
storage:
  backend: "s3"
  s3:
    s3_region: "eu-central-1"
    s3_access_key: "<REDACTED>"
    s3_secret_key: "<REDACTED>"
backup:
  backend: inherited

Note

At this point it is possible to start the OpsControl deployment. But during that process Environment configuration is downloaded and used for some initial Environment integration with OpsControle. So to avoid the rerun it is advised to first update Environment configuration.

Configure environments

Configure environments for your purposes. Create or use existing repository and create following structure:

  • config.json - main configuration file with all Environments and users that can access jumpboxes
  • env dir - directory for Environment configuration file, should use the same name as Environment
    • env/config.json - Environment configuration file

Note

URL and branch of that repository should be correctly filled in the "Terraform configuration" step. Also git private key in sensitive data container must work for that repository.

Main config file

Configuration file with the list of all Environments and additional list of users with ssh public keys that will be able to access Environments jumpboxes.

Example

{
    "envs": [
    {
        "name": "env1-test"
    }],
    "users": [
    {
        "name": "test",
        "email_address": "test@grapeup.com",
        "ssh_keys": [
            "<REDACTED>"
        ]
    }]
}

Environment configuration file

Configuration file with all pipelines and variables defined for a specific Environment. Details can be found in environment configuration details

Example

{
    "pipelines": [
        { "name": "deploy_infra", "file": "cb-env/ci/pipelines/deploy-pipeline.yml", "vars": [] },
        { "name": "destroy_infra", "file": "cb-env/ci/pipelines/destroy-pipeline.yml", "vars": [] },
        { "name": "deploy_k8s", "file": "cb-k8s-deployment/ci/pipelines/deploy-pipeline.yml", "vars": [] },
        { "name": "destroy_k8s", "file": "cb-k8s-deployment/ci/pipelines/destroy-pipeline.yml", "vars": [] },
        { "name": "backup_k8s", "file": "cb-k8s-deployment/ci/pipelines/backup-pipeline.yml", "vars": [] },
        { "name": "restore_latest_k8s", "file": "cb-k8s-deployment/ci/pipelines/restore-latest-pipeline.yml", "vars": [] },
        { "name": "security_upgrade_k8s", "file": "cb-k8s-deployment/ci/pipelines/upgrade-pipeline.yml", "vars": [
            { "name": "upgrade_type", "value": "security" },
            { "name": "upgrade_serial", "value": "20%" },
            { "name": "reboot_policy", "value": "never" }] },
        { "name": "smoke_tests_k8s", "file": "cb-k8s-deployment/ci/pipelines/smoke-tests-pipeline.yml", "vars": [
            { "name": "maximum_backups_interval_seconds", "value": "0" },
            { "name": "smoke_test_prefix", "value": "smoke-tests" },
            { "name": "time_interval", "value": "15m" }] }
    ],
    "vars": [
        { "name": "vsphere_host", "vault_var": "/opscontrol/vsphere_host" },
        { "name": "vsphere_username", "vault_var": "/opscontrol/vsphere_username" },
        { "name": "vsphere_password", "vault_var": "/opscontrol/vsphere_password" },
        { "name": "vsphere_allow_unverified_ssl", "vault_var": "/opscontrol/vsphere_allow_unverified_ssl" },
        { "name": "vsphere_datacenter", "vault_var": "/opscontrol/vsphere_datacenter" },
        { "name": "vsphere_datastore", "vault_var": "/opscontrol/vsphere_datastore" },
        { "name": "vsphere_cluster", "vault_var": "/opscontrol/vsphere_cluster" },
        { "name": "vsphere_resource_pool", "vault_var": "/opscontrol/vsphere_resource_pool" },
        { "name": "nsxt_host", "vault_var": "/opscontrol/nsxt_host" },
        { "name": "nsxt_username", "vault_var": "/opscontrol/nsxt_username" },
        { "name": "nsxt_password", "vault_var": "/opscontrol/nsxt_password" },
        { "name": "nsxt_allow_unverified_ssl", "vault_var": "/opscontrol/nsxt_allow_unverified_ssl" },
        { "name": "nsxt_remote_auth", "vault_var": "/opscontrol/nsxt_remote_auth" },
        { "name": "nsxt_max_retries", "vault_var": "/opscontrol/nsxt_max_retries" },
        { "name": "nsxt_retry_min_delay", "vault_var": "/opscontrol/nsxt_retry_min_delay" },
        { "name": "nsxt_retry_max_delay", "vault_var": "/opscontrol/nsxt_retry_max_delay" },
        { "name": "overlay_transport_zone_name", "vault_var": "/opscontrol/overlay_transport_zone_name" },
        { "name": "tier0_router_name", "vault_var": "/opscontrol/tier0_router_name" },
        { "name": "edge_cluster_name", "vault_var": "/opscontrol/edge_cluster_name" },
        { "name": "opscontrol_dns_ip", "vault_var": "/opscontrol/dns_instance_public_ip" },
        { "name": "env_base_domain", "value": "((env_base_domain))" },
        { "name": "enable_lb_service", "value": "false" },
        { "name": "public_dns_ips", "value": ["((public_dns_ip1))", "((public_dns_ip2))"] },
        { "name": "dns_instance_public_ip", "value": "((dns_instance_public_ip))" },
        { "name": "dns_instance_private_ip", "value": "((dns_instance_private_ip))" },
        { "name": "jumpbox_template_name", "value": "((template_name))" },
        { "name": "jumpbox_public_ip", "value": "((jumpbox_public_ip))" },
        { "name": "jumpbox_private_ip", "value": "((jumpbox_private_ip))" },
        { "name": "network_cidr", "value": "((network_cidr))" },
        { "name": "mgmt_subnet_cidr", "value": "((mgmt_subnet_cidr))" },
        { "name": "mgmt_gateway_ip", "value": "((mgmt_gateway_ip))" },
        { "name": "mgmt_router_ip", "value": "((mgmt_router_ip))" },
        { "name": "mgmt_dhcp_server_ip", "value": "((mgmt_dhcp_server_ip))" },
        { "name": "mgmt_dhcp_server_range_start", "value": "((mgmt_dhcp_server_range_start))" },
        { "name": "mgmt_dhcp_server_range_end", "value": "((mgmt_dhcp_server_range_end))" },
        { "name": "services_subnet_cidr", "value": "((services_subnet_cidr))" },
        { "name": "services_gateway_ip", "value": "((services_gateway_ip))" },
        { "name": "services_router_ip", "value": "((services_router_ip))" },
        { "name": "services_dhcp_server_ip", "value": "((services_dhcp_server_ip))" },
        { "name": "services_dhcp_server_range_start", "value": "((services_dhcp_server_range_start))" },
        { "name": "services_dhcp_server_range_end", "value": "((services_dhcp_server_range_end))" },
        { "name": "dmz_subnet_cidr", "value": "((dmz_subnet_cidr))" },
        { "name": "dmz_gateway_ip", "value": "((dmz_gateway_ip))" },
        { "name": "dmz_router_ip", "value": "((dmz_router_ip))" },
        { "name": "dmz_dhcp_server_ip", "value": "((dmz_dhcp_server_ip))" },
        { "name": "dmz_dhcp_server_range_start", "value": "((dmz_dhcp_server_range_start))" },
        { "name": "dmz_dhcp_server_range_end", "value": "((dmz_dhcp_server_range_end))" },
        { "name": "k8s_subnet_cidr", "value": "((k8s_subnet_cidr))" },
        { "name": "k8s_gateway_ip", "value": "((k8s_gateway_ip))" },
        { "name": "k8s_router_ip", "value": "((k8s_router_ip))" },
        { "name": "k8s_dhcp_server_ip", "value": "((k8s_dhcp_server_ip))" },
        { "name": "k8s_dhcp_server_range_start", "value": "((k8s_dhcp_server_range_start))" },
        { "name": "k8s_dhcp_server_range_end", "value": "((k8s_dhcp_server_range_end))" },
        { "name": "k8s_lb_public_ip", "value": "((k8s_lb_public_ip))" },
        { "name": "k8s_lb_enabled", "value": "true" },
        { "name": "k8s_lb_app_profile_name", "value": "default-tcp-lb-app-profile" },
        { "name": "k8s_lb_ingress_active_monitor_paths", "value": ["/infra/lb-monitor-profiles/default-icmp-lb-monitor"] },
        { "name": "k8s_lb_api_active_monitor_paths", "value": ["/infra/lb-monitor-profiles/default-icmp-lb-monitor"] },
        { "name": "k8s_lb_cidr", "value": "((k8s_lb_cidr))" },
        { "name": "k8s_lb_gateway", "value": "((k8s_lb_gateway))" },
        { "name": "k8s_lb_allocation_start", "value": "((k8s_lb_allocation_start))" },
        { "name": "k8s_lb_allocation_end", "value": "((k8s_lb_allocation_end))" },
        { "name": "k8s_node_ports_enabled", "value": "true" },
        { "name": "k8s_node_ports_tcp", "value": ["30000-32767"] },
        { "name": "k8s_node_ports_udp", "value": ["30000-32767"] },
        { "name": "k8s_node_ports_whitelist", "value": ["0.0.0.0/0"] },
        { "name": "jumpbox_whitelist_ssh_in", "value": ["0.0.0.0/0"] },
        { "name": "k8s_api_whitelist_in", "value": ["0.0.0.0/0"] },
        { "name": "env_whitelist_out", "value": ["0.0.0.0/0"] },
        { "name": "ntp_servers", "value": ["ntp.ubuntu.com", "ntp.ubuntu.local"] },
        { "name": "ssh_allowed_hosts", "vault_var": "/opscontrol/ssh_allowed_hosts" },
        { "name": "enable_gateway_policy", "vault_var": "/opscontrol/enable_gateway_policy" },
        { "name": "firewall_logging_enabled", "vault_var": "/opscontrol/firewall_logging_enabled" },
        { "name": "opscontrol_cidr", "vault_var": "/opscontrol/opscontrol_cidr" },
        { "name": "vm_hardware_version", "vault_var": "/opscontrol/vm_hardware_version" },
        { "name": "vmware_tools_upgrade_policy", "vault_var": "/opscontrol/vmware_tools_upgrade_policy" },
        { "name": "cpu_hot_add_enabled", "vault_var": "/opscontrol/cpu_hot_add_enabled" },
        { "name": "memory_hot_add_enabled", "vault_var": "/opscontrol/memory_hot_add_enabled" },
        { "name": "git_private_key", "vault_var": "/opscontrol/git_private_key" },
        { "name": "jumpbox_private_key", "vault_var": "/opscontrol/jumpbox_private_key" },
        { "name": "jumpbox_public_key", "vault_var": "/opscontrol/jumpbox_public_key" },
        { "name": "dns_private_key", "vault_var": "/opscontrol/dns_private_key" },
        { "name": "dns_public_key", "vault_var": "/opscontrol/dns_public_key" },
        { "name": "k8s_private_key", "vault_var": "/opscontrol/k8s_private_key" },
        { "name": "k8s_public_key", "vault_var": "/opscontrol/k8s_public_key" },
        { "name": "trusted_ca_crt", "vault_var": "/uber_pipeline/trusted_ca_crt" },
        { "name": "k8s_key", "vault_var": "/uber_pipeline/((env_name))/k8s_key" },
        { "name": "k8s_crt", "vault_var": "/uber_pipeline/((env_name))/k8s_crt" },
        { "name": "thanos_key", "vault_var": "/uber_pipeline/((env_name))/thanos_key" },
        { "name": "thanos_crt", "vault_var": "/uber_pipeline/((env_name))/thanos_crt" },
        { "name": "thanos_ca_chain", "vault_var": "/uber_pipeline/((env_name))/thanos_ca_chain" },
        { "name": "docker_registry_url", "vault_var": "/concourse/harbor_url" },
        { "name": "docker_registry_username", "vault_var": "/concourse/harbor_username" },
        { "name": "docker_registry_password", "vault_var": "/concourse/harbor_password" },
        { "name": "docker_registry_proxy_cache_project", "vault_var": "/concourse/harbor_proxy_cache_project" },
        { "name": "docker_registry_cloudboostr_project", "vault_var": "/concourse/harbor_cloudboostr_project" },
        { "name": "cloudboostr_image_name", "vault_var": "/concourse/cloudboostr_image_name" },
        { "name": "cloudboostr_image_tag", "vault_var": "/concourse/cloudboostr_image_tag" },
        { "name": "config_repository_url", "vault_var": "/opscontrol/config_repository_url" },
        { "name": "config_repository_branch", "vault_var": "/opscontrol/config_repository_branch" },
        { "name": "k8s_create_standard_storage_class", "vault_var": "/opscontrol/k8s_create_standard_storage_class" },
        { "name": "k8s_set_standard_storage_class_as_default", "vault_var": "/opscontrol/k8s_set_standard_storage_class_as_default" },
        { "name": "k8s_standard_storage_class_name", "vault_var": "/opscontrol/k8s_standard_storage_class_name" },
        { "name": "k8s_storage_class", "vault_var": "/opscontrol/k8s_storage_class" },
        { "name": "elasticsearch_deployment_enabled", "vault_var": "/opscontrol/elasticsearch_deployment_enabled" },
        { "name": "elasticsearch_host", "vault_var": "/opscontrol/elasticsearch_host" },
        { "name": "elasticsearch_port", "vault_var": "/opscontrol/elasticsearch_port" },
        { "name": "elasticsearch_ca_chain", "vault_var": "/uber_pipeline/elk_root_crt" },
        { "name": "elasticsearch_username", "vault_var": "/uber_pipeline/((env_name))/elasticsearch_username" },
        { "name": "elasticsearch_password", "vault_var": "/uber_pipeline/((env_name))/elasticsearch_password" },
        { "name": "infrastructure_state_bucket_name", "value": "((infrastructure_state_bucket_name))" },
        { "name": "extensions_bucket_name", "value": "((extensions_bucket_name))" },
        { "name": "backups_bucket_name", "value": "((backups_bucket_name))" },
        { "name": "extensions_terraform_directory", "value": "env/terraform" },
        { "name": "extensions_terraform_properties", "value": "terraform.tfvars" },
        { "name": "extensions_dns_directory", "value": "env/dns" },
        { "name": "extensions_dns_properties", "value": "dns-properties.yml" },
        { "name": "extensions_k8s_directory", "value": "env/k8s" },
        { "name": "extensions_k8s_properties", "value": "k8s-properties.yml" },
        { "name": "extensions_k8s_terraform_directory", "value": "env/k8s-terarform" },
        { "name": "extensions_k8s_terraform_properties", "value": "terarform.tfvars" },
        { "name": "k8s_template_name", "value": "((template_name))" },
        { "name": "k8s_master_ips", "value": "((k8s_master_ips))" },
        { "name": "k8s_master_cpu", "value": "4" },
        { "name": "k8s_master_num_cores_per_socket", "value": "1" },
        { "name": "k8s_master_ram", "value": "8192" },
        { "name": "k8s_master_network", "value": "((k8s_master_network))" },
        { "name": "k8s_master_network_cidr", "value": "((k8s_master_network_cidr))" },
        { "name": "k8s_master_gateway", "value": "((k8s_master_gateway_ip))" },
        { "name": "k8s_worker_ips", "value": "((k8s_worker_ips))" },
        { "name": "k8s_worker_cpu", "value": "4" },
        { "name": "k8s_worker_num_cores_per_socket", "value": "1" },
        { "name": "k8s_worker_ram", "value": "16384" },
        { "name": "k8s_worker_disk", "value": "200" },
        { "name": "k8s_worker_network", "value": "((k8s_worker_network))" },
        { "name": "k8s_worker_network_cidr", "value": "((k8s_worker_network_cidr))" },
        { "name": "k8s_worker_gateway", "value": "((k8s_worker_gateway_ip))" },
        { "name": "delete_k8s_resources_on_destroy", "value": "true" },
        { "name": "filebeat_release_state", "value": "present" },
        { "name": "nginx_ingress_release_state", "value": "absent" },
        { "name": "traefik_ingress_release_state", "value": "present" },
        { "name": "traefik_ingress_redirect_web_to_websecure", "value": "false" },
        { "name": "prometheus_release_state", "value": "present" },
        { "name": "thanos_release_state", "value": "present" },
        { "name": "velero_release_state", "value": "present" },
        { "name": "velero_backup_schedule", "value": "0 0 * * *" },
        { "name": "velero_backup_ttl", "value": "72h" },
        { "name": "velero_snapshot_volumes", "value": "false" },
        { "name": "ingress_disable_create_tls_secret", "value": "false" },
        { "name": "thanos_ingress_disable", "value": "false" },
        { "name": "velero_cb_annotation", "value": "cloudboostr-velero" },
        { "name": "traefik_ingress_cb_annotation", "value": "cloudboostr-traefik" },
        { "name": "nginx_ingress_cb_annotation", "value": "cloudboostr-nginx" },
        { "name": "prometheus_cb_annotation", "value": "cloudboostr-prometheus" },
        { "name": "filebeat_cb_annotations", "value": "cloudboostr-filebeat" }
    ]
}

Run installation

Start the deployment operation in directory "terraform" in the OpsControl repository:

export CB_IMAGE=""                # Unique name of the Docker image.
export CB_REGISTRY=""             # Docker registry from which Docker image should be pulled.
export CB_VERSION="2.32.0"        # Cloudboostr version to be installed
export CB_PROVIDER="vsphere"      # Name of the provider to use.
export CB_CONFIG_HOST_DIR=""      # Absolute path to Opscontrol configuration directory on the host machine.
export CB_CONFIG_CONTAINER_DIR="" # Absolute path to Opscontrol configuration directory inside Docker containter.

docker run \
  --rm \
  --tty \
  --interactive \
  --platform 'linux/amd64' \
  --name "${CB_IMAGE}" \
  --hostname "${CB_IMAGE}" \
  --mount "type=bind,source=${CB_CONFIG_HOST_DIR},target=${CB_CONFIG_CONTAINER_DIR}" \
  "${CB_REGISTRY}/${CB_IMAGE}:${CB_VERSION}" \
  opscontrol-infra-plan "${CB_PROVIDER}" "${CB_CONFIG_CONTAINER_DIR}"

docker run \
  --rm \
  --tty \
  --interactive \
  --platform 'linux/amd64' \
  --name "${CB_IMAGE}" \
  --hostname "${CB_IMAGE}" \
  --mount "type=bind,source=${CB_CONFIG_HOST_DIR},target=${CB_CONFIG_CONTAINER_DIR}" \
  "${CB_REGISTRY}/${CB_IMAGE}:${CB_VERSION}" \
  opscontrol-infra-deploy "${CB_PROVIDER}" "${CB_CONFIG_CONTAINER_DIR}"

The process of deployment may take from 30 minutes to two hours depending on the underlying infrastructure. If the error occurs there is an error message like: 

module.custom_scripts.null_resource.deploy_dns (remote-exec): Task 31904 | 12:51:59 | Preparing deployment: Preparing deployment
module.custom_scripts.null_resource.deploy_dns (remote-exec):  (00:00:00)
module.custom_scripts.null_resource.deploy_dns (remote-exec):                      L Error: Instance group 'opscontrol-dns-master' references an unknown network 'dmz'
module.custom_scripts.null_resource.deploy_dns (remote-exec): Task 31904 | 12:51:59 | Error: Instance group 'opscontrol-dns-master' references an unknown network 'dmz'


Error: Error applying plan:

1 error(s) occurred:

* module.custom_scripts.null_resource.deploy_dns: error executing "/tmp/terraform_1281010007.sh": Process exited with status 1

The description of the error is in most cases enough to find out which configuration is wrong. After fixing the problem you can re-run the installation and it will continue from where it finished.

After successfull deployment there will be an information message with the jumpbox IP addres. Next step is the Environment deployment which is described on this page.