Kubernetes VMware vSphere infrastructure – Phase 1

Simple Steps

Below are the basics of how I created my template in preparation for deploying Kubernetes into my VMware vSphere environment.

tl;dr to success:

  1. gather some data
  2. make a VM that will be the template
  3. continue through phase(s)
  4. profit

Operating System Environment or OSE

I chose Ubuntu 22.04 LTS as my base and built out my VM Template using a minimal install, and my scripts are based on the apt system toolset.

  • Your linux-user for the template should allow for SSH public-key logins
  • Your linux-user should be able to utilize passwordless sudo (more on this below)
  • Run a script to prepare the OSE for later use (script at the bottom of this post)

I used Terraform v1.5, and my HCL should be compatible with v1.2 or higher.

Templating your source VM

I created a relatively light VM configuration to build my template.

  • 2 vCPU
  • 2 GiB RAM
  • 32 GiB boot disk

The CPU and RAM configurations are easy to deal with later on (as you will see in my Terraform HCL) though the boot disk resizing is beyond the scope of this post.

I downloaded the ISO image for Ubuntu 22.04 LTS for my minimal install. I attached it to the new VM I created and booted it up. This can also be scripted, but since I reuse the same template for longer, I have yet to feel the need to automate this further. (and the exact opposite of my AWS infrastructure; I am a hypocrite)

Go through the installer and customize as you need. Make sure the ssh server is installed. Very hard to use ssh when sshd is not answering. Once your system reboots, you should double-check the basics and add your SSH public key to ~linux-user/.ssh/authorized_keys for password-less access for provisioning via Terraform remote-exec.

Review the templating script at the end of the post. You should never run someone else’s script without investigating what is happening or going to happen.

Once you have the templating script ready, you need to get it to the currently running VM and can be as simple as scp set-template.sh user@hostname:set-template.sh, and once you are logged into your VM, you can just bash set-template.sh and let it do work.

Since we want password-less sudo, the script will launch sudo visudo, and here is the change:

From:

  • %sudo ALL=(ALL:ALL) ALL

To:

  • %sudo ALL=(ALL:ALL) NOPASSWD:ALL

Then save it.

If you are using my script as-is, it will shut down the VM.

Summary of my templating script though not in order

  • Install some base tools I know I want to be installed
  • Update the system and clean up the snap system (should not be installed in a minimal system)
  • Clean out old log items
  • Clean out shell history
  • Clean out temporary directories
  • Clear current SSH keys for the host (a new rc.local file will be installed, which will regenerate upon first boot)
  • Reset hostname

Documentation

Now document a few things:

  • The name of your VM or VM Template, as you will need this for the next phase
  • The choice you made in linux-user
  • Your life goals, of course.

And onwards to the next phase; having Terraform deploy your templated OSE into useable infrastructure!

Templating script

You can download my set-template.sh script just by clicking or reviewing the code below.

#!/usr/bin/env bash

# We don't need any interactiveness
echo 'debconf debconf/frontend select Noninteractive' | \
  sudo debconf-set-selections

# patch baby patch, and clean up snap (which should not be installed
# on a minimal system)
sudo snap refresh ; \
  LANG=C snap list --all | \
    awk '/disabled/{print $1, $3}' | \
      while read -r snapname revision; do sudo snap remove "$snapname" --revision="$revision"; \
    done ; \
  sudo apt update ; \
  sudo apt -y dist-upgrade ; \
  sudo apt -y autoremove ; \
  dpkg -l | grep ^rc | awk '{print $2}' | xargs sudo dpkg --purge

# Install some things we want
# By the way: rsyslog is not installed in a minimal Ubuntu setup
sudo apt update
sudo apt -y install open-vm-tools rsyslog less

# Stop services for cleanup
sudo service rsyslog stop

# Clear logs
if [ -f /var/log/wtmp ]; then
  sudo truncate -s0 /var/log/wtmp
fi
if [ -f /var/log/lastlog ]; then
  sudo truncate -s0 /var/log/lastlog
fi
if [ -f /var/log/syslog ]; then
  sudo truncate -s0 /var/log/syslog
fi

# Cleanup /tmp directories
sudo rm -rf /tmp/*
sudo rm -rf /var/tmp/*

# Cleanup leftover VMware items
sudo rm -rf /var/log/vmware*.log

# Cleanup current ssh keys
sudo rm -f /etc/ssh/ssh_host_*

# Add check for ssh keys on reboot...regenerate if necessary
# (first boot for sure)
echo ""
echo "Writing out /etc/rc.local to regenerate SSH keys."
echo ""
cat << 'EOL' | sudo tee /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser run level.
#
# Check to see if a key is out there, and if not (it should not be
# upon first boot) execute regeneration of SSH server keys

test -f /etc/ssh/ssh_host_dsa_key || dpkg-reconfigure openssh-server
exit 0
EOL

# Ensure the script is executable
sudo chmod +x /etc/rc.local

# Reset hostname
sudo truncate -s0 /etc/hostname
sudo hostnamectl set-hostname localhost

# Cleanup apt caches
sudo apt clean

# Cleanup shell history
cat /dev/null > ~/.bash_history && history -c
history -w

# Now you need to update your sudoers file correctly
#   search for group sudo
#   change the following line:
#     %sudo	ALL=(ALL:ALL) ALL
#   to look like
#     %sudo	ALL=(ALL:ALL) NOPASSWD:ALL
sudo visudo

# neckbeard
sync ; sync ; sync

# Shutdown as our work is done
sudo shutdown -h now