Development k3d Cluster Automation💣
NOTE: This script does not install Flux or deploy Big Bang. You must handle those deployments after your k3d dev cluster is ready.
The instance will automatically terminate 8 hours after creation.
Install and Configure Dependencies💣
- Install aws cli
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
# sudo apt install unzip -y
unzip awscliv2.zip
sudo ./aws/install
rm -rf aws
rm awscliv2.zip
aws --version
- Configure aws cli
aws configure
# aws_access_key_id - The AWS access key part of your credentials
# aws_secret_access_key - The AWS secret access key part of your credentials
# region - us-gov-west-1
# output - json
# Verify configuration
aws configure list
-
Install jq
Follow jq installation instructions for your workstation operating system.
https://stedolan.github.io/jq/download/ -
Mac users will need to install the GNU version of the sed command.
https://medium.com/@bramblexu/install-gnu-sed-on-mac-os-and-set-it-as-default-7c17ef1b8f64
Usage💣
The default with no options specified is to use the EC2 public IP for the k3d cluster and the security group.
./docs/assets/scripts/developer/k3d-dev.sh -h
AWS User Name: your.name
Usage:
k3d-dev.sh -b -p -m -d -h
-b use big M5 instance. Default is t3.2xlarge
-p use private IP for security group and k3d cluster
-m create k3d cluster with metalLB
-d destroy related AWS resources
-h output help
Export KUBECONFIG💣
In order to use kubectl commands, export your KUBECONFIG
export KUBECONFIG=~/.kube/your.name-dev-config
Update IP💣
To access packages via the web, your IP needs to be updated. Open up /etc/hosts
and update the IP corresponding to <package>.bigbang.dev
sudo vi /etc/hosts
<package>.bigbang.dev
field does not exist, create <package>.bigbang.dev
for each package you are using, or plan to open on the web, then apply the cluster IP to the front of that list
Additional Step for WSL users💣
Using WSL requires users to update both their Windows Hosts File with the cluster IP as well as updating the /etc/hosts
file on WSL.
PowerToys - It is recommended to install the PowerToys application to update your Windows Hosts File using the Host File Editor.
- After opening PowerToys, navigate to
Host File Editor
and update the IP field for<package>.bigbang.dev
- If the
<package>.bigbang.dev
field does not exist, create<package>.bigbang.dev
for each package you are using, or plan to open on the web, then apply the cluster IP
Alternative to using PowerToys:
-
Open Notepad or another text editor like Notepad++
-
In the text editor, select File > Open and open the HOST file location at
C:\Windows\System32\drivers\etc\.
-
Select Text Documents (*txt) in the bottom-right of the Open window and change it to All Files.
-
When files appear in the folder, double click hosts to open it.
-
Edit the HOSTS file and update the IP field for
<package>.bigbang.dev
5a. If the
<package>.bigbang.dev
field does not exist, create<package>.bigbang.dev
for each package you are using, or plan to open on the web, then apply the cluster IP -
Save your changes
Install FluxCD💣
The Big Bang product is tightly coupled with the GitOps tool FluxCD. Before you can deploy Big Bang you must deploy FluxCD on your k8s cluster. To guarantee that you are using the version of FluxCD that is compatible with the version of Big Bang that you are deploying use the Big Bang provided script ./scripts/install_flux.sh
. You will need your Iron Bank pull credentials and command line access to the k8s cluster from your workstation.
./scripts/install_flux.sh -u <your-user-name> -p <your-password>
Deploy Bigbang💣
From the bigbang directory deploy Big Bang via helm
helm upgrade -i bigbang chart/ -n bigbang --create-namespace --set registryCredentials.username=XXXXX --set registryCredentials.password='XXXXX' -f chart/values.yaml
Overrides can be supplemented by adding references to the specific yaml file, the right-most values file will take highest precedence:
-f ../other-overrides.yaml
It is recommended to create a registry-values.yaml
file in an overrides
directory, located outside of the bigbang directory, that contains your credentials instead of having to --set
them each time.
registry-values.yaml💣
registryCredentials:
registry: registry1.dso.mil
username: "XXXXXXXXXXX"
password: "XXXXXXXXXXX"
email: "XXXXXXXXXXX"
Using this overrides file:
helm upgrade -i bigbang ./chart -n bigbang --create-namespace -f ../overrides/registry-values.yaml
To use a different AWS profile, VPC, or AMI💣
The ./docs/assets/scripts/developer/k3d-dev.sh
script uses the default AWS profile and defaults the VPC id and AMI id.
To change this default behavior you can export environment variables before running the script.
Run the script with a specific AWS profile by first exporting the AWS_PROFILE environment variable:
export AWS_PROFILE=my-aws-profile
export VPC_ID=vpc-XXXXXXXXXXXX
export AMI_ID=ami-XXXXXXXXXX
export K3S_IMAGE_TAG=sometag
You can also run these inline with the script, without exporting them to your environment. Example:
K3S_IMAGE_TAG=v1.24.12-k3s1 ./docs/assets/scripts/developer/k3d-dev.sh
Troubleshooting💣
- If you are on a Mac insure that you have GNU sed command installed. Otherwise you will see this error and the kubeconfig will not be updated with the IP from the instance.
copy kubeconfig
config 100% 3019 72.9KB/s 00:00
sed: 1: "...": extra characters at the end of p command
-
If you get a failure from the script study and correct the error. Then run script with “-d” option to clean up resources. Then re-run your original command.
-
Occasionally a ssh command will fail because of connection problems. If this happens the script will fail with “unexpected EOF”. Simply try again. Run the script with
-d
to clean up resources. Then re-run your original command.
Termination and Teardown💣
Termination💣
Run the k3d-dev.sh script with the -d
flag
./docs/assets/scripts/developer/k3d-dev.sh -d
Teardown💣
-
Delete the bigbang namespace
helm delete bigbang -n bigbang
1a. If problems occur during tear where other namespaces do not get deleted, run the remove-ns-finalizer.sh script for each namespace having issues
2. Remove bigbang./scripts/remove-ns-finalizer.sh <namespace>
kubectl delete ns bigbang