Dev Workflow with OCI¶

⚠️ NOTE: This doc is a work in progress as OCI is not the expected or default workflow in Big Bang yet. Changes might be made to the structure or process at any time. ⚠️

If you want to test deployment of a package off of your dev branch, you have two options. This doc covers the OCI workflow. The Git workflow requires nothing more than the values specified in example git values pointed to your development branch.

Package Chart for OCI¶

After making your changes to a chart you will need to package it with helm package chart. You should see output similar to the following:

Successfully packaged chart and saved it to: /Users/me/bigbang/anchore/anchore-1.19.7-bb.4.tgz

Note that Helm strictly enforces the OCI name and tag to match the chart name and version (see HIP 0006), and artifacts will always match the above syntax.

Pushing OCI “Somewhere”¶

In order to use this OCI artifact, you will need to push it to an OCI compatible registry. There are multiple options here: you can push to a self-hosted docker registry, push to Registry1 staging, or push to a Big Bang registry.

Push to self-Hosted Docker Registry¶

The preferred option for OCI storage is in your own personal registry. This example uses the Iron Bank registry image. Log in to Registry1 with Docker before pulling it.

NOTE: We have to host this as a TLS registry due to limitations with Helm.

You will want to spin up the registry on the same host as your cluster (i.e., your ec2 instance, if following the normal developer workflow).

Extract the evaluation *.dev.bigbang.mil certificate with curl and yq v4. The certificate is for development only; use a certificate trusted by both Helm and Flux for your own registry hostname.

umask 077
mkdir -p certs
chmod 700 certs
curl -fsS https://repo1.dso.mil/big-bang/bigbang/-/raw/master/chart/ingress-certs.yaml | yq -r '.packages.istioGateway.values.gateways.public.gatewayCerts[] | select(.name == "public-cert") | .tls.key' > certs/tls.key
curl -fsS https://repo1.dso.mil/big-bang/bigbang/-/raw/master/chart/ingress-certs.yaml | yq -r '.packages.istioGateway.values.gateways.public.gatewayCerts[] | select(.name == "public-cert") | .tls.cert' > certs/tls.crt
chmod 600 certs/tls.key

Set up a Docker registry, mounting the certificate and a writable data directory. Run the container as your local user so it can read the private key and write the data directory:

mkdir -p registry-data
docker run -d -p 5000:5000 --restart=always --name registry --user "$(id -u):$(id -g)" \
  -v "$(pwd)/registry-data:/var/lib/registry" \
  -v "$(pwd)/certs:/certs:ro" \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/tls.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/tls.key \
  registry1.dso.mil/ironbank/opensource/docker/registry-v2:2.8.3

Spin up your development cluster as you normally would. Do not install Flux or Big Bang on top of the cluster yet.

Modify CoreDNS for your cluster to resolve your registry address to the private IP of your cluster host. In the example below we are using oci.dev.bigbang.mil. Run the commands from your cluster host (i.e., ec2 instance if using it) listed in the following:

# Note that these commands assume a Linux host and k3d cluster
export PRIVATE_IP=$(hostname -I | cut -d " " -f1)
kubectl get configmap -n kube-system coredns -o jsonpath='{.data.NodeHosts}' > newhosts
echo "${PRIVATE_IP} oci.dev.bigbang.mil" >> newhosts
hosts=$(cat newhosts) yq e -n '.data.NodeHosts = strenv(hosts)' > patch.yaml
kubectl patch configmap -n kube-system coredns --patch "$(cat patch.yaml)"
kubectl rollout restart deployment -n kube-system coredns

If your cluster is not on your local machine, also modify /etc/hosts to resolve your registry address (i.e. oci.dev.bigbang.mil) to your cluster/registry host’s public IP.

# Run on your registry/cluster host to print public IP
curl http://checkip.amazonaws.com/ 2> /dev/null

# From developer machine add this IP to /etc/hosts
sudo sh -c "echo '<IP from curl above> oci.dev.bigbang.mil' >> /etc/hosts"

Push the packaged chart to this registry. Replace the filename with the output from helm package:
```
helm push anchore-enterprise-4.4.1-bb.1.tgz oci://oci.dev.bigbang.mil:5000
```

Configure these umbrella values to create an OCI HelmRepository and select it for Anchore Enterprise. Retain any other entries your deployment needs: the helmRepositories list is replaced by an override. The Registry1 source uses the private-registry Secret in the bigbang namespace. Install Flux and Big Bang as usual. Anchore Enterprise also requires a license and its other package prerequisites.

helmRepositories:
  - name: registry1
    repository: oci://registry1.dso.mil/bigbang
    existingSecret: private-registry
    type: oci
  - name: k3d
    repository: oci://oci.dev.bigbang.mil:5000
    type: oci
    username: ""
    password: ""

packages:
  anchoreEnterprise:
    enabled: true
    sourceType: helmRepo
    helmRepo:
      repoName: k3d
      chartName: anchore-enterprise
      tag: 4.4.1-bb.1

Push to Registry1 Staging¶

Another option is to push your OCI artifacts to the Big Bang Staging area of Registry1. This is a SHARED area that internal Big Bang team members have access to.

NOTE: You may overwrite other developer’s artifacts if you take this approach.

Login to registry1 with helm: helm registry login registry1.dso.mil. Follow the prompts to add your normal username and CLI token for registry1 auth.
```
❯ helm registry login registry1.dso.mil
Username: myusername
Password: 
Login Succeeded
```
Push OCI artifact to the staging area with helm push <artifact name> oci://registry1.dso.mil/bigbang-staging.
```
helm push anchore-enterprise-4.4.1-bb.1.tgz oci://registry1.dso.mil/bigbang-staging
```

Configure these umbrella values to create an OCI source and select it for Anchore Enterprise. The private-registry Secret must exist in the bigbang namespace and contain Registry1 credentials usable by Flux. Retain any other required helmRepositories entries and configure the Anchore license and prerequisites separately.

helmRepositories:
  - name: registry1
    repository: oci://registry1.dso.mil/bigbang
    existingSecret: private-registry
    type: oci
  - name: staging
    repository: oci://registry1.dso.mil/bigbang-staging
    existingSecret: private-registry
    type: oci

packages:
  anchoreEnterprise:
    enabled: true
    sourceType: helmRepo
    helmRepo:
      repoName: staging
      chartName: anchore-enterprise
      tag: 4.4.1-bb.1

Push to a Big Bang Registry¶

Note that this has a limited use case, since this requires at minimum Istio + Registry to be installed in advance. This may not work well if you are testing Istio or the registry package itself.

Currently you could leverage any of the following as your OCI registry:

Gitlab Project Registries (in a Big Bang installed Gitlab, not Repo1)
Harbor (a built-in Big Bang addon package)
Install a minimal Big Bang on your cluster, not including the package you want to test. You should at least install Istio and the registry (i.e., Gitlab and/or Harbor).
Modify CoreDNS for your cluster to route traffic to x.dev.bigbang.mil (e.g., harbor.dev.bigbang.mil) to the IP of the public ingress gateway.
Modify /etc/hosts to route x.dev.bigbang.mil to the Public IP of your instance (if using a remote/ec2 based cluster).
Push Helm tgz to your chosen registry.
Configure your Big Bang values to set up an additional HelmRepository and point the package to that repository.