How to upgrade the HAProxy Package chart📜
BigBang makes modifications to the upstream helm chart. The full list of changes is at the end of this document.
- Find the current and latest release notes from the release page. Be aware of changes that are included in the upgrade. Take note of any manual upgrade steps that customers might need to perform, if any.
- Clone the repo locally and checkout the release tag from the upstream repo.
- Do a diff of the current BB package against the upstream chart using your favorite diff tool to become aware of any significant chart changes. You can see where the current helm chart came from by inspecting the chart/Kptfile.
- Create a development branch and merge request from the Gitlab issue.
- Merge changes from the upstream chart and template into the existing HAProxy package code, again using your favorite diff tool -or- (if you’re feeling lucky) by using
kpt
as in the two steps below:- In
chart/Kptfile
update.upstream.git.ref
to the tag of the release you found earlier. kpt pkg update chart --strategy force-delete-replace
- In
- Reference the “Modifications made to upstream chart” section below and re-apply these changes to the merged package. If for some reason additional changes to the chart or template are required to get the new version of
haproxy
to work with Big Bang… take note of these additional changes in the same modification section below. This will preserve the Big Bang specific changes for future upgrades. - Update CHANGELOG.md by adding an entry at the top detailing the changes of the new haproxy chart as appropriate.
- Update README.md following the gluon library script
- Update chart/Chart.yaml to the appropriate
version
andappVersion
. The annotation version should match theappVersion
(you will likely need to change that in the modifications below).version: X.X.X-bb.X appVersion: X.X.X annotations: bigbang.dev/applicationVersions: | - HAProxy: vX.X.X
- Use a development environment to deploy and test HAProxy. See more detailed testing instructions below. Also test an upgrade by deploying the old version first and then deploying the new version.
- When the Package pipeline runs expect the cypress tests to fail due to UI changes.
- Update the README.md and the “Modifications made to upstream chart” section below again if you have made any additional changes during the upgrade/testing process.
Testing a new HAProxy version📜
- Create a k8s dev environment. One option is to use the Big Bang k3d-dev.sh with no arguments which will give you the default configuration. The following steps assume you are using the
dev
script in this fashion. - Connect to the k8s cluster and ./install_flux.sh
- Deploy HAProxy with these minimal values overrides; core apps are disabled for quick deployment.
istio: enabled: true monitoring: enabled: true sso: enabled: true istio: injection: "disabled"
- Once deployment is complete, verify that the pod
authservice-haproxy-sso-*
is ready. - Browse to http://prometheus.dev.bigbang.mil
- You should get redirected to an SSO page.
OPTIONAL: If you’d like to test further you can spin up a local sso workflow and test it to get to prometheus using haproxy📜
- BB Integration Test; deploy with the below values:
- Ensure that authservice came up and that there is an haproxy pod in the authservice namespace
- Ensure that there are authorization policies in the authservice namespace
- Navigate to http://prometheus.dev.bigbang.mil and you should get pulled to an SSO page to login
- Delete all of the authorization policies in the authservice namespace, except the default deny - you should start getting 403/503 on prometheus.dev.bigbang.mi depending on prometheus’ state
See also: instructions for authservice and suggested haproxy-sso-workflow test.
istio: enabled: true monitoring: enabled: true sso: enabled: true istio: injection: "disabled" addons: haproxy: git: tag: null branch: "15-implement-istio-authorization-policies" values: istio: enabled: true hardened: enabled: true authservice: git: tag: null branch: "81-implement-istio-authorization-policies" values: istio: enabled: true hardened: enabled: true
Modifications made to upstream chart📜
This is a high-level list of modifications that Big Bang has made to the upstream helm chart. You can use this as as cross-check to make sure that no modifications were lost during the upgrade process.
chart/Chart.yaml📜
- Append
-bb.0
to.version
. - Update the
.appVersion
to the value that you find for.image.tag
inchart/values.yaml
- Add the
annotations."bigbang.dev/applicationVersions"
as described above.
chart/values.yaml📜
- After the license add the
BigBang additions
back.## BigBang additions: imagePullSecrets: - name: private-registry usePSP: false
- Switch the
.image.repository
back to the registry1 upstream.repository: registry1.dso.mil/ironbank/opensource/haproxy/haproxy22 # can be changed to use CE or EE images
- Switch the
.image.tag
to the latest version on ironbank. - Update the
.existingImagePullSecret
toprivate-registry
. - Merge this in.
podSecurityPolicy: create: false
- Update the
.securityContext
to fix non-root user and group so it could pass kyverno.securityContext: enabled: true capabilities: drop: - ALL fsGroup: 1111 fsGroupChangePolicy: OnRootMismatch runAsUser: 1111 runAsGroup: 1111 runAsNonRoot: true
- Update the
.resources
resources: limits: cpu: 100m memory: 500Mi requests: cpu: 100m memory: 500Mi
automountServiceAccountToken📜
The mutating Kyverno policy named update-automountserviceaccounttokens
is leveraged to harden all ServiceAccounts in this package with automountServiceAccountToken: false
. This policy is configured by namespace in the Big Bang umbrella chart repository at chart/templates/kyverno-policies/values.yaml.
This policy revokes access to the K8s API for Pods utilizing said ServiceAccounts. If a Pod truly requires access to the K8s API (for app functionality), the Pod is added to the pods:
array of the same mutating policy. This grants the Pod access to the API, and creates a Kyverno PolicyException to prevent an alert.