Code Changes for Updates📜
Fluentbit within Big Bang is a modified version of an upstream chart. kpt
is used to handle any automatic updates from upstream. The below details the steps required to update to a new version of the Fluentbit package.
-
Navigate to the upstream fluentbit helm chart repo and find the latest chart version that works with the image update. For example, if updating to 1.8.11 I would look at the Chart.yaml
appVersion
field and switch through the latest git tags until I find one that matches 1.8.11. For this example that would befluent-bit-0.19.16
. -
From the top level of the repo run
kpt pkg update chart@{GIT TAG} --strategy alpha-git-patch
replacing{GIT TAG}
with the tag you found in step one. You may run into some merge conflicts, resolve these in the way that makes the most sense. In general, if something is a BB addition you will want to keep it, otherwise go with the upstream change. -
Append
-bb.0
to theversion
inchart/Chart.yaml
. -
Update
CHANGELOG.md
adding an entry for the new version and noting all changes (at minimum should includeUpdated Fluentbit to x.x.x
). -
Generate the
README.md
updates by following the guide in gluon. -
Open an MR in “Draft” status and validate that CI passes. This will perform a number of smoke tests against the package, but it is good to manually deploy to test some things that CI doesn’t. Follow the steps below for manual testing.
-
Once all manual testing is complete take your MR out of “Draft” status and add the review label.
Manual Testing for Updates📜
NOTE: For these testing steps it is good to do them on both a clean install and an upgrade. For clean install, point fluentbit to your branch. For an upgrade do an install with fluentbit pointing to the latest tag, then perform a helm upgrade with fluentbit pointing to your branch.
The following overrides can be used for a bare minimum FluentBit deployment:
elasticsearchKibana:
enabled: true
sso:
enabled: true
client_id: platform1_a8604cc9-f5e9-4656-802d-d05624370245_bb8-kibana
eckOperator:
enabled: true
fluentbit:
enabled: true
git:
tag: null
branch: renovate/ironbank
monitoring:
enabled: true
loki:
enabled: true
promtail:
enabled: false
neuvector:
enabled: false
Testing Steps:
- Login to Prometheus, validate under Status
-> Targets
that all fluentbit targets are showing as up
- Login to Grafana, then navigate to Dashboards
> fluentbit-fluent-bit
and validate that the dashboard displays data
- Login to Kibana, then navigate to https://kibana.dev.bigbang.mil/app/management/kibana/indexPatterns and add an index pattern for logstash-*
- Navigate to Analytics
-> Discover
and validate that pod logs are appearing in the logstash
index pattern
Note: as of BB 2.0, the following secrets will need to be copied from the logging namespace to fluentbit in order to successfully test fluentbit log shipping to elasticsearch
- logging-ek-es-http-certs-public
- logging-ek-es-http-certs-internal
- logging-ek-es-elastic-user
The following script can be run to copy the secrets over from the logging namespace. The yq package install instructions can be found here.
kubectl get secret -n logging logging-ek-es-http-certs-public -o yaml | yq '.metadata.namespace = "fluentbit"' - | kubectl apply -f -
kubectl get secret -n logging logging-ek-es-http-certs-internal -o yaml | yq 'del(.metadata["creationTimestamp","resourceVersion","selfLink","uid","ownerReferences"])' | yq '.metadata.namespace = "fluentbit"' - | kubectl apply -f -
kubectl get secret -n logging logging-ek-es-elastic-user -o yaml | yq '.metadata.namespace = "fluentbit"' - | kubectl apply -f -
When in doubt with any testing or upgrade steps ask one of the CODEOWNERS for assistance.
Modifications made to upstream chart📜
Note that this list is likely incomplete currently.
chart/templates/configmap.yaml📜
- Add
fluent-bit.conf:
[OUTPUT]s, lines 11 to 226
chart/templates/_pod.tpl📜
- Add
additionalElastic
toadditionalLoki
(lines 50 to 77) with the adjustment in order toenvFrom
in the middle (lines 55-58) - Add
Values.additionalOutputs
(lines 122 to 137 and lines 162-180) - Change container name to
name: {{ default .Chart.Name .Values.nameOverride }}
chart/values.yaml📜
- Added values for
elasticsearch
,istio
,additionalOutputs
,storage_buffer
,networkPolicies
,openshift
, andbbtests
- Changed image to default to Ironbank image
- Set default
securityContext
,imagePullSecrets
,extraVolumes
,extraVolumeMounts
, andconfig
- Added commented out values for
serviceMonitor.scheme
andserviceMonitor.tlsConfig
chart/Chart.yaml📜
- Name changed to
fluentbit
- Annotations added for versioning, images
- Gluon dependency added for helm tests