Deploying Flux as part of the AKS IaC
Flux is a tool that enables continuous delivery of things like Helm Apps and your applications. While the suggested method of installation is using the Flux CLI tool, is this the way to go in a world of IaC? Let's take a look at how we can install Flux using Bicep, the IaC language for Azure!
4 min read
·
By Thomas Oddsund
·
December 20, 2022
So, what's Flux?
To quote the Flux website:
"Flux is a set of continuous and progressive delivery solutions for Kubernetes that are open and extensible."
Cool, it enables continuous delivery, a staple of modern software development! It works by keeping track of various configurations and acting accordingly to the declared configurations. The configurations are stored as custom resources in Kubernetes, which in practical terms, just means a new ApiVersion and Kind in the Kubernetes manifests.
One example of a custom resource is a GitRepository, where Flux stores information such as the URL, authentication, retry interval, and what to use for git commit verification. A GitRepository can then be used as a basis for other resources - like a Kustomization resource that says where to look for Helm apps, or how your application is deployed. For an in-depth introduction to Flux, see their pages on Core Concepts, Getting Started, or Flux from End-to-End.
Note also that it's a set of solutions, so you can use just one of the Flux controllers it if that's your wish. One example would be to use the Helm Controller to install Helm apps by applying HelmRelease manifests to the cluster, instead of running helm install!
And how do you install Flux?
To summarize the Flux docs, you;
- Install the Flux CLI tool
- Run the relevant
flux bootstrapcommand
And that's it! Now, the bootstrap command does multiple things to make all of this work. If we take GitHub as an example, it..
- .. adds the Flux manifests at the specified path in the repository
- .. installs the Flux components to the Kubernetes cluster
- .. configures a deploy key in the repository for read access
- .. configures Flux in the cluster to sync against the given repository
So after the bootstrap command is done, you have a cluster capable of continuous delivery by pulling configuration. The alternative is to push the configuration with commands like kubectl apply or helm install.
While a pull-based approach adds a little complexity it also has some advantages, such as;
- Security, there's less permanent access to the cluster needed
- Consistency, it regularly pulls the configuration and checks for configuration drift
- History, the benefit of Git also applies to your k8s configuration
So, what's the catch?
It does sound good, especially the Flux components and the ease of installation. But, infrastructure should be managed as code, so using a CLI tool for installation? Of central infrastructure components?
Also, there's the question about how to run the CLI tool. From your machine against production? Eh, rather not. Let's avoid that privilege as much as possible. How about the CI system? Sure, it does so already, but wouldn't it be nice if you could cut off that access to the cluster? Taking Github Actions as an example, they use 2584 IP blocks. That's a pretty big door to leave open to your cluster.
Another thing - how long does it take after your IaC run is finished before you remember to trigger the job that installs Flux? 😏
The AKS IaC solution
If you're working in Azure, there's a good chance you're using Bicep to manage your IaC. To enable Flux in your cluster, all you need is to add these lines of code to your relevant Bicep file;
resource fluxExtension 'Microsoft.KubernetesConfiguration/extensions@2021-09-01' = {
scope: aks
name: 'flux'
properties: {
extensionType: 'microsoft.flux'
autoUpgradeMinorVersion: true
releaseTrain: 'Stable'
scope: {
cluster: {
releaseNamespace: 'flux-system'
}
}
}
}And with that, your cluster has Flux! It now supports pull-based declarative configuration of your cluster configuration, your applications and Helm apps. You can also configure it to send notifications when necessary. No manual steps, and minimal downtime between the readiness of the AKS cluster ready and Flux.
But we're not done. While all of the above is possible in theory, we still haven't told it what configurations to sync. However, with a few more lines of Bicep, we can also kick of syncing immediately once it's installed;
resource fluxConfiguration 'Microsoft.KubernetesConfiguration/fluxConfigurations@2022-03-01' = {
scope: aks
name: 'my-k8s-repo'
properties: {
scope: 'cluster'
namespace: 'flux-system'
sourceKind: 'GitRepository'
gitRepository: {
url: gitOpsBootstrappingRepoSSHUrl
timeoutInSeconds: 180
syncIntervalInSeconds: 300
repositoryRef: {
branch: gitOpsBootstrappingRepoBranch
tag: null
semver: null
commit: null
}
sshKnownHosts: <base64 encoded known hosts>
httpsUser: null
httpsCACert: null
localAuthRef: null
}
configurationProtectedSettings: {
sshPrivateKey: <base64 encoded PEM key>
}
kustomizations: {
unified: {
path: './clusters/${environment}'
dependsOn: []
timeoutInSeconds: 300
syncIntervalInSeconds: 300
retryIntervalInSeconds: 300
prune: true
force: false
}
}
}
dependsOn: [
fluxExtension
]
}This adds your Git repository as a source in Flux, and Flux will read your initial cluster configuration from the environment-specific path ./clusters/${environment}. From there, you can branch out to other sources and/or configurations for your cluster as normal for Flux. A few notes to be explicit;
- sshPrivateKey: Base64 encoded pem key. This means base64 encoding the entire content of the pem key, headers and all
- sshKnownHosts: One Base64 encoded string with all public keys that the remote might present
- gitOpsBootstrappingRepoSSHUrl and gitOpsBootstrappingRepoBranch are parameters sent in to the Bicep file
As you might have understood, it's a small downside to this over the CLI version; you have to manage the access for Flux to the Git repository manually. If we use Github as an example(again), this can be as simple as a read-only deploy key already installed on the repository.
Up next...
Loading…
