Managing multiple clusters with ArgoCD on k3s running in Azure and secured by Traefik and Let’s Encrypt

Code available at https://github.com/ams0/argocd-azure-k3s-traefik

1. Infrastructure installation
2. Ingress installation
3. Argo installation
4. GitOps flow
5. Conclusions

I heard about ArgoCD many times (recently from my friends at Fullstaq) but never tried kicking its tires until now. If you don’t know, ArgoCD is a platform for declarative continuous deployment of Kubernetes applications, and it’s quickly becoming an exceedingly popular choice to deploy and manage applications at scale on multiple clusters. It also recently become an incubated CNCF project.

Since I want to use it for deploying to a cluster, my plan is to have an ArgoCD instance outside my clusters that can manage them independently from the clusters’ lifecycle; hence, I devised this method of deploying ArgoCD into a VM in Azure that’s running the lightweight distribution of Kubernetes from Rancher Labs called “k3s” installed using the k3s helper tool and exposed via the Traefik ingress controller and secured with Let’s Encrypt certificates. Let’s get to it!

Infrastructure installation

Prerequisites:

• Azure CLI
• Azure subscription (already logged in)
• kubectl and jq installed

Start by cloning the repo and entering it:

git clone https://github.com/ams0/argocd-azure-k3s-traefik
cd argocd-azure-k3s-traefik

Let’s create the infrastructure:

• One VM with ports open to access the Kubernetes APIs and ports 80/443 opened for our application):

./deploy.sh <rg> <dns_name> <location> <size>

dns_name should be unique in the region of choice. After a couple of minutes, you’ll have the config file for the k3s cluster (of one VM, with two virtual nodes inside as docker containers).

Ingress installation

You may have noticed that I skipped the installation of k3s’ built-in Traefik ingress; that’s because it still includes the 1.7 branch and I want to use the newer 2.x branch that introduced the IngressRoute CRD (you can follow the progress on this issue). Let’s now install Traefik with this script (you’ll need to pass your email for the Let’sEncrypt certificate authority):

./install-traefik <email>

Argo installation

Finally you can add one or more clusters to be managed by ArgoCD — provided you already have the kubeconfig file available (use the azure-cli to retrieve it) — with this command:

./install-argo.sh <dns_name> <region> <password>

That’s it! The script will patch argocd-server to run over http (SSL termination is done by Traefik) and will patch the secret with the bcrypt-encoded version of your password. Navigate to https://dns_name.region.cloudapp.azure.com and login to ArgoCD. You can also download the CLI and login with:

argocd login --username admin \
--password Password \
dns_name.region.cloudapp.azure.com

Finally you can add one or more clusters to be managed by argo with (provided you already have the kubeconfig file available, for example using the azure cli to retrieve it):

az aks get-credentials -g rg -n cluster_name -f kubeconfig
argocd cluster add --kubeconfig  ./kubeconfig manageme

Note that the last option on the above command line must match the context inside the kubeconfig configuration.

Now, let’s deploy some apps!

GitOps flow

I’m a big fan of the Helm-controller project and I wanted to use it with ArgoCD. In a nutshell, the controller lets you create objects of kind HelmRelease in your cluster (representing Helm releases) and manages the lifecycle of those helm releases programmatically (create/update/destroy). So in the manifests/ folder in my repository, you’ll find the templates to deploy an helm-controller that will in turn deploy helmreleases also present in the same folder.

Fork the repository I provided at the top of this post and head over to the Argo UI tocreate a new app (call it to your liking, and choose the default project) pointing to the your fork and to the manifests/ path.

Importantly, make sure that directory recurse is on. You can also create an object of type “Application” inside your Argo/k3s cluster to achieve the same result:

The app will start syncing right away, installing the Helm operator first and then an nginx ingress controller and you’ll see the tree of resources being created.

That’s it! Now every change to the github repo will be reflected in your cluster.

Conclusion

In conclusion, you might be tempted to ask: is this production ready? Absolutely not, there are still some aspects of the deployment I want to improve (AAD authentication for Argo, for instance, and persistence of data as well). However, it’s a quickstart with ArgoCD in Azure and it will help me learn more in the future. I hope you enjoy it too!

thanks to Gino Filicetti for a draft review of the article.