AI Assistants and Platform Engineering
How does generative AI fare in speeding up platform development?
AI assistants certainly show promise, but more testing needs to be done for platform and developer teams to get the most value. Dave Sudia shares his experience attempting to use generative AI to speed up Kubernetes development.
My background is in DevOps/platform engineering. While I’ve spent a fair share of time writing web services, I have spent much more writing Ansible and Terraform code. I’ve read articles showing how helpful Copilot, GitHub’s AI assistant based on OpenAI’s GPT-3, can be for Data/ML and app code, which made me wonder how useful it could be for my use case. Now that I’m in Developer Relations, I often create new environments and clusters for learning and doing demos. I usually do this by copy-pasting from my previous infrastructure-as-code repos. If I could speed that process up, it would be beneficial.
But how good would Copilot be at helping with my kind of work? It was trained on public code samples, and there is much more public application code than infrastructure code, which tends to be private simply because it describes a specific set of resources a company uses. When I worked at an open-source non-profit, our infrastructure code was public, but that’s a rare occurrence. So was there enough training data to make Copilot useful?
The Goal
Two of my most commonly used tools in Kubernetes are the CNCF projects cert-manager and Emissary-Ingress. I use Emissary-Ingress as my go-to API gateway for getting traffic into my cluster and cert-manager for creating and renewing certificates using Let’s Encrypt, which Emissary uses to authenticate over HTTPS. I use Cloudflare to manage my domains. Thus every time I create a new cluster, I have to create the following:
- secret that holds my Cloudflare API token
- cert-manager ClusterIssuer to issue certificates
- cert-manager Certificate for my domain
- Emissary-Ingress Host to cause Emissary to listen on a given domain
- Emissary-Ingress Mappings to correctly route traffic to my apps
A lot of boilerplate code is pretty much the same every time, except for the Mappings, which have more variance depending on the needs of the applications they route traffic to. These are all standard Custom Resource Definitions (CRDs) that should have lots of public examples. I would try exclusively using Copilot prompts to generate my code and see how well it performed.
See More: 3 Reasons to Implement Platform Engineering into Your Business
The Test
I signed up for a free trial and installed the GitHub Copilot extension for Visual Studio Code. At its simplest, Copilot works through code comments. In a comment, you input a prompt, then hit return to go to the following line, and Copilot tries to generate the code it thinks should follow. You see the suggested code, and if you hit the tab, it auto-completes the suggestion. You can move to the following line and see the next suggestion.
Creating Cert-manager Resources
Here’s how the process flows for building cert-manager resources.
Creating the Cloudflare API token
Right off the bat, I hit some limitations with Copilot’s experience in my coding niche. I created a YAML file to work in and started with the comment, “Make a cert-manager secret containing a Cloudflare API token.” This got me successive links to documentation but no actual code.
I tried jumping ahead and trying “make a cert-manager Cloudflare dns01 cluster issuer.”
Copilot suggested the following lines:
Interestingly, this response is intended for the command line or a shell script, not for a YAML file. What I would expect is something like the object in the cert-manager’s documentation for this, a Secret object.
I tried starting over with the same command but adding “in yaml format” to it: “make a cert-manager Cloudflare dns01 cluster issuer in yaml format.” That didn’t make a difference. All I could get was a CLI command, which could be considered a best practice! You don’t want to commit your secret values to source code, so I gave Copilot a mulligan and pressed on.
Creating the ClusterIssuer
Next, I put in the prompt “Create a Cloudflare DNS01 ClusterIssuer using the secret above.” The first response back was “kubectly apply -f – <<EOF” Oof, yet another CLI command, but let’s give it a chance:
Not bad this time! I wanted the production server, so I simply added “and the production acme server” to the original prompt and got the same object as above, but using the correct server. But, since I’m in a YAML file, working in YAML, what I’d like to see instead of “kubectl apply” is the “—” that prepends a new object in YAML. I tried adding a few more lines to my prompt:
- “Formatted for a yaml file.”
- “To be put into a yaml file.”
- “Beginning with three dashes for a yaml file.”
The only thing that made a difference was “to be put into a yaml file,” which got me “cat > clusterissuer.yaml” starting the command so that the text would end up in a new yaml file. Everything else began with “kubectl apply.” GitHub Copilot didn’t understand my use case of a yaml file I can apply as a whole, or I was just unable to figure out the right words to explain it, and it seemed to be pulling exclusively from tutorials for people who are working on a CLI. I’ll note that you can ask Copilot for alternative suggestions. I tried, and it didn’t have any. I resigned myself to needing to run a few Find and Replace commands at the end of my experiment, to take all the “kubectly apply -f – <<EOF” lines out and replace them with “—”.
Creating the certificate
This process went more smoothly. I was getting a bit more experienced with crafting my prompts and tried
Which got me the following result:
This is exactly the object I needed (minus the surrounding CLI command lines that I had to accept alongside it)!
So for the first part of the test, creating the objects for cert-manager, Copilot proved it could generate very close to the correct code, just not in the format appropriate for my file. But the secret would have been correct upon creation, and the ClusterIssuer and Certificate code were exactly correct. There are enough examples of cert-manager code out there for it to pull from for a basic use case. If this were my first time doing this work, I think I would have saved time.
Creating the Emissary-ingress resources
For Emissary-Ingress to terminate TLS, it needs a Host that references a Certificate secret. My first attempt was using the prompt “Create an emissary-ingress host resource for the domain test.company.com.” This got me an object using the API version “emissary-ingress.io/v1alpha1.” Nope! I added “using the v3alpha1 api version” (the latest) to my prompt and got a better result. I got the following result:
Close! I did three more rounds of prompt revising and ended up with the following:
It took several rounds of revising because, in the intermediate rounds, I ended up with a Host with no hostname (not valid) or with the acmeProvider field (not necessary when tlsSecret is provided). I learned I needed to be much more specific with Copilot with this object than with the cert-manager objects. And the more specific I have to be, the less time I feel I’m saving. If I need to know every field of an object and what I want to put in there, am I doing much more by just typing the object itself out over the prompt?
This issue was even more apparent when I tried to create Mappings. These are highly variable objects, and I don’t think there are many public examples compared to other codes, as they are specific to services an organization is deploying. I felt I was getting better at being specific in my prompts, and I started with the following prompt and result:
This was the first time that Copilot had given me terrible results. To start, “use_websocket” and “tls” are not fields in this version of the Mapping object. The “service” field is incorrect for using HTTPS to reach a service. The correct version would be:
I tried several more versions of the prompt, particularly trying to explain to Copilot how to format the “service” field accurately, and I could never get it to work. Eventually, I gave up trying to get a correct result and approached it from a different angle. Since you must have a Mapping for every service you want to reach and, depending on the use case, different paths for a single service, there can be a lot of duplication and copy-pasting done. My final task for Copilot was taking a Mapping I had already written and updating it:
This worked well! I got four more accurate Mapping objects that did exactly what I wanted.
Training AI Assistants for Success
Is GitHub Copilot useful for this kind of DevOps work? The results are a mixed bag. I think my initial theory that there would not be enough public training code was mostly right. For cert-manager, which has many more public examples than Emissary-Ingress, Copilot was a legitimate time saver. For Emissary-Ingress, I had to rephrase my Host code as a prompt, and I couldn’t get a valid Mapping out of Copilot.
But I could use it to generate new code from examples I gave it, creating my own little training data set. And I spend much time copy-pasting and making small changes to Mappings. If I learned anything, it’s that as a developer advocate for Emissary-Ingress, I need to learn how to get more examples into the training data set because the AI assistant train isn’t going back into the station. We should make it so that users get the best experience with our project when we can.
How are you improving your platform engineering processes with AI assistants? Share with us Facebook, X(Twitter), and LinkedIn. We’d love to hear from you!
Image Source: Shutterstock
MORE ON AI ASSISTANTS
