How This Blog Is Served
walked through the full path from git push to a served page, touching every layer along the way. This post stays in one place: Kargo, the piece that sits between “CI built a new image” and “ArgoCD is told to deploy it.” It’s the part of the pipeline most likely to look like magic from the outside, so this is a proper look at what it’s actually doing — plus a couple of real gotchas hit while getting auto-promotion working correctly.
Source: git.wagner.org.za/cw/blog
— Kargo manifests live in deployment/blog/kargo.yaml.
Why Not Just Let ArgoCD Do Everything?#
ArgoCD is excellent at one job: making the cluster match what’s in Git. It has no opinion about when a new image tag should go into that Git manifest, whether it should land in a staging environment first, or whether it’s safe to promote further. Something upstream of ArgoCD has to make those calls.
That’s Kargo : it watches a container registry for new image tags, decides when and where they’re allowed to be promoted, writes the resulting Git commit, and then verifies the outcome before calling it done. ArgoCD stays exactly as simple as it’s supposed to be — sync Git to cluster — while Kargo owns the promotion logic in front of it.
The Core Model#
Kargo has five resource types that matter for this pipeline:
Warehouse — subscribes to one or more artifact sources. This blog’s Warehouse watches the OCI registry for tags matching ^v\d{4}\.\d{8}$ (the vYYYY.MMDDhhmm format the build pipeline stamps on every image).
Freight — an immutable snapshot of “here’s a specific artifact version.” Every time the Warehouse sees a new matching tag, it wraps it in a Freight object. Freight is the unit of promotion — you don’t promote an image tag directly, you promote the Freight that references it.
Stage — an environment, plus the recipe for getting Freight into it. A Stage declares which Freight it’s allowed to accept (requestedFreight), the steps to run when promoting (promotionTemplate), and how to verify the result (verification.analysisTemplates).
Promotion — the actual execution record of moving one piece of Freight into one Stage. Each Promotion runs its Stage’s promotionTemplate.spec.steps in order and records a phase (Succeeded, Failed, Errored) plus per-step output.
For this blog, two Stages exist:
# preview — subscribes directly to the Warehouse
requestedFreight:
- origin: {kind: Warehouse, name: blog}
sources: {direct: true}
# production — only accepts Freight that has already passed through preview
requestedFreight:
- origin: {kind: Warehouse, name: blog}
sources: {stages: [preview]}
That sources: {stages: [preview]} line is what makes this a pipeline rather than two independent deploy targets — production can only ever receive Freight that preview has already accepted.
What a Promotion Actually Does#
The preview Stage’s promotionTemplate is five steps, run in order:
Steps 1–4 are Kargo acting as a bot committer: clone, edit a manifest with the new tag, commit, push — using its own kargo-git-creds secret, distinct from any human’s credentials. Step 5 then directly patches the ArgoCD Application resource’s desiredRevision to point at the commit Kargo just made, so ArgoCD doesn’t have to wait for its next poll interval to notice.
After the Promotion finishes, verification.analysisTemplates kicks in — an AnalysisTemplate runs the Playwright e2e suite (tests/playwright/blog.spec.ts) against cw-preview.apps.wagner.org.za. If Chromium can’t complete the test run, the Stage is marked unhealthy even though the Promotion itself “succeeded” — deploying and verifying are tracked as separate concerns.
Where Argo Rollouts Comes In#
Kargo’s job ends the moment it pushes a Git commit and pokes ArgoCD. Everything from there — actually standing up a new version and cutting traffic to it — belongs to Argo Rollouts, watching the same Rollout resource ArgoCD just synced.
The blue-green mechanics (why two ReplicaSets exist, how the preview/active Services split traffic) are covered in more depth in the routing post
— the short version is that Kargo’s Promotion only ever touches Git and the Application’s desired revision. It never talks to the Rollout directly, and it never decides when to cut production traffic. That decision stays a deliberate, separate kubectl argo rollouts promote — Kargo can record that a Freight reached the production Stage, but the traffic switch is still a human action.
Two Gotchas That Bit This Pipeline#
Wiring this up looked correct on paper — Warehouse polling, Stages defined, promotion steps in place — but nothing was actually promoting. Two separate, non-obvious issues turned out to be responsible.
1. Auto-promotion needs an explicit opt-in#
The preview Stage subscribes directly to the Warehouse specifically so it can auto-promote every new build without waiting on a human. That’s not the default behavior, though — AutoPromotionEnabled defaults to false, and it isn’t a field on the Stage at all. It lives on a separate ProjectConfig resource:
apiVersion: kargo.akuity.io/v1alpha1
kind: ProjectConfig
metadata:
name: blog
namespace: blog
spec:
promotionPolicies:
- stage: preview
autoPromotionEnabled: true
Without this resource, new Freight arrives, sits fully “Fulfilled” against the Stage’s requestedFreight, and nothing happens — no error, no event, just silence. The fix is a resource that has to exist somewhere for auto-promotion to work at all, and it’s easy to build the Warehouse/Stage pair, watch it sit there, and assume the Warehouse itself is broken.
2. One Argo CD Application, one authorized Stage#
Kargo won’t touch an Argo CD Application unless that Application explicitly opts in via an annotation:
metadata:
annotations:
kargo.akuity.io/authorized-stage: blog:preview
The natural instinct, since both preview and production target the same shared Application (this blue-green setup deploys through one Rollout, not two), is to authorize both: blog:preview,blog:production. That looks reasonable, and newer Kargo builds actually do support comma-separated values — but the version this cluster runs parses the annotation with strings.SplitN(value, ":", 2), which takes the whole rest of the string as the stage name. blog:preview,blog:production becomes project=blog, stage=preview,blog:production — which matches neither real Stage, and every argocd-update step fails with is not authorized.
Until either a Kargo release ships multi-stage authorization or this Application gets split in two, the annotation carries a single value — blog:preview — which unblocks the auto-promoted path this post is about, at the cost of production promotion needing a follow-up fix.
Both were silent-failure modes: no crash, no obvious log line pointing at the cause, just a Stage sitting at 0/1 Fulfilled or a Promotion erroring several steps in. Kargo’s own status.conditions and the controller logs (kubectl -n kargo logs -l app.kubernetes.io/name=kargo) were what actually surfaced both root causes.
Summary#
Kargo doesn’t replace ArgoCD or Argo Rollouts — it sits upstream of both, deciding what gets promoted and when, while leaving “sync Git to cluster” and “how to cut traffic” to the tools that already do those jobs well. For this blog that means: a Warehouse polling the registry, Freight objects marking discovered builds, a preview Stage that auto-promotes and verifies with Playwright, and a production Stage that requires a deliberate human call before Kargo — and separately, kubectl argo rollouts promote — will touch live traffic.
This post itself went through that exact path to get published: built by the CI pipeline, promoted to preview by Kargo, and verified against cw-preview.apps.wagner.org.za before merging to production.