Performing per-environment staged rollouts of dependency updates with Renovate

In a lot of organisations, you'll find a fairly consistent structure in teams' (GitOps) repositories, where they set up a branch per deployable environment, such as:

* dev
* staging
* prod

Within this structure, the approach for releasing upgrades would be to stage/promote upgrades through environments, starting with dev, through staging and up to prod.

If you're using Renovate - which I have been recommending long before it became my job - you can handle this same workflow very nicely.

Today, I learned a new couple of tricks to make this work even nicer from Renovate maintainer Michael Kriese, who's been working on this setup upstream with the Forgejo maintainers, where it's been working very well for the team.

Prerequisites

As a starting point, you would want to set up your repository with baseBranchPatterns to make Renovate manage each of the branches you use, i.e.

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "baseBranchPatterns": ["$default", "dev", "staging", "prod"]
}

When you specify baseBranchPatterns, you'll start seeing that PRs raised by Renovate are i.e. titled chore(deps): update nginx docker tag to v1.27.3 (dev), which makes it easier to know which of your PRs are for which environment.

This minimal config actually gets you quite far towards this setup, and you could probably stop here. But one thing you'll find is that - depending on how long your release pipelines take - in the time it takes for you to release a PR to dev and staging, when it comes to the release to prod, there may now be a new version of the dependency available.

This is more of an issue with fast releasing packages - like with Renovate itself - but happens often enough that it can scupper your plans, as you go to review the PR for prod, and the PR updates to the new version, which restarts the release-and-testing flow.

Ensuring that these versions can only be rolled out in a specific order is the key problem we're trying to solve here - how do you ensure that Renovate only proposes an update to staging that is definitely on dev, and so on?

Using a Custom Datasource

One option we've got is to utilise Renovate's Custom Datasource functionality, and tell Renovate to look at the version currently on the dev branch, when it's updating staging.

For instance, say that the dev branch has docker-compose.yml:

// dev branch, `docker-compose.yml`
services:
  app1:
    image: nginx:1.27.3

To make sure that staging branch can only ever update NGINX to the version that is in dev, we can write a mix of Custom Datasource (to check what's on dev) and a Custom Manager (to update the version accordingly) config:

// with help from Claude Sonnet 5
// main branch, `renovate.json`
{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "baseBranchPatterns": ["$default", "dev", "staging", "prod"]
  "customDatasources": {
    "app1NginxDevVersion": {
      "description": "NGINX version currently deployed to `app1` on the `dev` branch",
      "defaultRegistryUrlTemplate": "https://github.com/JamieTanna-Mend-testing/staged-releases-branch-per-env/raw/refs/heads/dev/docker-compose.yml",
      "format": "yaml",
      "transformTemplates": [
        // NOTE that the use of `app1` here is more to flex that we can do this, but it may not be the easiest way to show how this works
        // Also, we use `sourceUrl` to make sure that we still have changelog fetching
        "{\"releases\": [{\"version\": $substringAfter(services.app1.image, \":\")}], \"sourceUrl\": \"https://github.com/nginx/nginx\"}"
      ]
    }
  },
  "customManagers": [
    {
      "customType": "jsonata",
      "description": "Track the `app1` NGINX version against what's on the `dev` branch",
      "fileFormat": "yaml",
      "managerFilePatterns": ["/^docker-compose\\.ya?ml$/"],
      "matchStrings": [
        "services.app1.{ \"depName\": \"app1-nginx\", \"packageName\": \"nginx\", \"currentValue\": $substringAfter(image, \":\") }"
      ],
      "datasourceTemplate": "custom.app1NginxDevVersion",
      "versioningTemplate": "docker"
    }
  ],
  "packageRules": [
    // existing rules
    {
      "description": "`app1`: Disable regular updates for NGINX, and instead use `custom.app1NginxDevVersion`",
      "matchManagers": ["docker-compose"],
      "matchFileNames": ["docker-compose.yml"],
      "matchPackageNames": ["nginx"],
      "enabled": false
    }
  ]
}

We can then do something similar on prod to fetch the version from staging, and we'll be able to happily promote upgrades through our environments!

You will notice, however, that this looks like it may get cumbersome as the list of packages grows, especially as we need to disable the existing manager, and add a new Custom Datasource.

Note that looking up the raw file path via the GitHub API should work without any additional wiring of authentication, but if it doesn't, drop us a message in the GitHub Discussions.

Using presets

A simpler solution Michael's come to is the use of a shared preset that lives on the branches that are being updated, which is then referenced by the next branch.

For instance:

* dev
  * renovate.json                 (regular branch-level config)
  * .github/renovate-dev.json     (preset to upgrade `staging` to the version in `dev`)
* staging
  * renovate.json                 (references `.github/renovate-staging.json` on `dev`)
  * .github/renovate-staging.json (preset to upgrade `prod` to the version in `staging`)
* prod
  * renovate.json                 (references `.github/renovate-prod.json` on `staging
  * .github/renovate-prod.json    (preset to upgrade any other repos to the version in `prod`)

These presets then look like i.e.:

// `dev` branch, `.github/renovate-dev.json`
{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "description": "Update versions of dependencies in `staging` to the current version deployed into `dev`",
  "packageRules": [
    {
      "matchBaseBranches": [
        "staging"
      ],
      "description": "nginx",
      "matchDepNames": [
        "nginx"
      ],
      "allowedVersions": "<=1.27.3"
    }
  ]
}

We then take into account this preset by making sure it's in our extends, and that we manage the allowedVersions constraint, in our configuration on main:

// with help from Claude Sonnet 5
// main branch, `renovate.json`
{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    // ensure branches pick up their lower environment's versions
    "github>JamieTanna-Mend-testing/staged-releases-branch-per-env//.github/renovate-dev.json#dev",
    "github>JamieTanna-Mend-testing/staged-releases-branch-per-env//.github/renovate-staging.json#staging"
  ],
  "customManagers": [
    // keep the version in sync
    {
      "customType": "jsonata",
      "description": "Keep the `nginx` `allowedVersions` packageRule in sync with the regular NGINX Docker image",
      "fileFormat": "json",
      "managerFilePatterns": [".github/renovate-*.json"],
      "matchStrings": [
        "packageRules[\"nginx\" in matchDepNames].{ \"depName\": matchDepNames[0], \"packageName\": matchDepNames[0], \"currentValue\": $substringAfter(allowedVersions, \"<=\") }"
      ],
      "datasourceTemplate": "docker",
      "versioningTemplate": "docker"
    }
  ]
}

From here, Renovate will ensure that each branch is only updated as far as the allowedVersions allows it, which is then automagically updated when we're updating the dependency itself on that branch.

This is my preference too, as it keeps things more atomic, and you can see the changes that are being applied more reasonably than decoding a Custom Datasource's transformations.

I'd also recommend the prod branch having a preset that stores the allowedVersions for the versions of packages that are in production, which can then be used when you're performing cross-repo updates. For instance, if you have internal documentation that wants to note the current version of NGINX deployed, you can use that shared preset from the prod branch.

A note about single-branch repos

If you're performing a similar setup, but want to use a single i.e. main branch for this, you'll want to follow the pattern of how the presets manage this with allowedVersions, but will as it's on the same branch, you need to specify the allowedVersions logic in your main branch's renovate.json.