Breaking CI with Dependabot — Weekly Telescope Podcast

In this article I will talk about setting up automatic dependencies updates, its pros and cons…and how to break your CI with it.

Photo by Matthew Henry on Unsplash

Why Do You Need to Know about Dependabot?

Dependabot is an automation bot that would check dependencies in your project for you and create PRs to update outdated ones for you. What is even more exciting is that ever since it’s been acquired by GitHub, it is completely free to use for both your personal accounts and open-source projects. Being an integral part of the GitHub Security features, it will take extra care of patching security vulnerabilities in your repository as well.

Configuration Process

In order to configure Dependabot on GitHub, you will need to create a dependabot.yml config file under .github folder of your repository. Use package-ecosystem to specify the package files to monitor. There is a number of configuration options available. For Telescope, we identified schedule, commit-message, reviewers, and labels to be the most helpful. Our configuration file looked like this:

version: 2
updates:
  - package-ecosystem: 'npm'
    directory: '/'
    schedule:
      interval: 'daily'
      time: '07:00'
      # Use Eastern Standard Time (UTC -05:00)
      timezone: 'America/Toronto'
    open-pull-requests-limit: 10
    commit-message:
      prefix: 'chore: '
    reviewers:
      - 'Seneca-CDOT/winter-2021-telescope'
      - 'c3ho'
      - 'manekenpix'
      - 'raygervais'
      - 'cindyledev'
    labels:
      - 'dependencies'
  - package-ecosystem: 'npm'
    directory: '/src/frontend/gatsby'
    schedule:
      interval: 'daily'
      time: '07:00'
      timezone: 'America/Toronto'
    open-pull-requests-limit: 10
    commit-message:
      prefix: 'chore: '
    reviewers:
      - 'Seneca-CDOT/winter-2021-telescope'
      - 'c3ho'
      - 'manekenpix'
      - 'raygervais'
      - 'cindyledev'
    labels:
      - 'dependencies'
      - 'area: gatsbyjs'
  - package-ecosystem: 'npm'
    directory: '/src/frontend/next'
    schedule:
      interval: 'daily'
      time: '07:00'
      timezone: 'America/Toronto'
    open-pull-requests-limit: 10
    commit-message:
      prefix: 'chore: '
    reviewers:
      - 'Seneca-CDOT/winter-2021-telescope'
      - 'c3ho'
      - 'manekenpix'
      - 'raygervais'
      - 'cindyledev'
    labels:
      - 'dependencies'
      - 'area: nextjs'
  - package-ecosystem: 'npm'
    directory: '/tools/autodeployment'
    schedule:
      interval: 'daily'
      time: '07:00'
      timezone: 'America/Toronto'
    open-pull-requests-limit: 10
    commit-message:
      prefix: 'chore: '
    reviewers:
      - 'Seneca-CDOT/winter-2021-telescope'
      - 'c3ho'
      - 'manekenpix'
      - 'raygervais'
      - 'cindyledev'
    labels:
      - 'dependencies'
      - 'area: autodeployment'

With the help of those tags, we can specify how often and at what time (and in which timezone) should the bot do checks. Since we have multiple package files in the project, we wanted to assigned distinctive labels for each package to be able to quickly identify what module is the update related to. It was also helpful to add a prefix to the commit message to add category of the fix to be displayed in out changelog. Finally, we wanted all of our active contributors to be assigned as reviewers. Oh, and one more thing…there is this open-pull-requests-limit option, which lets you tell Dependabot at what point should it stop creating new PRs. As it turned out, this one can easily exhaust CI quotas and send you a few bills (I’m sorry, Dave).

Fun Part

So as mentioned in the heading, our experience with Dependabot also included breaking a few things. We originally agreed to keep the limit of open pull requests at 10, but we did not consider that in the configuration file that limit is set to each package module. In our case, that was 4 modules = max of 40 open PRs at a time. At first, it seemed to be okay…However, I should’ve taken into account that each PR created by Dependabot is automatically rebased by it whenever it gets outdated. As a result, this triggered an astonishing amount of CI runs on our repository. Dependabot managed to open about 34 PRs, and our tests were long gone…

Useful Remarks

Dependabot also understands a number of commands when you comment them in the Pull Requests. For example, you can tell it to merge the PRs as soon as the checks pass successfully and the required number of approval is collected. You can also tell it to stop rebasing, but that doesn’t necessarily help when you have 34 PRs open.

Conclusion

Dependabot gives you an easy way to update your dependencies as far as you are careful with it. In case your project already has a long list of outdated dependencies, it might be better to update most of them manually first or it least set the number of open PRs limit to a very low number, for each package module.