Merge tools address a shift in the role of the pull request

Today’s software teams have embraced a new workflow for merging and releasing code: trunk-based development. Instead of developing code in feature branches over weeks, they are merging and deploying small chunks of code daily.

Trunk-based development helps developers ship code faster, but as teams adopt the workflow, they’re finding the day-to-day to be rocky. Trunk based workflows can lead to tough oncall shifts, more frequent customer-facing bugs, and higher day-to-day stress when merging code.

A new wave of tools is emerging to help companies stick the landing on their transition to trunk based development: merge tools.

From one headache to the next

In many ways, trunk-based development is a natural extension of the embrace of continuous integration and continuous deployment.

Years ago, most companies followed a strict “GitFlow” approach to software development. In GitFlow teams maintain a unique Git branch for each stage of the software lifecycle: the “main” branch goes out to production; the “dev” branch goes out to a dev environment; “feature” branches are used to develop features locally. Developers ship code by progressively merging branches, copying commits from “feature” branches to “dev” to “main”:

Teams have now soured on GitFlow. The sheer number of branches in GitFlow adds overhead to deployment, and over time, engineers have grown tired of frequent merge conflicts while merging large branches. Reliably porting commits across branches can be tricky, even if you have a Git-Fu expert in house. Strong teams want to deliver code to production continuously, and GitFlow is simply too slow and ungainly.

Trunk-based development simplifies things. There’s only one branch – “main” – which every developer branches off of. Instead of building long-running feature branches, developers merge small atomic changes daily. Once a change gets merged, it often goes out to a QA or production environment immediately. Trunk based development helps teams ship fast.

The speed and simplicity does come with a cost. Because developers merge directly into trunk, it’s easy to introduce breaking changes that affect your whole team. According to CircleCI, ~17% of builds on production branches fail. Changes also get less soak time. For companies running a continuous release train, a change that’s merged in the morning might be in customers’ hands by lunch! The fast turnaround makes it easy to introduce customer-facing bugs.

Most teams are still struggling to make trunk based development sustainable. Research from Devops Research and Assessment confirms that the workflow is now firmly a best practice: they report that 74% of top tier organizations use trunk based development. But their research also shows that adoption comes at a cost. Teams reported that of the new workflows they’ve adopted recently, trunk-based development is the biggest driver of engineer burnout.

Merge tools are reducing risk PR risk and improving developer efficiency

Trunk-based development pull requests can be dangerous. Because every pull request points at the main branch, any breakages a developer introduces will immediately affect every other developer on the branch. Depending on the company, those breakages may also be fast tracked for deployment, sometimes reaching customers in hours. This puts a massive onus on engineers in a trunk-based workflow to write reliable and safe code. Features like stacked PRs, intelligent code analysis, and merge queues are helping engineers accomplish this. 

STACKED PRs

With stacked PRs, engineers break large features into small, atomic, and testable units of code. Each chunk of code is ~100 lines and can stand alone if merged. Engineers then build features by stringing together these small chunks of code into “stacked” PRs. Stacked PRs read like a logical plan – one at a time, an engineer introduces changes which incrementally move the feature towards completion. Because each PR stands alone and has a test plan, different teammates can review different parts of the stack. The developer can also merge individual PRs as soon as they’re accepted, helping them get unblocked faster. While stacked PRs are a nightmare to work with in vanilla git, tool vendors like Graphite and Aviator are making these workflows accessible to teams. 

MERGE QUEUES

Another category of merge tools is helping companies get code into trunk more quickly with automatic merge support. Tools like Mergify, Kodiak, and bors-ng automate the process of merging code so that engineers introducing small commits frequently can reduce the overhead involved. 

INTELLIGENT ANALYSIS

A final way to increase merge safety is through high quality PR analysis tools. Startups are combining AI-based feedback with deterministic static analysis to help engineers assess risk more intelligently. Shepherdly, a pre-seed startup, provides automatic risk scoring for new PRs to help engineers flag particularly dangerous changes. CodeRabbit, which raised a Series A from CRV in August, is employing LLMs to give detailed code feedback. Socket, a security company, is employing deep package analysis to assess the security implications of new changes.

Trunk-based development is here to stay

As a natural extension of the shift to continuous deployment, teams are shipping code to prod on a daily basis. New tools will help teams adopt this practice safely and sustainably.