Saturday, March 22, 2014

Code Review Tools - Stash "versus" Crucible


We've been using Atlassian Crucible for code reviews, but Atlassian Stash becomes the scm/git server standard over the last 12 months, some of the teams are starting to use Pull Requests in Stash as their code review vehicle. As the Gospel According To Stash spreads, inevitably managers are asking "Hey, what is the deal? Is Crucible going away? Should we be switching to Stash Pull Requests?"

So, let's look at it. Some background - this is the state of the products of early 2014. We're on Crucible 3.3 (March 2014) and Stash 2.10 (Feb 2014) - there is a newer release of Stash out right now, and that's mentioned. We're also almost entirely git, but using other scm products is brought into play. 

Stash

  • Stash reviews work best when all your changes can be encapsulated in single pull request from a single repository. 

Crucible

  • Crucible reviews work best when your changes contain commits from multiple repos - Stash cannot span multiple repos in a single review. 
  • Crucible also supports pulling in changes from multiple source control systems (cvs/svn/git/et al) & multiple scm servers (i.e. two different svn servers) in a single review - Stash code reviews have to be pinned to a pull request in Stash only (so git is your only scm choice), and only from a single repo local to the Stash server you create the request on. It's sort of a fine point that 99% of people wouldn't care about, but it needs to be pointed out if you require a certain flexibility. 
  • Crucible will let you review any (set of) commit(s), Stash only lets you review pull requests. So if you wanted to do a formal review of an entire release, or maybe a point in time snapshot before you upload the changes to a public github repo -Crucible would be your tool. Stash cannot free-review commits. 

Conclusion

Crucible is a very established, stable product with a wide audience; it has a very active development team behind it but the audience it caters to is very broad. The Crucible platform probably won’t radically change in the future. Crucible has a long future in front of it. 

That said, Stash is a new product and has a very focused audience - git users only, and catering to the most popular git workflows. It also has very active development team behind it, but because the audience is so narrow compared to Crucible, the change velocity appears much higher. The upgrade to Stash (to 2.11 - the March 2014 release) I'm doing this weekend adds side by side diff, commit commenting outside of pull requests/code reviews, comments at the file level in a commit (for instance, the deletion of a file, or a binary file that can’t be parsed), and a much tighter integration in Jira. Pending pull-requests tied to an issue will now be visible in Jira after this weekend. 

Stash has evolved insanely quickly; Stash 2.0 came out last January, and Stash 2.11 came out a few weeks ago - that’s 11 major(ish) releases in 14 months. Many of the features in Stash that our development teams use on a daily basis simply weren’t there just 12 months ago. I can’t imagine what the product will look like 12 months from now.  

Pull Request code reviews really shine when you can embrace the idea; I highly recommend using them if it fits your team’s needs. If you've gotten this far and pull requests are an unfamiliar term, a quick refresher:
Pull requests are like saying "request to merge". 
  1. You branch server side on any change - a feature, a bug, a hotfix. 
  2. You make your changes, commit them and push your changes back up to the server.
  3. Your Continuous Integration system shows your peers that your change builds & passes the automated test framework.
  4. You create the Pull Request to merge it back into the parent branch.
    1. Add appropriate reviewers. 
    2. They review your change, if they comment & request changes you make those changes and this iterates until they approve the pull request. 
    3. You complete the pull request and Stash automagically merges your changes back to the parent source line and deletes your branch.
If you do this, or could do this, pull request code reviews can be a very powerful thing. 

Tl;dr - Crucible & Stash are both awesome, use what works best for your team.

-Kelly Schoenhofen