When I first joined our Platform Infrastructure team back in August 2017, I was worried of getting lost in the mix of server management and working on requests coming in from the Engineering team. This was the time when our re-architecture project was cancelled and we were going to back to doing things the old way (at least that’s what I had at the back of my mind).  But then we became Site Reliability Engineering (SRE) team.

To give you my background, I am a Staff Software Engineer on the team and have been working with QuickBase for 7+ years.  I have been part of different projects across different frameworks and stacks that we use at Quick Base.

Our first task was to migrate our product code base from Subversion to Git.  Here is my experience with my first Site Reliability Engineering (SRE) project and learnings.

Migration to Git and GitHub

The Quick Base repository is 18 years old and consists of about 40,000+ files with 76,000+ commit history.  We had been using Subversion on a server we operated in our Waltham, MA office since 2005.

Why GitHub?

Consistency

During our re-architecture days, we started using git. We had a few projects deployed in production that are source controlled using git. So, we wanted to be consistent across the board and have all of our projects under one source control.

Integrated Code reviews

With Subversion either hosted or with any other provider, it was hard to do code reviews. You could create a patch file and upload it to a tool that would create a review for you but you have to switch context to see how functions are being accessed or look for code around the piece of code that is changed. With GitHub pull requests you can do all of that and compare your changes with any other branch (bonus).

Whole repository local

With git, you have the whole repository available locally which means that you don’t need to be connected to your Subversion server to look at the logs.

Prepare for Open Source projects

As we are growing as a tech company, we wanted to contribute to the community and made plans to open source some of our tools that we use at Quick Base. GitHub is a great place to host such open source projects.

Add Ons / Extensibility

With GitHub APIs, it is easy to add extensions and webhooks to GitHub. We had a Jenkins plugin to post the status of the builds to the pull requests to make sure there aren’t any breaking changes happening.

Usage

The concentration on the re-architecture project was high over the past 3+ years and we had most of our development team working on git and GitHub.

Challenges/ Concerns

When we decided to move off of Subversion, we had to convince people about the benefits and also address their concerns before we start migration.

Losing history

This was a no brainer, everyone wanted to make sure that we maintain history of our 18 year old product with more than 76k commits.

Handling OEM modules (external modules)

We had a few OEM modules in our project and integrated with our Visual Studio solution. These were not updated for a very long time and some of it was not even in use anymore, but wanted to make sure that all the external modules do work when we migrate.

Handling Quick Base release branches

We do release our code once a month and we mostly work on the trunk branch and in the last week we create a release branch which is a long lived branch (forever). We work on the release branch until its ready to be deployed to production. Any change that we make to the release branch needs to be added to TRUNK as well (a manual process which is highly error prone). Also, for every patch that we do in production, we create a new branch off of the release branch which is also a long lived branch. Any change that goes to this branch needs to go to TRUNK as well but not in the previously released code.

The build

Since most of our deployments are done via running scripts, we discovered that we had a lot of places where we were depending on the SVN revision number which was incremented by 1 on every successful commit. Our artifact version was also the SVN revision number, so we relied on being able to sort by revision number. We started using short git hash (“GIT_HASH=%GIT_COMMIT:~0,7%” in Windows Batch) to be used for artifact versions. But since the git hash is alphanumeric string, sorting doesn’t provide chronological order. So we decided to use the build number on our Jenkins job that generates the artifact as the artifact version. That solved a lot of our problems since it was incremented by 1 for each build.

Fear of change

This was the biggest hurdle we had in mind when we started thinking of migrating our repository to Git. People were worried about the change causing script failures that are hard coded to use the code in a certain way.

Today (6 months later)

After working with Git for almost 6 months and about 6 releases, here is our view:

Every change is a branch

Every change being made to the source code is done via a separate branch and merged only if reviewed and approved. This ensures that any code that goes to develop branch is actually reviewed to keep develop clean.

Try build results

Since all of our changes are separate branches, we run try builds against the branch to make sure that the change doesn’t break the develop branch. You can read more about Try Builds here 

Pull requests improve code reviews

Pull requests have been one of the best tools to share knowledge and provide constructive feedback to new developers and get valuable feedback from peers.