Skip to content

Instantly share code, notes, and snippets.

@Rob--W
Last active October 22, 2020 16:56
Show Gist options
  • Save Rob--W/b812be6e40e3ad489f6441ce65fdeb83 to your computer and use it in GitHub Desktop.
Save Rob--W/b812be6e40e3ad489f6441ce65fdeb83 to your computer and use it in GitHub Desktop.
Tutorial to walk through the creation of a Git repository for Firefox/Gecko development, initialized from a git bundle.

The git-cinnabar wiki explains how to set up a repository for Firefox development in Mozilla: A git workflow for Gecko development. The first step of that tutorial, git clone hg::https://hg.mozilla.org/mozilla-unified gecko may fail if your internet connection is unreliable. Mercurial Bundles are an alternative but take 3 hours to initialize. Git bundles for Mozilla development are also available and a recommended alternative.

This tutorial walks you through the creation of a gecko repository for Firefox/Gecko development, initialized using a Git Bundle. Commands are shown on the lines starting with $, and the output is shown for reference. You can copy and paste the command in the shell.

You need at least 10 GB of free disk space for this. On October 22nd, 2020: 3.2GB for the bundle, 7GB for the repository, and a bit more for dependencies and creating an artifact build. To create non-artifact builds, at least 40 GB of disk space is recommended.

Set up git-cinnabar

git-cinnabar allows git to interact with Mercurial repositories. It needs to be installed first.

$ git clone https://github.com/glandium/git-cinnabar
Cloning into 'git-cinnabar'...
remote: Enumerating objects: 282, done.
remote: Counting objects: 100% (282/282), done.
remote: Compressing objects: 100% (170/170), done.
remote: Total 9391 (delta 191), reused 187 (delta 112), pack-reused 9109
Receiving objects: 100% (9391/9391), 2.83 MiB | 735.00 KiB/s, done.
Resolving deltas: 100% (6851/6851), done.
$ export PATH=$PATH:/tmp/path/to/git-cinnabar
Downloading from https://community-tc.services.mozilla.com/api/index/v1/task/project.git-cinnabar.helper.e7e446b707ce7a9a3bdf50a8233bc8237c421f06.linux.x86_64/artifacts/public/git-cinnabar-helper...
 100%

The last command adds git-cinnabar to the PATH environment variable for the current shell session only. To make sure that git cinnabar works in the future, put the export PATH=$PATH:/tmp/path/to/git-cinnabar in your .bashrc file (or .profile).

Download git bundle

Download https://community-tc.services.mozilla.com/api/index/v1/task/project.git-cinnabar.bundle.mozilla-unified/artifacts/public/bundle.git , for example with wget.

$ wget https://community-tc.services.mozilla.com/api/index/v1/task/project.git-cinnabar.bundle.mozilla-unified/artifacts/public/bundle.git
--2020-10-22 14:15:08--  https://community-tc.services.mozilla.com/api/index/v1/task/project.git-cinnabar.bundle.mozilla-unified/artifacts/public/bundle.git
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving community-tc.services.mozilla.com (community-tc.services.mozilla.com)... 34.102.144.36
Connecting to community-tc.services.mozilla.com (community-tc.services.mozilla.com)|34.102.144.36|:443... connected.
HTTP request sent, awaiting response... 303 See Other
Location: https://community-tc.services.mozilla.com/api/queue/v1/task/bI8bQqk9TJaJZ13N6wviVA/artifacts/public%2Fbundle.git [following]
--2020-10-22 14:15:08--  https://community-tc.services.mozilla.com/api/queue/v1/task/bI8bQqk9TJaJZ13N6wviVA/artifacts/public%2Fbundle.git
Reusing existing connection to community-tc.services.mozilla.com:443.
HTTP request sent, awaiting response... 303 See Other
Location: https://community.taskcluster-artifacts.net/bI8bQqk9TJaJZ13N6wviVA/0/public/bundle.git [following]
--2020-10-22 14:15:08--  https://community.taskcluster-artifacts.net/bI8bQqk9TJaJZ13N6wviVA/0/public/bundle.git
Resolving community.taskcluster-artifacts.net (community.taskcluster-artifacts.net)... 99.86.89.26, 99.86.89.84, 99.86.89.71, ...
Connecting to community.taskcluster-artifacts.net (community.taskcluster-artifacts.net)|99.86.89.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3337315178 (3.1G) [application/octet-stream]
Saving to: ‘bundle.git’
bundle.git                                            15%[============>                                                                                                       ] 484.38M  11.1MB/s    eta 4m 28s^C

If the download failed, you can resume the download, e.g. with the --continue flag of wget. While it is possible to use the original URL, you should use the URL after redirects (visible in the output of the previous wget call) to make sure that the same bundle.git file is being downloaded.

$ wget https://community.taskcluster-artifacts.net/bI8bQqk9TJaJZ13N6wviVA/0/public/bundle.git --continue
--2020-10-22 14:16:04--  https://community.taskcluster-artifacts.net/bI8bQqk9TJaJZ13N6wviVA/0/public/bundle.git
Resolving community.taskcluster-artifacts.net (community.taskcluster-artifacts.net)... 99.86.89.26, 99.86.89.84, 99.86.89.71, ...
Connecting to community.taskcluster-artifacts.net (community.taskcluster-artifacts.net)|99.86.89.26|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 3337315178 (3.1G), 2828203516 (2.6G) remaining [application/octet-stream]
Saving to: ‘bundle.git’

bundle.git                                           100%[===================================================================================================================>]   3.11G  11.3MB/s    in 4m 22s

2020-10-22 14:20:26 (10.3 MB/s) - ‘bundle.git’ saved [3337315178/3337315178]

Initializing a git repo from the git bundle

This can take a long while (20 minutes). The input is the 3.2GB bundle from the previous step, the output is a 3.8GB .git directory.

$ git init gecko
Initialized empty Git repository in /tmp/path/to/gecko/.git/
$ cd gecko/
$ git fetch ../bundle.git refs/cinnabar/metadata:refs/cinnabar/metadata
Receiving objects: 100% (14849683/14849683), 3.10 GiB | 44.11 MiB/s, done.
Resolving deltas: 100% (9716159/9716159), done.
From ../bundle
 * [new ref]         refs/cinnabar/metadata -> refs/cinnabar/metadata

The bundle.git file can be removed after the import, and the next step is to actually configure a remote:

$ git config fetch.prune true
$ git remote add mozilla hg::https://hg.mozilla.org/mozilla-unified -t bookmarks/central
$ git config remote.mozilla.fetch +refs/heads/bookmarks/*:refs/remotes/mozilla/*

It could be that the repository has been updated since you had downloaded the bundle. You may want to fetch the latest revisions.

Fetch latest revisions

To ensure that you have the latest version of the code, use git fetch. This only updates the data in your .git database, it does not change anything in your working directory. See the next section for updating your local repository.

$ git fetch mozilla bookmarks/central
WARNING Mercurial libraries not found. Falling back to experimental native access.
Reading 194 changesets
Reading and importing 193 manifests
Reading and importing 1408 revisions of 842 files
Importing 194 changesets
From hg::https://hg.mozilla.org/mozilla-unified
 * branch                      bookmarks/central -> FETCH_HEAD
 * [new branch]                bookmarks/central -> mozilla/central

The "central" part of "bookmarks/central" (aka "mozilla/central" because we created an alias) is the "master" branch of Firefox's source code. Other available bookmarks are listed at https://hg.mozilla.org/mozilla-unified/bookmarks

Create branches to develop

Whenever you want to create a new feature, create a new branch, branched off the mozilla/central branch:

$ git checkout -b bugxxxxxxx mozilla/central 
Updating files: 100% (284741/284741), done.
Branch 'bugxxxxxxx' set up to track remote branch 'bookmarks/central' from 'mozilla'.
Switched to a new branch 'bugxxxxxxx'

Now your gecko git repository is ready for use. See https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html for getting started. If you are contributing to the WebExtensions code in Firefox, see https://wiki.mozilla.org/WebExtensions/Contribution_Onramp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment