Git Hands on Demo for the PNI

01_BasicIntro.md

Basic git intro

A few notes, all commands in this tutorial will assume you are working in a linux shell. The same commands should work for OSX but have not been tested. This primer is not intended to give you a full breadth of coverage of the git toolset but show you the basics of making a repository, recording changes, reviewing differences over time, and pushing and pulling from a remote repository. We won't be working with any of the graphical tools for the time being as they are all very different, but the underlying tech to all of them is the same commands we will be doing on the command line.

To get started we'll need a place to work, so lets create that and populate it with some files. I'll be using the files from https://gist.github.com/Wildcarde/9079ead23d5651d871e0 for this demo but feel free to use other files if you would like to.

mkdir gitclass
cd gitclass
wget https://gist.github.com/Wildcarde/9079ead23d5651d871e0/raw/e68a6db01c3e9786d1a49a1b14064dc30f1e18ac/drawfigure.m
wget https://gist.github.com/Wildcarde/9079ead23d5651d871e0/raw/e68a6db01c3e9786d1a49a1b14064dc30f1e18ac/testdraw.sh

We have a folder with some content now but it's not under version control, first lets enable version control of the gitclass folder with the following:

git init
git status
git log

This will create a .git folder adding the structures needed for git to work inside and show you the current status of the working directory vs. that git repository. Now we will add those files to the commit log and commit them to git with the following:

git add *
git status
git commit -m "adding the initial commit to the repository"
git log (q to quit)

These files are now under version control in the source control system, so lets add a few changes to the files to see what happens. First I'm going to make some modifications to drawfigure.m changing the output filename and scaling variable in the file. These changes can then be reviewed, added to staging and committed.

git diff
git commit -a -m "changing scaling factor and correcting output filename"
git log (q to quit)

To add a few more commits I'm going to update the testdraw.sh script with some comments, then to use xvfb-run instead of manually creating it's own xvfb instance since it's safer overall.

#--- make comment updates
git diff
git commit -a -m "adding some notes to testdraw.sh"
git log
#--- make xvfb-run changes
git diff
git commit -a -m "correcting some errors based on knowing how xvfb-run works now."
git log

Now that there are a few lets review all the changes, to do this we will first need to retrieve the hash for the first commit and copy it. After that we can issue a git diff against a specific version like so:

git diff -r e0943ae15b979fa930522730d7eb552da220f6a6 (replace this has with your local git log hash number)

If you want to see the revision history for only a single file add it to the end like so:

git diff -r e0943ae15b979fa930522730d7eb552da220f6a6 testdraw.sh

02_AddingRemoteTarget.md

Remote Targets For Push/Pull/Clone Operations

This document covers the basics of adding a remote server for using the push and pull commands as well as using these remote targets for cloning to new machines if you need to do so.

First we should discuss what each of these operations do.

Clone simply copies the change log of a target repository and places it in a subfolder of the repository name. This is used when you want to create a copy of a project locally to work on, whether that be a github target or a repository stored in your lab area on bucket (using ssh on apps for instance).

Pull updates a cloned local working area to the state of a remote repository. It performs both a fetch and a merge to bring things up to date on your local work. These steps can be performed separately with fetch and merge if you'd like but for basic use the pull command should be sufficient.

Push is the opposite of a fetch, not a pull. The difference is subtle but important, all a push does is send the new list of ref's for your current branch to a remote repository. It can do quite a bit more but at the moment we will be leveraging this to create a central code store located on apps.pni.

First, we need to pick a location for this repository to live for the class I'll be adding one to my fastscratch folder, but you should store yours in a lab area instead to make sure it's backed up. So we will need to ssh to apps and create a repository (replace gmcgrath with your pni user ID):

ssh apps.pni.princeton.edu
cd /fastscratch/gmcgrath
mkdir gitrepo
cd gitrepo
git init --bare ./
pwd

The --bare command tells git to make the folder itself a git repo, instead of hiding it in a .git folder. This repository has no working directory and will never display the raw contents of a file if browsed as it stores the above mentioned ref files only. This repository currently has no identity so we'll have give it some content.
Now that it exists switch back to the local terminal with the gitclass repo and issue the following commands to add our work to the remote ssh target:

git remote add origin ssh://[email protected]:/fastscratch/gmcgrath/gitrepo
git push --set-upstream origin master

From now on you will be able to push and pull from this repository with a simple git push and git pull command. If you do not have ssh keys configured you will be prompted for a password each time you do this but otherwise that's it.

Q: How do I get a copy of my code for use on our clusters?

Now that we have a repository we can talk to / from we have what we need to push updates quickly to our cluster scripts. This can be achieved in the current example with the following commands on apps.pni.

cd /fastscratch/gmcgrath
git clone ./gitrepo ourlivecode
cd ourlivecode
ls -lh

This creates a local working copy of the code living in fastscratch that can similarly be used to push and pull changes from the repository (no configuration should be required the clone will set an origin automatically). Similarly a clone can be issued from another computer over ssh with the following.

cd /<target>/<director>
git clone ssh://[email protected]:/fastscratch/gmcgrath/gitrepo ourlivecode
cd ourlivecode
ls -lh

Q: How do I publish my code on Github?

The short answer is found here.
However this sets github as the origin of the repository which may not always be desirable. So instead you can add it as an alternative destination with the following commands.

git remote add github https://github.com/Company_Name/repository_name.git
git push github master

From now on you can push / pull from github with 'git push github' and pull similarly. And the github repo will reflect the contents of your master branch. Some groups use this to display a clean master branch on github while doing all their actual work in a separate development branch that's only merged into master when it's functional and tested.

Reminder: Never Publish Sensitive Information to Github. This includes configuration files with passwords in them, cryptographic keys, PII, and any other sensitive data

03_AdditionalTopics.md

Additional Topics

This document is a work in progress, not all sections may be complete

Centralized Workflows for Teams

This is by far the easiest team based workflow to wrap your head around. It works off the same model as the the Remote Targets documentation above but with a few tweaks that allow mutliple users to interact with the repository instead of making it owned only by a single user. The overall workflow has to change a bit and will be enforced by the git commandline tool:

git add . #add all changes to the commit
git commit -m "committing my work" 
git pull ## this will pull updates from your teammates

## If there are no issues merging upstream work with the local work
# you will be prompted to add a comment for this new pulled version (:wq will close this vi / vim terminal if you don't wish to add additional notes)
# you can view the changes between your old code and the current source version with:
git diff @~..@  #this is a quick shortcut to compare the current working directory copy with the immediate previous copy

## If the merge can not be completed cleanly you must resolve
# any conflicts that are encountered and inspect changes
git diff #view changes brought in by other team members assuming the merge completed cleanly
#fix conflict changes
git add . # add combine work to commit
git commit -m "merging in teams work"
git push # this will now take the combine code base and push it back up to the server
#Other users will see these changes on their next 'git pull' 

Conflict Resolution

Basic conflict resolution is handled via the same add/commit loop used to add work to the respository. Conflicts in code will appear similar to below:

<!-- Author: Alexander Shvets ([email protected]) -->
<html>
  <head>
<<<<<<< HEAD
    <link type="text/css" rel="stylesheet" media="all" href="style.css" />
=======
    <!-- no style -->
>>>>>>> master
  </head>
  <body>
    <h1>Hello,World! Life is great!</h1>
  </body>
</html>

The <<<<< and >>>>> indicate the start and end of the conflicting block of code with the ===== in between indicates where your code stops and the conflicting incoming code starts. HEAD here refers to your local work, where as 'master' refers to the upstream changes your code conflicts with. To resolve this issue you simply have to remove the lines and make the code correct to what it needs to be like this:

<!-- Author: Alexander Shvets ([email protected]) -->
<html>
  <head>

  <link type="text/css" rel="stylesheet" media="all" href="style.css" />

  </head>
  <body>
    <h1>Hello,World! Life is great!</h1>
  </body>
</html>

After that the merge is treated as any other commit so you can cleanly do the following to push your code up to the remote server:

git add .
git commit -m "fixing merge conflicts"
git push

For more complex cases there are graphical tools that will help manage the merge issues and help you create a new correct file faster than git diff and a terminal text editor, these tools are highly specialized to the task and outside the scope of the discussion today.

NOTE: This pattern does not use the rebase command, it is entirely reliant on the git 'fetch' and 'merge' behaviors, this will lead to more verbose logging in the master repository including all local work but isn't capable of doing some of the damage a bad rebase can do. If you'd like to use rebase instead you should use a branching pattern instead of the shared history setup we are using here, this workflow is more complex but keeps the core master version of the code signifigantly cleaner.

Types of Merging

Basic three way merges are explained visually here: https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging#_basic_merging

Rebase base merging is explained visually here: https://git-scm.com/book/en/v2/Git-Branching-Rebasing

Branches and Tags

Useful Online Tools

• Github offers a practice / getting started space at try.github.io, this will walk you through creating a git repository, making commits, and basic pushing and pulling.
• For practice with the different types of branching strategies and issue resolution an entire tool has been built at http://learngitbranching.js.org/ that is incredibly helpful.
• For a truly deep dive into git, the Git SCM is published that gives both a history of the tool and indepth technical explanation of what the tool is doing.