Git From the Shell

Dave White

2019-05-8

Setup

git clone https://github.com/portalgun/hellogitworld
git clone https://github.com/portalgun/mergeConflict

A case for terminal

Most of learning git is just figuring out what you can do
Many things difficult or impossible in GUI
GUI not always available on compute servers
Some things are easier with a client

Overview

Improve basic workflow
Parse and navigate the history
Branching & Merging
Other misc stuff

Improving workflow

Bad commit practices

Non descriptive commit messages ("pushed lots o' stuff")
Large commits
Commit -a (unless responsibly)
Pushing to origin with every commit

Solution

Practice using git at each working feature (function/class scale)

#+ATTR_REVEAL: :frag (appear)

Workspace (untracked/unstaged)
Index (add/stage)
History (commit)

Details

Git should be a continual flow

#+ATTR_REVEAL: :frag (appear)

Branch - begin working on more than a couple lines of code
Add - begin testing, cleaning up
Commit - tested
Push - share/backup (no neeed to do this so frequently, commit frequently instead)

Creating a branch

git branch <branch-name>
git checkout -b <branch-name>

Commit

Some things you maybe didn't know…

COMMIT BODY

line 1 - title
line 2 - blank
line 3 - longer description

OR

#+begin_src bash -i
git commit -m "title" -m "description"

INTERACTIVE ADD

git add -i <larger>
git add -p <smaller>

Commit A Single File With Staged Files

git commit <file>
git reset <file>

Small demo…

Reset

Note that these will not affect a push; reset applies to just 1 branch.

I forgot to test my last commit:

git reset --soft

I need to add/fix something to my last commit:

git reset --mixed

My last commit is utter garbage and I want to trash it completely:

git reset --hard

Details

Note that there is a difference between

git reset --soft

and

git reset --soft HEAD~1

Without specifying a commit (latter), the reset only applies to your index and workspace.
To undo a commit, you need to explicitly specify which commit you are referring to.

Undo a reset

git reset HEAD@{n}

replace n for how many times you did the git reset command
If I did 'git reset HEAD~1' or 'git reset HEAD~20', then n is 1
If you did 'git reset HEAD~1; git reset HEAD~1', then n is 2

Stash

git stash save [<message>]

git stash list

git stash show

git stash apply stash@{n} #n is a number

git stash drop stash@{n}

git stash -u

git stash pop stash@{n} #drop and apply

Revert

git revert <ref>

Like reset, but appropriate for your origin
It doesn't undo the commit, but creates a new commit that is identical to the commit you want to 'reset to'.

Details

How would we undo two commits, but retain changes

git reset --soft HEAD~2

This also works on individual files

#+ATTR_REVEAL: :frag (appear)
Other notation - next branch point:

HEAD^

Example:

Create a branch called "cleanup"
Add "Forked by <your name>" to README.txt
Add this change to index
Stash
Add "Created by Matthew McCullough" to README.txt
Commit with a message and a body
Undo commit and move them to workspace only
Stash these changes
Apply first stash
Commit with a message
Apply second stash
Commit with a messsage and a body
Remove all your stashes

Solution:

git checkout -b cleanup
echo "\nForked by Dave" >> README.txt
git add -u README.txt
git stash save Dave

#MAKE SURE THINGS ARE GOOD
git status
cat README.txt

echo "\nCreated by Matthew McCullough" >> README.txt
git commit -a -m "Signed Matthew McCullough" -m "What a great dude"
git reset --mixed HEAD~1 #HEAD~1 must be included
git stash save -u Matt #-u must come right after save
git stash list
git stash apply stash@{1}
git stash drop stash@{1}
git add -u README.txt
git commit -m  "Added name Dave"
git stash pop stash@{0}

#This last line will create a conflict, which we will talk about later
#Fix the conflict by removing lines with:
#<<<<<<<
#=======
#>>>>>>>
#You can do this manually or with a stream editor:
sed -i -r '/^<{3,}/d; /^>{3,}/d; /^={3,}/d' README.txt

cat README.txt

git commit -a -m "Signed Matthew McCullough" -m "What a great dude!"

History parsing

Log

View commits in a tree

git log --graph --oneline --decorate --all

Bash alias

Git was meant to be scripted/aliased

#$HOME/.bashrc in linux
#$HOME/.bash_profile in mac
alias glg="git log --graph --oneline --decorate --all"

Basics

git log <tag, hash, branch,remote>

git log RELEASE_1.1

git log 8d2636d

git log origin/master

git log origin/feature_subtraction_polished

Limiting

git log -3

git log --since="5 years ago"

git log --until="8 years ago"

git log --author="Peter"

Specific details

git log -L <startrange>,<endrange>:<file>
    git log -L 1,2:pom.xml
    git log -L 1,3:pom.xml

git log -L :funcname:file
    git log -L :repositories:build.gradle
    git log -L :repositories:build.gradle -L :dependencies:build.gradle

git log -S''
    git log -S'maven'

git log -G''
    git log -G'dep.+'

git log --grep'Search messages'
    git log --grep='script'

Combine as many options as you want

Other help

Shell tools

git log | less
git log | grep -ei "some string"

Gitk, Gitkraken, Magit, etc..

Checkout

How to move around.
Try this:

git checkout master
git status

Detached head

Try this:

git checkout RELEASE_1.0
git status

Detatched - not on newest commit

Any commits without explicitly branching will not be attached to the tree

Create a new branch off the checked out branch if you are making a commit to an older commit

Example:

Find where sum function was added (Trick question)
Navigate to where that was committed (also tricky)
Navigate to the next branch point from there
create a branch called "testbranch"
Create a new file called TRASH
Commit the file
Navigate to master
Find the commit associated with testbranch

Solution:

grep -Ri "sum"
git log -L :sum:src/Sum.groovy
git checkout 333fd9
git checkout HEAD^
git checkout -b "testbranch"
touch TRASH
git add TRASH
git commit -a -m "created trash"
git checkout master
git log testbranch

Branching & Merging

1. Branching discussion

Branch/Merge fear

People don't like branching because it means merging

However:

#+ATTR_REVEAL: :frag (appear)

You cannot break anything
You are already creating a branch for every time you push/pull
Merging all at once can be better than multiple doing it multiple times (example)
There is an easy way to update your branch with master if needed

Problems clean branching helps with

Simply, branches allow for

Organizing code by stability
Organize code by individual and/or feature

Workflow

Think of commits as a sections in a textbook.
A branch is going to be chapters in your textbook
You can work on multiple chapters of a book at a time.

#+ATTR_REVEAL: :frag (appear)
1 feature per branch, rule of thumb

Smallish change for many

Switch your usual large commits, with a merged branch.
Actually will make things easier to merge

2. Basic Merging

Remotes basics

Master & origin/master are different branches
Feature1 & feature1/master are different branches
Push/pull is just fetch + merge

Pushing, pulling, merging

Same thing in practice, but different vantage point
If you are not a manager you would need to do a pull request

#+ATTR_REVEAL: :frag (appear)
How to pull request

Push your branch
Submit a pull request
Wait for me to merge it

Example:

Send me a pull request to merge "cleanup" with master

Ways of Merging

Merging is always towards the head
Think of master as a giant git beast

Two main ways of merging

FF - no separate 'branch'
No-ff - no merge commit

#+ATTR_REVEAL: :frag (appear)
Default:

If no divergence, ff

If divergence, no-ff

Defaults

Why set your default to no-ff? keeps things separated by feature/user

git config --global --add merge.ff false
git config --add merge.ff false

Also

Only merge if nothing new at merge point

git merge ff-only

Example:

create a backup of your repo
merge "cleanup" with master, no-ff
undo your merge
merge "cleanup", ff
try to merge tesbranch with –ff-only

Solution:

cp ../hellogitworld ../hellgitworld_Backup #linux
cp -a ../hellogitworld ../hellgitworld_Backup #mac
git checkout master
git merge cleanup --no-ff
git reset --hard HEAD~1
git merge --ff-only

Merge vs rebase

Rebase is ugly
Few cases when you should use it

Pull –rebase

Makes things easier, eliminates many merge issues as you go along

git config --global pull.rebase true

When you push, it will look like…

Pull –rebase note

You do NOT need your repo to look exactly like remote
Nor should you try to do this, or expect it.
What matters is that both are clear
In a merge it doesn't matter where code came from
Small differences in master, larger differences between your branches

Example:

Pull master into testbranch

Solution:

checkout testbranch
git pull –rebase master

Deleting a branch

git branch -D <branch>

Doesn't delete the commits and their structure, but opens up the namespace
May be useful to add a note here (notes later)

Restore a branch

git checkout -b <branch> <sha>

Example:

Delete testbranch
Look at where test branch was
restore test branch

Solution:

git branch -D testbranch
git checkout -b testbranch <sha>

3. Tag/Notes

Note on tags/branches

When a ref is created, it needs to be push - this includes new branches and tags
Everything point to point (one branch at a time), unless required or specified
eg. commits have dependencies

Explicitly need to push:

Deleted branches
Tags
New branches unless included with a commit

Branch example

Notes

Reserve tags for versions numbers, notes can be human readable version
Notes are for additional detail attached to commits
Do not affect the history, hash number, commit message etc.
You can edit at any time
Does not clutter namespace

Details

git notes add -m "message"

git log -p notes/commits

git push <remote> refs/notes/*

git fetch origin refs/notes/*:refs/notes/*

git notes --help

Example:

Tag commit 435ffce as '\(\text{RELEASE}\_ 1.1.1\)'
Add a note to RELEASE 1.1

Solution:

git checkout 435ffce
git tag RELEASE_1.1.1
git checkout RELEASE_1.1
git notes add -m "Missing log files"

Different branch models

Industry standard:

1. hotfix
1. master
1. dev
1. <Features>

Everything gets pulled top to bottom

#+ATTR_REVEAL: :frag (appear)
New project/single individual/small project

1. master
1. <Features>

#+ATTR_REVEAL: :frag (appear)
Why make MASTER stable only? Default

Merge conflicts

Overview/Flow

Fetch
Diff
Blame
Cherry-pick/merge parts
Merge
Branch delete

Fetch & Diff

Fetch, then diff

#commit or stash then...
git fetch
git log ..<branch> #list changes that will be merged
git diff ..<branch>
git difftool ..<branch>

diff Notation

difference between a & b

a..b
..b

difference between common ancestor and b

a...b
...b

Log Notation

everything that is in b but not in a

a..b

only in one or other

a...b

Fetch & Diff

Many different diff/merge conflict tools

meld - popular
kdiff3 - popular
vimdiff - vim users

Many graphical git wrappers have their own

Diff tool setup and use

git config --global merge.tool <merge-tool>
git config --global diff.tool <merge-tool>
#git config --global diff.external <merge-tool>

git difftool <COMMIT_HASH> file_name
git difftool <BRANCH_NAME> file_name
git difftool <COMMIT_HASH_1> <COMMIT_HASH_2> file_name

git difftool RELEASE_1.1 RELEASE_1.0
git difftool RELEASE_1.1 RELEASE_1.0 runme.sh

Blame

git blame <file>

git blame -e <file>

git blame -L <start>,<end><file>

git blame -L :function <file>

git blame -L ... -M <file>

Who do I talk to, yell at?
Merges should often be a group effort

Example:

get the email of whoever added line three to runme.sh

Solution:

git blame -e -L 3,3 runme.sh

Cherry pick

git cherry-pick (commits in order to append, first to last)

Example:

Create a branch 'hotfix' off of Release 1.1
Cherry-pick commits from cleanup
Tag as \(\text{RElEASE}\_1.1\text{h}\)

Solution:

git checkout Release_1.1
git checkout -b hotfix
git cherry pick <SHA> <SHA>
git tag Release_1.1h1

Initial merging

Complete failure to merge - updated index, not pushed

git stash
git reset etc..

Strategy

merge -s <strategy>

Recursive (default)
Patience - slower, more effective recursive
Resolve - pick one of the possible merge bases, and hope for the best.
Ours - only use our changes, ignore theirs (but keep history)
Theirs - only use their changes, ignore ours (but keep history)
Octopus - multiple head default
Subtree - (subtree)

Strategy option

merge -s <strategy> --strategy-option <option>

Don't confuse with -s

Ours - take ours if there is a conflict
Theirs - take theirs if there is a conflict

Conflict mode

git log --merge

Diff

git diff --ours
git diff --theirs
git diff --base

Show

git show :1:<filename> #common ancestor

git show :2:<filename> #your versions

git show :3:<filename> #their version

git clean -f #remove any extra files used for merge resolution

These can be piped.

#+ATTR_REVEAL: :frag (appear)

checkout --theirs <filename>

Reset or start over

git merge --abort  #RESET
git checkout -m -- <filename> #START MERGE CONFLICT OVER FOR A FILE
git checkout -m -- .* #for everything

conflict markers

conflictstyle diff3 (shows common ancestor too)

<<<
Towards you, you are always first (left)
|||
COMMON ANCESTOR
========
THEIRS
>>>

#+begin_src bash -i
git config –global merge.conflictstyle diff3
git checkout –conflict=diff3 <file>

Handle merge conflicts systematically with a merge tool (see diff tools from before)

git mergetool <file1> <file2>
git mergetool <branch1> <branch2>
git mergetool <sha> <sha> <file>
#...

Cleanup

git clean -n #check what clean will do
git clean -f #clean

Example:

Move into mergeConflict repo
Set your config to use diff3
View the differences between all three branches
Merge testMerge1 into test2Merge2
View the differences between ours and thiers using diff
Make some changes to test.h
Start conflict mode over
Abort the merge
Merge testMerge2 into testMerge1
Pipe the different versions into files for viewing side by side
Abort the merge
Merge theirs
Cleanup

Solution:

git branch mergeTests
git config --global merge.conflictstyle diff3
git diff testMerge1 testMerge2 testMerge3
git difftool testMerge1 testMerge2
git checkout testMerge2
git merge testMerge1
git diff --ours
git diff --theirs
echo "this" >> test.h
git checkout -m -- .*
git merge --abort
get checkout testMerge1
git merge testMerge2
git show :2:test.h > testOURS.h
git show :3:test.h > testTHEIRS.h
git merge --abort
git merge -s theirs
git clean -n
git clean -f

Cleanup:

Set back to normal:

git config --global merge.conflictstyle merge

Rerere

Record how you applied conflict resolutions before and automatically apply them
Simply enable:

git config --global rerere.enabled true
git config --global rerere.autoupdate true

Handle merge conflicts the same way as you did before
Recorded preimage for ...
Recorded resolution for ...
Resolved ... using previous resolution.

Review: sane defaults

git config –list

git config --list

git config --global \
    merge.conflictstyle diff3 (shows common ancestor too) \
    merge.ff false \
    merge.tool <meld,p4merge,ediff(emacs,smerge-mode)> \
    diff.tool <...> \
    core.fileMode=false \
    pull.rebase true \
    rerere.enabled true \
    rerere.autoupdate true

Conventions!

Other than a clean workflow, proper conventions and communication will solve most potential conflicts

Some conventions we went over

Commit often
One feature per branch

…

Conventions to consider

Whitespace, comments, text format (UTF-8)
When to merge with master?
Who should be working on what file?
What is the scope of your feature?
When should you communicate with your team?
How should code be organized when you create/append?
Who is allowed to publish to the master repository?
- Do you have pull requests?
- Who handles pull requests?
What continuous branches does your lab have?
When should you create a new repository
How and when does code get tagged?

Other

Bisect

Regression debugger's best friend

git checkout <commit that you know has regression>
git bisect start
git bisect bad
git bisect good <commit that you know doesn't have regression>

Git will automatically checkout a commit and expect you to label it as good:

git bisect good

or bad

git bisect bad

Git will automatically checkout another commit.
This process continues until it reaches the point where the regression was found

Automation can be done with

git bisect run ./somescript

When you are done:

git bisect reset

pastebin
sync/backup your workspace with dropbox/nextcloud

Preview

…of some things in future advanced class

Large files

Terrible idea to track large binary files (can't open in a text editor) in the default way

Meta data is constantly changing
Looking at different commits in these files is nonsense
Will make repo big and slow

Solution git-lfs

git-lfs track <*.filetype>
git add <largefile.filetype>

Git commit stuff like usual
Uses pointers, does not upload your file
Version control your data and figures!

Details

If you push your lfs to a remote and somebody clones your repo:
The data files will NOT be there.
They would need to get the files from your local machine.
You need to set:

git config lfs.url = "https://my_local_machine.com/foo/bar/info/lfs"

And this would also mean setting up your local machine to point to the www in a secure manner.
With this in mind, its wise to have a lab server that can

Store the data
Act as origin for your data
Provide web services

If this is the case, you do not need to set the above configuration.

Hooks

.git/hooks
exmples located there

pre-commit
post-commit
post-checkout
pre-rebase
…
Essentially before or after any step

Shared Templates

git config --global init.templateDir <directory>
git init

Dependencies

Subtrees - If you don't plan on editing dependencies
Very straightforward to use, not very flexible

Submodules - If you don't plan on editing dependencies
Very flexible, requires very careful usage

Setup

A case for terminal

Overview

Improving workflow

Bad commit practices

Solution

Details

Creating a branch

Commit

COMMIT BODY

INTERACTIVE ADD

Reset

Details

Undo a reset

Stash

Revert

Details

Example:

Solution:

History parsing

Log

View commits in a tree

Bash alias

Basics

Limiting

Specific details

Other help

Checkout

Detached head

Example:

Solution:

Branching & Merging

1. Branching discussion

Branch/Merge fear

Problems clean branching helps with

Workflow

Smallish change for many

2. Basic Merging

Remotes basics

Pushing, pulling, merging

Example:

Ways of Merging

Two main ways of merging

Defaults

Also

Example:

Solution:

Merge vs rebase

Pull –rebase

Pull –rebase note

Example:

Solution:

Deleting a branch

Restore a branch

Example:

Solution:

3. Tag/Notes

Tags

Note on tags/branches

Notes

Details

Example:

Solution:

Different branch models

Merge conflicts

Overview/Flow

Fetch & Diff

diff Notation

Log Notation

Fetch & Diff

Diff tool setup and use

Blame

Example:

Solution:

Cherry pick

Example:

Solution:

Initial merging

Strategy

Strategy option