Intermediate Git

Setup

git clone https://github.com/portalgun/hellogitworld
git clone https://github.com/portalgun/mergeConflict

A case for terminal

Most of learning git is just figuring out what you can do
Many things difficult or impossible in GUI
GUI not always available on compute servers
Some things are easier with a client

Overview

  1. Improve basic workflow
  2. Parse and navigate the history
  3. Branching & Merging
  4. Other misc stuff

Improving workflow

Bad commit practices

  • Non descriptive commit messages ("pushed lots o' stuff")
  • Large commits
  • Commit -a (unless responsibly)
  • Pushing to origin with every commit

Solution

  • Practice using git at each working feature (function/class scale)


  • Workspace (untracked/unstaged)
  • Index (add/stage)
  • History (commit)

Details

  • Git should be a continual flow


  • Branch - begin working on more than a couple lines of code
  • Add - begin testing, cleaning up
  • Commit - tested
  • Push - share/backup (no neeed to do this so frequently, commit frequently instead)

Creating a branch

git branch <branch-name>
git checkout -b <branch-name>

Commit

Some things you maybe didn't know…

COMMIT BODY

line 1 - title
line 2 - blank
line 3 - longer description

OR

git commit -m "title" -m "description"

INTERACTIVE ADD

git add -i <larger>
git add -p <smaller>

Commit A Single File With Staged Files

git commit <file>
git reset <file>

Small demo…

Reset

Note that these will not affect a push; reset applies to just 1 branch.

I forgot to test my last commit:

git reset --soft

I need to add/fix something to my last commit:

git reset --mixed

My last commit is utter garbage and I want to trash it completely:

git reset --hard

Details

Note that there is a difference between

git reset --soft

and

git reset --soft HEAD~1

Without specifying a commit (latter), the reset only applies to your index and workspace.
To undo a commit, you need to explicitly specify which commit you are referring to.

Undo a reset

git reset HEAD@{n}

replace n for how many times you did the git reset command
If I did 'git reset HEAD~1' or 'git reset HEAD~20', then n is 1
If you did 'git reset HEAD~1; git reset HEAD~1', then n is 2

Stash

git stash save [<message>]
git stash list
git stash show
git stash apply stash@{n} #n is a number
git stash drop stash@{n}
git stash -u
git stash pop stash@{n} #drop and apply

Revert

git revert <ref>

Like reset, but appropriate for your origin
It doesn't undo the commit, but creates a new commit that is identical to the commit you want to 'reset to'.

Details

How would we undo two commits, but retain changes

git reset --soft HEAD~2

This also works on individual files

Other notation - next branch point:

HEAD^

Example:

  1. Create a branch called "cleanup"
  2. Add "Forked by <your name>" to README.txt
  3. Add this change to index
  4. Stash
  5. Add "Created by Matthew McCullough" to README.txt
  6. Commit with a message and a body
  7. Undo commit and move them to workspace only
  8. Stash these changes
  9. Apply first stash
  10. Commit with a message
  11. Apply second stash
  12. Commit with a messsage and a body
  13. Remove all your stashes

Solution:

git checkout -b cleanup
echo "\nForked by Dave" >> README.txt
git add -u README.txt
git stash save Dave

#MAKE SURE THINGS ARE GOOD
git status
cat README.txt

echo "\nCreated by Matthew McCullough" >> README.txt
git commit -a -m "Signed Matthew McCullough" -m "What a great dude"
git reset --mixed HEAD~1 #HEAD~1 must be included
git stash save -u Matt #-u must come right after save
git stash list
git stash apply stash@{1}
git stash drop stash@{1}
git add -u README.txt
git commit -m  "Added name Dave"
git stash pop stash@{0}

#This last line will create a conflict, which we will talk about later
#Fix the conflict by removing lines with:
#<<<<<<<
#=======
#>>>>>>>
#You can do this manually or with a stream editor:
sed -i -r '/^<{3,}/d; /^>{3,}/d; /^={3,}/d' README.txt

cat README.txt

git commit -a -m "Signed Matthew McCullough" -m "What a great dude!"

History parsing

Log

View commits in a tree

git log --graph --oneline --decorate --all

Bash alias

Git was meant to be scripted/aliased

#$HOME/.bashrc in linux
#$HOME/.bash_profile in mac
alias glg="git log --graph --oneline --decorate --all"

Basics

git log <tag, hash, branch,remote>
git log RELEASE_1.1
git log 8d2636d
git log origin/master
git log origin/feature_subtraction_polished

Limiting

git log -3
git log --since="5 years ago"
git log --until="8 years ago"
git log --author="Peter"

Specific details

git log -L <startrange>,<endrange>:<file>
    git log -L 1,2:pom.xml
    git log -L 1,3:pom.xml
git log -L :funcname:file
    git log -L :repositories:build.gradle
    git log -L :repositories:build.gradle -L :dependencies:build.gradle
git log -S''
    git log -S'maven'
git log -G''
    git log -G'dep.+'
git log --grep'Search messages'
    git log --grep='script'

Combine as many options as you want

Other help

Shell tools

git log | less
git log | grep -ei "some string"

Gitk, Gitkraken, Magit, etc..

Checkout

How to move around.
Try this:

git checkout master
git status

Detached head

Try this:

git checkout RELEASE_1.0
git status

Detatched - not on newest commit

Any commits without explicitly branching will not be attached to the tree

Create a new branch off the checked out branch if you are making a commit to an older commit

Example:

  1. Find where sum function was added (Trick question)
  2. Navigate to where that was committed (also tricky)
  3. Navigate to the next branch point from there
  4. create a branch called "testbranch"
  5. Create a new file called TRASH
  6. Commit the file
  7. Navigate to master
  8. Find the commit associated with testbranch

Solution:

grep -Ri "sum"
git log -L :sum:src/Sum.groovy
git checkout 333fd9
git checkout HEAD^
git checkout -b "testbranch"
touch TRASH
git add TRASH
git commit -a -m "created trash"
git checkout master
git log testbranch

Branching & Merging

1. Branching discussion

Branch/Merge fear

People don't like branching because it means merging

However:

  • You cannot break anything
  • You are already creating a branch for every time you push/pull
  • Merging all at once can be better than multiple doing it multiple times (example)
  • There is an easy way to update your branch with master if needed

Problems clean branching helps with

Simply, branches allow for

  • Organizing code by stability
  • Organize code by individual and/or feature

Workflow

Think of commits as a sections in a textbook.
A branch is going to be chapters in your textbook
You can work on multiple chapters of a book at a time.

1 feature per branch, rule of thumb

Smallish change for many

Switch your usual large commits, with a merged branch.
Actually will make things easier to merge

2. Basic Merging

Remotes basics

Master & origin/master are different branches
Feature1 & feature1/master are different branches
Push/pull is just fetch + merge

Pushing, pulling, merging

Same thing in practice, but different vantage point
If you are not a manager you would need to do a pull request

How to pull request

  1. Push your branch
  2. Submit a pull request
  3. Wait for me to merge it
Example:

Send me a pull request to merge "cleanup" with master

Ways of Merging

Merging is always towards the head
Think of master as a giant git beast

Two main ways of merging
  • FF - no separate 'branch'
  • No-ff - no merge commit


Default:

  • If no divergence, ff
  • If divergence, no-ff
Defaults

Why set your default to no-ff? keeps things separated by feature/user

git config --global --add merge.ff false
git config --add merge.ff false
Also

Only merge if nothing new at merge point

git merge ff-only
Example:
  1. create a backup of your repo
  2. merge "cleanup" with master, no-ff
  3. undo your merge
  4. merge "cleanup", ff
  5. try to merge tesbranch with –ff-only
Solution:
cp ../hellogitworld ../hellgitworld_Backup #linux
cp -a ../hellogitworld ../hellgitworld_Backup #mac
git checkout master
git merge cleanup --no-ff
git reset --hard HEAD~1
git merge --ff-only

Merge vs rebase

Rebase is ugly
Few cases when you should use it

Pull –rebase

Makes things easier, eliminates many merge issues as you go along

git config --global pull.rebase true

When you push, it will look like…

Pull –rebase note

You do NOT need your repo to look exactly like remote
Nor should you try to do this, or expect it.
What matters is that both are clear
In a merge it doesn't matter where code came from
Small differences in master, larger differences between your branches

Example:

Pull master into testbranch

Solution:

checkout testbranch
git pull –rebase master

Deleting a branch

git branch -D <branch>
  • Doesn't delete the commits and their structure, but opens up the namespace
  • May be useful to add a note here (notes later)
Restore a branch
git checkout -b <branch> <sha>
Example:
  1. Delete testbranch
  2. Look at where test branch was
  3. restore test branch
Solution:
git branch -D testbranch
git checkout -b testbranch <sha>

3. Tag/Notes

Tags

git tag <tag-name>
git push --follow-tags
git push <remote> <tag-name> --force #if moved
git tag -a <tag-name> -m <message>
git checkout <tag-name>
git tag -d <tag-name> #local
git push --delete origin <tag-name> #remote

Note on tags/branches

When a ref is created, it needs to be push - this includes new branches and tags
Everything point to point (one branch at a time), unless required or specified
eg. commits have dependencies

Explicitly need to push:

  • Deleted branches
  • Tags
  • New branches unless included with a commit

Branch example

Notes

  • Reserve tags for versions numbers, notes can be human readable version
  • Notes are for additional detail attached to commits
  • Do not affect the history, hash number, commit message etc.
  • You can edit at any time
  • Does not clutter namespace

Details

git notes add -m "message"
git log -p notes/commits
git push <remote> refs/notes/*
git fetch origin refs/notes/*:refs/notes/*
git notes --help

Example:

  1. Tag commit 435ffce as '\(\text{RELEASE}\_ 1.1.1\)'
  2. Add a note to RELEASE 1.1

Solution:

git checkout 435ffce
git tag RELEASE_1.1.1
git checkout RELEASE_1.1
git notes add -m "Missing log files"

Different branch models

Industry standard:

  • 1. hotfix
  • 2. master
  • 3. dev
  • 4. <Features>

Everything gets pulled top to bottom

New project/single individual/small project

  • 1. master
  • 2. <Features>


Why make MASTER stable only? Default

Merge conflicts

Overview/Flow

  1. Fetch
  2. Diff
  3. Blame
  4. Cherry-pick/merge parts
  5. Merge
  6. Branch delete

Fetch & Diff

Fetch, then diff

#commit or stash then...
git fetch
git log ..<branch> #list changes that will be merged
git diff ..<branch>
git difftool ..<branch>

diff Notation

difference between a & b

a..b
..b

difference between common ancestor and b

a...b
...b

Log Notation

everything that is in b but not in a

a..b

only in one or other

a...b

Fetch & Diff

Many different diff/merge conflict tools

  • meld - popular
  • kdiff3 - popular
  • vimdiff - vim users

Many graphical git wrappers have their own

Diff tool setup and use

git config --global merge.tool <merge-tool>
git config --global diff.tool <merge-tool>
#git config --global diff.external <merge-tool>

git difftool <COMMIT_HASH> file_name
git difftool <BRANCH_NAME> file_name
git difftool <COMMIT_HASH_1> <COMMIT_HASH_2> file_name

git difftool RELEASE_1.1 RELEASE_1.0
git difftool RELEASE_1.1 RELEASE_1.0 runme.sh

Blame

git blame <file>
git blame -e <file>
git blame -L <start>,<end><file>
git blame -L :function <file>
git blame -L ... -M <file>

Who do I talk to, yell at?
Merges should often be a group effort

Example:

get the email of whoever added line three to runme.sh

Solution:

git blame -e -L 3,3 runme.sh

Cherry pick

git cherry-pick (commits in order to append, first to last)

Example:

  1. Create a branch 'hotfix' off of Release 1.1
  2. Cherry-pick commits from cleanup
  3. Tag as \(\text{RElEASE}\_1.1\text{h}\)

Solution:

git checkout Release_1.1
git checkout -b hotfix
git cherry pick <SHA> <SHA>
git tag Release_1.1h1

Initial merging

Complete failure to merge - updated index, not pushed

git stash
git reset etc..

Strategy

merge -s <strategy>
  • Recursive (default)
  • Patience - slower, more effective recursive
  • Resolve - pick one of the possible merge bases, and hope for the best.
  • Ours - only use our changes, ignore theirs (but keep history)
  • Theirs - only use their changes, ignore ours (but keep history)
  • Octopus - multiple head default
  • Subtree - (subtree)

Strategy option

merge -s <strategy> --strategy-option <option>

Don't confuse with -s

  • Ours - take ours if there is a conflict
  • Theirs - take theirs if there is a conflict

Conflict mode

git log --merge

Diff

git diff --ours
git diff --theirs
git diff --base

Show

git show :1:<filename> #common ancestor
git show :2:<filename> #your versions
git show :3:<filename> #their version
git clean -f #remove any extra files used for merge resolution

These can be piped.

checkout --theirs <filename>

Reset or start over

git merge --abort  #RESET
git checkout -m -- <filename> #START MERGE CONFLICT OVER FOR A FILE
git checkout -m -- .* #for everything

conflict markers

conflictstyle diff3 (shows common ancestor too)

<<<
Towards you, you are always first (left)
|||
COMMON ANCESTOR
========
THEIRS
>>>


git config --global merge.conflictstyle diff3
git checkout --conflict=diff3 <file>


Handle merge conflicts systematically with a merge tool (see diff tools from before)

git mergetool <file1> <file2>
git mergetool <branch1> <branch2>
git mergetool <sha> <sha> <file>
#...

Cleanup

git clean -n #check what clean will do
git clean -f #clean

Example:

  1. Move into mergeConflict repo
  2. Set your config to use diff3
  3. View the differences between all three branches
  4. Merge testMerge1 into test2Merge2
  5. View the differences between ours and thiers using diff
  6. Make some changes to test.h
  7. Start conflict mode over
  8. Abort the merge
  9. Merge testMerge2 into testMerge1
  10. Pipe the different versions into files for viewing side by side
  11. Abort the merge
  12. Merge theirs
  13. Cleanup

Solution:

git branch mergeTests
git config --global merge.conflictstyle diff3
git diff testMerge1 testMerge2 testMerge3
git difftool testMerge1 testMerge2
git checkout testMerge2
git merge testMerge1
git diff --ours
git diff --theirs
echo "this" >> test.h
git checkout -m -- .*
git merge --abort
get checkout testMerge1
git merge testMerge2
git show :2:test.h > testOURS.h
git show :3:test.h > testTHEIRS.h
git merge --abort
git merge -s theirs
git clean -n
git clean -f

Cleanup:

Set back to normal:

git config --global merge.conflictstyle merge

Rerere

Record how you applied conflict resolutions before and automatically apply them
Simply enable:

git config --global rerere.enabled true
git config --global rerere.autoupdate true
Handle merge conflicts the same way as you did before
Recorded preimage for ...
Recorded resolution for ...
Resolved ... using previous resolution.

Review: sane defaults

git config –list

git config --list

git config --global \
    merge.conflictstyle diff3 (shows common ancestor too) \
    merge.ff false \
    merge.tool <meld,p4merge,ediff(emacs,smerge-mode)> \
    diff.tool <...> \
    core.fileMode=false \
    pull.rebase true \
    rerere.enabled true \
    rerere.autoupdate true

Conventions!

Other than a clean workflow, proper conventions and communication will solve most potential conflicts

Some conventions we went over

  • Commit often
  • One feature per branch


Conventions to consider

  • Whitespace, comments, text format (UTF-8)
  • When to merge with master?
  • Who should be working on what file?
  • What is the scope of your feature?
  • When should you communicate with your team?
  • How should code be organized when you create/append?
  • Who is allowed to publish to the master repository?
    • Do you have pull requests?
    • Who handles pull requests?
  • What continuous branches does your lab have?
  • When should you create a new repository
  • How and when does code get tagged?

Other

Bisect

Regression debugger's best friend

git checkout <commit that you know has regression>
git bisect start
git bisect bad
git bisect good <commit that you know doesn't have regression>

Git will automatically checkout a commit and expect you to label it as good:

git bisect good

or bad

git bisect bad

Git will automatically checkout another commit.
This process continues until it reaches the point where the regression was found

Automation can be done with

git bisect run ./somescript

When you are done:

git bisect reset

Sharing incomplete chunks

pastebin
sync/backup your workspace with dropbox/nextcloud

Preview

…of some things in future advanced class

Large files

Terrible idea to track large binary files (can't open in a text editor) in the default way

  • Meta data is constantly changing
  • Looking at different commits in these files is nonsense
  • Will make repo big and slow

Solution git-lfs

git-lfs track <*.filetype>
git add <largefile.filetype>
  • Git commit stuff like usual
  • Uses pointers, does not upload your file
  • Version control your data and figures!

Details

If you push your lfs to a remote and somebody clones your repo:
The data files will NOT be there.
They would need to get the files from your local machine.
You need to set:

git config lfs.url = "https://my_local_machine.com/foo/bar/info/lfs"

And this would also mean setting up your local machine to point to the www in a secure manner.
With this in mind, its wise to have a lab server that can

  1. Store the data
  2. Act as origin for your data
  3. Provide web services

If this is the case, you do not need to set the above configuration.

Hooks

.git/hooks
exmples located there

  • pre-commit
  • post-commit
  • post-checkout
  • pre-rebase

  • Essentially before or after any step

Shared Templates

git config --global init.templateDir <directory>
git init

Dependencies

Subtrees - If you don't plan on editing dependencies
Very straightforward to use, not very flexible

Submodules - If you don't plan on editing dependencies
Very flexible, requires very careful usage

End

Date: 2019-05-8

Author: Dave White

Created: 2019-05-30 Thu 11:58

Validate