Git

Git Community Book
Git Documentation & Pro Git book
Github & Git Foundations Training from youtube.
git/github guide a minimal tutorial from Karl Broman
Some examples of projects hosted in github: RHadloop, shiny, Qt.
http://www.atlassian.com/git/tutorial/undoing-changes
Git reference (5 pages of summary)
Distributed Version Control with Git (nice tutorial including Github, BitBucket. )
Cheat sheet:
- github.com (2 pages)
- byte.kde.org (font too small)
- git-tower.com (1 page as an image)
- atlassian.com (immediate download)
- jan-krueger.net (cool too)
- patrickzahnd.ch (visual cheat sheet)
https://www.kernel.org/pub/software/scm/git/docs/everyday.html
http://haacked.com/archive/2012/05/21/introducing-github-for-windows.aspx
Delete a repository

git config

This has been run the very first time you use git commit.

git commit (commit to the local repository) will ask username/password if we have not logged in (that is, this step has been done before we run git commit). The identity will be saved under ~/.gitconfig file.

git config --global user.name "Your Name Here"
git config --global user.email [email protected]

OR to have a project specific git identity (this will write your identify in ./.git/config file)

git config user.name "Your Name Here"
git config user.email [email protected]

Type

git config --list

to confirm the configuration.

git stage area

If you have run git add, git rm or git mv command, you put some files in the stage area. In other words git knows about the change, but it is not permanent in the repository. The next commit will include the changes staged.

Should you decide not to commit the change, the status command will remind you that you can use the git reset command to unstage these changes.

For example, if your IDE accidentally deletes a file for you, you can run git reset HEAD FILENAME and git checkout FILENAME to recover the file.

git push, git clone and remote repository

git push (push to a remote repository) will ask the username and/or password. To avoid that

git remote set-url origin https://name:[email protected]/name/repo.git
git remote set-url origin https://[email protected]/name/repo.git # skip password for security
git remote -v

OR when you run git clone, use the format

git clone https://[email protected]/name/repos.git
git remote -v

Either way, we can check the the remote repository by using git config --list or cat .git/config.

When I commit (git commit) a directory containing resource file, it shows warning: LF will be replaced by CRLF in xxxx.qrc.
Password cache
github tutorial (for statistician)
git add -A

git config --global credential.helper cache

Note that if the local repository is new, it won't work to use git remote set-url. You need to use git remote add origin, such as

git remote add origin [email protected]:USERNAME/REPOSITORY.git

git clone when no 'master' branch exists

Warning: remote HEAD refers to nonexistent ref, unable to checkout

In my case, I need to run

git clone ssh://[email protected]/home/git/seqtoolsweb.git --branch gh-pages --single-branch seqtoolsweb

git fork

https://help.github.com/articles/fork-a-repo

First example

My example of working on a new repository called 'network'.

Follow https://help.github.com/articles/create-a-repo to create a new repository. For convenience, I also check the button to create README file.
Click 'GitHub' icon on Windows Desktop. Look at the LHS and click on the word 'github'. Click 'clone' button (This can be accomplished by git clone https://github.com/arraytools/network.git). The new repository will appear under C:\Users\USERNAME\Documents\Github\ directory. Now Click 'Git Shell' icon on the Windows Desktop and go to C:\Users\USERNAME\Documents\Github\network directory where 'network' is my repository's name.

git config --global color.ui auto  # colorize the output of git
git init

git add client.c
git add server.c
git add server2.c
git commit -m 'first commit'

git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.

# git remote add origin https://github.com/arraytools/network.git
# git push <remote> <branch>
git push origin master
# It seems 'git push' is the only chance we use the format 'origin master' instead of 'origin/master'.

# If we modify any file, we need to go through git add, git commit and git push 3 commands.

# get commit_id
git log
# get a specific version
git checkout commit_id

# after an examination, we want to get the latest version
git pull origin master
# If you do not want to merge the upstream changes wit your local repository, run git fetch to 
# fetch the changes and then git merge to merge the changes. 
# git pull is just a combination of fetch and merge.

# To rename a file
git mv originafile newfile
git commit -m "rename orginalfile"
git push

# To remove a file
rm myfile
git add . -A
git commit -m "remove a file"
git push

# Revert to origin's master branch's version of file
# http://stackoverflow.com/questions/1817766/revert-to-origins-master-branchs-version-of-file
# 1. Assuming you did not commit the file, or add it to the index, then:
git checkout filename
# 2. Assuming you added it to the index, but did not commit it, then:
git reset HEAD filename
git checkout filename

Check the https://github.com/arraytools/network. The commit goes to the repository!

In summary: add and commit are local operations, push, pull and fetch are operations that interact with a remote.

If we want to checkout a specific commit on a new computer, we can use (here we use Qt repository as an example)

git clone https://xxxxxx/xxxxxx/Qt.git
cd Qt       
git log --oneline
git checkout SHA1  (7 digits obtained from git log --oneline commandis enough)

After that we can run

# move HEAD to origin
git checkout origin/master

# Visualize using text mode
git log --graph --oneline --date-order --decorate --color --all

Using GUI client: gitg

sudo apt-get install gitg

To switch among different branch (eg. github project page is located in gh-pages branch of a repository),

git checkout gh-pages
git checkout master

GUI version of Git software

Windows

Go to http://git-scm.com/download. The Windows version contain 'Git Bash', 'Git CMD', and 'Git GUI'. The 'Git GUI' software (based on Tcl/Tk) works pretty cool. It can 'Rescan' the project, compare the changed filefs and visualize the master's history too. The Git comes with 2 built-in tools: Git-gui is for committing and gitk is for browsing. Screenshots of gitk can be found below.

Note that Git-gui cannot run git pull directly. We have to go through two steps: Fetch and merge. See this message.

When installing Git for Windows, there's a tricky option you need to be careful with. You can configure Git to change line endings on text files from Linux (LF only) to/from Windows-DOS (CR and LF).

Linux

Use gitk or gitg. For example,

gitk --all

If we run the gitk/gitg in background (gitg --all &), we will see the tree changed in real time when we run git commands in a terminal.

Gitk colors and bold

The missing gitk documentation

local branch names are in a green background
remote branch names are in a mixed orange/green background
the currently checked out branch name is in bold
a yellow dot marks the current HEAD (it seems the yellow dot and bold branch name are always together)
tags are on a yellow background

See a screenshot at Branch - Pull.

Set up a new local/remote repository

mkdir /path/to/your/project
cd /path/to/your/project
git init
git remote add origin https://[email protected]/arraytools/REPOSITORYNAME.git
# git remote add origin ssh://[email protected]/home/git/REPOSITORYNAME.git
# git clone             ssh://[email protected]/home/git/REPOSITORYNAME.git

git config --global user.name "YOUR NAME"
git config --global user.email "YOUR EMAIL ADDRESS"
git config --list # confirm

echo "arraytools" >> contributors.txt
git add contributors.txt
git commit -m 'Initial commit with contributors'
git push -u origin master

ssh key

See Linux > Multiple ssh keys.

Github
Bitbucket. Remember to change the remote repository from https protocol to git protocol on local repository.

If you add a new key to your bitbucket account, you will receive an email about the change.

Already has a git repository on my computer

cd /path/to/my/repo
git remote add origin https://[email protected]/arraytools/REPOSITORYNAME.git
git push -u origin --all # pushes up the repo and its refs for the first time
git push -u origin --tags # pushes up any tags

'master' and 'origin'

master is a branch name. You can use git branch to find out all branches. The current branch has a asterisk in the command line output and has a bold font in the gitk program.
origin is a repository name. You are free to create a new one and delete origin especially in situation that you are working with multiple remotes.

https://lostechies.com/joshuaflanagan/2010/09/03/use-gitk-to-understand-git/

'origin master' vs 'origin/master' format

origin master format: git push, git pull, git fetch where the branch name is optional.
origin/master format: git checkout, git merge, git diff, git log, git reset.

See stackoverflow.

Multiple remotes

Suppose I have a remote at github.com. I add another remote (bitbucket.com) in my current project.

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
$ git push -u bitbucket master
$ git status
On branch master
Your branch is up-to-date with 'bitbucket/master'.

$ git branch -a
* master
  remotes/bitbucket/master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master

$ git checkout origin/master             # seems not what I want to do
Note: checking out 'origin/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 980fd0b... test fetch +1
$ git status
HEAD detached at origin/master

$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'bitbucket/master'.
$ git status
On branch master
Your branch is up-to-date with 'bitbucket/master'.

nothing to commit, working directory clean

$ nano linuxfile
$ git commit -am "multiple remote +1"
$ git push origin master
$ git status
On branch master
Your branch is ahead of 'bitbucket/master' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working directory clean

$ # Question: How to switch back to origin repository to sync?
$ #           How do I change the remote a git branch is tracking?
$ # Method 1: "-u" in git push; see next section about "-u"

$ git push -u origin master
Branch master set up to track remote branch master from origin.
Everything up-to-date
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

$ # Method 2: "-u" in git branch; see next section about "-u"
$ git branch master -u bitbucket/master
Branch master set up to track remote branch master from bitbucket.
$ git status
On branch master
Your branch is ahead of 'bitbucket/master' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working directory clean

How to know which remote's master branch to pull/push

$ git status
# On branch master
# Your branch is up-to-date with 'bitbucket/master'

-u option

The key is "argument-less git-pull". When you do a git pull from a branch, without specifying a source remote or branch, git looks at the branch.<name>.merge setting to know where to pull from. git push -u sets this information for the upstreaming branch you're pushing.

“git push -u origin master” vs “git push origin master”

What does the '-u' do in git push

git branch

How do I change the remote a git branch is tracking?

The git branch command can be used to connect a local branch and a remote branch.

$ git status
# suppose the local master is connected to bitbucket/master

$ git branch master -u origin/master
# now the local master is connected to origin/master
# the local branch name and the remote branch name can be different if you like!
$ git status

Note that gitk can not see the difference when we change the connection of a local branch and a remote branch. Only the git status can reveal the connection.

Example

Multiple remotes

Setup editor

See man git-commit.

git config --global core.editor "nano"
# OR
export GIT_EDITOR=nano
# OR for other programs to use too
export EDITOR=nano

After that

$ cat ~/.gitconfig
[core]
        editor = nano

.gitignore file

http://stackoverflow.com/questions/8527597/how-do-i-ignore-files-in-a-directory-in-git
Microsoft Office temporary files. It works if we haven't made a mistake to run commit.

git help gitignore
# or
man gitignore

A leading slash indicates that the ignore entry is only to be valid with respect to the directory in which the .gitignore file resides. Specifying *.o would ignore all .o files in this directory and all subdirs, while /*.o would just ignore them in that dir, while again, /foo/*.o would only ignore them in /foo/*.o.

git status, multiple branches

Your branch is up-to-date with 'origin/master'

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

http://stackoverflow.com/questions/27828404/why-does-git-status-show-branch-is-up-to-date-when-changes-exist-upstream

When git status says up to date, it means "up to date with the upstream status that was retrieved last time we did a fetch" which is not the same as "up to date with the latest live status of the upstream".

nothing to commit, working directory clean

If we have a new branch on local which does not exist on remote, it is useful to use the git remote show REMOTENAME command to find out what branches are tracked.

$ git status
On branch newbranch
nothing to commit, working directory clean

$ git remote show bitbucket
* remote bitbucket
  Fetch URL: ssh://[email protected]/XXXXXX/toy.git
  Push  URL: ssh://[email protected]/XXXXXX/toy.git
  HEAD branch: master
  Remote branch:
    master tracked
  Local ref configured for 'git push':
    master pushes to master (up to date)
$ git diff bitbucket/master
diff --git a/linuxfile b/linuxfile
index 894674f..b9f3a6f 100644
--- a/linuxfile
+++ b/linuxfile
@@ -1,6 +1,6 @@
 test ssh key (mod in mint again)
 changed from 'newbranch'
-another line 
+mod from newbranch 
$ git push bitbucket master
Everything up-to-date

Note that

The last command is NOT working since only the local/master can be pushed to remote/master. Local/newbranch cannot be pushed to remote/master.
The local 'master' branch can be pushed to either remote1/master or remote2/master.

Remove untracked files

Go to the top directory of the local repository (or it won't work), then run

$ git clean -f -n  # dry run
$ git clean -f     # real run

See the discussion.

Remove deleted files

git ls-files --deleted
git rm `git ls-files --deleted`

Undo

https://git-scm.com/book/en/v2/Git-Basics-Undoing-Things

git commit --amend

If you commit too early and possibly forget to add some files, or you mess up your commit message.

Assumption: Commit has not been pushed online

$ git commit -m 'initial commit'
$ git add forgotten_file
$ git commit --amend
# In your text editor, edit the commit message and save the commit.

You end up with a single commit.

If the commit has been pushed online, see https://help.github.com/articles/changing-a-commit-message/.

Unmodifying a Modified File (not staged)

$ git checkout -- Filename  # no output
$ git checkout -- .         # All unstaged files
$ git stash save --keep-index 
# http://stackoverflow.com/questions/52704/how-do-you-discard-unstaged-changes-in-git/52713#52713

Unstaging a Staged File

$ git reset HEAD Filename

Undo git pull or merge

http://stackoverflow.com/questions/101752/i-ran-into-a-merge-conflict-how-can-i-abort-the-merge

$ git reset --hard HEAD

Undo git commit

http://stackoverflow.com/questions/927358/how-do-you-undo-the-last-commit

$ git reset --soft HEAD~

Undo git push

http://stackoverflow.com/questions/1270514/undoing-a-git-push

git checkout master
git reset --hard cc4b63bebb6
git push origin +master

change commit message

https://help.github.com/articles/changing-a-commit-message/#commit-has-not-been-pushed-online

To change the last committed message

git commit --amend # Enter and save
git push --force origin master

Remove a file from a Git repository without deleting it from the local filesystem

git rm --cached mylogfile.log   # file
git rm --cached -r mydirectory  # directory

Git reset revert rebase commands

How to reset, revert, and return to previous states in Git

git fetch vs git pull

search Briefly

git fetch origin master

git merge origin/master

# git push origin master

git fetch and gikt

After we run git fetch origin master, we will see the remote commits from gitk if we add --all option.

gitk --all
# OR
gitg --all

branch - not the same as in CVS

Gitk shows all of the commits as a single straight line. In git, a branch is a label for a commit. The label moves to new commits as they are created. When you create a git branch, you are not changing anything in the structure of the repository or the source tree. You are just creating a new label.

Below are some info borrowed from Pro Git.

master branch

When you clone a repository, it generally automatically creates a master branch that tracks origin/master.

To set up other tracking branches, see Remote branches.

git checkout --track origin/serverfix

Create a new branch

git branch testing    # do not switch
git log --oneline --decorate

Pay attention to the keywords 'HEAD', 'master' and 'testing' in this case.

To create a new branch and switch to the new branch

git status
git checkout -b myBranch
git status

Delete a branch locally or from the remote

Delete a local copy of a branch.

# Since you can't delete a branch you're on, switch to the master branch. 
git branch
git checkout master
git branch -d myBranch

To delete the branch from the remote:

git push origin :myBranch

# OR
git push -d origin myBranch

Switch branches in local repository

git checkout testing
git branch
nano test.rb
git commit -a -m 'made a change'
git checkout master
nano test.rb
git commit -a -m 'made other changes'
git log --oneline --decorate --graph --all

Note that when you run git pull without specify the branch name (eg gh-pages from the current local branch), it won't pull the latest from another branch (eg master).

Basic merging and conflicts

git checkout -b iss53 
# shorthand for 
# git branch iss53
# git checkout iss53

nano index.html
git commit -a -m 'added a new footer [issue 53]'
git checkout master

git checkout -b hotfix
nano index.html
git commit -a -m 'fixed the broken email address'

git checkout master
git merge hotfix
git branch -d hotfix # delete the branch

git checkout iss53
nano index.html
git commit -a -m 'finished the new footer [issue 53]'

Basic Merging:

git checkout master
git merge iss53

Basic merge conflicts:

git merge iss53
git status # Look at the standard conflict-resolution markers to the top of files

git mergetool
git status

push the new branch to remote

# make sure we are at the right branch
$ git branch -a 
  master
* newbranch
  remotes/bitbucket/master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
$ git push -u bitbucket newbranch
$ git status
On branch newbranch
Your branch is up-to-date with 'bitbucket/newbranch'.

nothing to commit, working directory clean

If another machine runs git pull, it will get the new branch. The result can be seen by gitk --all.

pull

Suppose we are at the 'newbranch'. Some files are modified and committed to 'newbranch' (these actions are only done locally).

If someone modified files and committed to the 'master' branch, then when we run git pull (keep it in mind that we are still on newbranch) the files we just modified & committed OR even any files aren't affected since git pull is pulling files from the 'master' branch.

If we use the gitk --all or git log --graph --all --oneline --decorate command, we may see (remote has an orange background color, branch has a green background color, a yellow dot marks the current HEAD)

remotes/bitbucket/newbranch
remotes/bitbucket/master
newbranch (local)
master (local)

These 4 branches could be on different nodes. Note. gitk output is easy to read but git log gives the SHA information. The screenshot from git log below does NOT use the --decorate option.

Branch management

git branch
git branch -v          # see the last commit on each branch
git branch -vv         # see the last commit and what remote branch a local branch is tracking?
git branch --merged    # Filter the list to branches that you have merged into the branch you're currently on
git branch --no-merged # See the branches that contain work you haven't yet merged in
git branch -a          # show all remotes' branches too
git branch -d testing

Branch workflows

Inspect a Remote

https://git-scm.com/book/ch2-5.html

brb@brb-P45T-A:~/github/SIK$ git remote
origin
brb@brb-P45T-A:~/github/SIK$ git remote -v
origin	https://[email protected]/arraytools/SIK.git (fetch)
origin	https://[email protected]/arraytools/SIK.git (push)
brb@brb-P45T-A:~/github/SIK$ git remote add pb https://github.com/paulboone/ticgit
brb@brb-P45T-A:~/github/SIK$ git remote -v
origin	https://[email protected]/arraytools/SIK.git (fetch)
origin	https://[email protected]/arraytools/SIK.git (push)
pb	https://github.com/paulboone/ticgit (fetch)
pb	https://github.com/paulboone/ticgit (push)
brb@brb-P45T-A:~/github/SIK$ git fetch [remote-name]
brb@brb-P45T-A:~/github/SIK$ git fetch pb
warning: no common commits
remote: Counting objects: 634, done.
remote: Total 634 (delta 0), reused 0 (delta 0), pack-reused 634
Receiving objects: 100% (634/634), 109.18 KiB | 0 bytes/s, done.
Resolving deltas: 100% (231/231), done.
From https://github.com/paulboone/ticgit
 * [new branch]      master     -> pb/master
 * [new branch]      ticgit     -> pb/ticgit

brb@brb-P45T-A:~/github/SIK$ git remote show origin
* remote origin
  Fetch URL: https://[email protected]/arraytools/SIK.git
  Push  URL: https://[email protected]/arraytools/SIK.git
  HEAD branch: master
  Remote branch:
    master tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)

brb@brb-P45T-A:~/github/SIK$ git remote show pb
* remote pb
  Fetch URL: https://github.com/paulboone/ticgit
  Push  URL: https://github.com/paulboone/ticgit
  HEAD branch: master
  Remote branches:
    master tracked
    ticgit tracked
  Local ref configured for 'git push':
    master pushes to master (local out of date)

brb@brb-P45T-A:~/github/SIK$ git remote rm pb
brb@brb-P45T-A:~/github/SIK$ git remote -v
origin	https://[email protected]/arraytools/SIK.git (fetch)
origin	https://[email protected]/arraytools/SIK.git (push)

So if we use gitg program, we will see there are following branches

origin/master
pb/master
pb/ticgit

brb@brb-P45T-A:~/github/SIK$ git remote rm pb brb@brb-P45T-A:~/github/SIK$ git remote -v origin https://[email protected]/arraytools/SIK.git (fetch) origin https://[email protected]/arraytools/SIK.git (push)

Remote branches, tracking branches

https://git-scm.com/book/ch3-5.html

Remote references are references (pointers) in your remote repositories, including branches, tags, and so on.

Pushing:

git ls-remote (remoteName)
git remote show (remoteName)  # add RSA key fingerprint if the git is using the ssh protocol

git push origin serverfix   # serverfix is a branch name

Do not type your password every time: you can set up a credential cache. The simplest is to keep it in memory for a few minutes, which you can set up by running

git config --global credential.helper cache

One collaborator fetches from the server. They will get a reference to where the server's version of serverfix is under the remote branch origin/serverfix:

git fetch origin
git checkout -b serverfix origin/serverfix

Tracking Branches:

Note: Tracking means that a local branch has its upstream set to a remote branch. Tracking can occur when we use clone or checkout commands.

When you clone a repository, it creates a master branch that tracks origin/master. However you can set up other tracking branches if you wish - ones that track branches on other remotes, or don't track the master branch.

FAQ: git checkout --track origin/branch VS git checkout -b branch origin/branch. Basically '-b' allows a different branch name.

git checkout --track origin/serverfix

# if the branch name you're trying to checkout (a) does not exist and (b) exactly
# matches a name on only one remote, Git will create a tracking branch for you
git checkout serverfix

# set up a local branch with a different name than the remote branch
git checkout -b sf origin/serverfix # your local branch sf will auto pull from origin/serverfix

# if you ALREADY have a local branch and want to set it to a remote branch
# you just pulled down, or want to change the upstream branch you're tracking.
git branch -u origin/serverfix

# see what tracking branches you have set up in your local repo
git branch -vv

Pulling:

git pull   
# Equivalent to two actions
git fetch origin  # get the contents of the remote repository (origin), but keep them under origin/branch branch
                  # requires the password  
git merge origin/master # merge the master branch of the remote repository (origin) with your current branch
                        # no password required

See git fetch and git-merge.

Deleting Remote Branches:

git push origin --delete serverfix

Rebasing - integrate changes from 2 branches

In Git, there are two main ways to integrate changes from one branch into another: the merge and the rebase.

Detached head/HEAD detached from xxxxxxx

I got the above message when I run

Gitgui -> checkout -> Revision Expression (some previous version)
Gitgui -> checkout -> Revision Expression (the latest version)
Git Bash -> git status

At this time, I cannot continue as usual to make changes to files and do commits.

The solution is run

git checkout master

See this post from stackoverflow.com.

git merge

See an example of merging a temporary branch with the master branch in the local repository.

This is another example. It teaches how to make sure my master branch is in synch with the central repository on github (which I refer to using the remote “origin”) before I merge my changes into master.

Merge conflict 1

Your branch is ahead of 'origin/master'

Merge conflict 2

Your branch and 'origin/master' have diverged

Manual merge and auto merge

If a file looks like

line1
line2
line3
line4

(Manual merge) Then suppose machine 1 modifies line3 and push to the remote. Machine 2 at the same time modifies line4 and tries to run git fetch & git merge from the remote. Then machine 2 will need to do merge manually (git mergetool) because line 3 and line 4 are next to each other. After fixing the conflict by git mergetool, we can run git commit & git push. A temporary file called FILENAME.orig will be created.

(Auto merge) On the other hand, if machine modifies line 2 and push to the remote. Machine 2 at the same time modifies line 4 and tries to run git fetch & git merge to the remote. Then auto merge will be done because line 2 and line 4 are separated. Machine 2 only needs to provide the commit message.

Whether there are conflicts or not, the practice is to run

git fetch
git diff master origin/master (- part is from master, + part is from origin/master)
git merge origin/master (possibly need to run nano FILENAME + git add + git commit or run git mergetool + git commit if there is a conflict, OR a text editor will be opened to let you enter the commit message if there is no any conflict). Use git config --global core.editor nano to set the default text editor if this was not done yet; see Git configuration.
git push.

Note that

If we are using git pull instead of git fetch + git merge, we will be directed to the nano editor to enter a commit message if there are no conflicts; however, if we use git fetch, we will NOT be directed to nano editor).

If we directly run git push (even on non-conflict file) without running git pull, we will be welcome with the message

brb@brb-P45T-A:~/github/toy$ git add .
brb@brb-P45T-A:~/github/toy$ git commit -m "add a line from p45t-a"
[master 3b69283] add a line from p45t-a
 1 file changed, 1 insertion(+)
brb@brb-P45T-A:~/github/toy$ git push
To https://[email protected]/arraytools/toy.git
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://[email protected]/arraytools/toy.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
brb@brb-P45T-A:~/github/toy$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working directory clean

This goes back to the same problem mentioned above in Merge conflict 2.

Summary:

When we run git pull, if there is a conflict and that conflict can be resolved automatically, we will be directed to the text editor to enter the commit message.
When we run git fetch, we will NOT be directed to the text editor regardless of conflicts. We need to enter the commit message when we run git merge origin/master. Also gitk will NOT show origin/master on graph BEFORE we run git merge origin/master for some reason.

Take home message:

Before going to run git merge, run git checkout master first.
Instead of running git pull, we'd better run git fetch & git merge origin/master if the bash code was run in a background.

Practices

Suppose we want to experiment on a new plan, we can create a new branch ('newbranch') in our local repository, merge newbranch to master and push to the remote.

git branch newbranch    # step 1 create a new branch called 'newbranch'
git checkout newbranch  # step 2 switch to the 'newbranch' branch
nano FILENAME           # step 3 modify some files
git commit              # step 4 commit changes to newbranch
git checkout master     # step 5 check out the master branch
git branch -a           # (optional) verify we are on the master branch
git merge newbranch     # step 6 merge newbranch to master
git push origin master  # step 7 push (local) master to remote/master

Question: If we have multiple remotes, how do we know if the local master is the same as the origin/master or origin2/master? Answer: git diff.

git rebase

https://lostechies.com/joshuaflanagan/2010/09/03/use-gitk-to-understand-git-merge-and-rebase/

The simple git merge method can make the history very complicated; see and the following cons.

Branching paths in the history can be unnecessarily complicated
The extra merge commit.
Your branch is now no longer a private, local concern. Everyone now knows that you worked in an issue123 branch. Why should they care?

git rebase can be used to avoid these issues.

The good approach (as in that example) is

git checkout issue123
git rebase master
git checkout master
git merge issue123
git branch -d issue123
git push origin master

.ref directory

brb@brb-P45T-A:~/github/toy$ ls -l .git
total 60
drwxrwxr-x  2 brb brb 4096 May  6 11:55 branches
-rw-rw-r--  1 brb brb   13 May  7 16:12 COMMIT_EDITMSG
-rw-rw-r--  1 brb brb  273 May  6 11:55 config
-rw-rw-r--  1 brb brb   73 May  6 11:55 description
-rw-rw-r--  1 brb brb   95 May  7 16:32 FETCH_HEAD
-rw-rw-r--  1 brb brb 3676 May  6 21:44 gitk.cache
-rw-rw-r--  1 brb brb   23 May  6 11:55 HEAD
drwxrwxr-x  2 brb brb 4096 May  6 11:55 hooks
-rw-rw-r--  1 brb brb  176 May  7 16:32 index
drwxrwxr-x  2 brb brb 4096 May  6 11:55 info
drwxrwxr-x  3 brb brb 4096 May  6 11:55 logs
drwxrwxr-x 85 brb brb 4096 May  7 16:12 objects
-rw-rw-r--  1 brb brb   41 May  7 16:32 ORIG_HEAD
-rw-rw-r--  1 brb brb  107 May  6 11:55 packed-refs
drwxrwxr-x  5 brb brb 4096 May  6 11:55 refs

brb@brb-P45T-A:~/github/toy$ tree .git/refs
.git/refs
├── heads
│   └── master
├── remotes
│   └── origin
│       ├── HEAD
│       └── master
└── tags

brb@brb-P45T-A:~/github/toy$ ls .git/objects/
03  0f  28  39  47  52  5f  65  6e  8c  ad  b7  ce  da  e5  f2  ff
07  12  29  3b  49  53  60  67  6f  9a  b1  bb  cf  dc  e9  f5  info
08  13  2c  3d  4d  56  61  68  84  9e  b2  bc  d0  de  ea  f6  pack
09  16  36  43  4e  57  63  6c  89  a9  b3  c0  d5  e1  ef  fc
0c  1b  37  45  51  58  64  6d  8a  aa  b5  c8  d6  e2  f1  fe

brb@brb-P45T-A:~/github/toy$ ls .git/branches/
brb@brb-P45T-A:~/github/toy$ ls .git/info/
exclude
brb@brb-P45T-A:~/github/toy$ ls .git/logs
HEAD  refs

Tagging

https://git-scm.com/book/en/v2/Git-Basics-Tagging

$ git tag

# Create annotated tags
$ git tag -a v1.4 -m "my version 1.4" 
$ git tag
v1.4
$ git show v1.4

# Tagging Later
$ git log --pretty=online
$ git tag -a v1.2 9fceb02

By default, the git push command does not transfer tags to remote servers. You have to explicitly push tags to a shared server after you have created them.

$ git push origin v1.5
$ git push origin --tags # push up all tags at once

Rename a remote repository

Go to github.com, open the project and click Settings button on the left-bottom corner. Change the repository name on top.
On local machine, rename the directory. Go to the directory. Issue

git remote -v

to get the ULR for the current working copy. Suppose the url is [email protected]:someuser/someproject.git. Now issue the following command to change to the new repository

git remote set-url origin [email protected]:someuser/newprojectname.git

Suppose we have added ssh key to git server and we want to use ssh key/protocol to automatically access the server instead of entering the password (https protocol). See

$ # the REMOTE below could be 'origin' or any name you define
$ git remote show REMOTE
$ git remote set-url REMOTE ssh://[email protected]/USERNAME/REPOS.git
$ git remote show REMOTE
$ git status  # show local branch is update-to-date with REMOTE/BRANCH.

setting up SSH access on the server side

Follow the instruction on git-scm.com. It works.

I tested it by

create a git account (called 'git') on my host machine.
sudo to create a directory called /opt/git/project.git. Change the owner to 'git'. Cd to the directory and initialize it.
Create two virtual machines (vb1 and vb2). Creating a username 'david' on vb1 and a user name 'joseph' on vb2.
Create ssh key for both 'david' and 'joseph'. Ssh to copy their ssh keys to git account.
Create a new directory on vb1 and initialize it. Run git commands to commit & push files to the server (no password is needed). Note that when we use 'git commit', git will ask to create a username and email by first running 'git config' command.
Switch to vb2 and run git clone (no password is needed). The user can modify the code and commit & push files to the server.
Run 'git log' to check if each user's name/email are shown on the log.

Some important points:

There is no daemon to be installed. We only need to install the 'git' program on the server.
When a client uses the 'git' command to communicate with the server, it is actually using the 'ssh' to access the server.
We still need to create a user for this git server. Then the developers' rsa keys can be saved to the git user's <.ssh/authorized_keys> file.
The instruction asks to chmod 700 (rwx------) for the .ssh directory and chmod 600 (-rw------) for the <.ssh/authorized_keys> file.
If the git repository directory is not saved under the user's directory, we need to make sure the owner of the directory is the git user.

Create a git server

How to Install HTTP Git Server with Nginx on Ubuntu 18.04 LTS

Create a git server (github like w/ web interface)

If we like to create a github-like web interface, check out GitLab. See

How to Install Gitlab on Debian 8 (Jessie),
How to Run Your Own Git Server. This includes the installation of postfix for sending emails.

Below is my note

https://about.gitlab.com/downloads/ contains steps of setting up Gitlab.
By default, the domain name you have entered in setting up gitlab will be the URL you will use to access gitlab.
Use the recommended method to install gitlab. Nginx will be installed as an http server.
The root username and password is root and 5iveL!fe.
When new users are created by root, we can put a faked email there (eg [email protected]). The root account can create password for the user.
User's password is used to access GitLab web interface only. It is not used for pushing commits.
After a new user is created, log out of root account and log in using the new user account. Click 'Profile setting' icon and then select SSH > Add SSH key. Copy your <id_rsa.pub> content there. To create your ssh key, use the command line "ssh-keygen -t rsa". The <id_rsa.pub> is located under ~/.ssh directory. The title should be auto populated. If ssh key is added successfully to gitLab, we won't get a pop-up asking password when we run 'git push'.
A new project should be created by users (not root). If I create a project by root, I keep getting a permission issue when I run 'git push'.
The username will affect path to all personal projects; e.g. [email protected]:newuser/test2.git.

How to Install Gitlab with PostgreSQL and Nginx on Ubuntu 15.04 from howtoforge.com.

Gitolite (favored by Ubuntu)

This approach involves the following steps

Installing a gitolite server
Gitolite configuration
Managing gitolite users and repositories
Using your server

Not sure about any advantages of this approach?

Graphical tool

http://askubuntu.com/questions/227554/what-are-some-gui-clients-for-git

Command line approach

$ git log --oneline --decorate --graph --all
*   63af56a (HEAD, origin/master, origin/HEAD, mybranch, master) merged in vm
|\  
| * 67d5aad modify linux by mint 4th line
* | 5f3267a modify linuxfile by vm 4th line
|/  
* adf545a new modify linux by mint
*   f22932b resolve the conflict in vm
|\  
| * b2f1ee4 modify linuxfile by mint
* | fe50c32 modify linuxfile by vm
|/  
* a95258a add linuxfile
* b1a71a8 commit from linux
* d02b570 2nd commit
* 3700939 first commit

gitk and git-gui

sudo apt-get update; sudo apt-get install gitk
git clone [email protected]:joshuaflanagan/gitk-demo.git

cd gitk-demo
gitk --all   # show all refs (branches, tags, etc.)

What I find is gitk will take a little time to rebuild something because after I run git pull, it will show several files (if not all) are uncommitted.

Gitk worked with git-gui program (sudo apt-get install git-gui) which is a gui program to run rescan/stage/commit/push. You launch git-gui from gitk-File-Start git-gui.

It looks these 2 gui tools are sufficient enough.

Others

QGit - QGit is a git GUI viewer built on Qt/C++.
git-cola ???
gitg
Giggle - Similar to gitk
SmartGit - Support push, pull, fetch

Navigating your Git History; git grep, git show, git log

Git Tips

old mode new mode

See this post

git config core.filemode false

Temporarily move to a branch and then move back

# suppose we have 3 commits so far and we are at commit 3 now.
git status
git log                        # show 3 commits
git checkout sha1_commit_1     # move to commit 1
git log                        # only show 1 commit
git checkout master            # go back to the original 3 commits
git status

git log on a specific file

git log FILENAME

# to see the difference/patch in each commit
git log -p FILENAME

git log message is too long

Normally when we use 'git log' command, long log messages get truncated.

Use the following to view the complete log messages instead of truncated log messages.

git log | less

Common standard is 78 characters. See this and this posts.

git push error (file is created by Windows but edited by Linux), Your branch is ahead of 'origin/master'

Suppose there are two users A & B (that me). A modified something, commits and pushes to the remote. I modified the same file, commit (so far OK since the actions are all local) and try to push to the remote.

If I forget to run git pull and after running git push, we will get something like if there is a conflict.

C:\Users\XXX\Documents\GitHub\toy [master]> git push
To https://github.com/arraytools/toy.git
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'https://github.com/arraytools/toy.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

C:\Users\XXX\Documents\GitHub\toy [master]> git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
nothing to commit, working directory clean

Follow the suggestion from this post to use git pull for git to do merge. It does not work in this case.

C:\Users\XXX\Documents\GitHub\toy [master]> git pull origin master
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), done.
From https://github.com/arraytools/toy
 * branch            master     -> FETCH_HEAD
warning: Cannot merge binary files: README.md (HEAD vs. b1a71a846e17416a3b248dfd5829547f74fd812c)
Auto-merging README.md
CONFLICT (content): Merge conflict in README.md
Automatic merge failed; fix conflicts and then commit the result.

C:\Users\XXX\Documents\GitHub\toy [master +0 ~0 -0 !1 | +0 ~0 -0 !1]> dir

    Directory: C:\Users\XXX\Documents\GitHub\toy

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---          5/3/2016   4:33 PM        102 README.md

C:\Users\XXX\Documents\GitHub\toy [master +0 ~0 -0 !1 | +0 ~0 -0 !1]> git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
# You have unmerged paths.
#   (fix conflicts and run "git commit")
#
# Unmerged paths:
#   (use "git add <file>..." to mark resolution)
#
#       both modified:      README.md
#
no changes added to commit (use "git add" and/or "git commit -a")

The README.md file was created on Windows/DOS. For some reason, it is recognized as binary files by git. If I look at the file on Linux, it shows the file is ASCII text but with CR line terminators.

$ file README.md
README.md: ASCII text, with CR line terminators.

Since we already run git pull, it won't work to run git reset --soft HEAD~ to undo a commit. Two solutions

git fetch origin; git reset --hard origin/master to throw away my local modified file OR
git reset --hard HEAD to undo git pull; see this post. After that, we can run git reset --soft HEAD~ to undo a commit. See below.

C:\Users\XXX\Documents\GitHub\toy [master +0 ~0 -0 !1 | +0 ~0 -0 !1]> git reset --hard HEAD
HEAD is now at cd2d105 commit from Windows
C:\Users\XXX\Documents\GitHub\toy [master]> git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
nothing to commit, working directory clean

C:\Users\XXX\Documents\GitHub\toy [master]> git reset --soft HEAD~

C:\Users\XXX\Documents\GitHub\toy [master +0 ~1 -0]> git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       modified:   README.md

C:\Users\XXX\Documents\GitHub\toy [master +0 ~1 -0]> cat README.md
# toy
second commit
3rd commit (Windows MACHINE)
C:\Users\XXX\Documents\GitHub\toy [master +0 ~1 -0]> git reset HEAD README.md
Unstaged changes after reset:
M       README.md
C:\Users\XXX\Documents\GitHub\toy [master +0 ~1 -0]> git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   README.md
#
no changes added to commit (use "git add" and/or "git commit -a")

C:\Users\XXX\Documents\GitHub\toy [master +0 ~1 -0]> git checkout -- README.md

C:\Users\XXX\Documents\GitHub\toy [master]> cat README.md
# toy
second commit

C:\Users\XXX\Documents\GitHub\toy [master]> git status
# On branch master
nothing to commit, working directory clean

Now the README.md file is the original one. If we want, we can make a copy of the modified file before unmodifying a modified file (git checkout -- Filename).

Summary

(undo git pull) git fetch; git reset --hard origin/master # OR git reset --hard HEAD
(undo commit) git reset --soft HEAD~
(undo stage) git reset HEAD Filename
(undo change) git checkout -- Filename

git push error - Permission denied (publickey)

odroid@odroid:~/bitbucket/qt$ git push
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

The error only happens if we ssh to the remote server and run git commands. This is done on the bitbucket server.

https://confluence.atlassian.com/bbkb/permission-denied-publickey-302811860.html

On my odroid machine,it shows

odroid@odroid:~/bitbucket/qt$ ssh -T [email protected]
logged in as USERNAME.

You can use git or hg to connect to Bitbucket. Shell access is disabled.
odroid@odroid:~/bitbucket/qt$ ssh-add -h
unknown option -- h
usage: ssh-add [options] [file ...]
Options:
  -l          List fingerprints of all identities.
  -E hash     Specify hash algorithm used for fingerprints.
  -L          List public key parameters of all identities.
  -k          Load only keys and not certificates.
  -c          Require confirmation to sign using identities
  -t life     Set lifetime (in seconds) when adding identities.
  -d          Delete identity.
  -D          Delete all identities.
  -x          Lock agent.
  -X          Unlock agent.
  -s pkcs11   Add keys from PKCS#11 provider.
  -e pkcs11   Remove keys provided by PKCS#11 provider.

But if I connect to odroid through ssh, I get

odroid@odroid:~/bitbucket/qt$ ssh -T [email protected]
Could not open a connection to your authentication agent.
odroid@odroid:~/bitbucket/qt$ ssh-add -h
Could not open a connection to your authentication agent.
odroid@odroid:~$ ssh-add -l
The agent has no identities.

The solution is to follow this.

odroid@odroid:~$ eval `ssh-agent`   # start the ssh-agent 
odroid@odroid:~$ ssh-add ~/.ssh/personalid    # note that personalid is a ssh identity created before
Identity added: /home/odroid/.ssh/personalid (/home/odroid/.ssh/personalid)
odroid@odroid:~$ ssh-add -l
2048 SHA256:1OQfvlXgVl+AyDwjfA0IpHBlgnM2i1dlt0hlHmjJck /home/odroid/.ssh/personalid (RSA)
odroid@odroid:~$ cd bitbucket/qt
odroid@odroid:~/bitbucket/qt$ ssh -T [email protected]
logged in as USERNAME.

You can use git or hg to connect to Bitbucket. Shell access is disabled.
odroid@odroid:~/bitbucket/qt$ git pull
Already up-to-date.

git push error (both clients are linux), Your branch and 'origin/master' have diverged

Two clients (mint and vm) running on Linux. The following is coming from vm machine.

brb@ubuntu:~/toy$ git push
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://[email protected]/arraytools/toy.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
brb@ubuntu:~/toy$ git pull
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), done.
From https://github.com/arraytools/toy
   a95258a..b2f1ee4  master     -> origin/master
Auto-merging linuxfile
CONFLICT (content): Merge conflict in linuxfile
Automatic merge failed; fix conflicts and then commit the result.

brb@ubuntu:~/toy$ cat linuxfile
first line
<<<<<<< HEAD
second line edited by vm
=======
second line edited by mint
>>>>>>> b2f1ee4b1f9c2a428a1af061c7b40316a8f85d3d

brb@ubuntu:~/toy$ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 1 and 1 different commit each, respectively.
  (use "git pull" to merge the remote branch into yours)

You have unmerged paths.
  (fix conflicts and run "git commit")

Unmerged paths:
  (use "git add <file>..." to mark resolution)

	both modified:      linuxfile

no changes added to commit (use "git add" and/or "git commit -a")

master branch and 'origin/master' have diverged, how to 'undiverge' branches'?

The solution now is to run git mergetool (assume we have install something like meld, opendiff, kdiff3, tkdiff, or xxdiff). See Git Mergetool – Merging With a GUI from gitguys.com.

After we edit the file in the middle of meld, we save it and quit meld. Go back to the terminal. Strangely, git knows the conflict has been resolved (shown from git status).

brb@ubuntu:~/toy$ cat linuxfile
first line
second line. conflict resolved in vm
brb@ubuntu:~/toy$ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 1 and 1 different commit each, respectively.
  (use "git pull" to merge the remote branch into yours)

All conflicts fixed but you are still merging.
  (use "git commit" to conclude merge)

Changes to be committed:

	modified:   linuxfile

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	linuxfile.orig

brb@ubuntu:~/toy$ git commit -m "resolve the conflict in vm"
[master f22932b] resolve the conflict in vm
brb@ubuntu:~/toy$ git status
On branch master
Your branch is ahead of 'origin/master' by 2 commits.
  (use "git push" to publish your local commits)

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	linuxfile.orig

nothing added to commit but untracked files present (use "git add" to track)
brb@ubuntu:~/toy$ git push

The repository now looks like:

warning: push.default is unset

Git 2.0 from 'matching' to 'simple'. To squelch this message ...

See stackoverflow.

It only affects what happens when you don't specify which branches you want to push; e.g. 'git push' or 'git push origin' instead of 'git push -u origin master'.

$ git push bitbucket
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:

  git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

  git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

In Git 2.0, Git will default to the more conservative 'simple'
behavior, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

I guess 'simple' (the default) is what we/beginners usually want. Run

git config --global push.default simple

git difftool & meld

Setting up and using Meld as your git difftool and mergetool

git diff can only show the differences on the terminal. git difftool will show the difference on the GUI program.

git difftool FILENAME -y will launch 'meld' (if it has been installed before) to compare the file between revisions by using custom tools. It has to be run before we call git add. This is quite convenient since you can double check before running git commit. The '-y' argument is used to launch a diff tool without a prompt. See the documentation here.

To compare two revisions (sha1sum, branch name)

git difftool <revision_1> <revision_2>

It will open the first file in Meld. After we close the first file, it will launch the 2nd file in Meld, and so on.

To create a (temporary) branch, use git branch -b BRANCHNAME. To delete a (temporary) branch, use git branch -D BRANCHNAME.

It may be helpful to run the following too

git config --global diff.tool meld

Search/find a deleted file from the commit history

https://stackoverflow.com/questions/7203515/git-how-to-find-a-deleted-file-in-the-project-commit-history

For example, the noweb/code.pdf file was mentioned in the Readme file but it is missing from the CRAN's survival package.

$ git log --all --full-history -- noweb/code.pdf
commit 1f0dbc538e9147ab264c5215cf205b9b3bc171cb
Author: Terry M Therneau <[email protected]>
Date:   Sun Jun 26 14:19:51 2016 +0000

    version 2.39-5

commit ec6b50c973b9298a5f1aeb839b2d04d3448e69de
Author: Terry M Therneau <[email protected]>
Date:   Wed May 11 11:45:26 2016 +0000

    version 2.39-4

commit 38a8ccc0fb014c2b2bfbb2b5153b419e615b01ba
Author: Terry M Therneau <[email protected]>
Date:   Sat Apr 16 19:26:46 2016 +0000

    version 2.39-2

Here we see code.pdf file existed in version 2.39-2 and was last seen in version 2.39-5.

To restore the deleted, we use

git checkout 38a8ccc -- code.pdf

Note that I follow the instruction to add the caret sign (^) at the end of SHA, I'll get an error pathspec 'code.pdf' did not match any file(s) known to git. My git version is 2.7.4.

However, for another file, I do need to use caret.

git log --all --full-history -- src/coxfit2.c

git checkout 68e5adc1d1486e2b1daa7db9203fd9226e029992^ -- src/coxfit2.c

diff between two revisions

git diff <revision_1>:<file_1> <revision_2>:<file_2>

# Better to run git config --global diff.tool meld BEFORE
git difftools <revision_1> <revision_2> # ask to go each file separately
git difftools <revision_1> <revision_2> FILENAME -y
                                        # just compare the file <FILENAME>

git diff master..test                   # between the tips of the two branches

git diff test                           # between your current working directory and the (remote) snapshot on the 'test' branch.

git diff --name-only SHA1 SHA2          # show only file names

git diff --name-only HEAD~10 HEAD~5     # between the tenth latest commit and the fifth latest

Monitor/find files that have been changed since last pull

git branch
git fetch origin master    # or git fetch on Windows
git diff --name-only origin/master

git log                    # local
git log origin/master      # remote repository

$ git status
On branch master
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

nothing to commit, working directory clean

We can also use the gitk to view the log. The following is a screenshot from Window's git gui (Windows Start > Git > Git Gui. Then Repository > Visualize All Branch History). As you can see the local repository (master w/ yellow circle) is 2 commits behind the remote repository (remotes/origin/master w/ blue circle).

Pull and overwrite local files

If we want to run git pull and also overwrite possibly changed local file, we use (see stackoverflow)

git fetch --all
git reset --hard origin/master

Your branch is ahead by X commits after running git pull

git status shows Your branch is Ahead by X commits after running git pull. See this post. The solution is to run git fetch after git pull.

This repository currently has approximately XXXX loose objects

http://stackoverflow.com/questions/21457407/git-gui-perpetually-getting-this-repository-currently-has-approximately-320-lo

git gc --aggressive

git commit -am

works only for simple edits. It does not work for renaming files, etc.

git diff --staged

git checkout -- FileName

to recover a modified/deleted file where -- is related to branches

rm FileName
git pull   # do nothing
git status # still miss 'FileName'
git checkout FileName

git reset HEAD FileName

to un-stage a file

Rewrite a commit

ps. This is different from the case of Pull and overwrite local files

Suppose someone made a stupid commit. We want to get a previous version of the code and commit that one.

git checkout XXXXXX -- FileName
git commit -am CommitMessage

origin and nickname

git remote add NickName https://XXX
git remote 
git push -u NickName master

Why is my .git folder so big?

https://www.reddit.com/r/git/comments/4xsh26/why_is_my_git_folder_so_big/

git gc --aggressive --prune

Large files and storage limit

Less than 100MB

$ git push -u origin master
Counting objects: 115, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (100/100), done.
Writing objects: 100% (115/115), 97.54 MiB | 4.48 MiB/s, done.
Total 115 (delta 16), reused 0 (delta 0)
remote: warning: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: warning: See http://git.io/iEPt8g for more information.
remote: warning: File GSE48215/breastcancer-bt20_raw.vcf is 79.57 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
To https://github.com/arraytools/seqtools_testdata.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

It looks like all files are uploaded successfully.

One annoying thing about github is when we download the repository it automatically add a folder name (REPOSITORY-BRANCH) at the top. For example, it creates 'seqtools_testdata-master'.

$ unzip -l ~/Downloads/seqtools_testdata-master.zip 
Archive:  /home/odroid/Downloads/seqtools_testdata-master.zip
9f5e132bcfd2007f5fc1fb4ee465c5b307b3e85a
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2016-06-01 21:05   seqtools_testdata-master/
        0  2016-06-01 21:05   seqtools_testdata-master/GSE11209-master/
 46737374  2016-06-01 21:05   seqtools_testdata-master/GSE11209-master/SRR002051.fastq
 46675548  2016-06-01 21:05   seqtools_testdata-master/GSE11209-master/SRR002059.fastq
        0  2016-06-01 21:05   seqtools_testdata-master/GSE11209-master/Saccharomyces_cerevisiae/
...

Larger than 100MB

remote: error: File hg19/chr1.fa is 242.46 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File hg19/chr1.fa.bwt is 237.70 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File hg19/chr1.fa.sa is 118.85 MB; this exceeds GitHub's file size limit of 100.00 MB
To https://github.com/arraytools/GSE48215.git
 ! [remote rejected] master -> master (pre-receive hook declined)

Import from one remote to another remote

Suppose the new remote is on bitbucket.

rsync -av DIRECTORY .
cd SUBDIRECTORY
git remote -v # Examine the current URL
git remote set-url origin [email protected]:USERNAME/DIRECTORY.git
git remote -v # Examine the new URL
git push -u origin --all
git push origin --tags

Other instructions

Import from CVS

http://git-scm.com/docs/gitcvs-migration
http://stackoverflow.com/questions/20869710/migrate-from-cvs-to-git-without-losing-history (see also comments after the suggestion using git-cvsimport)

sudo apt-get install git-cvs  # cvsimport command
tar xzvf tmp.tar.gz
cd DirectoryName

# The next command requires a connection to the CVS server even we have a copy of CVS in the local pc
git cvsimport -C ~/Downloads/tmp ModuleName # SLOW & give up

Zip up the folder but exclude the .git subfolder

zip -r myproject.zip myproject -x *.git*

On Mac, we should add --exclude=*.DS_Store* to exclude Mac OS X directory display metadata files.

Preview HTML files from Github and Bitbucket

http://htmlpreview.github.io/

General resources

Git hosting services

Self-hosted

Gitlab

gitlabr R package

Compared to Github, Gitlab can

host private projects for free
host static web pages on http://pages.gitlab.io/.

Host web site

http://pages.gitlab.io/

http://monitor.gitlab.net/

How to install and configure Gitlab on Ubuntu

https://www.howtoforge.com/tutorial/how-to-install-and-configure-gitlab-on-ubuntu-16-04/

Bitbucket

New IP July 2, 2018
Storage limit warning after 1GB, hard limit for 2GB.

Github

Github Status

https://status.github.com/

Large files (eg binary)

Disk Quota. Repositories < 1GB and files < 100MB.
Releases allow you to include binary files, such as compiled programs. See https://help.github.com/articles/distributing-large-binaries/

Create a new repository

After we use the web interface to create a new empty repository, we are instructed to do one of the following

create a new repository on the command line

echo "# toy" >> README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin https://github.com/arraytools/toy.git
git push -u origin master

Compare git remote add (set up remote url) and git remote set-url (switch remote url)
push an existing repository from the command line

git remote add origin https://github.com/arraytools/toy.git
git push -u origin master

import code from another repository. You can initialize this repository with code from a Subversion, Mercurial, or TFS project.

Host web site

Updating gh-pages (assuming your vignette file is named 'index.Rmd'). See a note.

# pip install ghp-import
Rscript -e "devtools::build_vignettes()"
ghp-import inst/doc
git push

Using GitHub Pages To Host Your Website using a Custom Domain. The IP address information for Github is changed to new ones.

# Create a new repository in Github using the web interface
# clone it to the local pc
git clone https://github.com/user/repository.git

# create a new branch
cd repository
git checkout --orphan gh-pages
git status # double check we are in the gh-pages branch
git rm -rf .

# create new files. 
touch index.html
git add -a 
git commit -m "Adding pages"

# push to github
git push origin gh-pages

# see the web page at
# http://username.github.io/repository/.

Some Examples:

PiScope
dygraphs for R
networkD3 for R
BRB-DGE
read depth calculator It can run some script!!!

12 件可以用 GitHub 完成的很酷的事情

12 cool things you can do with GitHub

Create Websites

Documentation generator

Use a Static Site Generator to Build a Quick Website. Jekyll, Hugo, Hexo, Gitbook and Pelican. Jekyll was explained in detail.
Comparison of documentation generators

Jekyll

https://www.makeuseof.com/tag/static-site-generator-build-website/
Using Jekyll with Pages
Get Started With GitHub Pages (Plus Bonus Jekyll)
Cookbook for R powered by knitr and Jekyll.
Slower than Hugo as pointed out by YiHui Xie.
Setup website of your Github account with Jekyll

Install Jekyll

on Ubuntu 14.04 I use (the second command will take a while to start),

sudo apt-get install ruby1.9.1-dev
sudo gem install jekyll

Example

http://www.masalmon.eu/2017/11/16/wheretoliveus/ and its source code in Github.
https://privefl.github.io/blog/one-year-as-a-subscriber-to-stack-overflow/ and its source

Script

cd /tmp
jekyll new MyNewSite
# It will create a new folder 'MyNewSite' with about.md, _config.yml, css (folder),  
#    _includes (folder), index.html, _layouts (folder), _posts (folder) and _sass (folder).
cd MyNewSite
jekyll serve --watch
# It'll say the Server Address: http://127.0.0.1:4000/
# We can open a browser to see a template of html page created by '''jekyll new''' command.

hexo

Hexo is a fast, simple and powerful blog framework. You write posts in Markdown (or other languages) and Hexo generates static files with a beautiful theme in seconds.

Example

http://mutse.github.io/

Sphinx & Read The Docs

Simple GitHub repo and ReadTheDocs set up

http://tutos.readthedocs.io/en/latest/source/git_rtd.html#creating-repository-on-github

Examples

Raspberry Pi/Python

Creating a project website with Sphinx from Documenting your code.

Hugo

R blogdown package depends on Hugo.

Install snap from the Snap Store.

Install

Start a blog in 30 minutes with Hugo, a static site generator written in Go (Unix and Windows). Note Hugo generates HTML for you. You then take that HTML and serve it on some web server, such as Apache HTTPD, or Nginx.

On Ubuntu

sudo apt-get install hugo

How To Install and Use Hugo, a Static Site Generator, on Ubuntu 14.04. See the latest binary version (32/64-bit or ARM/ARM64, Windows/Linux/Mac) at https://github.com/gohugoio/hugo/releases/.

wget https://github.com/gohugoio/hugo/releases/download/v0.31.1/hugo_0.31.1_Linux-64bit.deb
sudo dpkg -i hugo*.deb

# Install the Hugo Themes
# SKIP FOR NOW

# Install the Pygments Syntax Highlighter
sudo apt-get install python-setuptools python-dev build-essential
sudo easy_install pip
pip install --user Pygments # https://github.com/pypa/pip/issues/4186

Create a new site

Follow https://gohugo.io/getting-started/quick-start/

(trusty)brb@localhost:~/Downloads$ hugo new site quickstart
Congratulations! Your new Hugo site is created in /home/brb/Downloads/quickstart.

Just a few more steps and you're ready to go:

1. Download a theme into the same-named folder.
   Choose a theme from https://themes.gohugo.io/, or
   create your own with the "hugo new theme <THEMENAME>" command.
2. Perhaps you want to add some content. You can add single files
   with "hugo new <SECTIONNAME>/<FILENAME>.<FORMAT>".
3. Start the built-in live server via "hugo server".

Visit https://gohugo.io/ for quickstart guide and full documentation.

(trusty)brb@localhost:~/Downloads$ cd quickstart;\
git init;\
git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke;\

# Edit your config.toml configuration file
# and add the Ananke theme.
echo 'theme = "ananke"' >> config.toml

(trusty)brb@localhost:~/Downloads/quickstart$ hugo new posts/my-first-post.md
/home/brb/Downloads/quickstart/content/posts/my-first-post.md created

(trusty)brb@localhost:~/Downloads/quickstart$ ls -l content/posts
total 12
-rw-rw-r-- 1 brb brb 76 Dec 19 16:00 my-first-post.md

(trusty)brb@localhost:~/Downloads/quickstart$ ls -l
total 40
drwxrwxr-x 2 brb brb 4096 Dec 19 15:58 archetypes
-rw-rw-r-- 1 brb brb   91 Dec 19 16:03 config.toml
drwxrwxr-x 3 brb brb 4096 Dec 19 16:00 content
drwxrwxr-x 2 brb brb 4096 Dec 19 15:58 data
drwxrwxr-x 2 brb brb 4096 Dec 19 15:58 layouts
drwxrwxr-x 6 brb brb 4096 Dec 19 16:20 public
drwxrwxr-x 2 brb brb 4096 Dec 19 15:58 static
drwxrwxr-x 3 brb brb 4096 Dec 19 16:00 themes

(trusty)brb@localhost:~/Downloads/quickstart$ cat config.toml 
baseURL = "http://example.org/"
languageCode = "en-us"
title = "TAICHIMD"
theme = "ananke"

# Start the server
(trusty)brb@localhost:~/Downloads/quickstart$ hugo server -D # prompt will not be returned, Press Ctrl+C to stop
                                                             # -D/--buildDrafts: include content marked as draft
Started building sites ...

Built site for language en:
1 of 1 draft rendered
0 future content
0 expired content
1 regular pages created
8 other pages created
0 non-page files copied
1 paginator pages created
0 tags created
0 categories created
total in 27 ms
Watching for changes in /home/brb/Downloads/quickstart/{data,content,layouts,themes,static}
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop

Deploy Your Website

After running hugo server for local web development, you need to do a final hugo run without the server part of the command to rebuild your site. You may then deploy your site by copying the public/ directory to your production web server.

(trusty)brb@localhost:~/Downloads/quickstart$ hugo 
Started building sites ...

Built site for language en:
0 of 1 draft rendered
0 future content
0 expired content
0 regular pages created
6 other pages created
0 non-page files copied
0 paginator pages created
0 tags created
0 categories created
total in 15 ms

Dev vs Deploy Destinations

Directory Structure

Configure Hugo

Hugo themes

(trusty)brb@localhost:~$ cd Downloads
(trusty)brb@localhost:~$ git clone https://github.com/gesquive/hugo-slate-demo
(trusty)brb@localhost:~$ cd hugo-slate-demo/
(trusty)brb@localhost:~/Downloads/hugo-slate-demo$ hugo server -t slate

Examples

http://noreastrconf.com/

roxygen

Pelican

Markdown

CommonMark - A strongly defined, highly compatible specification of Markdown

Cheat sheet

Printable Markdown Cheat Sheet for Beginners and Experts

Software

Editor/viewer on Ubuntu. See this discussion. Retext
Multiple platforms.
- Atom opensource. The ATOM editor has a documentation at here.

- Haroopad free, not opensource
Windows:
- Markdownpad nonfree.
- Texts notfree
Python solution. Grip
Online StackEdit

R markdown

For code chunks, use

```{r eval=FALSE echo=FALSE}
library(dplyr); library(ggplot2); library(ggthemes);
```

For internal links, see How to link to part of the same document in Markdown?

Tip: just one # for all heading sizes, no space between # and anchor name, anchor tag names must be lowercase, and delimited by dashes if multi-word.

[click on this link](#my-multi-word-header)

### My Multi Word Header

Github Markdown

To show images on readme.md file use

<img src="https://raw.github.com/arraytools/arduino/master/images/%E5%9B%9B%E6%B5%B7%E4%B8%80%E5%AE%B6.jpg"/>
# or relative paths (assume readme.md is at the root of your repository
<img src="master/images/%E5%9B%9B%E6%B5%B7%E4%B8%80%E5%AE%B6.jpg"/>

For code, use

Some basic Git commands are:
```
git status
git add
git commit
```

```R
library(dplyr); library(ggplot2); library(ggthemes);
```

Bitbucket Markdown

https://bitbucket.org/tutorials/markdowndemo/src

To show images on readme.md, use for example

![Alt text](https://bitbucket.org/mingchung/qt/raw/master/example/drawer/drawer0.png)

Note that it seems there is no way to resize the image if we use markdown files.

For code syntax highlight or preformatted text, use 3 ticks (```). See https://confluence.atlassian.com/bitbucketserver/markdown-syntax-guide-776639995.html. For example,

```c++
MainWindow w;
```

Continuous Integration

Mobile apps

iOS: Working Copy works fine. It will create a new SSH key and added to your bitbucket account.

Software copyrighted by NIH

BMDExpress-2.0 - analyze high dimensional dose-response data, in particular gene expression data.

@@ Line 1,709: / Line 1,709: @@
 ** [https://www.howtoforge.com/tutorial/how-to-install-gogs-go-git-service-on-ubuntu-1604/ How to Install Gogs Go Git Service on Ubuntu 16.04]
 * [https://gitea.io/en-US/ gitea]
+** [https://www.howtoforge.com/tutorial/install-gitea-using-docker-on-ubuntu/ How to Install Gitea Self-hosted Git Service using Docker on Ubuntu 18.04]
 == [https://about.gitlab.com/ Gitlab] ==

Github: Difference between revisions

Revision as of 14:15, 1 December 2018

Git

git config

git stage area

git push, git clone and remote repository

git clone when no 'master' branch exists

git fork

First example

Windows

Linux

Gitk colors and bold

Set up a new local/remote repository

ssh key

Already has a git repository on my computer

'master' and 'origin'

'origin master' vs 'origin/master' format

Multiple remotes

How to know which remote's master branch to pull/push

-u option

“git push -u origin master” vs “git push origin master”

git branch

Example

Setup editor

.gitignore file

git status, multiple branches

Your branch is up-to-date with 'origin/master'

nothing to commit, working directory clean

Remove untracked files

Remove deleted files

Undo

git commit --amend

Unmodifying a Modified File (not staged)

Unstaging a Staged File

Undo git pull or merge

Undo git commit

Undo git push

change commit message

Remove a file from a Git repository without deleting it from the local filesystem

Git reset revert rebase commands

git fetch vs git pull

git fetch and gikt

branch - not the same as in CVS

master branch

Create a new branch

Delete a branch locally or from the remote

Switch branches in local repository

Basic merging and conflicts

push the new branch to remote

pull

Branch management

Branch workflows

Inspect a Remote

Remote branches, tracking branches

Rebasing - integrate changes from 2 branches

Detached head/HEAD detached from xxxxxxx

git merge

Merge conflict 1

Merge conflict 2

Manual merge and auto merge

Practices

git rebase

.ref directory

Tagging

Rename a remote repository

setting up SSH access on the server side

Create a git server

Create a git server (github like w/ web interface)

Gitolite (favored by Ubuntu)

Graphical tool

Command line approach

gitk and git-gui

Others

Navigating your Git History; git grep, git show, git log

Git Tips

old mode new mode

Temporarily move to a branch and then move back

git log on a specific file

git log message is too long

git push error (file is created by Windows but edited by Linux), Your branch is ahead of 'origin/master'