Linux: Difference between revisions
| Line 778: | Line 778: | ||
=== Run a command from previous history === | === Run a command from previous history === | ||
http://www.thegeekstuff.com/2008/08/15-examples-to-master-linux-command-line-history/ | * http://www.thegeekstuff.com/2008/08/15-examples-to-master-linux-command-line-history/ | ||
* http://unixhelp.ed.ac.uk/shell/tcsh_hist3.html | |||
# ^P: Move up through the command history list one command at a time. | |||
# ^N: Move down through the command history list one command at a time. | |||
# !!: Run the previous command. | |||
# !n: Run command number n | |||
# !string: Run most recent command starting with characters in string | |||
# !?string: Run most recent command containing characters that match string | |||
For example, | For example, | ||
<pre> | <pre> | ||
Revision as of 10:08, 10 November 2014
Handy Linux tips
Beautiful desktop
How much resource is used by a process
Find the process ID first by ps -ef | grep APPLICATIONAME. Then
ps -p <pid> -o %cpu,%mem,cmd
For example,
$ ps -ef | grep akregator brb 15013 1942 1 10:41 ? 00:00:05 akregator --icon akregator -caption Akregator brb 15186 24045 0 10:50 pts/11 00:00:00 grep --color=auto akregator $ ps -p 15013 -o %cpu,%mem,cmd %CPU %MEM CMD 1.0 0.8 akregator --icon akregator -caption Akregator
Query whether the OS is 64-bit or 32-bit
SYSTEM_ARCH=getconf LONG_BIT echo $SYSTEM_ARCH
Switch user in command line
use
su newusername
to switch to another user.
Find binary file location
- which
$ which ls /bin/ls
- whereis
$ whereis ls ls: /bin/ls /usr/share/man/man1p/ls.1p.gz /usr/share/man/man1/ls.1.gz
- type -a
$ type -a ls ls is aliased to `ls --color=tty' ls is /bin/ls
Use locate command mindfully. It is used to find the location of files and directories. Note that locate does not search the files on disk rather it searches for file paths in a database. For example, the following command will search .png files over the system (not only the personal directory).
locate "*.png"
IP address fundamental
http://www.howtogeek.com/133943/geek-school-learning-windows-7-ip-addressing-fundamentals/
Bash shell programming
http://bash.cyberciti.biz/guide/Main_Page
Redirect standard error
http://bash.cyberciti.biz/guide/Standard_error. Use 2> operator.
command 2> errors.txt
ls | more without lose color
ls --color | more
Most likely your ls is aliased to ls --color=auto. If you do ls --color (which is morally equivalent to ls --color=always), that will force it to turn on colors.
ls output with color background
In my case, after I apply chmod 755 -R XXXX, the weird green background color goes away.
colordiff-color in terminal diff
Need to install first. apt-get install colordiff. http://www.cyberciti.biz/programming/color-terminal-highlighter-for-diff-files/
sudo apt-get install colordiff diff -y file1 file2 | colordiff
where -y option means to show the output in two columns.
Interpretation of the diff output:
The first line of the diff output will contain:
- line numbers corresponding to the first file,
- a letter (a for add, c for change, or d for delete), and
- line numbers corresponding to the second file.
In our output above, 2,4c2,4 means: "Lines 2 through 4 in the first file need to be changed in order to match lines 2 through 4 in the second file." It then tells us what those lines are in each file:
- Lines preceded by a < are lines from the first file;
- lines preceded by > are lines from the second file.
- The three dashes ("---") merely separate the lines of file 1 and file 2.
2,4c2,4 < I need to run the laundry. < I need to wash the dog. < I need to get the car detailed. --- > I need to do the laundry. > I need to wash the car. > I need to get the dog detailed.
Run diff with large files
Meld freezes When I tested it with two large files (800k & 936k lines coming from human gtf files). Actually the whole linux system became unresponsive.
output colored console to html
Use ansi2html.sh. It only requires gawk.
- Use wget to download it
- sudo apt-get install gawk
- chmod +x ansi2html.sh
- colordiff file1 file2 | ./ansi2html.sh > diff.html
using a the result of a diff in a if statement
ls -lR $dir > a
ls -lR $dir > b
DIFF=$(diff a b)
if [ "$DIFF" != "" ]
then
echo "The directory was modified"
fi
Another example
if [ "$(diff file1.html file2.html)" == "" ]; then echo Same; else echo Different; fi
Listen to pandora in Europe: install squid proxy
http://www.cyberciti.biz/faq/access-pandora-radio-using-proxy-server-outside-usa/
Interestingly, the firefox connection settings should choose HTTP Proxy instead of 'SOCKS host'.
Bypass blocked website from work computer
http://www.panix.com/~ruari/censorship.html
ssh -D 4096 user@remoteip ssh -D 4096 -p 23 user@remoteip
This will require you to enter the password and leave you in the remote machine. If a nonstandard port is required, we can use -p option.
Now in the firefox, we need to go to Edit -> Preferences -> Advanced -> Network tab -> Settings... Check 'Manual proxy configuration' (The default is 'Use system proxy settings') and enter 'localhost' for SOCKS Host and '4096' for the Port. Don't enter 'localhost' in the HTTP Proxy.
On Windows, we can use Putty. In short, in the left-hand panel, navigate through Connection > SSH > Tunnels. Enter 4096 in the Source Port box and select the Dynamic radio button. Click Add and “D4096″ will appear in the Forwarded Ports list. The setting in the firefox end is the same.
Linux journal also put a video on youtube. We can use http://www.ipligence.com/geolocation to check the current location. The port number is 1080 in the example. The example actually also use '-N' option which means no interaction; i.e. ssh -N -D 1080 user@remoteip. So we won't see anything after we type our password. Once we want to stop SOCK proxy, we just need to hit Ctr+C on terminal.
Graphical way to display disk usage
For example, to use xdiskusage, we run apt-get install xdiskusage and launch it by xdiskusage ~/.
Display files sorted by modified date in a directory recursively
stat --printf="%y %n\n" $(ls -tr $(find DIRNAME -type f))
find -type f -printf '%T+\t%p\n' | sort -n
Display directory size with sorting and human readable
du -k ~/ --max-depth=1 --exclude ".*" | sort -nr | cut -f2 | xargs -d '\n' du -sh
The --exclude is to hide hidden directories.
Note that the 'du' commands may be cheating. See the following screenshot.
The discrepancy is explained by 'sector'. See http://askubuntu.com/questions/122091/difference-between-filesize-and-size-on-disk.
$ sudo dumpe2fs /dev/sda1 | grep -i "block size" dumpe2fs 1.41.14 (22-Dec-2010) Block size: 4096
To show a file size in terms of blocks, we can use
ls -s
So for example, if a file takes 150 blocks, and if a block takes 4096 bytes, then the file takes 150*4096/1024 KB on disk.
Apache benchmark (ab) testing
ab -n 100 -c 10 http://taichi.selfip.net:81/ ab -n 100 -c 10 http://taichi.selfip.org/
rsync with exclude files/directories
See http://www.thegeekstuff.com/2011/01/rsync-exclude-files-and-folders/. The key is excluded files are relative to the current directory even we specify the absolute path. For example /path1/path2/file does not mean the file is located under /path1/path2; it means the file is located under ./path1/path2.
rsync -avz --exclude '/path1/path2/file' source/ destination/
rsync with non-standard port
Use -e option
rsync -avz -e "ssh -p 23" mydir user@remoteip:
rsync with progress bar
Use --progress option.
rsync -avz --progress file1 file2
The 'rsync' command works on transferring files local to local too.
rsync on Windows
Download and install command line rsync from http://www.rsync.net/resources/howto/windows_rsync.html. The website also provides a documentation. Some people are concern about the license issue. The website here provides a link to the free, old but usable version 4.0.5 which is newer than I tested v3.1.0.
Below are my note by using cwrsync v3.1.0 installer got from http://www.rsync.net.
cd C:\Program Files (x86)\cwRsync\bin ssh-keygen -t rsa -N '' rsync -av "/cygdrive/c/Users/brb/.ssh/id_rsa.pub" [email protected]:.ssh/authorized_keys rsync -av "/cygdrive/c/Users/brb/Downloads/cytokineMC.txt" [email protected]:Downloads/
scp with non-standard port
Use -P argument.
ssh -P 23 myfile user@remoteip:
scp or ssh without password
http://www.thegeekstuff.com/2008/06/perform-ssh-and-scp-without-entering-password-on-openssh/
- Verify that local-host and remote-host is running openSSH
- Generate key-pair on the local-host using ssh-keygen
- Install public key on the remote-host
- Give appropriate permission to the .ssh directory on the remote-host
- Login from the local-host to remote-host using the SSH key authentication to verify whether it works properly
- Start the SSH Agent on local-host to perform ssh and scp without having to enter the passphrase several times
- Load the private key to the SSH agent on the local-host
- Perform SSH or SCP to remote-home from local-host without entering the password
Another option is to use ssh -i IDENTITY_FILE. See superuser.com.
ssh key
SSH key is useful if you want a password-less login to a remote system. Some useful resources:
- https://help.ubuntu.com/community/SSH/OpenSSH/Keys
- https://help.github.com/articles/generating-ssh-keys
The steps are
- Check if there is an existing key
ls -al ~/.ssh
- Create a new RSA key pair:
ssh-keygen -t rsa
- Copy the public key to a remote host ([email protected]) over ssh. The current user (eg brb) and the remote user (eg git)have not any relationship (they most likely have different user names):
ssh-copy-id [email protected]
Or
cat ~/.ssh/id_rsa.pub | ssh [email protected] "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
- Test if this is working by trying 'ssh [email protected]'.
- To disable the password for root login. Type sudo nano /etc/ssh/sshd_config
PermitRootLogin without-password
Then run the following to put the changes into effect:
reload ssh # Or service ssh restart
If we like to ask all users to use key-based to log in, we can modify the line
PasswordAuthentication no
in sshd_config.
Multiple ssh keys
If we want to use a specific key in ssh, use
ssh -i ~/.ssh/xxx_id_rsa [email protected]
See also ssh-add & ssh-agent.
- https://confluence.atlassian.com/pages/viewpage.action?pageId=271943168
- http://askubuntu.com/questions/269140/how-to-use-multiple-ssh-keys-with-different-accounts-and-hosts
Copy ssh keys to another computer
http://askubuntu.com/questions/134975/copy-ssh-private-keys-to-another-computer
ssh log files
- /var/log/syslog
- /var/log/auth.log (see who is trying to connect; check out http://ip-lookup.net/index.php to see their geolocation)
It is also helpful to check /etc/hosts.allow and /etc/hosts.deny for any possible wrong configuration.
The device is busy
See http://ocaoimh.ie/2008/02/13/how-to-umount-when-the-device-is-busy/
# fuser -m /dev/sdc1 /dev/sdc1: 538 # ps auxw|grep 538 donncha 538 0.4 2.7 219212 56792 ? SLl Feb11 11:25 rhythmbox
Another handy one is:
umount -l /dev/sdwhatever
Kill a process
kill -9 PID
Create an ext3 file system on a USB flash drive
umount /dev/sdb1 (depending on the device of course) sudo mkfs.ext3 /dev/sdb1 sudo e2label /dev/sdb1 usbdrive (change the label)
We can create MS-DOS file system by
sudo mkfs.vfat /dev/sdb1
Port forwarding
For example, we like to use source port number 8080, destination port 80 (the http port), and destination server www.ubuntuforums.org.
ssh -L 8080:www.ubuntuforums.org:80 <host>
Where <host> should be replaced by the name of your laptop. The -L option specifies local port forwarding. For the duration of the SSH session, pointing your browser at http://localhost:8080/ would send you to http://www.ubuntuforums.org/.
To stop the ssh session, use ps -ef to find the process id and kill it.
Add a new user with home directory
adduser xxx
adduser is better than useradd since useradd does not create home directory and it does not even ask the password for new user. adduser will interactively ask user information.
To delete the user and home directory, use
deluser --remove-home xxx
Compress a folder without full path name
Suppose we want to compress the folder ~/Documents and its subfolders. We want to include Documents folder name but not /home/brb/Documents name.
tar -czvf tmp.tar.gz -C ~/ Documents tar -tzvf tmp.tar.gz
List contents of tar.gz
tar -tzvf myfile.tar.gz
Extract tar.xz file
tar --xz -xvf filename.tar.xz
Note the double slash in front of xz.
Extract one files from tar.gz
Extract a file called etc/default/sysstat from config.tar.gz tarball:
$ tar -zxvf config.tar.gz etc/default/sysstat
Noe that a new directory etc/default will be created under the current directory if it does not exist.
You can also extract those files that match a specific globbing pattern (wildcards). For example, to extract from cbz.tar all files that begin with pic, no matter their directory prefix, you could type:
$ tar -xf cbz.tar --wildcards --no-anchored 'pic*'
To extract all php files, enter:
$ tar -xf cbz.tar --wildcards --no-anchored '*.php'
Show folder size for one level only
du --max-depth=1 -h
The graphical tool is called Disk Usage Analyze which is already available on Ubuntu.
Soft link
ln -s /full/path/of/original/file /full/path/of/soft/link/file
Backup
- Use dropbox folder.
If we don't want to install dropbox software, we can install bash dropbox uploader: http://www.andreafabrizi.it/?dropbox_uploader OR https://github.com/andreafabrizi/Dropbox-Uploader
It allows to upload/download/delete/list files and show info of user. The version I am using is v0.9.7. It works on linux, Windows/Cygwin, Raspberry Pi, etc.
I install it under ~/Downloads/andreafabrizi-Dropbox-Uploader-cdc2466 directory
Instruction with screenshots: http://www.jobnix.in/dropbox-command-line-interface-cli-client/
Sample usages:
./dropbox_uploader.sh list / ./dropbox_uploader.sh upload ~/Desktop/ConfigurateNote.txt
Running a cron job as a user
- crontab cron-file-winter; crontab -l
Make sure the .sh file gives a complete path. For example,
#!/bin/sh R --vanilla < arraytoolsip.R
does not work in cron job although it works perfect when we manually run it from the right path. The sh file should be
#!/bin/sh R --vanilla < $HOME/Dropbox/scripts/arraytoolsip.R
Cron knows nothing about your shell; it is started by the system, so it has a minimal environment. If you want anything, you need to have that brought in yourself. For example, to use 'ifconfig' command, I need to give it a complete path in my script file.
$ cat syncIP
/sbin/ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}'
and the cron job
06 15 * * * /home/MYUSERNAME/Ubuntu\ One/syncIP > $HOME/Ubuntu\ One/ip.txt 2>&1
See here on how to add environment variable into cron environment.
Running crontab as root
Use sudo crontab -e to edit. After saving it, no need to initialize it. Use sudo crontab -l to list the cron job.
Mount usb drive
Run
fdisk -l
OR
dmesg | grep -i "SCSI"
to confirm the USB device is detected.
Now suppose the usb device is found in dev/sdb1.
mkdir /mnt/usb mount -t vfat -o rw,users /dev/sdb1 /mnt/usb
The above mount command assumes the usb drive has Windows vfat partition and users give non-root users the ability to unmount the drive. If the USB drive is partitioned linux ext2/3, we can merely run mount command as
mount /dev/sdb1 /mnt/usb
At the end, run umount command like
umount /mnt/usb
To make the mounting automatically, edit the file /etc/fstab.
/dev/sdb1 /mnt/usb vfat defaults 0 0 /dev/sdb1 /mnt/usb ext3 defaults 0 0
and run
mount -a
Mount NTFS usb drive in xubuntu
http://xflinux.blogspot.com/2011/01/mount-ntfs-volumes-automatically-in.html
sudo apt-get install ntfs-config
Now go to Applications>> System>> Ntfs Configuration Tool
Expand the "Advanced Configuration" and select all those partitions you want to be auto mounted and writable( The tool will detect all partitions at its startup).
Make sure the " Enable write support for internal devices" option is selected. Now click Close.
run commands in a background and allow log off
nohup /path/to/script >output 2>&1 &
vi keys
- editmode / insert: "i"
- editmode / append: "a"
- leave edit mode: "ESC"
- save: ":w"
- quit: ":q"
- save and quit: ":wq"
- quit without saving: ":q!"
Some helps
Cloud
http://slidedecks.wilmoore.com/2012-confoo/diy-private-cloud-using-virtualBox-and-chef/#66
http://www.datacentermap.com/blog/cloud-software-389.html
- OpenStack:Open source software for building private and public clouds. Great for large infrastructures. Cf: Amazon Elastic Compute Cloud (EC2).
- CloudStack
- Eucalyptus API compatible with Amazon EC2
- ownCloud. Cf: Dropbox.
U-boot
http://www.denx.de/wiki/U-Boot
Pandora linux client
http://kevinmehall.net/p/pithos/
COW (copy on write) file system
The cow filesystem was found on xubuntu live CD. See http://en.wikipedia.org/wiki/Copy-on-write
Apache redirection
http://cran.r-project.org/mirror-howto.html
Important linux directories
- /bin - executables used by the base system
- /boot
- /dev
- /etc - configuration files
- /media
- /mnt
- /opt - optional application packages
- /proc - process information only
- /sbin - critical executables for running the system, but should be used by superuser
- /usr - non-critical files. Inside is /usr/bin, which contains most of the libraries used by apps.
- /var - variable data such as databases, mails spools and system logs.
DHCP lease time
On Windows, it is 8 days.
Check a file's encoding
file -bi myfile
For example,
mli@PhenomIIx6:~$ file -bi Downloads/hmv_.rc text/x-c++; charset=utf-16le mli@PhenomIIx6:~$
System monitor tools
Git and Github (arraytools)
- Some examples of projects hosted in github: RHadloop, shiny, Qt.
- https://help.github.com/articles/fork-a-repo
- http://www.atlassian.com/git/tutorial/undoing-changes
- Git reference (5 pages of summary) and Pro Git book
- Distributed Version Control with Git (nice tutorial including Github, BitBucket. )
- Cheat sheet
- https://www.kernel.org/pub/software/scm/git/docs/everyday.html
- http://haacked.com/archive/2012/05/21/introducing-github-for-windows.aspx
- Delete a repository
- git push will ask username/password if we have not logged in (eg use linux machine).
- Under Windows, we need to provide username/password when we install Git program provided by Github. Even that, when we use powershell program to commit codes to github, the username was not yet shown on the github website. We need to use git config --global user.name "Your Name" to fix it.
- When I commit (git commit) a directory containing resource file, it shows warning: LF will be replaced by CRLF in xxxx.qrc.
- Password cache
- git add -A
git config --global credential.helper cache
My example of working on a new repository called 'network'.
- Follow https://help.github.com/articles/create-a-repo to create a new repository. For convenience, I also check the button to create README file.
- Click 'GitHub' icon on Windows Desktop. Look at the LHS and click on the word 'github'. Click 'clone' button (This can be accomplished by git clone https://github.com/arraytools/network.git). The new repository will appear under C:\Users\USERNAME\Documents\Github\ directory. Now Click 'Git Shell' icon on the Windows Desktop and go to C:\Users\USERNAME\Documents\Github\network directory where 'network' is my repository's name.
git config --global color.ui auto # colorize the output of git git init git add client.c git add server.c git add server2.c git commit -m 'first commit' git status # On branch master # Your branch is ahead of 'origin/master' by 1 commit. # git remote add origin https://github.com/arraytools/network.git # git push <remote> <branch> git push origin master # If we modify any file, we need to go through git add, git commit and git push 3 commands. # get commit_id git log # get a specific version git checkout commit_id # after an examination, we want to get the latest version git pull origin master # If you do not want to merge the upstream changes wit your local repository, run git fetch to # fetch the changes and then git merge to merge the changes. # git pull is just a combination of fetch and merge. # To rename a file git mv originafile newfile git commit -m "rename orginalfile" git push # To remove a file rm myfile git add . -A git commit -m "remove a file" git push # Compare difference in local and remote repository git fetch (the syntax is the same as git pull, but it doesn't automatically merge) git status git diff master origin/master --color --name-only (color, Show only names of changed files.) # To force to overwrite local files on pull git fetch --all git reset --hard origin/master
Check the https://github.com/arraytools/network. The commit goes to the repository!
In summary: add and commit are local operations, push, pull and fetch are operations that interact with a remote.
If we want to checkout a specific commit on a new computer, we can use (here we use Qt repository as an example)
git clone https://xxxxxx/xxxxxx/Qt.git cd Qt git log --oneline git checkout SHA1 (7 digits obtained from git log --oneline commandis enough)
After that we can run
# move HEAD to origin git checkout origin/master # Visualize using text mode git log --graph --oneline --date-order --decorate --color --all
Using GUI client: gitg
sudo apt-get install gitg
To switch among different branch (eg. github project page is located in gh-pages branch of a repository),
git checkout gh-pages git checkout master
To avoid being asking the username each time when you use 'git push', you can modify the .git/config file of your local repository. This file contains a section called 'remote' with an entry called 'url'. The 'url' entry should contains the https link of repository you're talking about. When you prefix the host url with your username, git shouldn't be asking for your username anymore. Here's an example: url = https://[email protected]
Set up a new local repository
mkdir /path/to/your/project cd /path/to/your/project git init git remote add origin https://[email protected]/arraytools/REPOSITORYNAME.git git config --global user.name "YOUR NAME" git config --global user.email "YOUR EMAIL ADDRESS" echo "arraytools" >> contributors.txt git add contributors.txt git commit -m 'Initial commit with contributors' git push -u origin master
Already has a git repository on my computer
cd /path/to/my/repo git remote add origin https://[email protected]/arraytools/REPOSITORYNAME.git git push -u origin --all # pushes up the repo and its refs for the first time git push -u origin --tags # pushes up any tags
Rename a repository
- Go to github.com, open the project and click Settings button on the left-bottom corner. Change the repository name on top.
- On local machine, rename the directory. Go to the directory. Issue
git remote -v
to get the ULR for the current working copy. Suppose the url is [email protected]:someuser/someproject.git. Now issue the following command to change to the new repository
git remote set-url origin [email protected]:someuser/newprojectname.git
Git Tips
- git difftool FILENAME -y will launch 'meld' (if it has been installed before) to compare the file between revisions by using custom tools. This is quite convenient since you can double check before running git commit. The '-y' argument is used to launch a diff tool without a prompt. See the documentation here.
- If we want to monitor/find files that have been changed since last pull, run
git fetch origin/master git diff --name-only origin/master
Create a git server (command line only)
Follow the instruction on git-scm.com. It works. I tested it by
- create a git account (called 'git') on my host machine.
- sudo to create a directory under /opt/git/project.git. Change the owner to 'git'. Cd to the directory and initialize it.
- Create two virtual machines (vb1 and vb2). Creating a username 'david' on vb1 and a user name 'joseph' on vb2.
- Create ssh key for both 'david' and 'joseph'. Ssh to copy their ssh keys to git account.
- Create a new directory on vb1 and initialize it. Run git commands to commit & push files to the server (no password is needed). Note that when we use 'git commit', git will ask to create a username and email by first running 'git config' command.
- Switch to vb2 and run git clone (no password is needed). The user can modify the code and commit & push files to the server.
- Run 'git log' to check if each user's name/email are shown on the log.
Create a git server (github like w/ web interface)
If we like to create a github-like web interface, check out GitLab.
Below is my note
- https://about.gitlab.com/downloads/ contains steps of setting up Gitlab.
- By default, the domain name you have entered in setting up gitlab will be the URL you will use to access gitlab.
- Use the recommended method to install gitlab. Nginx will be installed as an http server.
- The root username and password is root and 5iveL!fe.
- When new users are created by root, we can put a faked email there (eg [email protected]). The root account can create password for the user.
- User's password is used to access GitLab web interface only. It is not used for pushing commits.
- After a new user is created, log out of root account and log in using the new user account. Click 'Profile setting' icon and then select SSH > Add SSH key. Copy your <id_rsa.pub> content there. To create your ssh key, use the command line "ssh-keygen -t rsa". The <id_rsa.pub> is located under ~/.ssh directory. The title should be auto populated. If ssh key is added successfully to gitLab, we won't get a pop-up asking password when we run 'git push'.
- A new project should be created by users (not root). If I create a project by root, I keep getting a permission issue when I run 'git push'.
- The username will affect path to all personal projects; e.g. [email protected]:newuser/test2.git.
General resources
- https://www.atlassian.com/git/tutorial/remote-repositories
- http://www.sbf5.com/~cduan/technical/git/git-4.shtml
- http://www.acquia.com/blog/getting-started-collaborative-development-git
- http://documentup.com/skwp/git-workflows-book
- http://www.eqqon.com/index.php/Collaborative_Github_Workflow
Bitbucket (free for 5 users)
- Unlimited private repos
- Code reviews
- JIRA integration
- REST API
- Custom domains
See this post to know how to fix the problem of unknown author. In short, when I uncheck "Use global user setting" from Repository-> Repository Settings -> Advanced does the commit author change as expected.
See here for a list of Android apps related to bitbucket.
Convert an image file to a different format (eg icon)
Using imagemagic program.
convert winamp-ncrow.png -resize 32x32 winamp-ncrow.ico
Run a command from previous history
- http://www.thegeekstuff.com/2008/08/15-examples-to-master-linux-command-line-history/
- http://unixhelp.ed.ac.uk/shell/tcsh_hist3.html
- ^P: Move up through the command history list one command at a time.
- ^N: Move down through the command history list one command at a time.
- !!: Run the previous command.
- !n: Run command number n
- !string: Run most recent command starting with characters in string
- !?string: Run most recent command containing characters that match string
For example,
!-1 !4
Listen to HiChannel internet radio
Use Radio Tray
- http://endroid.blogspot.com/2012/02/listen-hichannel-radio-online-by.html
- http://abcde9990101.blogspot.com/2011/05/ubunturadio-tray.html
I use it to listen m3u file (VLC also supports it too).
Web Analytics Reporting Tools
Paint.NET like program
Read ebook in epub format
Use Calibre.
$ sudo -v && wget -nv -O- https://github.com/kovidgoyal/calibre/raw/master/setup/linux-installer.py | sudo python -c "import sys; main=lambda:sys.stderr.write('Download failed\n'); exec(sys.stdin.read()); main()"
2014-03-19 15:54:28 URL:https://raw.github.com/kovidgoyal/calibre/master/setup/linux-installer.py [25423/25423] -> "-" [1]
Installing to /opt/calibre
Downloading tarball signature securely...
Will download and install calibre-1.28.0-x86_64.tar.bz2
Downloading calibre-1.28.0-x86_64.tar.bz2
100% [===============================================================================================]
Downloaded 63255897 bytes
Checking downloaded file integrity...
Extracting files to /opt/calibre ...
Extracting application files...
Creating symlinks...
Symlinking /opt/calibre/fetch-ebook-metadata to /usr/bin/fetch-ebook-metadata
Symlinking /opt/calibre/lrf2lrs to /usr/bin/lrf2lrs
Symlinking /opt/calibre/ebook-convert to /usr/bin/ebook-convert
Symlinking /opt/calibre/ebook-meta to /usr/bin/ebook-meta
Symlinking /opt/calibre/ebook-edit to /usr/bin/ebook-edit
Symlinking /opt/calibre/lrfviewer to /usr/bin/lrfviewer
Symlinking /opt/calibre/calibre to /usr/bin/calibre
Symlinking /opt/calibre/markdown-calibre to /usr/bin/markdown-calibre
Symlinking /opt/calibre/calibre-debug to /usr/bin/calibre-debug
Symlinking /opt/calibre/calibre-parallel to /usr/bin/calibre-parallel
Symlinking /opt/calibre/web2disk to /usr/bin/web2disk
Symlinking /opt/calibre/calibre-server to /usr/bin/calibre-server
Symlinking /opt/calibre/calibre-customize to /usr/bin/calibre-customize
Symlinking /opt/calibre/ebook-polish to /usr/bin/ebook-polish
Symlinking /opt/calibre/ebook-viewer to /usr/bin/ebook-viewer
Symlinking /opt/calibre/calibre-smtp to /usr/bin/calibre-smtp
Symlinking /opt/calibre/lrs2lrf to /usr/bin/lrs2lrf
Symlinking /opt/calibre/ebook-device to /usr/bin/ebook-device
Symlinking /opt/calibre/calibredb to /usr/bin/calibredb
Setting up command-line completion...
Installing bash completion to /etc/bash_completion.d/calibre
Setting up desktop integration...
Creating un-installer: /usr/bin/calibre-uninstall
Run "calibre" to start calibre
export DISPLAY
If we want to run a GUI app on a remote computer (such as Raspberry Pi/Beaglebone Black) using ssh, we can issue the following command before running the app.
export DISPLAY=:0.0
LVM Demystified
http://www.linuxjournal.com/content/lvm-demystified
Install binary software using sudo
One example (Calibre) is like
sudo -v && wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py | sudo python -c "import sys; main=lambda:sys.stderr.write('Download failed\n'); exec(sys.stdin.read()); main()"
Note that in wget the option "-O-" means writing to standard output and "-nv" means no verbose.
If the option "-O-" is not used, we'd better to use "-N" option in wget to overwrite an existing file.
See the logging and download options in wget's man page.
Log files
ls -lt /var/log
Linux command similar to top to show hard disk activity
Use iotop. On ubuntu, we can use sudo apt-get install to install it. Use sudo iotop to launch it.
sudo iotop -o -u $USER
Install Apache HBase
Follow the Quick Start to downloaded hbase tar ball. Suppose we save the tar ball under ~/Downloads folder and extract it in the same directory. We shall edit conf/hbase-site.xml file according to their instruction. The following is my case.
$ tar xzvf hbase-0.98.3-hadoop2-bin.tar.gz
$ cd hbase-0.98.3-hadoop2/
$ cat conf/hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///home/brb/Downloads/hbase-0.98.3-hadoop2/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/brb/Downloads/hbase-0.98.3-hadoop2/zookeeper</value>
</property>
</configuration>
Before we follow the getting started guide to launch HBase, we shall make sure JAVA_HOME environment variable is created.
$ ls /usr/lib/java $ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64
Note that the last line may be replaced by
export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:bin/javac::")
Then we can launch HBase,
$ ./bin/start-hbase.sh starting master, logging to /home/brb/Downloads/hbase-0.98.3-hadoop2/bin/../logs/hbase-brb-master-brb-P45T-A.out brb@brb-P45T-A:~/Downloads/hbase-0.98.3-hadoop2$ ./bin/hbase shell 2014-07-06 09:51:34,621 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 0.98.3-hadoop2, rd5e65a9144e315bb0a964e7730871af32f5018d5, Sat May 31 19:56:09 PDT 2014 hbase(main):001:0> create 'test', 'cf' 2014-07-06 09:51:49,510 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 0 row(s) in 2.0770 seconds => Hbase::Table - test hbase(main):002:0> list 'test' TABLE test 1 row(s) in 0.0530 seconds => ["test"] hbase(main):003:0> exit brb@brb-P45T-A:~/Downloads/hbase-0.98.3-hadoop2$ ./bin/hbase shell2014-07-06 09:53:37,480 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 0.98.3-hadoop2, rd5e65a9144e315bb0a964e7730871af32f5018d5, Sat May 31 19:56:09 PDT 2014 hbase(main):001:0> list 'test' TABLE 2014-07-06 09:53:44,373 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable test 1 row(s) in 1.4800 seconds => ["test"] hbase(main):002:0> put 'test', 'row1', 'cf:a', 'value1' 0 row(s) in 0.4460 seconds hbase(main):003:0> put 'test', 'row2', 'cf:b', 'value2' 0 row(s) in 0.0140 seconds hbase(main):004:0> put 'test', 'row3', 'cf:c', 'value3' 0 row(s) in 0.0050 seconds hbase(main):005:0> scan 'test' ROW COLUMN+CELL row1 column=cf:a, timestamp=1404654837532, value=value1 row2 column=cf:b, timestamp=1404654856976, value=value2 row3 column=cf:c, timestamp=1404654866298, value=value3 3 row(s) in 0.0560 seconds hbase(main):006:0> get 'test', 'row1' COLUMN CELL cf:a timestamp=1404654837532, value=value1 1 row(s) in 0.0280 seconds hbase(main):007:0> disable 'test' 0 row(s) in 1.6050 seconds hbase(main):008:0> drop 'test' 0 row(s) in 0.2290 seconds hbase(main):009:0> exit brb@brb-P45T-A:~/Downloads/hbase-0.98.3-hadoop2$
curl vs wget
sudo apt-get install curl
For example, the Download link at the National Geographic Travel Photo Contest 2014 works for curl but not wget. I can use curl with -o option but not wget with -o option in such a case.
Apply a patch to source code
http://www.cyberciti.biz/faq/appy-patch-file-using-patch-command/. For example Tophat 2.0.12 compatibility with Samtools 1.0,
brb@brb-VirtualBox:~/Downloads$ ls support_for_tophat_1.patch tophat-2.0.12 tophat-2.0.12.tar.gz brb@brb-VirtualBox:~/Downloads$ grep -r -i "check_samtools" tophat-2.0.12/ tophat-2.0.12/src/tophat.py:def check_samtools(): tophat-2.0.12/src/tophat.py: check_samtools() brb@brb-VirtualBox:~/Downloads$ cp support_for_tophat_1.patch tophat-2.0.12/src/ brb@brb-VirtualBox:~/Downloads$ cd tophat-2.0.12/src/ brb@brb-VirtualBox:~/Downloads/tophat-2.0.12/src$ patch tophat.py < support_for_tophat_1.patch patching file tophat.py Hunk #1 succeeded at 1540 (offset 3 lines). Hunk #2 succeeded at 1563 (offset 3 lines). brb@brb-VirtualBox:~/Downloads/tophat-2.0.12/src$ ls
Monitor network by command line
Use iftop command. On ubuntu, we need to use sudo apt-get install iftop and then run it by sudo iftop -i eth0 -n where -n is to disable dns lookup. After that, we can press some keys to toggle options.
- p: port
- s: source
- d: destination
Copy text to a clipboard to be used in other apps
Install the xclip program. See here or here.
sudo apt-get install xclip # Example sort -n -k 3, -k 2 file.txt | xclip -selection clipboard
Works.
Debian/Ubuntu/Bio-Linux
Debian
Download Debian
Go to http://www.debian.org/distrib/ and download "small installation image", for example debian-7.1.0-amd64-netinst.iso. It is about 222 MB.
Screenshots of step-by-step installation can be found in here.
After installation, you got a desktop environment of Debian based on GNOME 3 (Virtual machine will use recovery mode, but still works. Some people suggest to install the guest additions (in the guest) and make sure that you enable the 3D acceleration in the guest settings.). Also 'free -m' command shows it uses 202 MB memory and the whole system takes up 3.3 GB. I am testing on a Chinese desktop environment.
When Debian is in recovery mode, the desktop interface is like old fashion. Application and Place on top of the screen. When GNOME 3 is working (in my test of Debian 7.1.0, I cannot run VBoxLinux.run, but it still works after I did other steps), the interface is sort of Ubuntu with application launched from the left hand side. It differs from Ubuntu because the side bar appears by clicking a 'preview' button on top left corner.
The default browser in Debian is Iceweasel with AdBlock Plus preinstalled.
I don't know why the default user does not have root privilege.
When I installed the Chinese version, the keyboard switch icon (SCIM) is automatically available. However once the desktop is in regular GNOME 3, the switch icon disappeared. Fortunately, we can use Ctrl + Space to switch languages. Thanks to the hint there.
Server version
There is no a server version of iso to download. At the end of installation, it will ask what software to install: Debian desktop environment, Web server, SSH server, Laptop, SQL database, ... We can uncheck 'Debian desktop environment' item.
Compared to Desktop version, the server version takes 1.3GB space and 33MB memory.
The server version does not have 'sudo' command. Use 'su' to switch to 'root' user.
Note that even we installed 'sudo', we can not use 'sudo' from the default user. It will complain xxx is not in the sudoers file. This incident will be reported..
Browse iso files
http://ftp-nyc.osuosl.org/debian-cdimage/7.1.0/amd64/iso-cd/
Ubuntu/Kubuntu/Lubuntu/Xubuntu
Ubuntu flavors and derivatives
For some reason, when I try the Ubuntu (13.04) live CD, the screen resolution looks perfect. But when I installed the OS, the screen resolution is always too low. The propriety graphics driver cannot be installed successfully. Fortunately, when I try the Kubuntu (13.04), the display resolution problem automatically works!
Update: Kubuntu failed to respond after I install SCIM related programs. A freshly installed linuxmint OS also has a similar problem that the desktop does not respond to mouse or keyboard. Luckily, the Xubuntu works fine and the Chinese input works out of box if I choose Chinese as desktop evvironment (339MB was used).
Server version
Elementary OS
Lightweight and beautiful. Elementary is crafted by designers and developers who believe that computers can be easy, fun, and gorgeous.
Dock is pre-installed.
AV Linux
AV Linux features a complete customized Debian Linux XFCE4 4.10 Desktop Environment with the added bonus of a handpicked selection of pre-tested and pre-configured Audio, Graphics and Video content creation software demonstrating the excellence of Open-Source and also includes many unique Commercial Demos.
After running update/upgrade in Ubuntu
- Virtualbox does not work. After initial fix, the guest machine cannot connect to internet
Selection of desktop environments
- Unity
- Genome
- KDE
- LXQt
- LXDE
- Cinnamon
- MATE
Customize the desktop
- Install Cairo-Dock.
Create customized ubuntu iso
- Ubuntu customization kit linux.com howtogeek
- http://www.ubuntu-mini-remix.org/
- http://amjjawad.blogspot.com/2013/07/ubuntu-mini-iso-installation-process.html
- http://askubuntu.com/questions/409607/how-to-create-a-customized-ubuntu-server-iso
- http://razvangavril.com/linux-administration/custom-ubuntu-server-iso/
- Ubuntu builder
Change the default session when using auto login
See this post. We need to edit the file /etc/lightdm/lightdm.conf. Note that on my Ubuntu 12.04, I have the following desktop options.
$ ls -lt /usr/share/xsessions/ total 16 -rw-r--r-- 1 root root 205 Apr 19 2012 gnome.desktop -rw-r--r-- 1 root root 188 Apr 19 2012 gnome-shell.desktop -rw-r--r-- 1 root root 208 Apr 19 2012 ubuntu-2d.desktop -rw-r--r-- 1 root root 185 Apr 19 2012 ubuntu.desktop
Search package full name
Use apt-file command. See this post
Find package information before installing it
Use "-s" option for simulation. No sudo is necessary.
apt-get -s install PACKAGENAME
The output is too much (include other dependences)
A better way is to use
aptitude search <package> -F "%c %p %d %V"
For example,
debian@beaglebone:~/qt-4.8.5/bin$ aptitude search qtcreator -F "%c %p %d %V" p qtcreator lightweight integrated development environme 2.5.0-2 p qtcreator:armel lightweight integrated development environme 2.5.0-2 p qtcreator-dbg debugging symbols for Qt Creator IDE 2.5.0-2 p qtcreator-dbg:armel debugging symbols for Qt Creator IDE 2.5.0-2 p qtcreator-doc documentation for Qt Creator IDE 2.5.0-2
If a package is already installed, we can use the following way to check version number.
apt-show-versions <package>
Chinese Input
http://pinyinjoe.com/linux/ubuntu-12-chinese-setup.htm
Note: If ibus does not have chewing selection, we should install it first.
sudo apt-get install ibus-chewing
- type 'language' in Dash search and click 'Language Support' (this may not be installed from (x)Ubuntu automatically. In such case, IBUS will be missing eg Chewing method). The 'Language Support' is under Settings menu in xUbuntu.
- choose 'IBUS' for keyboard input method. IBUS is under 'System' menu in xUbuntu.
- Click 'Install/Remove Lanagues'. Choose Chinese -> Chewing.
- Click 'Close' button.
Log out and Log in. Click Keyboard icon on top right corner. Modify its Preferences. Go to 'Input Method' tab. Add Chinese-Chewing and click 'Close' button. We can use Ctrl+Space to switch language input.
On Ubuntu 13.10, type 'language support' in the Dash. Click 'Install/Remove Languages' and check Chinese. In the 'Keyboard input method system', change from default to 'IBus'. We shall see a keyboard icon on the top right of menu. Click it and choose Text Entry Settings. Add Chinese Chewing input. Then change 'switch to next source using' method to 'Ctrl + Space' by pressing both keys on your keyboard. Bingo!
Samba
sudo apt-get install samba samba-common sudo apt-get install python-glade2 sudo apt-get install system-config-samba
Use Dash and search for 'samba'. It will ask for the user's password first.
The 'Access' tab in my samba configuration window, I cannot select users. So I choose 'Allow access to everyone'.
A non-gui way to configuration samba is adding the following to the end of /etc/samba/smb.conf file, sudo nano -w /etc/samba/smb.conf
[brb]
path = /home/brb
; writeable = no
; browseable = yes
guest ok = yes
- Share between ubuntu and linux On the client file manager, type
smb://192.168.1.3/
On Windows PC, go to start and open 'Run' then enter ip with double backslash. Like this (\\192.168.1.3).
Change time zone
http://wiki.debian.org/TimeZoneChanges
$sudo dpkg-reconfigure tzdata
The command launches an ncurses based interface which allows arrow keys to be used to select the region specific time zone.
Make script run at boot time with init.d directory
http://www.debian-administration.org/articles/28
For example, see here from running a python script for raspberry pi.
- Create a script /etc/init.d/lcd
- Make the script executable
- Make the script known to the system by using the update-rc.d' command
sudo update-rc.d lcd defaults
wireless connection randomly drop off
My wireless adapter is TP-LINK, TL-WN722N.
http://askubuntu.com/questions/73607/wireless-connection-keeps-dropping-with-an-intel-3945abg-card
sudo iwconfig wlan0 power off
5 Best Linux/Ubuntu compatible USB Wifi cards:
- AirLink101 AWLL6075 Wireless N Mini USB Adapter
- Medialink – Wireless N USB Adapter – 802.11n
- ASUS (USB-N13) Wireless-N USB Adapter
- Panda Mini Wifi (b/g/n) 150Mbps Card
- TP-Link TL-WN722N 150Mbps High Gain Wireless USB Adapter
To show (USB) wireless adapter information
sudo lshw -C network sudo lsusb -v iwconfig
To check wifi connection information (SSID, channel, address, frequency, qualiyt, signal level, ...)
sudo iwlist wlan0 scan
My experience is quality should be at least 50/70 and signal level should be larger than -60dBm.
Write an iso or img file to a USB flash drive
Use dd
First, get to know the USB drive device name like /dev/sdb. When using 'dd', the USB drive has to be unmounted (using 'umount' command, not click 'reject' button in File Manager). Note that this will irrevocably destroy all data on /dev/sdX.
The instruction can be found in a lot of places like Archlinux wiki page.
sudo fdisk -l sudo dd bs=4M if=xxx.img of=/dev/sdb && sync
where /dev/sdb is a device name, not a partition name. We can also adjust bs to a smaller value like 1M, 4m.
For some reason, I cannot boot from the USB drive. sudo gparted also gives me some error.
UNETBOOTIN
http://xubuntugeek.blogspot.com/2013/04/how-to-install-xubuntu-on-usb-device.html
sudo apt-get install unetbootin extlinux -y
The following is a screenshot of the contents of xubuntu 12.04. The usb drive needs to be formatted to fat32 on Windows OS to repair partition table error. The partition table error was discovered when I use sudo gparted program to view the USB drive.
Universal USB Installer
http://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/
Create an iso file from a CD or HD
Method 1 (Better than Method 2). No need to umount the cdrom.
mkisofs -o /tmp/DVD.iso /tmp/directory/
to make an ISO from files on your hard drive.
Method 2. Make sure the cdrom is NOT mounted. Type mount to confirm it. If cd was mouted automatically unmout it with umount command: like umount /dev/cdrom or umount /mnt/cdrom. Note that no extra forward slash after /dev/cdrom for the command below.
dd if=/dev/cdrom of=~/DVD.iso
We can mount the iso file to a directory to check the iso file content is correct.
mount -t iso9660 -o loop,ro DVD.iso /mnt
Have fun with /etc/hosts file
su -c "nano /etc/hosts"
74.125.67.100 DNS_NAME1 DNS_NAME2
Find a file based on file name
$ find . -iname '*.txt'
Find files modified in one day.
$ find . -mtime -1
Find files modified in one day and contain string 'est'
$ find . -mtime -1 -exec grep --with-filename est {} \;
If the search directory is not the current directory, we need to add a forward slash to the directory name.
$ find ~/Desktop -iname '*.txt' # Not working $ find ~/Desktop/ -iname '*.txt' # Working
The following example shows we can list multiple search criteria. The “‑r” option in tar appends files to an archive. xargs is a handy utility that coverts a stream of input (in this case the output of find) into command line arguments for the supplied command (in this case tar, used to create a backup archive).
find / -type f -mtime -7 | xargs tar -rf weekly_incremental.tar gzip weekly_incremental.tar
Find a file by searching contents
grep -r -i "Entering" ~/Downloads/R-3.0.0/
where -r means recursively searching the directory and -i means case insensitive.
Mount iso file
$ sudo mkdir /mnt/iso $ sudo mount -o loop /tmp/file.iso /mnt/iso $ sudo umount /mnt/iso
Check ubuntu version from command line
lsb_release -a
keyboard shortcuts
Go to keyboard app to change the settings. Note: Super key is also Windows key.
- Ctrl + Super + D: To minimize all windows
- Ctrl + Alt + Arrows: move to another workspace
- Ctrl + Alt + Shift + Arrows: move current application to another workspace
BioLinux
Bio-Linux 7.0 (2012/11/21) screenshot.
Biolinux can be installed in two ways.
- One is to download iso image file. http://nebc.nerc.ac.uk/downloads/
- The other way is to install Bio-linux software/package by using apt-get install method. See http://nebc.nerc.ac.uk/tools/bio-linux/other-bl-docs/package-repository
Apache configuration
https://help.ubuntu.com/11.10/serverguide/httpd.html
Device manager
By default, ubuntu does not provide any graphical tool like device manager on Windows. A very close one is lshw (hardware lister). A GUI tool based on it is called lshw-gtk and can be installed by sudo apt-get install lshw-gtk in Ubuntu/Debian or yum install lshw in Red Hat/Fedora/CentOS.
$ sudo lshw -short
H/W path Device Class Description
===================================================
system To Be Filled By O.E.M. (To Be Filled By O.E.M.)
/0 bus 970 Extreme4
/0/4 processor AMD Sempron(tm) 145 Processor
/0/4/5 memory 128KiB L1 cache
/0/4/6 memory 1MiB L2 cache
/0/10 memory 8GiB System Memory
/0/10/0 memory 2GiB DIMM DDR3 Synchronous 667 MHz (1.5 ns)
/0/10/1 memory 2GiB DIMM DDR3 Synchronous 667 MHz (1.5 ns)
/0/10/2 memory 2GiB DIMM DDR3 Synchronous 667 MHz (1.5 ns)
/0/10/3 memory 2GiB DIMM DDR3 Synchronous 667 MHz (1.5 ns)
/0/0 memory 64KiB BIOS
/0/100 bridge RD890 PCI to PCI bridge (external gfx0 port B)
/0/100/2 bridge RD890 PCI to PCI bridge (PCI express gpp port B)
/0/100/2/0 display RV770 [Radeon HD 4850]
/0/100/2/0.1 multimedia RV770 HDMI Audio [Radeon HD 4850/4870]
/0/100/9 bridge RD890 PCI to PCI bridge (PCI express gpp port H)
/0/100/9/0 bus ASM1042 SuperSpeed USB Host Controller
/0/100/a bridge RD890 PCI to PCI bridge (external gfx1 port A)
/0/100/a/0 bus ASM1042 SuperSpeed USB Host Controller
/0/100/11 storage SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode]
/0/100/12 bus SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
/0/100/12.2 bus SB7x0/SB8x0/SB9x0 USB EHCI Controller
/0/100/13 bus SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
/0/100/13.2 bus SB7x0/SB8x0/SB9x0 USB EHCI Controller
/0/100/14 bus SBx00 SMBus Controller
/0/100/14.1 storage SB7x0/SB8x0/SB9x0 IDE Controller
/0/100/14.2 multimedia SBx00 Azalia (Intel HDA)
/0/100/14.3 bridge SB7x0/SB8x0/SB9x0 LPC host controller
/0/100/14.4 bridge SBx00 PCI to PCI Bridge
/0/100/14.5 bus SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
/0/100/15 bridge SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
/0/100/15.2 bridge SB900 PCI to PCI bridge (PCIE port 2)
/0/100/15.2/0 bus VT6315 Series Firewire Controller
/0/100/15.3 bridge SB900 PCI to PCI bridge (PCIE port 3)
/0/100/15.3/0 eth0 network RTL8111/8168B PCI Express Gigabit Ethernet controller
/0/100/16 bus SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
/0/100/16.2 bus SB7x0/SB8x0/SB9x0 USB EHCI Controller
/0/101 bridge Family 10h Processor HyperTransport Configuration
/0/102 bridge Family 10h Processor Address Map
/0/103 bridge Family 10h Processor DRAM Controller
/0/104 bridge Family 10h Processor Miscellaneous Control
/0/105 bridge Family 10h Processor Link Control
/0/1 scsi2 storage
/0/1/0.0.0 /dev/cdrom disk CDDVDW SH-S223Q
/0/2 scsi4 storage
/0/2/0.0.0 /dev/sda disk 80GB MAXTOR 4K080H4
/0/2/0.0.0/1 /dev/sda1 volume 66GiB EXT4 volume
/0/2/0.0.0/2 /dev/sda2 volume 8186MiB Extended partition
/0/2/0.0.0/2/5 /dev/sda5 volume 8186MiB Linux swap / Solaris partition
/0/3 scsi6 storage
/0/3/0.0.0 /dev/sdb disk SCSI Disk
/0/3/0.0.1 /dev/sdc disk SCSI Disk
/0/3/0.0.2 /dev/sdd disk SCSI Disk
/0/3/0.0.3 /dev/sde disk SCSI Disk
/1 wlan0 network Wireless interface
For storage part, ubuntu provides a graphical tool. See "disk utility" on gnome based ubuntu or search for "disk" in launcher.
Set static IP
root@debian:~# cat /etc/network/interfaces auto lo eth0 iface lo inet loopback iface eth0 inet dhcp
Now edit the file /etc/network/interfaces
# The primary network interface auto eth0 iface eth0 inet static address 192.168.1.3 gateway 192.168.1.1 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 dns-nameservers 192.168.1.1 8.8.8.8
After it, restart network
/etc/init.d/networking restart
OR sudo reboot
Note: It does not work by editing /etc/resolv.conf since this file will be overwritten.
Change IP address from the command line
/sbin/ifconfig eth0 192.168.1.17 netmask 255.255.255.0 up /sbin/ifconfig eth0
Dyndns
See https://help.ubuntu.com/community/DynamicDNS
nano /etc/ddclient.conf
daemon=600 protocol=dyndns2 use=web, web=checkip.dyndns.com, web-skip='IP Address' server=members.dyndns.org login=YOURNAME password='YOURPASSWORD' taichi.selfip.net
And run ddclient -daemon=0 -debug -verbose -noquiet to confirm it is working.
- check the option of Mail Routing: I have mail server with another name and would like to add MX hostname...
- In 'MX hostname' entering aspmx.l.google.com
- In 'Primary' choose 'Yes, use it as my primary mail relay.'
After that, see http://yaui.me/postfix-gmail-smtp-server-relay-ubuntu/ for setup on Ubuntu machine.
no-ip
Similar to Dyndns. It has its own client program. Needs to build it yourself.
Also see the troubleshooting guide.
See http://ducky-pond.com/posts/12 for instruction of setting autostart on Debian system.
See http://www.coulterfamily.org.uk/pages/PCs/Linux/FAQ-LINUX-NO-IP-CLIENT.php for another approach.
Note: If noip2 cannot start automatically or noip2 does not update even it can be seen from ps -ef command, use sudo crontab -e command. For some reason, after I use sudo crontab, noip2 can update IP. So the only problem right now is it cannot update every 30 minutes even sudo noip2 -S says so. The problems may be 1. ps -ef shows the command runs from nobody user 2. sudo noip2 -S says it updates every 30 minutes via /dev/eth0 with NAT enabled.
Update: An alternative is to use ddclient. However, ddclient never updates the IP.
- Allow only one MX record for each host for free no-ip account.
- Click Host/Redirects > Manage Hosts > Modify.
webmin
See http://www.webmin.com/deb.html
wget http://prdownloads.sourceforge.net/webadmin/webmin_1.600_all.deb dpkg --install webmin_1.600_all.deb
The install will be done automatically to /usr/share/webmin, the administration username set to root and the password to your current root password. You should now be able to login to Webmin at the URL http://localhost:10000/. Or if accessing it remotely, replace localhost with your system's IP address.
Install Virtualbox on ubuntu
See here.
Remote desktop
Remote desktop connection from Windows 7
Install xrdp on Ubuntu. That's it. See
- http://www.ubuntututorials.com/remote-desktop-ubuntu-12-04-windows-7/
- https://community.hpcloud.com/article/using-windows-rdp-access-your-ubuntu-instance
sudo apt-get install xrdp sudo /etc/init.d/xrdp start
Remote desktop connection to Windows 7 from xubuntu
sudo apt-get update sudo apt-get install rdesktop rdesktop xxx.xxx.x.x -f -u USERNAME -p PASSWORD rdesktop 192.168.1.4 -g 1280x720 -u USERNAME -p PASSWORD
where -f option means full screen and -g means geometry.
Remote desktop connection to NCI
ts.nci.nih.gov:1494
DOMAIN: NIH
resolution: 1024 x 768
Use remmina or rdesktop or freerdp (sudo apt-get install freerdp-x11)
rdesktop ts.nci.nih.gov:1494 -d NIH -u XXXXX -g 1024x768
Remote desktop connection to Ubuntu
Network -> Remmina. Protocol: VNC-virtual network. Input IP, username and password.
Allow for remote desktop connection
- Go to System -> Preference -> Remote desktop. Allow other uses to view your desktop & uncheck you must confirm each access & require user to enter this password.
- Go to System -> Preference -> Monitor. Change monitor resolution to 1280 x 720.
Allow for remote desktop connection when vino failed
Try teamviewer. It works fine. After launching it, the software automatically creates an ID and password. We can change the password so it is fixed. Then launch the software on the client. Use the partner's ID and password to connect to it.
Remote desktop connection through vmware workstation
We shall be able to remote desktop connect to a Windows guest machine if the guest machine has configured to use bridge connection and a static IP. If there is a problem, it is likely caused by Window's firewall. See the two screenshots. We can just turn off the firewall of home network but keep the firewall on for the public network.
We don't need to use port forward for the remote desktop connection.
Instal sshd
apt-get update apt-get install openssh-server
Install LAMP
See the page [1]
apt-get install apache2 a2enmod rewrite apt-get install mysql-server [I choose branch name as MYSQL root password] apt-get install php5 php-pear php5-suhosin apt-get install php5-mysql /etc/init.d/apache2 restart
Another instruction including how to set up user directories for Apache web server http://wiki.debian.org/LaMp
apt-get install mysql-server mysql-client mysql_secure_installation apt-get install apache2 apache2-doc apt-get install php5 php5-mysql libapache2-mod-php5 apt-get install python libapache2-mod-python
The apache configuration file is in /etc/apache2/apache2.conf.
For PHP, it is also useful to install php for command line.
sudo aptitude install php5-cli
Restarting apache before testing on web browser
/etc/init.d/apache2 restart
Install mediawiki using tar ball
http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu
Install mediawiki using aptitude
And also the link http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Debian_GNU/Linux
aptitude install mediawiki php5 apache2 mediawiki-extensions libapache2-mod-php5
This will install latex. After the end, we can use Synaptic package manager to see what were installed. Now following the instruction in https://help.ubuntu.com/community/MediaWiki, we remove the '#' from the third line so that it reads 'Alias /mediawiki /var/lib/mediawiki':
sudo nano /etc/mediawiki/apache.conf sudo /etc/init.d/apache2 restart
Now we can start mediawiki by opening a browser and pointing it to http://localhost/mediawiki.
We need to enter
Site config admin username: WikiSysop password: Database config Database name: wikidb DB username: DB password: Superuser name: root Superuser password: [depend on how it was chosen when installing MYSQL]
Press the button of "Installing mediawiki". We will be welcomed to the wiki page. Following the instruction,
For security reason, I remove new account creation and anonymous editing. I also remove edit counters.
Backup and Restore mediawiki
See docs.google.com note. The process involves 3 parts: mediawiki system, mysql and images.
Install moinmoin
- Comparison of mediawiki vs moinmoin http://www.wikimatrix.org/compare/MediaWiki+MoinMoin
- Comparison of wiki software http://en.wikipedia.org/wiki/Comparison_of_wiki_software
- Moinmoin website http://moinmo.in/
- moinmoin was used by Bioconductor, ubuntu, etc.
MYSQL security
Just execute mysql_secure_installation from the command line.
- You can set a password for root accounts.
- You can remove root accounts that are accessible from outside the local host.
- You can remove anonymous-user accounts.
- You can remove the test database, which by default can be accessed by anonymous users.
See http://www.mysql-optimization.com/mysql-secure-installation-program.html
Compiling R
Use the following command to download required components before building any R packages. See also ubuntu package for R
sudo apt-get build-dep r-base
Package maintenance
https://help.ubuntu.com/community/AptGet/Howto#Maintenance_commands
Show package information
apt-cache show <package_name>
List files in a package
dpkg -L <package_name>
List racing games package (kind of search packages by key words)
apt-cache search racing game
List all packages installed in the system
apt-cache pkgnames
List of installed packages
dpkg --get-selections
List of available (uninstalled) packages
aptitude -F "%p" search "?not(?installed)"
View logs
- Apache /var/log/apache2/access.log
Torrent
Popular search sites
- torrentz
- thepiratebay
- isohunt
- kickass torrents
Torrent client: transmission-cli
Torrent client command line: rtorrent
We first need to create an .rtorrent.rc file under $HOME directory. Then run
rtorrent XXX.torrent
- https://wiki.archlinux.org/index.php/RTorrent
- http://harbhag.wordpress.com/2010/06/30/tutorial-using-rtorrent-on-linux-like-a-pro/ [download a template]
- http://mylinuxbook.com/rtorrent-bit-torrent-client/
In the simplest case, .rtorrent.rc looks like
download_rate = 0 upload_rate =50 directory = ~/Downloads session = ~/Downloads/rsession port_range = 55556-55560 scgi_port = 127.0.0.1:5000 use_udp_trackers = yes encryption = allow_incoming,try_outgoing,enable_retry
As you can see here, I have created a sub-directory rsession under ~/Downloads/.
Keybinding:
- ctrl + q = quit application
- ctrl + s = start downloading
Torrent client command line: aria2c
http://linuxconfig.org/aria2-all-in-one-command-line-download-tool
sudo apt-get install aria2 aria2c magnet:?xt=urn:btih:1e99d95f1764644a86a8e99bfd80c ...
Install webmin
apt-get install perl libnet-ssleay-perl openssl libauthen-pam-perl libpam-runtime libio-pty-perl apt-show-versions python wget http://prdownloads.sourceforge.net/webadmin/webmin_1.600_all.deb dpkg --install webmin_1.600_all.deb
File does not exist: /var/www/favicon.ico from /var/log/apache2/error.log
The reason? You have not created a favicon, also known as a website icon, for your website. It’s the icon that displays in the address bar of your web browser when you connect to a website. A web browser will request this icon file from every website.
If you choose to create one. Use a program such as Gimp and create a 16×16 pixel image and save it as a .ico filetype. Then upload that file to the DocumentRoot of your website. You will need one for each VirtualHost. If you don’t have Gimp, there are online resources such as favicon.cc where you can create a .ico file and download it for your own use.
As you know by now, not having a favicon.ico file, won’t stop web browsers from requesting it each time. But you can tell Apache not to log the event as an error message. You will still see the request in the access.log, but at least you will have a cleaner error.log file.
Add the following block of code to each VirtualHost, or at least the ones which don’t have a favicon file.
Redirect 404 /favicon.ico <Location /favicon.ico> ErrorDocument 404 "No favicon" </Location>
Don’t forget to restart apache after making the change. If you want make a “global” change, which would apply to any and all VirtualHosts, you can create a file in Apache’s conf.d folder with a name such as nofavicon.conf and then add that block of code to the file. That would disable favicon across the board and save you from having to edit each VirtualHost.
Or, you create an empty file with the name “favicon.ico” in the directory root of Apache (for exemple /var/www/).
Show weather on the taskbar
http://www.noobslab.com/2012/10/important-thingstweaks-to-do-after.html
Screencaster
- RecordMyDesktop: http://www.youtube.com/watch?v=A0Tn3Z8OklQ. We need to run ffmpeg to convert video to flv (Quality seems to be reduced) OR we can use online service (http://video.online-convert.com/convert-to-mp4) to convert ogv file to mp4 file (Same quality as I can tell).
- https://wiki.ubuntu.com/ScreenCasts
- Create animated Gif of a screencast
- kazam
Watch TV
Running concrete5 On Nginx (LEMP)
http://www.howtoforge.com/running-concrete5-on-nginx-lemp-on-debian-squeeze-ubuntu-12.10
Open mms stream in google chrom in linux
http://www.thermetics.net/2011/12/07/how-to-open-mms-links-from-chrome-under-ubuntu/
Download mms stream (suitable if mms is an extension; for example, studioclassroom)
- Use mimms (will NOT output to speaker at the same time; so is best in terms of performance)
sudo apt-get install mimms mimms -t 60 mms://example.com/video
where -t option specifies number of minutes. See http://linuxers.org/howto/how-download-mms-streaming-videos-ubuntu The output will have the same extension as the input. But it is not always clear. For example
$ mimms -t 3 mms://bcr.media.hinet.net/RA000073 mms://bcr.media.hinet.net/RA000073 => RA000073.wmv 1.48 MB / ∞ B (8.79 kB/s, ∞ s remaining) Download stopped after user-specified timeout.
- Use mplayer (will output to speaker at the same time, so suffer from performance, anyway do not get output)
mplayer mms:/link/something.xxx -dumpstream -dumpfile file.xxx
mms:/link/something.xxx - link to the stream you wish to download file.xxx - file to which you wish to download the stream, be careful to write the same extension xxx
Wait for the file to download and that's it. See http://ubuntuhowtos.com/howtos/download_mms_stream
Record audio out from command line
http://www.pantz.org/software/alsa/recording_sound_from_your_web_browser_using_linux.html
Step 1: Install required programs
sudo apt-get install gnome-media pavucontrol lame
Step 2: Create a script file <recordfm.sh>
#!/bin/bash
# Get pulseaudio monitor sink monitor device then pipe it to
# sox to record wav, lame to encode to mp3, or flac to encode flac
FILENAME="$1"
STOPTIME="$2"
# Encoding options for lame and flac.
LAMEOPTIONS="--preset cbr 192 -s 44.1"
if [ -z "$FILENAME" ]; then
echo -e "
Usage: $0 /path/to/output.mp3
Usage: $0 /path/to/output.mp3 stopinseconds" >&2
exit 1
fi
# Get sink monitor:
MONITOR=$(pactl list | egrep -A2 '^(\*\*\* )?Source #' | \
grep 'Name: .*\.monitor$' | awk '{print $NF}' | tail -n1)
echo "set-source-mute ${MONITOR} false" | pacmd >/dev/null
# Record it raw, and pipe to lame for an mp3
echo "Recording to $FILENAME ..."
if [[ $FILENAME =~ .mp3$ ]]; then
if [ -z $STOPTIME ]; then
parec -d $MONITOR | lame $LAMEOPTIONS -r - $FILENAME
else
echo -e "\nStopping in $STOPTIME seconds"
parec -d $MONITOR | lame $LAMEOPTIONS -r - $FILENAME 2>&1 &
SPID=$!
sleep $STOPTIME
kill -9 $SPID
fi
fi
Step 3: play the music or launch a browser with a desired url.
Step 4: run the bash script
chmod +x recordfm.sh ./recordfm.sh test.mp3 10
where <test.mp3> is the output filename and 10 is recording length (seconds). It works.
Note the script teaches us how to find out the ID for a process we just launched (cleaner than using ps -ef | grep commands). The command is
SPID=$! echo $SPID
Use VLC to record internet radio (suitable if the stream is continuous)
sudo apt-get install vlc browser-plugin-vlc
sudo apt-get install ubuntu-restricted-extras
sudo apt-get install lame libmp3lame0 sudo apt-get install ffmpeg sudo apt-get install libavcodec-extra-53 libavdevice-extra-53 libavfilter-extra-2 libavformat-extra-53 libavutil-extra-51 libpostproc-extra-52 libswscale-extra-2
A successful run will have an output like
VLC media player 2.0.3 Twoflower (revision 2.0.2-93-g77aa89e) [0x2329ca8] dummy interface: using the dummy interface module... [0x7fac2c007428] mux_dummy mux: Open [0x7fac2c003598] access_mms access: selecting stream[0x1] audio (39 Kib/s) [0x7fac2c003598] access_mms access: connection successful [0x7fac2c003598] access_mms access error: failed to send command [0x7fac2c005fe8] idummy demux: command `quit'
CVS server
# Original data dir: /home/mli/Downloads/hmv_arc
# CVS dir on server (CVSROOT): /home/mli/cvsrep
# CVS dir on local: /home/mli/Downloads/localcvs
# Project name: mycvs
# cvs user name: mli
sudo apt-get install cvs
mkdir ~/cvsrep
export CVSROOT=/home/mli/cvsrep
cvs init
sudo groupadd mycvsgrp
sudo useradd -G mycvsgrp mli # Assume mli is a new user
sudo usermod -a -G mycvsgrp mli # assume mli is an existing user
groups mli # view groups a user is in use
sudo chown -R :mycvsgrp /home/mli/cvsrep # change the group ownership of cvsrep directory to mycvsgrp.
cd ~/Downloads/hmv_arc
cvs import -m "initial" mycvs mli START # import files to CVS repository
# the new subfolder mycvs has owner mli.mli
Note 1. It is OK to use the same CVSROOT for multiple modules/projects since each module/project will be saved under a separate subfolder.
Note 2. The cvs version that I have installed in my ubuntu server is 1.12.13.
$ cvs -v Concurrent Versions System (CVS) 1.12.13-MirDebian-6 (client/server)
Quick test to checkout project to the same machine
cd ~/Downloads mkdir localcvs cd localcvs cvs checkout mycvs
This will create a new subfolder 'mycvs' under ~/Downloads/localcvs.
CVS repository on server has a structure
cvsrep/CVSROOT cvsrep/mycvs
CVS sandbox on local machine has a structure
mycvs/CVS mycvs/[files1] mycvs/[files2]
CVS client (ubuntu)
cvs -d :ext:[email protected]:/home/mli/cvsrep checkout mycvs
OR
export CVSROOT=:ext:[email protected]:/home/mli/cvsrep cvs checkout mycvs
To add/commit a new file
cvs add mynewfile cvs commit -m "my log message" mynewfile cvs update filename
The file's encoding is correct for 'hmv_.rc' file.
brb@ubuntu:~/Downloads$ cvs -d :ext:[email protected]:/home/mli/cvsrep checkout mycvs brb@ubuntu:~/Downloads$ file -bi mycvs/src/hmvUnicode.rc text/x-c; charset=utf-16le
CVS client (windows)
I use WinCVS for a demonstration
- Remote -> checkout module
- Module name: mycvs
- CVSROOT:
- protocol: ssh
- repository path: /home/mli/cvsrep
- user name:
- host name: taichi.selfip.net
If I use TortoiseCVS (1.12.5 from 1/24/2011), I need to choose ext as protocol instead ssh. Still the checked out file 'hmv_.rc' still contains unreadable Chinese characters. The cvsnt is the latest free version (2.5.05). If I want to use WinCVS + cvsnt from TortoiseCVS, the options in the CVSROOT dialog looks weird and cannot create a connection.
For the unicode encoding. If I commit the file at first from ubuntu os, but check out in Windows. The checked out file has right encoding (using Notepad ++, or from VS2010). However, the file does not have right line ending and it shows Chineses character when I open it in either Notepad++ or VS2010.
To see hidden characters in Linux, try either one of the following 2 methods:
- Open the file in EMACS and do a M-X hexl-mode
- geany editor.
The solution I have found to overcome accessing unicode (utf-16) file on Windows OS is using Cygwin.
- Download setup.exe from http://cygwin.com/install.html
- Root directory = c:\cygwin
- Local package directory = C:\Users\brb\Downloads
- Direct connection
- Download site: ftp://cygwin.mirrors.pair.com (Some mirrors are not updated & contain old version of packages! For example, make sure the cvs version is 1.12.13.)
- Search: cvs. Click plus sign next to "Devel". Click 'Skip' corresponding to cvs package.
- Search: ssh. Click plus sign next to "Net". Click 'skip' correspond to openssh package.
- Click 'Next' button.
- Click 'Finish' button.
- Now open 'Cygwin Terminal' icon on Windows Desktop.
export CVSROOT=:ext:[email protected]:/home/mli/cvsrep cvs checkout mycvs
The 'mycvs' directory should be under C:\cygwin\home\brb (a.k.a. /home/brb in cygwin) directory. We can open 'hmv_.rc' file in Notepad++ to double check if the file looks normal OR use md5sum to check.
Difference between CRLF (Windows), LF (Linux, Mac) and CR
This is a good summary I found: http://stackoverflow.com/questions/1552749/difference-between-cr-lf-lf-and-cr-line-break-types
- The Carriage Return (CR) character (0x0D, \r) moves the cursor to the beginning of the line without advancing to the next line. This character is used as a new line character in Commodore and Early Macintosh operating systems (OS-9 and earlier).
- The Line Feed (LF) character (0x0A, \n) moves the cursor down to the next line without returning to the beginning of the line. This character is used as a new line character in UNIX based systems (Linux, Mac OSX, etc)
- The End of Line (EOL) character (0x0D0A, \r\n) is actually two ASCII characters and is a combination of the CR and LF characters. It moves the cursor both down to the next line and to the beginning of that line. This character is used as a new line character in most other non-Unix operating systems including Microsoft Windows, Symbian OS and others.
Common CVS commands
- http://ximbiot.com/cvs/manual/
- http://mu2e.fnal.gov/public/hep/computing/cvsCheatSheet.shtml
- http://refcards.com/docs/forda/cvs/cvs-refcard-a4.pdf, http://www.bravegnu.org/cvscheat/cvscheat.pdf
cvs checkout MODULE cvs checkout DIR cvs checkout DIR/SUBDIR cvs co DIR/SUBDIR/FILENAME # check out a specific tag and put it in a specified directory. # the specified directory name will replace the module name in output. mkdir localcvs cvs checkout -r v4_3 -d localcvs MODULE cvs add myfile.c cvs add -kb myfile.bin # If you accidentally add a file, simply skip the commit for that file. cvs update –dA DIR/SUBDIR # -d: Create any directories that exist in the repository if they're missing from the working directory. # -A: Reset any sticky tags, dates, or -k options. Needed after you use "cvs update -D" or "cvs update -r". cvs update –A DIR/SUBDIR/FILENAME cvs commit cvs commit –m "add test suite" DIR/SUBDIR/FILENAME mkdir ~/original touch ~/original/newfile cvs import ~/original VENDORTAG RELEASETAG mkdir ~/localcvs cd ~/localcvs cvs checkout common/too cvs diff -r1.23 -r1.24 SUBDIR/FILENAME # Difference between specified versions 1.23 & 1.24. cvs diff -D "1 hour ago" MODULE cd LOCALCVS; cvs diff cvs checkout -D "1 hour ago" MODULE cvs checkout -D "2013-02-27 01:30" MODULE rm file(s); cvs remove file(s); cvs commit -m "Comment text" file(s) # You must rm the file before issuing the cvs remove command. The remove is not final until the commit has been issued. # cvs does not let you remove directories. However it does let you ignore any directories that are empty. cvs co -P Offline cvs update -PdA # P: Prune empty directories. d: create new directories. A: Reset sticky tags cvs history -c -a -D "1 day ago" -z "-0500" # find all changes submitted to the repository by anyone in the past day. # -z is used to adjust the time zone. cvs history -c -a -D "1 day ago" -f Mu2eG4/src # find all changes submitted to Mu2eG4/src (or any other subdirectory) by anyone in the past day cvs history -c -u USER "1 day ago" -f Mu2eG4/src # find all changes submitted by USER to Mu2eG4/src in the past day cvs log FILENAME
Bypass SSH password login (convenient for CVS, git etc)
- ssh-keygen -t rsa
- (make sure the remote server has .ssh directory)
- cat ~/.ssh/id_rsa.pub | ssh user@hostname 'cat >> .ssh/authorized_keys'
- ssh user@hostname
It helps with CVS log in too when the CVS works by using ssh protocol. Note that step 3 allows to run a shell command at a remote machine.
See https://help.github.com/articles/generating-ssh-keys also for similar instruction when work on github.
The ssh key can be copied to another a machine (pay attention to mode). Or let the new machine to create its own key pair and use ssh-copy-id to append the identity file to remote machine's ~/.ssh/authorized_keys file. See http://superuser.com/questions/332510/how-to-transfer-my-linux-ssh-keys-to-another-machine.
We can even have multiple ssh key on local machine by using. ssh/config file. See http://www.karan.org/blog/index.php/2009/08/25/multiple-ssh-private-keys.
Install Ubuntu to a USB flash drive
The following approach assumes the boot loader will not be put on internal hard disk if you are careful enough. See also http://fernhilllinuxproject.com/guidesandhowtos/installubuntutousbdrive.html
- Boot from live CD in ubuntu.
- Insert the USB flash drive in USB Port.
- On desktop double click on icon 'install ubuntu 12.04'
- Click on continue and it will ask if you want to unmount the flash drive click on yes
- Choose some thing else when ask to where to install ubuntu.
- Next you will see your hard disk partitions and flash drive. Click on flash drive partition and then click on change. Change the partition type to ext4 and mount as to '/'. Click on ok to close the dialogue box.
- In last you will see a drop down menu on where to install the the boot loader. Initially it will be showing sdc,sdd but on drop down you must select sdc1 or sdd1. Failing this step you may render your computer unbootable.
- Then click on install and linux will be installed on your USB Flash drive.
Install a new hard drive
https://help.ubuntu.com/community/InstallingANewHardDrive
- Use sudo fdisk command to create partition table. Then 'n', 'p', '1', 'w' and several returns.
- Use sudo mkfs -t ext4 /dev/sdb1 to create a new partition
- Use sudo mkdir /mnt/ssd to create a new mount point
- Use sudo nano -Bw /etc/fstab to do auto mount on boot
- sudo chown -R USERNAME:USERNAME /mnt/ssd to give the ownership to the USERNAME.
Building a simple Beowulf Like Cluster with Ubuntu
MPICH2
Resource:
- http://byobu.info/article/Building_a_simple_Beowulf_cluster_with_Ubuntu/
- https://help.ubuntu.com/community/MpichCluster
Here is my record for creating a cluster environment based on ubuntu 13.04. The master node is running on ubuntu 13.04 desktop with virtualBox 4.2. The virtualBox has added a host-only adapter (vboxnet0) with ip 192.168.56.1. This adapter will be added to the master node so I can use this ip to identify the master node in the host-only network. Creating cluster using VirtualBox is just for the education purpose, not for real practice.
- (virtualBox) Create two virtual machines running on ubuntu 13.04 server. The ssh server was checked during installation. The host name for each of them is ubuntuNode1 and ubuntuNode2 respectively. The network adapter is left by default (NAT) during installation. But after the installation is done, I shutdown the system and add a host-only adapter (vboxnet0) to each of them. Then after the system is up again, I change the IP so it is static. Do sudo nano /etc/network/interfaces and append the following before running sudo /etc/init.d/networking restart to take the change in effect.
auto eth1 iface eth1 inet static address 192.168.56.101 # use 192.168.56.102 for ubuntuNode2 netmask 255.255.255.0 network 192.168.56.0 broadcast 192.168.56.255
Note that it is better not to add host-only network before installation, or during installation it will ask what is the primary network (confusing). So at the end each new nodes should have both eth0 and eth1 adapters and they should not reside in same subset. ifconfig may not show all adapters so we should use ifconfig -a instead. Although host-only network is used for communication between guest and guest OR guest and host, since each guest node has NAT adapter by default so the guest machine can still access the internet.
- (master node) Edit /etc/hosts so it like like
127.0.0.1 localhost 192.168.56.1 ubuntu1304 192.168.56.101 ubuntuNode1 192.168.56.102 ubuntuNode2
Note that the master node will be used to start jobs on the cluster although it is OK to let the master node as one of nodes to execute the jobs.
- (all nodes) Run
sudo adduser mpiuser --uid 999
It is recommeneded to use the same password for the user. This will create a new directory /home/mpiuser. This is the home directory for user mpiuser and we will use it to execute jobs on the cluster.
- (master node) Run
sudo apt-get install nfs-kernel-server
- (other nodes) Run
sudo apt-get install nfs-client
- (master node) Add the following to the file /etc/exports
/home/mpiuser *(rw,sync,no_subtree_check)
or something like
/home/mpiuser 192.168.56.0/24(rw,sync,no_subtree_check)
Some people create a shared folder under /srv directory.
Now run
sudo service nfs-kernel-server restart
- (master node)
sudo ufw allow from 192.168.56.0/24
- (other nodes)
sudo mount ubuntu1304:/home/mpiuser /home/mpiuser
And if we want to mount the NFS shared directory when the compute nodes are booted, edit /etc/fstab by adding
ubuntu1304:/home/mpiuser /home/mpiuser nfs
- (master node)
sudo apt-get install ssh su mpiuser ssh-keygen ssh-copy-id localhost
We can test if the ssh works without passwords
ssh ubuntuNode1 echo $HOSTNAME
- (all nodes)
sudo apt-get install mpich2 which mpirun which mpiexec
- (master node, mpiuser)
Go to the home directory of mpiuser and create a new file hosts. Include host names for computing nodes (it is OK to include master node, ubuntu1304, as one of computing nodes)
ubuntuNode1 ubuntuNode2
- (all nodes)
sudo apt-get install build-essential
- (master node, mpiuser). Creating a test file <mpi_hello.c>
#include <stdio.h>
#include <mpi.h>
int main(int argc, char** argv) {
int myrank, nprocs;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
printf("Hello from processor %d of %d\n", myrank, nprocs);
MPI_Finalize();
return 0;
}
And compile and run it by
mpicc mpi_hello.c -o mpi_hello mpiexec -n 3 -f hosts ./mpi_hello
We should see something like
Hello from processor 0 of 2 Hello from processor 1 of 2
Successful! That's it.
OpenMPI
- http://auriza.site40.net/notes/mpi/openmpi-on-ubuntu-904/
- http://particlephysicsandcode.wordpress.com/2012/11/04/installing-open-mpi-1-6-3-ubuntu-12-04-fedora/
- http://randomusefulnotes.blogspot.com/2010/12/setting-up-mpi-cluster-on-ubuntu.html
With R
- http://www.open-mpi.org/papers/tr-uni-muenchen-8991/parallelR_techRep.pdf
- http://www.cybaea.net/Blogs/R-tips-Installing-Rmpi-on-Fedora-Linux.html (Fedora)
- http://kiradi.blogspot.com/2011/10/high-performance-computing-with-openmpi.html (Ubuntu)
- http://cran.r-project.org/web/packages/doMPI/vignettes/doMPI.pdf
- http://biowulf.nih.gov/user_guide.html#parallel
Sharing internet via wifi
I cannot make it to work to share the internet via wifi on my Xubuntu 13.04. However the solution in http://askubuntu.com/questions/287251/creating-an-infrastructure-hotspot-using-ubuntu-12-10 works for me.
$ sudo su - # add-apt-repository ppa:nilarimogard/webupd8 # aptitude update # aptitude install ap-hotspot # ap-hotspot configure # ap-hotspot start
In the step of 'ap-hotspot configure' it will ask for Access Point name and WPA passphrase.
When I run the final line 'ap-hotspot start', it will ask me to disconnect my current wifi first.
At the end, I found my original configuration in ubuntu does not work even its setting is there. apt-hotspot create an infrastracture AP instead of adhoc.
~/.xsession-errors file is filling the hard disk
It is related to vino-server. See
- http://askubuntu.com/questions/130768/my-home-partition-slowly-fills-up-until-the-system-is-unable-to-complete-even-si
- http://filthypants.blogspot.com/2013/02/xsession-errors-log-filling-hard-drive.html
Short solution is to use "kill -9 xxx" to kill the process and rm to remove ~/.xsession-errors file. The long time solution is to uninstall vino.
JRE and JDK
Install openjdk or Sun jdk. See http://www.maketecheasier.com/install-java-runtime-in-ubuntu/
If we have multiple versions of JRE/JDK, we can use the following command to set the default version
sudo update-alternatives --config java
This approach seems to be working in the case JAVA_HOME cannot be honored.
CPU-G
See this instruction to install CPU-G program for viewing hardware information (process, mb, graphic, memory, system).
Unlock keyring
I got the prompt of unlocking keyring every time I open google chrome browser.
- http://askubuntu.com/questions/867/how-can-i-stop-being-prompted-to-unlock-the-default-keyring-on-boot (this works)
- http://askubuntu.com/questions/184266/what-is-unlock-keyring-and-how-do-i-get-rid-of-it (this does not help)
It is also helpful to remove ubuntu one from Ubuntu 12.04. See http://hex.ro/wp/blog/removing-ubuntuone-from-ubuntu-12-04/
nmap
nmap - Network exploration tool and security / port scanner
sudo apt-get install nmap nmap -T4 -F 192.168.1.99-255
Google Drive
sudo add-apt-repository ppa:thefanclub/grive-tools sudo apt-get update sudo apt-get install grive-tools
On Dash, type grive. It will ask you to enter a sequence of characters. After that, a new folder 'Google Drive' will be created and files will be downloaded.
- http://www.thefanclub.co.za/how-to/ubuntu-google-drive-client-grive-and-grive-tools
- http://www.thefanclub.co.za/node/151
Edit a video using OpenShot
OpenShot software. An article from PC World.
sudo apt-get install openshot
When I needed to export the video (choose 'youtube' profile, 'youtube-HD' target, 'HD 270, 29 .97 bps', and 'high' quality), I found I need to install libx264 code. On Ubuntu, I open software center and seach 'libavformat'. I choose 'libavformat-extra-53'.
Download Youtube video
sudo apt-get install youtube-dl
To convert the video to mp3 format, use soundconverter
sudo apt-get install soundconverter
Create a desktop shortcut
Navigate to your application in Nautilus. Right-click, select "Make Link". Then drag shortcut to your desktop. Works in Ubuntu 12.04.
Create Unity Launcher
Take RStudio for example,
- Desktop icons are saved under /usr/share/icons/hicolor/16x16/apps/rstudio.png, where other than 16x16 there are also icons from 24x24, 32x32, 48x48 sizes (directories). Another place is ~/.local/share/icons/hicolor/16x16/ .
- Unity launcher is created at /usr/share/applications/rstudio.desktop or ~/.local/share/applications/XXXX.desktop.
- Desktop shortcut can be created by copy XXXX.desktop to ~/Desktop directory.
The icon size in the launcher can be adjusted by opening System Settings -> Apperance -> Launcher Icon Size (at the bottom).
k2pdfopt has an example how to do it for creating a desktop shortcut and be in the right click menu.
What date was the system installed
ls -l /var/log/installer
What is the last log in time for users
lastlog
What is the reboot time
last reboot
Crop an image
sudo apt-get install gthumb
Open an image. Click color palette icon on top-right corner (or use keyboard shortcut 'e'). Click 'crop'. There will be a rectangle on image where you can resize the rectangle.
Mind-mapping
Update Firefox
See https://help.ubuntu.com/community/FirefoxNewVersion. The following instruction is used to get security-testing packages.
sudo add-apt-repository ppa:ubuntu-mozilla-security/ppa sudo apt-get update sudo apt-get install firefox
Change default audio player
Right-click an MP3 file, and from the menu select “Properties”. In the window that appears, go to the “Open With” tab and select VLC or whatever. Click the “Set As Default” button to set VLC as the default player.
You might also want to set the default player in the sound menu.
Screensaver
I like gluqlo (flip clock) screensaver. However, it is not easy to make it to replace the genome-screensaver.
sudo apt-add-repository ppa:alexanderk23/ppa sudo apt-get update sudo apt-get install gluqlo
- Follow the instruction to install it on Ubuntu machine.
- Follow the instruction there to remove genome-screensaver and install xscreensaver.
- Configure xscreensaver to use only 1 screensaver. Edit ~/.xscreensaver file and add a line like
gluqlo -root \n\
- Still follow the instruction to allow xscreensaver to start when the machine starts up. Don't try to edit ~/.xinitrc file as other sites suggested; start Dash and type 'startup' and follow the screen dialog to add xscreensaver -no-splash.
- Still follow the instruction to add lock screen keyboard shortcut.
RHEL/CentOS
Change hostname
- Change the ^HOSTNAME line in /etc/sysconfig/network
- Change the hostname in /etc/hosts
- Run /bin/hostname new_hostname for the hostname change to take effect immediately.
- Run /sbin/service syslog restart for syslog to log using the new hostname.
Note that using the command line 'hostname' to change the machine's hostname works only for the current session.
sudoer
Some distributions do not come with sudo command.
As root type:
visudo
and add a line
MyUserName ALL = ALL
Add an existing user to have sudo privilege
sudo adduser USERNAME sudo
See help.ubuntu.com.
What is my IP address
ifconfig eth0
What is my DNS server
/etc/resolv.conf
What services get started at boot time
chkconfig --list
Is xxx service running
xxx status
What services are currently running
ps -e
and
lsof -i
will show you services that are listening to TCP or UDP endpoints.
What network ports are open
lsof -i
or
netstat -aut
What firewall rules do I in place
iptables -L
What packages do I have installed
rpm -qa | less
or
rpm -qa | grep xxx
What version of package xxx do I have installed
rpm -qi xxx
List of available (uninstalled) packages
yum list available
List All Configured Repositories
yum -v repolist yum -v repolist | less yum repolist
To list only enabled repositores
yum repolist enabled
To list only disabled repositories
yum repolist disabled
To list available packages under a repo called ksplice-uptrack, enter:
yum --disablerepo="*" --enablerepo="ksplice-uptrack" list available
yum equivalent of apt-get update
yum check-update
Open a port
Edit the file /etc/sysconfig/optables. Create a new rule by following the examples of other rules in the file.
Power Manager for GNOME
The configuration defaults for GNOME power manager have not installed correctly. Cannot login
This error will results in a log-in problem except root account. The symptom is 50GB in root (/) is used up.
The problem was caused by a bug in yum where /var/cache/yum/x86_64/6Workstation takes about 42GB space. The 'yum' does not remove old generated .sqlite files.
See https://bugzilla.redhat.com/show_bug.cgi?id=632391
I use 'du -k' command to find out which directory took space. I use 'rm' command to delete the contents.
Even I delete the content, the directory still grows up daily.
Upgrade Python from 2.6.x to 2.7.x
This instruction tells how to install Python 2.7 from source.
yum -y update yum groupinstall -y 'development tools' yum install -y zlib-devel bzip2-devel openssl-devel xz-libs wget wget http://www.python.org/ftp/python/2.7.8/Python-2.7.8.tar.xz xz -d Python-2.7.8.tar.xz tar -xvf Python-2.7.8.tar # Enter the directory: cd Python-2.7.8 # Run the configure: ./configure --prefix=/usr/local # compile and install it: make make altinstall # Checking Python version: [root@nicetry ~]# python2.7 -V Python 2.7.8 wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz # Extract the files: tar -xvf setuptools-1.4.2.tar.gz cd setuptools-1.4.2 # Install setuptools using the Python 2.7.8: python2.7 setup.py install curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py | python2.7 - pip2.7 install virtualenv
Install Meld
Have not found a solution yet. We need to install it from source. However, the source depends on
- Python 2.7 (see above for the instruction)
- GTK+ 3.6
- GLib 2.34
- PyGObject 3.8
- GtkSourceView 3.6
VirtualBox guest addition
Check out this post.
su # click VirtualBox -> Devices -> Install guest addition mkdir /media/VirtualBoxGuestAdditions mount -r /dev/cdrom /media/VirtualBoxGuestAdditions rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm yum install gcc kernel-devel kernel-headers dkms make bzip2 perl KERN_DIR=/usr/src/kernels/`uname -r` export KERN_DIR cd /media/VirtualBoxGuestAdditions ./VBoxLinuxAdditions.run
(Update for 64-bit CentOS 6.5 + VirtualBox 4.3.18) The installation still failed and it showed a missing package which can be installed with
yum install kernel-devel-2.6.32-431.el6.x86_64
Then I re-run ./VBoxLinuxAdditions.run to finish the installation of guest addition. Reboot and GA works.
Online Tools
- Non-boring presentation: Prezi
- Flow chart/Mind-mapping: MindMup, LucidChart
- Image editor: Pixlr Editor (vs Pinta or Shotwell in Ubuntu)
- Video editor: WeVideo (vs OpenShot in Ubuntu)
- Virus : Virus Total
- Finance: Mint