[http://www.makeuseof.com/tag/linux-distro-space/ Install Linux to Save Space! These Tiny Linux Distros Are Super Small] 10/10/2017
[http://www.makeuseof.com/tag/linux-distro-space/ Install Linux to Save Space! These Tiny Linux Distros Are Super Small] 10/10/2017
== Debian/Ubuntu/Bio-Linux ==
=== Debian ===
==== Download Debian ====
Go to http://www.debian.org/distrib/ and download "Small CDs or USB sticks", for example ''debian-8.1.0-amd64-netinst.iso'' (Jessie, released June 2015). It is about 280 MB.
Screenshots of step-by-step installation can be found in [http://debian-handbook.info/browse/wheezy/sect.installation-steps.html here].
At the end of installation, it will offer a collection of software to install. Below 'Debian desktop environment', it has a selection of GNOME, Xface, KDE, Cinnamon, MATE and LXDE (new in Jessie/8.0). Note that the [http://cdimage.debian.org/debian-cd/8.0.0/amd64/bt-cd/ cd images download page] only provides a selection of kde, lxde and xfce desktop. The default is 'GNOME' in Jessie.
[[File:DebianInstall.png|200px]]
After installation, you got a desktop environment of Debian based on GNOME 3 (Virtual machine will use recovery mode, but still works. Some people suggest to install the guest additions (in the guest) and make sure that you enable the 3D acceleration in the guest settings.). Also 'free -m' command shows it uses 202 MB memory and the whole system takes up 3.3 GB. I am testing on a Chinese desktop environment.
When Debian is in recovery mode, the desktop interface is like old fashion. Application and Place on top of the screen. When GNOME 3 is working (in my test of Debian 7.1.0, I cannot run VBoxLinux.run, but it still works after I did other steps), the interface is sort of Ubuntu with application launched from the left hand side. It differs from Ubuntu because the side bar appears by clicking a 'preview' button on top left corner.
The default browser in Debian is Iceweasel with AdBlock Plus preinstalled.
I don't know why the default user does not have root privilege.
When I installed the Chinese version, the keyboard switch icon (SCIM) is automatically available. However once the desktop is in regular GNOME 3, the switch icon disappeared. Fortunately, we can use Ctrl + Space to switch languages. Thanks to the [http://verahill.blogspot.com/2012/01/debian-testing-64-wheezy-chinese.html hint] there.
==== How hard it is to install Debian ====
Tested on Lenovo thinkpad t420s. When it is about to detect network hardware, it popped up a message:
Some of your hardware needs non-free firmware files to operate. The firmware can be loaded from removable media, such as a USB stick or floppy.
The missing firmware files are: iwlwifi-6000g2a-6.ucode iwlwifi-6000g2a-5.ucode.
There is no a server version of iso to download. At the end of installation, it will ask what software to install: Debian desktop environment, Web server, SSH server, Laptop, SQL database, ... We can uncheck 'Debian desktop environment' item.
Compared to Desktop version, the server version takes 1.3GB space and 33MB memory.
The server version does not have 'sudo' command. Use 'su' to switch to 'root' user.
Note that even we installed 'sudo', we can not use 'sudo' from the default user. It will complain '''xxx is not in the sudoers file. This incident will be reported.'''.
==== Virtualbox guest addition installation ====
See [[Virtualbox#Debian|Virtualbox -> Debian]].
==== Browse iso files ====
Note that if we want to download the iso image, we should consider using the torrent method.
The can see a variety of download options from
http://www.debian.org/CD/ > Downloading Debian CD/DVD images via HTTP/FTP
The script uses /bin/sh as shell and on Debian Wheezy/Jessie /bin/sh is symlinked to /bin/dash. Dash is a more light-weight replacement for Bash, and it turns out that the VirtualBox script does not work as it should when run with "dash".
An easy workaround is to explicitly run the script with "bash" with the following command:
<pre>
sudo bash ./VBoxLinuxAdditions.run
</pre>
==== Browse source code ====
* http://archive.ubuntu.com/
* https://wiki.ubuntu.com/Kernel/SourceCode
* [[Linux#Count_number_of_rows_in_a_file_-_wc|Find a command's package name]]
Download links for all versions from [https://wiki.ubuntu.com/ wiki.ubuntu.com] > [http://releases.ubuntu.com/ releases.ubuntu.com].
Ubuntu [https://wiki.ubuntu.com/UbuntuFlavors flavors] and [http://www.ubuntu.com/about/about-ubuntu/derivatives derivatives]
For some reason, when I try the Ubuntu (13.04) live CD, the screen resolution looks perfect. But when I installed the OS, the screen resolution is always too low. The propriety graphics driver cannot be installed successfully. Fortunately, when I try the Kubuntu (13.04), the display resolution problem automatically works!
Update: Kubuntu failed to respond after I install SCIM related programs. A freshly installed linuxmint OS also has a similar problem that the desktop does not respond to mouse or keyboard. Luckily, the Xubuntu works fine and the Chinese input works out of box if I choose Chinese as desktop environment (339MB was used).
==== Download mirror ====
https://launchpad.net/ubuntu/+cdmirrors
For me, mirrors.acm.jhu.edu (only visible from Ubuntu's Software & Updates -> Download from, Ubuntu 16.04) or mirror.umd.edu are closest.
==== Installation ====
Installing Ubuntu (or xubuntu, Mint) still requires an internet connection for downloading '''language packs'''. ''This could be very time consuming.'' However, in the installation process I can click the 'skip' button to skip downloading language packs. This saves a lot of time when the internet connection is slow. After ubuntu desktop appears, it still pops up a message to give an instruction to install language packs.
The installation takes about 10 minutes when I installed ubuntu 14.04 (unity) on virtualBox.
===== How to upgrade =====
* [http://www.ubuntugeek.com/upgrade-ubuntu-desktopserver-from-16-04-xenial-xerus-to-16-10-yakkety-yak.html 16.04 to 16.10].
* [http://fridge.ubuntu.com/2016/04/21/ubuntu-16-04-lts-xenial-xerus-released/ New version of Ubuntu contains newer versions of Qt and GTK] and [http://planet.ubuntu.com/ planet.ubuntu.com].
===== Install via PXE Network Boot Server =====
[https://www.howtoforge.com/tutorial/install-debian-9-stretch-via-pxe-network-boot-server/ Install Debian 9 (Stretch) via PXE Network Boot Server]
==== End of life date of Ubuntu release ====
https://wiki.ubuntu.com/Releases
Each time I log into my Ubuntu 12.04.5 LTE, I'll receive a message
<pre>
New release '14.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Your current Hardware Enablement Stack (HWE) is no longer supported
since 2014-08-07. Security updates for critical parts (kernel
and graphics stack) of your system are no longer available.
For more information, please see:
http://wiki.ubuntu.com/1204_HWE_EOL
There is a graphics stack installed on this system. An upgrade to a
supported (or longer supported) configuration will become available
on 2014-07-16 and can be invoked by running 'update-manager' in the
Dash.
</pre>
===== Ubuntu 12.04 =====
As of Dec 1, 2015, [http://www.pcworld.com/article/3010404/browsers/googles-killing-chrome-support-for-32-bit-linux-ubuntu-1204-and-debian-7.html Chrome for 32-bit Ubuntu 12.04 is not supported] anymore.
==== Black screen on boot Ubuntu 14.04 ====
[http://ubuntuforums.org/showthread.php?t=2245141 Press e when you see the first menu].
==== Server version ====
[[File:UbuntuServerInstall.png|150px]]
==== [http://www.bandshed.net/AVLinux.html AV Linux] ====
AV Linux features a complete customized Debian Linux XFCE4 4.10 Desktop Environment with the added bonus of a handpicked selection of pre-tested and pre-configured Audio, Graphics and Video content creation software demonstrating the excellence of Open-Source and also includes many unique Commercial Demos.
* https://www.offensive-security.com/kali-linux-vmware-arm-image-download/ In addition to regular linux image, Kali provides images for VirtualBox, VMWARE and ARM devices like Raspberry Pi, Beaglebone Black, Chromebook, Odroid, et al.
==== Create customized ubuntu iso ====
* [http://www.ubuntu-mini-remix.org/ Ubuntu Mini Remix] (~200MB). Note that this is a live ubuntu which can't be installed even we can remaster it to include Desktop Environment, packages, et al. See [https://answers.launchpad.net/ubuntu-mini-remix/+faq/35 this FAQ].
** [http://sourceforge.net/projects/uck/ Ubuntu customization kit] - [http://www.linux.com/learn/tutorials/739139-roll-your-own-customized-ubuntu-with-uck linux.com] [http://www.howtogeek.com/109736/how-to-create-a-custom-ubuntu-live-cd-or-usb/ howtogeek]. The project has not been updated since 2013-01-16.
** [https://launchpad.net/ubuntu-builder Ubuntu Builder] - [http://lifehacker.com/5921054/ubuntu-builder-lets-you-build-your-own-customized-linux-distribution lifehacker]. It looks the project is abandoned.
** [https://github.com/fluxer/Customizer Customizer] - quite information from its website. The [https://github.com/clearkimura/Customizer/wiki/Manuals manuals/user guide 3.x] p46 talks about how to make the iso installable instead of just a live CD (e.g. '''apt-get install ubiquity ubiquity-frontend-gtk'''). '''N.B.''' ''Installing [https://en.wikipedia.org/wiki/Ubiquity_(software) ubiquity] should be run once we have installed all software we want; i.e. if we want to install xfce4 we should install xfce4 before we install ubiquity.'' Also for some reason, Customizer crashed when I tried to create an iso if I have installed xubuntu-desktop, ubiquity and ubiquity-frontend-gtk.
Note that the Ubuntu Mini Remix by default contains only 3 repositories. We may want to add some more.
<pre>
deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb http://security.ubuntu.com/ubuntu/ trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
</pre>
while for example an official v14.04 xubuntu contains 22 sources,
<pre>
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://us.archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ trusty main restricted
## Major bug fix updates produced after the final release of the
## distribution.
deb http://us.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://us.archive.ubuntu.com/ubuntu/ trusty universe
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
deb http://extras.ubuntu.com/ubuntu trusty main
deb-src http://extras.ubuntu.com/ubuntu trusty main
</pre>
* http://askubuntu.com/questions/409607/how-to-create-a-customized-ubuntu-server-iso It gives a long instruction based on command line.
* http://razvangavril.com/linux-administration/custom-ubuntu-server-iso/ The instruction is organized and is very similar to the above.
* http://amjjawad.blogspot.com/2013/07/ubuntu-mini-iso-installation-process.html It does not talk about creating a customized iso. It talks about how to install Ubuntu from the minimal CD (~40M). The minimal CD will download the packages in the installation process.
==== MultiSystem – Create a MultiBoot USB from Linux ====
* [https://itsfoss.com/multiple-linux-one-usb/ How to Install Multiple Linux Distributions on One USB] using [http://multibootusb.org/ MultiBootUSB] (Linux/Windows)
The minimal ubuntu iso is about 50MB. It will download files when we install the Ubuntu.
In the halfway of installing the minimal Ubuntu, there is a dialog called 'Software selection'. It says ''At the moment, only the core of the system is installed. To tune the system to your needs, you can choose to install one or more of the following predefined collections of software. Choose software to install:''.
In addition to some default selections (like 'standard system utilities'), I choose '''Ubuntu MATE minimal installation''' (not 'Ubuntu MATE desktop'). This
action will retrieve about 1228 files from the internet. After finishing install them, the installer also installed GRUB and set up system clock. Then the installation was complete. We have to reboot the system (for virtual machine case we need to power off the guest machine and remove the virtual drive).
For the ''Ubuntu MATE minimal installation'' selection, it still includes several software. The 'df' command shows 3.3GB space was used in this minimal Ubuntu MATE 16.04.
* Accessories: Character Map, Engrampa Archive Manager, Calculator, MATE Search Tools, Passwords and Keys, Pluma Text Editor, Take Screenshot
* Graphics: Eye of MATE image Viewer, MATE Color Selection, Simple Scan
* Internet: Firefox
* Office: Atril Document Viewer, MATE Dictionary
* Sound & Video: Sound
* System Tools:Avahi Zeroconf Browser, Caja, dconf Editor, GDebi Package Installer, Log File Viewer, MATE Disk Usage Analyzer, MATE System Monitor, MATE Terminal, Power Statistics
Actually, if we do not select ''Ubuntu MATE minimal installation'' but rather choose to install it later on from [http://askubuntu.com/questions/53822/how-do-you-run-ubuntu-server-with-a-gui the command line] ('''sudo apt-get install --no-install-recommends ubuntu-mate-core''') we still end up with the same Ubuntu MATE desktop environment (3.3GB).
==== The Perfect Server ====
* [https://www.howtoforge.com/tutorial/perfect-server-ubuntu-16.04-with-apache-php-myqsl-pureftpd-bind-postfix-doveot-and-ispconfig/ Ubuntu 16.04] from howtoforge.
* [https://www.howtoforge.com/tutorial/perfect-server-debian-8-4-jessie-apache-bind-dovecot-ispconfig-3-1/ Debian 8.4] from howtoforge.
==== Live USB with persistent storage ====
[http://www.howtogeek.com/howto/14912/create-a-persistent-bootable-ubuntu-usb-flash-drive/ Linux Live USB Creator]
=== Selection of [http://en.wikipedia.org/wiki/Desktop_environment desktop environment] ===
* https://help.ubuntu.com/community/Installation/LowMemorySystems How to install desktops from the command line.
* [http://askubuntu.com/questions/125062/how-can-i-find-which-desktop-enviroment-i-am-using To query the desktop environment] using the command line, use
Unity as the default user interface instead of GNOME Shell, beginning April 2011, with Ubuntu 11.04 according to the wikipedia.
Use 'unity --version' to check the unity version. If something was screwed up (eg after we remove gnome-desktop), we can reinstall the unity desktop by
<pre>
sudo apt-get install ubuntu-desktop
</pre>
Don't forget [https://github.com/freyja-dev/unity-tweak-tool Unity Tweak Tool]. That is required if you want to install, saying, [https://github.com/horst3180/arc-theme Arc theme]. I install it on Ubuntu 16.04 by running ''sudo apt install unity-tweak-tool''.
[[File:Unity tweak tool.png|150px]]
Note that there is no screensaver anymore starting with Ubuntu 12.04. Read [http://askubuntu.com/questions/292995/configure-screensaver-in-ubuntu this post]. If we want to add a screensaver program, read [http://www.howtogeek.com/114027/how-to-add-screensavers-to-ubuntu-12.04/ How to Add Screensavers to Ubuntu 12.04]
[http://ubuntugnome.org/ Ubuntu GNOME] (GNOME 3). The build-in screensaver is a digital clock showing the current time & date. Cool! This seems to be a new feature in GNOME 3.6 optimized for touch screen devices. See [https://help.gnome.org/misc/release-notes/3.6/users-lock-screen.html.en_GB this] and [http://www.linuxuser.co.uk/reviews/gnome-3-6-review-against-the-grain this].
Note that we can install the gnome desktop by using the command line. It will keep the current wallpaper. The clock in screensaver will not be shown until we shake the mouse or keyboard.
<pre>
sudo apt-get install gnome-shell
# Choose 'gdm' (Gnome Desktop Manager) as the display manager instead of 'lightdm' the Ubuntu's default
# when it is configuring gdm as only GDM offers GNOME-specific features such as lock-screen notifications.
If the lock screen does not work, use Settings > Brightness and Lock, or use the command line
<pre>
gsettings set org.gnome.desktop.lockdown disable-lock-screen 'false'
</pre>
If the screensaver is not working, try
<pre>
sudo apt-get install gnome-screensaver
</pre>
In my case, the screen turns off (black). But if we wake the PC up, the time and date screen shows up.
Some differences (inconvenience): 1. No maximize, minimize windows buttons 2. Have to click 'Activities' button (or 'Windows' key) to switch applications. These complains also appeared in other [http://www.linuxuser.co.uk/reviews/gnome-3-6-review-against-the-grain review].
[http://www.makeuseof.com/tag/xfce-explained-linux-speediest-desktop/ Xfce Explained: A Look at One of Linux’s Speediest Desktops] ''"Xfce doesn’t support HiDPI, which can be a deal breaker on newer machines. Even on a laptop with a 1920 x 1080 screen resolution, I find Xfce’s interface and windows to be absolutely tiny."''
[http://xubuntu.org/ Xubuntu]. The response is quicker when I compare the speed by clicking the top-left icon (app menu) in Xfce and Acitvities button in ubuntu-GNOME. This is tested when both Xubuntu and ubuntu-GNOME are installed in VirtualBox.
<syntaxhighlight lang='bash'>
# Note the sources.list should contain 'universe' repositories.
# Install XFCE alone, without Xubuntu, with this command:
sudo apt-get install xfce4
# Install the entire Xubuntu package, which includes a full suite of software and a lot of improvements:
sudo apt-get install xubuntu-desktop
#
# Note that installed terminal is XTerm and UXTerm which looks awful. We will want
# xfce4-terminal, a modern, lightweight and low memory cost terminal emulator for X11,
# which was included in the Xubuntu desktop.
sudo apt-get install xfce4-terminal
</syntaxhighlight>
The default display manager (used e.g. log in screen) can be found by
<syntaxhighlight lang='bash'>
cat /etc/X11/default-display-manager
</syntaxhighlight>
To install lightdm display manager
<syntaxhighlight lang='bash'>
sudo apt-get install lightdm
</syntaxhighlight>
After running the above command, I found 1. a GUI login screen came out, but login failed (my case). Use Ctrl+Alt+F1 to switch to the command line approach. 2. startx failed.
[http://lubuntu.net/ Lubuntu], [http://lxle.net/ LXLE] and [http://lxqt.org/ LXQt]. LXDE is the default desktop environment for Raspbian, LXLE, BBB, and Lubuntu.
The default browser in LXLE is '''SeaMonkey''' (Debian Jessie has '''Iceweasel''' which is even similar to Firefox; see [[Odroid#Minimal_Debian_Jessie|Odroid]]). In the Internet category, it also includes FireFTP (SeaMonkey extension), Transmission, Gitso (VNC), uGet. In the Sound/Video category, it includes Arista transcoder, Audacity, Guaydadeque Music Player, guncviewer, Pithos, RecordMyDesktop, Videos/Totem. Gedit, GParted, KeepassX and LibreOffice are standard. There is also a Games category.
The [http://www.lxle.net/forums/discussion/972/lxle-ls-first-beta-available LXLE LS] version provides several business apps to anyone on the network.
LXQt is the Qt port and the upcoming version of LXDE, the Lightweight Desktop Environment.
The download links (if we want to skip answering the question) are https://sourceforge.net/projects/lxle/files/Final/OS/.
[http://www.lxle.net/articles/ LXLE key features]
One cool thing in lxle is the random wallpaper. On regular Ubuntu, we can install a wallpaper changer program - [http://askubuntu.com/questions/134/how-do-i-create-a-desktop-wallpaper-slideshow Wallch] or [http://peterlevi.com/variety/how-to-install/ Variety]. See also [http://www.omgubuntu.co.uk/2016/12/8-bit-day-wallpaper-changes-day OMGUBUNTU].
[[File:Variety pref.png|150px]]
In Lxle, the random wallpaper is done through a simple command
[https://help.ubuntu.com/community/Lubuntu/Keyboard A list of keyboard shortcuts] for Lubuntu.
The file manager in LXLE is [https://wiki.archlinux.org/index.php/PCManFM PCManFM] (not to confuse with the package manager [https://wiki.archlinux.org/index.php/pacman '''pacman'''] in ArchLinux). It does not support drag and drop to copy a file name to a terminal but it can execute a file in a terminal by PCManFM -> Tools -> Run a Command in Current Folder.
[http://www.linuxmint.com/ Linux Mint]. A GTK+-based desktop environment. Note that [https://sites.google.com/site/easylinuxtipsproject/mint Mint releases only LTS versions] (5 years support as Ubuntu).
To check the desktop environment, use '''echo $DESKTOP_SESSION'''. To check the cinnamon version, use '''cinnamon --version'''.
Note
# To change to other workspace, use Ctrl + Alt + Up or Alt + F1 keybind to enter ''Expo'' mode and then select one. To directly switch to the next workspace, use Ctrl + Alt + Right/Left arrow key.
# To change the title bar color from gray to black follow [https://www.reddit.com/r/linuxmint/comments/3mv7w3/how_to_change_the_color_of_the_top_title_bar/ this instruction]. Go to Preferences -> Themes and click 'Add/Remove desktop themes'. Type the keyword 'nightfall' and install the theme. Close the 'Themes' window and open it again. Click 'Windows borders' and select 'nightfall'.
# Mint has its own package update manager. Click Menu > Administration > Update Manager. Security updates are level 5. Kernel updates are level 4/5. See their meaning on the [https://www.linuxmint.com/documentation.php official user's guide].
#* Level 1 and Level 2 updates are risk-free and you should always apply them. Level 3 updates “should be safe” but, although we recommend you take them, make sure you look over them on the list of updates.
#* Level 4 is "Unsafe packages". It could potentially affect the stability of the system". Level 5 is "Dangerous packages".
# We don't have to install Linux Mint in order to get cinnamon desktop. [http://www.webupd8.org/2016/04/how-to-install-cinnamon-30-in-ubuntu.html How To Install Cinnamon 3.0 In Ubuntu 16.04 Or 15.10 Via PPA]
# this Cinnamon desktop has a digital clock as a screen saver while the MATE desktop only has a black screen as a screen saver. I still get the digital clock as a screen saver when I install cinnamon desktop on top of regular Ubuntu.
# Restricted - Supported software that is not available under a completely free license.
# Universe - Community maintained software, i.e. not officially supported but enabled by default software.
# Multiverse - Software that is not free.
See the pitfall in the [http://www.pcworld.com/article/2845072/ubuntu-owncloud-and-a-hidden-dark-side-of-linux-software-repositories.html PCWorld] article.
==== Slow download ====
* [http://askubuntu.com/questions/37753/how-can-i-get-apt-to-use-a-mirror-close-to-me-or-choose-a-faster-mirror How can I get apt to use a mirror close to me, or choose a faster mirror?]
==== Check if a repository exists ====
For example, consider the CRAN repository at cran.rstudio.com server.
<syntaxhighlight lang='bash'>
if grep -q "deb http://cran.rstudio.com/bin/linux/ubuntu" /etc/apt/sources.list; then
echo http://cran.studio.com/bin/linux/ubuntu was found
Now run ''nano /etc/apt/sources.d'' to check if duplicated repository has been added.
==== GPG/Authentication key ====
* [https://linuxconfig.org/how-to-encrypt-and-decrypt-individual-files-with-gpg How to Encrypt and Decrypt Individual Files With GPG]
* [http://askubuntu.com/questions/131397/what-is-a-repository-key-under-ubuntu-and-how-do-they-work What is a repository key under Ubuntu and how do they work?]
* [https://wiki.archlinux.org/index.php/GnuPG GnuPG] from archlinux.org.
'''gpg (GNU Privacy Guard)''' is the tool used in secure apt to sign files and check their signatures. See https://help.ubuntu.com/community/SecureApt or https://wiki.debian.org/SecureApt.
'''apt-key''' is a program that is used to manage a keyring of gpg keys for secure apt. Note The keyring is kept in the file '''/etc/apt/trusted.gpg'''. Not to be confused with the related but not very interesting /etc/apt/trustdb.gpg.
sub 2048R/1CFF3E8F 2010-10-19 [expires: 2020-10-16]
/etc/apt/trusted.gpg.d/peterlevi_ubuntu_ppa.gpg
-----------------------------------------------
pub 1024R/A546BE4F 2012-06-28
uid Launchpad PPA for Peter Levi
</syntaxhighlight>
Note that the 3 commands we have use (gpg for import, gpg for export from your keyring, apt-key for adding) can be combined into one '''apt-key''' command). See [[R#Ubuntu.2FDebian|R installation]].
'''GPG key error''': If the machine is behind a proxy, we may get the following error. See [http://askubuntu.com/questions/147311/no-gpg-key-from-behind-a-firewall this post].
<pre>
gpg: requesting key E084DAB9 from hkp server keyserver.ubuntu.com
My experience aligns with the statement: '''dist-upgrade''' is more likely to break stuff badly than '''upgrade'''.
=== Troubleshooting ===
==== [http://askubuntu.com/questions/406229/there-was-no-etc-default-grub-file Grub2 cannot boot after timeout] ====
There is not /etc/default/grub with GRUB2.
<syntaxhighlight lang='bash'>
sudo apt-get --reinstall install grub-pc
</syntaxhighlight>
==== [http://askubuntu.com/questions/454037/disk-drive-boot-problem-the-disk-drive-for-tmp-is-not-ready-yet-or-is-not-pres The disk drive for /tmp is not ready yet] ====
Change the download location to 'Main Server' or another server close to the country you live in and try to sudo apt-get update again.
=== Use '''parted''' command to format a new hard disk ===
* The '''fdisk''' won’t create partitions larger than 2 TB. Use '''parted''' with GPT partition table.
* http://askubuntu.com/questions/517354/terminal-method-of-formatting-storage-drive. The bottomline is not to use '''fdisk''' since it does not support GPT. Use '''parted''' (the CLI brother of GParted).
It is best to use the latest release of GParted https://github.com/GNOME/gparted/.
The gparted version on Mint 17.2 is 0.18.0 (Feb 19, 2014) while the current one is 0.28.1 (Feb 17, 2017).
After I create a bootable USB drive, the partition format is fat32 (I use an old version of GParted to format the drive as fat32, and then I use unetbootin to create the bootable USB drive).
If we want to use unetbootin, the USB drive has better to be pre-formatted as fat32 first for unetbootin to detect it. Once unetbootin detect the drive, we can use gparted to reformat it as NTFS before going to burn the Windows bootable USB drive. Read [https://askubuntu.com/questions/162174/how-do-i-use-unetbootin-to-make-a-bootable-windows-usb-installer How do I use Unetbootin to make a bootable Windows USB installer?].
==== '''gparted''' on a 4TB disk ====
Get an error when I tried to create partitions on a 4T disk. Search 4TB gparted sectors exceeds the msdos-partition-table-imposed maximum of 4294967295
It seems it is necessary to use '''GPT''' instead of '''MBR/msdos''' as a partition table to overcome 2TB limitions.
It is also a good idea to use a live gparted os since the one in Ubuntu may not be up-to-date and gparted takes forever to scan devices. When boot from live USB, we need to turn off the Ext hard disk first.
* The 4TB hard disk can be recognized and used normally in an internal hd in Ubuntu 12.04.
* The 4TB hard disk can also be used in Windows 7 as an external hd if I formatted it (e.g. from gparted in Linux) as an NTFS partition. The Disk Management (command prompt -> diskmgnt.msc) shows it has 3726.02 GB (3726.02 * 1024 * 1024 * 1024 = 4.000784e12 Bytes) capacity and the windows manager shows it is 3.63TB (3726.02/1024=3.638) total space.
* It cannot be used as 4TB in the case when I use a docking station in Dell Precision T3500. Unsolved problem:
** Running the command chown from root to user takes forever on 4T partition.
** The ext dock station (StarTech) will halt the shutdown until I power off the station?
Conclusion:
* 4TB using NTFS works on Ubuntu.
* rsync will not stop spinning for some reason even the command is finished (through StarTech dock station).
Below is a screenshot I got from Gparted on a pre-formatted (NTFS) 4T portable drive from Seagate. It seems not harmful because I can still umount, change labels, etc on the disk.
[[File:Gparted gpt.png|200px]]
=== tracker-miner-fs ===
See http://askubuntu.com/questions/346211/tracker-store-and-tracker-miner-fs-eating-up-my-cpu-on-every-startup how to disable it.
=== Health check of the hdd ===
I got an input/output error when I use ''sudo rm'', ''sudo reboot'' or ''Ctrl + Del'' commands.
When I use the power button to force shutdown, I could not boot again. The BIOS does find the hdd and the Ubuntu Live USB does find the internal hdd too.
To force to reboot/shutdown, follow the suggestion [http://linoxide.com/how-tos/inputoutput-error-bad-blocks-how-to-restart-linux/ here]
'''dmesg''' command shows there are a few bad sectors on that hdd.
http://www.howtogeek.com/howto/37659/the-beginners-guide-to-linux-disk-utilities/ shows a few ways to run a health check on the hdd. The gnome disk utility cannot run S.M.A.R.T. on the external hdd.
<pre>
sudo badblocks -v /dev/sdb1
sudo badblocks /dev/sdb > /home/zainul/bad-blocks
sudo fsck -l bad-blocks /dev/sdb
</pre>
This article http://linoxide.com/linux-how-to/how-to-fix-repair-bad-blocks-in-linux/ talks about how to fix/repair bad blocks in Linux
.
This article http://unix.stackexchange.com/questions/25902/what-does-this-hard-disk-error-message-mean-current-pending-sector-count talks about how to do with bad sectors.
It is an indicator that hdd is going to die http://www.linuxquestions.org/questions/linux-hardware-18/34-bad-blocks-what-should-i-do-927224-print/.
=== Burn/Write an iso or img file to a USB flash drive ===
[http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-windows Official instruction] on www.ubuntu.com and from [https://en.wikipedia.org/wiki/List_of_tools_to_create_Live_USB_systems wikipedia].
* If your current OS is windows => [http://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/#button Universal USB Installer/Live Linux USB Creator].
* If your current OS is Ubuntu => [https://help.ubuntu.com/community/Installation/FromUSBStick#Creating_a_bootable_Ubuntu_USB_flash_drive_from_Ubuntu Several choices] like '''Startup Disk Creator/usb-creator''' ([[has an option on the GUI to erase the usb drive]]). If your ubuntu derivative does not have it, install it by sudo apt-get install usb-creator-gtk. UNETBOOTIN (no option to erase the USB so it can fail) or mkusb.
* If your current OS is Mint => Right click the iso file and select Make bootable USB stick. No software to install.
==== Use dd ====
* [http://www.cyberciti.biz/faq/unix-linux-dd-create-make-disk-image-commands/ How to make disk image with dd on Linux or Unix]
First, get to know the USB drive device name like /dev/sdb. When using 'dd', the USB drive has to be unmounted (using 'umount' command, not click 'reject' button in File Manager). Note that this will irrevocably destroy all data on /dev/sdX.
The instruction can be found in a lot of places like [https://wiki.archlinux.org/index.php/USB_Flash_Installation_Media Archlinux wiki] page.
<syntaxhighlight lang='bash'>
sudo fdisk -l
sudo dd bs=4M if=xxx.img of=/dev/sdb && sync
</syntaxhighlight>
where /dev/sdb is a device name, not a partition name. We can also adjust bs to a smaller value like 1M, 4m.
* [http://www.cyberciti.biz/faq/linux-unix-dd-command-show-progress-while-coping/ Monitor the progress],
** adding a parameter '''status=progress''' in '''dd''' (not working on Ubuntu 14.04)
** Linux '''dialog''' command
** following [[Raspberry#Install_an_image_to_a_SD_card|Raspberry Pi]]
<syntaxhighlight lang='bash'>
sudo pkill -USR1 -n -x dd
</syntaxhighlight>
For some reason when I use dd to create ubuntu 14.04 on usb drive, sudo gparted also gives me a Libparted warning ''/dev/sdc contains GPT signature, indicating that it has a GPT table. However, it does not have a valid fake msdos partition table, as it should... Is it a GPT partition table?'' messsage. If I click 'Yes', Gparted shows no partition on the usb drive??? Nevertheless, the usb drive can be used to boot into ubuntu 14.04.
In another case, the gparted compalins the usb drive "Invalid partition table - recursive partition on /dev/sdb". Someone suggests to issue a command
* Create GPT (for UEFI) or MBR partition table. See [http://www.howtogeek.com/193669/whats-the-difference-between-gpt-and-mbr-when-partitioning-a-drive/ What’s the Difference Between GPT and MBR When Partitioning a Drive?]
* [https://medium.com/@tomac/installing-ubuntu-mint-linux-on-onda-820w-tablet-727747a376b#.o4osscrty installing Ubuntu / Mint Linux on Onda 820w tablet]
* It successfully burns ESXi and Ubuntu iso images to USB drives while the USB drives created by the 'dd' command does not work??
For creating a Windows bootable USB drive, we cannot use Etcher program. On UDoo-X86 [https://www.udoo.org/get-started-x86/ Get started] site, it suggests to use Unetbootin if the host machine is Ubuntu. NOTE. the USB drive has to be formatted as FAT32 (this can be done by '''GParted''' program); otherwise Unetbootin cannot recognize the drive. Unfortunately the USB drive can not be booted from UDoo-X86. See also the trick by [https://askubuntu.com/questions/162174/how-do-i-use-unetbootin-to-make-a-bootable-windows-usb-installer this post].
* The command ''sudo mount /dev/sdc1 /mnt'' should be '''sudo mkdir /media/$USER/usb; sudo mount /dev/sdc1 /media/$USER/usb''' where /dev/sdc1 should be changed appropriately
* This approach works
* Rufus is good but not always (eg the USB drive is not bootable??)
The GUI is written by Qt so the program is cross-platform. See its [http://sourceforge.net/p/unetbootin/wiki/compile/ wiki].
The following is a screenshot of the contents of xubuntu 12.04. '''The usb drive needs to be formatted to fat32''' on Windows OS to repair partition table error. The partition table error was discovered when I use sudo gparted program to view the USB drive.
Note that Unetbootin (Windows & Linux) and Universal USB installer (Windows only) are quite similar although Universal USB installer provides more options in its interface while Unetbootin does not have any other options.
I first used ''dd'' command to successfully created a xubuntu 14.04 usb drive but there seems to be a problem with the partition since the gparted program will give a warning message about that and also the whole 16GB was used when I insert the drive onto a Windows machine.
Note that the '''fdisk''' utility cannot handle this new partition format so we have to use the '''gparted''' program.
Open the '''gparted''' program. Use it MULTIPLE TIMES to create a partition table (Device -> Create Partition Table...). Then we create a FAT32 partition with all of the space. After that, we can use the UUI program to create an Ubuntu USB drive.
The following screenshots are from a 16GB USB drive.
YUMI works much like Universal USB Installer, except it can be used to install more than one distribution to run Live from your USB.
It can be used to create a Multiboot USB Flash Drive containing multiple operating systems, antivirus utilities, disc cloning, diagnostic tools, and more.
==== [http://multibootusb.org/ MultiBootUSB] ====
The program is included by LXLE.
It supports [https://wiki.ubuntu.com/LiveUsbPendrivePersistent persistence] up to 4GB for ubuntu and its derivatives.
It also supports multi-thread (check by top or htop). The %cpu > 100.
It will take space as needed. So we can still use the USB drive to write data.
For some reason, the USB drive could not be boot after I use the program. The ubuntu does not show/recognize it though gparted still finds it. Maybe it is because the partition format (ext4 by gparted) is right. For YUMI program, it says to use fat16/fat32/NTFS; otherise syslinux will fail. But it seems not to help:( Maybe it is the partition table (I choose gpt instead of the default msdos).
On Ubuntu the default window manager is '''Compiz''', for xubuntu it is '''Xfwm4''' and for BBB it is '''Openbox'''.
=== Add date and time to the clock indicator on the top panel ===
Method 1. GUI approach. Right click the clock indicator and choose Time and Date Settings. Click on 'clock' tab and check 'date and month' option.
Method 2. [https://www.liberiangeek.net/2012/05/show-the-date-and-time-on-the-clock-indicator-in-ubuntu-12-04-precise-pangolin/ Command line approach].
<pre>
gsettings set com.canonical.indicator.datetime show-date true
</pre>
=== Jenkins ===
[https://www.howtoforge.com/tutorial/how-to-install-jenkins-with-apache-on-ubuntu-16-04/ How to Install Jenkins Automation Server with Apache on Ubuntu 16.04]
Virtualbox does not work. After initial fix, the guest machine cannot connect to internet:(
==== Crashes network manager (no internet connection, no applet) ====
The solution on [http://askubuntu.com/questions/727127/last-upgrade-crashes-network-manager-no-internet-connection-no-applet here] works on my Ubuntu 14.04.4. Download 3 deb files and install them (downgrade packages).
=== Graphics driver ===
* https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia By default Ubuntu will use the open source video driver '''Nouveau''' for your NVIDIA graphics card. This driver lacks support for 3D acceleration and may not work with the very latest video cards or technologies from NVIDIA.
# If glxinfo is not find, run sudo apt-get install mesa-utils
$ glxinfo | grep OpenGL
</syntaxhighlight>
Under Ubuntu-Unity, we can search Additional Drivers to install propriety party drivers instead of using X.org.
On Dell T3600, it shows (pay attention to the line '''driver=nouveau'''). See also [https://nouveau.freedesktop.org/wiki/InstallNouveau/ Installing Nouveau on your Linux computer].
<syntaxhighlight lang='bash'>
$ sudo lshw -C video
[sudo] password for brb:
*-display
description: VGA compatible controller
product: GF108GL [Quadro 600]
vendor: NVIDIA Corporation
physical id: 0
bus info: pci@0000:03:00.0
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
If I change to use the nvidia driver on T3600 which has NVIDIA Quadro 600, the GLX error will happen (OpenGL related. Some programs like Qt Creator, Blender will not be able to run). See [https://taichimd.us/mediawiki/index.php/Qt#Could_not_initialize_GLX._Aborted the detail].
==== AMD vs NVIDIA ====
[http://www.makeuseof.com/tag/use-amd-nvidia-gpus-linux/ Should You Use AMD or NVIDIA GPUs on Linux?]
Change value of "enabled" from 1 to 0. Then reboot.
=== Autostart when log into your desktop ===
1. '''~/.config/autostart/''' (hidden directory)
Add .desktop files to here to have programs start automatically at startup. These .desktop files are application shortcuts — you can often create them by dragging-and-dropping an application onto your desktop or even just into the ~/.config/autostart/ window.
If you’re not using a desktop environment check out '''~/.bash_profile''' file.
2. '''/etc/xdg/autostart'''
Or search 'Startup Applications Preferences' in Dash/Menu. [https://askubuntu.com/questions/48321/how-do-i-start-applications-automatically-on-login How do I start applications automatically on login?]
==== gnome-screenshot (preinstalled in Ubuntu) ====
[https://www.howtoforge.com/tutorial/taking-screenshots-in-linux-using-gnome-screenshot/ A comprehensive guide to taking screenshots in Linux using gnome-screenshot]
* Each taken screenshots are placed in different tabs in Shutter
* If we close Shutter, the unsaved screenshots are still there because by default it automatically save files in the ~/Pictures folder. We can modify the settings from Edit -> Preferences.
* It also supports 'cropping'. Click 'Edit' button, Tool -> Crop, Select a region, Click 'Crop' button on the RHS.
==== GIMP ====
File -> Create -> Screenshot
==== '''import''' ====
command from ImageMagick
==== '''scrot''' ====
from scrot package. Note that by default the title bar will not be included (see the '-b' option below).
It seems '''scrot''' is better for my need.
Note: there seems no way to copy the screenshot to the clipboard.
<syntaxhighlight lang='bash'>
scrot -c -d 5 screenshot.png # -c is count down, -d is delay
scrot -b -c -d 5 -u screenshot.png # -b is title bar, -u is the current foc'u'sed window
scrot -u # current window
scrot -s screenshot.png # select an area
scrot --thumb 50 # 50% of the original screenshot
scrot abc.png -e 'gthumb $n' # open gThumb to open the file after taking the screenshot
Although Kazam can do screenshots, [[#Shutter|Shutter]] (can sit on the system tray) is more convenient for taking care of screenshots.
The default frame rate is only 15. If we want to increase it, go to File -> Preferences -> Screencast tab.
When recording, it will have 5 seconds (adjustable) to wait. After launching Kazam, it will show an icon (video recorder) on the top-right corner. Keyboard shortcuts are available. For example (Windows key=Super key),
* Record=Ctrl + Windows + r,
* Finish=Ctrl + Windows + f,
* Pause=Ctrl + Windows + p.
<syntaxhighlight lang='bash'>
sudo apt-get install kazam
</syntaxhighlight>
You can choose fullscreen, window or a specific area. However, if you want to change the window or area once you have chosen one, you have to restart the program. One nice thing with Kazam is the output video is in .mp4 format (not ogv or ogg format). For a 14-seconds video with 15 frames per second (default), the video file size is 1.4MB.
A good introduction [https://www.maketecheasier.com/create-screencast-videos-using-kazam/ Create Screencast Videos With Ease Using Kazam]
[https://github.com/MaartenBaert/ssr source code] and [https://beeznest.wordpress.com/2014/10/16/the-comprehensive-guide-to-screencasting-in-ubuntu-14-04/ this article]
** We need to run '''ffmpeg''' to convert video to flv (Quality seems to be reduced) OR we can use online service (http://video.online-convert.com/convert-to-mp4) to convert ogv file to mp4 file (Same quality as I can tell).
* [http://askubuntu.com/questions/107726/how-to-create-animated-gif-images-of-a-screencast Create animated Gif of a screencast] and the command line tool [https://github.com/jclem/gifify/ Gifify].
* [https://www.howtoforge.com/tutorial/record-screen-to-animated-gif-on-linux/ How to record a region of your desktop as animated GIF on Linux]
=== Wallpaper ===
[https://www.howtogeek.com/305974/how-to-use-bings-background-of-the-day-as-your-ubuntu-wallpaper/ How to Use Bing’s Background of the Day as Your Ubuntu Wallpaper]
=== Conky ===
[http://conky.sourceforge.net/ Conky] is a free, light-weight system monitor for X, that displays any information on your desktop.
* https://github.com/zenzire/conkyrc (this one works). Check [http://woeid.rosselliot.co.nz/ for the WOEID] for your city used in the [http://www.yahooapis.com/weather/ Yahoo weather API].
Step 2. create ~/.conkyrc file. This file can be downloaded from web.
Step 3. Run it: $ conky. If we want to run a specific configuration file, use conky -c CONKYRCFILE
Step 4. If you want to stop Conky: $ killall conky
Note that conky works automatically on Ubuntu's Unity.
For Lubuntu (tested on 14.04), the Conky's transparent function does not work at first. But [http://gnome-look.org/content/download.php?content=170851&id=1&tan=63968661 This conkyrc] works on Lubuntu desktop (mainly, tranparent function). To deal with the autostart, follow the suggestion from [http://askubuntu.com/questions/81383/how-can-i-add-new-autostart-programs-in-lubuntu askubuntu.com]. That is, go to ~/.config/autostart folder, create or copy+paste the file called '''conky.desktop''' with a content like
<pre>
[Desktop Entry]
Type=Application
Exec=sh "/home/brb/.conky/conky-startup.sh"
Hidden=false
NoDisplay=false
X-GNOME-Autostart-enabled=true
Name[en_IN]=Conky
Name=Conky
Comment[en_IN]=
Comment=
</pre>
and
<pre>
brb@brb-VirtualBox:~$ cat .conky/conky-startup.sh
conky &
exit 0
brb@brb-VirtualBox:~$ ls -l .conky/conky-startup.sh
-rw-rw-r-- 1 brb brb 37 Aug 30 20:17 .conky/conky-startup.sh
</pre>
Another way to configure conky is to install conky-manager. See [http://www.itworld.com/article/2696428/install-conky-manager-2-1-in-ubuntu-14-04.html this] and [http://www.teejeetech.in/p/conky-manager.html project website] page. But it seems it does not work well with desktop wallpaper.
On Mint, go to Preferences -> Window Tiling -> Enable Window Tiling and snapping -> Off.
=== [http://www.gnu.org/software/grub/ Grub2] ===
To show the grub2 screen, run 'sudo nano /etc/default/grub' and comment out the line GRUB_HIDDEN_TIMEOUT=0 and change the line GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" to GRUB_CMDLINE_LINUX_DEFAULT="". Remember to run 'sudo update-grub' after any change to grub.
To add a splash image, follow the instruction at https://help.ubuntu.com/community/Grub2/Displays. Note that Grub2 will search the image based on some priority and there are also some minor requirements on the images. To test
(Ubuntu 16.04 on Odroid) For some reason, pinging my domain always goes to the wrong IP. If I go to System -> Administration -> Network to change DNS from 127.0.1.1 to 8.8.8.8, it fixes the problem. But if I reboot, the DNS entry goes back to 127.0.1.1 again.
When I follow this post [http://askubuntu.com/questions/627899/nameserver-127-0-1-1-in-resolv-conf-wont-go-away nameserver 127.0.1.1 in resolv.conf won't go away], the DNS will be changed to local IP of my router and the problem is fixed (even I reboot the computer).
[http://askubuntu.com/questions/597116/how-to-disable-wireless-from-command-line The command-line equivalent of unchecking the nm-applet's 'Enable Wi-fi' should be]
<syntaxhighlight lang='bash'>
nmcli nm wifi off
# OR
nmcli radio wifi off
# OR
sudo ifconfig wlan0 down
</syntaxhighlight>
==== Proxy ====
* [http://askubuntu.com/questions/3807/how-to-check-if-network-proxy-is-really-applied How to check if Network Proxy is really applied?]
==== Internet Shut Down ====
* [http://gizmodo.com/this-is-probably-why-half-the-internet-shut-down-today-1788062835 This Is Why Half the Internet Shut Down Today], [http://pcworld.com/article/3134056/hacking/an-iot-botnet-is-partly-behind-fridays-massive-ddos-attack.html An IoT botnet is partly behind Friday's massive DDOS attack] and [http://www.appledaily.com.tw/appledaily/article/international/20161023/37425973/ 駭客襲美 推特等主要網站大掛點] (Oct 21 2016)
* [https://en.wikipedia.org/wiki/Denial-of-service_attack Distributed denial-of-service/DDoS attack] from wikipedia.
=== DHCP Server ===
[http://www.tecmint.com/install-dhcp-server-in-ubuntu-debian/ How to Install a DHCP Server in Ubuntu and Debian]
=== File Server ===
==== [https://help.ubuntu.com/lts/serverguide/network-file-system.html Network File System (NFS)] ====
NFS allows a system to share directories and files with others over a network. By using NFS, users and programs can access files on remote systems almost as if they were local files.
Some of the most notable benefits that NFS can provide are:
* Local workstations use less disk space.
* There is no need for users to have separate home directories on every network machine. Home directories could be set up on the NFS server and made available throughout the network.
'''Server part''':
<syntaxhighlight lang='bash'>
sudo apt-get install nfs-kernel-server
</syntaxhighlight>
Configure the directories to be exported by adding them to the '''/etc/exports''' file.
where 'ro' means read only. See [http://www.brennan.id.au/19-Network_File_System.html Linux Home Server Howto]. The '''no_root_squash''' option will not prevent root on a client machine from writing files to the server as root; by default, NFS will map any requests from root on the client to the 'nobody' user on the server. See [https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-14-04 digitalocean].
To start the NFS server:
<syntaxhighlight lang='bash'>
sudo service nfs-kernel-server start
</syntaxhighlight>
'''Client part''':
PS.
* ''There is no need to enter any password related to the NFS server.''
* See [https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-14-04 digitalocean] for other ways to specify the arguments in </etc/fstab>.
<syntaxhighlight lang='bash'>
sudo apt-get install nfs-common
sudo mount example.hostname.com:/ubuntu /mnt/ubuntu
# The mount point directory /mnt/ubuntu must exist.
[http://www.ubuntugeek.com/how-to-configure-nfs-server-and-client-configuration-on-ubuntu-16-10-yakkety-yak.html How to configure NFS Server and Client Configuration on Ubuntu 16.10]
With NFS booting, the core kernel and file systems are kept on a central server and then pushed out onto client systems to be booted on there. That means your files and desktop will always be available wherever you want to log in.
At the end, the client computer does not need any internal storage. Cool!
==== [https://en.wikipedia.org/wiki/Server_Message_Block CIFS (Common Internet File System)] and [https://en.wikipedia.org/wiki/NT_LAN_Manager NTLMv2 (NT LAN Manager)] ====
''Note that this method is most suitable for transferring small files. Users transferring large amounts of data to and from Helix/Biowulf should continue to use scp or sftp.'' ([https://hpc.nih.gov/docs/transfer.html#mapped nih.gov])
==== Samba - allows Linux to transfer files with Windows clients ====
* http://www.krizna.com/ubuntu/setup-file-server-ubuntu-14-04-samba/ (Anonymous share and Secured share via username/password)
* [http://www.noobslab.com/2012/03/configure-samba-sharing-between-ubuntu.html Share 'between' ubuntu and Windows]
* [http://www.linuceum.com/Server/srvSambaDaemons.php Start and stop the samba daemons]
<syntaxhighlight lang='bash'>
$ ps -ef | grep smbd # see if the Samba daemon (smbd)
$ ps -ef | grep nmbd # see if the NetBIOS name server daemon (nmbd) is running
$ sudo service smbd stop # does not stop nmbd
$ sudo service nmbd stop
$ sudo service smbd start
$ sudo service nmbd start
</syntaxhighlight>
* [https://www.samba.org/samba/docs/man/manpages/smb.conf.5.html '''smb.conf''' — The configuration file for the Samba suite]
<syntaxhighlight lang='bash'>
sudo apt-get install samba samba-common
sudo apt-get install python-glade2
sudo apt-get install system-config-samba
</syntaxhighlight>
Use Dash and search for 'samba'. It will ask for the user's password first. The samba password can also be set by
<syntaxhighlight lang='bash'>
sudo smbpasswd -a USERNAME
</syntaxhighlight>
A non-gui way to configuration samba is adding the following to the end of /etc/samba/smb.conf file, '''sudo nano -w /etc/samba/smb.conf''' (-w means no-wrap). Any line beginning with a semicolon (“;”) or a hash (“#”) character is ignored.
<pre>
[brb]
path = /home/brb
; writeable = no
; browseable = yes
guest ok = yes
</pre>
* [http://www.unixmen.com/howto-install-and-configure-samba-share-in-ubuntu/ Share between ubuntu and linux] On the client file manager, type
<pre>
smb://192.168.1.3/
</pre>
* https://help.ubuntu.com/community/Samba
On Windows PC, go to start and open 'Run' then enter ip with double backslash. Like this (\\192.168.1.3).
=== Change the default session when using auto login ===
See [http://askubuntu.com/questions/62833/how-do-i-change-the-default-session-for-when-using-auto-logins this post]. We need to edit the file /etc/lightdm/lightdm.conf. Note that on my Ubuntu 12.04, I have the following desktop options.
When you login by SSH, then use the username you have chosen when you installed Debian as the root user is disabled for remote logins. Then run the command "su" to become root user. See [https://www.howtoforge.com/tutorial/debian-8-jessie-minimal-server/2/ howtoforget.com].
=== Terminal ===
==== Directory color on ssh ====
When I ssh to biowulf from Gnome/Terminal, the folder color became dark blue which is hard to read when the background color is black. A solution is https://serverfault.com/questions/137649/changing-terminal-colors-in-ubuntu-server.
2. create/edit your ~/.dir_colors, for example with
<pre>
$ dircolors --print-database > .dir_colors
</pre>
3. Then force a read of your .bashrc file with:
<pre>
$ source ~/.bashrc
</pre>
Everything should be pretty.
==== Remember terminal tabs ====
The trick on [http://askubuntu.com/questions/310705/some-fast-way-to-save-and-restore-tabs-of-terminal this post] works for me on my GNOME Terminal 3.6.2 (gnome-terminal --version).
<syntaxhighlight lang='bash'>
# To save configuration into /home/$USER/terminal.cfg:
To recall the titles, follow [http://vadim-kirilchuk-linux.blogspot.com/2013/05/gnome-terminal-how-to-save-tabs-and.html this simple hack]. That is, add an entry Title=xxxx to each tab section.
==== Terminal tab color ====
If we open several tabs in the (GNOME) Terminal, the default color of the active tab is not quite different from the other tabs. We need a solution to change the tab colors.
# type 'language' in Dash search (Or click Power button on the top-right corner -> System Settings...) and click 'Language Support' (this may not be installed from (x)Ubuntu automatically. In such case, IBUS will be missing eg Chewing method). The 'Language Support' is under Settings menu in xUbuntu.
# Click 'Install' if a message popped up with the message 'The language support is not installed completely'. After the installation is done, the 'Language support' dialog comes back.
# Choose between ibus (ubuntu < 16.04) /fcitx (ubuntu 16.04).
#* Choose 'IBUS' for keyboard input method system (bottom of the dialog). IBUS is under 'System' menu in xUbuntu.
#* Choose 'fcitx'
# Click 'Install/Remove Languages'. Choose Chinese. Click 'Apply'. An 'Applying changes' dialog will appear.
# Click 'Close' button.
# Select Chinese
#* System Settings -> Text Entry -> '+' -> Chinese. Or Settings > Keyboard Input Method > Input method > Select an input method > Show only input methods for your region > Chinese 酷 Chewing. If we cannot find this dialog, we can launch it by '''ibus-setup''' command. If Chewing is not shown as one of Chinese input methods, we can log out and log in the desktop. If ibus does not have the chewing selection, we should try to install it '''sudo apt-get install ibus-chewing''' and/or run '''ibus restart'''.
#* Dash -> Fcitx Configuration. '+' -> Chewing. PS. On Ubuntu 16.04 it only shows a Chinese option without further options even I installed ''ibus-chewing''.
# Make sure 'Show current input source in the menu bar' is checked.
# Click 'Close' button.
On Ubuntu 14.04, the super key is '''Windows'''.
Note that do not use 'Text Entry' application (shown as one options when we search 'language'). This application will conflict with ibus.
* On Linux Mint, it cannot show a list of candidate words. The trick of using '''killall fcitx-qimpanel''' works.
* On Ubuntu Mate, the fcitx works fine. ps -ef | grep fcitx-qimpanel shows nothing.
=== /usr/lib/ibus/ibus-ui-gtk3 high memory usage ===
For some reason, the ubuntu 14.04 is not responsive. Htop shows ibus-ui-gtk3 is the culprit. The [http://askubuntu.com/questions/490317/how-can-i-find-out-what-ibus-daemon-and-ibus-ui-gtk3-are-doing suggestion] is to run
<syntaxhighlight lang="bash">
killall ibus-daemon
</syntaxhighlight>
After I run this command and wait a few seconds, the memory leak problem is solved.
=== Change time zone ===
http://wiki.debian.org/TimeZoneChanges
<pre>
$sudo dpkg-reconfigure tzdata
</pre>
The command launches an ncurses based interface which allows arrow keys to be used to select the region specific time zone.
=== Make script run at boot time with init.d directory ===
http://www.debian-administration.org/articles/28
For example, see [http://learn.adafruit.com/drive-a-16x2-lcd-directly-with-a-raspberry-pi/init-script here] from running a python script for raspberry pi.
# Create a script '''/etc/init.d/lcd'''
# Make the script executable
# Make the script known to the system by using the '''update-rc.d' command
<pre>
sudo update-rc.d lcd defaults
</pre>
==== /etc/rc.local file ====
[http://linuxtechlab.com/executing-commands-scripts-at-reboot/ Executing Commands and Scripts at Reboot & Startup in Linux]
For CentOS, we use file ‘/etc/rc.d/rc.local’ instead of ‘/etc/rc.local’. We also need to make this file executable before adding any script or command to the file.
Note:- When executing a script at startup, make sure that the script ends with ‘exit 0’.
==== crontab ====
[http://linuxtechlab.com/executing-commands-scripts-at-reboot/ Executing Commands and Scripts at Reboot & Startup in Linux]
[http://linuxplained.com/how-to-fix-wireless-problems-in-ubuntu-1204-precise-pangolin/ 5 Best Linux/Ubuntu compatible USB Wifi cards]:
* AirLink101 AWLL6075 Wireless N Mini USB Adapter
* Medialink – Wireless N USB Adapter – 802.11n
* ASUS (USB-N13) Wireless-N USB Adapter
* Panda Mini Wifi (b/g/n) 150Mbps Card
* TP-Link TL-WN722N 150Mbps High Gain Wireless USB Adapter
To show (USB) wireless adapter information
<pre>
sudo lshw -C network
sudo lsusb -v
iwconfig
</pre>
To check wifi connection information (SSID, channel, address, frequency, qualiyt, signal level, ...)
<pre>
sudo iwlist wlan0 scan
</pre>
My experience is quality should be at least 50/70 and signal level should be larger than -60dBm.
=== Create an iso file from a CD or HD ===
Method 1 (Better than Method 2). No need to umount the cdrom.
<pre>
mkisofs -o /tmp/DVD.iso /tmp/directory/
</pre>
to make an ISO from files on your hard drive.
Method 2.
Make sure the cdrom is NOT mounted. Type '''mount''' to confirm it. If cd was mouted automatically unmout it with umount command:
like '''umount /dev/cdrom''' or '''umount /mnt/cdrom'''. Note that no extra forward slash after /dev/cdrom for the command below.
<pre>
dd if=/dev/cdrom of=~/DVD.iso
</pre>
We can mount the iso file to a directory to check the iso file content is correct.
<pre>
mount -t iso9660 -o loop,ro DVD.iso /mnt
</pre>
=== Have fun with /etc/hosts file ===
su -c "nano /etc/hosts"
<pre>
74.125.67.100 DNS_NAME1 DNS_NAME2
</pre>
==== Block malware, adware ====
[https://www.howtogeek.com/319700/how-to-use-your-computers-hosts-file-to-block-tons-of-malware-porn-and-other-types-of-websites/ How to Use Your Computer’s Hosts File to Block Tons of Malware, Porn, and Other Types of Websites]
=== DNS tricks ===
==== 5 DNS Servers Guaranteed to Improve Your Online Safety ====
* "%k" - [http://askubuntu.com/questions/47775/how-can-i-set-the-path-variable-in-a-desktop-file-to-be-relative-to-the-locat How can I set the 'Path' variable in a .desktop file to be relative to the location where the desktop file is located?] and [http://unix.stackexchange.com/questions/144422/getting-the-current-path-in-desktop-exec-command Getting the current path in .desktop EXEC command]
* [http://stackoverflow.com/questions/8980464/how-do-i-access-an-environment-variable-in-a-desktop-files-exec-line How do I access an environment variable in a .desktop file's exec line?]
==== Open a terminal ====
<syntaxhighlight lang='bash'>
sudo apt-get install nautilus-open-terminal
nautilus -q
</syntaxhighlight>
In Ubuntu 15.10, the functionality is already included in nautilus!
Go to keyboard app to change the settings. Note: '''Super''' key is also Windows key. Use 'Backspace' key to disable a shortcut.
* List from [https://help.ubuntu.com/community/KeyboardShortcuts ubuntu.com]
* Super: Open the Dash. Press and hold it to see a cheat sheet with a bunch of other nifty shortcuts.
* Super + Number: Open the application that is at that position in the dock
* '''Alt + Mouse''': Move an application. Useful if an application's title bar is too high.
* Alt + F2: Run an application by typing its name in the box which appears (same function as the Super key).
* Alt + Space: Activate the window menu. Not useful so I disable it.
* Ctrl + Super + D: To minimize all windows
* Ctrl + Alt + Arrows: move to another workspace
* Ctrl + Alt + Shift + Arrows: move current application to another workspace
On Xubuntu,
* Alt + F1 (Ctrl + ESC): Application menu.
* Alt + F2 (Super + R): Application Finder.
==== Add a new keyboard shortcut ====
Let's say I want to assign Ctrl+Alt+s to bring the shutdown dialog.
# System Settings -> Keyboard -> Shortcuts -> Windows -> Toggle shaded state -> Backspace to disable it. For some reason, I cannot assign a new keybinding using the method of '''Custom Shortcuts' method.
# Open ''Ubuntu Software Center ''and install '''Compiz Config Setting Manager''' program. Open the program by search ''Compiz'' and then create a new command '''gnome-session-quit --power-off --force''' with a name like 'Shutdown' and the keyboard binding we want. CCSM -> General -> Commands.
With this approach, it will give a dialog with 4 options (lock suspend restart shutdown) but no one is pre-selected. If we want the shutdown icon being preselected, use [http://askubuntu.com/questions/554576/how-do-i-call-gnome-session-quit-with-countdown-from-unity this script] where the '''zenity''' command was used to create a dialog. The '''zenity''' program was pre-installed in Ubuntu. See [http://www.howtogeek.com/107537/how-to-make-simple-graphical-shell-scripts-with-zenity-on-linux/ How to Make Simple Graphical Shell Scripts with Zenity on Linux].
# One is to download iso image file. http://nebc.nerc.ac.uk/downloads/
# The other way is to install Bio-linux software/package by using apt-get install method. See http://nebc.nerc.ac.uk/tools/bio-linux/other-bl-docs/package-repository
[[File:BioLinux.png|100px]]
==== Check Biolinux version ====
<pre>
cat /etc/bio-linux.version # 8.0.5 as of June 2015
(Excerpt from [http://www.thegeekstuff.com/2016/06/nginx-reverse-proxy-to-apache/ thegeekstuff]) For example, let us say we have an enterprise application that is running on Apache and PHP on app.thegeekstuff.com, and we also have Nginx running on example.com.
In this example scenario, when someone goes to example.com, we can setup Nginx as a reverse proxy so that it will serve the enterprise apache/php application that is running on app.thegeekstuff.com.
But, for the end-user, they’ll only see example.com, they won’t even know anything about app.thegeekstuff.com. End-user will think the whole apache/php application is getting served directly from example.com.
* [http://www.ubuntugeek.com/install-tomcat-8-on-ubuntu-15-10-server.html Install Tomcat 8 on Ubuntu]
=== Device manager ===
By default, ubuntu does not provide any graphical tool like device manager on Windows. A very close one is '''[http://ezix.org/project/wiki/HardwareLiSter lshw]''' (hardware lister). A GUI tool based on it is called '''lshw-gtk''' (seems not as informative as the command line one) and can be installed by ''sudo apt-get install lshw-gtk'' in Ubuntu/Debian or ''yum install lshw'' in Red Hat/Fedora/CentOS.
As you can see the line containing 'display' shows the motherboard (P45T-A), CPU (Intel Core 2 Duo E8400), graphical card (GeForce 9400 GT in this case), et al.
/0/1/0.0.0/2/5 /dev/sda5 volume 8190MiB Linux swap / Solaris partition
/0/1/0.1.0 /dev/sdb disk 2TB ST2000DM001-9YN1
/0/1/0.1.0/1 /dev/sdb1 volume 1863GiB EXT4 volume
/0/2 scsi2 storage
/0/2/0.0.0 /dev/cdrom disk DVDRAM GH24NS90
/1 power Nikon Ultra Plus
/2 power To Be Filled By O.E.M.
</pre>
For storage part, ubuntu provides a graphical tool. See "disk utility" on gnome based ubuntu or search for "[https://live.gnome.org/Design/Apps/Disks disk]" in launcher.
=== Set static IP - /etc/network/interfaces===
<syntaxhighlight lang='bash'>
root@debian:~# cat /etc/network/interfaces
auto lo eth0
iface lo inet loopback
iface eth0 inet dhcp
</syntaxhighlight>
Now edit the file /etc/network/interfaces
<pre>
# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.3
gateway 192.168.1.1
netmask 255.255.255.0
network 192.168.1.0 # Optional
broadcast 192.168.1.255 # Optional
dns-nameservers 192.168.1.1 8.8.8.8 # Or skip 192.168.1.1
</pre>
After it, restart the network by issuing
<syntaxhighlight lang='bash'>
/etc/init.d/networking restart
</syntaxhighlight>
OR
sudo reboot
Note: It does not work by editing /etc/resolv.conf since this file will be overwritten.
=== Change IP address from the command line ===
<pre>
/sbin/ifconfig eth0 192.168.1.17 netmask 255.255.255.0 up
The default firewall configuration tool for Ubuntu is '''ufw'''. Developed to ease '''iptables''' firewall configuration, ufw provides a user friendly way to create an IPv4 or IPv6 host-based firewall. By default UFW is disabled.
Suppose I have a virtual machine running a web server at port 8888 (The vm may be initialized by the Vagrant command). I can access the webpage from my host machine using http://localhost:8888.
Now I want the web page to be accessible from other local machines. We can use the '''ufw''' command to enable the firewall wall and open ports for certain services.
<syntaxhighlight lang='bash'>
sudo ufw allow 8888/tcp
sudo ufw show added
sudo ufw enable
sudo ufw status
nmap localhost
</syntaxhighlight>
Now I can go to another machine, open a browser at http://hostip:8888/. I should be able to get the same result as I got from the host machine.
In one instance sshing to a server failed (connection time) for some reason. The solution is to run the following command on the server
<syntaxhighlight lang='bash'>
sudo ufw allow 22/tcp
</syntaxhighlight>
A graphical interface program is called [http://gufw.org/ Gufw Firewall].
Other things '''ufw''' can do:
* Allow/Deny by ports and (optional) protocols
* Allow/Deny by service name
* Disable ping requests.
* Allow by specific IP
* Allow by subnet
* Allow by specific port and IP address
* Deny by certain IP address
* Deny by certain IP address and certain port
=== DNS Server ===
[https://linuxconfig.org/protecting-your-privacy-with-firefox-on-linux Protecting Your Privacy With Firefox on Linux]
Note that the specification should depend on the domain name registrar (eg namecheap). For namecheap, the login/password is NOT your actual credential from your domain name registrar. The password should be obtained from the domain name registrar website. The last line is about the host. If I am setting it up for a subdomain, I should enter the subdomain name (and skip the domain name part). The '''ssl=yes''' is to ensure the [https://help.dyn.com/ddclient/ connection is made over https instead of http].
And run '''sudo ddclient -daemon=0 -debug -verbose -noquiet''' to verify ddclient is working. You shall get a long return with the last line looks like
<pre>
SUCCESS: updating YOURSUBDOMAIN: good: IP address set to XX.XXX.XXX.XXX
</pre>
No matter which method we use, we can go to our DNS account (in namecheap, go to Dashboard -> MANAGE button -> Domains -> Advanced DNS) and temporarily change the global IP address to another one, run the update script and then check if the global IP address has been updated to the correct one.
==== namecheap ====
* [https://www.namecheap.com/support/knowledgebase/article.aspx/583/11/how-do-i-configure-ddclient How do I configure DDClient?]
* To create a subdomain, go to Dashboard -> Manage -> Advanced DNS tab. Click '''+ ADD NEW RECORD'''. In the 'HOST RECORDS' section, pick 'A + dynamic dns record' and enter the subdomain name (HOST) with the IPv4 address (Value). In the 'DYNAMIC DNS' section, we can download the client software too (scroll down to get the download link). See [https://www.namecheap.com/support/knowledgebase/article.aspx/319/78/how-can-i-setup-an-a-address-record-for-my-domain How can I set up an A (address) record for my domain?]
* To understand different records (A record, AAAA record, CNAME record, NS record, SRV record, TXT record, URL redirect record) See [https://www.namecheap.com/support/knowledgebase/article.aspx/434/2237/how-do-i-set-up-host-records-for-a-domain How do I set up host records for a domain?]
* If you've purchased an SSL certificate, you'll want to visit your Account Panel soon to enter your CSR and activate the certificate. [https://www.namecheap.com/support/knowledgebase/category.aspx/14/ Instructions on how to create a CSR and install the certificate on your server].
==== Mail ====
* check the option of '''Mail Routing: I have mail server with another name and would like to add MX hostname...'''
* In 'MX hostname' entering '''aspmx.l.google.com'''
* In 'Primary' choose 'Yes, use it as my primary mail relay.'
=== no-ip ===
Similar to Dyndns. It has its own client program. Needs to build it yourself.
Also see the [http://support.no-ip.com/customer/portal/articles/375955-basic-troubleshooting-guide troubleshooting guide].
See http://ducky-pond.com/posts/12 for instruction of setting autostart on Debian system.
See http://www.coulterfamily.org.uk/pages/PCs/Linux/FAQ-LINUX-NO-IP-CLIENT.php for another approach.
Note: If noip2 cannot start automatically or noip2 does not update even it can be seen from ps -ef command, use '''sudo crontab -e''' command. <span style="color: red"> For some reason, after I use sudo crontab, noip2 can update IP.</span> So the only problem right now is it cannot update every 30 minutes even '''sudo noip2 -S''' says so. The problems may be 1. ps -ef shows the command runs from nobody user 2. sudo noip2 -S says it updates every 30 minutes via /dev/eth0 with NAT enabled.
Update: An alternative is to use ddclient. However, ddclient never updates the IP.
==== Mail ====
* Allow only one MX record for each host for free no-ip account.
The install will be done automatically to ''/usr/share/webmin'', the administration username set to ''root'' and the password to your current root password. You should now be able to login to Webmin at the URL http://localhost:10000/. Or if accessing it remotely, replace localhost with your system's IP address.
model name : AMD Phenom(tm) II X6 1055T Processor # home
model name : Intel(R) Xeon(R) CPU X7560 @ 2.27GHz # helix
model name : Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz # office
</syntaxhighlight>
==== Find out if the OS is running in a virtual environment ====
Linux adds the '''hypervisor''' flag to '''/proc/cpuinfo''' if the kernel detects running on some sort of a hypervisor. See [http://unix.stackexchange.com/questions/3685/find-out-if-the-os-is-running-in-a-virtual-environment here].
<syntaxhighlight lang='bash'>
cat /proc/cpuinfo | grep hypervisor
</syntaxhighlight>
==== Virtualbox ====
See [[Virtualbox|here]].
==== VBoxClient: the VirtualBox kernel service is not running ====
Got the above message on the upper right corner of the screen for every booting. Note I don't install VirtualBox. A [https://www.linuxliteos.com/forums/other/vboxclient-the-virtualbox-kernel-service-is-not-running/15/ solution] is run
* [http://www.havetheknowhow.com/Configure-the-server/Install-LTSP.html How to install LTSP (Linux Terminal Server Project) on Ubuntu Server]
* Full Tutorial - Install and configure LTSP server and clients (ubuntu, ltsp, howto) [https://youtu.be/s1cguYmZ3-c Part 1] and [https://youtu.be/2aFcDlp9-xw Part 2]
==== [http://www.ltsp.org/ LTSP] ====
==== Raspberry Pi ====
[http://www.uzerp.com/blog/running-raspberry-pis-as-thin-clients-with-ubuntu-14-04-lts/ Running Raspberry Pi’s as Thin Clients with Ubuntu 14.04 LTS]
where '''-f''' option means full screen and '''-g''' means geometry.
==== Remote desktop connection to NCI ====
Note the information here is outdated now.
The version 1.7.1 of rdesktop program in Ubuntu 14 has a bug with mouse cursor (invisible). To fix the bug, download the version 1.8.3 and build it. See http://ubuntuforums.org/showthread.php?t=2266743
The new secure connection relies on the SmartCard is using ncits-p111.nci.nih.gov.
==== [https://www.realvnc.com/ RealVNC] ====
The default vnc server included in Ubuntu is not quite compatible with other clients. For example I can connect to Ubuntu 14.04 from Ubuntu 16.04 but not from a Mac. The 3rd party software is better. See the following screenshot after I have installed RealVNC server.
[[File:Realvnc-server.png|250px]]
Note that '''RealVNC Server is not free for commercial users but is free for home users''' up to 5 computers & 3 users; see https://manage.realvnc.com/. An email and a password can be used to sign in your account for activating the license in VNC Server.
To start VNC server, use (https://www.realvnc.com/docs/debian-install-remove.html)
<syntaxhighlight lang='bash'>
sudo /etc/init.d/vncserver-x11-serviced start
</syntaxhighlight>
The RealVNC will have an icon sitting on the tray at the top-right corner.
Before using VNC viewer, we also need to create a VNC password in VNC Server.
On Mac, the viewer can be launched from Applications -> VNC Viewer (if we have drag and drop the app to the Applications folder).
On Chrome OS, there is an [https://chrome.google.com/webstore/detail/vnc%C2%AE-viewer-for-google-ch/iabmpiboiopbgfabjmgeedhcmjenhbla VNC Viewer for Google Chrome] to use. The IP address is special. For example, 192.168.5.127:80 for port 5980. See https://www.realvnc.com/docs/faq/connect-fail.html. Unfortunately I cannot connect successfully:(
==== Remote desktop connection from Ubuntu to Ubuntu ====
* [http://ubuntuhandbook.org/index.php/2016/07/remote-access-ubuntu-16-04/ How to Remote Access to Ubuntu 16.04 from Windows]. The instruction is the same for older versions of Ubuntu.
** On a Ubuntu server. Go to Dash, type 'desktop sharing' and select it. Check sharing. Close the dialog. Open a terminal and run '''ps -ef | grep vino''' to make sure the server is running.
** [http://askubuntu.com/questions/477947/what-do-i-need-for-remotely-accessing-my-ubuntu-14-04-desktop Disable encryption]. Run ''sudo apt install dconf-editor''. Go to Dash and type 'dconf' and select 'dconf editor'. When it opens, navigate to '''org -> gnome -> desktop -> remote-access''', and uncheck the value of 'require-encryption.' Another way is to run '''gsettings set org.gnome.Vino require-encryption false'''.
** On a client machine (eg another Ubuntu), open a remote desktop client program (eg Remmina). Choose '''VNC''' as the protocol. Enter necessary information to connect to the server.
# Go to System -> Preference -> Remote desktop. Allow other uses to view your desktop & uncheck you must confirm each access & require user to enter this password.
# Go to System -> Preference -> Monitor. Change monitor resolution to 1280 x 720.
==== Allow for remote desktop connection when vino failed ====
* [http://www.teamviewer.com/en/index.aspx Teamviewer]. It works fine. After launching it, the software automatically creates an ID and password. We can change the password so it is fixed. Then launch the software on the client. Use the partner's ID and password to connect to it.
* [https://www.nomachine.com/ NoMachine]. I ran the service on my Ubuntu 12.04 server. I also tested the client on my Android 6.0 tablet and Odroid xu4 running Ubuntu 15.10. I am using the version 5.0.63. I need to use my server's user account info to connect.
If for some reason some nx programs disappeared, restart the service by
<syntaxhighlight lang='bash'>
sudo /usr/NX/bin/nxserver --restart
</syntaxhighlight>
It is interesting that nomachine is faster than 'ssh -X' method when I tested running a Qt application launched from Qt Creator. For example, the progress bar is not moving when it is supposed to move forward and backward when the app is launched through 'ssh -X'.
used in ''Linux Data Science Virtual Machine'' by Microsoft.
==== Remote desktop connection through vmware workstation ====
We shall be able to remote desktop connect to a Windows guest machine if the guest machine has configured to use bridge connection and a static IP. If there is a problem, it is likely caused by Window's firewall. See the two screenshots. We can just turn off the firewall of home network but keep the firewall on for the public network.
We don't need to use port forward for the remote desktop connection.
XAMPP is a free, open source cross platform web server solution stack package for Windows, Mac, and Linux. AMPPS is a software stack from Softaculous enabling Apache, MySQL, MongoDB, PHP, Perl, Python, and Softaculous auto-installer on a desktop.
This will install latex. After the end, we can use Synaptic package manager to see what were installed. Now following the instruction in https://help.ubuntu.com/community/MediaWiki, we remove the '#' from the third line so that it reads 'Alias /mediawiki /var/lib/mediawiki':
<syntaxhighlight lang='bash'>
sudo nano /etc/mediawiki/apache.conf
sudo /etc/init.d/apache2 restart
</syntaxhighlight>
Now we can start mediawiki by opening a browser and pointing it to http://localhost/mediawiki.
We need to enter
<syntaxhighlight lang='bash'>
Site config
admin username: WikiSysop
password:
Database config
Database name: wikidb
DB username:
DB password:
Superuser name: root
Superuser password: [depend on how it was chosen when installing MYSQL]
</syntaxhighlight>
Press the button of "Installing mediawiki". We will be welcomed to the wiki page. Following the instruction,
For security reason, I remove new account creation and anonymous editing. I also remove edit counters.
Backup and Restore mediawiki: The process involves 3 parts: mediawiki system, mysql and images. See docs.google.com note.
=== Install moinmoin ===
* Comparison of mediawiki vs moinmoin http://www.wikimatrix.org/compare/MediaWiki+MoinMoin
* Comparison of wiki software http://en.wikipedia.org/wiki/Comparison_of_wiki_software
* Moinmoin website http://moinmo.in/
* moinmoin was used by [http://wiki.fhcrc.org/bioc/HowTo Bioconductor], [https://wiki.ubuntu.com/MoinMoin ubuntu], etc.
=== UpnP server ===
==== PS3 Media Servver ====
https://github.com/ps3mediaserver/ps3mediaserver
==== minidlna ====
http://bbrks.me/rpi-minidlna-media-server/
It works even I use my phone to tether data (I don't need to turn on wifi on my phone).
<syntaxhighlight lang='bash'>
sudo apt-get update
sudo apt-get install minidlna
sudo nano /etc/minidlna.conf # the default location of media files is on /var/lib/minidlna
# rebuild the database. See the comments in <etc/minidlna.conf>
sudo service minidlna force-reload
sudo service minidlna start
sudo update-rc.d minidlna defaults # ask minidlna to start up automatically upon boot.
</syntaxhighlight>
Too bad is when I played certain videos the program crashed. The /var/log/syslog showed ''kernel: [96495.690373] minidlna[1627]: segfault at 0 ip 00007f4af2de9964 sp 00007fffa43014f8 error 4 in libc-2.15.so[7f4af2d54000+1b4000]''. Also the minidlna process becomes 2 instead of 1 after the crash.
==== Kodi ====
Too bad the Kodi's upnp function is not stable. Kodi server disappeared so the client cannot find it.
==== PLEX ====
This seems to be the best from my test.
* [http://www.makeuseof.com/tag/reasons-do-not-need-plex-pass/ 5 Reasons Why You Don’t Actually Need a Plex Pass] - live TV channels!
* [http://lifehacker.com/5975362/five-best-desktop-media-servers Best media server] from lifehacker.
* [http://lifehacker.com/home-theater-software-showdown-kodi-vs-plex-1746501974 Kodi vs Plex] from lifehacker.
* [http://www.howtogeek.com/252261/how-to-set-up-plex-and-watch-your-movies-on-any-device/ How to set up Plex (and Watch Your Movies on Any Device)] from howtogeek.
* Plex is running as a service. After we use web to configure, we can close the website.
* Users needs to sign up/sign in before completing the installation
* Access the admin page by http://IP-address:32400/web
* If new files were added, it will update the library. To do that, go to settings and check automatically update.
* Soft links works.
* Iso file cannot be read. Use [https://launchpad.net/~stebbins/+archive/ubuntu/handbrake-releases HandBrake] to create m4v files from iso files (seems to be fast enough; e.g. 5 minutes for a DVD).
* [http://www.howtogeek.com/259529/how-to-share-your-plex-media-library-with-friends/ How to Share Your Plex Media Library with Friends]
* [http://gizmodo.com/plex-just-killed-my-rental-cable-box-and-it-could-kill-1795679132 Plex Just Killed My Cable Box Rental, and It Could Kill Yours Too]
* [http://www.makeuseof.com/tag/improve-online-security-defend-privacy/ The Complete Guide to Improving Your Online Security and Defending Your Privacy]. It covers web browsers, browser extensions, email providers, search engines, VPN, password managers, OS, mobile, messaging, cloud storage & encryption tools.
==== Automatic security update ====
[https://www.howtoforge.com/tutorial/how-to-setup-automatic-security-updates-on-ubuntu-1604/ How to Setup Automatic Security Updates on Ubuntu 16.04]
==== https connection ====
* [http://security.stackexchange.com/questions/2914/can-my-company-see-what-https-sites-i-went-to Can my company see what HTTPS sites I went to?]
* [http://www.wsj.com/articles/SB118539543272477927 Ten Things Your IT Department Won't Tell You]
Connecting to ftp-trace.ncbi.nlm.nih.gov (ftp-trace.ncbi.nlm.nih.gov)|130.14.250.7|:443... connected.
ERROR: cannot verify ftp-trace.ncbi.nlm.nih.gov's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
Unable to locally verify the issuer's authority.
To connect to ftp-trace.ncbi.nlm.nih.gov insecurely, use `--no-check-certificate'.
Connecting to ftp-trace.ncbi.nlm.nih.gov (ftp-trace.ncbi.nlm.nih.gov)|130.14.250.11|:443... connected.
WARNING: cannot verify ftp-trace.ncbi.nlm.nih.gov's certificate, issued by ‘CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US’:
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 63707890 (61M) [application/x-gzip]
Saving to: ‘sratoolkit.2.7.0-ubuntu64.tar.gz’
sratoolkit.2.7.0-ubuntu64.tar. 100%[==================================================>] 60.76M 6.50MB/s in 9.2s
Use the following command to download required components before building any R packages. See also [http://cran.r-project.org/bin/linux/ubuntu/ ubuntu package for R]
<pre>
sudo apt-get build-dep r-base
</pre>
=== [http://appimage.org/ AppImage] file - new way of installing an application ===
[http://askubuntu.com/questions/774490/what-is-an-appimage-how-do-i-install-it What is an “AppImage”? How do I install it?]
* AppImages can be downloaded and run without installation or the need for root rights.
* The key idea of the AppImage format is one app = one file. Every AppImage contains an app and all the files the app needs to run. In other words, each AppImage has no dependencies other than what is included in the targeted base operating system(s).
* [http://www.tecmint.com/useful-basic-commands-of-apt-get-and-apt-cache-for-package-management/ 25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management]
* [http://askubuntu.com/questions/766823/apt-get-being-replaced-with-apt apt-get being replaced with apt? in Ubuntu 16.04] '''apt''' now comes with a progress bar, coloring, etc.
==== Package repository and /etc/apt/sources.list ====
echo "deb https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list
# Update and install syncthing:
sudo apt-get update
sudo apt-get install syncthing
</syntaxhighlight>
==== E: Could not get lock /var/lib/dpkg/lock ====
I get the above message when I run '''sudo apt-get update'''.
'''Solution 1:'''
[http://askubuntu.com/questions/15433/unable-to-lock-the-administration-directory-var-lib-dpkg-is-another-process Unable to lock the administration directory (/var/lib/dpkg/) is another process using it?].
I can reproduce the error from a clean boot. The most possible explanation is the answer from poolie:
''the command-line apt overlaps with update-manager automatically polling. So if you try again in a few minutes that should fix it.''
From my experience, after I wait about 10 minutes, '''ps -A | grep apt''' won't show anything.
'''Solution 2:'''
[http://www.tecmint.com/fix-unable-to-lock-the-administration-directory-var-lib-dpkg-lock/ Fix “Unable to lock the administration directory (/var/lib/dpkg/)” in Ubuntu]
<syntaxhighlight lang='bash'>
# Method 1: Find and Kill all apt-get or apt Processes
$ ps -A | grep apt
1760 ? 00:00:00 apt.systemd.dai
3489 ? 00:00:00 aptd
$ sudo kill -9 1760
$ sudo kill -9 3489
$ sudo apt-get install keepass2
E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
# Method 2: Delete the lock Files
sudo rm /var/lib/apt/lists/lock
# and
sudo rm /var/lib/dpkg/lock
</syntaxhighlight>
==== dpkg error: Package is in a very bad inconsistent state ====
However, installing from a .deb file through dpkg does not resolve dependency issues the same way installation through apt will. See [https://askubuntu.com/questions/784935/unable-to-install-r-studio here].
So you should run sudo apt-get install -f to fix and complete the installation if any errors were encountered by dpkg.
<syntaxhighlight lang='bash'>
sudo apt install -f dep1.deb dep2.deb dep3.deb
</syntaxhighlight>
A better way to install a deb file is using the '''gdebi''' tool.
==== Remove deb packages ====
See the [http://manpages.ubuntu.com/manpages/trusty/man1/dpkg.1.html man page].
<syntaxhighlight lang='bash'>
dpkg -l | grep 'tcl'
sudo dpkg -r tcl8.4 # -r means remove/uninstall
sudo dpkg -P tcl8.4 # -P means purge
dpkg -l | grep 'tcl' (optional)
</syntaxhighlight>
==== gdebi: an improvement over dpkg ====
[https://askubuntu.com/questions/621351/gdebi-vs-dpkg-how-does-gdebi-automatically-gets-missing-dependancies-can-i-u How does gdebi automatically gets missing dependancies ? Can I use gdebi for all other installations of .deb packages?]
<syntaxhighlight lang='bash'>
sudo apt-get install gdebi-core
# wget newPackage.deb
# sudo gdebi -n newPackage.deb
</syntaxhighlight>
==== deb files caches ====
See http://askubuntu.com/questions/444240/ubuntu-updates-blocked-apt-get
<pre>
sudo rm -f /etc/apt/sources.list.d/*
sudo dpkg -i /var/cache/apt/archives/*.deb
sudo dpkg --configure -a
</pre>
==== List all available packages (from repositories) ====
<syntaxhighlight lang='bash'>
apt-cache pkgnames # pkgnames is part of the command
</syntaxhighlight>
==== Show package information ====
<syntaxhighlight lang='bash'>
apt-cache show <package_name>
</syntaxhighlight>
==== Check dependencies for specific packages ====
<syntaxhighlight lang='bash'>
apt-cache showpkg <package_name>
</syntaxhighlight>
==== Upgrade a package ====
<syntaxhighlight lang='bash'>
apt-get install <package_name>
</syntaxhighlight>
==== Install specific version of the package ====
Use the "=" with the package-name and append desired version.
<syntaxhighlight lang='bash'>
sudo apt-get install vsftpd=2.3.5-3ubuntu1
</syntaxhighlight>
==== Remove a package ====
<syntaxhighlight lang='bash'>
sudo apt-get remove <package_name>
</syntaxhighlight>
To completely remove a package including their configuration files
Note, selecting 'libgl1-mesa-dev-lts-quantal' for regex 'libgl1-mesa.dev'
Note, selecting 'libgl1-mesa-dev' for regex 'libgl1-mesa.dev'
Note, selecting 'libgl1-mesa-dev-lts-saucy' for regex 'libgl1-mesa.dev'
Note, selecting 'libgl1-mesa-dev-lts-trusty' for regex 'libgl1-mesa.dev'
Note, selecting 'libgl1-mesa-dev-lts-utopic' for regex 'libgl1-mesa.dev'
Note, selecting 'libgl1-mesa-dev-lts-vivid' for regex 'libgl1-mesa.dev'
Note, selecting 'libgl1-mesa-dev-lts-raring' for regex 'libgl1-mesa.dev'
libgl1-mesa-dev is already the newest version.
libgl1-mesa-dev set to manually installed.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libgl1-mesa-dev : Conflicts: libgl-dev
Conflicts: libgl1-mesa-dri-dev
libgl1-mesa-dev-lts-utopic : Depends: mesa-common-dev-lts-utopic (= 10.3.2-0ubuntu1~trusty2) but it is not going to be installed
Depends: libgl1-mesa-glx-lts-utopic (= 10.3.2-0ubuntu1~trusty2) but it is not going to be installed
Conflicts: libgl-dev
Conflicts: libgl1-mesa-dev
Conflicts: libgl1-mesa-dri-dev
libgl1-mesa-dev-lts-vivid : Depends: mesa-common-dev-lts-vivid (= 10.5.9-2ubuntu1~trusty2) but it is not going to be installed
Depends: libgl1-mesa-glx-lts-vivid (= 10.5.9-2ubuntu1~trusty2) but it is not going to be installed
Conflicts: libgl-dev
Conflicts: libgl1-mesa-dev
Conflicts: libgl1-mesa-dri-dev
Conflicts: xorg-renamed-package-lts-utopic
E: Unable to correct problems, you have held broken packages.
</syntaxhighlight>
[https://askubuntu.com/questions/914428/unmet-dependencies-when-trying-to-install-r-base/914429 Unmet dependencies when trying to install r-base] (Works on Ubuntu 16.04).
<syntaxhighlight lang='bash'>
sudo apt --fix-broken install
sudo apt-get update
sudo apt-get upgrade
</syntaxhighlight>
=== Terminal Assistant ===
* [https://github.com/pickhardt/betty Betty]: [http://www.makeuseof.com/tag/use-natural-language-with-betty-your-digital-assistant-in-the-linux-terminal/ makeuseof] and [http://www.infoworld.com/article/2696910/open-source-tools/install-the-command-line-virtual-assistant-betty-in-linux-mint-17.html infoworld]
* [https://github.com/semicode-ltd/sarah Sarah]
=== Torrent ===
[http://www.makeuseof.com/tag/top-6-torrent-alternatives-to-pirate-bay-downloads/ Top 6 Torrent Alternatives to The Pirate Bay] (7/5/2017)
(old) Popular search sites
* torrentz
* thepiratebay
* isohunt
* kickass torrents
==== How to set up torrent scheduling on Linux ====
https://www.howtoforge.com/tutorial/how-to-set-up-torrent-scheduling-on-linux/. It covers 'Deluge', 'Transmission' and 'Rtorrent'.
This can be useful for people who want to take advantage of their computer while they are not using it, like during the nighttime for example. This way, large portions of huge files can be downloaded without delaying your work activities, or interrupting/undermining your media consumption.
aria2 is a lightweight multi-protocol & multi-source command-line download utility. It supports '''HTTP/HTTPS, FTP, BitTorrent''' and '''Metalink'''. aria2 can be manipulated via built-in '''JSON-RPC''' and '''XML-RPC''' interfaces.
* [https://www.digitalocean.com/community/tutorials/how-to-set-up-a-host-name-with-digitalocean How To Set Up a Host Name with DigitalOcean] This includes information about WHOIS, changing domain server, configuring domain (A record, AAAA record, CNAME record, MX record, et al)
-rw-r--r-- 1 root wheel 9390 May 15 2016 openssl.cnf
drwxr-xr-x 2 root wheel 68 May 15 2016 private
$ ls -l /System/Library/OpenSSL/certs/
$ # empty results
</syntaxhighlight>
==== openssl & patch bug ====
* Check out https://launchpad.net/ubuntu/+source/openssl to see the latest openssl version (number may be different for each of Ubuntu version). As of this writing, the latest openssl on Ubuntu 14.04 is 1.0.1f-1ubuntu2.15 and for Ubuntu 12.04 it is 1.0.1-4ubuntu5.31 (this kind of representation can be obtained using the sudo dpkg -l | grep openssl command; see below). '''The full list of the publishing history can be accessed through [https://launchpad.net/ubuntu/+source/openssl/+publishinghistory View full publishing history link].''' From there, we can restrict to Target = Trusty, for example.
* http://askubuntu.com/questions/444702/how-to-patch-the-heartbleed-bug-cve-2014-0160-in-openssl. The following is an output after running ''sudo apt-get update; sudo apt-get upgrade''.
<pre>
brb@vm-1404:~$ sudo dpkg -l | grep openssl
ii libgnutls-openssl27:amd64 2.12.23-12ubuntu2.2 amd64 GNU TLS library - OpenSSL wrapper
ii python-openssl 0.13-2ubuntu6 amd64 Python 2 wrapper around the OpenSSL library
</pre>
* http://www.liquidweb.com/kb/update-and-patch-openssl-on-ubuntu-for-the-ccs-injection-vulnerability/. As you can see although a bug in OpenSSL has been found affecting versions 1.0.1 through 1.0.1f (inclusive), and openssl version is still 1.0.1f in Ubuntu 14.04.1, the build date is on June 2014. So it is safe.
- debian/patches/CVE-2014-0224-regression2.patch: accept CCS after
- debian/patches/CVE-2014-0224.patch: set the CCS_OK flag when using
- debian/patches/CVE-2014-0224-1.patch: only accept change cipher spec
- debian/patches/CVE-2014-0224-2.patch: don't accept zero length master
- debian/patches/CVE-2014-0224-3.patch: allow CCS after resumption in
- CVE-2014-0224
</pre>
* [http://www.cyberciti.biz/faq/howto-openssl-security-update-cve20150291-cve20150204-cve20150290-cve20150207-cve20150286/ How To Patch and Protect OpenSSL Vulnerability # CVE-2015-0291 CVE-2015-0204 [19/March/2015]]
* [https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-16-04 How To Secure Apache with Let's Encrypt on Ubuntu 16.04]: install '''python-certbot-apache''' package
* [https://www.upcloud.com/support/install-lets-encrypt-apache/ How to Install Let’s Encrypt on Apache2]: install '''python-letsencrypt-apache''' package
It takes less than 5 minutes for the whole process!
<syntaxhighlight lang='bash'>
sudo apt-get update
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache
sudo certbot --apache
</syntaxhighlight>
Let’s Encrypt certificates are valid for 90 days, but it’s recommended that you renew the certificates every 60 days to allow a margin of error. The Let's Encrypt client has a renew command that automatically checks the currently installed certificates and tries to renew them if they are less than 30 days away from the expiration date.
So we can add the following line to the cron job that will execute the letsencrypt-auto renew command every Monday at 2:30 am. '''sudo crontab -e'''
<pre>
30 2 * * 1 certbot renew >> /var/log/le-renew.log
</pre>
'''Note''' if you currently have already had a working SSL certificate (eg from a purchased one), then even you have removed the certificate from the relevant Apache files you still can't run through ''sudo certbot''. But once your current working SSL certificate expires, ''sudo certbot'' will successfully go through even you don't manually remove the certificate from your system.
=== File does not exist: /var/www/favicon.ico from /var/log/apache2/error.log ===
The reason? You have not created a favicon, also known as a website icon, for your website. It’s the icon that displays in the address bar of your web browser when you connect to a website. A web browser will request this icon file from every website.
If you choose to create one. Use a program such as Gimp and create a 16×16 pixel image and save it as a .ico filetype. Then upload that file to the DocumentRoot of your website. You will need one for each VirtualHost. If you don’t have Gimp, there are online resources such as [http://www.favicon.cc/ favicon.cc] where you can create a .ico file and download it for your own use.
As you know by now, not having a favicon.ico file, won’t stop web browsers from requesting it each time. But you can tell Apache not to log the event as an error message. You will still see the request in the access.log, but at least you will have a cleaner error.log file.
Add the following block of code to each VirtualHost, or at least the ones which don’t have a favicon file.
<pre>
Redirect 404 /favicon.ico
<Location /favicon.ico>
ErrorDocument 404 "No favicon"
</Location>
</pre>
Don’t forget to restart apache after making the change. If you want make a “global” change, which would apply to any and all VirtualHosts, you can create a file in Apache’s conf.d folder with a name such as nofavicon.conf and then add that block of code to the file. That would disable favicon across the board and save you from having to edit each VirtualHost.
Or, you create an empty file with the name “favicon.ico” in the directory root of Apache (for exemple /var/www/).
Install '''cheese''' or '''guvcview'''. [https://thinkrpi.wordpress.com/2013/04/05/step-3-install-softwares-for-webcam-and-computer-vision/ An example of using computer vision] on Raspberry Pi.
==== [https://www.maketecheasier.com/listen-tunein-radio-in-vlc/ Use VLC to listen tunein radio] ====
Download link http://addons.videolan.org/content/show.php?content=152788. Right click the links and save them (3 lua files)
Create two new subfolders and place the lua files
* move the “tunein.lua” file to the VLC folder so it can be found and run by VLC.
** In Linux, move the “tunein.lua” file to ~/.local/share/vlc/lua/sd/
** In Windows, move the “tunein.lua” file to %AppData%/VLC/VideoLAN/lua/sd.
** In Mac OS X, move the “tunein.lua” file to VLC.app/Contents/MacOS/share/lua/sd
* move the “radiotimes.lua” and “streamtheworld.lua” files from the Playlist folder to “/lua/playlist” folder in the VLC directory.
Open VLC. Go to “View -> Playlist”. At the sidebar, under the Internet section, you should now see “TuneIn Radio” in the list. Click on it and you will be able to access your Favorites.
==== Taiwan radio ====
* rtsp://live2.rti.org.tw/rti4.sdp (discovered by open http://music.rti.org.tw/music in Linux Chrome browser)
* rtmp://news98-t24.griffinmas.com/live/news98
==== Use VLC to record internet radio (suitable if the stream is continuous) ====
[0x7fac2c003598] access_mms access error: failed to send command
[0x7fac2c005fe8] idummy demux: command `quit'
</pre>
==== Play audio using a command line mode ====
See also my [[Beaglebone#Playing_music_using_command_line_tools|Beaglebone]] page for a comparison of different possibilities. For VLC, there are 3 [https://wiki.videolan.org/Console/ interface modes]. The following example is to run vlc in a [http://www.tldp.org/REF/VLC-User-Guide/x625.html text mode] with the ncurses library.
<syntaxhighlight lang='bash'>
sudo apt-get install vlc-nox
vlc -I ncurses XXX.mp3
vlc --help
</syntaxhighlight>
==== Play youtube using VLC from a command line ====
See [http://raspberrypi.stackexchange.com/questions/1556/why-wont-flash-player-on-chromium-work this post]
[https://www.cheatography.com/shaunumb/cheat-sheets/vlc-windows-and-os-x/ cheatography.com] and [http://www.howtogeek.com/196371/master-vlc-with-these-23-keyboard-shortcuts/ howtogeek]
* Ctrl + arrow: forward/backward 1 minute (Add the 'CMD' key on macOS)
* Alt + arrow: forward/backward 10 seconds (Add the 'CMD' key on macOS)
* Shift + arrow: forward/backward 3 seconds (Add the 'CMD' key on macOS)
* [ or -: decrease speed
* ]: increase speed
==== VLC & screensaver ====
It seems the screensaver will be deactivated if some video is opened (even not it is not playing).
=== SMPlayer ===
* [http://www.ubuntugeek.com/install-smplayer-on-ubuntu-15-10.html Install SMPlayer on Ubuntu 15.10]
If I use TortoiseCVS (1.12.5 from 1/24/2011), I need to choose ext as protocol instead ssh. Still the checked out file 'hmv_.rc' still contains unreadable Chinese characters. The cvsnt is the latest free version (2.5.05). If I want to use WinCVS + cvsnt from TortoiseCVS, the options in the CVSROOT dialog looks weird and cannot create a connection.
For the unicode encoding. If I commit the file at first from ubuntu os, but check out in Windows. The checked out file has right encoding (using Notepad ++, or from VS2010). However, the file does not have right line ending and it shows Chineses character when I open it in either Notepad++ or VS2010.
To see hidden characters in Linux, try either one of the following 2 methods:
* Open the file in EMACS and do a M-X hexl-mode
* geany editor.
The solution I have found to overcome accessing unicode (utf-16) file on Windows OS is using Cygwin.
* Download setup.exe from http://cygwin.com/install.html
* Root directory = c:\cygwin
* Local package directory = C:\Users\brb\Downloads
* Direct connection
* Download site: ftp://cygwin.mirrors.pair.com (Some mirrors are not updated & contain old version of packages! For example, make sure the cvs version is 1.12.13.)
* Search: cvs. Click plus sign next to "Devel". Click 'Skip' corresponding to cvs package.
* Search: ssh. Click plus sign next to "Net". Click 'skip' correspond to openssh package.
* Click 'Next' button.
* Click 'Finish' button.
* Now open 'Cygwin Terminal' icon on Windows Desktop.
The 'mycvs' directory should be under C:\cygwin\home\brb (a.k.a. /home/brb in cygwin) directory. We can open 'hmv_.rc' file in Notepad++ to double check if the file looks normal OR use md5sum to check.
==== Difference between CRLF (Windows), LF (Linux, Mac) and CR ====
This is a good summary I found: http://stackoverflow.com/questions/1552749/difference-between-cr-lf-lf-and-cr-line-break-types
* The Carriage Return (CR) character (0x0D, \r) moves the cursor to the beginning of the line without advancing to the next line. This character is used as a new line character in Commodore and Early Macintosh operating systems (OS-9 and earlier).
* The Line Feed (LF) character (0x0A, \n) moves the cursor down to the next line without returning to the beginning of the line. This character is used as a new line character in UNIX based systems (Linux, Mac OSX, etc)
* The End of Line (EOL) character (0x0D0A, \r\n) is actually two ASCII characters and is a combination of the CR and LF characters. It moves the cursor both down to the next line and to the beginning of that line. This character is used as a new line character in most other non-Unix operating systems including Microsoft Windows, Symbian OS and others.
It helps with CVS log in too when the CVS works by using ssh protocol. Note that step 3 allows to run a shell command at a remote machine.
See https://help.github.com/articles/generating-ssh-keys also for similar instruction when work on github.
The ssh key can be copied to another a machine (pay attention to mode). Or let the new machine to create its own key pair and use '''ssh-copy-id''' to append the identity file to remote machine's ~/.ssh/authorized_keys file. See http://superuser.com/questions/332510/how-to-transfer-my-linux-ssh-keys-to-another-machine.
We can even have multiple ssh key on local machine by using <'''.ssh/config'''> file. See
The following approach assumes the boot loader will not be put on internal hard disk if you are careful enough. See also http://fernhilllinuxproject.com/guidesandhowtos/installubuntutousbdrive.html
# Boot from live CD in ubuntu.
# Insert the USB flash drive in USB Port.
# On desktop double click on icon 'install ubuntu 12.04'
# Click on continue and it will ask if you want to unmount the flash drive click on yes
# Choose some thing else when ask to where to install ubuntu.
# Next you will see your hard disk partitions and flash drive. Click on flash drive partition and then click on change. Change the partition type to ext4 and mount as to '/'. Click on ok to close the dialogue box.
# In last you will see a drop down menu on where to install the the boot loader. Initially it will be showing sdc,sdd but on drop down you must select sdc1 or sdd1. Failing this step you may render your computer unbootable.
# Then click on install and linux will be installed on your USB Flash drive.
=== Install a new hard drive ===
See also https://help.ubuntu.com/community/InstallingANewHardDrive and [[#Mount_drive|Mount drive]]
# Use sudo fdisk command to create partition table. Then 'n', 'p', '1', 'w' and several returns.
# Use sudo mkfs -t ext4 /dev/sdb1 to create a new partition
# Use sudo mkdir /mnt/ssd to create a new mount point
# Use sudo nano -Bw /etc/fstab to do auto mount on boot
# sudo chown -R USERNAME:USERNAME /mnt/ssd to give the ownership to the USERNAME.
=== How to install a Ceph Storage Cluster on Ubuntu ===
Here is my record for creating a cluster environment based on ubuntu 13.04. The master node is running on ubuntu 13.04 desktop with virtualBox 4.2. The virtualBox has added a host-only adapter (vboxnet0) with ip 192.168.56.1. This adapter will be added to the master node so I can use this ip to identify the master node in the host-only network. Creating cluster using VirtualBox is just for the education purpose, not for real practice.
* (virtualBox) Create two virtual machines running on ubuntu 13.04 server. The ssh server was checked during installation. The host name for each of them is ubuntuNode1 and ubuntuNode2 respectively. The network adapter is left by default (NAT) during installation. But after the installation is done, I shutdown the system and add a host-only adapter (vboxnet0) to each of them. Then after the system is up again, I change the IP so it is static. Do '''sudo nano /etc/network/interfaces''' and append the following before running '''sudo /etc/init.d/networking restart''' to take the change in effect.
<pre>
auto eth1
iface eth1 inet static
address 192.168.56.101 # use 192.168.56.102 for ubuntuNode2
netmask 255.255.255.0
network 192.168.56.0
broadcast 192.168.56.255
</pre>
Note that it is better not to add host-only network before installation, or during installation it will ask what is the primary network (confusing). So at the end each new nodes should have both '''eth0''' and '''eth1''' adapters and they should not reside in same subset. '''ifconfig''' may not show all adapters so we should use '''ifconfig -a''' instead. Although host-only network is used for communication between guest and guest OR guest and host, since each guest node has NAT adapter by default so the guest machine can still access the internet.
* (master node) Edit /etc/hosts so it like like
<pre>
127.0.0.1 localhost
192.168.56.1 ubuntu1304
192.168.56.101 ubuntuNode1
192.168.56.102 ubuntuNode2
</pre>
Note that the master node will be used to start jobs on the cluster although it is OK to let the master node as one of nodes to execute the jobs.
* (all nodes) Run
<pre>
sudo adduser mpiuser --uid 999
</pre>
It is recommeneded to use the same password for the user. This will create a new directory ''/home/mpiuser''. This is the home directory for user ''mpiuser'' and we will use it to execute jobs on the cluster.
* (master node) Run
<pre>
sudo apt-get install nfs-kernel-server
</pre>
* (other nodes) Run
<pre>
sudo apt-get install nfs-client
</pre>
* (master node) Add the following to the file '''/etc/exports'''
Some people create a shared folder under /srv directory.
Now run
<pre>
sudo service nfs-kernel-server restart
</pre>
* (master node)
<pre>
sudo ufw allow from 192.168.56.0/24
</pre>
* (other nodes)
<pre>
sudo mount ubuntu1304:/home/mpiuser /home/mpiuser
</pre>
And if we want to mount the NFS shared directory when the compute nodes are booted, edit '''/etc/fstab''' by adding
<pre>
ubuntu1304:/home/mpiuser /home/mpiuser nfs
</pre>
* (master node)
<pre>
sudo apt-get install ssh
su mpiuser
ssh-keygen
ssh-copy-id localhost
</pre>
We can test if the ssh works without passwords
<pre>
ssh ubuntuNode1
echo $HOSTNAME
</pre>
* (all nodes)
<pre>
sudo apt-get install mpich2
which mpirun
which mpiexec
</pre>
* (master node, mpiuser)
Go to the home directory of mpiuser and create a new file '''hosts'''. Include host names for computing nodes (it is OK to include master node, ubuntu1304, as one of computing nodes)
<pre>
ubuntuNode1
ubuntuNode2
</pre>
* (all nodes)
<pre>
sudo apt-get install build-essential
</pre>
* (master node, mpiuser). Creating a test file <mpi_hello.c>
<pre>
#include <stdio.h>
#include <mpi.h>
int main(int argc, char** argv) {
int myrank, nprocs;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
printf("Hello from processor %d of %d\n", myrank, nprocs);
* [http://ubuntuhandbook.org/index.php/2014/09/3-ways-create-wifi-hotspot-ubuntu/ 3 Ways to Create Wifi Hotspot in Ubuntu 14.04]
* https://www.quora.com/How-do-I-create-hotspots-in-Ubuntu-14-04. Notice the last step.
When I tested the method with my rtl8187 wifi adapter by Rosewill (output by '''lsusb''') on Odroid Xu4 running Ubuntu 14.04, I got an error on Step 3 (Create New Wi-Fi Network) ???
<pre>
Failed to activate connection
(32) Access Point (AP) mode is not supported by this device
</pre>
When I tried the wifi adapter RT2870/RT3070 802.11n by MediaLink (MWN-USB150N), there is no problem to create it. My android device can connect to it. However, there is no internet access:( ...... After some tweaks on command line (iptables; see ''Sharing internet via wifi: iptables'' below), it works!!!
And the output of '''nm-tool''' command also confirms the wifi device is connected.
The ap hotspot from the wifi adapter has an IP 10.42.0.1 (ifconfig).
==== Sharing internet via wifi: ap-hotspot ====
I cannot make it to work to share the internet via wifi on my Xubuntu 13.04. However the solution in http://askubuntu.com/questions/287251/creating-an-infrastructure-hotspot-using-ubuntu-12-10 works for me.
<syntaxhighlight lang='bash'>
$ # Under rare situation, we need to run the following line to get 'add-apt-repository'
In the step of 'ap-hotspot configure' it will ask for Access Point name and WPA passphrase.
When I run the final line 'ap-hotspot start', it will ask me to disconnect my current wifi first.
At the end, I found my original configuration in ubuntu does not work even its setting is there. '''apt-hotspot''' create an infrastracture AP instead of adhoc.
==== Sharing internet via wifi: iptables ====
1. Assume the primary wired network connection, eth0 is connected to Internet.
2. Using your distro's network managment tool, create a new ad hoc wireless connection with the following settings:
<pre>
IP address: 10.99.66.55
Subnet mask: 255.255.0.0 (16)
</pre>
3. Use the following shell script to share the internet connection
iptables -A FORWARD -i $1 -o $2 -s 10.99.0.0/16 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A POSTROUTING -t nat -j MASQUERADE
</syntaxhighlight>
4. Run the script as follows:
<syntaxhighlight lang='bash'>
sudo ./netsharing.sh eth0 wlan0
</syntaxhighlight>
5. Connect your devices to the wireless network you just created with the following settings (Question: how about the SSID here?)
<pre>
IP address: 10.99.66.56 (and so on)
Subnet mask: 255.255.0.0
</pre>
To make this more convenient, you might want to install a DHCP and DNS server on your machine, so it's not necessary to configure IPs on devices manually. A handy tool for this is '''dnsmasq''' which you can use for performing both DHCP and DNS operations.
Credit: [https://www.packtpub.com/application-development/linux-shell-scripting-cookbook-second-edition Linux Shell Scripting Cookbook]
[http://forum.odroid.com/viewtopic.php?f=112&t=9321 A similar approach] discussed on the Odroid forum.
==== iptables ====
* [[Beaglebone#Internet_over_USB|Beaglebone]]
=== ~/.xsession-errors file is filling the hard disk ===
Short solution is to use "kill -9 xxx" to kill the process and rm to remove ~/.xsession-errors file. The long time solution is to uninstall vino.
=== JRE and JDK ===
Install openjdk or Sun jdk. See http://www.maketecheasier.com/install-java-runtime-in-ubuntu/ (Ubuntu 12.04)
If we have multiple versions of JRE/JDK, we can use the [https://www.digitalocean.com/community/tutorials/how-to-install-java-on-ubuntu-with-apt-get following command] to set the default version
<pre>
sudo update-alternatives --config java
</pre>
This approach seems to be working in the case JAVA_HOME cannot be honored.
Question: [http://askubuntu.com/questions/64329/how-to-replace-openjdk-6-with-openjdk-7 how to switch to 1.6 or 1.7 version of java?] (For example, snpEff requires java 1.7)
Question: [http://askubuntu.com/questions/464755/how-to-install-openjdk-8-on-14-04-lts How to install OpenJDK 8 on 14.04 LTS?] (for example, [https://github.com/broadinstitute/picard Picard 2] requires Java 1.8)
<syntaxhighlight lang='bash'>
$ sudo apt-get -y install openjdk-8-jdk # works for Ubuntu 14.10 and later
# Unable to locate package openjdk-8-jdk on my Ubuntu 14.04
echo debconf shared/accepted-oracle-license-v1-1 seen true | sudo debconf-set-selections
sudo apt-get install oracle-java8-installer
java -version
</syntaxhighlight>
=== CPU/system load ===
==== I-Nex ====
[https://www.ghacks.net/2017/07/23/a-look-at-i-nex-a-cpu-z-like-application-for-gnulinux/ A look at I-Nex – a CPU-Z like Application for GNU/Linux]
==== CPU-G ====
See [http://blog.hostonnet.com/how-to-view-cpu-info-in-ubuntu-cpu-z-alternative-for-linux this instruction] to install CPU-G program for viewing hardware information (process, mb, graphic, memory, system).
[http://www.howtogeek.com/118908/10-awesome-indicator-applets-for-ubuntus-unity-desktop/ System Load Indicator]: it is used to view system information (cpu, memory, network) in real-time.
[ 1.512669] ata1: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f100 irq 19
[ 1.512672] ata2: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f180 irq 19
[ 1.512675] ata3: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f200 irq 19
[ 1.512677] ata4: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f280 irq 19
[ 1.832349] ata4: SATA link down (SStatus 0 SControl 300)
[ 1.832418] ata2: SATA link down (SStatus 0 SControl 300)
[ 2.004290] ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 2.004313] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
</syntaxhighlight>
To test the disk performance, follow [http://www.cyberciti.biz/faq/howto-linux-unix-test-disk-performance-with-dd-command/ cyberciti.biz]. Note that the parameter '''oflag=dsync''' makes a difference.
Here I have a 7200rpm Seagate (ST2000DM001-1CH164) and a 5400rpm WD-blue (WD30EZRZ-00WN9B0)
<syntaxhighlight lang='bash'>
# Hard disk info https://en.wikipedia.org/wiki/Hdparm
sudo hdparm -I /dev/sda
sudo hdparm -I /dev/sdb
# Writing speed, where /tmp is a directory from the disk
See [[Linux#Display_directory_size_with_sorting_and_human_readable|Display directory size with sorting and human readable]] by using the '''ncdu''' utility.
==== Monitor/run a short test on a hard disk health by command line/GUI ====
Using [https://www.howtoforge.com/tutorial/monitor-harddisk-with-smartmon-on-ubuntu/ smartctl]
<syntaxhighlight lang='bash'>
sudo apt-get install smartcontrol
sudo smartctl -a /dev/sda
# Run a short test
$ sudo smartctl -t short /dev/sda # It will start the test in the background
where [https://ubuntuforums.org/archive/index.php/t-2010895.html nouveau is an open-source driver set for Nvidia cards]. It is not clear about acpitz-virtual-0. Some suggested to use [http://code.google.com/p/inxi/ '''inxi'''] which will produce human readable system info.
And on a Dell T3600 machine (running the samtools mpileup & bcftools programs),
<pre>
brb@T3600 ~ $ sensors
nouveau-pci-0300
Adapter: PCI adapter
fan1: 3510 RPM
temp1: +52.0°C (high = +95.0°C, hyst = +3.0°C)
(crit = +105.0°C, hyst = +5.0°C)
(emerg = +135.0°C, hyst = +5.0°C)
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +70.0°C (high = +81.0°C, crit = +91.0°C)
Core 0: +60.0°C (high = +81.0°C, crit = +91.0°C)
Core 1: +60.0°C (high = +81.0°C, crit = +91.0°C)
Core 2: +56.0°C (high = +81.0°C, crit = +91.0°C)
Core 3: +60.0°C (high = +81.0°C, crit = +91.0°C)
Core 4: +70.0°C (high = +81.0°C, crit = +91.0°C)
Core 5: +60.0°C (high = +81.0°C, crit = +91.0°C)
</pre>
When all cores are 100% used ('''htop'''), the fan is getting noisy
<pre>
brb@T3600 ~ $ sensors
nouveau-pci-0300
Adapter: PCI adapter
fan1: 4560 RPM
temp1: +61.0°C (high = +95.0°C, hyst = +3.0°C)
(crit = +105.0°C, hyst = +5.0°C)
(emerg = +135.0°C, hyst = +5.0°C)
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +82.0°C (high = +81.0°C, crit = +91.0°C)
Core 0: +78.0°C (high = +81.0°C, crit = +91.0°C)
Core 1: +81.0°C (high = +81.0°C, crit = +91.0°C)
Core 2: +78.0°C (high = +81.0°C, crit = +91.0°C)
Core 3: +80.0°C (high = +81.0°C, crit = +91.0°C)
Core 4: +81.0°C (high = +81.0°C, crit = +91.0°C)
Core 5: +77.0°C (high = +81.0°C, crit = +91.0°C)
</pre>
=== Unlock keyring ===
I got the prompt of unlocking keyring every time I open google chrome browser.
* http://askubuntu.com/questions/184266/what-is-unlock-keyring-and-how-do-i-get-rid-of-it (this does not help)
* http://superuser.com/questions/311216/why-does-chrome-ask-for-my-gnome-keyring-seahorse-password (works. I create a shortcut to launch chrome)
<pre>
google-chrome --password-store=basic
</pre>
It is also helpful to remove ubuntu one from Ubuntu 12.04. See http://hex.ro/wp/blog/removing-ubuntuone-from-ubuntu-12-04/
=== netcat (nc) - arbitrary TCP and UDP connections and listens ===
Netcat or nc is a networking utility for debugging and investigating the network.
The nc (or netcat) utility is used for just about anything under the sun involving TCP, UDP, or UNIX-domain sockets. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6.
For example, we can create simple network sockets and use them for text communication. We need two sockets: one listens for connections and the other connects to this one.
<syntaxhighlight lang='bash'>
# create a listening socket on the local machine
nc -l 1234
# connect to the socket from a 2nd computer
nc IP_LocalMachine 1234
# send messages.
# Type something and press Enter on the terminal from the 2nd computer.
# The message will appear on the terminal of the local machine.
</syntaxhighlight>
To transfer files over the network ('''no any password is needed!!''')
<syntaxhighlight lang='bash'>
# receiver machine
nc -l 1234 > destination_file
# sender machine
nc IP_Receiver 1234 < source_filename
</syntaxhighlight>
=== List of all services/daemons ===
Run service '''--status-all''' to get a list off all the Upstart services and their status. See
* man service
* man initctl
<pre>
service --status-all # output format is clean
sudo initctl list # show the process number too
</pre>
where in the output "+" means started, "-" stopped, and "?" unknown.
=== Google Drive or other cloud services ===
* [https://www.howtoforge.com/tutorial/how-to-access-your-google-drive-account-using-overgrive/ How to access your Google Drive account using overGrive]
* Use [http://rclone.org rclone] program. See http://wiki.linuxquestions.org/wiki/Rsync_with_Google_Drive
* [https://www.howtoforge.com/tutorial/how-to-access-google-drive-from-linux-gdrive/ How to access your Google Drive account from Linux command line using Gdrive]
* http://www.linux-magazine.com/Issues/2015/171/Video-Editor-Roundup/(offset)/9 with a conclusion
* [https://filmora.wondershare.com/video-editor/free-linux-video-editor.html Top 10 Linux Video Editor to Edit Videos on Linux with Ease] (11/12/2016)
==== [https://www.shotcutapp.com/ Shotcut] ====
It is a cross-platform open source software. See [https://www.pcworld.com/article/3240982/software/the-best-free-video-editing-software.html The best free video editing software: Great tools for YouTube stardom and more]
I tested inserting a text in a video. Compared to Youtube video editor
* The text is really a text. No pop-up shape to select
* Not sure how to control the text so it only appears at a certain time interval
Not as intuitive to use.
==== [http://www.pitivi.org/ Pitivi] ====
==== [http://www.blender.org/ Blender] ====
Looks very professional too. Windows/Linux/OSX (binary files are provided). Worth to try.
==== [https://kdenlive.org/ Kdenlive] ====
More complicated than OpenShot. Worth to try. Ubuntu 16.04.1 is needed.
==== [https://www.youtube.com/editor Youtube Video Editor] ====
'''Good'''
* Annotation and Title (Video Manager -> Videos -> Edit -> End screens and annotations -> Annotations -> Add annotations (Speech bubble, Note, Title, Spotlight, Label). However, the annotations do not show up on mobile. See [https://www.quora.com/Which-Video-Editor-will-let-you-add-clickable-links-and-annotations-inside-your-video-Is-YouTube-the-only-tool-that-allows-you-to-do-this this post] for a discussion.
* Add photos
'''Bad'''
* Music cannot have fade in/out
'''Notes'''
* [http://www.makeuseof.com/tag/youtube-annotations-everyone-hates/ YouTube Kills Annotations Because Everyone Hates Them] Mar 17, 2017
* [https://www.thanassis.space/youtube.html Offline playback of Youtube videos and their annotations]
* [https://www.4kdownload.com/blog/2015/04/28/youtube-annotations-and-subtitles-whats-difference/ YouTube Annotations And Subtitles: What’s The Difference?]
* [http://nekotears.blogspot.com/2013/07/download-annotation-and-cc-from-youtube.html Download Annotation or CC from Youtube] & [http://www.nikse.dk/SubtitleEdit Subtitle Edit] (free and open source software)
* [https://www.youtube.com/watch?v=AKMeiYJl5ec Save YouTube Annotations to Srt (Subtitle) File for Offline Viewing of Videos (Part 2 of 2)] & [https://github.com/germanger/youtubeannotations-to-srt Convert youtube XML annotations to SRT]
==== 7 Things You Need to Build a Low-Cost YouTube Studio ====
** Tutorials (video) A [https://www.youtube.com/watch?v=z7E6IyUY9ik Timing Subtitles] and a [https://www.youtube.com/watch?v=jNHbpU_xQBk How to Hardsub / subtitle a video] using [http://www.videohelp.com/software/XviD4PSP XviD4PSP].
** Tutorial (text) [http://www.makeuseof.com/tag/subtitles-text-editor-aegisub/ How To Make Your Own Subtitles With Any Text Editor & Aegisub]
** [http://docs.aegisub.org/manual/Attaching_subtitles_to_video Attaching subtitles to video], [http://forum.aegisub.org/viewtopic.php?f=5&t=1991&view=unread How to save your video in Aegisub after subbing?], [http://www.makeuseof.com/answers/put-hard-subtitles-video-aegisub/ How do I put hard subtitles in a video with Aegisub?], [https://www.reddit.com/r/PleX/comments/4fnus7/hardsub_ass_file_into_any_video_just_using_vlc_221/ Hardsub .ass file in a video with VLC], [https://www.macxdvd.com/mac-dvd-video-converter-how-to/handbrake-add-soft-hard-subtitle-to-movie.htm Hard sub with HandBrake], [https://chibimink.wordpress.com/2012/01/29/making-hardsub-hard-subtitle-with-virtualdub/ Hard sub with VirtualDub].
* [http://amara.org/en/ Amara] (Online editor, used in professional films). [https://www.youtube.com/watch?v=QPiEFc_lV9Q How to Caption YouTube Videos with Amara]
For TV captions, use white color font with black color for borders and transparent background.
==== [http://www.openshot.org/ OpenShot-qt] ====
* It is easier and simpler than Kdenlive. Good for beginners.
* Better if the CPU is good and has more cores
* When merge audio and video, put video at the last track (i.e. audio first). See [http://www.humans-enabled.com/2011/06/how-to-add-music-to-video-with.html here] on how to disable audio from the video track.
* Youtube
** https://www.youtube.com/user/xxlray/videos. [https://www.youtube.com/watch?v=U7BXKYlrf-g Slideshow video], [https://www.youtube.com/watch?v=EIxa11ze9E4 Cut], [https://www.youtube.com/watch?v=4oZwOVoYsUI Picture in picture], and [https://www.youtube.com/watch?v=KsCIuPQTbWo Chroma keying] (allow to change the background).
** [https://www.youtube.com/watch?v=l2V8RiqsrcA OpenShot vs KdenLive]
** Blur an area: [https://www.youtube.com/watch?v=I7Ls7YPqsOE one], [https://www.youtube.com/watch?v=_-IIWBcS6jM two] and [https://www.youtube.com/watch?v=I7Ls7YPqsOE three].
** [https://www.youtube.com/watch?v=mB64rtc5X8c Overlay a text]
* [http://www.makeuseof.com/tag/edit-video-linux-openshot-2-0/ How to Edit a Video in Linux With OpenShot 2.0] (11/19/2016)
* [http://www.pcworld.com/article/230531/free_openshot_video_editor_is_tremendous.html PC World] (2011)
<syntaxhighlight lang='bash'>
sudo apt-get install openshot
</syntaxhighlight>
When I needed to export the video (choose 'youtube' profile, 'youtube-HD' target, 'HD 270, 29 .97 bps', and 'high' quality), I found I need to install [https://answers.launchpad.net/openshot/+faq/1040 libx264 code]. On Ubuntu, I open software center and seach 'libavformat'. I choose 'libavformat-extra-53'.
[https://www.howtoforge.com/tutorial/an-introduction-to-video-editing-in-openshot-2-0/ An introduction to video editing in Openshot 2.0] from howtoforge.
Don't use the version (1.4.3 date 2009) because it crashed too often.
The new version 2.0.7 (date 2016) looks a little different (theme is black. Cool!). Its icon and command ('''openshot-qt''') are different too. The project saved from 1.4.3 cannot be opened in 2.0.7. The tools icons are different: Add track, Snapping tool, Add Marker, Previous Marker, Next Marker, Zoom in/out.
This version of OpenShot + (old) Core2Duo Ubuntu = Dynamic Heatmap Viewer video.
===== Audio library =====
You can download free music from [https://www.youtube.com/audiolibrary/music Youtube Audio Library]. If you use a copy righted music and upload your video to youtube, the video will show Ad eventually.
===== Take a snapshot =====
The keyboard shortcut Ctrl+d does not work.
[https://answers.launchpad.net/openshot/+question/294896 One suggestion] is to use VLC. VLC -> Video -> Take Snapshot. The snapshot will be automatically created in ~/Pictures/ directory (*.png format).
===== Procedure =====
# Put audio and video files in one folder
# Use openshot to create a new video. Also
#* Use the +/- sign for zoom in and zoom out
#* Right click video file and select Volume -> Entire clip -> level 0%
#* Right click audio and select Volume -> End of clip -> fade out (slow)
# openshot -> Save (arrow/download-like button)
# openshot -> Export (red circle button)
#* Modify the file name so it won't overwrite the original (openshot won't check it)
# Check the exported video (play it first by VLC).
#* On one instance the audio is fuzzy until the middle of the video. So I have to change the audio
#* On another instance the video length is longer than I expected because the final annotation slide lasts too long. A solution is to change the setting (Profile=All Formats, Target=MP4 (h.264), Video Profile=HD 1080p 24 or 23.98 fps). If I use 25 fps, the file will be wrong.
# Upload to Youtube. Use Youtube video editor to include annotation.
==== [https://trac.videolan.org/vlmc/ VideoLAN Movie Creator] ====
Using this approach to play audio only will save CPU power. Tested on Odroid.
However, it seems this approach does not work on a playlist, for example, https://www.youtube.com/playlist?list=PL6h94tLpXv3LabUa7B0tCz7K0pI5ZzZEi. See '''mpsyt''' for a solution!
By default, mps-youtube is basically a YouTube audio player (and downloader), but you can enable external video playback (via mpv or MPlayer) from its options. Check out
* An [http://www.webupd8.org/2014/11/youtube-command-line-player-and.html article] on webupd8.
* More keyboard controls can be found on the webpage of [https://github.com/np1/mps mps].
Installation on Linux (works on Odroid with low CPU usage for audio stream from youtube but Odroid gives dirty noise when I played the music).
<syntaxhighlight lang='bash'>
sudo apt-get install python3-pip
sudo pip3 install mps-youtube
sudo pip3 install youtube_dl
# On Ubuntu/Mint. Do not use mplayer. Use mpv instead.
# sudo add-apt-repository ppa:mc3man/mpv-tests
# sudo apt-get update && sudo apt-get install mpv
mpsyt # launch
set player mpv
pl https://www.youtube.com/playlist?list=PL6h94tLpXv3LabUa7B0tCz7K0pI5ZzZEi # a playlist
h # help
Space # pause
p # play
q # quit mpsyt
h search
url https://www.youtube.com/watch?v=hgIfZz8STLk # retrieve specific youtube video by url or id
</syntaxhighlight>
[http://lifehacker.com/mps-youtube-is-a-terminal-based-youtube-player-and-down-1651752380 Installation on Windows]
Some highlight
* Search
* Local playlist
* support YouTube Playlists
* Download
* Music Album Matching
If something is wrong with using 'set' command, just run ''rm -rf ~/.config/mps-youtube/'' and restart everything.
By default, it only streams audio. To watch the video, use '''set show_video true'''.
Below is what I got from the ''set'' command (as you can see the default player is '''mpv''')
<pre>
Key Value
order : relevance
user_order :
max_res : 2160p
player : mpv
playerargs :
encoder : 0 [None]
notifier :
checkupdate : True
show_mplayer_keys : True
fullscreen : False
show_status : True
columns :
ddir : /home/odroid/Downloads/mps
overwrite : True
show_video : False
search_music : True
window_pos :
window_size :
download_command :
audio_format : auto
api_key : AIzaSyCIM4EzNqi1in22f4Z3Ru3iYvLaY8tc3bo
</pre>
The screenshot shows mpsyt contains basic playing keyboard shortcuts.
[[File:Mps-youtube.png|200px]]
[Update 4-7-2017]: to fix an [https://github.com/mps-youtube/mps-youtube/issues/376 error] Signature extraction failed: Traceback (most recent call last):
==== Download youtube video using command line ====
<syntaxhighlight lang='bash'>
sudo apt-get install youtube-dl
</syntaxhighlight>
* Fix the error 'WARNING: Your copy of avconv is outdated and unable to properly mux separate video and audio files, youtube-dl will download single file media. Update avconv to version 10-0 or newer to fix this.':
(not tested) [https://forums.linuxmint.com/viewtopic.php?t=197316 Compile your own]. The source code is available at https://libav.org/.
* For videos with 1080p resolution, youtube-dl will download 720p by default. To download 1080p, see [http://askubuntu.com/questions/486297/how-to-select-video-quality-from-youtube-dl this post]
<syntaxhighlight lang='bash'>
# show the available resolutions
youtube-dl -F XXX
# download the desired resolution
youtube-dl -f 137+141 XXXX
# Note the two numbers 137, 141 is case dependent. They could be 137 & 140.
youtube-dl -f 22 XXX
# if the format 22 is mp4. If I do not specify the format, the mkv file will be downloaded
# But playing mkv file tends to be stuttering with VLC.
</syntaxhighlight>
This assumes Ubuntu has installed '''avconv'''; otherwise video(mp4) and audio(m4a) files will be downloaded separately. To install avconv, run
<syntaxhighlight lang='bash'>
sudo apt-get install libav-tools
</syntaxhighlight>
* To convert the video to mp3 format, use soundconverter
<syntaxhighlight lang='bash'>
sudo apt-get install soundconverter
</syntaxhighlight>
* [http://www.webupd8.org/2014/11/youtube-command-line-player-and.html Play/Download youtube video] using [https://github.com/np1/mps-youtube mps-youtube]
* [http://askubuntu.com/questions/127290/setting-program-logo-in-unity-launcher-missing-icons-folder Desktop icons] are saved under '''/usr/share/icons/hicolor/16x16/apps/'''rstudio.png, where other than 16x16 there are also icons from 24x24, 32x32, 48x48 sizes (directories). Another place is '''~/.local/share/icons/hicolor/16x16/''' .
* [https://help.ubuntu.com/community/UnityLaunchersAndDesktopFiles Unity launcher] is created at '''/usr/share/applications/'''rstudio.desktop or '''~/.local/share/applications/'''XXXX.desktop.
* Desktop shortcut can be created by copy XXXX.desktop to ~/Desktop directory.
The icon size in the launcher can be adjusted by opening System Settings -> Apperance -> Launcher Icon Size (at the bottom).
[http://www.willus.com/k2pdfopt/help/ubuntu.shtml k2pdfopt] has an example how to do it for creating a desktop shortcut and be in the right click menu.
==== How to Convert the Unity Launcher into a Dock-Style Launcher ====
See an article from [http://www.howtogeek.com/202708/how-to-convert-the-unity-launcher-into-a-dock-style-launcher/ Howtogeek.com].
==== How to Get Unity’s Global App Menu in Linux Mint Cinnamon ====
[http://www.omgubuntu.co.uk/2016/10/your-favourite-alfred-app-launcher-linux Synapse or Albert — What’s Your Favourite App Launcher for Linux?]
==== [http://do.cooperteam.net/ Gnome Do] ====
Release June 2009
=== What date was the system installed ===
<pre>
ls -l /var/log/installer
</pre>
=== What is the last log in time for users ===
<pre>
lastlog # all users
last # current user
</pre>
=== What is the reboot time ===
<pre>
last reboot
</pre>
=== Crop an image ===
<pre>
sudo apt-get install gthumb
</pre>
Open an image. Click color palette icon on top-right corner (or use keyboard shortcut 'e'). Click 'crop'. There will be a rectangle on image where you can resize the rectangle.
Pinta and mtPaint can also take a screenshot of the desktop and crop the image.
Search Ubuntu Software Center to install [http://www.keepassx.org KeePassX] (more reviews) or [http://keepass.info KeePass2]. There is no need to use ppa.launchpad.net to install from. Note that the kdbx file used in KeePass2 cannot be opened in KeePassX. To use the command line to install KeePass2,
<pre>
sudo apt-get install keepass2
</pre>
Note:
* Foe latest version, use [https://launchpad.net/~jtaylor/+archive/ubuntu/keepass Julian Taylor’s PPA].
<pre>
sudo add-apt-repository ppa:jtaylor/keepass
sudo apt-get update
sudo apt-get install keepass2
</pre>
* Android: KeePassDroid
* Chrome: [https://chrome.google.com/webstore/detail/chromeipass/ompiailgknfdndiefoaoiligalphfdae?hl=en ChromeIPass] with KeePassHttp. See the [https://github.com/pfn/passifox/ source code].
Security:
* [http://lifehacker.com/keepass-vulnerability-could-let-attackers-steal-your-pa-1781486764 KeePass uses unencrypted HTTP requests when retrieving updates]
Or use the official PPA method. See [http://libre-software.net/how-to-install-firefox-on-ubuntu-linux-mint/ How to install Firefox 57 “Quantum” on Linux Mint, Ubuntu, Debian, CentOS, Fedora…], [https://itsfoss.com/firefox-quantum-ubuntu/ How to Install Firefox Quantum in Ubuntu and other Linux Right Now]
* [http://raspberrypi.stackexchange.com/questions/934/how-do-i-install-bluetooth-for-my-mouse-and-keyboard How to install bluetooth for my mouse and keyboard] for my Raspberry Pi.
<syntaxhighlight lang='bash'>
* https://zach-adams.com/2014/07/bluetooth-audio-sink-stream-setup-failed/ Linux Mint
sudo apt-get install bluetooth
sudo apt-get install bluetooth bluez-utils blueman
</syntaxhighlight>
Then run '''lsusb | grep Bluetooth''' command which will shows the name of your bluetooth device.
* https://wiki.archlinux.org/index.php/Bluetooth
<syntaxhighlight lang='bash'>
sudo apt-get install bluez
</syntaxhighlight>
On my bluetooth adapter, the '''lsusb''' shows,
<syntaxhighlight lang='bash'>
$ lsusb
Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)
...
$ sudo hcitool dev
Devices:
hci0 00:1A:7D:DA:71:09
</syntaxhighlight>
I can use Ubuntu's bluetooth setting dialog to connect my bluetooth keyboard without any problem (It will ask me to enter some code on pairing).
Note that the MAC address of my bluetooth keyboard shown above is the from the controller/adapter. It is NOT the same as my bluetooth keyboard one (90:7F:61:8F:D0:38) as shown from the bluetooth setting (GUI) dialog.
==== Bluetooth Audio Receiver ====
* Bluetooth 4.0 Receiver A2DP Wireless Adapter from [http://www.amazon.com/Mpow-Streambot-Bluetooth-Receiver-Streaming/dp/B00MJMV0GU/ref=sr_1_4?s=electronics&ie=UTF8&qid=1463170351&sr=1-4&keywords=bluetooth+receiver Mpow Streambot] or [http://www.amazon.com/Bluetooth-Receiver-DBPOWER-Wireless-Devices/dp/B012FRBINI?ie=UTF8&psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s01 DBPOWER]
Remember to change the ''Mode'' from ''Telephony Duplex (HSP/HFP)'' to '''High Fidelity Playback (A2DP)''' from ''Sound Settings'' dialog (launched from Taskbar). See also [[Windows#Output_audio_to_bluetooth|Windows OS]].
[[File:BluetoothAudioLinux.png|200px]]
=== Default applications and mime-types ===
See '''/etc/gnome/defaults.list'''.
[https://www.howtoforge.com/tutorial/how-to-easily-open-a-pdf-file-from-command-line-in-ubuntu/ How to easily open a PDF (or any) file from command line in Ubuntu]
<syntaxhighlight lang='bash'>
sudo apt-get install libgnome2-bin
gnome-open [name-of-PDF-file]
gnome-open [dir-name-or-path] # To open a directory in Nautilus file manager
gnome-open https://www.howtoforge.com/ # To open a website in your system's default web browser
</syntaxhighlight>
=== Change default audio player ===
Right-click an MP3 file, and from the menu select “Properties”. In the window that appears, go to the “Open With” tab and select VLC or whatever. Click the “Set As Default” button to set VLC as the default player.
You might also want to set the default player in the sound menu.
=== Unity display timeout ===
Go to [http://www.techotopia.com/index.php/Image:Ubuntu_11_unity_battery_power_management_settings.jpg Dash -> power setting -> Brightness Settings]
Click Dash > xscreensaver or run the command line '''xscreensaver-demo'''. Mode = Only One Screen Saver & pick GLText. Click Setting & choose 'Display date and time' ,'Don't rotate'.
==== gluqlo ====
[Note that gluqlo uses too much cpu]
I like [https://github.com/alexanderk23/gluqlo gluqlo (flip clock) screensaver]. However, it is not easy to make it to replace the gnome-screensaver.
<pre>
sudo apt-add-repository ppa:alexanderk23/ppa
sudo apt-get update
sudo apt-get install gluqlo
</pre>
* Follow the instruction to install it on Ubuntu machine.
* Follow the instruction there to remove gnome-screensaver and install xscreensaver.
* Configure xscreensaver to use only 1 screensaver. Edit ~/.xscreensaver file and add a line like
<pre>
gluqlo -root \n\
</pre>
* Still follow the instruction to allow xscreensaver to start when the machine starts up. Don't try to edit ~/.xinitrc file as other sites suggested; start Dash and type 'startup' and follow the screen dialog to add xscreensaver -nosplash.
* Still follow the instruction to add lock screen keyboard shortcut.
Note that the above steps work for Ubuntu 12.04 & 13.10 but not 14.04 (Ubuntu 14.04 changed to use LockScreen instead LightDM program to lock the screen). A solution on Ubuntu 14.04 is to [http://askubuntu.com/questions/465194/how-to-disable-screen-lock-in-ubuntu-14-04-unity disable screen lock].
# Still follow the above instruction to remove gnome-screensaver and install xscreensaver. Set gluqlo as the only one screensaver.
# Go to Brightness & Lock panel from the Unity Launcher. And set Turn screen off when inactive: to ''Never''.
# Install "Unity Tweak Tool" with sudo apt-get install unity-tweak-tool. Run it from the Launcher and select System > Security > Enhance system security by disabling Desktop lock.
# To enable locking desktop (required a password to unlock the desktop), check 'Lock Screen After' option. I pick 1 minute.
For some reason, gluqlo suddenly uses all my cpu (6 cores) resource (Ubuntu 14.04). The computer thus makes some noise. I have to use 'kill' command to kill them.
On Ubuntu 12.04, there is no way to directly install the pepperflashplugin-nonfree plugin. So we have to use ppa from other people. See [http://linuxg.net/how-to-install-chromium-and-enable-pepper-flash-plugin-on-all-the-supported-ubuntu-systems-and-their-derivatives/ this post]
<pre>
sudo apt-get update
sudo apt-get install chromium-browser
sudo add-apt-repository ppa:skunk/pepper-flash
sudo apt-get update
sudo apt-get install pepflashplugin-installer
sudo update-pepperflashplugin-nonfree --install
</pre>
The last step gives me an error: sudo: update-pepperflashplugin-nonfree: command not found
=== Gedit ===
* To split a screen, Do "Documents -> New Tab Group. No extra plugin is needed to download. I am using version 3.10.4 from Ubuntu 14.04.
* [https://github.com/Quixotix/gedit-restore-tabs Restore tabs] plugin. It works on my gedit 3.4 (ubuntu 12.04). Follow the instruction there exactly.
* [https://github.com/Quixotix/gedit-source-code-browser Source code browser] plugin. This makes gedit a good IDE for developing C++/Java code since the left panel can show symbols. Click F9 to show the side panel.
gedit > Edit > preferences > font and colors > color scheme.
==== Plugins ====
* Gedit has no built-in options to show special characters except through [https://wiki.gnome.org/GeditPlugins gedit-plugins] (sudo apt-get install gedit-plugins). See [http://stackoverflow.com/questions/17162851/option-to-display-control-characters-in-gedit Draw Spaces].
* [https://cs50.stackexchange.com/questions/2026/how-do-i-open-a-terminal-window-in-gedit How do I open a Terminal window in gEdit?]
[https://en.wikipedia.org/wiki/Newline A DOS/Windows text file can be converted to Unix format by simply removing all ASCII CR characters with]
<pre>
$ tr -d '\r' < inputfile > outputfile # inputfile and outputfile cannot be the same
</pre>
or, if the text has only CR newlines, by converting all CR newlines to LF with
<pre>
$ tr '\r' '\n' < inputfile > outputfile
</pre>
=== [http://www.geany.org/ Geany] ===
'''Geany''' can be used to run a bash script file line by line. See [[Linux_Programming#Debugging_Scripts|Debugging_Scripts]].
==== Display special characters ====
Geany has a way to show special characters (Tabs/LF/CR). Edit > Preferences > Display > Tick, Show whitespace (tabs) & Show Line endings(CR/LF).
For DOS text file, the line ending is CR+LF.
For Unix text file, the line ending is LF.
==== Font size ====
Users can use either one of the following methods
# Edit -> Preferences -> Interface -> Fonts to adjust the font size.
# Keyboard bindings: Ctrl + Shift + '+' to increase the font size or Ctrl + '-' to decrease the font size. This does not affect the font size in Preferences.
==== Printing ====
The font size in Preferences affects the printing. The font size changed by using the keyboard bindings does not affect printing.
==== Remove vertical line ====
Edit -> Preferences -> Editor -> Display -> Uncheck Long line marker.
=== SQL ===
==== MySQL Workbench ====
http://www.mysql.com/products/workbench/
==== sqliteman ====
https://sourceforge.net/projects/sqliteman/
=== User Interface Designer ===
[https://glade.gnome.org/ Glade] - RAD tool to enable quick & easy development of user interfaces for the GTK+ toolkit and the GNOME desktop environment
[https://wiki.gnome.org/Apps/Devhelp Devhelp] - API documentation browser for GTK+ and GNOME
=== HTML editor ===
* Atom
* Bluefish
* [http://brackets.io/ Brackets] features
** Inline Editors
** Live Preview
** Preprocessor Support
* [http://www.monodevelop.com/ MonoDevelop]
* [http://kompozer.net/ Kompozer] and the installation instruction for [https://help.ubuntu.com/community/InstallKompozer Ubuntu].
* [http://bluegriffon.org/ BlueGriffon]: a new WYSIWYG content editor. The interesting thing is the software [http://www.bluegriffon-epubedition.com/BGEE.html BlueGriffon EPUB Edition]: a cross-platform Wysiwyg editor able to natively create and edit EPUB2 and EPUB3 ebooks!
=== npm and Javascript ===
See [[Javascript#npm_-_package_manager|npm package manager]] in Javascript.
=== chm reader ===
<pre>
sudo apt-get install xchm
</pre>
=== SCR3310 smart card ===
* The usb device should be recognized by Ubuntu/Mint. Thus, the smart card can be used by Windows virtual machine (tested on Windows 10 VM).
<syntaxhighlight lang='bash'>
brb@T3600 ~ $ lsusb
Bus 002 Device 003: ID 413c:2107 Dell Computer Corp.
Bus 002 Device 033: ID 09c3:0013 ActivCard, Inc.
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 3938:1031
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0b95:7720 ASIX Electronics Corp. AX88772
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
* http://www.binarytides.com/setup-chroot-ubuntu-debootstrap/ (Works after a little change)
Note that we have to change the conf file a little bit. The 'location' word needs to be changed to 'directory'. Also at the last step when we are ready to test a 32-bit GUI app, we need to issue DISPLAY in a separate line; such as
<pre>
export DISPLAY=:0.0
su brb # brb is my root user in the host system that can invoke the schroot program
The article also mentioned the home directories (Documents, Downloads, ...) of the users within the chroot are shared with the host. How to access them from the host?
* [https://blog.night-shade.org.uk/2013/12/building-a-pure-debian-armhf-rootfs/ Building a pure Debian armhf rootfs]
* [https://www.stgraber.org/2012/02/03/ever-wanted-an-armel-or-armhf-container-on-an-x86-machine-its-now-possible-with-lxc-in-ubuntu-precise/ Create an armhf container on your x86 machine?]
==== How Snappy packages are different from Deb ====
An article from [http://www.pcworld.com/article/2942267/why-ubuntu-plans-to-replace-traditional-linux-packages-with-something-better.html PCWorld]
* Applications are no longer installed system-wide. The base Ubuntu operating system is kept securely isolated from applications you install later. Both the base system and Snappy packages are kept as read-only images.
* Snappy packages can include all the libraries and files they need, so they don’t depend on other packages.
* An update can never fail, as a package installation could potentially fail and become incomplete with typical Linux packages.
* Snappy also supports “delta” updates, which means only the changed bits of the package need to be downloaded and installed.
* Snappy-based Ubuntu systems might be standard.
==== Snap commands ====
[http://www.omgubuntu.co.uk/2016/12/simple-guide-snapd-commands 6 Essential Ubuntu Snap Commands You Should Know]
==== Docker ====
I haven't found any tutorial yet!
==== Ova image ====
Sorry, I don't get the command line back. Booting stuck in the middle.
==== Beaglebone ====
http://beagleboard.org/snappy or http://www.ubuntu.com/things#try-beaglebone
(Mar 7, 2016). We may experience an error "Failed to fetch http://dl.google.com/linux/chrome/deb/dists/stable/Release" when we run sudo apt-get update. It is because the 32-bit chrome has been discontinued. The solution is to modify the file </etc/apt/sources.list.d/google-chrome.list>. See [https://www.reddit.com/r/chrome/comments/48oje6/linux_how_to_fix_failed_to_fetch/ reddit].
<syntaxhighlight lang='bash'>
$ sudo sed -i -e 's/deb http/deb [arch=amd64] http/' "/etc/apt/sources.list.d/google-chrome.list"
# You may comment out this entry, but any other modifications may be lost.
deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
</syntaxhighlight>
Another suggestion to modify </opt/google/chrome/cron/google-chrome> (though the file exists) does not work .
[http://www.makeuseof.com/tag/install-chrome-linux-easily-migrate-browsing-windows/ How to Install Chrome on Linux and Easily Migrate Your Browsing From Windows]
* [https://www.cyberciti.biz/faq/how-to-disable-ssh-motd-welcome-message-on-ubuntu-linux/ How to disable ssh motd welcome message on Ubuntu Linux]
=== .Trash-1000 folder ===
See [http://superuser.com/questions/169980/what-is-trash-and-trash-1000 this post]. Ubuntu will create such folders when a file is deleted from a USB drive. Presumably this would allow a file to be restored if you accidentally deleted it.
Try to empty the paperbin or delete the folder with the terminal-command as root: sudo rm -rf /path/to/folder/.Trash-1000
[https://plus.google.com/+CassidyJames/posts/EzxrE4UyYVF Birdie vs Corebird]
=== Elasticsearch & Kibana ===
[https://www.howtoforge.com/tutorial/elasticsearch-and-kibana-installation-and-basic-usage-on-ubuntu-1604/ Elasticsearch and Kibana : installation and basic usage on Ubuntu 16.04]
=== TexLive ===
* [https://askubuntu.com/questions/485514/how-to-properly-install-and-use-texlive-with-package-manager-in-14-04 How to properly install and use texlive with package manager in 14.04]
* [http://www.pcworld.com/article/2147063/linux-to-the-rescue-how-ubuntu-can-help-a-computer-in-distress.html Reset your Windows password]
== RHEL/CentOS ==
=== Download urls ===
* https://www.centos.org/download/
* [http://mirror.umd.edu/centos/7/isos/ U. of Maryland] mirror. Everything > DVD > Live > minimal.
=== Installation screenshots ===
* [https://www.tecmint.com/red-hat-enterprise-linux-7-3-installation-guide/ Installation of Red Hat Enterprise Linux (RHEL) 7.3 Guide]. UEFI + GPT, Legacy BIOS + MBR.
* [https://www.howtoforge.com/tutorial/centos-7-minimal-server/ How to Install a CentOS 7.3 Minimal Server] For a static IP address, I need to enter IP, subnet, gateway and DNS servers.
* [http://dev-random.net/how-to-install-a-centos-6-5-64bit-server/ How to install a CentOS 6 64Bit Server]
=== 30 Things to Do After Minimal RHEL/CentOS 7 Installation ===
=== Add an existing user to have sudo privilege ===
<syntaxhighlight lang='bash'>
sudo adduser USERNAME sudo
</syntaxhighlight>
See [https://help.ubuntu.com/community/RootSudo help.ubuntu.com].
=== What is my IP address ===
<syntaxhighlight lang='bash'>
ifconfig eth0
</syntaxhighlight>
=== What services get started at boot time ===
<syntaxhighlight lang='bash'>
chkconfig --list
</syntaxhighlight>
=== Is xxx service running ===
<syntaxhighlight lang='bash'>
xxx status
</syntaxhighlight>
=== What services are currently running ===
<syntaxhighlight lang='bash'>
ps -e
</syntaxhighlight>
and
<syntaxhighlight lang='bash'>
lsof -i
</syntaxhighlight>
will show you services that are listening to TCP or UDP endpoints.
=== Choosing a web hosting service for your website ===
* [http://www.makeuseof.com/tag/top-7-easy-and-free-web-hosting-services/ Top 7 Easy and Free Web Hosting Services] (7/18/2017)
* [http://www.lifehack.org/387041/10-things-you-should-know-when-choosing-web-hosting-service-for-your-website 10 Things You Should Know When Choosing A Web Hosting Service For Your Website]
'''The configuration defaults for GNOME power manager have not installed correctly. Cannot login'''
This error will results in a log-in problem except root account. The symptom is 50GB in root (/) is used up.
The problem was caused by a bug in yum where '''/var/cache/yum/x86_64/6Workstation''' takes about 42GB space. The 'yum' does not remove old generated .sqlite files.
See https://bugzilla.redhat.com/show_bug.cgi?id=632391
I use 'du -k' command to find out which directory took space. I use 'rm' command to delete the contents.
Even I delete the content, the directory still grows up daily.
=== Upgrade Python from 2.6.x to 2.7.x ===
[http://bicofino.io/blog/2014/01/16/installing-python-2-dot-7-6-on-centos-6-dot-5/ This instruction] tells how to install Python 2.7 from source.
Have not found a solution yet. We need to install it from source. However, the source depends on
* Python 2.7 (see above for the instruction)
* GTK+ 3.6
* GLib 2.34
* PyGObject 3.8
* GtkSourceView 3.6
(Update) A binary version of meld is already available in the git. See [http://linuxg.net/how-to-install-meld-3-11-2-on-ubuntu-linux-mint-debian-fedora-opensuse-mageia-and-their-derivative-systems/ this post].
[http://pixlr.com/editor/ Pixlr Editor] (vs [http://askubuntu.com/questions/164473/simple-image-editor Pinta or Shotwell] in Ubuntu)
[http://www.howtogeek.com/255596/the-easiest-ways-to-create-animated-gifs-on-any-platform/ Create gifs on any platform]. An online tool is called [http://giphy.com/create/slideshow Giphy]. The source file can be a video file (local < 100MB or youtube). If the source file is a series of png file, click slideshow option. The resulting file can be embedded in html files or downloaded to your local machines.
[http://www.makeuseof.com/tag/one-click-websites-photos-better/ 5 One-Click Websites to Make Your Photos Look Better]
=== Video editor ===
[https://www.wevideo.com/ WeVideo] (vs [[#Edit_a_video_using_OpenShot|OpenShot]] in Ubuntu)
Linux allows virtual consoles (aka virtual terminals) to be opened while an X Window System is executing.
Use Ctrl + Alt + FX to open a virtual console-- there are six virtual text-based consoles (F1 to F6). Use Ctrl + Alt + F7 (or Ctrl + Alt + F8 on Linux Mint) to return to the X Window System.
On X Window System, we can use Ctrl + Alt + -> or Ctrl + Alt + <- to switch workspaces.
When applying permissions to directories on Linux, the permission bits have different meanings than on regular files.
The write bit allows the affected user to create, rename, or delete files within the directory, and modify the directory's attributes
The read bit allows the affected user to list the files within the directory
The execute bit allows the affected user to enter the directory, and access files and directories inside
When we create a new directory, the attribute is 775. Some pre-created directories (Desktop, Documents, Music, Pictures, Public) have an attribute 755.
su - username -c 'command'
sudo su - # switch to root account
whoami
sudo su - -c "R -q -e \"install.packages('mypackage', repos='http://cran.rstudio.com/')\""
# OR
sudo su -c "COMMAND_REQUIRE_ROOT_ACCESS"
# OR
sudo "COMMAND_REQUIRE_ROOT_ACCESS"
man su
What’s the Difference Between Bash, Zsh, and Other Linux Shells
brb@T3600 ~ $ ls -l ~/GSE48215/*.fastq
-rw-r--r-- 1 brb brb 16226673016 Jun 14 14:13 /home/brb/GSE48215/SRR925751_1.fastq
-rw-r--r-- 1 brb brb 16226673016 Jun 14 14:13 /home/brb/GSE48215/SRR925751_2.fastq
brb@T3600 ~ $ ls -l '~/GSE48215/*.fastq'
ls: cannot access ~/GSE48215/*.fastq: No such file or directory
brb@T3600 ~ $ ls -l "~/GSE48215/*.fastq"
ls: cannot access ~/GSE48215/*.fastq: No such file or directory
sudo apt-get install pv
pv file1 > file2 # don't forget the ">" operator and the destination is a file, not a directory
After that, instead of clicking the reject icon from the file manager to eject it, it is better to use a command line to do that because there is no expect time for users to know when it will take for finish writing the data to a USB drive.
sudo apt-get install udisks
sudo udisks --unmount /dev/sdb1 # /dev/sdb1 is the partition
sudo udisks --detach /dev/sdb # /dev/sdb is the device
My testing shows this procedure works (tested by running md5sum after eject/plug-in) when I need to copy a 9GB file.
Reliable way: Split the large file and copy smaller chunks
# Use 'sudo iotop -o' to monitor the I/O
split -b 4G inputFile # create xaa, xab, ... files
cat x* > outputFile # merge them. md5sum check succeeds
type x* > outputFile # Windows OS. http://stackoverflow.com/questions/60244/is-there-replacement-for-cat-on-windows
It is interesting copying smaller files (eg 4GB) to USB drives is quite stable (just use the cp command). Even for a not-too large file (6.7GB), pv step looks OK but the unmount/detach step failed.
For a 6.7GB file, it will split it into a 4GB and 2.7GB files. Merge takes longer time if it is done on the USB drive. That is, it is best to do merge in the final destination (internal disk/storage).
split in the internal hdd: 1min 38sec
merge in the internal hdd: 37sec
merge in the USB 3.0 drive: 2min 17sec
Remember: Use a reliable USB drives.
The operation could not be completed because the volume is dirty
On a USB 2.0 drive, I can copy files to there but the drive cannot be rejected (Ubuntu has a pop-up showing it is still writing data to it).
When I forcibly rejects the drive and plug it in a Windows PC, Windows shows the message The operation could not be completed because the volume is dirty. This gives a way to run chkdsk (check and repair a file system).
Open a Windows File Manager
Right click the USB drive
Properties
Tools -> Check now... Start
Done. Now I can use the drive again.
The Linux equivalent to chkdsk is fsck. fsck is a front end that calls the appropriate tool (fsck.ex2, fsck.ex3, e2fsck, ...) for the filesystem in question.
umount /dev/sdb1 # thumb drive
sudo fsck /dev/sdb1
sudo fsck -a /dev/sdb1 # auto repair
For the root disk, you have to use a live CD. Otherwise, you will see a message like
$ fsck /dev/sdb1
fsck from util-linux 2.20.1
e2fsck 1.42.9 (4-Feb-2014)
/dev/sdb1 is mounted.
WARNING!!! The filesystem is mounted. If you continue you ***WILL***
cause ***SEVERE*** filesystem damage.
Do you really want to continue<n>? no
Most likely your ls is aliased to ls --color=auto. If you do ls --color (which is morally equivalent to ls --color=always), that will force it to turn on colors.
I install kdiff3 (<2 MB to download) and the 'File' -> 'Reload' (F5) function there works though it shows an extra space on the place I modified.
Beyond Compare (commercial $60, trial version can be downloaded)
diffuse. When I modified a file, diffuse can detect a change and ask me to reload the file. I am using the apt-get to install the software and the version number is 0.4.7 (2014). To copy lines from left panel to right panel, use 'Ctrl + Shift + >' or the Copy Selection Right icon. One drawback is it cannot save the history from the GUI though we can use the command line to include the file names in the arguments.
Alternatively we can use WinMerge on Linux. To do that, install Wine on Ubuntu. Download Winmerge (I am using 2.14.0). Then on a terminal, run the following command. At the end, WinMerge will be launched. WinMerge can also be launched from Mint Menu -> Wine -> WinMerge. One problem is I cannot increase the font size (though acceptable) from View -> Select Font.
wine WinMerge-2.14.0-Setup.exe
diff
Run diff with large files
Meld freezes When I tested it with two large files (800k & 936k lines coming from human gtf files). Actually the whole linux system became unresponsive.
Actually Meld is sluggish when it is used in small files in Odroid XU4 running Ubuntu 16.04 MATE. I have used Meld 3.14.2 and the latest 3.16.2.
Directory
diff -qr dir1 dir2
where -q means to report only when files differ and -r is to recursively compare any subdirectories found.
where -y option means to show the output in two columns.
Interpretation of the diff output:
The first line of the diff output will contain:
line numbers corresponding to the first file,
a letter (a for add, c for change, or d for delete), and
line numbers corresponding to the second file.
In our output above, 2,4c2,4 means: "Lines 2 through 4 in the first file need to be changed in order to match lines 2 through 4 in the second file." It then tells us what those lines are in each file:
Lines preceded by a < are lines from the first file;
lines preceded by > are lines from the second file.
The three dashes ("---") merely separate the lines of file 1 and file 2.
2,4c2,4
< I need to run the laundry.
< I need to wash the dog.
< I need to get the car detailed.
---
> I need to do the laundry.
> I need to wash the car.
> I need to get the dog detailed.
colordiff -ur path1 path2
# If you change -ur to -urN then that will also show the contents of files that are only present in one of the paths.
The meaning of colors can be found in /etc/colordiffrc (man colordiff)
which - Display the full path of shell commands. See examples from cyberciti.biz.
$ which ls
/bin/ls
whereis - locate the binary, source, and manual page files for a command. See examples from cyberciti.biz.
$ whereis ls
ls: /bin/ls /usr/share/man/man1p/ls.1p.gz /usr/share/man/man1/ls.1.gz
type -a
$ type -a ls
ls is aliased to `ls --color=tty'
ls is /bin/ls
Use locate command mindfully. It is used to find the location of files and directories. Note that locate does not search the files on disk rather it searches for file paths in a database.
For example, the following command will search .png files over the system (not only the personal directory).
locate "*.png"
find: Find a file based on file name
$ find . -iname '*.txt' # -iname or -name is necessary
It also works for searching files on subdirectories.
If the search directory is not the current directory, we need to add a forward slash to the directory name.
$ find ~/Desktop -iname '*.txt' # Not working
$ find ~/Desktop/ -iname '*.txt' # Working
The following example shows we can list multiple search criteria. The “‑r” option in tar appends files to an archive. xargs is a handy utility that converts a stream of input (in this case the output of find) into command line arguments for the supplied command (in this case tar, used to create a backup archive).
find / -type f -mtime -7 | xargs tar -rf weekly_incremental.tar
gzip weekly_incremental.tar
Using grep to search two different words (egrep -w 'word1|word2' /path/to/file)
Count line for matched words ("-c" option)
Grep invert match ("-v" option)
How to list only the names of matching files ("-l" option)
GUI
A GUI version of a tool to search files is searchmonkey (open source, Linux, Windows). On Ubuntu, we install it by
sudo apt-get install searchmonkey
It is also useful to change the settings so we can click a filename and open it in the desired text editor. To do that, go to Settings -> Preferences -> System Call -> Text Editor. I enter 'geany' since I want to use geany to open my C programs. Note. the v2.0 source code needs to be built using i386 gcc library and Qt 4.8.x. Still, I cannot get rid of some errors coming from the source code.
Summary of find and grep commands
Command
Examples
find
find [DIRECTORY] -iname '*.txt'
find [DIRECTORY] -maxdepth 2 -iname *.php
find -name '*.php' -o -name '*.txt' # OR operator
grep
grep -r -i "check_samtools" DIRECTORY/
dpkg -l libgtk* | grep '^i'
Count number of columns: awk
The following command shows the number of columns for the first few rows of a text file.
head MYFILE | awk '{ print NF}'
head MYFILE | awk -F '\t' '{ print NF}'
Count number of rows in a file: wc
wc -l MYFILE
The source code of wc (or any Linux command) can be found by using this method
brb@brb-T3500:~/Downloads$ which wc
/usr/bin/wc
brb@brb-T3500:~/Downloads$ dpkg -S /usr/bin/wc
coreutils: /usr/bin/wc
brb@brb-T3500:~/Downloads$ sudo apt-get source coreutils
[sudo] password for brb:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Need to get 12.3 MB of source archives.
Get:1 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main coreutils 8.21-1ubuntu5.1 (dsc) [1,635 B]
Get:2 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main coreutils 8.21-1ubuntu5.1 (tar) [12.3 MB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main coreutils 8.21-1ubuntu5.1 (diff) [31.6 kB]
Fetched 12.3 MB in 22s (559 kB/s)
gpgv: Signature made Tue 13 Jan 2015 10:33:04 PM EST using RSA key ID 9D8D2E97
gpgv: Cannot check signature: public key not found
dpkg-source: warning: failed to verify signature on ./coreutils_8.21-1ubuntu5.1.dsc
dpkg-source: info: extracting coreutils in coreutils-8.21
dpkg-source: info: unpacking coreutils_8.21.orig.tar.gz
dpkg-source: info: applying coreutils_8.21-1ubuntu5.1.diff.gz
As we can see from the coreutils-8.21/src directory, there are over 100 C programs including <cat.c>, <chmod.c>, <cp.c>, ...<wc.c>.
Print certain rows/lines of a text file
The following example will print out lines 10 to 60 of FILENAME.
sed -n '10,60p' FILENAME
Or to print out line 60,
sed -n '60p' FILENAME
It seems this method is not as fast as I expected. For example, the tail command will immediately print out the result without waiting!
For example, the following code will change the prompt to a light blue color.
PS1="\[\033[1;34m\][\u@\h:\w]$\[\033[0m\] "
In Odroid running Ubuntu mate, we can modify ~/.bashrc and un-comment the following line
force_color_prompt=yes
Add a timestamp to your Bash prompt
See man 3 strftime for the date, time format. In Linux Mint, the date applets uses the %A %B %e, %H:%M which gives a format like 'Friday July 15, 10:23'.
A second method which is useful for ssh and scp commands
$ ssh -L 9999:host2:22 user1@host1 # leave the terminal
# Open a new terminal
$ scp -P 9999 fileName user2@localhost:/path/to/dest/fileName # transfer from local to remote. Note: Upper P.
$ scp -P 9999 user2@localhost:/path/to/source/fileName fileName # transfer from remote to local. Note: Upper P.
# If we only want to use ssh
$ ssh -p 9999 user2@localhost # Note: lower p.
Verify that local-host and remote-host is running openSSH (ssh -V)
Generate key-pair on the local-host using ssh-keygen (Enter a passphrase here, do not leave it empty. A passphrase should be at least several words long, something you can easily remember. It's a bad idea to use a single word as a passphrase.)
Install public key on the remote-host
Give appropriate permission to the .ssh directory on the remote-host (chmod 755 ~/.ssh; chmod 644 ~/.ssh/authorized_keys)
Login from the local-host to remote-host using the SSH key authentication to verify whether it works properly
Start the SSH Agent on local-host to perform ssh and scp without having to enter the passphrase several times (ssh-agent $SHELL)
Load the private key to the SSH agent on the local-host (ssh-add, need to enter the passphrase 1 time only)
Perform SSH or SCP to remote-home from local-host without entering the password. It works for all remote machines containing the key from local-local.
Another option is to use ssh -i IDENTITY_FILE. See superuser.com.
/etc/hosts.deny will records the IPs that are blocked. If the normal ssh connection failed (e.g. get a message ssh_exchange_identification: read: Connection reset by peer), check /etc/hosts.deny file to see if your IP is in it. One method is to add your IP to /var/lib/denyhosts/allowed-host file so your IP won't be blocked.
A few minutes of run of denyhosts accumulates hundreds of IP in /etc/hosts.deny file. But I remove the service since I did not spend enough time to understand it.
Procedures: follow the README.txt file.
ssh key
SSH key is useful if you want a password-less login to a remote system. Some useful resources:
where the comment 'bitbucket' will appear at the end of <~/.ssh/personalid> file.
Copy the public key to a remote host ([email protected]) over ssh. The current user (eg brb) and the remote user (eg git)have not any relationship (they most likely have different user names):
ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected] # this will 'append' the key to the remote-host’s .ssh/authorized_key.
<Method 2> Another way is to use ssh-add & ssh-agent to manager your keys. ssh-agent keeps your key in its memory and pulls it up whenever it is asked for it.
$ ssh-keygen -f ~/.ssh/personalid -C "bitbucket"
$ eval $(ssh-agent -s) # Ensure ssh-agent is enabled:
$ ssh-add ~/.ssh/personalid # ssh-add program will ask you for your passphrase
$ ssh-add -l
If we do not change the permission correctly in <id_rsa>, we will get a warning: Unprotected private key file. Permissions 0664 for '/home/USERNAME/.ssh/id_rsa' are too open.
This port forwarding involves three computers (local, remote and hostname) as you can see from the SSH syntax.
For example, we like to access home's router (192.168.1.1) information from an outsider computer. Suppose the host 'hostname' is one computer in the home network and it can be accessed from outside world.
The -L option specifies local port forwarding. In this case, port 8080 on the local machine was forwarded to port 80 on the remote machine. For the duration of the SSH session, pointing your browser at http://localhost:8080/ would send you to http://192.168.1.1/ as if you are in the same local network of 'hostname'.
The reason it works is because the 'ssh' trick. In addition to being able to make yourself in the home network environment, the traffic on http://localhost:8080 is encrypted too.
Note that this forwarding uses port 8080 on the client rather than port 80. Binding to port 80 would require using root privileges every time we SSH.
To stop the ssh session, use ps -ef to find the process id and kill it.
This is most useful in situations where you have a machine which isn't publicly accessible from the internet, but you want others to be able to access a service on this machine. In this case, if you have SSH access to a remote machine which is publicly accessible on the internet, you can set up a reverse port forward on that remote machine to the local machine which is running the service.
ssh -R 8000:localhost:80 user@REMOTE_MACHINE
This will forward port 8000 on the remote machine to port 80 on the local machine. Using this method, if you browse to http://localhost on the remote machine, you will actually connected to a web server running on port 8000 of the local machine.
Example 2: Suppose you have two machine
machine A (userA): under firewall. cannot be directly accessed (like corporate machines)
machine B (userB): local machine (like home machines)
Our goal is to access machine A directly from machine B.
This will require you to enter the password and leave you in the remote machine. If a nonstandard port is required, we can use -p option.
Now in the firefox, we need to go to Edit -> Preferences -> Advanced -> Network tab -> Settings... Check 'Manual proxy configuration' (The default is 'Use system proxy settings') and enter 'localhost' for SOCKS (SOCKS5 by default) Host and '4096' for the Port. Don't enter 'localhost' in the HTTP Proxy.
Note that in addition to the Firefox, we can use SeaMonkey (seems better than Firefox since the form works better on 1024x600 resolution). The network setting option in my 32-bit maxthon browser does not work (cannot show options). For the Opera browser, it cannot connect to Internet after I made a change to the network setting.
On Windows, we can use Putty. In short, in the left-hand panel, navigate through Connection > SSH > Tunnels. Enter 4096 in the Source Port box and select the Dynamic radio button. Click Add and “D4096″ will appear in the Forwarded Ports list. The setting in the firefox end is the same. See also my Windows wiki page.
Linux journal also put a video on youtube. We can use http://www.ipligence.com/geolocation to check the current location. The port number is 1080 in the example. The example actually also use '-N' option which means no interaction; i.e. ssh -N -D 1080 user@remoteip. So we won't see anything after we type our password. Once we want to stop SOCK proxy, we just need to hit Ctr+C on terminal.
Backgrounding OpenSSH Forwarding
Use the -N flag to tell ssh to not run anything, including a terminal, on the remote server, and the -f flag to tell ssh to go into the background on the client.
ssh -fNL 2222:localhost:22 user@remotehost &
By backgrounding this command, you get your original terminal back.
Both of methods give the same output. Note the latest changed file is shown at the bottom of the output.
Sort files by their size
use the '-S' option.
ls -lS
df: Display disk space
df -h
df -h -T # show the 't'ype of the file system
df -h -t ext4 # show file systems of given type (ext4 in this example)
df -a # show all file system (include ones that have a size of zero blocks)
Note for the NTFS type, it will be reported as fuseblk by mount or df command.
du and ncdu : Display directory size with sorting and human readable
Use ncdu program (more interactive). Although it is a command line program, we can use the mouse to move through each directory to see its sub-directories.
du -sh ~/* # won't include hidden directories, Fast
du -h ~/ --max-depth=1 # include hidden directories, SLOW
du -h ~/ --max-depth=1 --exclude ".*" | sort -nr | cut -f2 | xargs -d '\n' du -sh
du -a -h ~/ # kilobytes will be used, '-a' is to see all files, not just directories.
du -a ~/ | sort -nr | head -n 10 # sort from the largest file size first
The --exclude is to hide hidden directories, '-n' is to compare according to string numerical value, and '-r' is to reverse the result.
Note that the 'du' commands may be cheating. See the following screenshot.
# Method 1: rsync
rsync --progress -a sourceDirectory destinationDirectory
rsync --info=progress2 source dest
# Method 2: pv
sudo apt-get install pv
## copy a single file
pv inputfile > outputfile
## multiple files or directories
tar c sourceDirectory | pv | tar x -C destinationDirectory
rsync
Copy large file
If we need to copy large file (say > 4GB), we shall
format USB drive to NTFS (exFat seems not work)
Run rsync --progress source dest
Run sync
The last step (rsync) is important. We can use sudo iotop to check if rsync is finished or not.
rsync with exclude files/directories
See http://www.thegeekstuff.com/2011/01/rsync-exclude-files-and-folders/. The key is excluded files are relative to the current directory even we specify the absolute path. For example /path1/path2/file does not mean the file is located under /path1/path2; it means the file is located under ./path1/path2.
We add add multiple --exclude to exclude more files/directories.
rsync to exclude hidden files/directories
--exclude=".*" # exclude both hidden files and directories
--exclude ".*" # same as above
--exclude ".*/" # exclude hidden directories ONLY
--exclude ".git" # exclude .git directory ONLY; relative to the directory to be synchronized.
rsync with -a option
The -a flag in there stands for “archive,” and it’s important to include. It makes sure that the sync command is recursive (meaning any sub-folders and files inside of old_movies are copied too) and it’s important for preserving all of those modification dates, symbolic links, permissions, and other goodies we talked about earlier.
rsync with non-standard port
Use -e option
rsync -avz -e "ssh -p 23" mydir user@remoteip:
rsync with progress bar
Use --progress option.
rsync -avz --progress file1 file2
The 'rsync' command works on transferring files local to local too.
Or it is better to use -P option which is the same as --partial --progress. When it is used you’ll get a progress dialog at the command line that shows you which file is currently transferring, what percentage of that transfer is complete, and how many more files are left to check. As each file completes, you’ll see an ever-growing list of completed file transfers, which is great for making sure everything transfers successfully. It also allows you to easily resume suspended or interrupted transfers. Combined, you can see how it’ll show you which file was the last one to go, where it failed, and if it failed, give you the option to resume. It’s a pretty powerful combination.
rsync on Windows
Download and install command line rsync from http://www.rsync.net/resources/howto/windows_rsync.html. The website also provides a documentation. Some people are concern about the license issue. The website here provides a link to the free, old but usable version 4.0.5 which is newer than I tested v3.1.0.
Below are my note by using cwrsync v3.1.0 installer got from http://www.rsync.net.
The browser uses a fork of Electron, called Muon, designed with a focus on browser features. For example, it has support for Chrome extensions, and a higher level of security.
brb@brb-P45T-A:~$ sudo umount /media/brb/TOSHIBA
[sudo] password for brb:
umount: /media/brb/TOSHIBA: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
brb@brb-P45T-A:~$ sudo umount /dev/sdc1
umount: /media/brb/TOSHIBA: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
brb@brb-P45T-A:~$ lsof /media/brb/TOSHIBA/
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
VBoxSVC 5600 brb 18w REG 8,33 4294967295 3 /media/brb/TOSHIBA/Windows 10.ova (deleted)
brb@brb-P45T-A:~$ kill -9 5600
brb@brb-P45T-A:~$ lsof /media/brb/TOSHIBA/
brb@brb-P45T-A:~$ sudo umount /dev/sdc1
brb@brb-P45T-A:~$
# find the PID
pgrep ProgramName
# Kill the ProgramName process
kill -9 PID
Another command is killall. For example, if Firefox is acting up (as Firefox will do from time to time) simply type killall firefox and it should kill the application completely.
In the rare circumstances that this doesn’t work you can always type xkill and then click on the window that won’t close; this will completely close a given window immediately. See this.
umount /dev/sdb1 (depending on the device of course)
sudo mkfs.ext3 /dev/sdb1
sudo e2label /dev/sdb1 usbdrive (change the label)
We can create MS-DOS file system by
sudo mkfs.vfat /dev/sdb1
Add a new user with home directory
adduser xxx
adduser is better than useradd since useradd does not create home directory and it does not even ask the password for new user. adduser will interactively ask user information.
To delete the user and home directory, use
deluser --remove-home xxx
gzip with multi cores
Use pigz utility. It makes a lot of difference. For example for a 21GB file, gzip can't finish the job after 30 minutes. But pigz only took 7 minutes on a 12-core machine.
sudo apt-get install pigz
pigz -9 FILENAME # compress & convert the file to FILENAME.gz
tar cf - paths-to-archive | pigz -9 -p 12 > archive.tar.gz
There is no need to use pigz to un-compress the file. gunzip is fast enough and only takes 4 minutes to decompress.
The '-9' (best compression) option does not make difference (6.6G vs 6.5G).
Compress a folder without full path name
Suppose we want to compress the folder ~/Documents and its subfolders. We want to include Documents folder name but not /home/brb/Documents name.
# Method 1. Include 'Documents' as the top folder name
cd ~/
tar -czvf tmp.tar.gz Documents
# Method 2. Mind the last dot. Not include 'Documents' as the top folder.
tar -czvf tmp.tar.gz -C /home/brb/Documents .
# Double check the tarball
tar -tzvf tmp.tar.gz
If we want to strip the upper directories when we uncompress a tar file, use --strip-components. For example, we can use --strip-components=1 to remove the Documents folder.
squashfs
squashfs is a heavy-compression based read-only filesystem that is capable of compressing 2 to 3 GB of data onto a 700MB. Linux liveCD are built using squashfs. These CDs make use of a read-only compressed filesystem which keeps the root filesystem on a compressed file. It can be loopback mounted and loads a complete Linux env. Thus when some file are required by processes, they are decompressed and loaded onto the RAM and used.
# create a squashfs file
sudo mksquashfs /etc test.squashfs
# mount the squashfs file
mkdir /mnt/squash
mount -o loop compressedfs.squashfs /mnt/squash
# you can acess the contents at /mnt/squashfs
# exclude files
sudo mksquashfs /etc test.squashfs -e /etc/passwd /etc/shadow
# or specify a list of exclude files given in a file
cat excludelist # /etc/passwd
sudo mksquashfs /etc test.squashfs -ef excludelist
List contents of tar.gz or tar.bz2
tar -tzvf myfile.tar.gz
tar -tjvf myfile.tar.bz2 # replace z with j
Extract files
Extract tar.gz or zip to a specified directory
tar xzvf XXXX.tar.gz -C DIRECTORY
unzip XXX.zip -d DIRECTORY
Extract gz file but keep the original gz file
gunzip -c x.txt.gz > x.txt
gunzip -c which simply writes the output stream to stdout
Extract .xz file
xz -d archive.xz
Extract tar.xz file
The bottomline is we don't need the 'z' parameter (used for gz ONLY but does not work for xz file) in the tar command for tar.xz files. And the method also works for tar.gz files. The argument '-f' means the archive file. Recall that the tar command can be used to store and extract files, so no default parameters.
tar xf archive.tar.xz
tar xf archive.tar.gz
Extract tar.bz2 file
tar -xjvf archive.tar.bz2 # replace z with j as we compare it to tar.gz file
You can also extract those files that match a specific globbing pattern (wildcards). For example, to extract from cbz.tar all files that begin with pic, no matter their directory prefix, you could type:
$ tar -xf cbz.tar --wildcards --no-anchored 'pic*'
To extract all php files, enter:
$ tar -xf cbz.tar --wildcards --no-anchored '*.php'
remove leading directory components on extraction with tar
If we want to extract certain files from a tarballj/archive, it is more efficient to use a virtual filesystem like AVFS. PS. for a large archive file, even extracting only a single file at the top directory it is terribly slow if we use the tar command directly.
Before we install the utility, let's look at the package dependecies of AVFS and Archivemount.
sudo apt-get install avfs
mountavfs
# Assume MyFile.tar.gz exists in the current directory
ls ~/.avfs/$PWD/MyFile.tar.gz#
# Alternatively, browse the content in Nautilus, but you need to add a trailing # character by hand to the path
# (Ctrl-L to access the address bar).
...
cat ~/.avfs/$PWD/MyFile.tar.gz#/README
# another tarball
ls ~/.avfs/$PWD/MyFile2.tar.gz#
umountavfs
https://github.com/Kickball/awesome-selfhosted This is a list of Free Software network services and web applications which can be hosted locally. Selfhosting is the process of locally hosting and managing applications instead of renting from SaaS providers.
Sovereign: A set of Ansible playbooks to build and maintain your own private cloud: email, calendar, contacts, file sync, IRC bouncer, VPN, and more.
POP works by contacting your email service and downloading all of your new messages from it. Once they are downloaded onto your PC or Mac, they are deleted from the email service.
IMAP allows you to access your email wherever you are, from any device. When you read an email message using IMAP, you aren't actually downloading or storing it on your computer; instead, you're reading it from the email service. As a result, you can check your email from different devices, anywhere in the world: your phone, a computer, a friend's computer.
Exchange offers the same syncing capabilities as IMAP, plus much more. Exchange is a Microsoft product, giving you the ability to still use Outlook as your email service and benefit from Exchange functionality.
It allows to upload/download/delete/list files and show info of user. The version I am using is v0.9.7. It works on linux, Windows/Cygwin, Raspberry Pi, etc.
I install it under
~/Downloads/andreafabrizi-Dropbox-Uploader-cdc2466 directory
MIN HOUR DOM MON DOW CMD
30 08 10 06 * /home/ramesh/full-backup
# 30 – 30th Minute
# 08 – 08 AM
# 10 – 10th Day
# 06 – 6th Month (June)
# * – Every day of the week
Twice a day
00 11,16 * * * /home/ramesh/bin/incremental-backup
# 00 – 0th Minute (Top of the hour)
# 11,16 – 11 AM and 4 PM
# * – Every day
# * – Every month
# * – Every day of the week
Every 10 minutes
*/10 * * * * /home/ramesh/check-disk-space
crontab
crontab cron-file-winter; crontab -l
Make sure the .sh file gives a complete path. For example,
#!/bin/sh
R --vanilla < arraytoolsip.R
does not work in cron job although it works perfect when we manually run it from the right path. The sh file should be
#!/bin/sh
R --vanilla < $HOME/Dropbox/scripts/arraytoolsip.R
To disable everything on crontab -l, run crontab -e then comment out each line you don't want to run with #. OR run crontab -r to empty the current crontab.
PATH and Shell
Cron knows nothing about your shell; it is started by the system, so it has a minimal environment. If you want anything, you need to have that brought in yourself. For example, to use 'ifconfig' command, I need to give it a complete path in my script file.
See here on how to add environment variable into cron environment.
Disable mail alert
If something went wrong with executing a cron job, cron will output a message "You have new mail in /var/mail/$USER". You can open this file using a text editor. To disable this alert, run 'crontab -e (see this post
Run mount -a to remount /etc/fstab without reboot, except the partitions with noauto option.
The following example shows a problem (as found from the output of df command) with </etc/fstab> where we use /dev/sdXY instead of UUID for specifying hard disks.
$ sudo blkid
/dev/sda1: LABEL="WD640" UUID="d3a0a512-bf96-4199-9674-f410f22f0a92" TYPE="ext4"
/dev/sdb1: UUID="afaa4bde-1172-4c54-8b0a-a324ad855355" TYPE="ext4"
/dev/sdb5: UUID="fb2a4ada-d80a-4e23-b4a2-67376b8b7e72" TYPE="swap"
$ sudo fdisk -l
Disk /dev/sda: 640.1 GB, 640135028736 bytes
...
Device Boot Start End Blocks Id System
/dev/sda1 2048 1250263039 625130496 83 Linux
Disk /dev/sdb: 640.1 GB, 640135028736 bytes
...
Device Boot Start End Blocks Id System
/dev/sdb1 * 2048 1217761279 608879616 83 Linux
/dev/sdb2 1217763326 1250263039 16249857 5 Extended
/dev/sdb5 1217763328 1250263039 16249856 82 Linux swap / Solaris
$ cat /etc/fstab
proc /proc proc nodev,noexec,nosuid 0 0
UUID=afaa4bde-1172-4c54-8b0a-a324ad855355 / ext4 errors=remount-ro 0 1
UUID=fb2a4ada-d80a-4e23-b4a2-67376b8b7e72 none swap sw 0 0
/dev/sdb1 /mnt/WD640 ext4 rw,nosuid,nodev 0 2
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 572G 413G 130G 77% /
...
/dev/sdb1 572G 413G 130G 77% /mnt/WD640
To fix the error here, modify the line starting /dev/sdb1 in /etc/fstab and replace it with the UUID. Then run sudo umount /mnt/WD640 and sudo mount -a. Done!
The above mount command assumes the usb drive has Windows vfat partition and users give non-root users the ability to unmount the drive.
If the USB drive is partitioned linux ext2/3, we can merely run mount command as
sudo mount /dev/sdb1 /mnt/usb
At the end, run umount command like
sudo umount /mnt/usb
To make the mounting automatically, edit the file /etc/fstab.
Now go to Applications>> System>> Ntfs Configuration Tool
Expand the "Advanced Configuration" and select all those partitions you want to be auto mounted and writable( The tool will detect all partitions at its startup).
Make sure the " Enable write support for internal devices" option is selected. Now click Close.
'Ctrl+Z' to stop (pause) the program and get back to the shell.
bg to run it in the background.
disown -h [job-spec] where [job-spec] is the job number (like %1 for the first running job; find about your number with the jobs command) so that the job isn't killed when the terminal closes.
# Note: OpenJDK 8 will not work
# We have to install Oracle Java
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt-get install netbeans # version 8.1 in my case
Netbeans has a built-in support for HTML/XML files. XML has an advantage over HTML since HTML cannot have any tags you want.
We need to download a plugin for markdown file support. Go to Tools -> Plugins. In the 'Settings' tab make sure the 3 items are checked. Go to 'Available plugins' tab
search 'markdown'. Install 'Markdown support'. It works on Netbeans 8.0 on x64 Ubuntu 14.04 and Netbeans 8.1 on my ARM Ubuntu 16.04.
For Markdown or XML, the comment syntax can be found here.
For some reason, the order of headlines on the navigator pane is not the same as they appeared on the file. So it is better to use XML file format.
My hack on Netbeans options (change to use a dark color on background).
It requires JDK. The community version is free. Download the tarball. Extract it and run bin/idea.sh. It even identifies a mismatch in my XML documentation that netbeans does not find.
SOLARIZED color. Copy icls file to ~/.IdeaIC2016.2/config/colors directory. Restart Intellij IDEA. Go to File -> Settings -> Editor -> Colors & Fonts -> Font.
To deactivate spelling checking: Ctrl + Alt + s -> Editor -> Inspections -> Spelling -> Typo. Uncheck it.
We need to create a project for IntelliJ IDEA to automatically open the file we are working on. IDEA will create a hidden folder call .idea. For git repository, we shall create the .gitignore file contains
The side panel contains a hierarchical view of the pages.
Right click on the page tab, we can create a new page or sub page.
On Ubuntu, the title bar is on the top of the desktop.
Auto save. Auto re-open the last session.
Handles several types of markup, like headings, bullet lists and of course bold, italic and highlighted. This markup is saved as wiki text so you can easily edit it with other editors.
When I run 'nano tmp', I got a message: Error reading /home/odroid/.nano/search_history: Permission denied. Press Enter to continue.
odroid@odroid:~$ ls -ld /home/odroid/.nano
drwxr-xr-x 2 root root 4096 Feb 12 08:01 /home/odroid/.nano
odroid@odroid:~$ ls -l /home/odroid/.nano
total 4
-rw------- 1 root root 15 Feb 12 08:01 search_history
A simple solution is
sudo chown -R odroid:odroid /home/odroid/.nano # note '-R' has to be capital
This seems to be a bug in nano after we use 'sudo nano [file]' (eg 'sudo nano /etc/chromium-browser/default') when the 'nano' program has not been run before.
superuser.com. A better habit to get into is to use sudoedit or sudo -e instead of 'sudo nano'.
$ ls /usr/share/nano/
asm.nanorc html.nanorc mutt.nanorc perl.nanorc ruby.nanorc
c.nanorc java.nanorc nanorc.nanorc pov.nanorc sh.nanorc
groff.nanorc man.nanorc patch.nanorc python.nanorc tex.nanorc
$ cat ~/.nanorc
include /usr/share/nano/sh.nanorc
include /usr/share/nano/c.nanorc
include ~/r.nanorc
R syntax highlight file r.nanorc. Note that I have to comment out line 29 starting with 'header'. A personal copy is saved in github.
To disable syntax highlight (useful if we use a terminal app on an Android ebook reader such as Boox), add -Ynone parameter.
vi keys
ctrl +f: page down. Ctrl +b: page up.
^: beginning of a line. $: end of a line.
command mode : this is the default when you run vi. Hit Esc key to return to the command mode. Command mode is right for moving about a file, copying or deleting a line, saving a file, etc.
Insert/edit mode : hit "i" (insert text before cursor position) or "a" (add text after cursor position) to enter the edit mode. The screen will show the text -- INSERT -- on the last line of the vi editor.
Command mode/leave edit mode: "ESC". In this mode, you can search, navigate or enter an insert model.
Last line mode: Press ':', vi editor will show ':' on the last line. If we continue to type 'q[Enter]' we will quit vi. If we continue to type 'wq', it will write the file and quits.
delete entire line: "dd"
undo: "u"
search forward: "/pattern" (case sensitive). Hit "n" to repeat search.
search backward: "?pattern" (case sensitive). Hit "n" to repeat search.
Difference between vi and vim. Especially Vim allows the screen to be split for editing multiple files. Use ":split" to split a screen using the same file, ":split filename" to split the screen using a new file and "Ctrl-w + Ctrl-w" to switch screens/viewports. More keyboard controls can be found at linux.com.
/sbin - critical executables for running the system, but should be used by superuser
/usr - non-critical files. Inside is /usr/bin, which contains most of the libraries used by apps.
/var - variable data such as databases, mails spools and system logs.
Difference of /bin, /sbin, /usr/local/bin, ...
/bin : For essential binaries; e.g. bash, cat, ls.
/sbin : is similar to /bin but for scripts with superuser (root) privileges required; e.g. shutdown command is located here. Local users have to use sudo to run binaries here.
/usr/bin : Same as first, but for general system-wide & non-essential binaries; e.g. grep, zip, docker, etc.
/usr/sbin : Same as above, but for scripts with superuser (root) privileges required.
/usr/local/bin or /usr/local/sbin for system-wide available (personal) scripts.
If you want to create your own scripts and make them available to all users, you’re pretty safe adding them to /usr/local/bin. Or to add my scripts to my local bin (~/bin) and then I create a symbolic link in /usr/local/bin to the commands I want to make public. As a result, I can manage all my scripts from the same directory but still make some of them publicly available since /usr/local/bin is added to $PATH. See this post.
DHCP lease time
On Windows, it is 8 days.
Get a New IP Address
dhclient -r # release your IP Address
dhclient # get your DHCP to issue you a new IP Address based on how it’s been configured.
See this post to know how to fix the problem of unknown author. In short, when I uncheck "Use global user setting" from Repository-> Repository Settings -> Advanced does the commit author change as expected.
See here for a list of Android apps related to bitbucket.
login shell - non desktop environment. ~/.bash_profile is sourced for the bash shell.
interactive shell - Ctrl+Alt+t to open a terminal from a graphical mode (desktop environment). ~/.bashrc is source. We usually edit ~/.bashrc to set up the environment to include fancy prompt, set aliases, set history options, or define custom shell functions.
export environment variables
Both a login shell and an interactive one. SSH (Putty) to connect to a remote machine.
When a shell runs a script or a command passed on its command line, it's a non-interactive, non-login shell.
Krita - professional painting program made by artists that want to see affordable art tools for everyone.
Pinta. It can be install by apt-get command. It works just line Window's paint. Ctr + v to paste an image and save to a file. To crop an image, click the selection tool (1st one) on the left hand side, then select a rectangle. Now click 'Image' > 'Crop to Selection' to finish.
mtPaint. It is included in Odroid - xu4 - Lubuntu 14.04. To crop an image, just select an area and click Image > Crop. It can be used to take a screenshot from the desktop by using either the application or through the command line (mtpaint -s). It will then display the screenshot in the application if you use the command line.
Several built-in sources like TIME, US NEWS report do not work. It'll show 'http error 404 not found'. The self added RSS feed method (see below) does work!
Run screen command first (run sudo apt-get install screen if necessary). You are now inside of a window within screen. This functions just like a normal shell except for a few special characters.
Create screen windows: Ctrl + a, then c. To close a screen window: exit. Once you close all screen windows, you shall see a message [screen is terminating] on the terminal.
View a list of open windows: Ctrl + a, then ".
Switch between windows: Ctrl + a and n for the next window and Ctrl +a and p for the previous window.
Attaching to and detaching screens: To detach (save) from the current screen session, Ctrl +a, and d (these keyboard shortcuts won't affect current execution). This will drop you into your shell. This is useful when you need to run a time-consuming job or your connection is dropped. To attach to an existing screen, use:
screen -r -d
Split screen:
To split the screen horizontally, Ctrl +a and S (capital).
To unsplit the screen, Ctrl +a and Q (capital).
To switch from one to the other: Ctrl +a and TAB.
Note: After splitting, you need to go into the new region and start a new session via Ctrl + a then c before you can use that area.
xrandr | grep maximum
gtf 800 480 59.9 # give some output used in the following line
xrandr --newmode "800x480_59.90" 29.53 800 816 896 992 480 481 484 497 -HSync +Vsync
xrandr --addmode "DISP3 BG" 800x480_59.90
xrandr --output "DISP3 BG" --mode 800x480_59.90
I cannot find the file /etc/X11/xorg.conf in my UDOObuntu 2 beta 2. It seems this file does not exist anymore. See this post about how to re-create it.
export DISPLAY
If we want to run a GUI app on a remote computer (such as Raspberry Pi/Beaglebone Black) and show the GUI app on the remote computer's screen using ssh, we can issue the following command before running the app.
-O file
--output-document=file
The documents will not be written to the appropriate files, but all
will be concatenated together and written to file. If - is used as
file, documents will be printed to standard output, disabling link
conversion. (Use ./- to print to a file literally named -.)
Follow the Quick Start to downloaded hbase tar ball. Suppose we save the tar ball under ~/Downloads folder and extract it in the same directory. We shall edit conf/hbase-site.xml file according to their instruction. The following is my case.
Before we follow the getting started guide to launch HBase, we shall make sure JAVA_HOME environment variable is created.
$ ls /usr/lib/java
$ export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64
Note that the last line may be replaced by
export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:bin/javac::")
Then we can launch HBase,
$ ./bin/start-hbase.sh
starting master, logging to /home/brb/Downloads/hbase-0.98.3-hadoop2/bin/../logs/hbase-brb-master-brb-P45T-A.out
brb@brb-P45T-A:~/Downloads/hbase-0.98.3-hadoop2$ ./bin/hbase shell
2014-07-06 09:51:34,621 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.98.3-hadoop2, rd5e65a9144e315bb0a964e7730871af32f5018d5, Sat May 31 19:56:09 PDT 2014
hbase(main):001:0> create 'test', 'cf'
2014-07-06 09:51:49,510 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
0 row(s) in 2.0770 seconds
=> Hbase::Table - test
hbase(main):002:0> list 'test'
TABLE
test
1 row(s) in 0.0530 seconds
=> ["test"]
hbase(main):003:0> exit
brb@brb-P45T-A:~/Downloads/hbase-0.98.3-hadoop2$ ./bin/hbase
shell2014-07-06 09:53:37,480 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.98.3-hadoop2, rd5e65a9144e315bb0a964e7730871af32f5018d5, Sat May 31 19:56:09 PDT 2014
hbase(main):001:0> list 'test'
TABLE
2014-07-06 09:53:44,373 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
test
1 row(s) in 1.4800 seconds
=> ["test"]
hbase(main):002:0> put 'test', 'row1', 'cf:a', 'value1'
0 row(s) in 0.4460 seconds
hbase(main):003:0> put 'test', 'row2', 'cf:b', 'value2'
0 row(s) in 0.0140 seconds
hbase(main):004:0> put 'test', 'row3', 'cf:c', 'value3'
0 row(s) in 0.0050 seconds
hbase(main):005:0> scan 'test'
ROW COLUMN+CELL
row1 column=cf:a, timestamp=1404654837532, value=value1
row2 column=cf:b, timestamp=1404654856976, value=value2
row3 column=cf:c, timestamp=1404654866298, value=value3
3 row(s) in 0.0560 seconds
hbase(main):006:0> get 'test', 'row1'
COLUMN CELL
cf:a timestamp=1404654837532, value=value1
1 row(s) in 0.0280 seconds
hbase(main):007:0> disable 'test'
0 row(s) in 1.6050 seconds
hbase(main):008:0> drop 'test'
0 row(s) in 0.2290 seconds
hbase(main):009:0> exit
brb@brb-P45T-A:~/Downloads/hbase-0.98.3-hadoop2$
For example, the Download link at the National Geographic Travel Photo Contest 2014 works for curl but not wget. I can use curl with -o option but wget with -o will not work in this case. Note with curl, we can also use the -O (capital O) option which will write output to a local file named like the remote file.
The main benefit of using the wget command is that it can be used to recursively download files.
The curl command lets you use wildcards to specify the URLs you wish to retrieve. And curl supports more protocols than wget (HTTP, HTTPS, FTP) does.
The wget command can recover when a download fails whereas the curl command cannot.
Actually curl supports continuous downloading too. But not all FTP connection supports continuous downloading. The following examples show it is possible to use the continuous downloading option in wget/curl for downloading file from ncbi FTP but not from illumina FTP.
$ wget -c ftp://igenome:[email protected]/Drosophila_melanogaster/Ensembl/BDGP6/Drosophila_melanogaster_Ensembl_BDGP6.tar.gz
--2017-04-13 10:46:16-- ftp://igenome:*password*@ussd-ftp.illumina.com/Drosophila_melanogaster/Ensembl/BDGP6/Drosophila_melanogaster_Ensembl_BDGP6.tar.gz
=> ‘Drosophila_melanogaster_Ensembl_BDGP6.tar.gz’
Resolving ussd-ftp.illumina.com (ussd-ftp.illumina.com)... 66.192.10.36
Connecting to ussd-ftp.illumina.com (ussd-ftp.illumina.com)|66.192.10.36|:21... connected.
Logging in as igenome ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /Drosophila_melanogaster/Ensembl/BDGP6 ... done.
==> SIZE Drosophila_melanogaster_Ensembl_BDGP6.tar.gz ... 762893718
==> PASV ... done. ==> REST 1706053 ...
REST failed, starting from scratch.
==> RETR Drosophila_melanogaster_Ensembl_BDGP6.tar.gz ... done.
Length: 762893718 (728M), 761187665 (726M) remaining (unauthoritative)
0% [ ] 374,832 79.7KB/s eta 2h 35m ^C
$ curl -L -O -C - ftp://igenome:[email protected]/Drosophila_melanogaster/Ensembl/BDGP6/Drosophila_melanogaster_Ensembl_BDGP6.tar.gz
** Resuming transfer from byte position 1706053
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 727M 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
curl: (31) Couldn't use REST
$ wget -c ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b147_GRCh37p13/VCF/common_all_20160601.vcf.gz
--2017-04-13 10:52:02-- ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b147_GRCh37p13/VCF/common_all_20160601.vcf.gz
=> ‘common_all_20160601.vcf.gz’
Resolving ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)... 2607:f220:41e:250::7, 130.14.250.10
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|2607:f220:41e:250::7|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /snp/organisms/human_9606_b147_GRCh37p13/VCF ... done.
==> SIZE common_all_20160601.vcf.gz ... 1023469198
==> EPSV ... done. ==> RETR common_all_20160601.vcf.gz ... done.
Length: 1023469198 (976M) (unauthoritative)
24% [===========================> ] 255,800,120 55.2MB/s eta 15s ^C
$ wget -c ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b147_GRCh37p13/VCF/common_all_20160601.vcf.gz
--2017-04-13 10:52:11-- ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b147_GRCh37p13/VCF/common_all_20160601.vcf.gz
=> ‘common_all_20160601.vcf.gz’
Resolving ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)... 2607:f220:41e:250::7, 130.14.250.10
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|2607:f220:41e:250::7|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /snp/organisms/human_9606_b147_GRCh37p13/VCF ... done.
==> SIZE common_all_20160601.vcf.gz ... 1023469198
==> EPSV ... done. ==> REST 267759996 ... done.
==> RETR common_all_20160601.vcf.gz ... done.
Length: 1023469198 (976M), 755709202 (721M) remaining (unauthoritative)
47% [++++++++++++++++++++++++++++++========================> ] 491,152,032 50.6MB/s eta 12s ^C
$ curl -L -O -C - ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b147_GRCh37p13/VCF/common_all_20160601.vcf.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
65 976M 65 639M 0 0 83.7M 0 0:00:11 0:00:07 0:00:04 90.4M^C
To download a copy of a complete web site, use the recursive option ('-r') By default it will go up to five levels deep. You can change the default level by using the '-l' option.
All files linked to in the documents are are downloaded to enable complete offline viewing ('-p' and '--convert-links' options). Instead of having the progress messages displayed on the standard output, you can save it to a log file with the -o option.
It seems there is no way to get the external IP address without not using external services.
# http://unix.stackexchange.com/questions/22615/how-can-i-get-my-external-ip-address-in-a-shell-script
sudo apt-get install dnsutils
dig +short myip.opendns.com @resolver1.opendns.com
# OR
curl http://ipecho.net/plain; echo
# OR
curl ipv4.ipogre.com
The above only gives the IP. The following method gives geo information too.
curl ipinfo.io
# OR give a specific IP (domain name does not work)
curl ipinfo.io/216.58.194.46
{
"ip": "216.58.194.46",
"hostname": "dfw25s12-in-f14.1e100.net",
"city": "Mountain View",
"region": "California",
"country": "US",
"loc": "37.4192,-122.0574",
"org": "AS15169 Google Inc.",
"postal": "94043"
}
Domain
WHOIS LOOKUP. For example, consider "r-pkg.org" domain,
The 'Sponsoring Registrar' shows who is the sponsoring registrar (eg GoDaddy.com).
The 'Registrant Name' shows who registered this domain.
Command Line Interface.
sudo apt-get install whois
whois r-pkg.org
IP-Lookup. For example, consider "r-pkg.org" domain.
The linux command line tool 'ping' shows the ip address.
When we use the ip address to search in the IP-Lookup, the IP owner info > 'Organization' or 'OrgName' field shows the owner of this IP (eg Digital Ocean, Inc).
The Domain owner info there gives the same (or less) information as WHOIS LOOKUP.
WhoIsHostingThis or webhostinghero (the returned result will be like Amazon, GoDaddy, CloudFlare, Github, Verizon, etc).
ping uses the ICMP Echo Message to force a remote host to echo a packet back to the local host. If packets can travel to and from a remote host, it indicates that the two hosts can successfully communicate.
Build a home network
Virtual router using vSphere. (Good ! It includes an introduction to vSphere installation)
Use the interface top iftop command. On ubuntu, we need to use sudo apt-get install iftop and then run it by sudo iftop -i eth0.
After that, we can press some keys to toggle options.
On Windows, we can use the tracert command. For example, tracert www.microsoft.com.
netstat
-l or --listening shows only the sockets currently listening for incoming connection.
-a or --all shows all sockets currently in use.
-t or --tcp shows the tcp sockets.
-u or --udp shows the udp sockets.
-n or --numeric shows the hosts and ports as numbers, instead of resolving in dns and looking in /etc/services.
-s --Print network stats
-r --print the network routing information
netstat -l # only listening ports
netstat -rn # displays the system's routing table
netstat -at
netstat -ant # For tcp
netstat -peanut # Easier to remember
netstat -anp | grep 3306 | wc -l # print the number of connections for the port mysql port i.e. 3306.
nmap - port scanning & IPs in local network
nmap - Network exploration tool and security / port scanner
Download the latest from https://svn.nmap.org/nmap/nmap-os-db. Note that the current revision number has to be found from the website. You can edit the file and insert the revision number on the 2nd line of your local copy.
Even I update the database, it cannot detect my Ubuntu 14.04 OS (it only shows OS details: Linux 3.8 - 4.9). For the Raspberry Pi, it can show information from the network adapter; e.g. MAC Address: AA:BB:CC:DD:EE:FF (Raspberry Pi Foundation) but not the OS name (OS details: Linux 3.2 - 4.8).
sudo mv /usr/share/nmap/nmap-os-db /usr/share/nmap/nmap-os-db-old
cd /usr/share/nmap
sudo wget https://svn.nmap.org/nmap/nmap-os-db
sudo apt-get install nmap
nmap 192.168.1.99 # does not require root privileges
# used to check open ports
nmap 192.168.1.* # show IPs and ports in LAN
sudo nmap -sP 192.168.1.1/24 # show connected IPs (no hostnames?) and MAC addresses
# If you don't use 'sudo' only partial devices can be found
# The output may contains the hostname. For example,
# Nmap scan report for brb-P45T-A.fios-router.home (192.168.1.xxx)
nmap -sV 192.168.1.1 # show Daemon name (in VERSION column) together with port number
nmap -T4 -F 192.168.1.99-255 # show connected IPs and open ports
# -F means fast
nmap -F taichimd.us
nmap -v taichimd.us
nmap -A 192.168.1.1 # Aggressive scan (more output)
nmap -p http,ssh,mysql taichimd.us # scan ports/services
# note that mysql will be shown as closed
nmap --open taichimd.us # scan open ports
sudo nmap -traceroute nih.gov
sudo nmap -sS -O 192.168.1.99 # -O shows operating system
# eth0 MAC
$ nmap localhost
Starting Nmap 7.01 ( https://nmap.org ) at 2017-10-09 15:01 EDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00016s latency).
Not shown: 996 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
631/tcp open ipp
$ host google.com
google.com has address 172.217.5.238
google.com has IPv6 address 2607:f8b0:4004:802::200e
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
$ nslookup google.com
Server: 127.0.1.1
Address: 127.0.1.1#53
Non-authoritative answer:
Name: google.com
Address: 172.217.7.238
Specify a new lower bitrate using the -b option in lame. For example if your starting mp3 has a quality of 256kbs you can lower its bitrate to 128kbps (or even lower like 64kbps) by:
Use Audacity. To raise (Amplify) volume: 1. Edit > Select All. 2. Effect > Amplify. Increase db from 0 to 15, for example. Check clip3. Export > MP3 or just start to listen.
Command line tool: avconv (replace ffmpeg program). See this post.
avconv -ss 00:00:10 -i OLD.mp3 -vol 2560 NEW.mp3
The anconv/ffmpeg -vol parameter amplifies the sound. The default value is 256 (no amplification), and you can adjust the number accordingly. Here it’s 2560, as it’s 10 times louder. Note that these are not decibel values or anything that sophisticated, but just an integer value. 512 equals to twice the volume, 768 three times, 1024 four times, etc. The -ss parameter specifies the start time offset. Here it will skip the first 10 seconds.
I found the converted file by sox has about one half file size compared to anconv/ffmpeg program (source file=47MB, anconv converted=135MB, sox converted file=54MB).
where '--norm' will normalize the audio and the 'trim' option set to skip the first 10 seconds. The silence parameter allows to trim silence at the beginning without a need to specify the number of seconds.
cut, delete or trim an audio
Open the audio file in audacity.
select a region in the waveform area. Do not select in the time interval area (above the waveform).
To precisely select a range from one position to the end. Click Zoom in several times. Click one position in the waveform and click Edit -> Select -> Cursor to the track end to select
Similarly, if we want to precisely select a range from the start to some position, we can click one position in the waveform and then click Edit -> Select -> Track start to cursor.
To move around the track, use the scrollbar (below the waveform and above the bottom toolbar, not quite clear in Ubuntu/Unity)
Click Edit -> Remove Audio or labels -> Cut/Delete/Trim Audio
play the new audio by clicking the green triangle.
The default one Evince seems slow when I try to view odroid magazine. I installed and tried MuPDF (github source code). It seems faster and I don't see blank pages when I view one odroid magazine. In terms of speed, mupdf >> xpdf >> okular >> Evince.
W - fit to width
H - fit to height
L - rotate page left (clockwise)
R - rotate page right (counter-clockwise)
12g - go to page 12
>,< - go to the next or previous page
+,- - zoom in or out
/ - search for text
n,N - Find the next or previous search result.
To copy a text, use right mouse button to select a text. Then use Ctrl+c to copy it.
Other pdf viewer choices are
acroread
xpdf
okular (KDE/Qt application), allow to change its background color
sudo apt-get install texlive-extra-utils
pdfcrop input.pdf output.pdf # no margins, works but seems too tight
pdfcrop --margins 5 input.pdf output.pdf # crop pdf but keep 5 bp from each side of page
pdfcrop --margins '5 10 20 30' input.pdf output.pdf
# left, top, right and bottom margins of 5, 10, 20, and 30 pt
# To actually crop something away, use negative values in the argument for crop.
# For example, to crops 50 pts from the left, top, right, bottom (in this order).
pdfcrop --margins '-50 -50 -50 -50' input.pdf output.pdf
One problem I found is (for newer PDFs with meta data) --margins initially removes the entire margin before implementing the adjustment. This will cause some pages being chopped out.
This java program gives me a better control on cropping
Download the file briss-0.9.tar.gz (8.7 MB) and extract it
Run java -jar briss-0.9.jar
Load the pdf file. It will ask what pages to be excluded from merging (This function does not work). Click 'Cancel' to continue.
It will automatically create two rectangle areas; one for odd (left) pages and the other for even (right)pages
Now we work on the left page first. Enlarge the selection to suit our need. Then right click & choose 'Select/Deselect rectangle' (a dash line will be added to the edges of the rectangle) and then 'Copy rectangles'.
Work on the right page. Right click and choose 'Delete rectangle'. Then 'Paste rectangles'.
Now we can click 'Action -> Preview' to preview the result. If we are satisfied with the result, we can click 'Action -> Crop PDF'. Done.
To highlight a line, click F6 (Tools -> Review) to turn on the annotation tool bar (it will be shown on the left hand side of the documentation). You can then click
the 4th icon to highlight a line (it may not be able to select the right texts we want. But when it works the result is nice)
the last icon to draw an ellipse or a rectangle (to change from an ellipse to a rectange you can click Settings -> configure Okular... -> annotation)
Another method is to use a windows program and run it using Wine. See the discussion here.
It seems there is no reliable way to find out when the power failed.
The linux command 'last' can show some information about system reboot.
Another way is to modify the BIOS to select the option like 'Power off and Reboot'. This won't automatically boot your computer when it is shutdown normally.
Bios: BIOS may have an easy-to-use wakeup scheduler
wakeonlan:
Eanble it: Check if it is enabled by default. If not, we can 1) enable it through a command (ethtool -s eth0 wol g) or 2) using the Network Manager
Send a wake up command: (from a second linux) /usr/bin/wakeonlan D0:50:99:82:E7:2B where D0:50:99:82:E7:2B is the IP on the machine you want to wake it up
sudo apt-get intall python-pip
sudo pip install speedtest-cli
# A slightly modified code that will create a one-line space/semi-colon
# delimited result is
git clone https://github.com/HenrikBengtsson/speedtest-cli-extras.git
speedtest-cli-extras/bin/speedtest-csv
works. But if I want to put it in cron, cron will issue an error speedtest-cli cannot be found. So I need to modify line 52 of the code in <speedtest-cli-extras/bin/speedtest-csv> to explicitly specify the location of speedtest-cli.
/usr/local/bin/speedtest-cli --share > $log
NOTE: 1. the results differ from the network connection. For example, the speed is good when I test it on the machine directly connected to the router. 2. It is helpful to modify the last line of the bash script to output what I need. 3. The separator is ";" in the output.
Lifehacker. Once it is launched, it is sitting on the task bar. Press Ctrl+Alt+W after selecting a word to look it up in Artha (a balloon tip will pop up on the screen top-right). It also supports using regular expressions to search words.
odroid@odroid:~/binary$ ./trans :zh-TW word
word
/wərd/
字
(Zì)
Definitions of word
[ English -> 正體中文 ]
noun
字
word, character, letter, calligraphy, symbol, style of writing
詞
word, term, speech, statement
單詞
word, individual word
話
words, word, dialect, saying, talk, speech
言
word, speech, character
言辭
words, word, what one says
筆墨
pen and ink, words, word, writings
約言
pledge, promise, word
verb
為 ... 措辭
word
odroid@odroid:~/binary$ time ./trans -brief :zh-TW word
字
real 0m4.249s
user 0m2.670s
sys 0m1.330s
http://www.eguidedog.net/ekho.php. Compilation/build works on x86 Ubuntu 14 and Odroid Ubuntu 15.10. On Odroid I have to follow their instruction to use 'make CXXFLAGS=-DNO_SSE' instead of 'make'. However, sound feels shaky on Odroid xu4.