Ssh

From 太極
Revision as of 14:36, 10 November 2024 by Brb (talk | contribs) (→‎xpipe - remote connection hub)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

SSH

Best security practices

  • The Best Ways to Secure Your SSH Server
  • OpenSSH Security Hardening Guide for Linux
  • 3 ways I configure SSH for privacy Change the default port, No more passwords, Decide who can log in.
  • Top 20 OpenSSH Server Best Security Practices
    1. Use SSH public key based login
    2. Disable root user login
    3. Disable password based login
    4. Limit Users’ ssh access
    5. Disable Empty Passwords
    6. Use strong passwords and passphrase for ssh users/keys
    7. Firewall SSH TCP port # 22
    8. Change SSH Port and limit IP binding
    9. Use TCP wrappers (optional)
    10. Thwart SSH crackers/brute force attacks such as using fail2ban and DenyHosts software
    11. Rate-limit incoming traffic at TCP port # 22 (optional)
    12. Use port knocking (optional)
    13. Configure idle log out timeout interval
    14. Enable a warning banner for ssh users
    15. Disable .rhosts files (verification)
    16. Disable host-based authentication (verification)
    17. Patch OpenSSH and operating systems
    18. Chroot OpenSSH (Lock down users to their home directories)
    19. Disable OpenSSH server on client computer
    20. Bonus tips from Mozilla

Install OpenSSL

How to Install the latest OpenSSL version from Source on Linux

Install ssh client

sudo apt install openssh-client

firewall

How to open ssh port using ufw on Ubuntu/Debian Linux 

sudo ufw allow ssh
# OR
sudo ufw allow 22/tcp

How to limit SSH connections with ufw

How to limit SSH (TCP port 22) connections with ufw on Ubuntu Linux

Configuration file ~/.ssh/config, avoid ssh connection timeout

Put the following in your ~/.ssh/config. 

Host remotehost
  HostName remotehost.com
  Port 4242
  User abcd
  IdentityFile "~/.ssh/id_rsa"
  ForwardX11 yes
  Compression yes
  TCPKeepAlive yes

Host github.com
    User YourUserName
    IdentityFile ~/.ssh/YourGithubSSHKey

After that we can run ssh remotehost.

To enable it for all hosts use: 

Host *
  ServerAliveInterval 240

Also make sure to run chmod 600 ~/.ssh/config

/etc/hosts

We can put some host name /etc/hosts as a shortcut to some IP address. After that we can run something like ssh ShortCutName

Change to a different port

$ sudo nano /etc/ssh/sshd_config  # looking for the line containing port 
$ sudo service ssh restart # tested on Ubuntu 14.04

Remember to change the Router settings.

On the client PC, use ssh USERNAME@HOSTNAME -p NEWPORT for a connection.

For security reason, use the port < 1024 (privileged ports and can only be opened by root)

ProxyJump, DynamicForward

How to Manage an SSH Config File in Windows and Linux

Avahi Daemon/Bonjour/Zeroconf and .local domain

  • "avahi-daemon" is needed in both remote server and local client in order to use "mydomain.local" instead of IP to access the remote machine.
  • sudo apt-get install avahi-daemon.

ipv6

  • https://wiki.ubuntu.com/IPv6
  • Check the file /proc/sys/net/ipv6/conf/all/disable_ipv6 to see if it is zero (0 means ipv6 is enabled).
  • To query the ipv6 address on the server, the following command from the post How to get the IPv6 IP address in Linux only return one ipv6. Note the command ip -6 address or /sbin/ip -6 addr | grep inet6 will return too many ipv6 addresses. 
    # Ubuntu 22.04 on x64
ip -6 addr show scope global dynamic mngtmpaddr up|egrep -o '([0-9a-f:]+:+)+[0-9a-f]+'

# Debian 12 on SBC
ip -6 addr show scope global dynamic up|egrep -o '([0-9a-f:]+:+)+[0-9a-f]+'
  • My ipv6 address looks like fdeb:2506:de09:4441:6604:3326:db42:d815
  • Consecutive groups of zeros can be represented with a double colon (::). However, the double colon can only be used once in an address. This allows for shorter representations of address with long sequences of zeros. For example, 2001:0db8:0000:0000:0000:0000:0000:abcd can be written as 2001:0db8::abcd.
  • The full ipv6 has a form of
    • 128 bits,
    • 8 groups and each group represents 16 bits,
    • 32 digits where each digit actually represents four bits.

ssh alias

  • Using linux's alias; eg put the following inside ~/.bashrc or ~/.zshrc 
    alias sshnocheck='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
alias scpnocheck='scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
alias openports='sudo netstat -tulpn | grep LISTEN'
alias checkport='sudo lsof -i -P -n | grep LISTEN'

    Don't forget to run sudo apt install net-tools lsof one time.

  • Using ssh config file to create aliases for hosts
  • 5 SSH alias examples in Linux

With this trick, ssh and scp (scp alias_name:Downloads/myfile .) work perfectly.

Modify ~/.ssh/config 

Host *
  ServerAliveInterval 120
  ServerAliveCountMax 30

Host your-alias_name
  User username
  HostName remote.sshserver.com
  Port 50001
  IdentifyFile ~/.ssh/id_file
  ServiceAliveInterval 120

Host work
  User abcde
  HostName work.workserver.com
  ServiceAliveCountMax 5
  StrictHostKeyChecking yes

According to the man of ssh_config:

  • ServerAliveCountMax: Sets the number of server alive messages (see below) which may be sent without ssh(1) receiving any messages back from the server. If this threshold is reached while server alive messages are being sent, ssh will disconnect from the server, terminating the session.
  • ServerAliveInterval: Sets a timeout interval in seconds after which if no data has been received from the server, ssh(1) will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server.

Running commands on a remote host

ssh user@host 'COMMANDS'

ssh user@host "command1; command2; command3"

COMMANDS="command1; command2; command3"
ssh user@host "$COMMANDS"

A practical example 

#!/bin/bash

IP_LIST="192.168.0.1 192.168.0.5 192.168.0.9"
USER="test"

for IP in $IP_LIST;
do
  utime=$(ssh ${USER}@${IP} uptime  | awk '{ print $3 }' )
  echo $IP uptime:  $utime
done

How to Run a Local Shell Script on a Remote SSH Server

How to Run a Local Shell Script on a Remote SSH Server

Disable password log in

If we like to ask all users to use key-based to log in, we can modify the line 

PasswordAuthentication no

in /etc/ssh/sshd_config and run sudo systemctl restart sshd.

Disable root log in

Modify /etc/ssh/sshd_config. Change this line: 

#PermitRootLogin yes

to 

PermitRootLogin no

and run /etc/init.d/sshd restart or sudo systemctl restart sshd. Also run sudo systemctl status sshd to check the status.

However, that line in my Ubuntu is 

PermitRootLogin without-password

According to this post, “without-password” means password authentication is disabled for root.

ssh log files

It is also helpful to check /etc/hosts.allow and /etc/hosts.deny for any possible wrong configuration.

Note that auth.log can show ssh security attacks. Am I compromised? 

$ grep sshd /var/log/auth.log
Feb 19 11:04:12 phenom sshd[16922]: Failed password for root from 92.62.131.23 port 49383 ssh2
Feb 19 11:04:12 phenom sshd[16922]: Received disconnect from 92.62.131.23: 11: Bye Bye [preauth]
Feb 19 11:04:14 phenom sshd[16924]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=92.62.131.23  user=root

Feb 19 11:04:36 phenom sshd[16998]: Invalid user enea from 113.160.227.93
Feb 19 11:04:36 phenom sshd[16998]: input_userauth_request: invalid user enea [preauth]
Feb 19 11:04:37 phenom sshd[16998]: pam_unix(sshd:auth): check pass; user unknown
Feb 19 11:04:37 phenom sshd[16998]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=113.160.227.93 
Feb 19 11:04:39 phenom sshd[16998]: Failed password for invalid user enea from 113.160.227.93 port 36090 ssh2
Feb 19 11:04:39 phenom sshd[16998]: Connection closed by 113.160.227.93 [preauth]
Feb 19 11:05:11 phenom sshd[17060]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:05:55 phenom sshd[17353]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:06:38 phenom sshd[17732]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:07:20 phenom sshd[17850]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:07:40 phenom sshd[17874]: refused connect from 221.194.47.221 (221.194.47.221)
Feb 19 11:08:01 phenom sshd[17955]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:08:41 phenom sshd[18118]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:09:22 phenom sshd[18280]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:10:02 phenom sshd[18353]: Invalid user support from 103.89.89.223
Feb 19 11:10:02 phenom sshd[18353]: input_userauth_request: invalid user support [preauth]
Feb 19 11:10:02 phenom sshd[18353]: pam_unix(sshd:auth): check pass; user unknown
Feb 19 11:10:02 phenom sshd[18353]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.89.89.223 
Feb 19 11:10:03 phenom sshd[18424]: refused connect from 58.218.198.170 (58.218.198.170)
Feb 19 11:10:04 phenom sshd[18353]: Failed password for invalid user support from 103.89.89.223 port 54218 ssh2
Feb 19 11:10:05 phenom sshd[18353]: fatal: Read from socket failed: Connection reset by peer [preauth]
Feb 19 11:10:07 phenom sshd[18425]: Did not receive identification string from 103.89.89.223
Feb 19 11:10:17 phenom sshd[18443]: Address 113.160.227.93 maps to static.vnpt.vn, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!

How to List Unsuccessful SSH Logins on Linux

How to List Unsuccessful SSH Logins on Linux 

grep "Failed password" /var/log/auth.log
# OR
journalctl _SYSTEMD_UNIT=ssh.service | egrep "Failed|Failure"

DenyHosts

Note that denyhosts package is no longer available in Ubuntu 14.04, 16.04 now. We can install install from its source DenyHosts-2.6.tar.gz.

Procedures: follow the README.txt file.

Fail2ban

# Check the status
$ /etc/init.d/fail2ban status 

$ sudo fail2ban-client status

# show banned IPs
$ sudo fail2ban-client status sshd

Status for the jail: sshd
|- Filter
|  |- Currently failed:	4
|  |- Total failed:	10
|  `- File list:	/var/log/auth.log
`- Actions
   |- Currently banned:	3
   |- Total banned:	3
   `- Banned IP list:	118.89.108.152 183.216.183.131 42.192.132.127

# manually unban an IP:
fail2ban-client set <JAIL-NAME> unbanip <IP-ADDRESS>
fail2ban-client set sshd unbanip <IP-ADDRESS>

# manually ban an IP:
fail2ban-client set <JAIL-NAME> banip <IP-ADDRESS>

# Another way to list banned IPs
sudo iptables -L -n

# Real-time Monitor 
sudo tail -f /var/log/fail2ban.log

Log in history: last command

The following command also shows how long a user has been logged in. 

last <username> | less

w/who can show who (and when) are currently logging in.

Generate a strong password

5 Ways To Generate A Random/Strong Password In Linux Terminal: pwgen, openssl, gpg, mkpasswd, makepasswd, ...

Put in your ~/.bashrc. See Top 20 OpenSSH Server Best Security Practices. 

$ genpasswd() {
	local l=$1
       	[ "$l" == "" ] && l=20
      	tr -dc A-Za-z0-9_ < /dev/urandom | head -c ${l} | xargs
}
$ genpasswd 16

login banners/messages, IP address

  • How to Disable SSH Welcome Message on Ubuntu
    • /etc/ssh/sshd_config
    • /etc/update-motd.d/
  • Beaglebone black (debian 11): the 'Message of the Day' file is located at /etc/issue 
    BeagleBoard.org Debian Bullseye IoT Image 2023-09-02
Support: https://bbb.io/debian
default username:password is [debian:temppwd]
  • For dietpi, after system startup, we see the following in the terminal 
    Debian GNU/Linux 12 DietPi tty1

DietPi login:
----------------------------
DietPi v9.8.0 : 14:43 - Sun 10/27/24
----------------------------
 - LAN IP : 192.168.1.2 (eth0)
 Please hit <return> to login

    See /boot/dietpi/postboot file.

    After login, it shows another message like Device model, CPU temp, LAN IP, MOTD, ...

  • https://kerneltalks.com/tips-tricks/how-to-configure-login-banners-in-linux/ There are two types of banners you can configure. Banner message to display before user log in (configure in file of your choice eg. /etc/login.warn) Banner message to display after user successfully logged in (configure in /etc/motd)
  • Where does the System Information information come from on login by SSH? 
    sudo run-parts /etc/update-motd.d/
  • Example on my Odroid xu4 running armbian. Each binaries under /etc/update-motd.d/ was used. 
    $ ssh [email protected]
  ___      _           _     _  __  ___   _ _  _   
 / _ \  __| |_ __ ___ (_) __| | \ \/ / | | | || |  
| | | |/ _` | '__/ _ \| |/ _` |  \  /| | | | || |_ 
| |_| | (_| | | | (_) | | (_| |  /  \| |_| |__   _|
 \___/ \__,_|_|  \___/|_|\__,_| /_/\_\\___/   |_|  
                                                   
Welcome to Armbian 23.05.0-trunk Bullseye with Linux 5.4.239-odroidxu4

No end-user support: built from trunk

System load:   1%           	Up time:       21 days 11:24	
Memory usage:  10% of 1.94G  	IP:	       192.168.86.86
CPU temp:      43°C           	Usage of /:    9% of 15G    	
storage/:      46% of 458G   	storage temp:  26°C           	

Tip of the day: Armbian is leading investor into software for custom ARM based hardware https://github.com/sponsors/armbian

[ 0 security updates available, 48 updates total: apt upgrade ]
Last check: 2023-10-15 00:00

$ ls /etc/update-motd.d/
10-armbian-header   35-armbian-tips     41-armbian-config           quotes.txt
30-armbian-sysinfo  40-armbian-updates  98-armbian-autoreboot-warn

Shutdown and quit

ssh user@hostname 'sudo shutdown -h now'

Double dash

What Does ‐‐ (double dash) Mean In SSH Shell Command?

ssh -A: forwarding of the authentication agent connection

Two-factor authentication

X11 forwarding

  • How to Run Graphical X Apps Over SSH in Linux. You'll have to enable X11 forwarding on the remote server. /etc/ssh/sshd_config file.
  • Mac connect to Linux
    • https://unix.stackexchange.com/a/46748
    • On Mac Mojave 10.14.6 (client side), I need to use -Y option (Enables trusted X11 forwarding) because "-X" does not work. The answer was found when I use "-v" option in ssh.
    • The 'ForwardX11' option in ~/ssh/config file does not help.
    • When I connect to my Linux machine, I can use display XXXX.png command to show the remote png file locally (display is part of imagemagick pack).
  • Linux connect to Linux does not have a problem.

Bypass SSH password login (convenient for CVS, git etc)

http://www.howtogeek.com/tips/bypass-ssh-logins-by-adding-your-key-to-a-remote-server-in-a-single-command/

  1. ssh-keygen -t rsa
  2. (make sure the remote server has .ssh directory)
  3. cat ~/.ssh/id_rsa.pub | ssh user@hostname 'cat >> .ssh/authorized_keys'
  4. ssh user@hostname

It helps with CVS log in too when the CVS works by using ssh protocol. Note that step 3 allows to run a shell command at a remote machine.

See https://help.github.com/articles/generating-ssh-keys also for similar instruction when work on github.

The ssh key can be copied to another a machine (pay attention to mode). Or let the new machine to create its own key pair and use ssh-copy-id to append the identity file to remote machine's ~/.ssh/authorized_keys file. See http://superuser.com/questions/332510/how-to-transfer-my-linux-ssh-keys-to-another-machine.

We can even have multiple ssh key on local machine by using <.ssh/config> file. See http://www.karan.org/blog/index.php/2009/08/25/multiple-ssh-private-keys.

Allow for a user

How to Enable/Disable SSH access for a particular user or user group in Linux

locale

-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8): No such file or directory Fix

SSH file server

How to create a personal file server with SSH on Linux

SSHFS: mount a remote file system over ssh

From my testing, files in local and remote machines are automatically synchronized. Convenient and dangerous. 

# Remote: only need to make sure openssh-server is installed

# Local
sudo apt install sshfs
mkdir /local/dir
sshfs user@ip:/remote/dir  /local/dir
# Unmount
fusermount -u /local/dir

To make this more practical, make sure /etc/fstab is modified appropriately and the ssh key is copied to the remote server. 

sudo nano /etc/fstab
sshfs#[email protected]:/remote/dir /local/dir

sudo nano /etc/fuse.conf # uncomment user_allow_other

sudo mkdir /media/roms
sudo chown -R $USER:$USER /media/roms
sshfs user@ip:/remote/roms /media/roms -o idmap=user,ro,allow_other

SSH server on Windows 10

Tmux

Enhancing SSH Login With A Tmux Session Selection Menu In Linux

xpipe/x-pipe - remote connection hub

ssh key

SSH key is useful if you want a password-less login to a remote system. Some useful resources:

Also there are different kinds of keys (see for example <~/.ssh/known_hosts file>): RSA, DSA and ECDSA (newer). They're keys generated using different encryption algorithms. See SSH key-type, rsa, dsa, ecdsa, are there easy answers for which to choose when?

The steps are

  • Check if there is an existing key
ls -al ~/.ssh
  • Create a new RSA key pair:
ssh-keygen -t rsa
ssh-keygen -f ~/.ssh/personalid -C "bitbucket"

where the comment 'bitbucket' will appear at the end of <~/.ssh/personalid> file.

  • Copy the public key to a remote host ([email protected]) over ssh. The current user (eg brb) and the remote user (eg git)have not any relationship (they most likely have different user names):
ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected] # this will 'append' the key to the remote-host’s .ssh/authorized_key.

Or (may not work:() 

cat ~/.ssh/id_rsa.pub | ssh [email protected] "mkdir -p ~/.ssh && cat >>  ~/.ssh/authorized_keys"
  • Delete the authorized key. Open the text file '.ssh/authorized_keys' and remove the offending lines.
  • Test if this is working by trying 'ssh [email protected]'.
  • To disable the password for root login. Type sudo nano /etc/ssh/sshd_config
PermitRootLogin without-password

Then run the following to put the changes into effect: 

reload ssh
# Or service ssh restart

Default key file

Force SSH Client To Use Given Private Key ( identity file )

The default is ~/.ssh/identity for protocol version 1, and ~/.ssh/id_rsa and ~/.ssh/id_dsa for protocol version 2.

Multiple ssh keys and ssh-add; ssh-agent

<Method 1> If we want to use a specific key in ssh, use 

ssh -i ~/.ssh/xxx_id_rsa [email protected]

<Method 2> Another way is to use ssh-add & ssh-agent to manager your keys. ssh-agent keeps your key in its memory and pulls it up whenever it is asked for it.

<Method 3> <~/.ssh/config> file.

Windows OS

Set up SSH public key authentication in WinSCP.

  • Use PuTTYgen to generate an rsa key.
  • We only need to save the private key since PuTTYgen can load a private key (right click the key file and choose Edit with PuTTYgen) and show the public key value.
  • The public key text box shown at the top of the PuTTYgen window is what we should copy to the server's ./ssh/authorized_keys file as instructed by the dialog.
  • NOTE: the "Key comment" part does not affect the authentication. It is located at the end of the public key. It's helpful to change that to more meaningful words.

ssh key management

Copy ssh keys to another computer

# Copy
$ ssh-copy-id -p 22 -i ~/.ssh/MyKey.pub [email protected]

# Test
$ ssh -p 22 -i ~/.ssh/MyKey [email protected]

http://askubuntu.com/questions/134975/copy-ssh-private-keys-to-another-computer 

$ chown brb:brb ~/.ssh/id_rsa*
$ chmod 600 ~/.ssh/id_rsa
$ chmod 644 ~/.ssh/id_rsa.pub

If we do not change the permission correctly in <id_rsa>, we will get a warning: Unprotected private key file. Permissions 0664 for '/home/USERNAME/.ssh/id_rsa' are too open.

Preserve ssh keys when upgrading computers

ls -l /etc/ssh/*key* > ~/key_list  # optional
mkdir ~/serverkeys && sudo cp -p /etc/ssh/*key* ~/serverkeys/ # back up, -p will preserve mode, ownership and timestamps
sudo cp -p ~/serverkeys/*key* /etc/ssh  # copy back
ls -l /etc/ssh/*key* | diff - ~/key_list # optional

If diff produces no output, you're finished.

Pay attention to the permissions. All the /etc/ssh/* files should be owned by root:root, with 644 permissions except for those that end in *key, which should be 600. 

udooer@udoo:~$ ls -l /etc/ssh/*key*
total 32
-rw------- 1 root root  668 Dec  8 14:43 ssh_host_dsa_key
-rw-r--r-- 1 root root  599 Dec  8 14:43 ssh_host_dsa_key.pub
-rw------- 1 root root  227 Dec  8 14:43 ssh_host_ecdsa_key
-rw-r--r-- 1 root root  171 Dec  8 14:43 ssh_host_ecdsa_key.pub
-rw------- 1 root root  399 Dec  8 14:43 ssh_host_ed25519_key
-rw-r--r-- 1 root root   91 Dec  8 14:43 ssh_host_ed25519_key.pub
-rw------- 1 root root 1679 Dec  8 14:43 ssh_host_rsa_key
-rw-r--r-- 1 root root  391 Dec  8 14:43 ssh_host_rsa_key.pub

udooer@udoo:~$ cd /etc/ssh; sudo tar -czvf ~/Downloads/sshkeys.tar.gz *key*
-rw------- root/root       668 2017-12-08 14:43 ssh_host_dsa_key
-rw-r--r-- root/root       599 2017-12-08 14:43 ssh_host_dsa_key.pub
-rw------- root/root       227 2017-12-08 14:43 ssh_host_ecdsa_key
-rw-r--r-- root/root       171 2017-12-08 14:43 ssh_host_ecdsa_key.pub
-rw------- root/root       399 2017-12-08 14:43 ssh_host_ed25519_key
-rw-r--r-- root/root        91 2017-12-08 14:43 ssh_host_ed25519_key.pub
-rw------- root/root      1679 2017-12-08 14:43 ssh_host_rsa_key
-rw-r--r-- root/root       391 2017-12-08 14:43 ssh_host_rsa_key.pub

udooer@udoo:~/$ cd /etc/ssh; sudo tar -xzvf ~/Downloads/sshkeys.tar.gz

Disable SSH host key checking

ssh -o UserKnownHostsFile=/dev/null \
    -o StrictHostKeyChecking=no \
     USERNAME@DOMAIN

To disable the checking for all hosts, in your ~/.ssh/config (if this file doesn't exist, just create it): 

Host *
    StrictHostKeyChecking no

Or for certain domains https://superuser.com/a/433621 

Host *.mydomain.com 
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  User foo
  LogLevel QUIET

Handling the ssh key change when connecting to a remote machine

An article from cybercitz.biz.

  • Method 1. Remove the key using ssh-keygen -R command.
$ ssh-keygen -R {server.name.com}
$ ssh-keygen -R {ssh.server.ip.address}
$ ssh-keygen -R server.example.com
$ ssh-keygen -f "/home/$USERNAME/.ssh/known_hosts" -R "xxx.xxx.x.xx"
  • Method 2. Add correct host key in /home/user/.ssh/known_hosts
  • Method 3. Just delete the known_hosts file If you have only used one ssh server
  • Method 4. Use ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no method. See Alias or Disable SSH host key checking.

What is tunnel

https://www.howtogeek.com/299845/why-is-a-network-tunnel-called-a-tunnel/. A tunnel provides a direct path that avoids some type of complexity you would otherwise have to deal with.

Cloudflare tunnel

See here.

awesome-tunneling

tunnel.pyjam.as

https://tunnel.pyjam.as/

Tested it on Slax

First we install openssh server on slax so we can remote access it and copy/paste commands. 

apt install openssh-server
systemctl start sshd.service
passwd # change root password

Testing tunnel.pyjam.as. 

apt update
apt install curl wireguard

touch newfile
python3 -m http.server # Serving http on 0.0.0.0 port 8000 (http://0.0.0.0:8000)

# open a new terminal/tab
curl https://tunnel.pyjam.as/8000 > tunnel.conf && wg-quick up ./tunnel.conf
# access http://0.0.0.0/8000 on https://<unique_slug>.tunnel.pyjam.as/

# To stop your tunnel
wg-quick down ./tunnel.conf

tailscale VPN

SSH Port forwarding

Local port forwarding: view a certain port web page on a remote domain locally

This port forwarding involves three computers (local, remote application server & remote SSH server). If the remote application server is the same as the remote ssh server, it will be reduced to involving 2 computers.

For example, we like to access home's router (192.168.1.1) information from an outsider computer. Suppose the host 'ssh_remote' is one computer in the home network and it can be accessed from outside world.

Another example is if we want to use Jupyter running on a remote machine from my local browser. Or if we want to view the remote router's page from my local browser (for some reason, it does not works the Verizon latest ax router. However, the SOCKS Proxy method still works).

The syntax is ssh -L localPort:app_remoteIP:app_remotePort username@ssh_remote 

ssh -L 8080:192.168.1.1:80 username@ssh_remote # access remote router's webpage
ssh -L 8080:localhost:80 username@ssh_remote   # access http://ssh_remote:80
ssh -i someKey.pem -L 443:127.0.0.1:8888 username@ssh_remote

The -L option specifies local port forwarding. In this case, port 8080 on the local machine was forwarded to port 80 on the remote machine. For the duration of the SSH session, pointing your browser at http://localhost:8080/ would send you to http://192.168.1.1/ as if you are in the same local network of 'ssh_remote'. We can try other ports like 1234 instead of 8080; it can be anything above 1024.

The reason it works is because the 'ssh' trick. In addition to being able to make yourself in the home network environment, the traffic on http://localhost:8080 is encrypted too.

Note that this forwarding uses port 8080 on the client rather than port 80. Binding to port 80 would require using root privileges every time we SSH.

To stop the ssh session, use ps -ef to find the process id and kill it.

Remote port forwarding (Reverse port forwarding)

This is most useful in situations where you have a machine which isn't publicly accessible from the internet, but you want others to be able to access a service on this machine. In this case, if you have SSH access to a remote machine which is publicly accessible on the internet, you can set up a reverse port forward on that remote machine to the local machine which is running the service. 

ssh -R 8000:localhost:80 user@REMOTE_MACHINE

This will forward port 8000 on the remote machine to port 80 on the local machine. Using this method, if you browse to http://localhost on the remote machine, you will actually connected to a web server running on port 8000 of the local machine.

Example 2: Suppose you have two machine

  • machine A (userA): under firewall. cannot be directly accessed (like corporate machines)
  • machine B (userB): local machine (like home machines)

Our goal is to access machine A directly from machine B.

We can run the following on the machine A 

# ssh -R remoteIP:remoteport:localIP:localport hostname
# ssh -R remoteport:localIP:localport hostname
ssh -R 2222:localhost:22 userB@machineB_IP
ssh -i /path/to/priv/key/id_rsa -f -N -R 2222:localhost:22 userB@machineB_IP

Then we can access machine A from machine B by 

ssh -p 2222 userA@localhost

If you want remote port forwarding configured every time you connect to a host, use the RemoteForward option in ssh_config . 

LocalForward server-IP:server-port client-IP:client-port

'D'ynamic port forwarding, SOCKS proxy, bypass blocked websites from work computer

ssh -D 4096 user@remoteip
ssh -D 4096 -p 23 user@remoteip

ssh -D 4096 -p 23 -f -N -q -C user@remoteip

This will require you to enter the password and leave you in the remote machine. If a nonstandard port is required, we can use -p option. -f is to fork process into background, -N means don't open a shell, -q means quiet and -C means compress data sent through tunnel. In order to close the connection, you need to find the process ID and kill it manually. You can use other port like 1080 (or something like 4321 which is also easy to remember too).

Now in the firefox, we need to go to Edit -> Preferences -> Advanced -> Network tab -> Settings... Check 'Manual proxy configuration' (The default is 'Use system proxy settings') and enter 'localhost' for SOCKS (SOCKS5 by default) Host and '4096' for the Port. Don't enter 'localhost' in the HTTP Proxy. Also check the option Proxy DNS when using SOCKS v5; otherwise someone on the network can see which site you are trying to get to even they are unable to see what you are seeing.

Open Google.com and search my ip to check if the change works.

Note that in addition to the Firefox, we can use

  • Chrome or chromium
  • SeaMonkey (seems better than Firefox since the form works better on 1024x600 resolution).
  • Brave browser does not support proxy

On Windows, we can use Putty. In short, in the left-hand panel, navigate through Connection > SSH > Tunnels. Enter 4096 in the Source Port box and select the Dynamic radio button. Click Add and “D4096″ will appear in the Forwarded Ports list. The setting in the firefox end is the same. See also my Windows wiki page.

Linux journal also put a video on youtube. We can use http://www.ipligence.com/geolocation to check the current location. The port number is 1080 in the example. The example actually also use '-N' option which means no interaction; i.e. ssh -N -D 1080 user@remoteip. So we won't see anything after we type our password. Once we want to stop SOCK proxy, we just need to hit Ctr+C on terminal.

Backgrounding OpenSSH Forwarding

Use the -N flag to tell ssh to not run anything, including a terminal, on the remote server, and the -f flag to tell ssh to go into the background on the client. 

ssh -fNL 2222:localhost:22 user@remotehost &

By backgrounding this command, you get your original terminal back.

ssh directly to a particular directory

How can I ssh directly to a particular directory? 

ssh -t xxx.xxx.xxx.xxx "cd /directory_wanted ; bash --login"

tunnel a ssh connection through an intermediate server (ssh -t)

Simple method is (-t means Force tty/pseudo-terminal allocation). 

$ ssh -t user1@domain1 ssh user2@domain2

Another method is to use ssh ProxyCommand to tunnel connections.

A third method is to 

$ ssh -L 9999:host2:22 user1@host1  # leave this terminal 
# open a new terminal tab
$ ssh -p 9999 user2@localhost

scp

file path with spaces (parentheses)

Use double quotes around the full path and a backslash to escape any space. 

scp [email protected]:"web/tmp/Master\ File\ 18\ 10\ 13.xls" .

Use the 'Tab' key to get the full path used by Linux. 

$ cd Calibre\ Library/calibre/Calibre-2\ days\ \(582\)/
$ exit
~/Downloads$ scp "taichimd:~/Calibre\ Library/calibre/Calibre-2\ days\ \(582\)/*.mobi" .

# Use the path we understand does not work
~/Downloads$ scp "taichimd:~/Calibre Library/calibre/Calibre-2 days (582)/*.mobi" .
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `scp -f ~/Calibre Library/calibre/Calibre-2 days (582)/*.mobi'

wildcard

You either need quotes, or a backslash before the star, but not both. And scp is not the one expanding it, the shell is. See How to use wildcards (*) when copying with scp? 

scp [email protected]:/abc/def/*.txt .   # no matches found:
scp [email protected]:/abc/def/\*.txt .

# macOS
scp 'helix:/data/xxx/file_*.rda' .

Copy multiple files

scp or sftp copy multiple files with single command

From local to remote: use space character. 

scp foo.txt bar.txt your_username@domain:

From remote to local: use escaped parentheses and comma sign. 

scp your_username@domain:~/\{foo.txt,bar.txt\} .

Some uses double quotes around the files with the space character to separate files but it does not work when I try to copy files from remote(biowulf) to local(mac).

Recursive copying

Use -r parameter.

Preserve permissions and modes

Use -p parameter.

Disable SSH host key checking

See here. scp is supposed to take the same command line options as ssh. Add the following options to the scp command. 

-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

scp files through one intermediate host

http://stackoverflow.com/questions/9139417/how-to-scp-with-a-second-remote-host

The following command is tested. 

scp -o 'ProxyCommand ssh user@remote1 nc %h %p' user@remote2:path/to/file .

A second method which is useful for ssh and scp commands 

$ ssh -L 9999:host2:22 user1@host1 # leave the terminal
# Open a new terminal
$ scp -P 9999 fileName user2@localhost:/path/to/dest/fileName   # transfer from local to remote. Note: Upper P.
$ scp -P 9999 user2@localhost:/path/to/source/fileName fileName # transfer from remote to local. Note: Upper P.
# If we only want to use ssh
$ ssh -p 9999 user2@localhost # Note: lower p.

scp with non-standard port: -P (capital)

Use -P argument. 

scp -P 23 myfile user@remoteip:

scp without a password

Assume the ssh key has been copied to the remote computer 

scp -p -P 22 -i ~/.ssh/MyKey [email protected]:MyFile .

Steps:

  1. Verify that local-host and remote-host is running openSSH (ssh -V)
  2. Generate key-pair on the local-host using ssh-keygen (Enter a passphrase here, do not leave it empty. A passphrase should be at least several words long, something you can easily remember. It's a bad idea to use a single word as a passphrase.)
  3. Install public key on the remote-host
  4. Give appropriate permission to the .ssh directory on the remote-host (chmod 755 ~/.ssh; chmod 644 ~/.ssh/authorized_keys)
  5. Login from the local-host to remote-host using the SSH key authentication to verify whether it works properly
  6. Start the SSH Agent on local-host to perform ssh and scp without having to enter the passphrase several times (ssh-agent $SHELL)
  7. Load the private key to the SSH agent on the local-host (ssh-add, need to enter the passphrase 1 time only)
  8. Perform SSH or SCP to remote-home from local-host without entering the password. It works for all remote machines containing the key from local-local.

Another option is to use ssh -i IDENTITY_FILE. See superuser.com.

ssh with password on the command line: sshpass

Rsync does use your ssh config

SCP user’s migration guide to rsync.

Some advantages of rsync over scp:

  • In-flight compression
  • Delta transfers
  • Syncing
Retrieved from "https://wiki.taichimd.us/index.php?title=Ssh&oldid=40580"