Apache

Installation on Ubuntu

Ubuntu

• How To Set Up Apache Virtual Hosts on Ubuntu 14.04/16.04
• How To Configure Apache Virtual Hosts In Ubuntu 18.04 LTS
  1. Install apache2 (sudo apt-get install apache2)
  2. Create the directory structure (sudo mkdir -p /var/www/example.com/public_html)
  3. Grant Permissions (sudo chown -R $USER:$USER /var/www/example.com/public_html)
  4. Create Demo Pages for Each Virtual Host (nano /var/www/example.com/public_html/index.html)
  5. Create a virtual host (sudo nano /etc/apache2/sites-available/example.com.conf)
  6. Enable a virtual host (sudo a2ensite example.com.conf)
• How to Configure Apache Virtual Hosts on Ubuntu 22.04

Cloudflare

• SSH service and R Shiny service should use DNS only (no HTTP proxy).
• Remember on Ubuntu we should open the necessary port using ufw.
• Raspberry Pi Home Server Episode 18: Install WordPress
  • Type: CNAME
  • Name: wp
  • Target: @ (OR something like example.com)
  • Proxy status: DNS only (temporarily). Change it back to Proxy once we have changed the settings in Nginx Proxy Manager & we can access the website on browser.

SSL certificate vs key

• https://superuser.com/questions/620121/what-is-the-difference-between-a-certificate-and-a-key-with-respect-to-ssl
• http://tldp.org/HOWTO/SSL-Certificates-HOWTO/x64.html
• https://en.wikipedia.org/wiki/Public_key_certificate
• How Do SSL Certificates Secure the Web?

Disable SSL

1. sudo nano /etc/apache2/sites-available/default-ssl.conf and change SSLEngine flag from on to off
2. sudo nano /etc/apache2/ports.conf and comment out sections containing port 443
3. sudo service apache2 restart

At this time, if I install Let's Encrypt I'll get an error message

$ sudo certbot --apache -d DOMAINAME
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for taichimd.us
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. DOMAINNAME (tls-sni-01): urn:acme:error:connection :: The server could not connect 
to the client to verify the domain :: Failed to connect to XX.XXX.XX.XX:443 for tls-sni-01 challenge

Domain: DOMAINNAME
   Type:   connection
   Detail:  Failed to connect to XX.XXX.XX.XX:443 for tls-sni-01
   challenge

To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

How to Create and Use Self-Signed SSL

• How to Create and Use Self-Signed SSL in Apache
• How to Create and Use Self-Signed SSL in Nginx

Secure Your Site Using HTTPS with Self-Signed or CA SSL Certificates

Secure Your Site Using HTTPS with Self-Signed or CA SSL Certificates on Ubuntu 22.04

• Distinction Between Self-Signed and CA Certificates
• Creating and Installing a Self-Signed SSL Certificate
• Creating and Installing a CA Certificate
  • The Difference Between Let's Encrypt & Other CA-Issued Certificates
  • How to Procure and Install non-Let's-Encrypt CA Certificates
  • How to Procure and Install Let's Encrypt Certificates

Installing fail2ban

• https://www.raspberrypi.org/documentation/configuration/security.md
• http://linux-sys-adm.com/ubuntu-16.04-lts-how-to-configure-firewall-iptables-fail2ban/
• How To Protect SSH and Apache Using Fail2Ban on Ubuntu Linux. Note: the text is better viewed by using the "Kindle Preview & Send" extension. It also shows how to test Fail2Ban for failed Login Attempts and for Apache DOS Attack.
• How to Prevent Brute-Force Attacks on Linux Using Fail2ban. Fail2ban works with most remote connection services such as SSH, FTP, SMTP, and HTTP.

Secure an Ubuntu server

How to secure an Ubuntu 16.04 LTS server - Part 1 The Basics

Optimize Apache on Ubuntu

https://thishosting.rocks/how-to-install-optimize-apache-ubuntu/

• Check if Apache is running

systemctl status apache2

• check what version you’re using with

apachectl -V

• Update your firewall. To allow traffic through both the 80 (http) and 443 (https) ports.

ufw allow 'Apache Full'

Install common Apache modules

• Speed up your website with the PageSpeed module

wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_amd64.deb
dpkg -i mod-pagespeed-stable_current_amd64.deb 
apt-get -f install
systemctl restart apache2

• Mod_pagespeed: How to Install and Configure mod_pagespeed Module with Apache on Ubuntu 18.04 LTS
• Enable rewrites/redirects using the mod_rewrite module

a2enmod rewrite
systemctl restart apache2

• Secure your Apache with the ModSecurity module

apt-get install libapache2-modsecurity
systemctl restart apache2

• Block DDoS attacks using the mod_evasive module

apt-get install libapache2-mod-evasive
nano /etc/apache2/mods-enabled/evasive.conf

Optimize Apache with the Apache2Buddy script

apt-get install curl
curl -sL https://raw.githubusercontent.com/richardforth/apache2buddy/master/apache2buddy.pl | perl

How Do You Protect Your Website Against DDoS Attacks?

• How Do You Protect Your Website Against DDoS Attacks?
• Fighting a DDoS Attack – Limiting Requests in NGINX
• How to Prevent a DDoS Attack with Nginx
• Mitigating DDoS Attacks with NGINX and NGINX Plus
• How to Configure mod_evasive for Apache DDoS Protection
• How I survived a DDoS attack (video)
• Rate Limiting With nginx

How To Speed Up a Slow Website

How To Speed Up a Slow Website

How to test your website

• How to Test Your Website Properly Before You Launch
• Load test
  • https://gatling.io/
• Using Google Lighthouse for Web Pages.

CentOS

• Genome desktop + Development tools
• http://www.dataarchitect.cloud/how-to-install-apache-php-7-1-and-mysql-on-centos-7-3-lamp/

• How To Install the Apache Web Server on CentOS 7 It works (7/10/2020)
  • Tested on VirtualBox with host-only netowork turned on
  • I am testing on CentOS 7 minimal iso
  • On <example.com.conf> file, I use 192.168.56.101 for ServerName
  • semanage command not found (Step 5). semanage command not found in CentOS 7 / 6 & RHEL 7 / 6 – Quick Fix helps.

    sudo yum provides /usr/sbin/semanage
     # yum provides command can find out the missing packages. In this case, it returns
     # policycoreutils-python-2.5-34.el7.x86_64 : SELinux policy core python utilities
    sudo yum -y install policycoreutils-python
    

  • In this example, the DocumentRoot is /var/www/example.com/html
• Configure Apache httpd to run as a service?

  # Installing the Apache package
  yum install httpd
  # Enable the http service on startup with the below command
  chkconfig httpd on
  # To start the httpd service :
  service httpd start
  

List all virtual hosts

sudo apache2ctl -S   # On Debian/Ubuntu 
sudo apachectl -S    # On CentOS/RHEL

sudo httpd -S

httpd.service failed

Job for httpd.service failed because the control process exited with error code. See “systemctl status httpd.service” and “journalctl -xe” for details

Enable cgi script

• Enable CGI Scripts on Apache. It works. The cgi file looks like a shell script file. When we put the cgi file in the designated path (/var/www/cgi-bin, the exact location depends on OS), the URL path will show the constrain.

  # Enable CGI Scripts in the Apache Configurations
  sudo nano /etc/httpd/conf/httpd.conf # add 2 lines in the "/var/www/cgi-bin" section
                                       # Options +ExecCGI
                                       # AddHandler cgi-script .cgi .pl
  sudo systemctl restart httpd
  
  # Upload the CGI Script and Set Permissions
  sudo nano /var/www/cgi-bin/test.cgi
  sudo chmod 755 /var/www/cgi-bin/test.cgi
  
  # Test it on a client browser
  hostname -I   # get the IP
  http://IP/cgi-bin/test.cgi  # It'll show Hello world
  

Note that if I just copy test.cgi file from /var/www/cgi-bin to /var/www/html and browse http://IP/test.cgi, then the browser just display the content of the cgi file.
• Run Perl CGI Scripts On CentOS 7 With Apache/Httpd. The cgi file is located under /var/www/html subfolder by creating a new conf/VirtualHost file under /etc/httpd/sites-available path.

  sudo nano /var/www/example.com/html/test.cgi
  sudo nano /etc/httpd/sites-available/example.com.conf
    # <VirtualHost *:80>
    #   ServerName 192.168.56.101
    #   DocumentRoot /var/www/example.com/html
    #   ErrorLog /var/www/example.com/log/error.log
    #   CustomLog /var/www/example.com/log/requests.log combined
    #   Options +ExecCGI
    #   AddHandler cgi-script .cgi .pl
    # </VirtualHost>
  sudo chmod 705 /var/www/example.com/html/test.cgi
  # Add IncludeOptional sites-enabled/*.conf
  # Add LoadModule cgi_module modules/mod_cgi.so
  #   to the end of /etc/httpd/conf/httpd.conf file
  sudo systemctl restart httpd
  

• CgiwithR (the tricky part is the where to put R.cgi and *.R files)

  R CMD INSTALL CGIwithR_0.73-0.tar.gz
  # It'll show the package is installed to '/usr/local/lib64/R/library'
  # It asks to copy the files R.cgi and .Rprofile in
  #    /usr/local/lib64/R/library/CGIwithR/cgi-bin/
  # to the cgi-bin area of your Web server. If necessary, 
  # modify the settings in the first part of R.cgi to 
  # suit your local configuration.
  
  cd /var/www/example.com/html/
  cp /usr/local/lib64/R/library/CGIwithR/examples/trivial.* .
  mkdir -p /home/brb/Sites/graphs
  chmod a+wx /home/brb/Sites/graphs
  nano trivial.R # change graphDir and graphURLroot
  
  nano trivial.html # change trivial.R location
                    # from "/cgi-bin/R.cgi/trivial.R" to "./R.cgi/trivial.R"
                    # Note: R.cgi is not a directory name
  
  sudo cp /usr/local/lib64/R/library/CGIwithR/cgi-bin/R.cgi /var/www/example.com/html/
  sudo cp /usr/local/lib64/R/library/CGIwithR/cgi-bin/.Rprofile /var/www/example.com/html/
  sudo chmod a+r /var/www/example.com/html/.Rprofile
  sudo chmod a+rx /var/www/example.com/html/R.cgi
  

Single board computer

An Excellent, Low-Cost Web Server: Using the ODROID-N2 For Internet Hosting

Apache2 Structure

/etc/apache2/
|-- apache2.conf
|-- envvars
|-- httpd.conf
|-- magic
|-- ports.conf
|-- conf-enabled
|       `-- *.conf
|-- mods-available
|       |-- *.load
|       `-- *.conf
|-- mods-enabled
|       |-- *.load
|       `-- *.conf
|-- sites-available
|       default, default-ssl
|-- sites-enabled
|       |-- 000-default  # points to ../sites-available/default
|       `-- default-ssl  # points to ../sites-available/default-ssl
|-- ssl
|       *.crt, *.key

• apache2.conf is the main configuration file. It puts the pieces together by including all remaining configuration files when starting up the web server.
• ports.conf is always included from the main configuration file. It is used to determine the listening ports for incoming connections, and this file can be customized anytime.
• Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/ directories contain particular configuration snippets which manage modules, global configuration fragments, or virtual host configurations, respectively.
• They are activated by symlinking available configuration files from their respective *-available/ counterparts. These should be managed by using our helpers a2enmod, a2dismod, a2ensite, a2dissite, and a2enconf, a2disconf . See their respective man pages for detailed information.
• The binary is called apache2. Due to the use of environment variables, in the default configuration, apache2 needs to be started/stopped with /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not work with the default configuration.

Where is the apache

ps aux | grep httpd

Some possible locations for httpd.conf

• /etc/httpd/httpd.conf
• /usr/local/apache2/conf/httpd.conf
• /opt/apache2/conf/httpd.conf

Document Root

By default, Ubuntu does not allow access through the web browser to any file apart of those located in /var/www, public_html directories (when enabled) and /usr/share (for web applications). If your site is using a web document root located elsewhere (such as in /srv) you may need to whitelist your document root directory in /etc/apache2/apache2.conf.

The default Ubuntu document root is /var/www/html (Ubuntu 14.04) or /var/www (Ubuntu 12.04). You can make your own virtual hosts under /var/www. This is different to previous releases which provides better security out of the box. In my case, the document roots for http and https are specified in the files

• </etc/apache2/sites-available/default>
• </etc/apache2/sites-available/default-ssl>

Important files

• http://www.basicconfig.com/linuxnetwork/ubuntu_web_server_setup Good cover
• http://www.htpcbeginner.com/how-to-setup-apache-web-server-on-ubuntu/2/. It includes how to set up DNS, running multiple websites
• How To Set Up Apache Virtual Hosts on Ubuntu 12.04 LTS from digitalocean.com. It teaches how to create a new virtual host file (instead of using the default one).

/etc/apache2/apache2.conf (important)

Main configuration file

/etc/apache2/httpd.conf

By default, this file is empty

/etc/apache2/ports.conf (important)

NameVirtualHost *:80
Listen 80

<IfModule mod_ssl.c>
    # If you add NameVirtualHost *:443 here, you will also have to change
    # the VirtualHost statement in /etc/apache2/sites-available/default-ssl
    # to <VirtualHost *:443>
    # Server Name Indication for SSL named virtual hosts is currently not
    # supported by MSIE on Windows XP.
    Listen 443
</IfModule>

<IfModule mod_gnutls.c>
    Listen 443
</IfModule>

/etc/apache2/mods-available/

Contains all the modules installed for your server.

/etc/apache2/mods-enabled/

Symbolic link in this directory that refers to the module file in /mods-available above to enable it.

/etc/apache2/sites-available/ (important)

Stores all the configuration files for the web sites serviced by Apache server. By default, only one file available, a default virtual host configuration file.

/etc/apache2/sites-available/default

This is the place to set up the document root for http port 80.

<VirtualHost *:80>
	ServerAdmin webmaster@localhost
        ServerName taichimd.us
	DocumentRoot /var/www/
	<Directory />
		Options FollowSymLinks
		AllowOverride None
	</Directory>
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>
...
</VirtualHost>

/etc/apache2/sites-available/default-ssl

This is the place to set up the document root for https port 443.

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
        ServerAdmin webmaster@localhost
        ServerName taichimd.us
        DocumentRoot /var/www
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
...
        #   SSL Engine Switch:
        #   Enable/Disable SSL for this virtual host.
        SSLEngine on
        SSLCertificateFile    /FullPathTo/CAName.crt
        SSLCertificateKeyFile /FullPathTo/KeyName.key
        SSLCACertificateFile "/FullPathTo/bundle.crt"
...
</VirtualHost>

/etc/apache2/sites-enabled/

Create a symbolic link to enable sites in /etc/apache2/sites-available.

udooer@udoo:~$ ls -l /etc/apache2/sites-enabled/
total 0
lrwxrwxrwx 1 root root 35 Dec 24 13:44 000-default.conf -> ../sites-available/000-default.conf

/etc/apache2/conf-available/, /etc/apache2/conf-enabled/

In UDOO, the dashboard webpage is pre-installed and the directory /var/www/html is empty. In Beaglebone, the apache is pre-installed (http://localhost/bone101/Support/bone101/) and /var/www/html is empty.

These directories have the same relationship as the sites-available and sites-enabled directories, but are used to store configuration fragments that do not belong in a Virtual Host. Files in the conf-available directory can be enabled with the a2enconf command and disabled with the a2disconf command.

udooer@udoo:~$ ls -lah /etc/apache2/conf-enabled/
total 8.0K
drwxr-xr-x 2 root root 4.0K Dec 25 08:15 .
drwxr-xr-x 8 root root 4.0K Dec 24 13:44 ..
lrwxrwxrwx 1 root root   30 Dec 24 13:44 charset.conf -> ../conf-available/charset.conf
lrwxrwxrwx 1 root root   40 Dec 25 08:13 javascript-common.conf -> ../conf-available/javascript-common.conf
lrwxrwxrwx 1 root root   44 Dec 24 13:44 localized-error-pages.conf -> ../conf-available/localized-error-pages.conf
lrwxrwxrwx 1 root root   46 Dec 24 13:44 other-vhosts-access-log.conf -> ../conf-available/other-vhosts-access-log.conf
lrwxrwxrwx 1 root root   33 Dec 25 08:15 phpmyadmin.conf -> ../conf-available/phpmyadmin.conf
lrwxrwxrwx 1 root root   31 Dec 24 13:44 security.conf -> ../conf-available/security.conf
lrwxrwxrwx 1 root root   36 Dec 24 13:44 serve-cgi-bin.conf -> ../conf-available/serve-cgi-bin.conf

See

• How To Install the Apache Web Server on Ubuntu 16.04
• conf-available vs sites-available. Both folders are loaded at startup of your apache. So you can place settings in both folders. The sites-enabled folder is more for Virtual-Host settings (subdomains, http/https) and the config-folder is for module configuration (can be as well webservices like owncloud or phpmyadmin that are available on a subfolder.

Commands

• Cheat-sheet for Apache2 commands Linux Mint / Ubuntu / Debian
• Simple commands to manage apache2 sites and Modules (a2ensite, a2dissite, a2enmod, a2dismod)
• Ubuntu httpd (Apache2 Web Server)

sudo a2ensite default      # activate the default site /etc/apache2/sites-available/default 
sudo a2ensite domain2.com  # activate each virtual host
sudo service apache2 reload

service apache2 status     # check if apache2 is running 
sudo service apache2 start # run this if apache2 is not running

Register a new domain

List of Internet top-level domains

Free dynamic dns/domain name

• How to Get a Free Domain Name for Your Website
• The 6 Best Free Dynamic DNS Providers
• https://freedns.afraid.org/
• https://www.noip.com/
• http://www.dot.tk/en/index.html?lang=en

Some free domain service provided by dot.tk

• .cf
• .ga
• .gq
• .ma
• .ml
• .nr
• .tk

Verify your domain

How to Verify Your Domain on Google Search Console

Multiple websites

Overview: Four steps

1. Create "site1.conf" and "site2.conf" under /etc/apache2/sites-available
2. sudo a2ensite site1.conf; sudo a2ensite site2.conf
3. sudo service apache2 reload
4. Modify DNS or change /etc/hosts on client's computer to see the effect locally

Optionally use sudo apache2ctl -S to see a list of virtual hosts

To avoid the error Cannot define multiple Listeners on the same IP:port: remove the line LISTEN 80. It seems this line is not necessary.

localhost, localtest.me

• localtest.me requires the system online. So it is not good for offline tasks like running multiple RStudio docker containers
• We can always add multiple domain to /etc/hosts for testing purpose
• Why does the registered domain name “localtest.me” resolve to 127.0.0.1?
• It is safe to use lvh.me instead of localhost for testing?

How To Enable And Run Multiple Websites Using Apache2 & /etc/hosts

The examples here assume you have multiple domain names pointing to 1 server with one IP. The end result is you can use different domain names to access websites hosted on the same server using the default port 80. Hint: you can change /etc/hosts file if you just want to do testing.

If we use nginx as a reverse proxy, we can even use the same domain name to have multiple applications running with different ports.

Some more practical example apps include Mediawiki and Nextcloud where the source is a folder that we can place the folder under /var/www/html/ and access on the browser using, for example, http://IP/FolderName.

We’re going to be using example.com and myexample.com domain names on a single Ubuntu server. See here.

sudo apt-get install apache2
sudo mkdir -p /var/www/html/example.com/public_html
sudo mkdir -p /var/www/html/myexample.com/public_html

sudo nano /var/www/html/example.com/public_html/index.html
cat /var/www/html/example.com/public_html/index.html
<html>
  <head>
    <title>Welcome to Example.com!</title>
  </head>
  <body>
    <h1>Success!  The example.com virtual host is working!</h1>
  </body>
</html>
sudo nano /var/www/html/myexample.com/public_html/index.html
sudo chown www-data:www-data /var/www/html
sudo chmod -R 755 /var/www/html

sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/example.com.conf
sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/myexample.com.conf

# Change ServerName, ServerAlias and DocumentRoot entries
sudo nano /etc/apache2/sites-available/example.com.conf
cat /etc/apache2/sites-available/example.com.conf
<VirtualHost *:80>
    ServerAdmin [email protected]
    ServerName example.com
    ServerAlias www.example.com
    DocumentRoot /var/www/html/example.com/public_html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
sudo nano /etc/apache2/sites-available/myexample.com.conf

sudo a2dissite 000-default.conf
sudo a2ensite example.com.conf
sudo a2ensite myexample.com.conf
sudo service apache2 restart 

sudo apache2ctl -S  # Get a list of all virtual hosts which are defined in all apache configuration files
ls -l /etc/apache2/sites-enabled/

# Edit /etc/hosts on client's computer
sudo nano /etc/hosts
# 127.0.0.1    example.com
# 127.0.0.1    myexample.com

dnsmasq

Use dnsmasq instead of /etc/hosts

dig foobar.test @127.0.0.1

How to create multiple virtual hosts

• It is OK to listen on the same port for different domains on the same server (tested)
• https://opensource.com/article/18/3/configuring-multiple-web-sites-apache. Only one httpd.conf was used. Multiple stanza can be put into this file.
• http://codingpad.maryspad.com/2012/03/14/how-to-create-multiple-virtual-hosts-in-ubuntu/
• http://httpd.apache.org/docs/2.2/vhosts/examples.html Virtual host examples for different scenarios
• http://www.tecmint.com/apache-ip-based-and-name-based-virtual-hosting/
• https://www.digitalocean.com/community/tutorials/how-to-set-up-multiple-wordpress-sites-on-a-single-ubuntu-vps
• http://www.unixmen.com/setup-virtual-hosts-apache-ubuntu-14-04-lts/
• How to Host Multiple Websites With One Apache Server. 2021
  • Configuring the Sites
  • Configuring Apache
  • Enabling Virtual Hosts
  • Site Aliases
  • Other Configuration Options
  • IP Address Hosts
  • Virtual Host Matching
• How to install multiple domains on a Nginx server

One IP two websites, Two IPs and two websites

• Run Multiple Site from one IP with reverse proxy Nginx
• How to create virtual host in NGINX (Both Name and IP based). Use /etc/hosts file to test name based.
• Useful if we use our http app that we cannot use port 80. NGINX as a Reverse Proxy (listening on port 80)
• Apache Web Server Complete Guide Dedoimedo
• See Reverse proxy

Two ports

How to Install NextCloud on OpenMediVault 5 with Remote Access and SSL. Nextcloud uses ports 80 & 443. OpenMediaVault uses port 81.

Monitor Apache

• sudo service apache2 restart
• htop and use Shift+f to highlight and follow this process (apache2).
  • Alternatively, use htop -p `pgrep -d ',' "apache2"` to list apache2 processes only with htop.
• sudo systemctl status apache2 . It shows the Active status (since what date/time), memory usage and CPU time.

$ sudo systemctl status apache2
● apache2.service - LSB: Apache2 web server
   Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           └─apache2-systemd.conf
   Active: active (running) since Thu 2019-12-05 22:11:59 EST; 11h ago
     Docs: man:systemd-sysv-generator(8)
  Process: 7495 ExecReload=/etc/init.d/apache2 reload (code=exited, status=0/SUCCESS)
  Process: 2874 ExecStart=/etc/init.d/apache2 start (code=exited, status=0/SUCCESS)
    Tasks: 11
   Memory: 411.1M
      CPU: 3h 9min 28.642s
   CGroup: /system.slice/apache2.service
           ├─  991 /usr/sbin/apache2 -k start
           ├─ 3077 /usr/sbin/apache2 -k start
...
Dec 06 07:35:02 phenom systemd[1]: Reloaded LSB: Apache2 web server.

See this solution if memory usage is not shown (works on raspbian stretch).

• 3 Ways to Check Apache Server Status and Uptime in Linux
  • Apachectl Utilities (Works, based on mod_status module which was enabled by default in Ubuntu, show server uptime and server load and how many requests currently being processed). Command line method (need to install lynx first): apachectl status. GUI method: http://localhost/server-status
  • Systemctl Utility (systemctl is not in Ubuntu 14.04 by default)
  • ps Utility (not useful)
• How to Monitor Apache Web Server Load and Page Statistics: mod_status module
• https://blog.serverdensity.com/monitor-apache/. The apache-top utility is cool! It can display the requested IP and the website. Use 'q' and wait 1-2 seconds for it to quit. (Don't use the one installed through apt-get)

  wget https://raw.githubusercontent.com/fr3nd/apache-top/master/apache-top.py
  python apache-top.py -u http://192.168.X.XXX/server-status

• Monitoring Apache with mod_status
• 7 Free Tools for Live Website Visitor Tracking (Mar 2019)

Log files

• ${APACHE_LOG_DIR} environment variable
• /var/log/apache2
  • access.log, access.log.1, access.log.2.gz, ...
  • error.log, error.log.1, error.log.2.gz, ...
• Suppose I try to access https://taichimd.us/mediawiki/index.php/Virtualbox. grep Virtualbox on access.log file shows several "Get" requests. The first one is "GET /mediawiki/index.php/Virtualbox HTTP/1.1", the 2nd one is skin and the rest are the thumbnails.

XXX.YYY.ZZZ.2 - - [06/Dec/2019:09:13:32 -0500] "GET /mediawiki/index.php/Virtualbox HTTP/1.1" 200 38206 "https://taichimd.us/mediawiki/index.php/Main_Page" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
...
XXX.YYY.ZZZ.2 - - [06/Dec/2019:09:13:33 -0500] "GET /mediawiki/images/thumb/7/7d/VBoxsnapshot2.png/600px-VBoxsnapshot2.png HTTP/1.1" 200 95778 "https://taichimd.us/mediawiki/index.php/Virtualbox" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

• On error.log we can see some alerting message

[Fri Dec 06 08:01:16.519555 2019] [php7:error] [pid 7536] [client 85.204.246.240:56106] script '/var/www/html/wp-login.php' not found or unable to s
   tat

GoAccess

• Installation
• How to See How Many Hits Your Server is Handling

$ echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list
$ wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install goaccess

Terminal display: Use "?" to get a list of keyboard shortcuts. For example, shift+1 to select the 1st module. Press Enter to get more information on this module. 'q' to go back to quit.

$ sudo nano /etc/goaccess/goaccess.conf # search apache and uncomment time-format & date-format
$ cd /var/log/apache2
# Method 1: only view the current log file
$ goaccess -f access.log access.log.1 --log-format=COMBINED 
# Method 2: view the last few days log
$ zcat access.log.*.gz | goaccess -f access.log access.log.1 --log-format=COMBINED 

Browser html display: For the real-time report to work, one needs to open the port 7890. Go to http://XXX.XXX.XXX.XXX/report.html to see the real-time change.

$ sudo ufw allow 7890/tcp
$ sudo goaccess /var/log/apache2/access.log --log-format=COMBINED \
  -o /var/www/html/report.html \
  --real-time-html 

$ sudo ufw delete allow 7890/tcp

Note that I still cannot see who is access apache in real-time.

truly real-time log viewer

• 4 Ways to Watch or Monitor Log Files in Real Time
• lnav – Watch and Analyze Apache Logs from a Linux Terminal
  • https://lnav.readthedocs.io/en/latest/commands.html#display
  • Understanding System Services with lnav
  • lnav – An Advanced Console Based Log File Viewer for Linux

# method 1:
tail -F /var/log/apache2/access.log

# method 2:
less +F  /var/log/apache2/access.log

# method 3: Cool but font color is too dim (brightness is too low)
sudo apt install lnav
sudo lnav /var/log/apache2/access.log /var/log/apache2/error.log

If we run lnav without anything, it will open /var/log/syslog file.

$ lnav # same as lnav /var/log/syslog

$ lnav /var/log/apt/

$ lnav /var/log/mysql

$ lnav /var/log/auth.log
$ lnav /var/log/mail.log
$ lnav /var/log/letsencrypt
$ lnav /var/log/unattended-upgrades

awstats

• https://www.awstats.org/
• AWStats. AWStats is a free powerful and featureful tool that generates advanced web, streaming, ftp or mail server statistics, graphically.
  • How to Set Up AWStats On Ubuntu Server
  • How to install AWStats on Ubuntu Linux. It includes a basic apache auth to AWStats using Apache's htpasswd.
  • Generating Web Site Statistics With AWStats & JAWStats On Debian Lenny & How To Install, Secure, And Automate AWStats (CentOS/RHEL)

Check connections

• How to Check Apache Concurrent Connections in Linux Using netstat and ss Command?
• https://serverfault.com/a/269293

netstat | grep http | wc -l 

• How to Find and Check Number of Connections to a Server

sudo netstat -plan|grep :80|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1

Check if a Website is up or down

• 6 Methods to Quickly Check if a Website is up or down from the Linux Terminal
• The Best Ways to Monitor If Your Website Goes Down
• Is website down or not? Free online website monitors to check website status

Misc

Add a User To Group www-data

Add a User To Group www-data. We can add an existing user or a new user to the www-data group.

Restrict Apache Information Leakage

https://www.thefanclub.co.za/how-to/how-secure-ubuntu-1604-lts-server-part-1-basics

How to Check Which Apache Modules are Enabled/Loaded in Linux

http://www.tecmint.com/check-apache-modules-enabled/

Running different sites on different ports

http://httpd.apache.org/docs/2.2/vhosts/examples.html#port

.htaccess file

• https://www.digitalocean.com/community/tutorials/how-to-use-the-htaccess-file
• See the example of Shorten the mediawiki URL.

Set Up Mod_Rewrite: redirect error

• https://www.digitalocean.com/community/tutorials/how-to-set-up-mod_rewrite-page-2 It contains an example to add www to a url.
• http://xmodulo.com/how-to-enable-mod_rewrite-in-apache2-on-debian-ubuntu.html
• How to Enable mod_rewrite in .htaccess File

Change the ownership and file permissions of the directory

How to Set Up an Apache Web Server on Linux

Forbidden You don't have permission to access /xxx/yyy on this server.

Solved: You Don’t Have Permission to Access on This Server. It's a good idea to have folders with permission mode of 755 and files with 644.

When I add a symbolic link file in /var/www/html to link to a sub-directory /home/$USER/Downloads/xxx, it does not work.

The detail error can be found in /var/log/apache2/error.log

Error: Symbolic link not allowed or link target not accessible

This post gives an explanation.

The solution in this case is to run

chmod 755 ~/Downloads

The problem seems to be specific to the attribute of the Downloads folder. If we untar/unzip to the $HOME folder, it does not have this problem because the attribute is already 755. The default attribute of Downloads in my Debian 8.4 is 700.

ServerName & ServerAlias

• How does ServerName and ServerAlias work?
• What is the difference between ServerName and ServerAlias in apache2 configuration?

Error. Could not determine the server’s fully qualified domain name

http://tuxtweaks.com/2009/07/how-to-configure-apache-linux/

echo "ServerName localhost" | sudo tee /etc/apache2/conf.d/fqdn
sudo service apache2 reload

On Ubuntu 18.04, I need to create /etc/apache2/apache2.conf and add the following ServerName localhost. After that, do sudo service apache2 reload. Then the command sudo apache2ctl configtest and sudo apache2ctl -S does not complain about the fqdn again.

There is no need of /etc/apache2/httpd.conf nor /etc/apache2/conf.d/fqdn.

How to set up a secure Apache webserver on Ubuntu

http://xmodulo.com/secure-apache-webserver-ubuntu.html

• Update TimeZone and Check Correct Time
• Disable AppArmor Conflicts
• Stop DDoS Attacks
• Stop Slowloris Attacks
• Stop DNS Injection Attacks
• Turn off Server Signature

Redirect vs rewrite

• Rewrite vs Redirect NGINX
  • URL rewrites are useful if you want to display beautiful SEO-friendly and intuitive URLs but serve content from non-pretty URLs
  • URL redirects are useful, if you have moved your page to a new location and want to serve content from this location.
• URL Rewrite vs. Redirect; What’s the difference? A redirect is a client-side request to have the web browser go to another URL. A rewrite is a server-side rewrite of the URL before it’s fully processed by IIS. This will not change what you see in the browser because the changes are hidden from the user.

Redirecting entire website to https

• How to Redirect HTTP to HTTPS on Apache
• http://www.tecmint.com/apache-htaccess-tricks/4/
• Redirect HTTP to HTTPS in Apache
• Apache Redirect to HTTPS

HSTS vs https

What Is HSTS and How Does It Protect HTTPS From Hackers?

A well-known public HTTP only site

• A list of HTTP test servers accepting GET/POST requests
• http://neverssl.com
• https://httpbin.org/
• http://1.1.1.1
• http://captive.apple.com/

sites-enabled vs sites-available directory

What is the difference between sites-enabled and sites-available directory?

The difference is that virtual sites listed in the sites-enabled directory are served by apache. In the sites-available directory there are the virtual sites that exist on your server but people can't access them because they are not enabled yet.

• sites-available: this directory has configuration files for Apache2 Virtual Hosts. Virtual Hosts allow Apache2 to be configured for multiple sites that have separate configurations.
• sites-enabled: like mods-enabled, sites-enabled contains symlinks to the /etc/apache2/sites-available directory.

A custom redirection example

http://www.tecmint.com/apache-htaccess-tricks/4/

How to Redirect Users to Maintenance Page

http://www.tecmint.com/apache-htaccess-tricks/4/

301 and 302 Redirects

What’s the Difference Between 301 and 302 Redirects?

How to Set Up Redirects with Just HTML

How to Set Up Redirects with Just HTML. HTML has redirect tools built in using meta tags.

<meta http-equiv="refresh" content="0; URL=https://www.example.com/" />

How to Perform Internal Redirection with mod_rewrite in Apache

http://www.tecmint.com/redirection-with-mod_rewrite-in-apache/

Redirect a Website URL from One Server to Different Server in Apache

• http://www.tecmint.com/redirect-website-url-from-one-server-to-different-server/
• How to Redirect a Web Page with Apache
• How can I redirect any ServerAlias to the respective ServerName?
• Redirecting and Remapping with mod_rewrite, When not to use mod_rewrite

Apache Virtual Hosting: IP Based and Name Based Virtual Hosts in RHEL/CentOS/Fedora

http://www.tecmint.com/apache-ip-based-and-name-based-virtual-hosting/

Virtual host file

• AllowOverride directive from the core module.

Forward proxy vs reverse proxy

• Difference between proxy server and reverse proxy server
• Differences Between Forward Proxy and Reverse Proxy
  • Forward proxy can be used by the client to bypass firewall restrictions in order to visit websites that are blocked by school, government, company etc. There’re many different kinds of forward proxy such as web proxy, HTTP proxy, SOCKS proxy etc.
  • Reverse proxy is mainly used by server admins to achieve load balancing and high availability. A website may have several web servers behind the reverse proxy.
• Proxy vs. Reverse Proxy (Explained by Example)

Proxy and reverse proxy

ProxyPass directive from the proxy module.

• Running multiple web applications on a Docker host with Apache
• Configure Apache as a Reverse Proxy Using mod_proxy on Ubuntu

25 Apache Interview Questions for Beginners and Intermediates

http://www.tecmint.com/apache-interview-questions/

Redirecting a non-www URL to a www URL

http://www.tecmint.com/apache-htaccess-tricks/3/

Add www to your domain name for your website

• https://www.linux.com/learn/tutorials/464510:weekend-project-create-virtual-hosts-with-apache Using ServerAlias or creating multiple virtualhost.
• https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-16-04

Disable directory browsing

• http://www.linuxscrew.com/2008/06/03/faq-how-to-disable-directory-browsing-in-apachehttpd/

Remove word Indexes from the following line in the file </etc/apache2/sites-available/default> & </etc/apache2/sites-available/default-ssl>

Options Includes Indexes FollowSymLinks MultiViews

Or try the following commands

sudo a2dismod autoindex
sudo service apache2 restart

Check remote server apache version

curl --head https://xxx.yyy.zzz

Hide version number

How to Hide Apache Version Number and Other Sensitive Info or this. It works (tested on Ubuntu 18.04). Add the following to /etc/apache2/apache2.conf and restart apache by sudo systemctl restart apache2

ServerTokens Prod
ServerSignature Off 

You can check the effect by visiting a wrong URL on a domain or using curl --head XXX or curl -I XXX or using Google Chrome's Inspect tool (tested on www.nih.gov). Pay attention to the Server key.

If we use Google Chrome, open 'Inspect' tool, go to 'Network' tab, reload the web page and click the html item.

403 Forbidden Error

• 403 Forbidden Error on Nginx Web Server
• Nginx Serving Static Content
• Set up nginx to serve files from subdirectories
• Configuring NGINX to Serve a Directory Listing

How to Configure a Custom 404 Error Page

How to Configure a Custom 404 Error Page — Apache Server Edition

Clean apache cache

How to Flush Apache's Cache

Disable a website through virtual host

http://manpages.ubuntu.com/manpages/hardy/man8/a2ensite.8.html

sudo a2dissite 000-default

List of all virtual hosts: apache2ctl -S

sudo apache2ctl -S

Show all loaded modules: apache2ctl -M

$ sudo apache2ctl -M

/usr/sbin/apache2ctl: 87: ulimit: error setting limit (Operation not permitted)
Loaded Modules:
 core_module (static)
 log_config_module (static)
 logio_module (static)
 mpm_prefork_module (static)
 http_module (static)
 so_module (static)
 alias_module (shared)
 auth_basic_module (shared)
 authn_file_module (shared)
 authz_default_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 cgi_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 mime_module (shared)
 negotiation_module (shared)
 php5_module (shared)
 reqtimeout_module (shared)
 setenvif_module (shared)
 ssl_module (shared)
 status_module (shared)
Syntax OK

favicon.ico

For some reason, if I just rename an animated gif file to <favicon.ico>, the file can be viewed locally and works when I put it on /var/www (http). For https, the default favicon does not show up and I have to manually put the favicon in the index.html file (good if you wish your pages to use different favicon sets).

<head>
...
<link rel="icon" href="yinyang_rot.gif" type="image/x-icon">
</head>

For mediawiki, I don't need to rename to <favicon.ico>.

See also Create an animated gif file on how I create an animated gif file from a single png file.

Note that chrome browser does not support animated gif favicons. IE does not support either. Firefox does support animated gif favicons.

Password Authentication and htpasswd

• Set Apache Password Protected Directories With .htaccess File (2015, Old)
  1. Make sure Apache is configured to use .htaccess file. Modify httpd.conf
  2. Create a password file with the htpasswd command and make the password file readable by Apache web server.
  3. Create .htaccess file under the directory where we want it to be protected. The username and the password file are specified here.
  4. Test it
• How To Set Up Password Authentication with Apache on Ubuntu 16.04
  1. Installing the Apache Utilities Package (apache2-utils)
  2. Creating the Password File using the htpasswd command
  3. Configuring Apache Password Authentication. Authentication is done on a per-directory basis.
     • Option 1: Configuring Access Control within the Virtual Host Definition 000-default.conf (Preferred)
     • Option 2: Configuring Access Control with .htaccess Files
  4. Restart apache2 and test it
• How to setup basic HTTP authentication on Apache

Reverse proxy

• The goal is to use http://addressA instead of http://addressB:port where machine A and machine B can be the same or different.

(Excerpt from thegeekstuff) For example, let us say we have an enterprise application that is running on Apache and PHP on app.thegeekstuff.com, and we also have Nginx running on example.com.

In this example scenario, when someone goes to example.com, we can setup Nginx as a reverse proxy so that it will serve the enterprise apache/php application that is running on app.thegeekstuff.com.

But, for the end-user, they’ll only see example.com, they won’t even know anything about app.thegeekstuff.com. End-user will think the whole apache/php application is getting served directly from example.com.

Caddy

Pi-Hosted : Reverse Proxy with Caddy

Nginx

Use proxy_pass in /etc/nginx/sites-available/default and a symbolic link is created under /etc/nginx/sites-enabled.

• Quickly getting started with a reverse proxy setup. In the example below, I use R to create two web pages (port 4000 and 4321). Then I edit /etc/hosts. Finally I can browse http://example.com and http://example2.com to see the results.

  sudo unlink /etc/nginx/sites-enabled/default
  sudo nano /etc/nginx/sites-enabled/reverse-proxy.conf
  # server {
  #  listen 80;
  #  server_name example.com;
  #  location / {
  #    proxy_pass http://127.0.0.1:4000;
  #  }
  # }
  # server {
  #  listen 80;
  #  server_name example2.com;
  #  location / {
  #    proxy_pass http://127.0.0.1:4321;
  #  }
  # }
  sudo service nginx configtest
  sudo service nginx restart
  
  sudo nano /etc/hosts
  # Add a new line 127.0.0.1   example.com example2.com
  
  Rscript -e "servr::httd('/tmp')" -p4000
  # Open another terminal
  Rscript -e "servr::httd('~/')" -p4321
  
  # Open http://example.com and http://example2.com in a browser
  

• Nginx Reverse Proxy Set Up (video). Edit </etc/nginx/sites-available/default>
• Video. Users do not need to specify a non-default port and the apps can be running on any ports on the same machine. In the videos, node.js was used to run http web servers.
  • One domain. Application is running on a different port. https://youtu.be/311DvLh8ems .
  • One domain and different directories to access different applications running on different ports. https://youtu.be/PTmFbYG0hK4
  • Multiple domains. Note: /etc/hosts was changed to mimic we have own different domains. https://youtu.be/tO2N0LLVZjI. See also Is it possible for two hostnames share the same IP address?.
  • Full Node.js Deployment - NGINX, SSL With Lets Encrypt and commands
• Node.js → Nginx as frontend server for Node.js
• R web → servr R package
• Multiple websites on the same host/a single system
  • https://www.techandme.se/set-up-nginx-reverse-proxy/
  • https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-web-server-and-reverse-proxy-for-apache-on-one-ubuntu-14-04-droplet (it includes how to Blocking Direct Access to Apache)
• Create a portable battery and solar powered Raspberry Pi Zero web server, especially step 3 for accessing the monitoring page without an ugly port number.
• https://www.howtoforge.com/tutorial/how-to-install-nginx-as-reverse-proxy-for-apache-on-ubuntu-15-10/.
• http://deanattali.com/2015/05/09/setup-rstudio-shiny-server-digital-ocean/
• How to Install Airsonic Media Server on Ubuntu 18.04 LTS (Letsencrypt is included!)

server {
    listen 80;
    server_name example.com;  # change /etc/hosts if necessary
    location / {
       proxy_pass http://127.0.0.1:81;
    }
    # location /shiny {
    #   proxy_pass http://127.0.0.1:3838;
    # }
    # location /rstudio {
    #   proxy_pass http://127.0.0.1:8787;
    # }
}

Nginx Proxy Manager

• https://nginxproxymanager.com/
• Youtube: Nginx Proxy Manager
• Install Nginx Proxy Manager on a Raspberry Pi 4 (only install Nginx Proxy Manager).
• Raspberry Pi Home Server Episode 18: Install WordPress (how to use Nginx Proxy Manager). See also some screenshots.
• Configuring a Reverse Proxy With NGINX Proxy Manager and Duck DNS

Run both Nginx and Apache at the same time

• How can i run both nginx and apache together on Ubuntu? Go to /etc/nginx/sites-available then modify the host file which should listen to a different port (if you didn't change anything here you will find a default file, enter to change it. In the file change listen: 80 to the port you want to listen to. Don't forget to reload the service: service nginx reload
• How To Configure Nginx as a Web Server and Reverse Proxy for Apache on One Ubuntu Server. Here we assume there are two websites w/ different servernames hosted by Apache using port 8080. We want to use Nginx as a reverse proxy using the default port 80.
  1. Install Apache
  2. Change Apache to use port 81 (/etc/apache2/ports.conf & /etc/apache2/sites-available/000-default.conf files)
  3. (optional) Create two document root directories & two virtual host files (port 81 for both, different servername)
  4. (optional) Reload apache2 (sudo service apache2 restart)
  5. Check open ports (sudo apt install net-tools; sudo netstat -tlpn)
  6. Install nginx
  7. Remove the default virtual host's symlink (/etc/nginx/sites-enabled/default)
  8. (optional) Create virtual hosts for Nginx using the same procedure we used for Apache
  9. (optional) Create a virtual host file for the domain example.com (/etc/nginx/sites-available/example.com & nano /etc/nginx/sites-available/sample.org)
  10. (optional) creating symbolic links to the sites-enabled directory
  11. (optional) Do an Nginx configuration test (sudo nginx -t) & reload Nginx (sudo service nginx restart)
  12. Create apache's virtual host (/etc/nginx/sites-available/apache)
  13. Create a symbolic link (/etc/nginx/sites-enabled/apache)
  14. Do an Nginx configuration test (sudo nginx -t) & reload Nginx (sudo service nginx restart)
  15. Open a browser and go to http://localhost or http://localhost/subdir to test

Apache: ProxyPass & ProxyPassReverse to hide the right port

Use ProxyPass and ProxyPassReverse in /etc/apache2/sites-available/default

• Navidrome Is the Perfect Self-Hosted Music Server for Raspberry Pi
• Apache redirect to another port (it still works on Ubuntu 20.04) After you make these changes, add the needed modules and restart apache

sudo a2enmod proxy && sudo a2enmod proxy_http && sudo service apache2 restart

Note that if we want to enable https, we just need to issue one more command: sudo certbot --apache. It will list all domains and we can select one of names or leave input blank to select all.

• https://serverfault.com/questions/472482/proxypass-redirect-directory-url-to-non-standard-port. The URL can be http://sub.mydomain.com or http://mydomain.com/app1.
• hide port from URL of Rails server?
• VirtualHost Examples. The URL in the example looks like http://sub.mydomain.com
• Apache Module mod_proxy
• How To Use Apache HTTP Server As Reverse-Proxy Using mod_proxy Extension/module
• Configure Reverse Proxies Using Apache2 HTTP Server On Ubuntu 18.04
• How to Setup Apache As Frontend Proxy for Node.js
• Ubuntu 18.04 Apache Reverse Proxy, Loadbalancer Configuration Step by Step

Varnish Reverse Proxy

How to Install Varnish Reverse Proxy with Nginx on Ubuntu 16.04 LTS

Shorten URL

• 3 open source link shorteners
• How to build a URL shortener with Apache

phpMyAdmin

• How To Install and Secure phpMyAdmin on Ubuntu 18.04 LTS
• How to Install phpMyAdmin Securely
  • Option 1: Lock Down Apache, and Use SSH Port Forwarding
  • Option 2: Lock Down Apache, and Use a VPN
  • Option 3: Secure Apache with HTTPS and Basic Auth

Make your website load faster

7 Ways to Make Your Website or Blog Load Faster for Visitors

Clean website URLs

We cleaned our website URLs with R

Static vs dynamic content

What Is Static Content, and How Does it Affect Your Website?

Serve files

h5ai modern HTTP web server index. An example from https://dl.omnirom.org/.

Maintaining remote Websites with Sitecopy

Maintaining remote Websites with Sitecopy on Ubuntu 18.04

Detect vulnerabilities in web applications/Web Application Security Scanner

• Netsparker

Let's Encrypt

• From the official website
  • User Guide
  • How it works
• Let's Encrypt DNS Challenge
  • How to use --preferred-challenges
  • Adding the CloudFlare DNS Authenticator Plugin
• Cloudflare
  • Can I use cloudflare with Let's Encrypt
  • How to get a Let’s Encrypt certificate while using CloudFlare
  • How to issue Let’s Encrypt wildcard certificate with acme.sh and Cloudflare DNS
  • How to Validate a Let’s Encrypt Certificate on a Site Already Active on Cloudflare
• Install Apache2 HTTP Server on Ubuntu 18.04 LTS with Let’s Encrypt Free SSL/TLS Certificates (Step-by-step)
• How To Secure Apache with mod_md Let’s Encrypt on Ubuntu 20.04 LTS
• Getting started with acme.sh Let's Encrypt SSL client
• How to manage Let's Encrypt SSL/TLS certificates with certbot

Ubuntu, Expiry Date

A real working domain is needed.

certbot instructions for Ubuntu running apache

1. Download the Let’s Encrypt Client
2. Set Up the SSL Certificate
3. Set Up Auto Renewal

To check the certificate expiration date, run sudo certbot certificates

Another more complicated way is sudo openssl x509 -dates -noout -in /etc/letsencrypt/live/DOMAIN_NAME/cert.pem as described here.

For some reason (related to the Challenge type), my domain is verified by cloudflare instead of let's encrypt. I receive an email reminding the expiration before 20 days. I can manually run sudo certbot renew --dry-run --preferred-challenges http to renew my certificate (remove the option --dry-run to make the command effective).

Configuration file location

/etc/letsencrypt/renewal/DOMAIN_NAME.conf

Standalone mode/plugin

• How To Use Certbot Standalone Mode to Retrieve Let's Encrypt SSL Certificates on Ubuntu 16.04
• Setting up a Reverse-Proxy with Nginx and docker-compose. We also learn where are the key pem files located.
• When to use Let's Encrypt's webroot and standalone authorization

Manage certbot certificates

How to manage Let's Encrypt SSL/TLS certificates with certbot

sudo certbot certificates

User Guide and renewing certificates

https://certbot.eff.org/docs/using.html

Obtain a SSL letsencrypt certificate only without installing it

How to Install Visual Studio Code - Server IDE on Ubuntu 18.04 LTS

$ sudo certbot certonly --standalone --agree-tos -m [email protected] -d vscode.hakase-labs.io
$ sudo ls -lah /etc/letsencrypt/live/vscode.hakase-labs.io/

Delete certificates

https://community.letsencrypt.org/t/correct-way-to-completely-remove-issued-certificate-s-for-a-domain/7409/12

DOMAIN=XXX.XXX.XXX
sudo rm -rf /etc/letsencrypt/{live,renewal,archive}/{${DOMAIN},${DOMAIN}.conf}

Forcefully renew Let’s Encrypt certificate

How to forcefully renew Let’s Encrypt certificate

(2021-03-18)

# force renew
# It will not ask more questions 'if' we specify "--nginx" and "-d"
sudo certbot --force-renewal --nginx -d DOMAIN1,DOMAIN2

# check, the notBefore date/time will be the current
#        the notAfter date/time will be 90 days afte
sudo openssl x509 -dates -noout -in /etc/letsencrypt/live/YOUR_DOMAIN/cert.pem

sudo service nginx reload

Auto renew

Configuring Let’s Encrypt SSL Cert for Apache on Ubuntu 18.04.

Certbot will automatically run twice a day and renew any certificate that is within thirty days of expiration. This seems to be true according to other sources:

• A renewal cron job was created automatically in /etc/cron.d/certbot.
• Certbot SSL Certification Auto Renew Cron Job.
• How To Secure Apache with Let's Encrypt on Ubuntu 16.04

(11/27/2020) certbot is now installed through snapd on Ubuntu 20.04. Use systemctl status snap.certbot.renew.timer to check the auto renew timer.

Acme

• https://github.com/acmesh-official/acme.sh
• How to Request a LetsEncrypt Certificate using Acme

Apache reverse proxy

• Apache as reverse proxy for letsencrypt free https certificates (Works!)
• How to Set Up a Reverse Proxy With Apache
• Let's encrypt ssl with apache reverse proxy
• Configuring an Apache-Server as Reverse-Proxy on Ubuntu. It teaches to make a regular website XXX.conf to a site with let's encryption and then to a reverse proxy by directly modifying XXX-le-ssl.conf file.
• Apache Reverse Proxy with LetsEncrypt SSL
• https://community.letsencrypt.org/t/using-certbot-behind-an-ssl-reverse-proxy/36783
• How to setup a reverse proxy with LetsEncrypt SSL for all your Docker apps

Nginx with Let's Encrypt

• How to Install Visual Studio Code - Server IDE on Ubuntu 18.04 LTS.
  • Note that it generates SSL Letsencrypt before installing Nginx.
• How to Install Nginx with Let's encrypt and get A+ from SSLLabs Test

Nginx

• https://www.nginx.com/resources/wiki/start/
• Learning NGINX (Lynda.com)
• How To Install Nginx on Ubuntu 16.04
• How To Set Up A WordPress Site On Your Own Servers
• Nginx Secure Web Server with HTTP, HTTPS SSL and Reverse Proxy Examples
• What is the different usages for sites-available/sites-enabled vs the conf.d directory for nginx?
  • See /etc/nginx/nginx.conf which includes both /etc/nginx/conf.d/*.conf and /etc/nginx/sites-enabled/* on my Ubuntu
  • Avoid sites-available & sites-enabled At All Costs. The official upstream package of nginx from nginx.org only includes /etc/nginx/conf.d/*.conf only in /etc/nginx/nginx.conf.

Apache vs. Nginx

Apache vs. Nginx: Which Web Server Is the Better Choice?

Nginx is faster than Apache for static site.

Default root directory

nginx -V

Look up the --prefix value. On Ubuntu 16.04, it is /usr/share/nginx.

Configuration file

Full Example Configuration from nginx.com, Example nginx configuration from nginx.org

Create a portable battery and solar powered Raspberry Pi Zero web server

How to Configure NGINX

All NGINX configuration files are located in the /etc/nginx/ directory. The primary configuration file is /etc/nginx/nginx.conf.

• server_name contains the URL we will use in the browser. See an example How to Setup NGINX as Reverse Proxy Using Docker

server {
    listen 80;
    server_name site1.test;

    location / {
        include /etc/nginx/includes/proxy.conf;
        proxy_pass http://site1_app_1;
    }

    access_log off;
    error_log  /var/log/nginx/error.log error;
}

• proxy_pass contains the IP for the real hosting service. See Nginx proxy_pass: examples for how does nginx proxy_pass map the request

server {
    listen        80;
    server_name   test.com;
    location /app/ {
        proxy_pass      http://192.168.154.102;
    }
}
# test.com/app/xxxxx =>  http://192.168.154.102/xxxxx

server {
    listen        80;
    server_name   test.com;
    location /app/ {
        proxy_pass      http://192.168.154.102/maped_dir/;
    }
}
# test.com/app/xxxxx =>  http://192.168.154.102/maped_dir/xxxxx

nginx: [warn] conflicting server name XXX.XXX on 0.0.0.0:80, ignored

I got this message when I ran sudo nginx -t

How to Fix: Nginx Conflicting Server Name. When you ran certbot it added a second server block to the bottom of the file for serving port 80 and redirecting to HTTPS. It added HTTPS and port 443 to the original server block. But it didn’t remove the port 80 listener from it. You can fix this yourself by removing the lines which make your new HTTPS server block listen on port 80.

Virtual host file

• Understanding Nginx Server and Location Block Selection Algorithms.
• location directive
  • Nginx Location Directive Explained
  • 13 Nginx Location Directive Examples including Regular Expression Modifiers from thegeekstuff.com (good explanation)
  • If a tilde modifier (~) is present, this location will be interpreted as a case-sensitive regular expression match.
  • Nginx location match tester
• rewrite directive: change (rewrite) incoming URL into a different URL
  • 7 Nginx Rewrite Rule Examples with Reg-Ex and Flags from thegeekstuff.com
  • rewrite ^/shiny(.*) $1 break; is required for regular shiny server (no forward slash after ^/shiny)
  • rewrite ^/shiny/(.*)$ /$1 break; is required for docker shiny server

Others

• What does [::] mean in my nginx config file
• How To Optimize Nginx Configuration

LEMP

• How to Install Nginx with PHP and MySQL (LEMP Stack) on Ubuntu 20.04 LTS
• How to Install LEMP Stack (Nginx, PHP and MariaDB) on Debian 12

Nginx + PHP

Simple case.

# https://devanswers.co/install-php-nginx-ubuntu-20-04/
sudo apt install php-fpm

For mediawiki we need to install PHP and extra packages like database. Some of the packages may be redundant.

# https://websiteforstudents.com/install-mediawiki-on-ubuntu-18-04-lts-beta-with-nginx-mariadb-and-php-7-1-supports
sudo apt install php7.1-fpm php7.1-common php7.1-mbstring php7.1-xmlrpc php7.1-soap php7.1-gd php7.1-xml php7.1-intl php7.1-mysql php7.1-cli php7.1-mcrypt php7.1-zip php7.1-curl

# https://websiteforstudents.com/setup-nginx-web-servers-with-php-support-on-ubuntu-servers/ (I use in U 20.04)
sudo apt-get install php-fpm php-mcrypt php-cli php-mysql php-gd php-imagick php-recode php-tidy php-xmlrpc

# https://www.osradar.com/how-to-install-nginx-with-php-fpm-on-ubuntu-20-04/ (I use in U 20.04)
sudo apt install php php-cli php-fpm php-json php-pdo php-mysql php-zip php-gd php-mbstring php-curl php-xml php-pear php-bcmath

# mariadb-server (I use in U 20.04)
sudo apt-get install php php-apcu php-intl php-mbstring php-xml php-mysql mariadb-server php-curl imagemagick build-essential
sudo mysql_secure_installation

PHP-FPM and FastCGI

• Mediawiki
• https://en.wikipedia.org/wiki/PHP#PHPFPM
• PHP-FPM: Multiple Resource Pools (Video)
• Differences and dis/advanages between: Fast-CGI, CGI, Mod-PHP, SuPHP, PHP-FPM
• https://arstechnica.com/information-technology/2012/12/web-served-part-3-bolting-on-php-with-php-fpm/
• Writing Hello World in FCGI with C++

LibreNMS + Nginx

How to Install LibreNMS with Nginx on Ubuntu 22.04

How To Troubleshoot Common Nginx Errors

How To Troubleshoot Common Nginx Errors

Load balancing

How to Configure NGINX for Basic Load Balancing

Secure Your Nginx Web Server

• Tips and Tricks to Secure Your Nginx Web Server
• How to remove the Server header in NGINX

How to Enable HTTP/2 in Nginx

https://www.howtoforge.com/how-to-enable-http-2-in-nginx/

Pitfalls and common mistakes

• https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/

Fine-Tune NGINX Performance

• 8 Ways You Can Fine-Tune NGINX Performance on Linux
• Top 13 Web Server Performance Testing Tools

Hide version number

How to Hide Nginx Server Version in Linux

How to Set Up Basic HTTP Authentication in NGINX

How to Set Up Basic HTTP Authentication in NGINX

Reverse proxy

• A Guide to running a Reverse proxy for HTTP(S), SSH and MySQL/MariaDB using NGINX
• How to Install Node.js on Debian 10 and configure Nginx as a Frontend(reverse) Proxy Server

A simple example

Resource: Tutorial - Nginx as a Reverse Proxy for security cameras. Below is tested on Armbian 20.04 Udoo Dual. Once it works, I can change "/app/" to "/app" and I can change "http://192.168.1.100:8888/" to "http://localhost:8888/" .

Note that at the beginning I kept getting this error with a simple server.

# Suppose the server ip is 192.168.1.100

sudo nano /etc/nginx/sites-available/default 
  # Add the following lines below the location / { } block.
  # ** Pay attention to the forward slash **
  location /app/ {
          proxy_pass http://192.168.1.100:8888/;
          proxy_redirect off;
          proxy_set_header Host $host:$server_port;
          proxy_set_header X-Real-IP $remote_addr;
      }
sudo service nginx restart

# Open another terminal
# Here I use ruby to start a file server
ruby -run -ehttpd . -p8888

# Open a browser on a client computer
http://192.168.1.100/app/
# Or using the command line
curl http://192.168.1.100/app/   # success
curl http://192.168.1.100/app    # 301 Moved Permanently

R/Shiny example

• Running Shiny Server with a Proxy
• First test if http://IP:3838 works! If yes, continue the next.
• Below is the virtual host file in /etc/nginx/sites-available/shiny
• Note that after we run sudo certbot --nginx -d sub.domain.com, certbot will automatically modify the virtual host file by adding more ssl information. So it makes sense to create a backup of the original virtual host file; sudo cp shiny shiny.bak.
• The http block at the beginning of the file listed in RStudio page will cause an error. So I take it off.

• This has been tested on Ubuntu 20.04 and Nginx. So server_name and location / determine the public URL. And proxy_pass determines the private/internal URL.

  # /etc/nginx/sites-available/shiny
  server {
      listen 80;
      server_name sub.domain.com;
      
      location / {
        proxy_pass http://localhost:3838;
        proxy_redirect / $scheme://$http_host/;
        proxy_http_version 1.1;
        proxy_read_timeout 20d;
        proxy_buffering off;
      }
  }
  

• Run the following commands for trouble-shooting

  sudo ln -s /etc/nginx/sites-available/shiny /etc/nginx/sites-enabled/shiny
  
  sudo service nginx configtest  # If 'fail', check the error log
  cat /var/log/nginx/error.log
  sudo service nginx restart
  
  # Optional: obtain a let's encrypt certificate
  sudo certbot --nginx -d sub.domain.com
  

Password Authentication and htpasswd

• How to password protect directory with Nginx .htpasswd authentication
• Traefik vs Nginx for Reverse Proxy with Docker on a Raspberry Pi

Books

• Nginx HTTP Server - Third Edition by Clement Nedelcu

Exploring Nginx workers load arbitration using R/Shiny

Exploring Nginx workers load arbitration

Cloudflare tunnel

See here

Caddy

• How to deploy a single web page to the Caddy server from a Caddyfile
• Installing Caddy Server on Ubuntu
• How to set up Caddy web server on local machine (video)
• How to install Caddy web server on Ubuntu 22.04

Lighttpd

• How to Install Lighttpd with PHP-FPM and MySQL on Ubuntu 20.04 LTS
• How to Install Lighttpd with PHP-FPM and MariaDB on CentOS 8

Quick HTTP server using command line

• How To Quickly Serve Files And Folders Over HTTP In Linux. Python, Ruby, NodeJS, Rust. Miniserve (Rust) also supports username/password.

  python3 -m  http.server 8000
  
  sudo apt install ruby-full
  ruby -run -ehttpd . -p8000
  

• R
  • https://cran.r-project.org/web/views/WebTechnologies.html
  • Creating a Web Service in R
  • plumber package
  • Rook package

Single sign on

buzzfeed sso

Resources

• https://httpd.apache.org/docs/
• https://www.lynda.com/Apache-tutorials/Welcome/164983/186356-4.html