Docker Applications: Difference between revisions
(56 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
* [[Self_hosting|Self hosting]] | * [[Self_hosting|Self hosting]] | ||
* [https://noted.lol/what-are-your-most-used-self-hosted-applications What are your Most Used Self Hosted Applications?] and [https://www.reddit.com/r/selfhosted/comments/udxlnp/what_are_your_most_used_self_hosted_applications/ more] | * [https://noted.lol/what-are-your-most-used-self-hosted-applications What are your Most Used Self Hosted Applications?] and [https://www.reddit.com/r/selfhosted/comments/udxlnp/what_are_your_most_used_self_hosted_applications/ more] | ||
* [https://youtu.be/9uF2us2PabM?si=ia76EPobqVRohIOy Best Docker Containers for Home Server!] 2022/11 (video) | |||
== Top/best == | |||
* https://hub.docker.com/ | |||
* [https://github.com/veggiemonk/awesome-docker Awesome Docker] | |||
* [https://fleet.linuxserver.io/ LinuxServer.io]. Need to sort by downloads. | |||
* Reddit [https://www.reddit.com/r/homelab/comments/zy5io8/what_are_your_must_docker_containers_for_home_lab/ what are your must docker containers for home lab?] | |||
* [https://www.blackvoid.club/docker-top-10-images-and-solutions/ Docker top (18) images and solutions] | |||
* [https://www.smarthomebeginner.com/best-docker-containers-for-home-server/ 60+ Best Docker Containers for Home Server Beginners 2023] | |||
= Templates = | = Templates = | ||
Line 36: | Line 45: | ||
= Bioinformatics analyses = | = Bioinformatics analyses = | ||
<ul> | |||
<li>[https://www.biorxiv.org/content/early/2017/10/17/204495 Hot-starting software containers for bioinformatics analyses] | |||
<li>[https://academic.oup.com/gigascience/article/6/8/1/3894236 Bio-Docklets: virtualization containers for single-step execution of NGS pipelines] | |||
<li>[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009244 Performance and scaling behavior of bioinformatic applications in virtualization environments to create awareness for the efficient use of compute resources] Hanussek, 2021 | |||
<li>[https://github.com/StaPH-B/docker-builds docker-builds] - Dockerfiles and documentation on tools for public health bioinformatics | |||
<li>[https://github.com/shmohammadi86/ACTIONet/tree/R-release/mini_docker ACTIONet] and the paper [https://pubmed.ncbi.nlm.nih.gov/36650127/ Transcriptional vulnerabilities of striatal neurons in human and rodent models of Huntington's disease] 2023. | |||
<li>[https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-024-05695-9 CREDO: a friendly Customizable, REproducible, DOcker file generator for bioinformatics applications] 2024 | |||
[[File:CredoError.png|240px]] | |||
</ul> | |||
= Cloud provider = | = Cloud provider = | ||
Line 55: | Line 69: | ||
== Dashy == | == Dashy == | ||
* https://github.com/Lissy93/dashy | * https://github.com/Lissy93/dashy | ||
* [https://youtu.be/QsQUzutGarA Dashy, a configurable, themable, flexible personal dashboard for your homelab!] (video) and [https:// | * [https://youtu.be/QsQUzutGarA Dashy, a configurable, themable, flexible personal dashboard for your homelab!] (video) and [https://wiki.opensourceisawesome.com/books/self-hosted-dashboards/page/dashy-powerful-informative-configurable-self-hosting-dashboard Notes] | ||
** https://github.com/walkxcode/dashboard-icons for icons | |||
* Icons are located in ~/dash/icons/dashboard-icons/png directory. | * Icons are located in ~/dash/icons/dashboard-icons/png directory. | ||
* Download an icon from [https://icons8.com/icons/ icons8.com] or [https://uxwing.com/ uxwing.com]. The default 48x48 works fine. | |||
* Better to work on one icon and then save the change. Repeat. | |||
<pre> | <pre> | ||
docker run -d \ | docker run -d \ | ||
Line 66: | Line 83: | ||
lissy93/dashy:latest | lissy93/dashy:latest | ||
</pre> | </pre> | ||
== Glance == | |||
* https://github.com/glanceapp/glance | |||
* [https://www.youtube.com/watch?v=2dGD4C2g8ig Glance Dashboard: Your New Favorite Docker Dashboard] | |||
== Homepage == | |||
https://github.com/gethomepage/homepage | |||
== Flame == | |||
https://github.com/pawelmalak/flame | |||
== Homer == | == Homer == | ||
Line 101: | Line 127: | ||
= Google photo = | = Google photo = | ||
* [https://docs.photoprism.app/getting-started/docker/ PhotoPrism] | * [https://docs.photoprism.app/getting-started/docker/ PhotoPrism] | ||
== Immich: Google photo replacment == | |||
* [https://github.com/immich-app/immich immich] - High performance self-hosted photo and video backup solution | |||
* [https://www.makeuseof.com/raspberry-pi-immich-google-photos-alternative/ Immich Is a Great Google Photos Alternative You Can Host on Raspberry Pi] | * [https://www.makeuseof.com/raspberry-pi-immich-google-photos-alternative/ Immich Is a Great Google Photos Alternative You Can Host on Raspberry Pi] | ||
* [https://missilepuppy.com/blog/2024/01/09/immich-finally-a-good-self-hosted-google-photos-replacement/ Immich – Finally, A Good Self-Hosted Google Photos Replacement] | |||
= ownCloud = | = ownCloud = | ||
Line 133: | Line 163: | ||
* Nextcloud uses owncloud mobile application. | * Nextcloud uses owncloud mobile application. | ||
* Nextcloud includes a small mp4 file. I can use browser to play mp4. Photos files can be opened in browsers too (including slideshow). | * Nextcloud includes a small mp4 file. I can use browser to play mp4. Photos files can be opened in browsers too (including slideshow). | ||
== Add external storage like SAMBA == | |||
To connect to a Samba drive in Nextcloud, you will need to use the '''External Storage''' app. This app allows you to mount external storage services, including Samba shares, as directories within your Nextcloud instance. Here are the steps to connect to a Samba drive in Nextcloud: | |||
First, make sure that the '''External Storage''' app is installed and enabled on your Nextcloud instance. You can do this by going to the '''Apps''' section in the Nextcloud '''admin panel''' and searching for the External Storage app. If it is not installed, you can install it from there. | |||
Once the External Storage app is installed and enabled, go to the Settings section in the Nextcloud admin panel and select the External Storages option under the '''Administration''' section (Left panel, scroll down to see the app). | |||
On the External Storages page, click on the '''Add storage''' button and select SMB / CIFS from the dropdown menu. | |||
In the configuration form that appears, enter the details of your Samba share, including the hostname or IP address of the server, share name, username, and password. You can also specify a subfolder within the share if you only want to mount a specific subfolder. | |||
Once you have entered all of the required information, click on the Save button to save your configuration. | |||
== [https://apps.nextcloud.com/ Nextcloud Apps] == | == [https://apps.nextcloud.com/ Nextcloud Apps] == | ||
* [https://docs.nextcloud.com/server/15/admin_manual/installation/installation_wizard.html#trusted-domains "access through untrusted domain" error]. On my Casaos, I found the file ''/DATA/AppData/nextcloud/var/www/html/config/config.php'' shows trusted domain is casaos.local:10081. So I just need to change the app's setting in CasaOS to use http://casaos.local:10081 instead of http://IP:10081 to open the page. | |||
* [https://apps.nextcloud.com/apps/audioplayer Audio Player]. When I play mp3 files, the web interface does not show a progress bar:( <syntaxhighlight lang='bash'> | * [https://apps.nextcloud.com/apps/audioplayer Audio Player]. When I play mp3 files, the web interface does not show a progress bar:( <syntaxhighlight lang='bash'> | ||
$ docker run --rm --name mynextcloud -v ~/Downloads/nextcloudapps:/var/www/html/custom_apps -p 81:80 nextcloud | $ docker run --rm --name mynextcloud -v ~/Downloads/nextcloudapps:/var/www/html/custom_apps -p 81:80 nextcloud | ||
Line 220: | Line 262: | ||
* [https://itsfoss.com/deploy-seafile-server-docker/ How to Deploy Seafile Server with Docker to Host Your Own File Synchronization and Sharing Solution] | * [https://itsfoss.com/deploy-seafile-server-docker/ How to Deploy Seafile Server with Docker to Host Your Own File Synchronization and Sharing Solution] | ||
= Run a Linux desktop VM with VNC = | = Run a Linux desktop = | ||
== Run a Linux desktop VM with VNC == | |||
[https://computingforgeeks.com/run-ubuntu-linux-in-docker-with-desktop-environment-and-vnc/ Run Ubuntu Linux in Docker with Desktop Environment and VNC] | [https://computingforgeeks.com/run-ubuntu-linux-in-docker-with-desktop-environment-and-vnc/ Run Ubuntu Linux in Docker with Desktop Environment and VNC] | ||
== Distrobox == | |||
* https://github.com/89luca89/distrobox | |||
* [https://www.howtogeek.com/run-any-app-on-any-linux-distro-with-distrobox/ DEB Files on Arch? Run Any App on Any Linux Distro With This Tool] | |||
* [https://youtu.be/Sj064D9ZUl8 Stop Distro Hopping! Use this AWESOME tool on Linux] 7/15/2023 | |||
** Linux subsystem in another Linux | |||
= Guacamole = | = Guacamole = | ||
Line 237: | Line 287: | ||
== Webtop: Linux desktop in a browser == | == Webtop: Linux desktop in a browser == | ||
* https://hub.docker.com/r/linuxserver/webtop/. Ubuntu, Alpine, Arch, and Fedora based Webtop images, Linux in a web browser supporting popular desktop environments. | * https://hub.docker.com/r/linuxserver/webtop/, or https://github.com/linuxserver/docker-webtop. Ubuntu, Alpine, Arch, and Fedora based Webtop images, Linux in a web browser supporting popular desktop environments. | ||
* An example. It seems the host's Disk space, Memory are all passed to VM. In this example, the host's directory "/tmp/downloads" will serve as the HOME directory in VM. | |||
:<syntaxhighlight lang='sh'> | |||
mkdir /tmp/downloads/ | |||
docker run -d --name=webtop \ | |||
-e PUID=1000 -e PGID=1000 \ | |||
-e TZ=US/Eastern \ | |||
-p 3000:3000 \ | |||
-v /tmp/downloads:/config \ | |||
--shm-size="1gb" \ | |||
--restart unless-stopped \ | |||
linuxserver/webtop:ubuntu-xfce | |||
</syntaxhighlight> | |||
* [https://www.linuxserver.io/blog/2021-05-05-meet-webtops-a-linux-desktop-environment-in-your-browser Meet Webtops A Linux Desktop Environment In Your Browser] | * [https://www.linuxserver.io/blog/2021-05-05-meet-webtops-a-linux-desktop-environment-in-your-browser Meet Webtops A Linux Desktop Environment In Your Browser] | ||
* [https://youtu.be/bQrtnZ1UrQQ Run Linux Desktops in your Browser with Docker and Webtops!] (video) and [https:// | * [https://youtu.be/bQrtnZ1UrQQ Run Linux Desktops in your Browser with Docker and Webtops!] (video) and [https://wiki.opensourceisawesome.com/books/webtops-for-remote-sessions/page/webtops-linux-desktops-in-docker Notes] | ||
* Default username/password is abc/abc | * Default username/password is abc/abc. But we don't need to enter the password when we use "sudo". | ||
* [https://youtu.be/Gd9bvdkIXOQ Linux desktop, inside of a container, inside of a browser??? Yes. A Webtop], [https://youtu.be/Gd9bvdkIXOQ?t=811 privileged: true] is required for installing something like vscode in VM (video) and [https://docs.technotim.live/posts/webtop-container/ Doc] | * [https://youtu.be/Gd9bvdkIXOQ Linux desktop, inside of a container, inside of a browser??? Yes. A Webtop], [https://youtu.be/Gd9bvdkIXOQ?t=811 privileged: true] is required for installing something like vscode in VM (video) and [https://docs.technotim.live/posts/webtop-container/ Doc] | ||
* If I'm not using the default desktop, I need to switch to use some user's account instead of root to run docker-compose command. If I use root to launch ubuntu-kde, for example, the desktop is blank. | * If I'm not using the default desktop, I need to switch to use some user's account instead of root to run docker-compose command. If I use root to launch ubuntu-kde, for example, the desktop is blank. | ||
Line 275: | Line 337: | ||
https://hub.docker.com/r/linuxserver/cloud9 | https://hub.docker.com/r/linuxserver/cloud9 | ||
== | == VS Code == | ||
* [https://github.com/coder/code-server code-server] VS Code in the browser. https://hub.docker.com/r/codercom/code-server | * [https://github.com/coder/code-server code-server] VS Code in the browser. https://hub.docker.com/r/codercom/code-server | ||
* [https://www.reddit.com/r/docker/comments/waebx5/development_container_with_visual_studio_code_how/ Development container with Visual Studio code - how does it work?] | * [https://www.reddit.com/r/docker/comments/waebx5/development_container_with_visual_studio_code_how/ Development container with Visual Studio code - how does it work?] | ||
Line 284: | Line 346: | ||
** Add & Start the Template. | ** Add & Start the Template. | ||
** For the VS-code template, I can launch a Terminal and work on it (Same RAM as my machine, 6GB storage of Debian image). | ** For the VS-code template, I can launch a Terminal and work on it (Same RAM as my machine, 6GB storage of Debian image). | ||
* [https://hub.docker.com/r/gitpod/openvscode-server OpenVSCode Server] 1M+ pulls. [https://github.com/coder/code-server/discussions/4267 Difference to OpenVSCode Server]. | |||
= | = Docker for R (r-base, rstudio) = | ||
See [[R_Docker|R Docker]]. | |||
[ | |||
= Docker for Shiny Server = | = Docker for Shiny Server = | ||
Line 788: | Line 355: | ||
<li>[https://hosting.analythium.io/user-2022-best-practicesfor-shiny-apps-with-docker-and-more/ UseR!2022: Best Practices for Shiny Apps with Docker and More] | <li>[https://hosting.analythium.io/user-2022-best-practicesfor-shiny-apps-with-docker-and-more/ UseR!2022: Best Practices for Shiny Apps with Docker and More] | ||
<li>[https://hosting.analythium.io/containerizing-interactive-r-markdown-documents/ Containerizing Interactive R Markdown Documents] 2022/7 | <li>[https://hosting.analythium.io/containerizing-interactive-r-markdown-documents/ Containerizing Interactive R Markdown Documents] 2022/7 | ||
<li>By default, Shiny Server looks for either: | |||
* A single-file Shiny app named '''app.R''' in the root of a directory. | |||
* A pair of files named '''ui.R''' and '''server.R''' within a directory. | |||
<li>[https://datawookie.dev/blog/2022/09/using-shiny-server-in-docker/ Using Shiny Server in Docker] 2022/9/7. Hello world example. Need to generate ''Dockerfile''. | <li>[https://datawookie.dev/blog/2022/09/using-shiny-server-in-docker/ Using Shiny Server in Docker] 2022/9/7. Hello world example. Need to generate ''Dockerfile''. | ||
<pre> | <pre> | ||
Line 921: | Line 491: | ||
* Best proxy - traefik, NPM, SWAG, caddy or other? | * Best proxy - traefik, NPM, SWAG, caddy or other? | ||
* [https://github.com/linuxserver/docker-swag swag] | * [https://github.com/linuxserver/docker-swag swag] | ||
== Nginx proxy manager == | == Nginx proxy manager == | ||
* [https://youtu.be/cjJVmAI1Do4 Putting it All Together - Docker, Docker-Compose, NGinx Proxy Manager, and Domain Routing - How To] (video) and [https://shownotes.opensourceisawesome.com/putting-it-all-together/ Notes] 2020-10-06 | * [https://youtu.be/cjJVmAI1Do4 Putting it All Together - Docker, Docker-Compose, NGinx Proxy Manager, and Domain Routing - How To] (video) and [https://shownotes.opensourceisawesome.com/putting-it-all-together/ Notes] 2020-10-06 | ||
* [https://youtu.be/UfCkwlPIozw Securing NGinX Proxy Manager - follow up - securing your admin console for this Open Source Software] (video) and [https://shownotes.opensourceisawesome.com/securing-nginx-proxy-manager/ Notes] 2020-10-20 | * [https://youtu.be/UfCkwlPIozw Securing NGinX Proxy Manager - follow up - securing your admin console for this Open Source Software] (video) and [https://shownotes.opensourceisawesome.com/securing-nginx-proxy-manager/ Notes] 2020-10-20 | ||
== Caddy == | |||
* [https://github.com/rocker-org/rocker/issues?q=caddy Rocker], [https://www.rocker-project.org/use/networking/ Networking: tips and tricks] from Rocker | |||
* [https://youtu.be/tv4A39yDDXs?t=526 Pi-Hosted : Reverse Proxy with Caddy]. We can play Caddy using IP address. | |||
* [https://linuxiac.com/how-to-set-up-caddy-as-reverse-proxy/ How to Set up Caddy as a Reverse Proxy] | |||
= Website analysis = | |||
* [https://www.howtoforge.com/how-to-install-plausible-analytics-with-docker-on-ubuntu-22-04/ How to Install Plausible Analytics with Docker on Ubuntu 22.04] | |||
= Nginx = | = Nginx = | ||
Line 1,338: | Line 913: | ||
https://github.com/solidnerd/docker-bookstack | https://github.com/solidnerd/docker-bookstack | ||
= HDHomeRun = | |||
== tvheadend == | |||
<ul> | |||
<li>docker-compose.yml. See [https://docs.linuxserver.io/images/docker-tvheadend/ linuxserver/tvheadend]. Open http://localhost:9981 . | |||
<pre> | |||
services: | |||
tvheadend: | |||
container_name: tvheadend | |||
environment: | |||
- PUID=1000 | |||
- PGID=1000 | |||
- TZ=Etc/UTC | |||
network_mode: host | |||
volumes: | |||
- ./data:/config | |||
- ./recordings:/recordings | |||
restart: unless-stopped | |||
image: lscr.io/linuxserver/tvheadend:latest | |||
</pre> | |||
<li>tvheadend setup. [https://www.youtube.com/watch?v=6Dgjn1IKrFM TVheadend and HDHomeRun using IPTV inputs]. | |||
<li>[https://www.youtube.com/watch?v=2Y-E4sQSb94 How to configure tvheadend server]. | |||
<li>[https://www.youtube.com/watch?v=Y4gnJRgMB9s How to install Tvheadend server on $40 Android box running Linux]. The configuration allows to scan channels on HDHomeRun. | |||
</ul> | |||
= Media server = | = Media server = | ||
Line 1,349: | Line 948: | ||
* [https://jellyfin.org/downloads/clients/ Official clients] | * [https://jellyfin.org/downloads/clients/ Official clients] | ||
* [https://youtu.be/aY732cSkopU ULTIMATE Jellyfin Guide - Hardware Acceleration, Codecs (PART 2) ] | * [https://youtu.be/aY732cSkopU ULTIMATE Jellyfin Guide - Hardware Acceleration, Codecs (PART 2) ] | ||
* Problems: | |||
** [https://www.reddit.com/r/jellyfin/comments/rf0taf/best_way_to_update_your_library_when_you_make/ Best way to update your library when you make changes to existing files?]. Answer: (manual) Settings - Dashboard - Libraries - Libraries - Scan all libraries, (automatic) scheduled tasks - scan media library (every 12 hour). | |||
** My (downloaded) videos got different video titles. | |||
== Youtube downloader == | == Youtube downloader == | ||
Line 1,361: | Line 963: | ||
# http://localhost:8081 | # http://localhost:8081 | ||
</pre> | </pre> | ||
The following example value for "YTDL_OPTIONS" embeds English subtitles and chapter markers (for videos that have them), and also changes the permissions on the downloaded video and sets the file modification timestamp to the date of when it was downloaded | |||
{{Pre}} | |||
docker run -d -p 8081:8081 \ | |||
-v /tmp/downloads:/downloads \ | |||
-e YTDL_OPTIONS='{"writesubtitles":true,"subtitleslangs":["en","-live_chat"],"updatetime":false,"postprocessors":[{"key":"Exec","exec_cmd":"chmod 0664","when":"after_move"},{"key":"FFmpegEmbedSubtitle","already_have_subtitle":false},{"key":"FFmpegMetadata","add_chapters":true}]}' \ | |||
--name metube \ | |||
--user 1000:1001 alexta69/metube | |||
</pre> | |||
<li>Chapters and subtitles: | |||
* Testing video: [https://www.youtube.com/watch?v=yG8sFwBISVU Macha Galette]. | |||
* In VLC, subtitle is available under "Subtitle" -> "Sub Track" -> "English". The subtitle can be extracted from the downloaded webm files using <syntaxhighlight lang='sh' inline>ffmpeg -i filename.webm -map 0:s:0? subs.srt</syntaxhighlight>. If a video's subtitle is auto-generated by Youtube, then there is no subtitle embedded in the downloaded webm files. | |||
* In VLC, chapters is available under "Playback" -> "Chapter" | |||
<li>[https://github.com/yt-dlp/yt-dlp yt-dlp] | |||
<li>[https://github.com/Tzahi12345/YoutubeDL-Material YoutubeDL-Material] | <li>[https://github.com/Tzahi12345/YoutubeDL-Material YoutubeDL-Material] | ||
</ul> | </ul> | ||
Line 1,373: | Line 988: | ||
== Plex == | == Plex == | ||
[https://github.com/plexinc/pms-docker Official Docker container for Plex Media Server] | * [https://github.com/plexinc/pms-docker Official Docker container for Plex Media Server] | ||
* https://hub.docker.com/r/linuxserver/plex/ | |||
* Live tv subtitles/captions: Server -> Settings -> Languages -> Subtitle mode -> Always enabled -> Save Changes. | |||
* [https://www.plex.tv/plex-pass/ Plex Pass] subscription | |||
** [https://support.plex.tv/articles/225877347-live-tv-dvr/ Live tv DVR] | |||
** [https://support.plex.tv/articles/115002178853-using-hardware-accelerated-streaming/ Using Hardware-Accelerated Streaming] | |||
<pre> | |||
--- | |||
services: | |||
plex: | |||
image: lscr.io/linuxserver/plex:latest | |||
container_name: plex | |||
network_mode: host | |||
environment: | |||
- PUID=1000 | |||
- PGID=1000 | |||
- TZ=US/Eastern | |||
- VERSION=docker | |||
- PLEX_CLAIM= #optional | |||
volumes: | |||
- /tmp/plex/library:/config | |||
- /tmp/plex/tvseries:/tv | |||
devices: | |||
- /dev/dri:/dev/dri | |||
restart: unless-stopped | |||
</pre> | |||
== Sonarr, Radarr, Prowlarr == | |||
* | * [https://www.youtube.com/watch?v=LD8-Qr3B2-o Make Automated Torrent Media Server with Emby, Sonarr, Radarr, Prowlarr, and qBittorrent on Windows!] | ||
* [https://www.youtube.com/watch?v=zfYthZrDpVU How to install and configure Sonarr] | |||
= DNS server = | = DNS server = | ||
Line 1,386: | Line 1,027: | ||
* https://github.com/IAmStoxe/wirehole | * https://github.com/IAmStoxe/wirehole | ||
* [https://youtu.be/DOJ39lyx6Js WireHole: WireGuard, Pi-Hole and Unbound in Docker] (video) | * [https://youtu.be/DOJ39lyx6Js WireHole: WireGuard, Pi-Hole and Unbound in Docker] (video) | ||
= Speed test = | |||
* https://myspeed.dev/, http://localhost:5216. I need to open the page in incognito mode when using SSH local forwarding. | |||
* [https://www.youtube.com/watch?v=tBJmhgn3ZOM MySpeed: Speedtest Automation Made Simple!] | |||
= Pi-hole = | = Pi-hole = | ||
Line 1,440: | Line 1,085: | ||
* [https://github.com/jupyter/docker-stacks Jupyter Docker Stacks], | * [https://github.com/jupyter/docker-stacks Jupyter Docker Stacks], | ||
* https://registry.hub.docker.com/r/jupyter/datascience-notebook/ | * https://registry.hub.docker.com/r/jupyter/datascience-notebook/ | ||
* [https://youtu.be/UXxUcZDSNwA How to customize the jupyter/datascience-notebook container with docker-compose]. | * [https://youtu.be/UXxUcZDSNwA How to customize the jupyter/datascience-notebook container with docker-compose]. Change the $PWD below for your case. | ||
<pre> | <pre> | ||
docker run --rm -p 8888:8888 \ | docker run --rm -p 8888:8888 \ | ||
-e JUPYTER_ENABLE_LAB=yes \ | -e JUPYTER_ENABLE_LAB=yes \ | ||
-v "$PWD":/home/jovyan/work \ | -v "$PWD":/home/jovyan/work \ | ||
jupyter/datascience-notebook:r-4. | --name jupyter \ | ||
jupyter/datascience-notebook:r-4.3.1 | |||
</pre> | </pre> | ||
Follow the screen output to access the app by going to http://IP:8888/?token=XXXXXX. This will take care of the authentication problem. | Follow the screen output to access the app by going to http://IP:8888/?token=XXXXXX. This will take care of the authentication problem. | ||
If we use 'docker compose up -d' method below, we need to use 'docker logs jupyter' to get the URL which includes the token password. | |||
<pre> | |||
version: '3.3' | |||
services: | |||
datascience-notebook: | |||
ports: | |||
- '8888:8888' | |||
environment: | |||
- JUPYTER_ENABLE_LAB=yes | |||
volumes: | |||
- '"$PWD":/home/jovyan/work' | |||
container_name: jupyter | |||
restart: unless-stopped | |||
image: 'jupyter/datascience-notebook:r-4.3.1' | |||
</pre> | |||
This includes python3, Julia 1.0.0 and R. However, the R version is pretty old if we don't specify the tag. | This includes python3, Julia 1.0.0 and R. However, the R version is pretty old if we don't specify the tag. | ||
Line 1,471: | Line 1,133: | ||
* https://hub.docker.com/r/freshrss/freshrss/ | * https://hub.docker.com/r/freshrss/freshrss/ | ||
* [https://hub.docker.com/r/linuxserver/freshrss/ FreshRSS] | * [https://hub.docker.com/r/linuxserver/freshrss/ FreshRSS] | ||
= Read it later/Pocket = | |||
[https://github.com/omnivore-app/omnivore Omnivore] | |||
= Home Assistant = | = Home Assistant = | ||
Line 1,512: | Line 1,177: | ||
= Draw.io for diagrams and more = | = Draw.io for diagrams and more = | ||
https://github.com/jgraph/drawio/tree/master/etc/docker | <ul> | ||
<li>https://github.com/jgraph/drawio/tree/master/etc/docker | |||
<pre> | <pre> | ||
docker run -d --restart=unless-stopped --rm --name="draw" -p 8080:8080 -p 8443:8443 jgraph/drawio | docker run -d --restart=unless-stopped \ | ||
--rm --name="draw" -p 8080:8080 \ | |||
-p 8443:8443 jgraph/drawio | |||
</pre> | </pre> | ||
It | <li>To allow to save to devices, the app and the file have to be on the same machine. | ||
<li>It is super easy to install draw.io in Nextcloud (just a few clicks). It is very natural to save the file in the Nextcloud storage. | |||
</ul> | |||
= Internet Archive = | = Internet Archive = | ||
Line 1,587: | Line 1,257: | ||
** [https://youtu.be/SmUDirCSaoE ChangeDetection.io with Browser Agent Support in Docker] | ** [https://youtu.be/SmUDirCSaoE ChangeDetection.io with Browser Agent Support in Docker] | ||
* [https://www.makeuseof.com/best-website-change-detection-tools/ The 7 Best Free and Paid Website Change Detection Tools] | * [https://www.makeuseof.com/best-website-change-detection-tools/ The 7 Best Free and Paid Website Change Detection Tools] | ||
= CasaOS = | |||
* https://casaos.io/, [https://wiki.casaos.io/en/guides Wiki] (Very limited) | |||
* https://blog.casaos.io/ Blog and new release announcement | |||
* [https://docs.zimaboard.com/docs/Self-Deploying-Applications.html Self-Deploying-Applications] from Zimaboard. | |||
* I installed CasaOS in Proxmox using Debian 12 LXC. I allocated 40GB space and 4GB ram. The memory can be increased without a need to shut down the LXC. | |||
* Files app is similar to NextCloud but it can connect to cloud storage like Google Drive, Dropbox and OneDrive. It can also connect to other network storage something like smb://192.168.1.1 | |||
* I guess we are supposed to use CasaOS in a full screen model. The backward button in the browser will possibly quit CasaOS. | |||
* Portainer, Uptime Kuma, Wireguard Easy are some of apps in the App Store. | |||
* CasaOS can expand their App Store. | |||
** Custom installer is another option; we can use Docker Compose or Docker CLI. | |||
** "Add Source". [https://awesome.casaos.io/content/3rd-party-app-stores/list.html Store list]. After I copy/paste CasaOS-LinuxServer-AppStore '''https://casaos-appstore.paodayag.dev/linuxserver.zip''' and click "Add +" button it immediately shows 226 apps. That is cool! It includes Firefox, Chromium, Opera. I installed FF but when I use the web interface of FF on my Android tablet, I could not scroll. FreshRSS not working on my tablets. Firefox and Chromium are hard to use on my tablet. | |||
** Apps are using ports other than 80. Clicking on an app (eg Snapdrop was changed to use port 88 instead of 80) sometimes opened a tab with a port 80. So the launching is not successfully. We need to click on each app's Settings and find out what the port it uses and modify the port on the Web UI field and 'Save'. | |||
** Kasm not working. | |||
** Snapdrop. I can't see other devices? [https://docs.linuxserver.io/images/docker-pairdrop/ pairdrop] from linuxserver.io works. | |||
** OpenSpeedTest & librespeed/speedtest works but gave quite different results than speedtest.net | |||
** [https://www.helpwire.app/blog/rdesktop-linux/ Rdesktop]. It is [https://youtu.be/jt_C3cStfAA?si=Mp-4UpV30Gn3lKVw&t=22 adding a desktop environment to your server]. So the IP should be IP:3389 when we use Remmina for example. The username/password is abc/abc. For example, I allocate 40GB to CasaOS and the remote desktop will show 40GB disk space for the server. The desktop includes the Chromium browser. We can connect to internet using the Chromium browser. | |||
** [https://shownotes.opensourceisawesome.com/webtops/ Webtop] is based on KasmVNC. Webtops is a project where you can setup a variety of Linux desktops, with a variety of Desktop Environments in Docker, and access them through a web browser! The screen/desktop is synchronized across devices. So we can use our tablet device as a monitor and our mouse and keyboard are still connected to our desktop computer. Interesting! But no internet connection? The font size on the desktop changes with the connected devices. The default distro is Arch Linux. | |||
** AdGuard: when I installed it, '''the default port for dashboard is 80'''. I changed it to 3000. But after the setup, the port 3000 is not working. Check the setting from CasaOS, it shows 3001. So the change of port in the initial set up was not taken (I ssh into the server and use '''ss''' to check open ports) though admin account was saved. The port 3001 is actually working. It works when I tested on a Android tablet (use only AdGuard IP as the only DNS, not including 1.1.1.1 or else). See [https://youtu.be/u9ylq5Gry_A?si=THSqnh84_VtdFhLv Adguard Home Installed on Docker] video for AdGuard setup. | |||
** ddclient, Duckdns, Duplicati, [https://github.com/healthchecks/healthchecks healthchecks] (A watchdog for your cron jobs). | |||
* [https://icewhale.community/t/casaos-tailscale-seamless-home-network-integration-remote-access-to-your-data/84 CasaOS+Tailscale: Seamless Home Network Integration + Remote access to your data] |
Revision as of 16:03, 3 September 2024
Homelab
- Self hosting
- What are your Most Used Self Hosted Applications? and more
- Best Docker Containers for Home Server! 2022/11 (video)
Top/best
- https://hub.docker.com/
- Awesome Docker
- LinuxServer.io. Need to sort by downloads.
- Reddit what are your must docker containers for home lab?
- Docker top (18) images and solutions
- 60+ Best Docker Containers for Home Server Beginners 2023
Templates
App store
Tipi
Dockerizing Compiled Software
- https://ram.tianon.xyz/post/2017/12/26/dockerize-compiled-software.html
- How to Containerize a Legacy Application
Crontab
How to Use Cron With Your Docker Containers
Install macOS
- Install macOS Big Sur Or Catalina In A Virtual Machine Using Docker-OSX
- This is Real!! MacOS Docker 2022/12/2
- How to Create a macOS Virtual Machine Using Docker
Raspberry Pi servers based on Docker
- https://github.com/pi-hosted/pi-hosted
- Raspberry Pi Server based on Docker, with VPN, Dropbox backup, Influx, Grafana, etc.
Data Science
Illumina BaseSpace
- http://blog.basespace.illumina.com/2013/10/21/introducing-the-native-app-engine/
- https://developer.basespace.illumina.com/docs/content/documentation/getting-started/overview
- https://developer.basespace.illumina.com/docs/content/documentation/native-apps/manage-docker-image
Bioinformatics analyses
- Hot-starting software containers for bioinformatics analyses
- Bio-Docklets: virtualization containers for single-step execution of NGS pipelines
- Performance and scaling behavior of bioinformatic applications in virtualization environments to create awareness for the efficient use of compute resources Hanussek, 2021
- docker-builds - Dockerfiles and documentation on tools for public health bioinformatics
- ACTIONet and the paper Transcriptional vulnerabilities of striatal neurons in human and rodent models of Huntington's disease 2023.
- CREDO: a friendly Customizable, REproducible, DOcker file generator for bioinformatics applications 2024
Cloud provider
Docker can be deployed on any cloud provider like Digital Ocean, Amazon Web Services (AWS), Microsoft Azure, Google Cloud Computing.
- https://www.digitalocean.com/community/tutorials/how-to-use-the-digitalocean-docker-application
- http://go.mkb.io/post/118463195802/docker-machine-with-digital-ocean
- http://networkstatic.net/running-docker-machine-on-digital-ocean/
- http://linoxide.com/linux-how-to/use-docker-machine-cloud-provider/
- HTC採用AWS的三大原因: 上市時間縮短、節省成本以及提供創新發想的低成本空間。
Dashboard
Dashy
- https://github.com/Lissy93/dashy
- Dashy, a configurable, themable, flexible personal dashboard for your homelab! (video) and Notes
- Icons are located in ~/dash/icons/dashboard-icons/png directory.
- Download an icon from icons8.com or uxwing.com. The default 48x48 works fine.
- Better to work on one icon and then save the change. Repeat.
docker run -d \ -p 8295:80 \ --volume /home/$USER/dashy/public/conf.yml:/app/public/conf.yml \ --volume /home/$USER/dashy/icons:/app/public/item-icons/icons \ --name dashy \ --restart=unless-stopped \ lissy93/dashy:latest
Glance
Homepage
https://github.com/gethomepage/homepage
Flame
https://github.com/pawelmalak/flame
Homer
- https://github.com/bastienwirtz/homer
- Homer Dashboard - Open Source, Self Hosted, Beautiful Shortcuts to all your sites and services, This Homelab Dashboard is simple and beautiful! (videos)
Uptime Kuma
heimdall
https://github.com/linuxserver/docker-heimdall
Dashboard for monitoring system: grafana
- Docker Dashboard Using Grafana, Prometheus & Node Exporter
- Another good tutorial: Server Monitoring // Prometheus and Grafana Tutorial 2021
- When I try to create a new stack in Portainer, the error message is not clear.
- When I try it on a LXC container in Proxmox, I got an error ERROR: for node-exporter Cannot start service node-exporter: path / is mounted on / but it is not a shared or slave mount. This is the same error as some users reported.
- It works when I try a regular container in Proxmox!
- Node exporter. The docker run command works if I run it in a regular Ubuntu system. You can check it is working by going to IP:9100.
- https://prometheus.io/docs/guides/node-exporter/
- curl http://localhost:9100/metrics works but using IP does not. So it is a firewall problem.
- How to Monitor Linux Servers with Prometheus and Node Exporter. sudo ufw allow 9090/tcp & sudo ufw allow 9100/tcp & sudo ufw reload
- Use IP:9100 to check if Node Exporter works. Use IP:9090 to check if Prometheus works (Status -> Targets). Use IP:3457 to access the Grafana where 3457 is the one I used in the docker-compose.yml (the default one is 3000).
- Use admin/admin to log in. Add data source. Select Prometheus. Add the IP address, e.g., http://IP:9090.
- Google: node exporter grafana dashboards. https://grafana.com/grafana/dashboards/. E.g. 11074 (1 Node Exporter for Prometheus Dashboard).
After I change the firewall by adding two ports (9090 & 9100), the docker-compose method of grafana works like a charm.
- Reboot the host OS will not affect the grafana (assuming we include restart: unless-stopped in the 3 containers). But if we modify docker-compose.yml and run docker-compose down; docker-compose up it will result in data lose and we will need to re-create the admin password and all the setup.
Google photo
Immich: Google photo replacment
- immich - High performance self-hosted photo and video backup solution
- Immich Is a Great Google Photos Alternative You Can Host on Raspberry Pi
- Immich – Finally, A Good Self-Hosted Google Photos Replacement
ownCloud
- https://hub.docker.com/_/owncloud/
- https://doc.owncloud.com/server/10.0/admin_manual/installation/docker/
- https://blog.securem.eu/serverside/2015/08/25/setting-up-owncloud-server-in-a-docker-container/
- How to put ownCloud behind Nginx with Docker?
- Migrating ownCloud to Docker
- "You are accessing the server from an untrusted domain" message when I use IP to access the server
- trusted_domains (config.php)
- https://forum.owncloud.org/viewtopic.php?t=34158
- https://blog.securem.eu/serverside/2015/08/25/setting-up-owncloud-server-in-a-docker-container/
- Fix ownCloud accessing the server from an untrusted domain
- My simple fix: use the IP instead of localhost when opening the browser to set up the first time
After I use docker run --rm --name myowncloud -p 81:80 owncloud:8.1, I find I can only access it through http://localhost:81. If I try to access it through another computer by http://xxx.xxx.xx.1:81, the URL is changed back to http://localhost:81 so it failed to load.
nextcloud
- https://hub.docker.com/_/nextcloud/
- https://github.com/nextcloud/docker
- App store
- Installing and Managing Apps
- Based on owncloud's github repositories or nextcloud's github repositories, android app has been updated more often than ios app.
- Setting Up NextCloud on Docker w/ NGINX and CloudFlare for Remote Access! (video)
- How to Install Nextcloud on Proxmox
Nextcloud looks more modern than owncloud though their interface are very similar. I got the same problem (URL is changed from IP to localhost) when I try to access its web from another computer. Using the IP instead of localhost to finish the initial setup can fix the error.
Note:
- The first we open the webpage, we need to create the admin username/password.
- Nextcloud uses owncloud mobile application.
- Nextcloud includes a small mp4 file. I can use browser to play mp4. Photos files can be opened in browsers too (including slideshow).
Add external storage like SAMBA
To connect to a Samba drive in Nextcloud, you will need to use the External Storage app. This app allows you to mount external storage services, including Samba shares, as directories within your Nextcloud instance. Here are the steps to connect to a Samba drive in Nextcloud:
First, make sure that the External Storage app is installed and enabled on your Nextcloud instance. You can do this by going to the Apps section in the Nextcloud admin panel and searching for the External Storage app. If it is not installed, you can install it from there.
Once the External Storage app is installed and enabled, go to the Settings section in the Nextcloud admin panel and select the External Storages option under the Administration section (Left panel, scroll down to see the app).
On the External Storages page, click on the Add storage button and select SMB / CIFS from the dropdown menu.
In the configuration form that appears, enter the details of your Samba share, including the hostname or IP address of the server, share name, username, and password. You can also specify a subfolder within the share if you only want to mount a specific subfolder.
Once you have entered all of the required information, click on the Save button to save your configuration.
Nextcloud Apps
- "access through untrusted domain" error. On my Casaos, I found the file /DATA/AppData/nextcloud/var/www/html/config/config.php shows trusted domain is casaos.local:10081. So I just need to change the app's setting in CasaOS to use http://casaos.local:10081 instead of http://IP:10081 to open the page.
- Audio Player. When I play mp3 files, the web interface does not show a progress bar:(
$ docker run --rm --name mynextcloud -v ~/Downloads/nextcloudapps:/var/www/html/custom_apps -p 81:80 nextcloud # above will create an empty folder ~/Downloads/nextcloudapps (www-data:root) $ tar xzvf Downloads/audioplayer-2.3.1.tar.gz -C Downloads/ $ docker cp Downloads/audioplayer mynextcloud:/var/www/html/custom_apps # Go to nextcloud and '+Apps', a new app 'Audio Player' should have shown up.
- Music. GOOD. The app is from the owncloud team though for some reason the app is not included in owncloud/nextcloud. Yes, a progress bar is shown when I played a song downloaded from Youtube:)
# No need to create a volume $ docker run -d --name mycloud -p 81:80 nextcloud # Open another terminal # Don't download the zip file from github. Get the zip file from the app store $ tar xzvf ~/Downloads/music_0.7.0_nc-signed.tar.gz -C ~/Downloads/ $ docker cp Downloads/music mycloud:/var/www/html/custom_apps $ docker exec mycloud chmod -R 0750 /var/www/html/custom_apps $ docker exec mycloud chown -R www-data:www-data /var/www/html/custom_apps
Now open the website. Click the gear icon (Settings) and click "Apps". Find out the 'music' app from the bottom of the list and click the 'Enable' button. A new Music icon will be created next to the Gallery icon at the top. Note: if we want to play music repeatedly, I should not play the music directly from Files. I should click the Music icon at the top and let it start to scan musics. Then once I can play a music from there, it will show two new icons; one is shuffle and the other is loop. To have a persistent data, we can 'run' using$ mkdir -p ~/nextcloud/{html,apps,config,data} $ docker run -d \ --name mycloud \ -p 81:80 \ -v ~/nextcloud/html:/var/www/html \ -v ~/nextcloud/apps:/var/www/html/custom_apps \ -v ~/nextcloud/config:/var/www/html/config \ -v ~/nextcloud/data:/var/www/html/data \ nextcloud
If we like to delete the contain and its unamed volume, run$ docker stop mycloud; docker rm -v mycloud
If we run the container again mounting all volumes, all apps,files,configuration,admin's username & password are kept. - PDF viewer. PDF files will be downloaded instead of opening on the browser. This can be easily fixed. Click '+Apps' from the personal settings (the response is slow when I tested using Docker). Search for PDF viewer. Click the 'Enable' button (it will be changed to 'Disable' after that)
- draw.io. After enable it, we can click the "+" sign and choose "Diagram" to create a new diagram (*.xml).
- Radio. After enable it, a new "Radio" icon will be shown at the top. Only 20 stations are available from the 'Top'. No response after I clicked "Categories".
Navidrome Is the Perfect Self-Hosted Music Server for Raspberry Pi
FileRun
- https://www.filerun.com/download
- How to Install FileRun using Docker
- Running Filerun Storage Sync Server in Docker Container. You can access, create and edit your files using Filerun via the web interface or the desktop sync app like NextCloud.
- Filerun vs NextCloud?
File manager
File browser
- https://filebrowser.org/installation/, https://github.com/filebrowser/filebrowser
- Pi-Hosted : Installing JDownloader and File Browser On The Pi Docker Server Part 4 (video)
The original image (filebrowser/filebrowser) does not work but linuxserver's image (filebrowser/filebrowser:s6) works fine.
$ touch filebrowser.db $ cat settings.json { "port": 80, "baseURL": "", "address": "", "log": "stdout", "database": "/database/filebrowser.db", "root": "/srv" } # Modify $FULL_PATH1 <- directory to share files # & $FULL_PATH2 <- directory contains filebrowser.db & settings.json $ docker run \ --name filebrowser \ -v $FULL_PATH1:/srv \ -v $FULL_PATH2/filebrowser.db:/database/filebrowser.db \ -v $FULL_PATH2/settings.json:/config/settings.json \ -e PUID=$(id -u) \ -e PGID=$(id -g) \ -p 8080:80 \ filebrowser/filebrowser:s6
FilGator
Seafile
- https://en.wikipedia.org/wiki/Seafile. Seafile's functionality is similar to other popular file hosting services such as Dropbox and Google Drive.
- How to Deploy Seafile Server with Docker to Host Your Own File Synchronization and Sharing Solution
Run a Linux desktop
Run a Linux desktop VM with VNC
Run Ubuntu Linux in Docker with Desktop Environment and VNC
Distrobox
- https://github.com/89luca89/distrobox
- DEB Files on Arch? Run Any App on Any Linux Distro With This Tool
- Stop Distro Hopping! Use this AWESOME tool on Linux 7/15/2023
- Linux subsystem in another Linux
Guacamole
- Guacamole is nothing more than a centralized front end to access multiple VNC/RDP devices. You are still using either VNC or RDP to access the device.
- Probably there is no point to use Guacamole if the VM was created in Proxmox since Proxmox provides a web interface to access the VM.
- Guacamole Remote Desktop - Open Source, Self Hosted remote access to your machines in the browser! (video). https://github.com/jwetzell/docker-guacamole
- One line script to install Docker-CE, Docker-Compose, Portainer-CE (not needed), and NGinX Proxy manager (not needed)
- Once creating docker-compose.yml, we can fire up it using docker-compose up -d
- We can create a new admin user and delete the default username/password guacadmin/guacadmin
- Open a browser tab and go to the IP:Port to open guacamole web page
- We can add various connections (Kubenetes/RDP/SSH/Telnet/VNC). After that, we can run connections through our web browser.
- After saving the new connection, go back to "Home" to use the new connections.
- Guacamole Remote Access Gateway on Docker (video). https://github.com/boschkundendienst/guacamole-docker-compose works fine. Your guacamole server should now be available at https://ip:8443/. The default username is guacadmin with password guacadmin. Four containers will be launched: nginx, postgres, guacamole and guacd.
- How To Access Your PCs and Servers from Anywhere Using Guacamole and CloudFlare Tunnels
Webtop: Linux desktop in a browser
- https://hub.docker.com/r/linuxserver/webtop/, or https://github.com/linuxserver/docker-webtop. Ubuntu, Alpine, Arch, and Fedora based Webtop images, Linux in a web browser supporting popular desktop environments.
- An example. It seems the host's Disk space, Memory are all passed to VM. In this example, the host's directory "/tmp/downloads" will serve as the HOME directory in VM.
mkdir /tmp/downloads/ docker run -d --name=webtop \ -e PUID=1000 -e PGID=1000 \ -e TZ=US/Eastern \ -p 3000:3000 \ -v /tmp/downloads:/config \ --shm-size="1gb" \ --restart unless-stopped \ linuxserver/webtop:ubuntu-xfce
- Meet Webtops A Linux Desktop Environment In Your Browser
- Run Linux Desktops in your Browser with Docker and Webtops! (video) and Notes
- Default username/password is abc/abc. But we don't need to enter the password when we use "sudo".
- Linux desktop, inside of a container, inside of a browser??? Yes. A Webtop, privileged: true is required for installing something like vscode in VM (video) and Doc
- If I'm not using the default desktop, I need to switch to use some user's account instead of root to run docker-compose command. If I use root to launch ubuntu-kde, for example, the desktop is blank.
- It's cool. I got Ubuntu 22.04 released 1.5 months ago.
- After I installed 'htop', the htop command shows the cpu & memory & uptime ... from the 'host' system. But docker stats shows webtop (ubuntu-mate) desktop is using around 300MB memory.
- When I want to copy text from VM to remote client, I need to open the clipboard using the black circle tool on the left-hand side.
- After I played around, KDE-konsole does not respond to my keyboard. Logged out and logged in again fixes the problem.
- I got the following errors when I try to install RStudio 2022.02.3 Build 492 on Ubuntu 22.04; this solution Installing R Studio on Ubuntu 22.04 works for me.
$ sudo apt install /config/Downloads/rstudio-2022.02.3-492-amd64.deb the following packages have unmet dependencies: rstudio : Depends: libssl1.0.0 but it is not installable or libssl1.0.2 but it is not installable or libssl1.1 but it is not installable
- Since the config subfolder is mounted, if we want to try different flavors of Linux/Desktop, we should create different folders.
- From scratch From Start to Finish Build a Desktop on a Browser Docker Container
instantbox
Firewall/router/Vlan
- OPNSense - a powerful, open source, network firewall and router 4/19/2022
- Building a Business - Ep. 2: Installing OPNSense or pfSense as our Firewall and Router. 9/4/2022
- *Building a Business - Ep 3 - VLAN Setup in pfSense and OPNSense four our segmented network 9/11/2022
IDE
cloud9
https://hub.docker.com/r/linuxserver/cloud9
VS Code
- code-server VS Code in the browser. https://hub.docker.com/r/codercom/code-server
- Development container with Visual Studio code - how does it work?
- Coder - Your Self-Hosted Remote Development Platform
- Instruction step 1 is necessary
- Workspace - Template
- After issuing docker run command, I need to go to the URL, create username/password. The username is for the app login purpose.
- Add & Start the Template.
- For the VS-code template, I can launch a Terminal and work on it (Same RAM as my machine, 6GB storage of Debian image).
- OpenVSCode Server 1M+ pulls. Difference to OpenVSCode Server.
Docker for R (r-base, rstudio)
See R Docker.
Docker for Shiny Server
- UseR!2022: Best Practices for Shiny Apps with Docker and More
- Containerizing Interactive R Markdown Documents 2022/7
- By default, Shiny Server looks for either:
- A single-file Shiny app named app.R in the root of a directory.
- A pair of files named ui.R and server.R within a directory.
- Using Shiny Server in Docker 2022/9/7. Hello world example. Need to generate Dockerfile.
FROM rocker/shiny:4.1.0 RUN rm -rf /srv/shiny-server/* WORKDIR /srv/shiny-server/ COPY ./app.R ./app.R
docker build -t shiny-in-docker . docker run --rm -p 3838:3838 shiny-in-docker
- https://hub.docker.com/r/rocker/shiny/. Pay attention to the Dockerfile there.
# sudo mkdir /srv/ sudo mkdir /srv/shinylog sudo mkdir /srv/shinyapps cd /srv/shinyapps sudo wget https://raw.githubusercontent.com/rstudio/shiny-examples/master/050-kmeans-example/ui.R sudo wget https://raw.githubusercontent.com/rstudio/shiny-examples/master/050-kmeans-example/server.R sudo lsof -i -P -n | grep LISTEN # nmap localhost docker run -d --restart always -p 3838:3838 \ -v /srv/shinyapps/:/srv/shiny-server/ \ -v /srv/shinylog/:/var/log/shiny-server/ \ rocker/shiny:4.0.3 nmap localhost
If the 3838 port method (http://IP:3838) works, we can move on to use Nginx + reverse proxy method; see Apache->R/Shiny example for the instruction.
- Shiny Administrator Guide
- Docker Reference. See Engine (docker) CLI from their LHS.
- Dockering a shiny app
- Dockerizing Shiny Applications
- Pin package versions in your production Docker image. Need to modify Dockerfile to install 'shiny' along with 'renv'. We need to create a directory 'shinyapp' with simple shiny files (eg kmeans example).
docker build -t pinrpackageversion . mkdir shinyapp wget https://raw.githubusercontent.com/rstudio/shiny-examples/master/050-kmeans-example/ui.R -P shinyapp wget https://raw.githubusercontent.com/rstudio/shiny-examples/master/050-kmeans-example/server.R -P shinyapp docker run --name shinyapp --rm -p 3838:3838 pinrpackageversion
- Question: what happened if there is a new version of R or shiny-sever?
# specify --name so we don't need to use container id to access it later $ docker run -d --name rshiny -p 81:3838 rocker/shiny $ docker ps -l # Run in container's bash # https://stackoverflow.com/questions/30172605/how-to-get-into-a-docker-container # non-interactive $ docker exec -it rshiny ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 22: eth0@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.17.0.2/16 scope global eth0 valid_lft forever preferred_lft forever # interactive shell $ docker exec -it rshiny bash root@5b78988d5619:/# who root@5b78988d5619:/# which R /usr/bin/R root@5b78988d5619:/# ls /home docker shiny root@5b78988d5619:/# ls /srv/shiny-server 01_hello 02_text 03_reactivity 04_mpg 05_sliders 06_tabsets 07_widgets 08_html 09_upload 10_download 11_timer index.html sample-apps root@5b78988d5619:/# R -e 'package.Description("glmnet")' root@5b78988d5619:/# exit $ docker stop rshiny $ docker rm rshiny # remove the container
It is also useful to use the -v option in docker run so the local directory (eg /srv/shinyapps/) can be mapped to the docker volume (/srv/shiny-server).
# Put your shiny app in host's /srv/shinyapps/appdir, $ docker run -d -p 81:3838 \ -v /srv/shinyapps/:/srv/shiny-server/ \ -v /srv/shinylog/:/var/log/shiny-server/ \ rocker/shiny
And check running status
$ docker ps -a
R packages installation: packrat
- https://cran.r-project.org/web/packages/packrat/index.html
- Installing R packages to a Docker image with packrat
- How to install packages through `packrat` while creating shiny Docker image
Check opened ports
See What network ports are open: lsof or netstat.
$ netstat -ant Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN ... tcp6 0 0 :::80 :::* LISTEN tcp6 0 0 :::81 :::* LISTEN tcp6 0 0 :::22 :::* LISTEN tcp6 0 0 ::1:25 :::* LISTEN
Blog Explorer
https://github.com/nz-stefan/blog-explorer, Winners of the 2nd Annual Shiny Contest
Raspberry Pi
The Raspberry Pi 4B as a shiny server
BioContainers
- https://biocontainers.pro/#/
- https://hub.docker.com/u/biocontainers/
- https://github.com/BioContainers
CMS, blog website
Ghost
Proxy
- Best proxy - traefik, NPM, SWAG, caddy or other?
- swag
Nginx proxy manager
- Putting it All Together - Docker, Docker-Compose, NGinx Proxy Manager, and Domain Routing - How To (video) and Notes 2020-10-06
- Securing NGinX Proxy Manager - follow up - securing your admin console for this Open Source Software (video) and Notes 2020-10-20
Caddy
- Rocker, Networking: tips and tricks from Rocker
- Pi-Hosted : Reverse Proxy with Caddy. We can play Caddy using IP address.
- How to Set up Caddy as a Reverse Proxy
Website analysis
Nginx
- Using the NGINX Open Source Docker Image
docker run -d --name web \ -p 8080:80 \ nginx curl http://localhost:8080 docker exec -it NGINX_container_ID bash
- Note nginx docker does not work if there is not index.html or index.php (it shows 403 Forbidden). Google: How to serve static file with nginx
- How To Use the Official NGINX Docker Image
docker run -it --rm -d \ -p 8080:80 \ --name web \ -v ~/Downloads/html:/usr/share/nginx/html \ nginx docker stop web # when finished
- How to Run NGINX Inside Docker (for Easy Auto-Scaling)
- How To Run Nginx in a Docker Container on Ubuntu 14.04. It teaches to put the whole web site in the host directory and the nginx configuration in the local host. You can edit the default file locally (to serve a new directory), or to use a proxy_pass to forward the traffic to another app/container like you would with a regular Nginx installation).
- How To Containerize and Use Nginx as a Proxy
- TIPS FOR DEPLOYING NGINX (OFFICIAL IMAGE) WITH DOCKER
- Docker 快速入门
- Deploying NGINX and NGINX Plus with Docker
- How to Install and Use Docker on Linux Reverse-proxy, LetsEncrypt ... built in.
docker run --name docker-nginx -p 80:80 nginx docker run --name my-nginx \ -v /host/path/nginx.conf:/etc/nginx/nginx.conf:ro \ -v /some/html:/usr/share/nginx/html:ro \ -p 8080:80 \ -d nginx
httpd/apache
https://hub.docker.com/_/httpd/
- How to Setup a Simple Apache Web Server in a Docker Container
- Docker – An apache ‘hello world’ example
- Note nginx docker does not work if there is not index.html or index.php (it shows 403 Forbidden); httpd image does not have this problem.
docker run -d --rm\ --name apache \ -p 8080:80 \ -v /home/brb/Videos/:/usr/local/apache2/htdocs/ \ httpd
Tomcat
https://hub.docker.com/_/tomcat/
ASP.NET
Forum
How to Install Discourse Forum with Docker on CentOS 8
Jenkins for continuous integration
- https://www.tutorialspoint.com/jenkins/jenkins_overview.htm
- https://jenkins.io/doc/book/installing/
- Example: https://dev.ropensci.org/
MySQL
- https://hub.docker.com/r/mysql/mysql-server/
- Deploying MySQL on Linux with Docker
- Note it seems impossible to use the 'grant' command interactively within the MySQL docker container.
- https://dzone.com/articles/using-docker-to-create-a-mysql-server
docker run --detach --name test-mysql --env="MYSQL_ROOT_PASSWORD=mypassword" mysql docker inspect test-mysql | grep IPAddress # 172.17.0.2 # https://dev.mysql.com/doc/index-other.html wget -O - http://downloads.mysql.com/docs/world.sql.gz | gunzip -c > world.sql mysql -uroot -pmypassword -h 172.17.0.2 -e 'CREATE DATABASE world' cat world.sql | mysql -uroot -pmypassword -h 172.17.0.2 world # zcat world.sql.gz | mysql -uroot -pmypassword -h 172.17.0.2 world mysql -uroot -pmypassword -h 172.17.0.2 -e 'USE world; SHOW TABLES' mysql -uroot -pmypassword -h 172.17.0.2 -e 'DROP DATABASE world'
# Let's see if we can access the database in R system("sudo apt install libmysqlclient-dev") library(RMySQL) con <- dbConnect(MySQL(), "world", user="root", password="mypassword", host="172.17.0.2") dbListTables(con) head(dbReadTable(con, "city")) head(dbReadTable(con, "country")) head(dbReadTable(con, "countrylanguage")) dbWriteTable(con, 'iris', iris) # Error in .local(conn, statement, ...) : # could not run statement: The used command is not allowed with this MySQL version dbDisconnect(con)
- How To Create And Test MySQL Database Server in Docker On Ubuntu Linux (video). Use Dockerfile.
- MySQL Docker Containers: Understanding the basics, https://youtu.be/qJQPckRYNis?t=16m29s
# Fire up docker run --detach --name=test-mysql --env="MYSQL_ROOT_PASSWORD=mypassword" mysql docker ps -a # Connecting to the Container docker inspect test-mysql | grep IPAddress # sudo apt install mysql-client # There are two methods to connect to the container # method 1. https://hub.docker.com/r/mysql/mysql-server/ # docker exec -it test-mysql mysql -uroot -p # method 2 mysql -uroot -pmypassword -h 172.17.0.20 -P 3306 docker run --detach --name test-wordpress --link test-mysql:mysql wordpress docker exec -it test-wordpress bash $ cat /etc/hosts # Expose the MySQL container to the outside world # Access the MySQL container directly from the machine’s port 6603 docker run --detach --name=test-mysql --env="MYSQL_ROOT_PASSWORD=mypassword" --publish 6603:3306 mysql mysql -u root -pmypassword -h 127.0.0.1 -P 6603 # Configuration management mkdir -p /root/container/test-mysql/conf.d nano /root/container/test-mysql/conf.d/my-custom.cnf docker run \ --detach \ --name=test-mysql \ --env="MYSQL_ROOT_PASSWORD=mypassword" \ --publish 6603:3306 \ --volume=/root/docker/test-mysql/conf.d:/etc/mysql/conf.d \ mysql mysql -uroot -pmypassword -h127.0.0.1 -P6603 -e 'show global variables like "max_connections"'; docker run \ --detach \ --name=test-mysql \ --env="MYSQL_ROOT_PASSWORD=mypassword" \ --publish 6603:3306 \ mysql \ --max-connections=200 \ --character-set-server=utf8mb4 \ --collation-server=utf8mb4_unicode_ci # Data Storage mkdir -p /storage/docker/mysql-datadir docker run \ --detach \ --name=test-mysql \ --env="MYSQL_ROOT_PASSWORD=mypassword" \ --publish 6603:3306 \ --volume=/root/docker/test-mysql/conf.d:/etc/mysql/conf.d \ --volume=/storage/docker/mysql-datadir:/var/lib/mysql \ mysql docker inspect test-mysql ls -al /storage/docker/mysql-datadir/ # If a MySQL container runs on top of an existing MySQL datadir, # the $MYSQL_ROOT_PASSWORD variable should be omitted from the “run” command line; docker stop test-mysql docker rm -f test-mysql docker run -d --name=another-new-mysql -p 6605:3306 -v /storage/docker/mysql-datadir:/var/lib/mysql mysql docker logs another-new-mysql
- Use Docker to run a typical MySQL server from Code Review Videos
docker volume create crv_mysql docker run \ -e MYSQL_ROOT_PASSWORD=my-secret-pw \ -e MYSQL_DATABASE=devdb \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ --mount type=volume,src=crv_mysql,dst=/var/lib/mysql \ -p 3306:3306 \ -d \ mysql:latest # Alternatively we can put username/password in a text file for security cat > .env <<EOF MYSQL_ROOT_PASSWORD=my-secret-pw MYSQL_DATABASE=devdb MYSQL_USER=dbuser MYSQL_PASSWORD=dbpassword EOF docker run \ --env-file .env \ --mount type=volume,src=crv_mysql,dst=/var/lib/mysql \ -p 3306:3306 \ -d \ mysql:latest
Backup and restore
# Backup docker exec CONTAINER /usr/bin/mysqldump -u root --password=root DATABASE > backup.sql # Restore # Method 1. docker exec + command docker exec -i CONTAINER /usr/bin/mysql -u root --password=root < backup.sql # Method 2. docker exec + bash # We can also use docker exec log into the contain and run any mysql commands there docker exec -it CONTAINER bash # Method 3. mysql command only # We directly use mysql command to import database without using docker commands # as long as we know the IP address of the container server docker inspect CONTAINER | grep IPAddress mysql -uroot -pPASSWORD -h 172.18.0.3 -e 'show databases' mysql -uroot -pPASSWORD -h 172.18.0.3 < backup.sql
PostgreSQL
Jekyll
Building Jekyll Sites within VS Code Dev Containers
node.js
Dockerizing Node.js. See the code here.
$ docker build -t my-node-app . $ docker run -p 3000:3000 my-node-app
How To Setup Your Local Node.js Development Environment Using Docker (not working)
$ cd notes-service $ cat Dockerfile FROM node:12.18.1 WORKDIR /code COPY package.json /code/package.json COPY package-lock.json /code/package-lock.json RUN npm install COPY . /code CMD [ "node", "server.js" ] $ docker build -t notes-service . $ docker run -it --rm -d -v mongodb:/data/db -v mongodb_config:/data/configdb -p 27017:27017 --network mongodb --name mongodb mongo $ docker run -it --rm -d --network mongodb --name notes -p 8081:8081 -e SERVER_PORT=8081 -e DATABASE_CONNECTIONSTRING=mongodb://mongodb:27017/yoda_notes notes-service # this does not work though no errors; omit "-d" to see the error message $ curl --request POST \ --url http://localhost:8081/services/m/notes \ --header 'content-type: application/json' \ --data '{ "name”: "this is a note", "text": "this is a note that I wanted to take while I was working on writing a blog post.", "owner": "peter" }'
React
Redis
https://hub.docker.com/_/redis/
Redis running inside Docker container on NVIDIA Jetson Nano, Redis is ideal for IoT and Embedded devices for several reasons.
Laravel
Dockerizing Laravel with Nginx MySQL and Docker Compose on Ubuntu 18.04 LTS
phpMyAdmin
- https://hub.docker.com/r/phpmyadmin/phpmyadmin/. The docker-compose.yml file works instantly.
- phpMyAdmin and MySQL server
docker run --name mysqlphp -e MYSQL_ROOT_PASSWORD=0000 -d mysql docker run --name myadmin -d --link mysqlphp:db -p 8080:80 phpmyadmin/phpmyadmin
Now go to your browser , tap http://localhost:8080 and login with root/0000.
ERROR mysqli_real_connect(): The server requested authentication method unknown to the client [caching_sha2_password]
The solution can be found here. After the change, I can log in phpMyAdmin.
docker exec -it mysqlphp bash mysql -u root -p0000 ALTER USER root IDENTIFIED WITH mysql_native_password BY '0000'; exit exit
MagicMirror
https://github.com/MichMich/MagicMirror/blob/master/README.md#docker
Mail/smtp server
- How to Set Up Your Own Secure Email Server 2021-11
- Setup Your Own Email Server with Docker
- Sending Emails with {curl} and Docker
Mediawiki
- https://www.mediawiki.org/wiki/Docker
- https://hub.docker.com/_/mediawiki/ or (better) https://docs.docker.com/samples/library/mediawiki/. The Dockerfile is based on jessie.
- bitnami/mediawiki (Verified but too complicated, 1M+)
- https://hub.docker.com/r/ubcctlt/mediawiki (10M+)
- Hosting Your Own Wiki With MediaWiki and Docker Nov 2020. The instruction is easy to follow.
- In mediawiki set up we page, we need to enter 'database' for the Database host
- The database information we entered will go to 'LocalSettings.php' file. But the name of wiki, administrator user name, password will go to the database.
- the instruction for the 'extensions' part seems too much. We just need to mount local 'extensions' to container's '/var/www/extensions' folder.
- Remember to run 'docker-compose down && docker-compose up -d' after we modify something
- docker搭建mediawiki
- https://github.com/kristophjunge/docker-mediawiki
- Setup MediaWiki and Postgres with Docker (based on simplyintricate/mediawiki, works on Mint). Visit http://localhost:8080 to see the result.
$ ls -l ~/wiki_root total 24 drwxr-xr-x 3 root root 4096 Jul 10 10:09 data drwxrwxr-x 3 brb brb 4096 Jul 10 09:23 db drwxrwxr-x 2 www-data www-data 4096 Jul 10 09:23 extensions drwxrwxr-x 2 www-data www-data 4096 Jul 10 09:23 images -rw-rw-r-- 1 www-data www-data 4152 Jul 10 11:12 LocalSettings.php
- Running Mediawiki as Docker container (based on synctree/mediawiki which is not updated anymore, works tested on GalliumOS)
- Follow Run MySql inside docker container first. 1. Setup host to store configuration, data and dumps. The 'dumps' and 'initdb.d' folders are optional 2. Override default mysql configuration 3. Executing scripts and sql on container startup (optional) 4. Starting mysql container
- Visit http://IP_address:8080 to see the mediawiki page running on Docker
brb@galliumos:~$ tree mysql mysql ├── conf.d │ └── mysql.cnf ├── data │ ├── auto.cnf │ ├── ca-key.pem │ ├── ca.pem │ ├── client-cert.pem │ ├── client-key.pem │ ├── ib_buffer_pool │ ├── ibdata1 │ ├── ib_logfile0 │ ├── ib_logfile1 │ ├── ibtmp1 │ ├── mediawiki [error opening dir] │ ├── mysql [error opening dir] │ ├── performance_schema [error opening dir] │ ├── private_key.pem │ ├── public_key.pem │ ├── server-cert.pem │ ├── server-key.pem │ ├── sys [error opening dir] │ └── wikidb [error opening dir] ├── dumps └── initdb.d 9 directories, 15 files brb@galliumos:~$ tree -L 2 mediawiki/ mediawiki/ ├── config │ └── LocalSettings.php ├── extensions │ ├── Cite │ ├── ConfirmEdit │ ├── Gadgets │ ├── ImageMap │ ├── InputBox │ ├── Interwiki │ ├── LocalisationUpdate │ ├── Nuke │ ├── ParserFunctions │ ├── PdfHandler │ ├── Poem │ ├── README │ ├── Renameuser │ ├── SpamBlacklist │ ├── SyntaxHighlight_GeSHi │ ├── TitleBlacklist │ └── WikiEditor └── images └── README 19 directories, 3 files
- https://linuxconfig.org/mediawiki-easy-deployment-with-docker-container (based on linuxconfig/mediawiki)
- http://collabnix.com/how-to-setup-mediawiki-in-seconds-using-docker/ (based on appcontainers/mediawiki)
- Isolated MediaWiki Setup on the Raspberry Pi using Docker, Part 1
BookStack
BookStack is a simple, self-hosted, easy-to-use platform for organising and storing information.
https://github.com/solidnerd/docker-bookstack
HDHomeRun
tvheadend
- docker-compose.yml. See linuxserver/tvheadend. Open http://localhost:9981 .
services: tvheadend: container_name: tvheadend environment: - PUID=1000 - PGID=1000 - TZ=Etc/UTC network_mode: host volumes: - ./data:/config - ./recordings:/recordings restart: unless-stopped image: lscr.io/linuxserver/tvheadend:latest
- tvheadend setup. TVheadend and HDHomeRun using IPTV inputs.
- How to configure tvheadend server.
- How to install Tvheadend server on $40 Android box running Linux. The configuration allows to scan channels on HDHomeRun.
Media server
Jellyfin
- https://hub.docker.com/r/jellyfin/jellyfin
- Jellyfin Update - An amazing, open source, and self hosted media server alternative to Plex or Emby (video)
- https://docs.linuxserver.io/images/docker-jellyfin (works)
- Official clients
- ULTIMATE Jellyfin Guide - Hardware Acceleration, Codecs (PART 2)
- Problems:
- Best way to update your library when you make changes to existing files?. Answer: (manual) Settings - Dashboard - Libraries - Libraries - Scan all libraries, (automatic) scheduled tasks - scan media library (every 12 hour).
- My (downloaded) videos got different video titles.
Youtube downloader
- Download YouTube videos, host locally?
- MeTube (simple but no thumbnails, deleted files seem not deleted from the disk/deleted files from disks are still on the list; no sync),
mkdir /tmp/downloads # O.W. /tmp/downloads generated by root cannot be written docker run -d -p 8081:8081 \ -v /tmp/downloads:/downloads \ --user 1000:1001 alexta69/metube # http://localhost:8081
The following example value for "YTDL_OPTIONS" embeds English subtitles and chapter markers (for videos that have them), and also changes the permissions on the downloaded video and sets the file modification timestamp to the date of when it was downloaded
docker run -d -p 8081:8081 \ -v /tmp/downloads:/downloads \ -e YTDL_OPTIONS='{"writesubtitles":true,"subtitleslangs":["en","-live_chat"],"updatetime":false,"postprocessors":[{"key":"Exec","exec_cmd":"chmod 0664","when":"after_move"},{"key":"FFmpegEmbedSubtitle","already_have_subtitle":false},{"key":"FFmpegMetadata","add_chapters":true}]}' \ --name metube \ --user 1000:1001 alexta69/metube
- Chapters and subtitles:
- Testing video: Macha Galette.
- In VLC, subtitle is available under "Subtitle" -> "Sub Track" -> "English". The subtitle can be extracted from the downloaded webm files using
ffmpeg -i filename.webm -map 0:s:0? subs.srt
. If a video's subtitle is auto-generated by Youtube, then there is no subtitle embedded in the downloaded webm files. - In VLC, chapters is available under "Playback" -> "Chapter"
- yt-dlp
- YoutubeDL-Material
Airsonic media server
https://airsonic.github.io/download/
My comment:
- Added mp3/mp4 files are not recognized.
- Cannot play by web browsers
- Special clients are needed:(
Plex
- Official Docker container for Plex Media Server
- https://hub.docker.com/r/linuxserver/plex/
- Live tv subtitles/captions: Server -> Settings -> Languages -> Subtitle mode -> Always enabled -> Save Changes.
- Plex Pass subscription
--- services: plex: image: lscr.io/linuxserver/plex:latest container_name: plex network_mode: host environment: - PUID=1000 - PGID=1000 - TZ=US/Eastern - VERSION=docker - PLEX_CLAIM= #optional volumes: - /tmp/plex/library:/config - /tmp/plex/tvseries:/tv devices: - /dev/dri:/dev/dri restart: unless-stopped
Sonarr, Radarr, Prowlarr
- Make Automated Torrent Media Server with Emby, Sonarr, Radarr, Prowlarr, and qBittorrent on Windows!
- How to install and configure Sonarr
DNS server
How to Deploy a Dynamic DNS Server with Docker on Debian 10
WireHole
Speed test
- https://myspeed.dev/, http://localhost:5216. I need to open the page in incognito mode when using SSH local forwarding.
- MySpeed: Speedtest Automation Made Simple!
Pi-hole
- https://hub.docker.com/r/pihole/pihole/, https://github.com/pi-hole/docker-pi-hole
- Raspberry Pi Pi-Hole: A Network Wide Ad-blocker
- Install Pi-hole on a Raspberry Pi with Docker and Portainer
- Running Pi-Hole as a Docker Container. Ubuntu.
- (Ubuntu) The command to disable the Systemd-Resolve Service (Network Name Resolution manager) seems to be outdated. sudo systemctl stop systemd-resolve returns a message Failed to stop systemd-resolve.service: Unit systemd-resolve.service not loaded.
- It looks sudo systemctl status systemd-resolved still works in Ubuntu 22.04
- Before deploying pi-hole, run the following commands (Ubuntu 22.04, Docker, Portainer, Pi-Hole):
sudo systemctl stop systemd-resolved sudo systemctl disable systemd-resolved
- How to install Pi-hole on Docker – Network-wide Ad Blocking
sudo docker volume create etc-pihole sudo docker volume create etc-dnsmasq.d sudo docker run -d --name pihole \ -p 53:53/tcp -p 53:53/udp -p 67:67 -p 80:80 -p 443:443 \ -e TZ="Asia/Kolkata" \ -v etc-pihole:/etc/pihole/ \ -v etc-dnsmasq.d:/etc/dnsmasq.d/ \ --dns=127.0.0.1 --dns=1.1.1.1 --restart=unless-stopped \ --hostname pi.hole \ -e VIRTUAL_HOST="pi.hole" \ -e ServerIP="127.0.0.1" \ pihole/pihole:latest
Authelia
- https://hub.docker.com/r/authelia/authelia Authelia is an open-source authentication and authorization server providing two-factor authentication and single sign-on (SSO) for your applications via a web portal
- Authelia - Free, Open Source, Self Hosted authorization and authentication for your web applications and Notes
Whoogle
https://github.com/benbusby/whoogle-search
Backup
Duplicati
- https://hub.docker.com/r/duplicati/duplicati/
- Backup Your Docker Server NOW! - Setup and Configure Duplicati (video)
Python
- Containerized Python Development – Part 1,
- Part 2
- Managing Project Configuration with Docker Compose
- Compose file
- Best practices for writing Compose files
- How Developers Can Get Started with Python and Docker
Python Jupyter including R
- Jupyter Docker Stacks,
- https://registry.hub.docker.com/r/jupyter/datascience-notebook/
- How to customize the jupyter/datascience-notebook container with docker-compose. Change the $PWD below for your case.
docker run --rm -p 8888:8888 \ -e JUPYTER_ENABLE_LAB=yes \ -v "$PWD":/home/jovyan/work \ --name jupyter \ jupyter/datascience-notebook:r-4.3.1
Follow the screen output to access the app by going to http://IP:8888/?token=XXXXXX. This will take care of the authentication problem.
If we use 'docker compose up -d' method below, we need to use 'docker logs jupyter' to get the URL which includes the token password.
version: '3.3' services: datascience-notebook: ports: - '8888:8888' environment: - JUPYTER_ENABLE_LAB=yes volumes: - '"$PWD":/home/jovyan/work' container_name: jupyter restart: unless-stopped image: 'jupyter/datascience-notebook:r-4.3.1'
This includes python3, Julia 1.0.0 and R. However, the R version is pretty old if we don't specify the tag.
Also the method does not work on arm64. WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested.
Keyboard shortcuts:
- ctrl+Enter to run.
- alt+Enter to run & insert a cell below.
- shift+Enter to run and shift to a cell below.
- Esc: command mode
- Enter: edit mode
Note by default the ipynb files are saved to /home/jovyan instead of /home/jovyan/work directory. So it's better to click the "work" icon on the LHS of the file panel in order to change the working directory to "work".
Calibre
- https://hub.docker.com/r/linuxserver/calibre. It works on x86 and it can be run as a service so we don't need to keep the app open like the Desktop version.
- See the Dockerfile and docker-baseimage-guacgui and Apache Guacamole.
- The ARM variants can be built on x86_64 hardware using multiarch/qemu-user-static. See https://github.com/multiarch/qemu-user-static.
- Calibre-Web is a web app providing a clean interface for browsing, reading and downloading eBooks using a valid Calibre database.
RSS
Read it later/Pocket
Home Assistant
- Install Proxmox and virtualize Home Assistant
- How to Build a $10 Wireless IP Security Camera for Your Home
Password manager
Vaultwarden
Vaultwarden - Open Source, self hosted backend for the Bitwarden password manager (video)
KeeWeb: password manager compatible with KeePass
https://github.com/keeweb/keeweb
Padloc: cloud-based password manager
- Padloc is a cloud-based password management solution that allows you to access and use your saved passwords across multiple platforms.
- How to Install Padloc Password Manager on Rocky Linux 8
Flask
Dockerizing Flask Application Using Docker on Debian 10
ONLYOFFICE
- https://www.howtoforge.com/tutorial/how-to-install-onlyoffice-8.9.0-with-docker-on-ubuntu/
- https://www.howtoforge.com/tutorial/how-to-update-onlyoffice-to-version-95-with-docker/ This involves backing up and restoring MYSQL data.
Gitlab
- https://docs.gitlab.com/omnibus/docker/
- https://hub.docker.com/r/gitlab/gitlab-ce/
- How to Install Gitlab Server with Docker on Ubuntu 18.04 LTS
- How to Install GitLab with Docker on Ubuntu 20.04 LTS
NAS & Docker
http://www.ithome.com.tw/tech/96500
Some examples
- GetLab專案管理工具
- Jenkins開發版本管理工具
Memo/note
Draw.io for diagrams and more
- https://github.com/jgraph/drawio/tree/master/etc/docker
docker run -d --restart=unless-stopped \ --rm --name="draw" -p 8080:8080 \ -p 8443:8443 jgraph/drawio
- To allow to save to devices, the app and the file have to be on the same machine.
- It is super easy to install draw.io in Nextcloud (just a few clicks). It is very natural to save the file in the Nextcloud storage.
Internet Archive
How To Self-host Your Own Internet Archive With ArchiveBox In Linux
Run a benchmark program on ARM devices
- Beaglebone black as an example. More device examples can be found on http://www.testing-software.org/.
Cheat.sh
https://github.com/chubin/cheat.sh#docker
Set up a Mastodon Instance on Linux
- https://joinmastodon.org/
- Your Own Social Network: How to Set Up a Mastodon Instance on Linux
- 3 Amazing Mastodon Clients for you to use to access a truly open social media platform & Fediverse
- What is Mastodon? What to know about the decentralized site some see as a Twitter alternative
- Should I join Mastodon? A scientists’ guide to Twitter’s rival
Bioinformatics tools (Tophat, BWA, et al)
https://github.com/dceoy/docker-bio
cbioportal
- Docker
- Deploy with Docker
- https://github.com/cBioPortal/cbioportal-docker-compose includes Connect to the database, Run different cBioPortal version, Change the heap size.
- https://github.com/cBioPortal/cbioportal
- https://groups.google.com/g/cbioportal
- docker-compose up is OK, but it requires me to restart the container (I wasn't instructed to do that. Documents need to improve).
- Import study does not work if we use the docker-compose method. It works if we use docker exec -it CONTAINER bash.
[brb@localhost cbioportal-docker-compose]$ docker-compose run cbioportal metaImport.py -u http://cbioportal:8080 -s study/lgg_ucsf_2014/ -o Creating cbioportal-docker-compose_cbioportal_run ... done Starting validation... CRITICAL: -: No meta files found in study/lgg_ucsf_2014/. Please make sure the directory is the path to the folder containing the files. ERROR: -: Cancer type needs to be defined for a study. Verify that you have a study file and have defined the cancer type correctly. ERROR: -: No valid study file detected
See also https://github.com/cBioPortal/cbioportal/issues/7972
- The version number shown on http://localhost:8080 is v3.4.17 (see .env variable) but the https://www.cbioportal.org/ website shows v3.5.3 (same as the current release version on Github & Docker hub)
- The 2nd field on the 1st row of data_cancer_type.txt contains the study name (blue color) shown on website
- meta_study.txt contains the study name (black color) shown on website
- study_es_0 (breast, contains mRNA expression) and study_es_3 can be imported successfully when I use docker-compose exec -it CONTAINER bash and follow Load sample cancer study.
- lgg_ucsf_2014 study (brain, CNA) should use the correct path docker-compose run cbioportal metaImport.py -u http://cbioportal:8080 -s study/lgg_ucsf_2014/lgg_ucsf_2014/ -o
- I have tested on CentOS 7 and Ubuntu 20.04 on KVM/QEMU w/ 2 cores, 2/4GB memory, 25GB storage. I can use host's terminal and browser for interaction (i.e. no VNC/Spice server needed).
- Gene panels in the cBioPortal patient view 2020-01-07, and other blogs related to cbioportal from thehyve.
GIVE - Genomic Interactive Visualization Engine
GeneTEFlow: Analysing gene and transposable elements expression from RNA-Seq data
GeneTEFlow: A Nextflow-based pipeline for analysing gene and transposable elements expression from RNA-Seq data. Source code.
Elastic Search
- https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
- https://docs.docker.com/samples/library/elasticsearch/
- https://github.com/triestpa/guttenberg-search
Webpage monitor tools
- statping
- changedetection.io, releases version v0.39.18
- The 7 Best Free and Paid Website Change Detection Tools
CasaOS
- https://casaos.io/, Wiki (Very limited)
- https://blog.casaos.io/ Blog and new release announcement
- Self-Deploying-Applications from Zimaboard.
- I installed CasaOS in Proxmox using Debian 12 LXC. I allocated 40GB space and 4GB ram. The memory can be increased without a need to shut down the LXC.
- Files app is similar to NextCloud but it can connect to cloud storage like Google Drive, Dropbox and OneDrive. It can also connect to other network storage something like smb://192.168.1.1
- I guess we are supposed to use CasaOS in a full screen model. The backward button in the browser will possibly quit CasaOS.
- Portainer, Uptime Kuma, Wireguard Easy are some of apps in the App Store.
- CasaOS can expand their App Store.
- Custom installer is another option; we can use Docker Compose or Docker CLI.
- "Add Source". Store list. After I copy/paste CasaOS-LinuxServer-AppStore https://casaos-appstore.paodayag.dev/linuxserver.zip and click "Add +" button it immediately shows 226 apps. That is cool! It includes Firefox, Chromium, Opera. I installed FF but when I use the web interface of FF on my Android tablet, I could not scroll. FreshRSS not working on my tablets. Firefox and Chromium are hard to use on my tablet.
- Apps are using ports other than 80. Clicking on an app (eg Snapdrop was changed to use port 88 instead of 80) sometimes opened a tab with a port 80. So the launching is not successfully. We need to click on each app's Settings and find out what the port it uses and modify the port on the Web UI field and 'Save'.
- Kasm not working.
- Snapdrop. I can't see other devices? pairdrop from linuxserver.io works.
- OpenSpeedTest & librespeed/speedtest works but gave quite different results than speedtest.net
- Rdesktop. It is adding a desktop environment to your server. So the IP should be IP:3389 when we use Remmina for example. The username/password is abc/abc. For example, I allocate 40GB to CasaOS and the remote desktop will show 40GB disk space for the server. The desktop includes the Chromium browser. We can connect to internet using the Chromium browser.
- Webtop is based on KasmVNC. Webtops is a project where you can setup a variety of Linux desktops, with a variety of Desktop Environments in Docker, and access them through a web browser! The screen/desktop is synchronized across devices. So we can use our tablet device as a monitor and our mouse and keyboard are still connected to our desktop computer. Interesting! But no internet connection? The font size on the desktop changes with the connected devices. The default distro is Arch Linux.
- AdGuard: when I installed it, the default port for dashboard is 80. I changed it to 3000. But after the setup, the port 3000 is not working. Check the setting from CasaOS, it shows 3001. So the change of port in the initial set up was not taken (I ssh into the server and use ss to check open ports) though admin account was saved. The port 3001 is actually working. It works when I tested on a Android tablet (use only AdGuard IP as the only DNS, not including 1.1.1.1 or else). See Adguard Home Installed on Docker video for AdGuard setup.
- ddclient, Duckdns, Duplicati, healthchecks (A watchdog for your cron jobs).
- CasaOS+Tailscale: Seamless Home Network Integration + Remote access to your data