NAS: Difference between revisions

From 太極
Jump to navigation Jump to search
 
(6 intermediate revisions by the same user not shown)
Line 49: Line 49:
* Method2: Storage - File systems
* Method2: Storage - File systems


== Hard drives ==
== Hard drives spindown ==
Click Storage -> Disks -> Select the disk -> Edit. Choose '''spindown time''' from "Disabled" to "20 minutes'.
<ul>
<li>Click Storage -> Disks -> Select the disk -> Edit. Choose '''spindown time''' from "Disabled" to "20 minutes'.
<li>After the spindown, we can use the command
<syntaxhighlight lang='sh'>
smartctl -i -n standby /dev/sdX
</syntaxhighlight> to check the status of the disk. In my case, the output shows "Device is in SLEEP mode, exit(2)". However, if I run the command again immediately, it will display a complete information about the device and the power mode is ACTIVE or IDLE.
</ul>


== Multiple SATA drives ==
== Multiple SATA drives ==
Line 56: Line 62:


== Sharing ==
== Sharing ==
* '''It's generally recommended to have a separate disk for share data in OMV.'''
* We need to partition the system disk if we want to create a shared folder on the system drive. [https://forum.openmediavault.org/index.php?thread/12104-shared-folder-on-the-system-disk/ shared folder on the system disk].
* We need to partition the system disk if we want to create a shared folder on the system drive. [https://forum.openmediavault.org/index.php?thread/12104-shared-folder-on-the-system-disk/ shared folder on the system disk].
* [https://forum.openmediavault.org/index.php?thread/18170-share-whole-disk/ share whole disk] (it works)
* [https://forum.openmediavault.org/index.php?thread/18170-share-whole-disk/ share whole disk] (it works)
* '''Remember to create a new user (it seems admin is not working for share folders)'''. When I create a new user (Groups = users) after I created a shared folder, the new user automatically appeared in a shared folder Permissions list.
* '''Remember to create a new user (it seems admin is not working for share folders)'''. When I create a new user (Groups = users) after I created a shared folder, the new user automatically appeared in a shared folder Permissions list.
** No need to create home directory from "Users - Settings".
** No need to create home directory from "Users - Settings".
** Create a user from "Users - Users". Enter a user name. In the '''Group''' attribute, select '''Users'''. That is enough.
** Create a user from "Users - Users". Enter a user name. In the '''Group''' attribute, select '''Users'''. That is enough. Why this is not the default in OMV?
** No need to worry from "Users - Groups".
** No need to worry from "Users - Groups".
* '''Storage - Shared Folders'''. Create a name.
* '''Storage - Shared Folders'''. Create a name.
** No need to click on the "Permissions" button
** Click on the '''Access control list''' button
*** '''File owner and group''': make sure Permissions=R/W/E in Groups=users.
*** '''File access control lists''': make sure Read/Write is clicked (yellow) for the samba user we have created in the "Users" section.
* To access the shared folder
** On Windows, type '''<nowiki>\\\\192.168.86.xx</nowiki>''' in the file manager.
** On Linux or Mac, type '''smb://192.168.86.xx''' in the file manager.
* Example videos
** [https://www.youtube.com/watch?v=oOvb5w5q-Uk OpenMediaVault 6.9 Part 2: Setup User, share and protocols] & the 1st video [https://www.youtube.com/watch?v=x24KdBMkCZ4 OpenMediaVault v6.9 Base Install and Updates]. PS use "ext4" filesystem is simpler instead of btrfs.
** [https://www.youtube.com/watch?v=nCTjWlwaRYQ Everything you need to know about OpenMediaVault 7 - MUST WATCH for Beginners!]


== Where are shared folders ==
== Where are shared folders ==
Line 97: Line 114:


== Update ==
== Update ==
[https://docs.openmediavault.org/en/latest/various/apt.html Software & Update Management]
<pre>
<pre>
apt-get update && apt-get dist-upgrade && omv-update
apt-get update && apt-get dist-upgrade && omv-update
Line 105: Line 123:
== SMART monitoring ==
== SMART monitoring ==
After enabling SMART monitoring in OpenMediaVault, you can do the following:
After enabling SMART monitoring in OpenMediaVault, you can do the following:
* '''View device informatio'''n: From the Devices tab, you can select a drive and click on the Information button to view detailed information about the drive, including its SMART attributes and status.
* '''Settings''': Enable SMART and specify Check Interval (I use 3600)
* '''Configure notifications''': You can configure OpenMediaVault to send you email notifications when changes in S.M.A.R.T values are detected. To do this, go to the System tab in the left-hand menu, select the Notification option, and configure your email settings.
* '''Devices -> View device information''': From the Devices tab, you can select a drive and click on the Information button to view detailed information about the drive, including its SMART attributes and status.
* '''Schedule tests''': From the Scheduled tests tab, you can schedule different types of tests to run on your drives, including short self-tests, long self-tests, conveyance self-tests, and offline immediate tests.
* '''Schedule tests''': From the Scheduled tests tab, you can schedule different types of tests to run on your drives, including short self-tests, long self-tests, conveyance self-tests, and offline immediate tests. I choose to run short tests weekly every Sunday 5:00AM. After we scheduled a short test, there is a run button to start a test immediately.
** Hour: 5am
** Day of Month: *
** Month: *
** Day of week: Sunday
* '''Configure notifications''': You can configure OpenMediaVault to send you email notifications when changes in S.M.A.R.T values are detected. To do this, go to the '''System -> Notification''' and configure your email settings.
** [https://docs.openmediavault.org/en/latest/administration/general/notifications.html#gmail Gmail]. Note that the sender email and username should be the same.
** To create an '''app password''', use this link https://myaccount.google.com/apppasswords
** Click the test button to check if the setting is working.


== NAS Not seen/visible in local network ==
== NAS Not seen/visible in local network ==
Line 245: Line 271:
** Debian 11 Bullseye's end of life is 2024/7.
** Debian 11 Bullseye's end of life is 2024/7.
** Debian 11/12 + CasaOS is using 2.9GB. The root partition has 24GB space in total. 7.8G allocated to /dev partition.
** Debian 11/12 + CasaOS is using 2.9GB. The root partition has 24GB space in total. 7.8G allocated to /dev partition.
* [https://www.xda-developers.com/projects-i-host-on-my-proxmox-home-lab/ 10 projects I host on my Proxmox home lab]


* [https://wiki.casaos.io/en/get-started Get Started]
* [https://wiki.casaos.io/en/get-started Get Started]

Latest revision as of 07:36, 30 October 2024

OpenMediaVault

  • Install & Configure openmediavault (omv6) | DIY NAS Part 3 - Software 2023/4/21. Quite complete !!
    • Set up Users first before messing up folders. The order is similar to the way clients access shared folders.
    • After creating a shared folder in Storage menu, go back to Users menu. Select a user and click "Shared folder privileges". Click on "Read/Write" to enable it (Yellow).
    • Go back to the shared folder. Click on "Privileges" icon. We can verify who has what kind of privileges.
    • After generating "Shared folders", we still need to go to Services and enable NFS or SMB/CIFS. This is two steps: one is to enable it and the other is to add individual shared folders to either SMB or NFS.
    • Now it's time to verify the shared folder on client machines.

Disk space and memory requirement

For disk space, 4GB is enough. It shows 2.4GB in my OMV 6.5.0.

The memory usage is 430MB when in idle.

Network, static IP & DNS for internet access

  • Remember to change DNS in the Advanced section. No need to modify anything in /etc/resolv.conf in terminal. You can check that "ping -c 3 1.1.1.1" works but "ping -c 3 google.com" did not.
  • Configure DNS is necessary when I need to install plugins.

apt update warnings

You still have the security repo in /etc/apt/sources.list. That needs to be removed.

Disk usage

  • Method1: Dashboard - File systems (/dev/sda1, /dev/sdb1, ...)
  • Method2: Storage - File systems

Hard drives spindown

  • Click Storage -> Disks -> Select the disk -> Edit. Choose spindown time from "Disabled" to "20 minutes'.
  • After the spindown, we can use the command 
    smartctl -i -n standby /dev/sdX
    to check the status of the disk. In my case, the output shows "Device is in SLEEP mode, exit(2)". However, if I run the command again immediately, it will display a complete information about the device and the power mode is ACTIVE or IDLE.

Multiple SATA drives

RISC-V NAS: BPI-F3 & OpenMediaVault. IDE-to-USB adapter (Molex output) + Molex to SATA adapter + SATA splitter (for 2 SATA disks).

Sharing

  • It's generally recommended to have a separate disk for share data in OMV.
  • We need to partition the system disk if we want to create a shared folder on the system drive. shared folder on the system disk.
  • share whole disk (it works)
  • Remember to create a new user (it seems admin is not working for share folders). When I create a new user (Groups = users) after I created a shared folder, the new user automatically appeared in a shared folder Permissions list.
    • No need to create home directory from "Users - Settings".
    • Create a user from "Users - Users". Enter a user name. In the Group attribute, select Users. That is enough. Why this is not the default in OMV?
    • No need to worry from "Users - Groups".
  • Storage - Shared Folders. Create a name.
    • No need to click on the "Permissions" button
    • Click on the Access control list button
      • File owner and group: make sure Permissions=R/W/E in Groups=users.
      • File access control lists: make sure Read/Write is clicked (yellow) for the samba user we have created in the "Users" section.
  • To access the shared folder
    • On Windows, type \\\\192.168.86.xx in the file manager.
    • On Linux or Mac, type smb://192.168.86.xx in the file manager.
  • Example videos

Where are shared folders

In OpenMediaVault, shared folders are a key functionality around which all services revolve. They are created as subvolumes on BTRFS file systems or simple directories on all other file systems supported by OpenMediaVault.

When you create a shared folder, you select a disk, and then a folder on that disk. If the folder doesn't exist, it is created for you². That's where the shared folder is located.

The actual path of the shared folder can be obtained using the UUID of the shared folder. On my OMV running in Proxmox, 

root@omv:~# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0   32G  0 disk 
├─sda1   8:1    0   31G  0 part /
├─sda2   8:2    0    1K  0 part 
└─sda5   8:5    0  975M  0 part [SWAP]
sdb      8:16   0  489G  0 disk 
└─sdb1   8:17   0  489G  0 part /srv/dev-disk-by-uuid-9c8dfd73-766f-49c0-acb4-2a26c1c2172c

root@omv:~# ls -l /srv/dev-disk-by-uuid-9c8dfd73-766f-49c0-acb4-2a26c1c2172c/
total 36
-rw------- 1 root root   6144 Oct 19 18:46 aquota.group
-rw------- 1 root root   6144 Oct 19 18:46 aquota.user
drwx------ 2 root root  16384 Oct 18 21:33 lost+found
drwxrwsr-x 2 root users  4096 Oct 19 07:40 OPMShare    <== Share name for a folder

Remove SMB sharing

  • (Necessary?) Plugin - type 'reset' (resetperm) - Install.
  • Step 1: Services - SMB/CIFS - Shares - click one and delete
  • Step 2: Storage - Shared folders - click one and delete.
  • Step 3: Apply the changes.

Note although we remove SMB shares, the Dashboard still shows the disk and its partitions. The Storage still shows the device.

Update

Software & Update Management 

apt-get update && apt-get dist-upgrade && omv-update

omv-release-upgrade

SMART monitoring

After enabling SMART monitoring in OpenMediaVault, you can do the following:

  • Settings: Enable SMART and specify Check Interval (I use 3600)
  • Devices -> View device information: From the Devices tab, you can select a drive and click on the Information button to view detailed information about the drive, including its SMART attributes and status.
  • Schedule tests: From the Scheduled tests tab, you can schedule different types of tests to run on your drives, including short self-tests, long self-tests, conveyance self-tests, and offline immediate tests. I choose to run short tests weekly every Sunday 5:00AM. After we scheduled a short test, there is a run button to start a test immediately.
    • Hour: 5am
    • Day of Month: *
    • Month: *
    • Day of week: Sunday
  • Configure notifications: You can configure OpenMediaVault to send you email notifications when changes in S.M.A.R.T values are detected. To do this, go to the System -> Notification and configure your email settings.

NAS Not seen/visible in local network

Ideally, if we go to the File Manager app -> Network (Windows)/Other Locations (GNOME) we shall see "HOSTNAME -SMB/CIFS" & "HOSTNAME - SSH" items. If we click on "HOSTNAME - SMB/CIFS", it will show the shared folder name.

View free/total space in client

PS. I don't this problem. To view free space, right click on any one of subfolders and choose Properties. It'll show "Free space" (at least on the GNOME file manager).

Unable to show real free/used disk space in SAMBA mounted drives under Windows. So add this line (OMV GUI) to the Extra options-field (bottom of the screen) in 'Edit share' and restart the samba service (systemctl restart smbd.service). It´s a per share command. 

dfree command = /usr/local/bin/dfree

Create a file /usr/local/bin/dfree and make it executable. 

#!/bin/sh
/bin/df -h $1 | grep sda1 | awk '{print $2" "$4}'
# 458G 241G

OMV-Extras

FileBrowser

  • Step 1: System - Plugins
  • Step 2: Services - File browser. Enable it. Port is 3670. Need to select a Shared folder. Save & OpenUI. Default is admin/admin. We can download/upload/copy/paste/delete/... files, create new files/folders, manage users, through the web interface.
  • The filebrowser cannot even preview png files like a regular file manager in a desktop

aquota.user, aquota.group

You can't remove them because they are in use by the quota service.

Backup configuration

Docker, portainer

Open Media Vault 6 (2023), omv-extras, Docker and Portainer Part 2

Proxmox

QNAP vs Synology

Virtualization Station from QNAP

http://www.qnap.com/event/station/en/virtualization.php

The supported NAS models are listed on http://www.qnap.com/event/station/en/virtualization.php?content=supported

Build a NAS like Synology

build a NAS like Synology 7.2 & Automated Redpill Loader (i18n)

Raspberry Pi

Raspberry Pi NAS

N100 PC

Dell Optiplex

ODroid xu4

Zimablade

  • Power by USB-C.
    • But it is 12V. We need to use the power supply purchased from zimaboard.
    • On the other hand, the USB-C power adapter should not be used in other 5V devices.
    • We can use a third party power adapter that supports 12v usb-c output such as Anker 317 Charger (100W) & Amazon
  • The power LED is located under the PCB. We cannot see the light directly from the front. We can see it from the side.
  • It supports two 3.5/2.5 SATA drives including power (note that the USB to SATA connector/adapter can't be used to power up 3.5" hard drive unless the adapter comes with its own power supply like this one UGREEN from amazon).
    • The notch on the power cable should be on top. The power cable should be twisted before it is connected to the board:(
    • A white LED around the SATA power port inside the case will flash when we power the board.
  • Hotkeys
    • While it's booting up, repeatedly tap the DEL key on your keyboard and it should take you into the BIOS/UEFI.
    • Use F11 to choose boot device
  • It supports Wake-on-lan. See Specs.

CasaOS

  • Wiki/guides - https://wiki.casaos.io/en/get-started
  • http://casaos.local/ is not working. I need to manually run sudo apt-get install avahi-daemon. Then it works without rebooting.
    • Currently it supports Debian 11, Ubuntu server 20.04, ... (2024/4)
    • Debian 11 Bullseye's end of life is 2024/7.
    • Debian 11/12 + CasaOS is using 2.9GB. The root partition has 24GB space in total. 7.8G allocated to /dev partition.
  • Tip
    • On the CasaOS dashboard, it can show the power consumption under the CPU. For example, it showed 2W/40C on my Zimablade with a 2.5 HDD attached. PS, playing a video file from CasaOS File browser does not take CPU load or increase power for a low-resolution h264 MP4 file.
    • The power consumption is for CPU only. For example, it shows 2.0W for the CPU but Kill-a-watt shows 5W for the computer (intel NUC7PJYH Pentium J5005, 8th gen, 2017) with a 2.5" SSD. Once I connected a 3.5" HDD in addition to a 2.5" SSD, the CPU shows 2.0W, but the Kill-a-Watt meter shows 7W for the entire computer.
    • Typical 3.5" SATA hard drive usually draws around 5-6 watts of power, and a 2.5" SATA SSD typically consumes 0.25-2 watts when idle.
    • If I have created a shared folder, I can click "..." and select "Copy Path". The path will look like "/mnt/toshiba500/CasaShare" where toshiba500 is my storage name and CasaShare is my (shared) folder's name. Then on Windows, I can paste "\\casaos.local\CasaShare" in the Windows Explorer to access the folder. On Mac/Linux, I can use "smb://casaos.local/CasaShare" to access the folder.
    • There is no way to set up a SMB password from GUI. So the shared folders can be accessed by anyone by default. SMB Sharing needs to support/enforce authentication. A solution is
      1. sudo nano /etc/samba/smb.casa.conf
      2. Change the line "guest ok = Yes" to "guest ok = No"
      3. Run "sudo smbpasswd -a USERNAME_OS"
      4. Run "sudo service smbd restart"
    • Upgrade a docker app seems to be easy. For example, if the Portainer has a new release, I click the "Settings" on the Portainer app on the CasaOS GUI. On the "Tag", I choose the "Latest" instead of "Stable(2.19.4)". Once I click "Save", it'll update the Portainer container. I do need to use the Portainer GUI or a command line to delete the 2.19.4 image.

Zimaboard

  • CasaOS internal storage full: My CasaOS shows the eMMC has only 27.56GB (instead of 32GB) in total. After installing 5 docker images, it now has only 16.6GB space left.
    • Run sudo docker system prune -a to remove all images without at least one container associated with them. The -a flag will remove all images not associated with a running container.
    • Removed unused volumes sudo docker volume prune
    • How to move docker images and volumes to a different storage
    • After I applied the following operations, CasaOS shows the eMMC has 22.38GB available.
sudo systemctl stop casaos*.service
sudo systemctl status casaos.service
sudo systemctl stop docker.*
sudo systemctl status docker.service

sudo mv /var/lib/docker /mnt/SP512/CasaShare512/
sudo ln -s /mnt/SP512/CasaShare512/docker /var/lib/docker

sudo mv /DATA/AppData /mnt/SP512/CasaShare512/
sudo ln -s /mnt/SP512/CasaShare512/AppData /DATA/AppData

sudo reboot

Custom Appstore

  • "App Store" -> "XXX apps" drop-down menu -> "More apps".
    1. Type in "https://casaos-appstore.paodayag.dev/linuxserver.zip" and press "Add".
    2. Refresh the page.
    3. On the left-hand-side, click on "All" drop-down menu and select "LinuxServer.io". It shows 180 apps.
  • https://awesome.casaos.io/content/3rd-party-app-stores/list.html
  • https://github.com/WisdomSky/CasaOS-LinuxServer-AppStore
  • Learning how to import custom app/manual install
  • Dashboard Icons: https://github.com/walkxcode/Dashboard-Icons.
  • {"message": "store app not found" } . Solutions: add linuxserver.io in app store - more apps. After adding linuxserver.io to the app store, webtop can be found in the app store when we do a search. When we check the setting of the webtop app, we see the Docker Image is from "linuxserver/webtop".
  • For the application of webtop, we shall change the Host directory for Container's /config to point to a larger disk instead of the default "/DATA/AppData/webtop/config" which resides in eMMC.
  • For the MeTube app, double quotes will be omitted when we use GUI to create the app. This causes an error for the message "YTDL_OPTIONS invalid option". My solution is to export the compose file, modify it and use "sudo docker compose up -d" to launch it. Test the app on another browser or in an incognito window. The advantage of using exporting the compose file is the file include an x-casaos: section that contains the icon file URL. So the app shows an icon in CasaOS GUI. 
    YTDL_OPTIONS='{"writesubtitles":true,"subtitleslangs":["en","-live_chat"],"updatetime":false,"postprocessors":[{"key":"Exec","exec_cmd":"chmod 0664","when":"after_move"},{"key":"FFmpegEmbedSubtitle","already_have_subtitle":false},{"key":"FFmpegMetadata","add_chapters":true}]}'
  • A better solution for the above situation is to Edit the CasaOS docker compose files in CasaOS. The script is on github here made by big-bear-scripts. PS: 1. we need to be in root in order to run the script 2. it won't save the script in the disk.
    • Automatic method: bash -c "$(wget -qLO - https://raw.githubusercontent.com/bigbeartechworld/big-bear-scripts/master/edit-casaos-apps/run.sh)"
    • Manual method:
      • Go to the folder /var/lib/casaos/apps/ to view all installed services/apps/containers
      • Edit the docker-compose file directly from "/var/lib/casaos/apps/sparkling_felix/" if, for example, my container is called "sparkling_felix"
      • Run casaos-cli app-management apply "$service_name" --file="$docker_compose_path" . For example, sudo casaos-cli app-management apply "sparkling_felix" --file=/var/lib/casaos/apps/sparkling_felix/docker-compose.yml. This will refresh the CasaOS GUI.

Backup/migration

NAS, SAMBA share

SAMBA share for multiple users with passwords. Edit this /etc/samba/smb.casa.conf, instead of the smb.conf.

Pogoplug

Basic statistic about Pogoplug E02

See http://archlinuxarm.org/platforms/armv5/pogoplug-v2-pinkgray for hardware specification.

  • architecture: ARMv5te
  • Processor: Marvell Kirkwood 1.2GHz
  • RAM: 256MB
  • 124MB NAND
  • Gigabit ethernet
  • 5W power
  • 4 USB 2.0 connector.

20121105 181258.jpg 20121105 181506.jpg

Install ArchLinux on pogoplug (Debian is better actually)

See the instruction in http://archlinuxarm.org/platforms/armv5/pogoplug-v2-pinkgray for how to install Archlinux on pogoplug (E02). I make a copy below. 

killall hbwd
cd /tmp
wget http://jeff.doozan.com/debian/uboot/install_uboot_mtd0.sh
chmod +x install_uboot_mtd0.sh
./install_uboot_mtd0.sh
/sbin/fdisk /dev/sda

At the fdisk prompt, delete old partitions and create a new one:

  • Type o. This will clear out any partitions on the drive.
  • Type p to list partitions. There should be no partitions left.
  • Now type n, then p for primary, 1 for the first partition on the drive, and then press ENTER, accepting default values.
  • Exit by typing w.
wget http://archlinuxarm.org/os/pogoplug/mke2fs
chmod 755 mke2fs
./mke2fs /dev/sda1
mkdir usb
mount /dev/sda1 usb
cd usb
wget http://archlinuxarm.org/os/ArchLinuxARM-armv5te-latest.tar.gz
tar -xzvf ArchLinuxARM-armv5te-*.tar.gz  # This will take a long time
rm ArchLinuxARM-armv5te-*.tar.gz
sync  # Takes a while when using a flash drive. Not in my case however.
cd ..
umount usb
/sbin/reboot

Log back in after your Pogoplug reboots using username root and password root.

At the end, the system is very lean. 

[root@alarm ~]# free
             total       used       free     shared    buffers     cached
Mem:        252492      28292     224200          0       4580      14072
-/+ buffers/cache:       9640     242852
Swap:            0          0          0
[root@alarm ~]# df -lh
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.4G  443M  6.6G   7% /
/dev/root       7.4G  443M  6.6G   7% /
devtmpfs        124M     0  124M   0% /dev
run             124M  224K  124M   1% /run
shm             124M     0  124M   0% /dev/shm
tmpfs           124M     0  124M   0% /tmp
[root@alarm ~]# uname -a
Linux alarm 3.1.10-13-ARCH #1 PREEMPT Mon Aug 20 15:46:18 UTC 2012 armv5tel GNU/Linux
[root@alarm ~]# cat /proc/cpuinfo
Processor	: Feroceon 88FR131 rev 1 (v5l)
BogoMIPS	: 1191.11
Features	: swp half thumb fastmult edsp 
CPU implementer	: 0x56
CPU architecture: 5TE
CPU variant	: 0x2
CPU part	: 0x131
CPU revision	: 1

Hardware	: Marvell SheevaPlug Reference Board
Revision	: 0000
Serial		: 0000000000000000
[root@alarm ~]# cat /proc/cpuinfo
Processor	: Feroceon 88FR131 rev 1 (v5l)
BogoMIPS	: 1191.11
Features	: swp half thumb fastmult edsp 
CPU implementer	: 0x56
CPU architecture: 5TE
CPU variant	: 0x2
CPU part	: 0x131
CPU revision	: 1

Hardware	: Marvell SheevaPlug Reference Board
Revision	: 0000
Serial		: 0000000000000000
[root@alarm ~]#

After it, follow the instruction on lifehacker and https://wiki.archlinux.org/index.php/DeveloperWiki:usrlib 

pacman -Syu --ignore glibc
pacman -Su
mkinitcpio -p linux

To install rsync, use 

pacman -S rsync

Unfortunately, I cannot bypass the error 

[root@alarm ~]# pacman -Suy rsync
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 alarm is up to date
 aur is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Targets (2): glibc-2.16.0-4  rsync-3.0.9-5

Total Installed Size:   34.26 MiB
Net Upgrade Size:       0.61 MiB

Proceed with installation? [Y/n] 
(2/2) checking package integrity                        [#############################] 100%
(2/2) loading package files                             [#############################] 100%
(2/2) checking for file conflicts                       [#############################] 100%
error: failed to commit transaction (conflicting files)
glibc: /lib exists in filesystem
Errors occurred, no packages were upgraded.

Install Debian on pogoplug

See the instruction on http://projects.doozan.com/debian.

  • Find your devices's IP address and connect via SSH:
  • Partition your flash drive with fdisk (or gparted):
fdisk /dev/sda
# Configure partion 1 as Linux (I'd recommend making this at least 512Mb. The default bare-bones installation uses 280Mb.)
# Configure partion 2 as Linux Swap (I used 256MB.  Adjust according to your anticipated memory usage.)
# Set partition 1 active

See http://www.terminally-incoherent.com/blog/2012/09/19/set-up-a-home-linux-server-for-30-with-pogoplug/ for clear explanation of using fdisk.

  • Download and run the Debian Wheezy installer:
cd /tmp
wget http://projects.doozan.com/debian/kirkwood.debian-wheezy.sh
chmod +x kirkwood.debian-wheezy.sh
export PATH=$PATH:/usr/sbin:/sbin
./kirkwood.debian-wheezy.sh
  • Alternatively, you can choose to install Debian Squeeze with the following commands:
cd /tmp
wget http://projects.doozan.com/debian/dockstar.debian-squeeze.sh
chmod +x dockstar.debian-squeeze.sh
export PATH=$PATH:/usr/sbin:/sbin
./dockstar.debian-squeeze.sh

The script will take some time to download the debian images and extract them to your flash drive. The total install time will vary with the speed of your flash drive and your Internet connection. On my system, it takes about 20 minutes. Once it's finished, you can reboot into your new Debian install.

After your device reboots, it may have a different IP address (it's identifying as 'Debian' to the DHCP server now instead of 'Pogoplug').

The default root password in Debian is 'root'. After you've logged in, you should change the root password and configure /etc/apt/sources.list to point to a Debian mirror near you. 

passwd
nano /etc/apt/sources.list

After reboot, the debian system is very lean too 


root@debian:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
rootfs                3.2G  260M  2.8G   9% /
none                  123M   36K  123M   1% /dev
/dev/sda1             3.2G  260M  2.8G   9% /
tmpfs                 125M     0  125M   0% /lib/init/rw
tmpfs                 125M     0  125M   0% /dev/shm
tmpfs                 125M     0  125M   0% /tmp
root@debian:~# free
             total       used       free     shared    buffers     cached
Mem:        255672      19636     236036          0        816      11104
-/+ buffers/cache:       7716     247956
Swap:       535544          0     535544

After I install LAMP, the memory uses 250MB. However, after I reboot the pogoplug, it reduces to 66MB. 

root@debian:~# free
             total       used       free     shared    buffers     cached
Mem:        255672      66076     189596          0       1244      37000
-/+ buffers/cache:      27832     227840
Swap:       535544          0     535544

If I want to add a USB drive, it is safe NOT to insert the 2nd USB drive when the system boot. It is better to insert the 2nd USB drive after the system has booted. In my experience, if I insert the 2nd USB drive when the system boot, the first USB drive will be mounted as /dev/sdb1 but I want the 2nd USB drive mounted as /dev/sdb1.

Install build-essential. Should be fine. See Home automation link below.

Other resources:

Make it more powerful

Backup server (RAID 1)

Crash plan.

Remote file access by AjaXplorer AjaXplorer

Cloud by OpenStack

BitTorrent by Transmission

Note: Must stop the transmission-daemon before modifying the setting. 

apt-get install transmission-cli transmission-common transmission-daemon
# stop transmission first before change the setting
/etc/init.d/transmission-daemon stop
nano /etc/transmission-daemon/settings.json
# change “download-dir”, “incomplete-dir”
# change rpc-whitelist to "*" or your remote IP
# change "rpc-username" ,
#        "rpc-password"  (this will then be hashed, so don't worry)
# change "speed-limit-up": 10,
#        "speed-limit-up-enabled": true,
# change upload-limit (KB/s) from 100 to 10
#        upload-limit-enabled from 0 to 1
mkdir /mnt/usb/Downloads
chmod -R 777 /mnt/usb/Downloads
transmission-daemon 

/etc/init.d/transmission-daemon stop

# Remember to shut down transmission before editing the settings.json file

/etc/init.d/transmission-daemon start

Open browser http://hostname:9091/transmission/web

For pirat*, we can download torr* by using (see here) 

http://torrents.thepiratebay.se/xxxxxx/My.torrent

So we don't need to download the torrent in our local machine. In fact, we can just right click 'Get this torrent' button and copy the link. Then we can open the url in transmission/web.

Note: the transfer speed from pogoplug to ubuntu is about 5.5MB/s.

SFTP server by Cyberduck

WebDAV by owncloud.org

SAMBA server

So every computer in LAN can access the contents there. On Windows computer, type \\192.168.1.3\ShareFolder, for example.

  • UPnP media server will stream the media content on your Pogoplug to any compatible media player such as the PlayStation 3, Microsoft Xbox 360, and D-Link Boxee Box over your local network.

Email alerts

Using sSMTP; see here.

Print server

Best NAS

Best NAS Hard Drives

TRIM

  • TRIM is like housekeeping for your SSD (Solid State Drive). When you delete a file, the drive doesn’t immediately erase the data. Instead, it just marks the space as available. Over time, this can slow things down because the drive has to sort through old data to write new stuff. TRIM helps by telling the SSD which parts of the data are no longer in use, so it can clean up more efficiently. Think of it as tidying up your room regularly so you don’t end up with piles of junk everywhere!
  • Solid state drive. To verify TRIM support, run:
    $ lsblk --discard
    And check the values of DISC-GRAN (discard granularity) and DISC-MAX (discard max bytes) columns. Non-zero values indicate TRIM support.
  • TRIM is indeed more relevant to SSDs. Here's why:
    • HDDs use spinning disks to store data. When you delete a file on an HDD, the drive can just overwrite the old data when it's ready to save new files. It's like writing over an old note on paper.
    • SSDs, on the other hand, use flash memory. When you delete a file on an SSD, it doesn't immediately erase the data. Instead, it marks the space as available. However, to write new data, the SSD must first erase the old data (a slower process). Without TRIM, this can lead to performance degradation over time because the drive has to manage this extra step of erasing before writing.
  • why SSDs and HDDs handle data differently:
    • SSDs use NAND flash memory, which stores data in memory cells. These cells must be erased before they can be written to again. Erasing is necessary because you can't write new data on top of old data without clearing the old data first. This is due to the way data is managed in blocks, which must be fully erased to ensure accuracy and performance.
    • HDDs, on the other hand, use magnetic storage on spinning disks. Writing new data simply involves changing the magnetic state of the disk, allowing it to directly overwrite old data. There’s no need to erase the old data first because the writing process inherently changes the data bits.

Different NAS

TrueNAS

NAS4Free vs FreeNAS vs Amahi vs OpenMediaVault

NAS4Free

  • NAS4Free from pcworld.com.
  • FreeNAS and UPnP setups from homemultimedianetwork.com.
  • A non-English youtube video demonstrates installing NAS4Free on Virtualbox (it works as I tested). The important thing is the NAS4Free is based on BSD. So we should select BSD & 64-bit instead Linux. I am using NAS4Free-x64-LivdCD-9.3.0.2.1349.iso. N.B. NAS4Free does not take much space as, saying, Ubuntu. See the next screenshot.

NAS4Free64.png

  • WebGUI interface and setup.
  • After I enable ssh and allows root login, I can see the default shell is tcsh. See the document in here.
  • If we have added IDE disks from virtual machine setting, we can add them using the web management. To do that, we need to Add disk by clicking '+' sign, apply changes, format disks, and mount disks (mount point name can be 'Data1', 'Data2', ...). These disks will be available via /mnt/Data1 or /mnt/Data2. See the document in here.
  • Lots of services: CIFS/SMB (Samba v4.x), FTP, NFS, TFTP, AFP, RSYNC, Unison, iSCSI (initiator and target), HAST, CARP, Bridge, UPnP, Webserver, and Bittorent.
  • UPnP service works like a charm. Note that I need to check the checkbox in 'Enable' (on the topright corner) for it to work. On the client side, I use ubuntu's VLC (v2.1.4) to test. In VLC, click View -> Playlist ('Ctrl+l') and then select Local Network -> Universal Plug 'n' Play. I have tested streaming mp3 and mp4 files. For Android devices, we can try the BubbleUPnP app.
  • How do I stream photos? Photos do not show up on VLC or Android UPnP software like BubbleUPnP, AirWire. The trick is to click Administrative Webgui url link and then click rebuild/update database. After the fix, VLC can play photos as video (10 seconds for each) automatically and BubbleUPnP can play the photos too (users decide how long to stay on one image).

FreeNAS

NAS hard disks

NAS4Free distribution

NextCloud

CentosConOnline.png

Cockpit

Turning Proxmox Into a Pretty Good NAS

Unraid

  • https://unraid.net/
  • Videos
  • NAS with unraid preinstalled.
  • Parity.
    • If a drive in the array fails, the data on the other drives can be combined with the parity data to reconstruct the missing data.
    • unRAID cannot protect you from two failed disks unless you use two parity disks.
  • Parity is not equivalent to backup
    • I did a simple test. One parity drive and one data drive. Once parity sync is done, I replace the data drive. Unraid does rebuilt the disk. And I can get my data back. The lesson is the parity disk is not used for data backup. It is used for reconstruct the disk.
    • Please help me understand parity vs backup
    • Unraid Server Backups Using LuckyBackup
    • It seems we don't need to schedule parity sync often, probably once per month or quarter.
  • What happens if the parity drive fails?
  • GUID & UUID
    • GUID (Globally Unique Identifier) is associated with the entire disk, especially in the context of GUID Partition Table (GPT) where it uniquely identifies the disk itself.
    • UUID (Universally Unique Identifier) is specific to partitions and is used to uniquely identify each partition on the disk.
    • Commands to get the GUID and UUID. In the following case, sda is PNY 1TB SATA drive, sdb is a SATA drive (sdb1 is FAT and sdb2 is ext4).
    $ lsblk -o name,uuid # it returns partition-specific uuid

NAME        UUID
sda         68ae4e09-51ae-4327-a080-8a662ca6e9b2
sdb         
├─sdb1      8C63-FBEB
└─sdb2      87210264-cc6e-430c-98cb-3eb8a247da20

$ sudo sgdisk -i 1 /dev/sdb # it returns disk-specific GUID

Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
Partition unique GUID: 2D2BBD4C-5409-4483-B435-1D4AE4A3246E
  • USB drive requirements Changing the flash device
    • USB 2.0 over USB 3.0: USB 2.0 drives are generally more reliable and universally recognized by computers
    • Capacity: The drive should be at least 2 GB in size, but no larger than 32 GB for manual installs. Larger drives can work, but 32 GB is recommended for simplicity.
    • Quality: Use a high-quality flash drive from reputable manufacturers such as Lexar, PNY, Samsung, and Kingston.
  • My testing
    • Zimablade with 16GB Ram. 525G SATA SSD as data disk, 2T WD 2.5" USB HDD as parity disk. Unraid is installed on a Samsung flash drive.
    • Main -> History shows that it took 6 hours & 11 min to run parity-sync.
    • I swapped the data disk and use a 1T SATA as data disk. It took 3 hours & 45 min to rebuild the data disk. All data are recovered.
    • The power usage is 4 Watt when it is idle and about 15 Watt when it is run parity sync. The official power adapter is 12V/3A=36W.
    • "hdparm -t --direct" shows nice speed for SATA SSDs but the WD USB HDD is just 25MB/s. Recall that the default filesystem is "xfs" on Unraid. When I benchmark the USB HDD in another machine, it also shows 25MB/s (so it's not an issue with the USB port on zimablade). If I reformat the USB HDD to ext4 and test it in Linux, it shows 110MB/s speed. A huge difference!
  • FAQs:
    • can i recover data in a 1 parity -1 data disk server ?
    • SSD and trim:
      • Issue: UNRAID does not fully support TRIM on parity disks, meaning the SSD's performance may degrade over time. TRIM is essential for maintaining SSD speed by clearing out unused data blocks.
      • Impact: Without TRIM, SSDs will wear out faster and suffer from slower write performance over time.
Retrieved from "https://wiki.taichimd.us/index.php?title=NAS&oldid=40506"