Compare commits

...

2 Commits

Author SHA1 Message Date
9caf3b1470 Update NAS docs
All checks were successful
Deploy to Fighter / Deploy (push) Successful in 7s
Deploy to Fighter / Notify (push) Successful in 2s
2024-02-20 19:46:50 -08:00
4a6ec48418 Hack for keycloak forwardauth containers not respecting depends_on 2024-02-20 19:13:01 -08:00
9 changed files with 128 additions and 129 deletions

View File

@ -14,7 +14,7 @@ We can get each drive's serial number, power on hours, and manufacture date with
```
for dev in {b..y}; \
do echo "### /dev/sd$dev" && \
smartctl -a /dev/sd$dev | awk '/Serial number:/{serial=$NF} /hours:minutes/{powerontime=$NF} {FS="\n"}/Manufactured in /{manufacture=$NF}END{print serial; print powerontime; print manufacture; printf "\n"}'; \
sudo smartctl -a /dev/sd$dev | awk '/Serial number:/{serial=$NF} /hours:minutes/{powerontime=$NF} {FS="\n"}/Manufactured in /{manufacture=$NF}END{print serial; print powerontime; print manufacture; printf "\n"}'; \
done
```

10
barbarian/DISKSHELFMAP.md Normal file
View File

@ -0,0 +1,10 @@
# Physical Disk Locations (DS4243)
Each cell contains the serial number for the drive in the mapped bay.
| | X1 | X2 | X3 | X4 |
|:--:|:---------------------------:|:---------------------------:|:---------------:|:---------------:|
| Y1 | VJGPS30X (da0) | 2EGP88WV (da1) | VJG2808X (da2) | VJG2PVRX (da3) |
| Y2 | VJGR6TNX (da4) | 2EG14YNJ (da5) | VJGR6TNX (da4) | 2EG14YNJ (da5) |
| Y3 | VJG282NX (da8) | 2EGXD27V (da9) | VJG1NP9X (da10) | VJG2UTUX (da11) |
| Y4 | VJGRGD2X (da12) | 001526PL8AVV2EGL8AVV (da13) | 2EKA903X (da14) | VJGRRG9X (da15) |
| Y5 | 001703PXPS1VVLKXPS1V (da16) | 001528PNPVWV2EGNPVWV (da17) | 2EKATR2X (da18) | VKH3Y3XX (da19) |
| Y6 | 001703PV9N8VVLKV9N8V (da20) | 001708P4W2VVR5G4W2VV (da21) | 2EKA92XX (da22) | VKGW5YGX (da23) |

View File

@ -1,93 +1,6 @@
# Pools
I have 2 pools, one for "Media" and one for everything else. All disks are 8 TB HGST/Hitachi drives with a sector size of 4096B. All pools use vdevs of 3 drives in RAIDZ1.
## Media - High-bandwidth sequential reads/writes
This pool contains one dataset: Media, which holds all non-text data (3d models, audio, images, video). As of writing, only the `./Media/Video/Movies` and `Media/Video/Shows` are accessed by services. Everything else is manipulated manually.
## Tank
This pool contains all other datasets.
# Physical Disk Locations (DS4243)
Each cell contains the serial number for the drive in the mapped bay.
| | X1 | X2 | X3 | X4 |
|:--:|:---------------------------:|:---------------------------:|:---------------:|:---------------:|
| Y1 | VJGPS30X (da0) | 2EGP88WV (da1) | VJG2808X (da2) | VJG2PVRX (da3) |
| Y2 | VJGR6TNX (da4) | 2EG14YNJ (da5) | VJGR6TNX (da4) | 2EG14YNJ (da5) |
| Y3 | VJG282NX (da8) | 2EGXD27V (da9) | VJG1NP9X (da10) | VJG2UTUX (da11) |
| Y4 | VJGRGD2X (da12) | 001526PL8AVV2EGL8AVV (da13) | 2EKA903X (da14) | VJGRRG9X (da15) |
| Y5 | 001703PXPS1VVLKXPS1V (da16) | 001528PNPVWV2EGNPVWV (da17) | 2EKATR2X (da18) | VKH3Y3XX (da19) |
| Y6 | 001703PV9N8VVLKV9N8V (da20) | 001708P4W2VVR5G4W2VV (da21) | 2EKA92XX (da22) | VKGW5YGX (da23) |
# Hardware
| Part | P/N | Specs |
|:----:|:---:|:-----:|
| CPU | Intel i7-5930k | [Intel Ark](https://www.intel.com/content/www/us/en/products/sku/82931/intel-core-i75930k-processor-15m-cache-up-to-3-70-ghz/specifications.html) |
| Motherboard | Gigabyte GA-X99-SLI | [Gigabyte](https://www.gigabyte.com/Motherboard/GA-X99-SLI-rev-10/sp#sp) |
| RAM | 8x GeIL EVO Potenza 8GB | [GeilMemory](http://www.geilmemory.com/product/?id=22) |
| GPU | Zotac Nvidia Geforce GT 710 2GB | [Zotac](https://www.zotac.com/us/product/graphics_card/geforce%C2%AE-gt-710-2gb) |
| NIC | Intel I218-V | [Intel Ark](https://ark.intel.com/content/www/us/en/ark/products/71305/intel-ethernet-connection-i218v.html)
| NIC2 | Mellanox MT26448 ConnectX-2 | [Nvidia](https://network.nvidia.com/related-docs/prod_adapter_cards/ConnectX-2_QSFP_IB_and_%20SFP_10GigE_Card_User_Manual_MHZH29.pdf)
| HBA | HP LSI SAS9217-4i4e | [SSDWorks](https://www.ssdworks.com/datasheets/LSI_SAS9217-4i4e_PB.pdf) |
| Expander | HP 4K1065 | [HP Enterprise](https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c04495958) |
<details>
<summary>Output of <code>inxi -CDGmMNPS</code></summary>
<pre><code>
System: Host: joey-nas.local Kernel: FreeBSD 12.2-RELEASE-p3 amd64 bits: 64 Console: tty pts/0 OS: FreeBSD 12.2-RELEASE-p3
Machine: Type: Desktop Mobo: Gigabyte model: X99-SLI-CF v: x.x serial: N/A UEFI: American Megatrends v: F24a rev: 5.6
date: 01/11/2018
Memory: RAM: total: 63.79 GiB used: 61.55 GiB (96.5%)
Array-1: capacity: 512 GiB note: check slots: 8 EC: None
Device-1: DIMM_A1 size: 8 GiB speed: 2133 MT/s
Device-2: DIMM_A2 size: 8 GiB speed: 2133 MT/s
Device-3: DIMM_B1 size: 8 GiB speed: 2133 MT/s
Device-4: DIMM_B2 size: 8 GiB speed: 2133 MT/s
Device-5: DIMM_C1 size: 8 GiB speed: 2133 MT/s
Device-6: DIMM_C2 size: 8 GiB speed: 2133 MT/s
Device-7: DIMM_D1 size: 8 GiB speed: 2133 MT/s
Device-8: DIMM_D2 size: 8 GiB speed: 2133 MT/s
CPU: Info: 12-Core model: Intel Core i7-5930K bits: 64 type: MCP cache: L2: 1.5 MiB note: check
Speed: 3500 MHz min/max: N/A Core speed (MHz): N/A
Graphics: Device-1: NVIDIA GK208B [GeForce GT 710] driver: vgapci
Display: server: No display server data found. Headless machine? tty: 164x47
Message: Unable to show advanced data. Required tool glxinfo missing.
Network: Device-1: Intel Ethernet I218-V driver: em
Device-2: Mellanox MT26448 [ConnectX EN 10GigE PCIe 2.0 5GT/s] driver: mlx4_core
Drives: Local Storage: total: raw: 174.73 TiB usable: 112.87 TiB used: 64.98 TiB (57.6%)
ID-1: /dev/ada0 vendor: Intel model: SSDSCKGW080A4 DC31 size: 74.53 GiB scheme: GPT
ID-2: /dev/da0 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-3: /dev/da1 vendor: HGST (Hitachi) model: HUH728080AL5204 CD05 size: 7.28 TiB scheme: GPT
ID-4: /dev/da10 vendor: HGST (Hitachi) model: H7280A520SUN8.0T PAG1 size: 7.28 TiB scheme: GPT
ID-5: /dev/da11 vendor: HGST (Hitachi) model: H7280A520SUN8.0T PD51 size: 7.28 TiB scheme: GPT
ID-6: /dev/da12 vendor: HGST (Hitachi) model: HUH728080AL4200 A7D8 size: 7.28 TiB scheme: GPT
ID-7: /dev/da13 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-8: /dev/da14 vendor: HGST (Hitachi) model: H7280A520SUN8.0T PD51 size: 7.28 TiB scheme: GPT
ID-9: /dev/da15 vendor: HGST (Hitachi) model: H7280A520SUN8.0T PAG1 size: 7.28 TiB scheme: GPT
ID-10: /dev/da16 vendor: HGST (Hitachi) model: HUH728080AL4200 A7D8 size: 7.28 TiB scheme: GPT
ID-11: /dev/da17 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-12: /dev/da18 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-13: /dev/da19 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-14: /dev/da2 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-15: /dev/da20 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-16: /dev/da21 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-17: /dev/da22 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-18: /dev/da23 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-19: /dev/da3 vendor: HGST (Hitachi) model: HUH728080AL5200 AD05 size: 7.28 TiB scheme: GPT
ID-20: /dev/da4 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-21: /dev/da5 vendor: HGST (Hitachi) model: HUH728080AL5200 AD05 size: 7.28 TiB scheme: GPT
ID-22: /dev/da6 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
ID-23: /dev/da7 vendor: HGST (Hitachi) model: H7280A520SUN8.0T PD51 size: 7.28 TiB scheme: GPT
ID-24: /dev/da8 vendor: HGST (Hitachi) model: HUH728080AL4200 A7D8 size: 7.28 TiB scheme: GPT
ID-25: /dev/da9 vendor: Hitachi model: HUH72808CLAR8000 M7K0 size: 7.28 TiB scheme: GPT
Partition: ID-1: / size: 62.2 GiB used: 26.85 GiB (43.2%) fs: zfs logical: freenas-boot/ROOT/FreeNAS-12.0-U2
ID-2: swap-1 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/mirror/swap0.eli
ID-3: swap-2 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/mirror/swap1.eli
ID-4: swap-3 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/mirror/swap2.eli
ID-5: swap-4 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/mirror/swap3.eli
ID-6: swap-5 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/mirror/swap4.eli
</code></pre>
</details>
## Replace a failing disk
1. Get the disknum (like `da4`), either in the web UI or with the ~/disklist.pl script, for the disk that needs to be replaced.
2. Refer to the [Physical Disk Locations](#physical-disk-locations-ds4243) chart to determine which shelf slot the disk is in. Remove the disk.

View File

@ -0,0 +1,62 @@
System:
Host: barbarian Kernel: 5.15.107+truenas arch: x86_64 bits: 64 Console: pty pts/1 Distro: Debian
GNU/Linux 11 (bullseye)
Machine:
Type: Desktop Mobo: Gigabyte model: X99-SLI-CF v: x.x serial: N/A UEFI: American Megatrends
v: F24a date: 01/11/2018
Memory:
System RAM: total: 64 GiB available: 62.64 GiB used: 39.28 GiB (62.7%)
Array-1: capacity: 512 GiB note: check slots: 8 modules: 8 EC: None
Device-1: DIMM_A1 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-2: DIMM_A2 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-3: DIMM_B1 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-4: DIMM_B2 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-5: DIMM_C1 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-6: DIMM_C2 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-7: DIMM_D1 type: DDR4 size: 8 GiB speed: 2133 MT/s
Device-8: DIMM_D2 type: DDR4 size: 8 GiB speed: 2133 MT/s
CPU:
Info: 6-core model: Intel Core i7-5930K bits: 64 type: MT MCP cache: L2: 1.5 MiB
Speed (MHz): avg: 1270 min/max: 1200/3700 cores: 1: 2026 2: 1201 3: 1200 4: 1201 5: 1201
6: 1200 7: 1206 8: 1201 9: 1202 10: 1201 11: 1202 12: 1202
Graphics:
Device-1: NVIDIA GK208B [GeForce GT 710] driver: N/A
Display: server: No display server data found. Headless machine? tty: 177x61
API: N/A Message: No API data available in console. Headless machine?
Network:
Device-1: Intel Ethernet I218-V driver: e1000e
Device-2: Mellanox MT26448 [ConnectX EN 10GigE PCIe 2.0 5GT/s] driver: mlx4_core
Drives:
Local Storage: total: raw: 174.73 TiB usable: 116.17 TiB used: 55.5 TiB (47.8%)
ID-1: /dev/sda vendor: Intel model: SSDSCKGW080A4 size: 74.53 GiB
ID-2: /dev/sdb vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-3: /dev/sdc vendor: Hitachi model: HUH728080AL5200 size: 7.28 TiB
ID-4: /dev/sdd vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-5: /dev/sde vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-6: /dev/sdf vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-7: /dev/sdg vendor: Hitachi model: HUH728080AL5200 size: 7.28 TiB
ID-8: /dev/sdh vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-9: /dev/sdi vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-10: /dev/sdj vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-11: /dev/sdk vendor: Hitachi model: HUH728080AL5200 size: 7.28 TiB
ID-12: /dev/sdl vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-13: /dev/sdm vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-14: /dev/sdn vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-15: /dev/sdo vendor: Sun Microsystems model: H7280A520SUN8.0T size: 7.28 TiB
ID-16: /dev/sdp vendor: Hitachi model: HUH728080AL4200 size: 7.28 TiB
ID-17: /dev/sdq vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-18: /dev/sdr vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-19: /dev/sds vendor: Sun Microsystems model: H7280A520SUN8.0T size: 7.28 TiB
ID-20: /dev/sdt vendor: Hitachi model: HUH728080AL4200 size: 7.28 TiB
ID-21: /dev/sdu vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
ID-22: /dev/sdv vendor: Sun Microsystems model: H7280A520SUN8.0T size: 7.28 TiB
ID-23: /dev/sdw vendor: Sun Microsystems model: H7280A520SUN8.0T size: 7.28 TiB
ID-24: /dev/sdx vendor: Hitachi model: HUH728080AL4200 size: 7.28 TiB
ID-25: /dev/sdy vendor: Hitachi model: HUH72808CLAR8000 size: 7.28 TiB
Partition:
ID-1: / size: 43.91 GiB used: 2.45 GiB (5.6%) fs: zfs logical: freenas-boot/ROOT/22.12.3
ID-2: swap-1 size: 2 GiB used: 26.5 MiB (1.3%) fs: swap dev: /dev/dm-0
ID-3: swap-2 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-1
ID-4: swap-3 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-2
ID-5: swap-4 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-3
ID-6: swap-5 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-4

View File

@ -1,20 +0,0 @@
System:
Host: barbarian Kernel: 5.15.107+truenas arch: x86_64 bits: 64 Console: pty pts/0 Distro: Debian
GNU/Linux 11 (bullseye)
Machine:
Type: Desktop Mobo: Gigabyte model: X99-SLI-CF v: x.x serial: N/A UEFI: American Megatrends
v: F24a date: 01/11/2018
CPU:
Info: 6-core Intel Core i7-5930K [MT MCP] speed (MHz): avg: 1201 min/max: 1200/3700
Graphics:
Device-1: NVIDIA GK208B [GeForce GT 710] driver: N/A
Display: server: No display server data found. Headless machine? tty: 177x60
API: N/A Message: No API data available in console. Headless machine?
Network:
Device-1: Intel Ethernet I218-V driver: e1000e
Device-2: Mellanox MT26448 [ConnectX EN 10GigE PCIe 2.0 5GT/s] driver: mlx4_core
Drives:
Local Storage: total: raw: 174.73 TiB usable: 116.17 TiB used: 61.66 TiB (53.1%)
Info:
Processes: 480 Uptime: 34d 1h 9m Memory: total: 64 GiB available: 62.64 GiB
used: 39.11 GiB (62.4%) Init: systemd target: multi-user (3) Shell: Zsh inxi: 3.3.30

View File

@ -18,3 +18,10 @@ for stack in /home/admin/homelab/fighter/config/*; do
docker compose up -d
cd /home/admin/homelab/fighter/config/
done
# extra thing because my keycloak healthcheck doesn't work properly
echo "Wait 15s, then bring Keycloak forwardauth containers online"
cd /home/admin/homelab/fighter/config/keycloak
sleep 15
docker compose up -d

14
monk/DISKINFO.md Normal file
View File

@ -0,0 +1,14 @@
We can get each drive's serial number, power on hours, and manufacture date with the following one-liner:
```
for dev in {a..e}; \
do echo "### /dev/sd$dev" && \
sudo smartctl -a /dev/sd$dev
done
```
Make sure to update this list *only* when a disk is newly installed. Also note the date of installation for the disk.
Or, if you're only updating one drive:
1. Set the `$dev` variable to the drive letter you want to check (e.g. for `/dev/sdr`, use `dev=r`)
2. Run the one-liner: `smartctl -a /dev/sd$dev'`

34
monk/inxi -CDGmMNPS.txt Normal file
View File

@ -0,0 +1,34 @@
System:
Host: monk Kernel: 5.15.107+truenas arch: x86_64 bits: 64 Console: pty pts/1 Distro: Debian
GNU/Linux 11 (bullseye)
Machine:
Type: Desktop Mobo: Gigabyte model: Z77X-UD5H serial: N/A UEFI: American Megatrends v: F16j
date: 11/14/2017
Memory:
System RAM: total: 16 GiB available: 15.51 GiB used: 10.96 GiB (70.7%) igpu: 64 MiB
Array-1: capacity: 32 GiB slots: 4 modules: 2 EC: None
Device-1: ChannelB-DIMM1 type: no module installed
Device-2: ChannelA-DIMM1 type: no module installed
Device-3: ChannelB-DIMM0 type: DDR3 size: 8 GiB speed: 1600 MT/s
Device-4: ChannelA-DIMM0 type: DDR3 size: 8 GiB speed: 1600 MT/s
CPU:
Info: quad core model: Intel Core i5-3570K bits: 64 type: MCP cache: L2: 1024 KiB
Speed (MHz): avg: 1602 min/max: 1600/3800 cores: 1: 1602 2: 1602 3: 1602 4: 1602
Graphics:
Device-1: Intel Xeon E3-1200 v2/3rd Gen Core processor Graphics driver: i915 v: kernel
Display: server: No display server data found. Headless machine? tty: 177x61
API: N/A Message: No API data available in console. Headless machine?
Network:
Device-1: Intel 82579V Gigabit Network driver: e1000e
Device-2: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet driver: atl1c
Drives:
Local Storage: total: raw: 29.55 TiB usable: 15.27 TiB used: 8.74 TiB (57.2%)
ID-1: /dev/sda vendor: SanDisk model: SDSSDX480GG25 size: 447.13 GiB
ID-2: /dev/sdb vendor: HGST (Hitachi) model: HUH728080ALE604 size: 7.28 TiB
ID-3: /dev/sdc vendor: HGST (Hitachi) model: HUH728080ALE604 size: 7.28 TiB
ID-4: /dev/sdd vendor: HGST (Hitachi) model: HUH728080ALE604 size: 7.28 TiB
ID-5: /dev/sde vendor: HGST (Hitachi) model: HUH728080ALE604 size: 7.28 TiB
Partition:
ID-1: / size: 428.76 GiB used: 2.65 GiB (0.6%) fs: zfs logical: boot-pool/ROOT/22.12.3
ID-2: swap-1 size: 2 GiB used: 268 KiB (0.0%) fs: swap dev: /dev/dm-0
ID-3: swap-2 size: 2 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-1

View File

@ -1,21 +0,0 @@
System:
Host: monk Kernel: 5.15.107+truenas arch: x86_64 bits: 64 Console: pty pts/1 Distro: Debian
GNU/Linux 11 (bullseye)
Machine:
Type: Desktop Mobo: Gigabyte model: Z77X-UD5H serial: N/A UEFI: American Megatrends v: F16j
date: 11/14/2017
CPU:
Info: quad core Intel Core i5-3570K [MCP] speed (MHz): avg: 1602 min/max: 1600/3800
Graphics:
Device-1: Intel Xeon E3-1200 v2/3rd Gen Core processor Graphics driver: i915 v: kernel
Display: server: No display server data found. Headless machine? tty: 177x61
API: N/A Message: No API data available in console. Headless machine?
Network:
Device-1: Intel 82579V Gigabit Network driver: e1000e
Device-2: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet driver: atl1c
Drives:
Local Storage: total: raw: 29.55 TiB usable: 15.27 TiB used: 8.74 TiB (57.2%)
Info:
Processes: 246 Uptime: 33d 18h 45m Memory: total: 16 GiB available: 15.51 GiB
used: 13.18 GiB (84.9%) igpu: 64 MiB Init: systemd target: multi-user (3) Shell: Zsh
inxi: 3.3.31