April Trip Forecast

I started thinking about trying to look at the forecast for my upcoming road trip while I was at work, but unfortunately it’s still just a little too far out. I’ll be hitting the road the 31st and that was just outside the window of the GFS, with 00/31 just coming into range with the 00z run tonight. I’ll be able to look at the first leg later today/tonight though at least.

ZFS Ponderings

So my TrueNAS has a primary pool of fifteen (3×5 RAIDZ2) drives in an external enclosure. However five of the drives (one VDEV) show as “UNKNOWN” status, though the pool otherwise reports as healthy. However, if I go to attempt to create a new pool, the five drives that should be in the above pool are listed as available. This has me wondering if something is potentially going to go borked up at some point. Should I copy the data off and rebuild the pool from scratch?

Winter Storm 1/8-9/24

So we’re getting a little snow overnight. A quick look at travel conditions at 11:30 PM CST.

Lab Update: Lab Version..3? I lost count..

So, I finally got around to finishing out my plans to try and do a HA setup in my lab, although I had to resort to eBay as the model of server I was using is not available from the refurb shop I was using before (not surprising really, given the age..) so I picked up 2 more R610s to add to the pool and a third to replace the 2950 as my OpnSense box. Initially I was looking for identical CPUs to the existing hosts (X5650s) but I stumbled upon the listing with 2 hosts with X5660s so I opted for those, repurposing the RAM out of the remaining unit as I don’t think I need 128+ GB of RAM for OpnSense. However, as luck would have it, the unit also included what appears to be a 10Gbps NIC in one of the PCIe slots so that could be handy someday.

The 2950 has been fully replaced, and the other new hosts have been added to my XCP-ng pool. I ended up tearing everything apart and redoing all of the port assignments on my switches as I had only planned for three VM hosts, not four. Most everything is up and running now except I came up a few cables short to be able to connect all of the NICs, at least to keep my color coding straight. This should hopefully be resolved this weekend provided my cable order arrives.

On a side note, forecast for Friday suggesting that we may see some severe weather, at least in the far eastern part of the state. Unfortunately I will be at work all day but I guess I should break the gear out this weekend on the off chance something pops up when I’m off work, plus to get ready for the road trip later in April.

It’s Done?

Finally got around to filling out (mostly? apparently it might support more than i thought) the RAM in the TrueNAS box. However, once everything booted back up I cannot get the XCP-ng hosts to connect to it over iSCSI for their shared storage. A simple reboot should not break things this badly, unless there was some glaring patch issue on the XCP-ng side. At the moment since things are still down I’m patching TrueNAS and will see what happens. Unfortunately the documentation on some of these things seems a bit lacking..

Short Storage Update

So one of the WD Red drives in the pool hosting my VMs started displaying errors in TrueNAS (pool degraded) recently. Now, like the previous NAS the errors did seem to go away after a reboot so not sure if this is truly a drive starting to fail or not, but to be on the safe side (drives are getting old) I picked up a few more 6TB IronWolf drives from NewEgg (this time the ‘Pro’ versions, I think the original 5 were just standard IronWolf drives). Prior to their arrival I built a new zvol on the other pool and migrated the VMs across as I would need to remove part of the WD pool to install the additional drives. So now I’m running two five disk RAID-Z2 vdevs, I’m thinking I’ll add one more max (as the JBOD is only 15 bays, anything else requires either 2.5″ drives or an additional JBOD). I haven’t decided on if I should start looking for SSDs to add to this as well. I did pick up 128GB of RAM to add to this system as well since it has plenty of room (I think it maxes out at 768 GB) but haven’t got around to installing it yet.

 

 

Storage Thoughts

So, I’m contemplating making some additions to my TrueNAS box (which was previously discussed here) either in the form of additional RAM or drives. The current pool configuration is the 5 drive RAID-Z2 of the IronWolf drives and the WD Reds have been rearranged into a 3 vdev mirror, with one drive just floating empty. Unfortunately it appears the 6TB IronWolf drives are no longer available so I’m not sure what I’m going to do for adding to that pool at the moment. I was consdering adding a pair of drives to the other pool which houses my VM store, but that pool is now showing one of the drives faulted, I have not been able to determine if the drive is having an issue or if its a repeat of the v1 build where it just goes back to normal after a reboot.

I need to work out a backup mechanism for the VMs, but other than the IronWolf pool I dont have a good one at the moment.

Tuesday Night Project

This was my project on Tuesday night this week. I’ll possibly make another post with a photo or three once I’ve started installing equipment.

So this is a thing…

Yep, definitely did not see this one coming. First in December.

SPC Day 1 Severe Outlook IWN

Moderate risk area carries a 45% hatched for damaging winds. Much of the area also has a 10% tornado probability.

You can walk across the river..

might be a bit of an understatement. The lack of rainfall has led to very low levels on area rivers, here is a look at the measurements around 4 PM this afternoon:

Stage (Feet)Flow Rate (CFS)
Des Moines River / Saylorville Lake / Beaver Creek
Des Moines River near Stratford5.61000
Saylorville Lake836.07
Des Moines River at Saylorville Tail (Johnston / Below Saylorville Dam)3.21870
Beaver Creek Near Grimes3.0344.1
Des Moines River at Des Moines 2nd Avenue15.491370
Des Moines River at Des Moines SE 6th Street (Below Raccoon River)10.231530
Raccoon River / Walnut Creek
North Raccoon River near Perry3.34181
South Raccoon River at Redfield2.49141
Raccoon River at Van Meter2.61377
Raccoon River near West Des Moines26.5323
Raccoon River at Iowa 28 / 63rd Street20.25391
Walnut Creek at Des Moines 63rd Street3.026.80
Raccoon River at Fleur Drive0.71263
Stream gage data via NWS / USGS / US Army Corps of Engineers

Saylorville Lake’s normal pool is 836.00.