Adventures in Chia

Chia 2.1.1 released. All disks are full of uncompressed plots. I need to get a Nvidia GPU with 8GB (or more) vRAM and replot to get some compressed plots. Farm currently consists of: Primary Storage server (Ryzen 5600X, 128GB RAM, 10 x 16TB HDD in raidz2). Has 1060 plots. Secondary storage server (Core i3-8100U, 64GB RAM, 7 x 8TB HDD in raidz1). Has 397 plots. Pi storage server #1 (Pi4, 4GB RAM, 2 x 16TB HDD in btrfs single mode). Has 290 plots. Pi storage server #2 (Pi4, 4GB RAM, 2 x 16TB HDD in btrfs single mode). Has 290 plots. That gives 2037 plots taking 201.6TiB. I also have a plotting machine that is a full node and mainly for plotting. It doesn't have a discreet GPU at the moment. I could add some more USB connected HDD to the Pi's fairly easily but have to organise another power board for all the power adapters. I have some 10TB Seagate Expansion drives and a USB 3 hub available.

Shortly after Chia 2 was released they had a Chia 2.0.1 release which corrects an issue with computing invalid compressed plots due to passing incorrect parameters to the Bladebit plotter. I have filled all the disks at this point. I might get some more Seagate Expansion drives to plug into the Pi4's but I would rather create compressed plots if I could. Unfortunately I don't have a graphics card with 8GB of memory. The ones I have only have 6GB of video memory and I don't want to buy a GPU just for Chia plotting. Hopefully they we have an updated plotter out soon that I can use for compressed plots, in the mean time I am simply farming. My main storage server is currently run via an nfs share so its latency is worse than even the Raspberry Pi4's running a harvester. The full node runs a local harvester instance and accesses the plots over a network share. I used this to initially get the plots across but haven't yet switched it to run as a harvester.

Chia 2.0 was released on the 24th of August with support for compressed plots. Unfortunately you will need 416GB of free memory to compute them on CPU or you'll need a Nvidia GPU with CUDA support. Most desktop CPU's only support 128GB of memory so that means a server grade machine or a GPU. Hopefully a later release of the Bladebit plotter will support disk-based plots with compression. After upgrading my full node to 2.0 all my harvesters were unable to communicate so I had to upgrade  them to 2.0 as well. To get support for compressed plots you need to tell the harvester(s) the parallel_cpu count and the number of threads they can use. Unfortunately there is no configuration option to do this via the command line so you'll need to manually edit config.yaml if you don't have access to the GUI. Apart from the additional CPU usage expect an additional 500MB of memory use on each harvester. One of the new features in the new GUI is latency stats for each harvester. My P...

Time for an update. I have 1562 plots taking 154.6 TiB scattered across 4 machines. Primary storage server Ryzen 5600X with 128GB of ECC memory 10GbE network card 8 port SAS controller in HBA mode 10 x 16TB HDD Secondary storage server Intel i3-8100U with 64GB of ECC memory 10GbE network card 4 port SATA card 7 x 8TB HDD Pi storage servers x2 Raspberry Pi4 4GB 2 x 18TB HDD via its USB 3 ports I have filled the two Pi storage servers with 290 plots each. They have a pair of Seagate Expansion 18TB drives using the Pi's USB3 ports. I left around 500GB free space on them but after running a btrfs defrag they now have no free space. It should have released the space after each file. Regardless I am running Chia in harvester mode on each one. I could create a plot in 40 minutes but it takes another 20 minutes to get the file onto the Pi due to the 1 gigabit network port. I have filled out the secondary storage server. Its got 397 plots. At the moment I am filling out the primary storage...

I added a 2nd Raspberry Pi harvester into the mix. Its the same specs as the other one, that is a Pi4 4GB with a couple of 16TB Seagate Expansion drives plugged into the USB3 ports and running btrfs. With the drives in a "single" configuration I can get around 298 plots on them. Once I have it filled I will install the Chia software and run it as a harvester. To get the drives filled I transfer the completed plots onto it as a nfs drive. Once filled I stop running the nfs server on it and its just running the harvester. This allows one to have a low-power harvester. I could use the USB2 ports as well and get 4 drives. I'm not sure if/how it would effect the performance. I could use a USB hub as well but given the Pi has two USB3 ports this will do for the moment. I don't know how many drives a single Pi could support, but it works fine with just the two drives. I have over 1000 plots at the moment and the Chia GUI is telling me I am still 2 months off winning, so it l...

The main storage server I built which has 10 x WD Red 16TB drives is having disk issues. I had one spare drive I purchased at the time. After getting a bunch of disk errors recently I bought 3 more drives as spares. The first spare I put in refused to spin up. The second drive (brand new out of its anti-static bag) was giving lots of write errors. I'm now using the 3rd spare to replace one of the original disks that reported tons of errors. I can hear disks retrying but because they are so close to each other can't tell which one from the noise. The drives state on the packaging they have a 5 year warranty so I will be sending some back. Recently WD are in the media with people reporting that WD drives have been giving warnings for drives that are older than 3 years (26,280 hours) and need to be replaced.  My original drives have run hours in the 11,000 range. If a drive is working fine there is no reason to replace it based simply on its age. I will look at mixing the drive br...

The replacement PCIe to U2 adapter card arrived and I tried the original Intel P4510 in it. It wouldn't work so I put another P4510 on it and after a few teething troubles its been running since. The U2 drive is used for work space when plotting. Chia released a 1.8.0 client as an urgent security update so I upgraded to it. That broke the Mad Max plotter so I was only able to use the default plotter. They have since released 1.8.1 that fixed it. One nice thing is the 1.8.0 client now tells you when a new version is available. I have been furiously plotting as fast as my one and only plotter can go. I have filled my secondary storage server (7 x 8TB drives in a ZFS raidz config) and a Raspberry Pi (2 x 16TB external drives in a BTRFS single drive config). Both of these are now harvesters. I have brought up the main storage server (10 x 16TB drives in a ZFS raidz2 config giving 106TB available) and I am working on filling it up. There is no sign of compressed plots in the 1.8.1 clien...