Storage server upgrade part 2

This is actually using a number of parts I already had and making a second storage server. Here is the parts list for this build. 

1 x Meshify 2 case (new)

1 x Seasonic 550w modular PSU

1 x Pair of extra drive trays (new)

1 x ASUS H370M motherboard

1 x Intel i3-8100 CPU

1 x Noctua NF-U9S CPU cooler

2 x 32GB DDR4-2666 ECC memory

1 x PCIe x1 SATA card (new)

1 x Intel X550 10GbE NIC

1 x 1TB Samsung M.2 SSD

7 x 8TB WD Red Pro drives

7 x SATA data cables

3 x SATA power cables

I already had the power supply, motherboard, CPU, CPU cooler, memory, network card, M.2 SSD, hard disk drives and SATA cables. The i3-8100 only supports up to 64GB of memory. I had some issues with the memory speed and had to set it to 2400MHz.

I bought a number of PCIe x1 4 port SATA cards off eBay. The motherboard has 6 SATA ports built-in and there are a couple of PCIe x1 slots. Using one of the 4 port SATA cards will allow for up to 10 drives. I don't have enough drive trays though. The shop only had one set of drive trays (they are a 2 pack) so I will need another set if I want to expand. The 4 drives that were in the external enclosure were removed but they were too hot to handle so I had to wait a couple of hours for them to cool down. After installing the drives ZFS found the existing pool of 4 drives and was up and running in minutes.

My next issue was how to expand the ZFS pool to include all 7 drives. For that I had to copy the data to another server. That took half a day. I then deleted the pool, recreated it with all 7 drives and copied the plots back. It took 2 days to get the 24TB of plots back. I pointed the Chia node at the new server and it saw the plots and off it went. Now that I have some free disk space I have resumed plotting. Technically the pool of drives is 50TB, but one needs to leave around 10% free so I will fill it to 45TB.

I have one spare 8TB Toshiba drive that I might add as a spare to the disk pool. That way if one drive fails it can start rebuilding the pool immediately. The four 8TB drives that were in the external disk enclosure have Power On Hours around 13000 at the moment. Given they were so hot I don't expect they will last more than another year or two.

Comments

Post a Comment

Popular posts from this blog

22nd of June

Image
Currently plotting away one plot at a time, using 4 threads (at least in phase 1). This is so as to not wear out the M.2 SSD so fast. I have a farmer or as Chia refers to it a "full node". I have a separate harvester machine (in a Meshify 2 case) that is basically a storage server. It has 6x WD Red 8TB drives at the moment with 60 plots taking around 5.9TiB. I have an M.2 to U.2 adapter on order, that I got on Amazon, but its not expected until next month. Must have sent it by sea.   I ordered one of these from one of my local PC shops in the hope it will turn up sooner. Its a PCIe card that you slot the U.2 drive onto it and then put the card into a PCIe x4 slot. Once this arrives I should be able to remove the M.2 SSD from the farmer and replace it with this and run multiple plots at a time. Its taking about 15 minutes to transfer a single plot from the farmer onto the harvester. I've setup a cron job to run every hour to pick up any completed plots from the local hard
Read more

More harvesting

I have 750 plots at the moment taking up 74TiB according to the the chia software. Storage server I purchased a pair of 32GB ECC memory modules to update the storage server. The machine wouldn't boot with them so I went back to using the pair of 16GB memory modules it had before. I purchased some more 16TB hard disks with a view to replacing the 8TB drives that are in the storage server. I am running ZFS and have 3 pools of drives: 5 x 16TB drives in Raidz (approx 32TB usable) 4 x 8TB drives in Raidz in an external enclosure (approx 24TB) 4 x 8TB drives (approx 24TB) The plan is to make a Raidz2 (two drive redundancy) of 10 x 16TB drives. This should yield approx 128TB usable space. The pool in the external enclosure can remain as is. I might upgrade it to larger drives at some point but don't have any plans at the moment. Due to the USB interface they are almost twice as slow as the drives inside the storage server, even though they are the same make and model 8TB drives. The
Read more

That took a while

I had to copy the plots off the 8TB disks so I could swap them out of the storage machine (aka harvester). That took 2 days, even with a 10GbE network connection between the two storage servers. I then removed the 8TB drives, put the 5 x 16TB drives in and copied the plots back. That took another 2 days with it doing 240MB/sec for most of that, which would mean the storage server simply can't write the data any quicker. After that I fired up the full node which told me 121 out of the 252 plots couldn't be opened. I ended up deleting them and I will have to plot fresh ones. Clearly they didn't survive being copied. It probably would have been quicker to upgrade the disks in the storage server and re-plot everything. At the moment I have put 4 of the 8TB drives into the external disk enclosure and I'm copying some of the remaining plots over to it. I ended up setting up a cron task to run every 10 minutes to write a small file to the disk enclosure in order to prevent the
Read more