Looking at a new Home Server Build

Working on something, Need advice... Post that here and we can help.
User avatar
Vinster
Site Admin
Posts: 356
Joined: Wed Feb 12, 2020 10:52 pm
Contact:

Re: Looking at a new Home Server Build

Unread post by Vinster »

well the LSI 9260CV-8i came in and it doesnt' have the cachevault power module... sigh... that part is hard to find on it's own... and too expensive when you do find one.. I'll test the card tomorrow then reach out to the seller about it..

otherwise I'll have to get a 9266-8i and get it's cachevault module, that one is easier to get and only $40.... it's the month of shipping that blows... this setup well over doubles what I was hoping. but the 9266 upgrades from PCIe 2.0 to 3.0, and the cache mem goes from 512MB of DDR2 to 1GB of DDR3. so there is a benefit.. just one I didn't think I needed as I believe this Server only supports PCIe 2.0...

I'll post up the software setup once I have the drives running in Hyper-V better.

Vin

EDIT: I was wrong, this server does support PCIe 3.0... so there is a benefit...

User avatar
Vinster
Site Admin
Posts: 356
Joined: Wed Feb 12, 2020 10:52 pm
Contact:

Re: Looking at a new Home Server Build

Unread post by Vinster »

So the card works, I can create an array, but the server doesn't like it. with the card installed I cannot get in to the TD340's bios. the bios hangs... and because of that I can't change the boot drive and the box is dunked... I've tried different slots... with 1 CPU the card needs to be in either Pcie slot 2 or 5, and the server behaves the same in either. with 2 CPUs installed the card should be in slot 4. so that's where I'm stuck.. I'm waiting for the 2nd CPU I ordered. hoping the server behaves differently/better with 2 CPU's and PCIe Slot 4...

tonight if I have time I want to try the card in the 2nd server to see if it behaves the same way...

Now 1 thing I haven't done is verify the bios on the server and update it if it's not the latest.. the 9260CV's firmware does need updating as it is dated 2011 and there was a release in I think 2017 to deal with windows 10... so it could be an UEFI issue.. the server is and the card might not.. though the card worked in a kabylake board/cpu... I don't know.. I'll have to play with it...

as for the ebay vendor. he's sending me $8 USD back.. so $12 CAD.. doesn't get me a card with a proper CacheVault... offered to refund if I send the card back.. but it's $20 to return for me.. so atleast this way I have something... I just need to figure out a use for it.

for software... I might be changing it up....

So a coworker that is very technical showed me his UnRaid Server (Linux based, costs associated for licensing)... really neat stuff and I'm considering it... though my raid setup would be quite different. UnRaid uses a software based array manager. So you just need all the disks to pass-through (HBA or IT mode) and unraid can't make use of onboard caching... unfortunate thing is the 9260CV can't be flashed to do that (IT mode). I'd need to go to a lower end card (9207, 9211, or an IBM 5110) those can be flashed to that mode...

There is good info from a Youtuber called SpaceInvaderOne... talks through setting up a server step by step for Unraid...

So I need to figure it out... cause if I go Unraid I need to change the raid card that I need... if I decide to stick with my current direction then I can get a 9270CV-8i for about $130 and that is the card that is used with this server when you get an IBM 710 Raid card...

So I need to figure that out...

Vin

User avatar
Vinster
Site Admin
Posts: 356
Joined: Wed Feb 12, 2020 10:52 pm
Contact:

Re: Looking at a new Home Server Build

Unread post by Vinster »

Finally making progress...

I installed a second E5-2420 v2 CPU. I then installed the 6x 4GB 1333mhz DDR3 memory from the backup server and then I also installed the 2x 8GB 1600mhz memory from the existing system and it is seeing all 40GB of memory...

Right now I'm running Memtest86+ and I'll let it run over night and see how that goes.

Here is a screenshot of memtest

20220115_175846 (Small).jpg
20220115_175846 (Small).jpg (131.92 KiB) Viewed 18017 times

Here are the different memory sticks... the 8GB are proper Lenovo mem's where the other server had HP memory installed.

20220115_173426 (Small).jpg
20220115_173426 (Small).jpg (146.27 KiB) Viewed 18017 times
20220115_173403 (Small).jpg
20220115_173403 (Small).jpg (154.22 KiB) Viewed 18017 times

Temps look ok overall .. I'll be keeping an eye on it

temps after 2cpu.PNG
temps after 2cpu.PNG (143.75 KiB) Viewed 18017 times

Vin

User avatar
Vinster
Site Admin
Posts: 356
Joined: Wed Feb 12, 2020 10:52 pm
Contact:

Re: Looking at a new Home Server Build

Unread post by Vinster »

another update... this one is a bit long....

I was able to flash the LSI 9260CV-8i to the last firmware I could find online (Dec 2012) and now the server will boot with the card. So that is great news... Bad news is, I don't think I need that card anymore... more down below...

also the 4 port server network card that I ordered was bad (Intel 1000/Pro)... 3 weeks of waiting... seller gave me a refund and I ordered a different card... Intel i350-4T which is the card that was optioned for this server anyway...

Memtest was good... I have 6x 4GB sticks and 2x 8GB sticks installed for a total of 40GB of ram...

So the original intent was to mimic my existing setup but have a modern OS, remote management and redundant power supplies and simplify the setup a bit... But I have changed direction a bit...

so Recap

Current System:
- X58 System, 6 core/24GB RAM, Windows 7 Pro as a host, VMWare Workstation with a Windows Server 2012r2 VM, Windows 10 Tinker Server (minecraft and other apps server) VM for my son.
- Server has 6 drives for home server shares and backup (2TB of usable space.. currently at 80% full)

issues with this setup
- can't do a redundant PSU setup (Define R5 case, can only install 1 PSU).. so when the UPS went bad, she was a goner..
- no low level remote management... it's just a normal Gigabyte X58 board...
- to add more data space... I need to copy all the data (need to do that twice to make sure there is no data loss) and a complete reconfigure the array and put the data back... a server shutdown is also required to do this as I don't have hot-swap capability.
- when I start the server, I have to manually start all the virtual environments... sucks when I'm away and it's only the family at home...
- When I have to shutdown the Server, I have to manually shutdown each VM individually... not doing so the VM's come back from an improper shutdown state...

--------------------------------------------------------------------------------------------------

Planned Setup:
- Lenovo Thinkserver TD340... so built in Remote Management, and redundant PSU's and 8 Hot-Swap trays for 3.5" drives... that that solves 3 big issues quick.
- use the LSI 9260CV-8i for a RAID6 array... this card allows for online array expansion.. so later if I want to add 1 drive or two.. I can. this controller can manage that.
- the drives can be added in the array using the MegaRAID windows utility.. so the server does not need to be shutdown...
- Use Windows 10 Pro, enable Hyper-V in the windows install functions. Then re-create the Windows Server 2012 r2, Windows 10 Pro for my son... VMWare and Hyper-V are not interchangeable and it's just as much work to re-create the VM's as it is to migrate them over...
- - Hyper-V runs as a service, you can set the VM to start with the OS (even delay the VM start) and when you shutdown the host, it manages hibernating the Hyper-V virtual environments...
- I have an 8tb drive that can hold all my data, I was going to assign this as a backup drive to Windows Server 2012 r2...

--------------------------------------------------------------------------------------------------

So this is all well and good... this solves the previous issues and put me in a very better place for future growth... But there is a different way to do this and I'm taking 30 days to try it... just a few days in and so far it exceeds anything I could had thought of doing...

--------------------------------------------------------------------------------------------------

I'm trying UnRAID... https://unraid.net/

This was featured on Linus Tech Tips a few times.. this is the OS they used to make the 1 PC 7 gamers stations system... It lives and loads from a USB flash drive... and lucky for me, my server has a Type-A port right on the Motherboard... So it can be tucked away and safe in the case...

But Unraid is a network attached storage system first... everything else is the topping on the cake...

- It's Linux based.. so you have the ability to use Dockers.
- - If you don't know what a docker is, check this link: https://docs.docker.com/get-started/overview/
- Licensing is pure... you buy a license and it's yours... any version updates that come down the pipes are available to you... no bullshit...
- You can run multiple VM's on it... it can also start on server start and it'll hibernate on server shutdown.
- Unraid managed the array differently that typical RAID5/6.. it's a software raid tool, though it has nearly the same capability... the benefit is the data lives on the drive natively, it's isn't striped across multiple disks.. so if the server dies, pull the drive and pop it in another PC and the data is there.
- so because it manages the drives like a "Storage Pool" you assign a drive for Parity... if you want to mimic RAID6.. then assign a drive to Parity 2.... then you can just add drives to the pool...
- it's got SSD Caching capabilities... so I ordered an NVMe to PCie adapter card.. it hold two NVMe's... I got the nmve's yesterday, the adapter card should arrive today...

- now unraid's initial intent is for you to use cheap or readily available hardware... it's intent is for you to not use a RAID card like the 9260... UNRaid wants native access to drives, just line linux or windows SATA port on the MB... so if you're like me and stuck with a raid card... I have to manually create a single disk virtual disk for EVERY disk on that card.... then the card passes the Virtual Disk (VD) to Unraid... it's a bit of a pain.. the card would have to be a card flashed to 'IT' mode or get an HBA card.... my family of card is an HB card... it doesn't have the 'A'... So I'll eventually have to get a different RAID card.. if you search on ebay for "IT Mode RAID", you can get pre-flashed cards, they are not expensive... and once I'm at my 30 day decision point.. or a "this is what I'm using" then I'll get one and replace this 9260...

--------------------------------------------------------------------------------------------------

Here is where it gets good...
- you can setup SMB usershares built in... so I don't need my Windows Server 2012r2 VM.. unraid can manage it...
- - Though there is 1 big drawback... you cannot create a user group...
- - - So I have to work around the "Parent" group and "Kids" group... I have to manually assign each user to each share....
- - - It does not create a user @Home\usershare by default... you have to create the share and assign the user to it...
- - - Now there are scripts available to do these things... and there is a very large community of people that help....
- with dockers, there is a Minecraft Server Docker... so I don't necessarily need that other Win10Pro VM...
- Hidden shares... In windows server, I could create hidden shares... but if the user on the network had "show hidden folders" enabled in file explorer.. he could see them... with unraid, if I set them as hidden... they are hidden.. with that option enabled in windows they don't come up... you have to type in the path...
- - I tried to do this a long time ago and I recall spending a lot of time and not being successful...
- The 8tb Backup drive... to do a store of the data.. well, unraid does not have a GUI utility to backup to a drive, as it expects you to be dependent to the array...
- - Which is fine... but that's not how I'm wired....
- - There is an application called RSYNC.. it's a command line utility and it does a great job synchronizing data from one location to another... I plan to write a script to copy all the shares to my 8tb disk.. The nice thing is the drive doesn't need to be mounted full time... in the script, I can mount the drive.. copy the data, then un-mount the disk... this is particularly beneficial if you want to protect against randsomware... it usually looks for mounted accessible drives... This would be a weekly or bi-weekly sync.. so the likely hood of this happening is greatly diminished... Note the greatly diminished... Shit can happen... but with this, there is benefit to it.

--------------------------------------------------------------------------------------------------

Here is where my mind went "POOF" I got to try this...
- with the available apps, you can create your own Cloud.. and when you setup your phones photos/docs/etc.. to sync to Google Drive... you have have it sync to your own and that is stored on your unraid server... it uses NextCloud which is free for personal use...

That's just the tip...

I've been wanting to create a PFSense router... this server being up 100% with power fail-over..., I can make a VM for that... and start testing that...

There is a plethora of dockers and apps that are available in the 'apps' section of UnRaid... The Scripting ability if phenomenal if you are in to that... But you can also do quite a lot without any scripting..

This is where I am at the moment... Server is up, shares are available... minus the minecraft server (waiting to install the NVMe's) it's doing everything, I have not YET written one script... I will... but you don't have too...

Once I have the remaining hardware here and I have all my drives installed I'll post up another update then... and at my 30 days I'll post if I buy the software or if I revert back to my original plan.

I also plan to create a thread on where I got the information for the setup... unless you're a Linux server admin, some of this isn't intuitive... but like I've said a few times, there is a lot of help out there... there is a community on their site and a few well versed youtubers on the subject that post great content on the matter.

Vin

User avatar
Vinster
Site Admin
Posts: 356
Joined: Wed Feb 12, 2020 10:52 pm
Contact:

Re: Looking at a new Home Server Build

Unread post by Vinster »

So I'm sticking with Unraid...

my drive setup is as follows

2x 4TB Drives for storage
2x 4TB Drives for Parity, so I can have two drives fail. just like RAID10 or RAID60. But here I can add more storage as I go, I don't have to break and rebuild the raid.. I can just add another disk and go.

I bought a StarTech Dual M.2 PCIe SSD Adapter Card (PEX8M2E2) https://www.amazon.ca/gp/product/B081SJ ... UTF8&psc=1
1x 1TB nvme for VM's
1x 1TB nvme for Dockers and Apps

4x 120GB SATA3 SSD's in a RAID0 for general Caching

1x 8TB drive for a data copy backup

I ended up not using the LSI 9260CV-8i card... I'm looking at getting a generic HBA or an LSI card flashed to IT mode... unfortunately the card I have can't support either of those modes

I also bought a Lenovo I350-T4 Quad Gigabit Network Card. This is so I can separate Server access to the Shares, Dockers and VM's... So now the Server has 6x Gigabit network cards... now I just need to get a Smart Manages Switch so I can use port Teaming... This is so I can plug 2 network ports from the server to the switch, 1 port is used to Send and the other port for Receive. then I'll have essentially a 2GB/s access to the server... I know I won't get that speed from a single user. but the idea is to load share to help avoid network bottlenecks...

I didn't go with 2.5GB, 5GB to 10GB networking because that HW is still very expensive in my area... just a basic 2.5GB 8 port switch is $250.. let alone the cards and other gear... for now this is my best option.

Vin

Post Reply