Updated June 30th, 2016.
Most people don’t know this but I have maintained a mirror server for more than 15 years (FreeBSD and Debian, Ubuntu for the last 7 years).
What is a mirror server? A system that keeps local copies of requested files for multiple UNIX-like systems. Click the link above to read what Wikipedia has to say about it.
I created a DNS entry for mirrors.iphouse.net recently so that it will feel more official though it still runs on my personal hardware and storage. The name has always been mirrors.geeks.org. I use this mirror for all of my personal systems and for ipHouse maintained UNIX-like servers when I ran things there.
I did this update as the UNIX-like repositories continue to grow and it was time to increase the storage available. Prior to todays update the old server had 1.7 TiB of usable storage.
Nevin Lyne of EngineHosting (now Arcustech) recently retired some gear and donated it to the cause. Because of him I have a new server online with an upgraded 3.8 TiB of usable storage. THANK YOU NEVIN! (see below for some specs on the new server)
There are 4 repository mirrors on the new server:
FreeBSD (i386, amd64, arm)
Ubuntu (i386, amd64, arm)
Ubuntu Releases (ISO images for i386, amd64, arm)
Debian (i386, amd64, arm, was an official mirror for years)
The server supports FTP and HTTP access with rsync access coming later this summer. I am also thinking of adding HTTPS access but so far none of the repository management systems support HTTPS as a transport protocol.
Drop me a message if you need help setting up your chosen UNIX-like system to use this mirror ahead of others in your download needs.
This server is also a cvsup mirror for FreeBSD systems, you can change the repository name to cvsup-freebsd.geeks.org for your src and ports updates.
The new server is a Dell PowerEdge 2950 with 6 3.5″ disks serviced by a PERC6/i RAID controller running FreeBSD 9-STABLE using ZFS.
We chose RAID6 for the 1 TiB disks as rebuild times on these slower disks can create potential unrecoverable failures in the case of a second disk failing.
The system boots from 2 OCZ RevoDrive cards with 1 bank of each physical card ZFS mirrored together. (overkill)
The other bank from each card is then carved into 2 10 GiB mirrored ZIL devices and the rest of the SSD storage used as L2ARC for higher read performance.
nginx serves as the web server delivering high throughput with very low overhead and latency.
Current usage as of June 30th, 2016
mirrors% df -h | egrep '(^Filesystem|ftp/pub/)' Filesystem Size Used Avail Capacity Mounted on data/ftp/pub/FreeBSD 320G 248G 71G 78% /ftp/pub/FreeBSD data/ftp/pub/debian 790G 585G 204G 74% /ftp/pub/debian data/ftp/pub/ubuntu 1.3T 923G 448G 67% /ftp/pub/ubuntu data/ftp/pub/ubuntu-releases 30G 16G 14G 53% /ftp/pub/ubuntu-releases
No doubt that’s a blazing fast mirror server, but it seems kind of overkill-ish.
That’s what, maybe a 300-400 watt server, possibly more? What I’m thinking is that for a mirror server, that’s a lot… an Atom box and two 5400RPM 4TB drives typically runs around 30-40 watts, or a tenth of yours. So if you’re paying 13c/kWh, that’s costing you maybe $33/month more ((400w-40w)*24h/day*30day/month*$.13/kWh) in electricity. You only have to run that for maybe two years before the Atom box pays for itself.
Now, knowing you, you had lots of fun and tested the heck out of the performance of your box, always good things to do of course, and entertaining for the geek in us. I know our mirror here couldn’t possibly compete with the performance of your box, but on the other hand, it serves stuff up at several hundred megabits per second and that’s mostly because it’s a crummy VM that’s accessing stuff off a NAS NFS share.
I’m kind of curious what your thoughts are about the advantages of going full out like you’ve done vs my conservative approach.
Hi Joe!
You’re funny and I like you (and nice new email address you have! :) but don’t take that as a belittlement.
The drives are WD Green 5400 RPM drives. Memory system is configured for low power, CPUs are configured to do demand based power.
Server is closer to (estimated) 225W but I’ll get a real measurement this week. That’s much higher than 40W, true. I’m hoping the new server uses around the same amount of power as the old server (even though it has more spindles) since the old system was a P4 without any type of power management.
Don’t see how it is overkill – old parts reused is old parts that aren’t tossed in the heap with the rest of the servers we have retired.
Only advantage I see right now is that the server cost close to zero to procure and I expect the power usage to match up with the previous server for a net-gain of zero for power consumption. I can replace the 1 TiB drives later with larger drives and let the PERC6/i rebuild onto them then grow the partitioning scheme as well.
Using ZFS was just some mental gymnastics (use another word for a better meaning) and the RevoDrive cards were unused test items from a previous experiment. So benefits of ZFS (see drive replacement above) included using the SSDs for some type of acceleration but with a very modest increase in power (site says 3W Idle, 8W active).
The advantage? No capital outlay + gymnastics + estimated zero sum power increase over old server.
Quick ‘wc’ of server logs (old and new) show an approximate of ~45K file requests per day over HTTP & FTP combined and ~200 connections a day for CSUP/CVSUP sync by external servers. I’m also going to increase the number of items mirrored (and reengage the Debian folks to make it official again which drives up usage by 3 zeros per day of accesses).