Ask Slashdot: Best Kit For a Home Media Server? 355
First time accepted submitter parkejr writes "I started off building a media library a few years ago with an old PC running Ubuntu. Folders for photos, ogg vorbis music from my CD collection, and x264 encoded mkv movies. I have a high spec machine for encoding, but over the years I've moved the server to a bigger case, with 8 TB of disk capacity, and reverted back to Debian, but still running with the same AMD Sempron processor and 2GB RAM. It's working well, it's also the family mail server, and the kids are starting to use it for network storage, and it runs both link and twonkyserver, but my disks are almost full, and there are no more internal slots. The obvious option to me is to add in a couple of SATA PCI cards, to give me 4 more drives, and buy an externally powered enclosure, but that doesn't feel very elegant. I'm a bit of an amateur, so I'd like some advice. Should I start looking at a rack system? Something that can accommodate, say, 10 3.5" drives (I'm thinking long term, and some redundancy)? Also, what about location — I could run some cat6 to the garage and move it out of the house, in case noise is an issue. Finally, what about file format, file system, and OS/software? I'm currently running with ext3 and Debian Squeeze. Happy with my audio encoding choice, but not sure about x264 and mkv. I'd also consider different media server software, too. Any comments appreciated."
Re:Why? (Score:3, Interesting)
Re:File System (Score:3, Interesting)
Another option with FreeBSD is ZFS, which is pretty sexy.
Re:HP Microserver (Score:5, Interesting)
If 8TB is full, you need to stop the obsessive collection of warez/pr0n/torrentz you are never likely to watch again.
As opposed to the obsessive collecting of physical media that can be scratched and takes up 10x+ the space?
My family since the 80s has amassed literally 10k CD/DVDs as well as almost 100 laserdisc titles. Not to mention a buttload of VHS tapes that we offloaded years ago.
It has all been converted to digital storage. Since it is on multiple RAID 5 devices and I run a cron job that checks the MD5 sigs against a database I know that it is in good condition.
Of course this requires constant rollover of the data from hard drive to hard drive. Half the drives have failed over the years and it has moved between multiple NAS systems. We still have all the data.
In addition to that, we have over 100k family photos collected from all of our relatives scanned and tagged as well.
Our collection is nearing 20 TB. With the low cost of drives we have backups in lead lined containers in safety deposit boxes at two banks. We swap them out every year or so adding to it. I am really looking forward to long term archival storage that is write once and designed to last 100 years plus. I'll pay for that.
Now I know you may be thinking obsession, but we *paid* for it. Paying twice for music or movies is just plain insane and we never fell for the HD/Bluesuck shit they were shoving down our throats. Well my parents did, but Spiderman solved that problem the first time it could not be played because the encryption changed. Since then they are back on DVD only and we are waiting for a HD storage method that does not involve constant Big Brother monitoring and DRM in our houses.
Then there is the most obvious benefit of all. You only have to rip the music or movie one time. Been years since we bought an actual CD, but you get my point.
The convenience of having all of your media at your fingertips without touching physical media is pretty damn nice.
Guess how much storage space you need for thousands of DVD/CDs when they are packed into spindles and put into storage? A heck of lot less than you would expect. Fits in a closet.
Comment removed (Score:5, Interesting)
Re:No reason to change from H.264 (Score:1, Interesting)
Why would you change away from x264 and mkv. They are the industry standards.
Stick with x264. It is open source and industry stanard. Not open licensed from a commercial point of view, but all the software side and standards are completely open.
Ditch .mkv as soon as possible. It's an almost completely unsupported container. Even among software that supposedly supports it there can be compatibility issues. It's popular in the ripping/pirate communities precisely because it's a pain to use. Just getting your videos to work on a regular basis is a mark of distinction.
Switch to a standard .mp4 container. Much better supported on hardware or software. Some day you will want to be able to stream from your server to a thin set-top box or load a file on your kid's phone. On that day .mkv will make you cry.
There are server remuxers floating around that repackage the video and audio streams into a new container. No re-compression or quality loss. You can fold any subtitle files or other extras into the file at the same time.
Re:I was saying maybe a raid card but non raid car (Score:5, Interesting)
useing software raid is ok most boards have about 6 ports so if you want like 10 then maybe a x4 or better pci-e card may be needed.
Or, get an actual server board (this is gonna be a server, right?), like this one [supermicro.com]. That's six SATA ports and 8 SAS ports. If you flash the SAS ROM to the "no-RAID" version, the controller is recognized natively by Linux. In addition, you get lots of PCIe connectivity, a pair of Gigabit Ethernet ports, and IPMI (allowing remote power cycle).
Then, find a full-tower case with lots of 5-1/4" drive bays, and add hot swap bays [newegg.com]. There are smaller versions, as well...just budget what you need for drives.
I use the motherboard I referenced along with an add-on 8-port SATA card (anything supported by Linux would be fine) and two of the drive bays for ten 2TB drives in RAID-10. I boot Fedora off a pair of SSDs in RAID-1 and also have four 2-1/2" 750GB drives in RAID-10. The 10TB array serves iSCSI over 10Gbit Ethernet to ESX systems that hold all my VMs, with the 1.5TB array as local and NFS storage. There's still PCIe slots available if you need more controller cards.
With this setup, the VMs are how everything is accessed, so you can pick whatever OS you want to face client machines.
Re:Mac mini or apple Tv (Score:5, Interesting)
Just because you don't understand the request doesn't mean it's gobbledegooke, it means your knowledge is limited.
Personally, I don't trust any auto-encoding solutions as they easily go haywire. I'd suggest doing that all by hand.
Ext3 is fine, & a rack mount is a necessity. If you want smooth operation of the system, at least 2 network cards are a must (I run 3, 2 bonded for media/SMB, 1 for management & VPN). I'd suggest having a decent 16 port switch in the house & running the 2 (or more) cables to the box.
For DLNA I just run miniDLNA & for torrents I've just set up uTorrent with a web interface. There's very little my desktop actually does.
Re:No reason to change from H.264 (Score:2, Interesting)
Although I do agree that using x264 in mp4 is a good idea, the fact is that MKV (Matroska) is a very well accepted/supported container. You must feel like a complete idiot for claiming that is not.
The issue with some MKV files is that they include UNSUPPORTED video / sound formats. (in case poeple didn't know, MKV is not a video/audio format ... it is a container ... a kind of zip for media files). For example a lot of completely delusional people love to use Ogg Vorbis as sound, which is barely supported outside of a few open source players and even there ... they don't work that well. Then a few crazy people like to mix WMV with Ogg .... and you get garbage.
There is nothing wrong with MKV ... except for "encoders" who can't follow common sense rules.
And about finding a remuxer ..... I would suggest Handbrake (http://handbrake.fr/). It is the best mufti-platform tool ... and it is free. Also, it automatically converts ogg sound into AC3 without losing audio sync (which is a problem with many other tools).
Re:No reason to change from H.264 (Score:5, Interesting)
1) I've never had problems with using Handbreak for chapters. But one anecdote is about as useless as another for this kind of thing. .mp4 but not the subtitles is the PS3, and all indications are that it was deliberate. So fuck Sony and the horse they rode in on (slung underneath). On the other hand, PS3 doesn't support mkv at all
2) Only system I've ever found that supports
3) This is not a good thing if your goal is to play back the content on any system but the one it was made on. Ever gone internet hunting for that one weird codec that you used for a few months a couple years ago? No? Me neither because I'm not dumb enough to think that 'can jam anything into it' is a good thing in a media format.
4) This is... debatable. Both formats are open standards and open source. You can look at the specs and the code for either. The patents for .mp4 are known and need to be licensed if you are a large commercial operation. The patents for mkv are god-knows-what and may or may not get eaten alive the first time that the patents ever become important. No one knows. Pick your poison.
Re:No reason to change from H.264 (Score:5, Interesting)
While Handbrake is an awesome piece of software, it's not a remuxer. It doesn't support passthrough for the video streams. They will always be re-compressed. Often with little or no loss of visible quality, but some loss will occur.
Re:Mac mini or apple Tv (Score:5, Interesting)
unraid sounds just like what I was *designing* for myself (doh!).
the idea of lots of disks always spinning is just moronic for home-users! we are NOT a data center! the game is different, here; and it has taken the industry a while to learn this.
I had a gazillion disks spinning in big-time home raid. for years. blech! lots of noise, heat, failure modes and like you said, you lose too much, you lose it all! the idea of being able to take a single spinner out, mount it on a 'dumb' system and read it, that's super powerful! don't discount that, folks! its worth its weight in gold.
my approach is to just be an idiot human (I do that part well) and save lots of copies of the files to lots of places. I do that anyway. then run a database job that will traverse your filesystem, get every file's size, date, md5 hash and any other tag info you want to peek inside, for. and run a smart differ on it. keep 'n' copies; report partial copies; trim the ones that are beyond what you need (if you want 3 copies, kill any 4,5,etc version you see). use ANY format you want for the disks, ntfs, ext3, jfs, whatever. each disk has a disk-id ('blkid' on linux shows this easily) and so each file has a disk-id to show where its located.
the final part of this is to keep all disks spun down and have the database know which disk the file is on (the one you want to watch/play) and have it do power-mgmt (as I call it) and spin the drive UP. either have it auto-spin down or spin down via mgmt when your 'session' is over.
spinning the disks is easy if they are all external esata/usb/fw. even simple x10 powerline relays can do this (my first proto will use this style of distrib tech). for internal drives, you power their molex's on/off via relays, via software control.
noise is stupid! power/heat wastage is stupid! raid is stupid for most of us. lets get beyond the gazillion spinners and be smarter about our large disk collections.
(oblig: I'm using arduinos and linux as the controller on all this)