Media Server & NAS project - Part 2 Hardware Update!

As with a lot of projects there are often overruns, mis planning and issues that arise later in the project that necessitate the changing of certain key assumptions. This project is no exception, lucky for me I had the ability to identify some of the issues before hand and was able to get the bulk of my work done in the initial time frame. Here is a rundown of some of the issues I ran into that necessitated me scrambling to find fixes at the expense of rush processing fees/shipping costs/restocking fees.

Problem: Storing lots of media files takes a lot of space

After archiving a few DVD's and using that small (20 or so disks) as a sample I was able to see that my raid array would be a bit on the small side. Also I moved my configuration from a RAID5 array to a RAID6 array. The reason was because I was using consumer grade drives in my setup, which after a bit of research I found had some issues being used in a raid array. Issues like kicking out of the array causing it to be rebuilt over and over :(.

Solution: Buy more stuff!

I picked up two more 1TB hard drives to fill out my enclosure, giving me a total storage capacity of around 6TB when all is told. Also, the two extra drives I picked up were the raid certified drives that do not suffer from the raid issues that the “consumer” drives I got do.
I even briefly considered getting 6 more of the raid versions of the drives just to make sure things were going to run smooth, but decided against it when I found that there was a trick to disable the problematic process on the consumer drives. It is called TLER (Time Limited Error Recovery). From what I understand, on the consumer drives which do NOT have TLER, drive goes into a error recovery mode every now and then as a normal part of the operation of the drive. This is interpreted as a drive failure by many raid utilities (including mdadm, the Linux software raid utilities that I will be using) and so marks the array as degraded. Once the drive comes back on-line after the recovery period the array recognizes it again and begins to rebuild the array. You can enable TYLE, which makes the error recovery period shorter, and prevents the software from marking the array as degraded. I found the explanation at this link: http://episteme.arstechnica.com/eve/forums/a/tpc/f/24609792/m/982006050931 as well as the tools to enable/disable TLER at this link: http://www.hardforum.com/showthread.php?t=1191548

Problem: Processor Doesn't Work in My Motherboard

It has been a bit since I have built a system from the ground up (2-3 years). That being said this mistake is pretty rookie. Basically the processor I chose was not compatible with the motherboard I chose. The processor was one of the new AMD socket AM3 processors and the motherboard only supported up to AM2+. There are lots of reasons why it might have happened, all of which could have been avoided if I had taken a bit more time to research, but such is the way of it.

Solution: Buy More Stuff!

My eventual solution was to get another motherboard, one that supported the AM3 socket. I ended up going with a small form factor (microATX) board since they were generally less expensive and had a lot of the features I was looking for on the board. That was nice in theory but I'll talk about later how that didn't really apply in practice. In the process I accidentally (lets just pretend it was an accident ok?) ordered another processor, when I thought I might go the route of getting a processor to fit into the motherboard. Of course with rush processing and shipping. Good fun!

Problem: eSATA card not compatible with my application

It was pointed out to me in the Ubuntu Forums that the eSATA expansion card that I had selected was not going to wok with my particular setup because it likely did not support port multiplication. This is what makes it possible to plug in one eSATA cable and have it be seen as more then one (usually 4 or 5) devices. This functionality was essential for my project, so the option had to be discarded. The external enclosure I had chosen came with a card that would serve as a stand in the trouble was that the stand in was much slower then what I was expecting. Basically the bandwidth of the card (PCI-Express 1x) was too low to handle the bandwidth of 8 drives. Another eSATA card had to be found.

Solution:Buy More stuff

This was a costly addition to the project. The port multiplier cards run around $200 give or take. Another issue is that they don't all have great documented compatibility with Linux. I ended up getting one and spending most of my setup time wasted on trying to get the crappy drivers that came with it installed on Ubuntu, or RedHat, or OpenSuse... all with no luck. My compromise was to continue setup with the slower 1x card that came with the enclosure, and consequently worked very well if not slow, and deal with it later. Which I did by purchasing another controller and installing. So I'll have to return two eSATA controllers.


So far this project has been fun, and it is starting to get pretty spendy. I've exceeded my basic mental budget by quite a bit, but I should be good and not in need of hardware upgrades for quite some time.

Media Server & NAS project - Part 2 Hardware

I've always liked buying new hardware and assembling new computers, it is my favorite part of a new build. Doing the research and figuring out the right components for my particular situation, just something about it that I have always liked.

I have been formulating my requirements for this particular project for quite some time. Mulling over the benefits of pulling the trigger sooner, or waiting and doing it later.

The Server

The server has always been on the agenda, it is shameful that I don't have a better way to keep my valuable data (read porn) safe from drive failures. As it stands I'm vulnerable if one of my drives fail, sure I may have backups someplace... but I'm not sure how current they are etc. Suffice to say I need something better.

For these reasons mostly, and also for the speed benefits RAID is one of the fundamental requirements for this project. I went back and forth with my self over weather I would be doing RAID in hardware or software, from the research I did it seems that for all but the most taxing scenarios software raid works as well and can be significantly less complicated when it comes time to restore your broke ass system. Add that to the fact that a true hardware raid card with capabilities of handling 4+ drives runs $600+ depending on your features and the choice was pretty obvious.

Originally I had intended the RAID setup to be something we could expand into a bit, however the addition of the media center and the plan to keep archival .ISO files of all of my DVD's means that I'll probably not be able to get by with the 4x1TB hard drives setup in a RAID 5 array (3TB of effective storage). So I added another 2 drives to the 4 giving me 6x1TB drives in a RAID 5 (5TB of total storage). Since the SATA capacity of my board is 6xSATA drives I needed something more for my 1 boot drive, and my 1 DVD drive. I had to pick up a separate SATA expansion card in order to accommodate. So I shall have a total of 8 SATA devices in my setup.

This puts me in a tight spot, I'll probably need to reconsider my setup if I plan to expand any farther. My options become somewhat limited, 6 is my maximum capacity, my original plan of dropping in a few more 1TB drives to expand my capacity doesn't work much past 6. There are other options but they might be a bit overboard.

Luckily for me I found a 4 port SATA adapter that has 2 external eSATA ports. This means I can use an external enclosure in the future to put my extra drives. I've seen these for around the $350.00 range for enclosures that house 8 drives in addition to what I already have.... DAYM that is a lot of hard drives. I'm not there yet, but it is good to know I have those options if I need them.

The rest of the setup is mostly mundane a pretty basic system. One area I did spend a little more was to upgrade the processor a few notches. This was mostly to allow for the ability of trans coding the video into another format at some point (iPhone version anyone?).

Total Cost for the Server:

• ASRock K10N78 AM2+/AM2 NVIDIA GeForce 8200 ATX AMD Motherboard: $67.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16813157159


• 6x Western Digital Caviar Green WD10EADS 1TB 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive $539.94 (6x$89.99) http://www.newegg.com/Product/Product.aspx?Item=N82E16822136317


• CORSAIR CMPSU-400CX 400W ATX12V V2.2 80 PLUS Certified Compatible with Core i7 Power Supply $59.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16817139008


• G.SKILL 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 1066 (PC2 8500) Dual Channel Kit Desktop Memory $54.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16820231166


• AMD Phenom 8650 Toliman 2.3GHz 3 x 512KB L2 Cache 2MB L3 Cache Socket AM2+ 95W Triple-Core Processor $78.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16819103253


• SAMSUNG Black 22X DVD+R 8X DVD+RW 16X DVD+R DL 22X DVD-R 6X DVD-RW 12X DVD-RAM 16X DVD-ROM 48X CD-R 32X CD-RW 48X CD-ROM 2MB Cache SATA 22X DVD Burner $25.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16827151171


• Rosewill RC-218 PCI Express x4 (x8 and x16 slot compatible) SATA II 3G Controller Card/ 4 internal SATA with 2 external eSATA Design $69.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16816132018

Total: $897.88

Media PC

One of the main reasons that got me excited about the media center concept was the very exciting stuff I was seeing out of the NVIDIA VDPAU project on the MythTV mailing lists. Essentially it allows you to lower your overall hardware costs by using your capable graphics chip to do the heavy lifting when it comes to the High Definition Video formats. Traditionally these tasks were pushed onto the CPU which was not optimal and often meant that if you wanted to play HD content you needed a pretty beefy machine.

Well with the new VDPAU technology you can divert your funds of purchasing a beefy machine towards purchasing more hard drive space, or more HD content ;).

To compound the coolness Intel recently launched the ION platform that combines the Atom processor in a super small form factor which should allow you to play HD video without breaking much of a sweat. AWESOME!

One of the major factors in the construction of such front end systems is often the amount of sound that it makes. The ION package will surely help with that, but in addition the other components of the system should be selected while spending close attention to the amount of noise produced. Things like fans, power supplies, and hard drives all make noise when they are operating, so keeping that in mind is important.

Ultimately I went with a disk-less system which I plan to network boot from my server. The front-end should have minimal cooling requirements and pull all of its content from the back-end server.

At this time I decided to NOT get a remote control for the Media PC, maybe that is a mistake, but I figure I can use the keyboard for most everything, and if I find out I truly need a remote then I can pick one up later.

Here is what I came up with for the Media PC:

• ZOTAC IONITX-A-U Atom N330 1.6GHz Dual-Core 441 NVIDIA ION Mini ITX Motherboard/CPU Combo $206.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16813500027


• JETWAY JC-300-B Black Mini-ITX Tower Computer Case 60W Power Supply $69.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16811120015


• G.SKILL 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel Kit Desktop Memory $46.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16820231122


• i-rocks RF-6572-BK Piano Black 104 Normal Keys USB 2.4 GHz RF Wireless Slim Keyboard/Mouse Combo w/ Travel Pouch $42.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16823204012

Total: $366.96

Not bad

Closing Thoughts

The prices I listed above do not include shipping & handling or any rush order processing fees I had applied in order to get all the parts before my long weekend from work over the 4th of July holiday. They also do not include mistakes in ordering that I may or may not have made. On a side note, know anyone who wants to buy an extra 3 core Phenom II AM3 processor? :D (hmmm maybe I'll be doing a PC build here soon.

I'm excited to actually get it all put together and fire it up for the first time. I'm worried that with the amount of time it is going to take for the raid array to build that if (when?) I muck it up it could take me a really long time to do over again. We will have to see. Next will be basic setup tasks of the back-end... I have a feeling that is going to take the longest. It may have to wait until I have the time off from work, though I probably will try and get started to see how it all goes when I get the parts in.

Media Server & NAS project - Part 1 Background

My house has modest computer needs. We have a few computers with what I consider quite a bit of data (maybe 1.5 TB total). Our computer sittuation looks like this:

• Laura's Computer: Mainly used for day to day tasks as well as Photoshop work for her business and her hobbies. She shoots with a Canon 5D and has been doing so for a while. When you figure in scans from her film days she is quickly approaching the 1TB storage limit on her computer. Laura's data is stored on Striped (RAID 1) array of 2x 1TB drives, she is rapidly approaching full on those. I should note, it is Windows Raid, and her desktop runs Vista Ultimate x64, which if my experience with windows RAID means it could be difficult to recover if, for example, we were forced to reformat and reinstall everything. Though it is protected against a single hard drive failure. She does NOT back up her data (I know begin the boo's now).


• Larry's Computer: This is mostly used for day to day stuff. I too dabble in photography, but to a much lesser degree then Laura. I have probably 100-200 GB's of photos. I also have a lot of music. I like to buy CD's and immediately rip them to my computer (I've been bitten before by the random car theft leaving me musicless). Because of my anal retentive OSS nutcase nature I tend to rip things in a lossless open format, so my cd's are ripped into FLAC (Free Lossless Audio Codec). I have some .MP3 files, but for the most part I'm a FLAC man. My music Collection runs about 100GB's. Throw in my willingness to support the Ubuntu project by seeding torrents of the latest various releases, as well as other .iso's I have accumulated over time and you can see I have a mish mash of data, running around 300GB). Most of this data is spread accross 2x 250GB hard drives that operate completly independently of each other. I occassionally backup the pair to a third 400GB drive that I keep in my case, but it is only irregular and manual backups. This is just on my main computer!


• My Laptop: There is not much stored here, because I am frequently reconfiguring and reinstalling things to tweak it a bit, i'm a glutton. I do on occasion copy things down from my main machine to play around with them, so lets say... 100GB here (an overestimate I would say). My laptop has one 250GB SSD drive in it, it is not backed up at all.

As you can see your storage needs are not insignificant. And adding some data redundancy via a recoverable RAID solution seems like a good idea. So that is what I decided to do!

I decided to setup a NAS for our storage needs. I was thinking something that could cover our storage needs for the next 6 months without buying extra hardware, but would allow us to basically drop in extra disks to expand capacity.

I looked at a lot of different options, some ready to go NAS solutions from different companies (DROBO, Netgear, DLink, etc) and finally decided to build my own using a tower case (should give lots of room to add drives) that a friend has been storing for me for a while (Thanks Luke) and some new hardware.

We have been enjoying the Instant watch thingy on Netflix the last few months and I had been thinking how cool it would be to have the same functionality for my own personal DVD collection. So I got to thinking about adding a Media Center PC to the project. I looked at different projects like MythTV (http://www.mythtv.org/) as well as XBMC (http://www.xbmc.org) and decided I would try and throw something together using XBMC, since it did everything I wanted it to do.

Once I decided what I wanted to do I just had to figure out how to make it happen. Next you will see my shopping list for both projects :)