Saturday, July 20, 2013

Low Power, Small Form Factor, and High Utility Home Server - Part III

Low Power, Small Form Factor, and High Utility
Home Server

Part III

(Be careful.  This definitely voids your warranty.)



Part I:  Results Summary - and some pics
Part II: Back Story - Why the heck would you spend your free time doing this?
Part III: Build Detail
Part IV: Future additions

Part III - Detail:

This section isn't meant to be a step-by-step, how-to document.  It is simply a more detailed description of the various hardware and software components used in this project and some links to various source material I used in the construction.
  • Ubuntu 12.04 LTS - Custom ARM build from miniand.com 
    • I used Linux dd to put the 1gb Lubuntu 720p version on a 32gb uSD card, then expanded the partition to fill the full card.  I subsequently removed (via apt) all the packages associated with the gui so I just had a headless/console SSH/CLI only server.
  • USB to Ethernet 10/100mb Adapter - the firmware and kmod drivers weren't in the image provided by miniand, although finding and adding them weren't that difficult.  Tests with iPerf to my desktop machine on a 100mb switch were close to the theoretical maximum of the 100mb ethernet.  Considering overhead of L2 frame and L3 packet encapsulation, I was happy with the performance of the USB Ethernet adapter.
  • Disabled the WiFi adapter once I got wired ethernet working reliably.  
  • Added some basic management tools:  screen, mc, iperf, webmin, MySQL, Apache2, Perl, etc.
  • Tweaked the fstab to avoid writing too often to the uSD card to avoid wearing it out.
    • changed swappiness to zero - this way we use RAM as much as possible but still allow for swapping if necessary.  I do have a swap partition on uSD, but it's never used.   :)
    • sysctl vm.swappiness=15
  • Added a 320 Gb hard drive with a USB 2.0 enclosure to store the large music (mp3) and video files.  I keep everything in one format so that I don't have to transcode on the fly (especially since the processor is completely incapable of doing it on the fly).  Lots of videophiles scoff at the MP4/M4V container and the H.264/AAC codecs, but I've opted for portability.  I have a mix of receivers that will play the content.  From Full PCs runing Windows 7, various flavors of Linux, iPad, Android Phones, even a LCD TV in the car to keep the kids from driving me nuts on road trips.  All these devices can play 1080p content in H.264/AAC.
  • Everything is powered from the Belkin powered USB hub.  The model I chose is currently discontinued, but any powered, High Speed, 480 Mbps hub will do.  
    • The power supply for the hub:  
      • Input: 110/220VAC
      • Output: 2 Amp, 5V DC switching power supply.  
      • Switching power supplies are great since they are highly efficient - almost always greater than 90% efficient and they don't suck lots of power even when the device is turned off but the power supply plugged into the wall.
    • The MK802's host port is plugged into the MiniUSB port on the hub and is drawing power exclusively from this port as well as serving as the data path for the HDD and the Ethernet NIC.  I've had no issues with under-power situations, reboots, kernel panics, etc.
  • SSH with two factor authentication for non-local IPs.
    • I wanted to be able to SSH into my server from where ever I happened to be.  But I didn't want to just have an open SSH server either.  Enter Google two-factor authentication.  Loosely following this guide, I setup the SSH server so that it prompts for the Google Auth code first when SSH'ing from non-local-lan IP addresses.  If you successfully enter the correct code, it will then prompt you for your user password.  Coupled with autoban, random script kiddies get shutdown pretty quickly.
  • Asterisk v11.4.0 - The default Ubuntu repositories available for ARM (at least on the distribution I am running) didn't have the latest version of Asterisk.  In fact it was two major releases behind. I try to stick to the available .deb packages whenever possible, but this was one of those situation where I couldn't.  I needed to run the latest Asterisk code to keep the Google Voice integration I wanted.  
    • I use Google Voice almost exclusively.  I distribute the number on my business card and to anyone/everyone who asks for my phone number.  Hopefully they keep the service around for another 7 years!  One of the great features is the ability to mark callers or texters as spam.  The next time they call they will get a "out of service" message.  WIN. 
    • With Asterisk, I can setup a voice trunk to Google Voice and use a physical SIP phone to make and receive calls, conference calls, transfer, etc.
    •  After installing all the prerequisites, I compiled the latest version of Asterisk from source on the MK802's ARM proc in Ubuntu.  Following the instructions to compile and install Asterisk and Freepbx (a great web gui for asterisk).  Surprisingly it wasn't all that hard or time consuming.  Everything built correctly with no modifications.  
    • I'm using a Cisco 7945 handset (mainly for it's excellent speaker phone) and for my familiarity with the brand and technology
    Yes, I know...a Cisco Phone, with a Juniper Logo, powered by a Juniper EX2200-C switch, running SIP firmware, connected to Asterisk, and making calls over Google Voice...what can I say?
    UPDATE: I now have a Cisco7940 in my garage that is connected to the rest of the network via a Linksys WRT54G running DD-WRT in "Bridge Client" mode.  This way I don't have to run network cable to my garage and I can have a WiFi, wall mounted phone.
      • The phone is running the latest 8.x SIP firmware.  The 9.x Code wouldn't register properly with Asterisk, so a downgrade to 8.x was necessary.  I haven't noticed any significant bugs and so far it's worked flawlessly.  
      • G.711 u-law is the codec of choice for Google Voice and calls to the PSTN.
      • G.722 is also supported in Asterisk and I've successfully tested this with internal calls to/from the Cisco 7945 and software sip clients including:
      • Voice Quality is Fantastic - I use this everyday I work from home and it's a mission critical part of my job as I live no where near where my customers or coworkers are.  I have never received a complaint regarding voice quality.  The only thing to be careful of is the 3 hour call limit enforced by Google. :)
      • To Do:  Setup OpenVPN on my home router:  I'll be able to VPN into my home network and fire up the CSipSimple app on my android phones and have HiDef G.722 calls back home. (It's kinda nice having a fully featured PBX in your house).
    • I also added Google Voice trunks for my two kids and my wife.
      • They each have their own Google voice number and corresponding extensions.  In/Outbound calls from their extensions or to their Google Voice numbers, use their personal Google voice trunk.
    • With a little python scripting magic and some bash scripting, I was able to set and unset the MWI (Message Waiting Indicator) lamp on the Cisco Phone and the various software SIP Clients.  A cron job runs every 5 minutes and checks Google voice for unread voice mail messages.  If there is one, it will create a dummy file in the appropriate directory on the asterisk voice mail directory.  This fools the phone into thinking there's a voice mail.  Asterisk takes care of lighting the MWI.
    • If there are no unread voice mails, the script will delete any files in the appropriate voice mail directory.  Asterisk takes care of turning off the MWI.
    • As I don't want voice mail on Asterisk (I love Google's voice mail too with Speech to text, email, mp3 download, and ubiquitous access), I don't actually download the mp3.  
  • MiniDLNA - UPnP Video streaming everywhere.
    • The MiniDLNA daemon is a nice (this was another one that I decided to compile from scratch as the latest version (with the features I wanted) weren't available in the apt repositories.  
    • The server and clients use Multicast to announce themselves to the rest of the network, so you may need to monkey with IGMP or IGMP snooping on your L3 devices to get everything working smoothly.  
    • Functionally this allows me to sit down on the couch, take my cell phone out of my pocket and start a UPnP app like BubbleUPnP on android or others on Apple iOS.  Once in the app, I select the device I want to stream from: "The Library" - in my case it's the MK802.  Next I select the device I want to stream to: "The Living Room Television" and the file I want played.  The "to" and "from" devices speak directly to each other via unicast and the file starts streaming.  
      • Full control is handled from any client device: Volume, start, stop, pause, seek, etc.
    • For the less technophillic (or technophobic), I also have a wireless (2.4Ghz RF) keyboard with integrated touch pad attached to the TV MK802IIIs.  This way you can drive Android directly from a keyboard/mouse interface.
  • mpd - Music Player Daemon
    • This isn't fully implemented yet.  The software is installed, working correctly, and performs well as a proof of concept.  
    • I'm still working through the use case and whether or not I will really use it as often as the Asterisk or MiniDLNA features.  Time will tell, but there are a bunch of clients and I'm pretty excited about it.  I'm contemplating using a USB battery pack, TP-Link ML-3020, running OpenWRT, USB Sound Card, and a set of USB Powered Speakers for a home made WiFi boom box.  But that's a project for a different blog post.
Here's a step by step - photo blog of the case build:

All the components - This was how the project ran, on my desk, for about six months.  Eventually I tired of the loose cables.

MK802 - The MK802 comes with it's own power brick with a small barrel connector on the end.  It plugs in right next to the USB connector shown in this photo.  You should note that I'm not using it.  The MK802 is acting as a USB Host (which normally provides power to peripherals), but in this case it is drawing power from the USB hub.

Random close in shot of the external HDD - This was a free usb HDD USB 2.0 HDD enclosure and drive I received at a trade show.  The drive inside is a PATA 2.5 inch 320gb 5200 RPM drive.  It is powered completely from USB.  The drive originally came with the cable you see plugged into it here.  It's a Y-Cable that allows you to power it with 1 Amp at 5V DC from two separate USB ports (0.5A x Two USB ports).  Modern USB ports aren't as limiting, but the USB Specification indicates a maximum of 0.5A @ 5 VDC.

Ooooo, ahhh.  The power supply.  This is now plugged into my Uninterruptable Power Suppy (UPS) so the entire project stays up and powered nicely regardless of any power fluctuations from the Utility. 

Random:  This is a USB battery charger I made a while ago and is still going strong.  I contemplated using this as an additional battery backup.  Just need a USB A Male to USB A Male cable going into the USB hub...instant battery backup.  Alas, I didn't bother.  I still use this as a charger for my cell phone when I'm away from a wall/car charger for too long.

The internals of the battery charger:  On top you have two 18650 Lithium Ion cells wired in series.  I salvaged these from a old laptop battery pack that was "useless".  Usually, even though the pack is dead, there are a few good cells still in the pack.  The trick is extracting them safely. :)   -There is a small circuit board attached to the bottom of one cell for added safety.  The board is the diameter of the cell (18mm) and keeps the cells within their correct tolerances to prolong cell life.  (Limits: High Voltage, low voltage, charge current, discharge current).  Under neath the cells you have three main components.  I purchased these off the shelf from various places.  On the left is the charge circuit: you provide a miniUSB charger (standard stuff) and it takes care of safely charging the cells until full.  In the center is the DC-DC converter.  This takes the voltage of the cells (which varies depending on state of charge) and produces a constant 5.0VDC.  It's rate limited to 1.5 Amps on the input side.  The far right component is simply a USB A Female connector.  This is where you get power from the pack to charge your devices.  There's also a small red quick disconnect so you can remove the battery pack.  The tips of the batteries are dipped in Plastidip to insulate them electrically from the case and the rest of the components.

Standard, 4-Port, USB, powered Hub from Belkin.  The MK802 has to be plugged into the port at the back.  The power cord is a barrel connector.  All the devices are plugged into the front.

Another shot of the USB hub.  Here I'm trying different configurations and layouts in preparation for mounting in the case.  

Upside down - just in case you want to see the labels, part numbers, etc.

MK802 - Here's the back side.  This is the side of the case I'll be removing later for better thermal control.  You'll note the factory drilled holes at the top of the case (furthest from the USB Connector).  This is near the processor.

This is a random box I grabbed off my work's electronic recycling pile.  They pile a bunch of old electronic stuff up and have it hauled away to be disposed of properly.  This is all metal construction (aluminum), quite solid, and had mounting points internally for the circuit board.

Front of the Case.  I have no idea what it did, but it had a couple 1/4" Head phone jacks built in.  I assume it was phone related.

The back side had this "TEL" cut out that matched up perfectly with the USB Ethernet NIC.  The "PWR" hole would also make a good spot to mount a power LED later.


This massive Velcro strip was glued down with industrial strength adhesive.  It took the better part of 5 minutes to remove by hand without scratching the black matte/finish of the case.

The two mounting holes bottom right and top/middle-left allow you to screw mount the whole case to a wall/desk/ceiling/whatever.  The case cover is also held on with six (6) screws.  Quite study.  You could probably drive a truck over this thing.  The right end of the box (in this photo) is not metal.  I'm not sure what it is exactly, but it's definitely some type of composite material.  

You can make out the metal stand-offs that were used to mount a circuit board.  They were limiting the layout inside the case and I needed the extra real estate, so I removed them by bending them off with a pair of channel lock pliers.  The right side of the box (in this photo) is the composite material.  It was also limiting the layout in the box, so I removed the two screws and used a hacksaw to cut off the "extra" piece.  (This photo has the piece still attached.  See later photo for the piece removed)

The upper/top cover.  No modifications made here.

Test fitting the layout and making sure the case closes.

Initial Placement of the power cord.  Meh.  Didn't like it here.

Hmm...does it still work?  Yes!  It's alive.  4.8 Amps here.  Must be booting.

Opening the MK802 - Pop off the top near the USB with a small flat blade screwdriver.

Top removed.

Popping off the back cover.  Note the placement of the retainer clips.  Be gentle and pry slowly - fortunately I didn't break anything, so if I ever want to put it back together...I'm good to go.
Detail shot of the MK802 interior.  Major Chip layout from Left to Right:
WiFi, Two x 512Mb RAM, A10 SOC CPU and GPU, 4Gb NAND Flash

Another test layout.  Since the damn cables were so long I was having trouble fitting everything in there.  Also, the combined height of the USB HDD Enclosure and the USB Hub were too tall for the case by about 3 mm.  This would have caused the case/cover to bend/expand and would apply pressure to the HDD.  I definitely didn't want to apply pressure to a Hard drive spinning at 5200 RPM.

I confirmed with a meter that the case was insulated electrically by the paint, but the remnants of the stand-offs were not.  Sounds like a job for some packing tape!

Lower interior insulated with packing tape.

Removed HDD from the aluminum enclosure.  This gave me and additional ~5 mm of clearance.  This way I could close the case w/o applying pressure on the drive.

Holding everything in place with some 3M Scotch Double Sided Foam.

Can you spare a square?

Three small strips to hold the MK802 in place.

Another test fitting.

Had to add quite a bit more mounting foam to stand up to the pressure of plugging and unplugging the Ethernet cable.

Final mounting.  Everything fit, even with the standard length cables.  I ordered some shorter cables so that I could keep power bearing cables away from the CPU.  Even with the standard length cables, the system ran perfectly under full load.

Not bad.  The case even allows access to the mini HDMI port on the MK802 - just in case you want to run it with a monitor/TV too.


Rookie move:  It wasn't until I took this picture that I realized that the power cord and the Ethernet cable, coming out opposite ends, wouldn't work very well.  Doh.

Fortunately, moving the power out the other side wasn't a big deal.  Again I ensured the power cable was routed as far away from the CPU as possible. 

New view of the front.  If I add a USB Sound Card, I've got plenty of holes to route a 1/8" headphone jack.  :)

Back up and running Asterisk - Providing Google Voice Service to my Cisco 7945!

Next up - Part IV - Future thoughts and plans for project improvement

14 comments:

  1. What additional features or changes would you have done if using the RK3188 chipset (the rockchip successor to the one inside the MK802)?

    For example, this one is slightly bigger but has provisions for a wired ethernet.. so you wouldn't need a usb-ethernet converter.

    http://www.geekbuying.com/item/T-R42-Quad-Core-Google-Android-4-2-2-RK3188-28nm-Cortex-A9-1-6-GHz-Mini-TV-BOX-HDMI-HDD-Player-2G-8G-BT-RJ45-External-Wifi-Antenna-with-IR-Remote-Controller-Black-317424.html

    ReplyDelete
  2. That's an interesting question. Fortunately for me, the MK802 has enough horsepower to do what I want. I am also running apache, MySQL, and perl to generate custom/dynamic web "stuff". The Load average is a bit high: 1.3 for the 5 minute average, but the CPU is generally not taxed at all.

    Bittorrent runs reliably on the MK802 too.

    If I had more CPU like the RK3188, I'd probably consider doing video transcoding on it. :) It'd still be crazy slow, but who cares? Kick off the transcode job and wait a day or so...no big deal. Especially at this low power.

    Alternatively to the T-R42, I'm considering this: http://utilite-computer.com/web/home for the next iteration. $99 including 4x USB 2.0 with 1Amp each, 2x Gig ethernet, audio card, 4gb ram, mSATA SSD, uSD card expansion, a great GPU, 802.11b/g/n, Bluetooth, USB OTG, Two RS232 serial ports, 3W – 8W (depending on system configuration and load). Sounds pretty sweet.

    Definetly getting something like the T-R42 or the Utilite will give you more CPU (and video) horsepower. Either one would definitely make a great "portable desktop". Bring your bluetooth keyboard and mouse (or a mouse/keyboard combo device) and the desktop with you. Plug it into the nearest TV or monitor and you have an instant desktop.

    ReplyDelete
  3. I am kind of working on the same thing with a raspberry. Your post made me order the IIIS model of the mk802 so ;-) more power.

    What I am thinking is: OpenVPN is great. Cisco ipsec VPN in my case. But it is annoying to have to vpn in to receive a call. The solution could be TLS. Make your own certificate and install it on the droid phone. Then you can securely receive and make calls all day long.

    I didn't start playing with it yet but I saw lots of documentation on it. Do you have any experience with it?

    ReplyDelete
    Replies
    1. If I had to do things over again, I'd probably start with a ubuntu/debian vanilla arm build and/or compile the kernel myself. As you can see in the post, the kernel that is in the miniand build doesn't support tun/tap interfaces. So OpenVPN, Cisco IPSec vpn, etc are all non-options with this kernel. A kernel rebuild is still on my to-do list.

      Cisco phone and SSL/TLS VPN: So I've used the Cisco ASA's and their VPN solution for the physical phones. It's all very closed source and proprietary.

      I haven't tried it without an ASA. Do you have any links you can share?

      Delete
    2. Absolutely. I actually finished implementing it yesterday. Had some spare time. First I implemented sip tls and srtp following the guide

      https://wiki.asterisk.org/wiki/display/AST/Secure+Calling+Tutorial

      Pretty basic. Works perfectly with the script in the source. Then I put the server in the dmz on the asa and forwarded 5061 and 1450-1500 after changing rtp ports in the sip.conf. The asa is configured to police connections to the server. This will protect it from dos attacks. I set it to 50 open syn sessions, randomize sequencing etc. Good old MPF.

      Then to protect the server itself from attacks I always used fail2ban on Linux servers in the past. To implement it here, I followed this guide loosely.

      http://sourceforge.net/p/raspbx/discussion/tutorials/thread/6288a838/

      I can see a few blocked hosts so far but nothing serious. The result however is: secure calls from anywhere without the hassle of vpn. Easy enough even for my girlfriend to use ;-)

      Delete
  4. Can you get Dolby DTS audio out of it?
    I own a MK808 stick which with android cannot ouput anything better than stereo, even if I try and enable 5.1.
    I suppose any other android stick has the same limitation.
    Do you get true 5.1 output from the HDMI port?

    ReplyDelete
  5. Hey Vince,
    I'm trying to set up a mk802 as an asterisk server to replace my big dell desktop server at home. I got the same DX MCS7830 usb-ethernet bridge as you. Where did you find the kernel modules for this (and for miniand's distro)? Thanks!

    ReplyDelete
    Replies
    1. raf,

      Great question! :) That one was a bit more difficult...

      I plugged it into an x86 PC (it worked out of the box) to determine what kernel modules loaded (lsmod before and after) and used lsusb to get some more info on the device. Then googled for the ARM versions of the kmods.

      Private message me on google+ and I can send them to you.

      Delete
    2. Hint: you need the following module "mcs7830" which also loads the "usbnet" module.

      Delete
    3. I sent an email to a Vince Loschiavo in G+, not sure if it's you. Still haven't found the driver for this :(

      Delete
    4. Yep, got your hangout/chat. I replied via chat with a link to the kmods. I have them permanently in this directory, although I'm sure I'll have problems with that location after updating kernels....
      /lib/modules/3.0.36-t1+/kernel/drivers/net/usb

      Delete
    5. Got it. Thanks for the help, Vince! Greatly appreciated!

      Delete
  6. Electronic devices are exposed to dangers producing troubles brought on by spikes in addition to surges associated with electrical currents. This specific 6 means surge & overload protection board was designed to safeguard your current devices by injury your current equipment. The item comes with a mains filtration & telephone line security.
    http://www.dueltek.com.au/collections/power-distribution-units/products/surge-protection-board-black

    ReplyDelete
  7. Whats the system user password for ubuntu from miniland? The root password as well please if possible.

    ReplyDelete