Creating.a.Raspberry.Pi-Based.Beowulf.Cluster_v2.pdf

Creating a Raspberry Pi-Based Beowulf Cluster

Joshua Kiepert

Updated: May 22 nd , 2013

Introduction

Raspberry Pis have really taken the embedded Linux community by storm. For those unfamiliar,

however, a Raspberry Pi (RPi) is a small (credit card sized), inexpensive single-board computer that is

capable of running Linux and other lightweight operating systems which run on ARM processors. Figure

1 shows a few details on the RPi capabilities.

Figure 1: Raspberry Pi Model B (512MB RAM)

The RPiCluster project was started a couple months ago in response to a need during my PhD

dissertation research. My research is currently focused on developing a novel data sharing system for

wireless sensor networks to facilitate in-network collaborative processing of sensor data. In the process

of developing this system it became clear that perhaps the most expedient way to test many of the

ideas was to create a distributed simulation rather than developing directly on the final target embedded

hardware. Thus, I began developing a distributed simulation in which each simulation node would

behave like a wireless sensor node (along with inherent communications limitations), and as such,

interact with all other simulation nodes within a LAN. This approach provided true asynchronous

behavior and actual network communication between nodes which enabled better emulation of real

wireless sensor network behavior.

For those who may not have heard of a Beowulf cluster before, a Beowulf cluster is simply a collection

of identical, (typically) commodity computer hardware based systems, networked together and running

some kind of parallel processing software that allows each node in the cluster to share data and

computation. Typically, the parallel programming software is MPI (Message Passing Interface), which

utilizes TCP/IP along with some libraries to allow programmers to create parallel programs that can split

a task into parts suitable to run on multiple machines simultaneously. MPI provides an API that enables

both asynchronous and synchronous process interaction. While my simulation did not necessarily need

a Beowulf cluster, since it simply required a number of computers on a common network, a cluster

environment provided a very convenient development platform for my project thanks to its common file

system and uniform hardware.

Here at Boise State University we have such a cluster in the “MetaGeek Lab” (or Onyx Lab), which is

run by the Computer Science Department. The Onyx cluster currently utilizes 32 nodes, each of which

has an 3.1GHz Intel Xeon E3-1225 quad-core processor and 8GB of RAM. Clearly, this setup has the

potential to offer quite good performance for parallel processing.

So, with Onyx available to me, you may ask why I would build a Beowulf cluster using Raspberry Pis?

Well, there are several reasons. First, while the Onyx cluster has an excellent uptime rating, it could be

taken down for any number of reasons. When you have a project that requires the use of such a cluster

and Onyx is unavailable, there are not really any other options on campus available to students aside

from waiting for it to become available again. The RPiCluster provides another option for continuing

development of projects that require MPI or Java in a cluster environment. Second, RPis provide a

unique feature in that they have external low-level hardware interfaces for embedded systems use,

such as I 2 C, SPI, UART, and GPIO. This is very useful to electrical engineers requiring testing of

embedded hardware on a large scale. Third, having user only access to a cluster is fine if the cluster

has all the necessary tools installed. If not however, you must then work with the cluster administrator

to get things working. Thus, by building my own cluster I could outfit it with anything I might need

directly. Finally, RPis are cheap! The RPi platform has to be one of the cheapest ways to create a

cluster of 32 nodes. The cost for an RPi with an 8GB SD card is ~$45. For comparison, each node in

the Onyx cluster was somewhere between $1,000 and $1,500. So, for near the price of one PC-based

node, we can create a 32 node Raspberry Pi cluster!

Although an inexpensive bill of materials looks great on paper, cheaper parts come with their own set of

downsides. Perhaps the biggest downside is that an RPi is no where near as powerful as a current x86

PC. The RPi has a single-core ARM1176 (ARMv6) processor, running at 700MHz (though overclocking

is supported). Additionally, since the RPi uses an ARM processor, it has a different architecture than

PCs, i.e. ARM vs x86. Thus, any MPI program created originally on x86 must be recompiled when

deployed to the RPiCluster. Fortunately, this issue is not present for java, python, or perl programs.

Finally, because of the limited processing capability, the RPiCluster will not support multiple users

simultaneously using the system very well. As such, it would be necessary to create some kind of time-

sharing system for access if it ever needed to be used in such a capacity.

The overall value proposition is pretty good, particularly if cluster program development is focused on

distributed computing rather than parallel processing. That is, if the programs being developed for the

cluster are distributed in nature, but not terribly CPU intensive. Compute-intensive applications will

need to look elsewhere, as there simply is not enough “horse power” available to make the RPi a

terribly useful choice for cluster computing.

Building the System

There are really only five major components needed for a working cluster: computer hardware, Linux

OS, an MPI library, an ethernet switch, and possibly a router. Of course, we have already chosen the

RPi as the computer hardware. Figure 2 (a) shows the overall network architecture. The system design

includes 32 RPi nodes, 48-port 10/100 switch, Arch Linux ARM, and MPICH3. The bill of materials can

be seen in Figure 2 (b).

(a) RPiCluster Network Architecture

(b) Bill of Materials

Figure 2: Cluster Design

There are several operating systems available as pre-configured Linux images for the RPi: Raspbian

“wheezy” (based on Ubuntu 11.10) and Arch Linux for ARM (aka “alarm”) ( raspberrypi

). Raspbian is

available in both hard and soft float versions. “Hard-float” implies it is compiled with support for the

hardware floating-point unit in the ARM processor, whereas soft-float implies it will use software

libraries to implement floating-point operations. Clearly, hard float is desirable for performance, but for

compatibility reasons soft-float is available. In the case of Raspbian, hard-float is not yet supported by

. org

Oracle’s Java Development Kit (JDK)/Java Virtual Machine (JVM) for ARM. It is worth noting that the

Oracle JDK is not the only Java implementation available, so there is no need to limit performance with

soft-float. The beauty of open source is that you can simply compile the program for your platform even

if the vendor has not taken the time to do so. In this case, OpenJDK is the open source implementation

of Java, and it is available on all Linux distributions, including Raspbian and Arch. Arch LInux for ARM

is only available in the hard-float configuration.

So, which Linux distribution makes sense for a cluster of RPis? This depends greatly on how

comfortable you are with Linux. Raspbian provides a turnkey solution in that it comes pre-installed with

the LXDE (a full desktop environment), and being Debian based, there is a huge set of packages

precompiled and available for the RPi. This is great for getting started with Linux and with the RPi in

general. The downside to this distribution is its weight. With support for so many features to begin with,

there is a staggering amount of daemons running all the time. As such, boot time is much longer than it

has to be, and you have many packages installed that you most likely will never need or use.

Arch LInux on the other hand, takes the minimalist approach. The image is tiny at ~150MB. It boots in

around 10 seconds. The install image has nothing extra included. The default installation provides a

bare bones, minimal environment, that boots to a command line interface (CLI) with network support.

The beauty of this approach is that you can start with the cleanest, fastest setup and only add the

things you need for your application. The downside is you have to be willing to wade through the

learning process of a different (but elegant) approach to Linux, which is found in Arch Linux.

Fortunately, Arch Linux has some of the best organized information out there for learning Linux. An

excellent place to start is the Arch

. I could go on about all the things I

appreciate about Arch Linux, but I digress. As you might have guessed from my previous statements,

and being a “Linux guy” myself (Linux Mint, primarily), I chose Arch Linux for the RPiCluster.

Linux

Beginner

’ s Guide

As for the MPI implementation, I chose MPICH primarily due to my previous familiarity with it through

Parallel Computing classes (OpenMPI is available also). If you are running Raspbian, MPI can be

installed by executing something like:

$ sudo apt-get install mpich2

That’s it. However, on Arch Linux MPICH must be compiled from source, or installed from the AUR

database. I did some looking on AUR, and only found an older version of MPICH (built for an old

version of gcc). So, I went with the compile-from-source method. You can get the source from

mpich

. o rg

. OpenMPI is available in the Arch Linux repositories, and it can be installed as follows:

$ sudo pacman -Syy openmpi

In the case of MPICH, compiling large amounts of source on a 700MHz machine can be tedious. So,

why not utilize our Core i7 Quad-Core to do the job? Enter: QEMU. QEMU is a CPU architecture

emulator and virtualization tool. It so happens that it is capable of emulating an ARM architecture very

similar to the RPi. This allows us to boot an RPi image directly on our x86 or x86_64 system! There is

one caveat, you need an RPI kernel for QEMU that knows how to deal with the QEMU virtualized

hardware which is not normally part of the RPi kernel image. This involves downloading the Linux

kernel source, applying a patch to support the RPi, adding the various kernel features and drivers

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika: