Colin Harrington

Tag: AWS

Amazon EC2 High-CPU instances

by on Aug.16, 2008, under Amazon Web Services, Distributed Computing, EC2

At the end of May (May 29th 2008), Amazon announced that Amazon Web Services Customers can now utilize "High-CPU Instances" on EC2.  According to their specs, there are currently 2 versions of their "High-CPU Instances" as described below:

High-CPU Instances

    Instances of this family have proportionally more CPU resources than memory (RAM) and are well suited for compute-intensive applications.

    High-CPU Medium Instance

      1.7 GB of memory
      5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
      350 GB of instance storage
      32-bit platform
      I/O Performance: Moderate
      Price: $0.20 per instance hour
    High-CPU Extra Large Instance

      7 GB of memory
      20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
      1690 GB of instance storage
      64-bit platform
      I/O Performance: High
      Price: $0.80 per instance hour

So the Extra-Large Instance has the computing Power equivalent to 20 EC2 compute units.  This means that CPU bound problems get 2.5 times the performance for the same amount of money.  In a post from earlier this year, I estimated that it would take 3,100,000 CPU hours to crack a 16384 bit RSA key pair based on stats I had found elsewhere.  This came out to be about 38.75 hours (less than a couple days!!) with 10,000 instances and would cost a maximum of $310k (for an insanely large RSA key pair)ie an average of $160k to locate a specific pair.  With the High-CPU instances, it would take approximately 15.5 hours to do the whole computing task from top to bottom.  At 15.5 hours, it would cost $124k or an average of $62k.  This definitely puts some CPU Bound computing jobs in closer reach of those who need it. 

I could only imagine what this would do for CPU bound utilities like Video encoding/transcoding, weather pattern simulators, or large Rendering farms (among many other applications).  I’d love the chance to work with a farm of machines again – Its like having a fleet of robots doing the work in a portion of the time that a traditional desktop could offer.  Photogrammetry, hmmm… Videogrammetry…

Does anyone know of some good Linux based/open Photogrammetry software?

3 Comments :, , more...

Distributed Computing

by on May.13, 2008, under Amazon Web Services, Distributed Computing, EC2, General

CloudWhen I was studying at Bethel College (now Bethel University) located in Arden Hills, Minnesota, I took a class called on Parallel Programming taught by Dr. Brian Turnquist.  I have to say that this class was my favorite.  I would stay up late just to solve the problems and projects that were presented to us.  I loved it!!!

We had a 40 CPU Beowulf cluster that we were able to work with.  It was a pretty standard AMD Dual Processor Configuration on a 10/100mbps ethernet network (which was usually the bottleneck).  Several students had the opportunity to help design and setup the cluster.  The cluster had its own housing inside one of the Computer Science labs. 

We ended up writing C++ programs that utilized MPI to communicate.  We ran calculations, rendered fractals, and simulated breaking passwords in a distributed form; Well maybe not passwords, but finding the seed and depth of how to replicate a series of "random" number’s generated by the stock random number generator could be easily substituted with other code .  I won’t get into how important the RNG (Random Number Generator) is to our modern systems (1,2) but it was a fun exercise none-the-less.  I ended up using the cluster briefly to render some intensive POV-Ray Fractals (See the contest results). 

I’ve always loved the concept of distributed computing.  I was really excited when I learned of Amazon’s Elastic Compute Cloud (EC2).  The concept of Pay as you go applied to Distributed computing is an interesting one!  And having a top-tier datacenter and Simple Storage Services (S3) makes it an attractive solution.  The concept of building scalable web applications is one that has caught my eye. 

I have some good ideas on how to utilize this service but haven’t made time to finish the concepts.  The Amazon Web Services crew have really started to round out ther services with the announcement of Persistent Storage for EC2 and SimpleDB.  Persistent Storage is, in my humble opinion, one of the last things that they needed to solve to service a fully viable, scalable, pay as you go/grow computing platform.  

72 Comments :, , , , , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!