?

Log in

No account? Create an account

Performance, It is about Caching Today

« previous entry | next entry »
Mar. 1st, 2008 | 08:35 am

CPU evolution has been about getting faster.

Toss in new instructions that do the work of four old instructions, into one.

Gamble on the outcome, branch prediction.

More layers of caching.

When I cook I take the ingredients out of the fridge and place them on the counter where I will do the prep. From there they move to the flat area next to the stove.

I am caching so I can make food more quickly.

In computing we do a lot of caching today.

This came out of the specs from a user who I have been talking to:

"80 nodes, 640 cores... with 21 nodes in our Lustre cluster serving 65T... which we can sustain ~2.5GB/s to"

Problem was?

It was not fast enough for their needs. The system worked well enough for "containing the data", but for executing on the data it could not keep up.

Solution?

Today they have a 100 nodes with 2gigs of ram a piece. They are using libmemcached to access their memcached servers on the nodes.

Classical web application?

No their business has nothing to do with the web :)

They do geo exploration.

Back a year ago there was a great quote, that I cannot find, from the CIO at Walmart.

What impression did it leave me with? That Walmart realized that they need to be more like a YouTube, and less like Big Iron 90's company.

Link | Leave a comment |

Comments {7}

Artur Bergman

(no subject)

from: crucially
date: Mar. 1st, 2008 05:36 pm (UTC)
Link

You have good experiences with lustre?

Reply | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Mar. 1st, 2008 05:40 pm (UTC)
Link

I have no personal experience. I can ask someone I know who has deployed it if that would mind me connecting you with them.

Reply | Parent | Thread

Artur Bergman

(no subject)

from: crucially
date: Mar. 1st, 2008 05:42 pm (UTC)
Link

Please do.

Thanks!

Reply | Parent | Thread

Brian "Krow" Aker

(no subject)

from: krow
date: Mar. 1st, 2008 06:22 pm (UTC)
Link

Mail sent, when he replies to me I'll drop you a note (he is down under so I suspect it will only take a day)

Reply | Parent | Thread