1st Spark: Matei Zaharia gives the first talk about Apache Spark, hosted by Alexy Khrabrov
📅 Aug 11, 2026FunctionalTV22 MIN15 VIEWS28 SEGMENTS · 4 SPEAKERS
On 12/12/2011, Alexy Khrabrov hosted the very first talk by Matei Zaharia at the Scala for Startups meetup at Klout. Alexy founded Scala for Startups and later merged it with SF Scala to build the world's largest Scala meetup. After hosting Matei, Alexy proposed establishing a new meetup, the Bay Area Spark Meetup, which they co-founded.
This is the legendary Berkeley vs Stanford run on the whole Wikipedia over HDFS with Spark.
What Matei Zaharia said
Written from the verified transcript and checked against it. Every figure links to the moment it was said.
Matei Zaharia demonstrated Apache Spark, an open-source in-memory computing framework, at a conference. He showed a 50 GB plain text file processed on 120 nodes in about 20 seconds, with repeated queries on in-memory data returning in under a second. He noted Spark is BSD-licensed, with releases starting in May and an active Google Groups user list. Zaharia said Spark will run on Hadoop next generation by grabbing nodes and launching Mesos, though Hadoop nextG is not designed for small tasks like the 300 tasks finishing in 0.8 seconds in his demo. He estimated Shark, a SQL interface, would have an alpha in the next few months, possibly January or February. He also discussed using MPI as an alternative, noting it is lower level. The talk included user presentations from Timothy Hunter on machine learning and Dip from Conviva on video analytics.
Key takeaways
Spark processes 50 GB on 120 nodes in about 20 seconds, with in-memory queries under a second.
Spark is BSD-licensed, with releases starting in May and an active user list on Google Groups.
Spark will run on Hadoop next generation by launching Mesos on grabbed nodes.
Shark alpha expected in the next few months, possibly January or February.
And you know it's going along and it will eventually hopefully give the same result.
Oh yeah, I didn't say that. So it's about 50 gigabytes, the plain text file. And then we're pulling on 120 nodes. So it's about 2.5.0. Yeah. So this took again about 20 seconds and it's similar performance to what you'd get with Hadoop. But now the data is in memory. So now if we run this again on the in-memory data, you get the answer back in less than a second. And this is really the advantage of having an in-memory data store. And now since the data is in memory we can ask all kinds of other questions about it as we want. So the question I always ask because I always know the answer is this one. So you can guess. So Berkeley was 13,000. How many will Stanford have? Oh, too bad.
And yeah, we can also do, you know, let's just be fair to East Coast brethren. Oh man. Okay. So this is the place to be. And just to finish this up, let's just see. So all the articles about Berkeley must be about the amazing research that's going on there, right? So let's just grab the titles of all the articles that mention Berkeley. So we can do that with the collect operation. Gives you back a data set as an array. And let's just look at some of the titles. Okay, so here they are. So where is some of the big data stuff? So we have exabytes. That's good. We have history of the hippie movement. We have list of free software lawsuits. And there are actually a lot of people named Berkeley so it's not all about the university but this is some of the stuff you can do using this.
Okay. So that's essentially it for my part of the talk. Last thing I want to do before we switch to someone else is just to say that this is open source. It's BSD licensed. We actually started putting out releases in May and we have a pretty active user list on Google Groups and we encourage you to try it out. And next you'll hear from Tim who's been using this for machine learning.
We'll have this on the next generation. So Hadoop next generation also has functionality to let you run multiple computing frameworks on the same cluster. And the way we're going to do that is that we'll actually grab some nodes from Hadoop nextG and launch Mesos on it and have Spark run in there. And the technical reason for that is that Hadoop nextG isn't really designed for small tasks. In the demo I did, you had 300 tasks finishing in, you know, 0.8 seconds. So each task is a few hundred milliseconds. And it's just the way that it's architected, you can't really have tiny tasks, but you'll be able to run it on nextG. Yeah.
When Shark is expected, right? So yeah, you know if you ask the professors who are very excited about it, they say like January or February. I want to be nice to the people that are working on it and say maybe a little later than that, but we do want to put out an alpha in the next few months.
Zaharia, M. (2026, August 11). 1st Spark: Matei Zaharia gives the first talk about Apache Spark, hosted by Alexy Khrabrov [Interview transcript]. FunctionalTV. CEOInterviews.AI. https://ceointerviews.ai/interview/1194694/
MLA
Matei Zaharia. "1st Spark: Matei Zaharia gives the first talk about Apache Spark, hosted by Alexy Khrabrov." FunctionalTV, 11 Aug. 2026. Transcript, CEOInterviews.AI, https://ceointerviews.ai/interview/1194694/.
BibTeX
@misc{zaharia2026_1194694,
author = {Matei Zaharia},
title = {1st Spark: Matei Zaharia gives the first talk about Apache Spark, hosted by Alexy Khrabrov},
howpublished = {Interview transcript, FunctionalTV. CEOInterviews.AI},
year = {2026},
month = {aug},
url = {https://ceointerviews.ai/interview/1194694/},
note = {Speaker-attributed transcript with timestamps}
}
