What’s Going on with Open Source and Standards?
By Keir Whitaker @keirwhitaker
19 April 2010 | Category: Code
At Future of Web Apps Miami 2010 David Recordon of Facebook took time out to share his thoughts on recent developments in the open source community and why they are important to the future of the web.
In this conversation with Ryan Carson of Carsonified he talks about Tornado, HipHop, Three20, PubSubHubbub and OAuth 2.0. A full transcript is available below.
Transcript
Ryan: The next session we have is actually a conversation between myself and a very talented man named David Recordon. David is leading the open source and standards movement at Facebook, and they’re doing a lot of very interesting things there that we want to let you know about. They’re not sponsoring the show. It’s not because we’re saying that they’re paying us at all.
There are a lot of interesting things that they’re doing that we feel you guys would benefit from. We’ll also talk a bit about where things are going with open source in general. Please give David a big welcome. Come on up, David.
I think we’re going to go over some topics we thought were interesting first. David is going to explain some current projects that Facebook are doing and then we’re going to have a little chat.
David: I joined Facebook about six months ago, and it’s been sort of non-stop since then. Some of it overlaps with what we’re talking about, some of it doesn’t. The first week I got there it was a few weeks after the FriendFeed acquisition. FriendFeed had an amazing piece of technology, a web server called Tornado, which we’ll talk about later. It was like help us open source this and help us do a really great job.
Since then, all sorts of sites are using it, and a few weeks ago we released a piece of technology called HipHop, which a team had been working on for about two and a half years. It focuses to PHP performance so it’s something we think is really valuable to all sorts of different people. It transforms your PHP source code into C++, compiles it using G++, produce binaries, reducing our CPU usage by about 50% on average.
Ryan: Just so the audience knows, what is the URL for all the open source projects?
David: For that sort of stuff, facebook.com/opensource, but I think there are a bunch of different things. Like identity, these are things which extend beyond just Facebook so that’s a lot of where my team is focusing, really how do we create technologies with others as well? How do we continue building on top of platforms like Facebook platform and connect with developers and startups? These are a few of the things I’m really interested in right now.
If you’ve ever seen my talk before, you’ve probably heard me talk about identity. I think we’re finally at a point where, for a startup, it’s really worth taking advantage of and investing. You’re able to go, whether it’s using Facebook Connect, whether it’s using OpenID with Google, Yahoo, or MySpace, or using Twitter OAuth, you’re able to go and get users signed up really quickly.
You’re able to get useful information about those users, able to have interactions with them so it’s not just that I want to sign up for your website, fill out this giant form, but it’s actually an ability to go and get people using your website much faster, which I think is very valuable from a business perspective as well.
Ryan: I agree. Let’s go through a couple of these. The first one is – I can’t read.
David: Three20 – this is something, how many of you have used Facebook’s iPhone app? All of you have used Three20. It’s an open source library that is responsible for a lot of the user interface aspects of our iPhone app, as well as a lot of the data accessors. It’s one of those things, if you’re building an application for iPhone, it gives you a giant jump start in terms of creating ways to have dialogs on the phone, have more complex dialogs.
I assume you’ve scrolled through your address book, so one of the things Three20 gives you is a scrolling list with the ability to pull photos from a remote server. This allows you – or what we use it for – is to have that list scrolling of your friends, we’re able to go pull their profile pictures from the Internet and have that something that’s just taken care of.
I really think Three20 is another of those open source technologies where if you’re a startup, being able to build on top of something really helps you get started that much faster.
Ryan: You might as well use it. Next, we have Tornado
David: Tornado – FriendFeed is written entirely in Python. Some of that team was also involved in App Engine at Google, so they were spending a lot of time working with infrastructure there. They created Tornado, which is like a web framework, a really high performance web server for Python, and its thing is being able to serve thousands of simultaneous requests, on relatively cheap hardware.
I think this is where we’ll go a bit further; it’s coming out of this idea of really creating real-time applications. If you’re on FriendFeed or Facebook, this feed is constantly updated. That’s using a technique where your browser is actually keeping a connection open to the server, which means when you think about a traditional website, user requests something and the server responds, then that connection is closed and you move onto the next user.
In this case, you’re actually keeping those connections open. Tornado is being used by Brizzly as well as a bunch of other people. I think it’s a technology where if you’re looking at building a web application, it’s another one of those that really gives you a good jumpstart in terms of getting started faster, being able to scale faster, and having something else you don’t need to worry about from an infrastructure.
Ryan: One of the things that David and I also chatted about is the fact that if you’re building a real-time service, which a lot of people are nowadays, these are just tools that have done a lot of the hard work for you. Facebook and FriendFeed, they have the engineering resources to build this stuff and you can benefit from it. I think we’re going to talk about PubSubHubbub later, as well.
David: I think that might actually be the next slide. PubSubHubbub is definitely named by an engineer, but we’ll get Alex Hunter talking to him a bit later. PubSubHubbub, if you’re creating content I think this is something that will be really interesting. PubSubHubbub is actually a standard. It’s not a piece of open source software. There are a bunch of open source implementations of it, but it was created by a few guys at Google originally and it’s a way to make feeds real time.
The analogy that one of those engineers often uses is how many times have you been on a road trip? You have that kid in the backseat saying, “Are we there yet? Are we there yet? Are we there yet?” If you’ve built an application that is consuming feeds, or if you’re publishing feeds, you know that’s sort of how the web works today. People are checking every fifteen minutes, pulling your RSS feeds to see if there is something new. His analogy with PubSubHubbub is that when you add that to your feeds, instead of the kid asking “Are we there yet…” you as a parent can say, “Shut up; I’ll tell you when we’re there.”
Ryan: Which is not nice. [Laughter]
David: Not nice, don’t do that, but that’s the idea. With PubSubHubbub, you’re creating content, you have feeds, you find a new feed. This is who to say I want to know when there are changes from, and that content will be pushed to you. That’s really useful in terms of being able to get content you’re creating distributed around the web much faster. It’s been implemented in a few dozen applications; things like Google Reader are now taking advantage of it as well so that if you subscribe to a blog in Reader, which is enabled with PubSubHubbub, you’re able to see that just about instantly when things publish. I think we also support it in FriendFeed.
Ryan: The last one, OAuth 2.0.
David: OAuth 2.0, how many people here have played with OAuth? OAuth is really interesting technology. It’s sort of like a valet key for the web. Instead of giving your password away when you’re accessing applications, you can use OAuth so you don’t have to give your password to all these applications. It was developed over the past 2 or 3 years. I think this is one of the themes that’s worth talking about, this idea of how do we modernize technologies, how do we evolve technologies?
One of the things that we’re really trying to do with OAuth 2.0, which is really just starting in the past few months, is a technology called Rab, is it makes OAuth much simpler. It’s gotten to the point where with this next version, we’re shooting for being able to make an API request in like lines of code by relying on SSL for doing a lot of that heavy lifting from a security perspective, whereas you might be ten times that amount of code with OAuth 1.0 today.
I think this is actually really interesting, this idea of how do you go and take something that was created a few years ago, maybe you just assume this is the status quo and actually would be willing to go reinvent it, re-implement it. I think it’s something that extends beyond technology.
We think about it a lot from a product perspective, in terms of just because you went and built your product in one way, getting a lot of information about how people are really using your product, you can’t be afraid to make changes. I think that’s true both from a product perspective as well as from a technology perspective.
Ryan: Let’s move on a bit and talk about where you think open source is going, specifically on the web, and how should the audience deal with the fact that a lot of new standards like OAuth and PubSubHubbub – is it worth building a web app on top of, should we be afraid that things will change? What does that mean?
David: I think open source is really stable in terms of you’re able to build gigantic websites. A lot of Facebook’s infrastructure is built on open source technology. We’ve released a lot of the technology we’ve created as open source as well. I think the lesson there, especially as a startup, is don’t be afraid of building on top of open source infrastructure. If you’re doing data analysis – if you’re not using Google Analytics or other packages like that but want to do something yourself, there is Apache Hadoop and Hive on top of that, it makes it pretty easy to look at data, to understand how people are using your site.
MySQL, open source database is really stable from a standards perspective. It depends on what you’re talking about. I think we’re now getting to the point where it’s better understood what that product should look like when we’re thinking about identity online, what does it mean in terms of being able to sign into another website, find people, interact with people you already know if you want to, be able to share what you’re doing, an entire virtuous cycle of sharing and connecting.
Ryan: On that note, I think probably some people are uncomfortable with big companies like Facebook or Google having so much – having their hooks into certain projects so deeply. Can people in the audience get involved with those projects? Are they truly open source?
David: A lot of these really are. In terms of SubPubHubbub, it’s a bunch of people on a mailing list. You have individuals who are hacking up libraries. Somebody just announced another PHP library this morning, as well as some of the largest companies on the Internet. A lot of these technologies are that way.
I think that’s what I spend a lot of my time really thinking about at Facebook. How do we do a better job of continuing to promote that, getting involved in more of these communities, working with more and more developers, and overall making things simple? I think that’s a huge thing when we look at how are technologies going to evolve from the developers’ standpoint to really make things as simple as possible.
From a product perspective, like the PayPal demo – it’s now really simple to accept payments inside of a mobile app you’re building. That’s making products simpler, and that’s also really important from a technology perspective, as well.
Ryan: Great, I think that’s all we have time for now. David is here. There is a lot of knowledge here as far as open source projects that Facebook are supporting that you guys could really benefit from. The URL is facebook.com/opensource.
Please give David a big hand.
David: Thank you.
Follow @thinkvitamin on Twitter Please check out Treehouse
