Curoverse at BarCamp Boston 9
Curoverse engineer Tim Pierce presents Keep at BarCamp Boston 9
BarCamp Boston was held October 11 and 12 at the Microsoft NERD Center in Cambridge! About 300 nerds and geeks from all over New England came on a rainy October weekend to share ideas, tips and tricks. It was a great and exciting weekend. Curoverse co-founder Jonathan Sheffi and I attended to check out the talks and share our work with other local hackers.
An unconference has a great kind of improvised, manic energy that makes it an excellent place to discuss new, interesting and nutty off-the-wall ideas. Which, of course, is ideal for us. :-)
On Saturday, I presented the work we've been doing on Keep. In this session, I explained why content-addressed storage helps solve critical problems of data provenance in scientific computing, presented an overview of Keep's architecture, and reviewed our team's experience porting a large Perl application to Go. The Keep presentation is published on Slideshare: Keep: Open Source Content-Addressed Storage (CC-BY-SA)
Conferences (both the formal and informal kinds) give us a very important opportunity: the chance to have our work criticized by others. That might not seem obvious at first, but we think it's a crucial part of the software engineering process for free software. Every software system needs to be able to stand up to criticism -- if it can't, that's likely to indicate a flaw that needs to be addressed. Conversely, if the software doesn't get thoroughly criticized, any significant flaws in the design are likely to go overlooked. This principle is often expressed as: "with enough eyeballs, all bugs are shallow." It's one of the fundamental advantages that free software has over proprietary software.
But it's important to be mindful that open source isn't a panacea for debugging. Just because the source code is available doesn't mean that it's actively being audited. Engineers and project leaders have to be proactive in seeking out reviews that will help uncover hidden flaws in their systems.
That's why presenting our work in public is important to us. It's not just about telling you how awesome it is. (Although we really think it is!) It's also about finding the flaws that we haven't been able to find ourselves. It's about tearing the work to shreds -- because that's also how we make it even better.
In the coming months, we're looking forward to bringing our work to more conferences for you to review. We'd love to hear your thoughts on Arvados and scientific computing -- both what's great and what's not. And we particularly want to hear about the latter!