wiki:WikiStart

The Bring-Your-Own-Storage Web

Today, data a user generates on the Web is siloed across multiple Web applications. Each Web application acts as the repository and gate-keeper for all data submitted to it. A user does not own this data, nor does she control how it gets used.

In the Bring-Your-Own-Storage (BYOS) Web, a user brings her own scalable cloud storage service to the Web application. Instead of uploading data into the application's servers, she puts it into her cloud storage service, and lets the application access it on her terms. The application does not host or serve data directly, but indexes data across its users' cloud storage services. In doing so, it decouples the presentation layer from the storage layer on the Web: the user's storage service hosts and delivers the data, and application lets users interpret the data without ceding its ownership and control.

Example: Webmail

If you use  Google Mail (Gmail) or similar, your e-mail is kept on Google's servers. Google ultimately owns your e-mails---Google can read them, delete them, and share them with people other than the intended recipients. There is no ownership nor control of your mail beyond what Google decides to give you. This is because Google owns both the storage and presentation layers for their webmail.

In BYOS webmail, you bring your own cloud storage when you sign up. When you compose e-mail, it gets saved to your cloud storage instead of the webmail servers. All the webmail service does is give you a UI for composing and reading mail hosted in cloud storage, and for informing users when you send them mail so they can go download it from your cloud storage. Since you own the data in your cloud storage, you own your emails and control how they get used independent of the webmail service.

What is Syndicate?

Syndicate is a virtual private cloud storage service, intended to facilitate BYOS Web applications. With Syndicate, a user pools multiple cloud storage and CDN accounts from existing services to create a virtual, always-on, scalable data repository called a Volume. Syndicate implements a trusted cloud storage abstraction layer for each Volume that keeps data consistent across the underlying services, and enforces end-to-end encryption to keep data private from all but the user's intended readers. Users put their Web data into their Volumes, and applications discover, read, and write to it with their permission.

The key academic contribution of Syndicate is a novel consistency protocol employed in both the data and control planes. It lets Syndicate defeat the weak consistency semantics of existing CDNs while continuing to use them to for scalable reads. It also lets Syndicate distribute certificates, signed code, and cryptographic keys to a scalable number of end-points, while simultaneously letting the user control trust between components as she sees fit.

Check out our  extended abstract for a technical overview of the Syndicate architecture.

Is Syndicate available for general use?

We are in the process of rolling out a public beta on  PlanetLab,  VINI, and  VICCI. VINI and VICCI will run a private CDN for Syndicate's use, and PlanetLab users will have the option to create a Syndicate Volume for their slice. Each PlanetLab VM (sliver) will access its Volume via a FUSE filesystem, so any data one sliver writes into Syndicate will be visible and readable by the rest of the slivers in the slice.

Can I get the Syndicate source code?

Currently in a private  GitHub repository. It will be released under the Apache 2.0 license when we make our first release.

Who's behind Syndicate?

 Jude Nelson, architect and primary developer

 Muneeb Ali, developer summer 2013

 Wathsala Vithanage, developer summer 2013

 John Whelchel, developer summer 2013

 Larry Peterson, adviser

Attachments