Shotton.com

I’ve been lending an occasional hand to Dave Winer on portions of his EC2 for Poets project that is building on the OPML Editor and other tools to create a “coral reef” alternative to Twitter-like services. One of the more thorny issues is how to deal with identity for users and their content across a big federation of systems, operating systems, and platforms.

The short answer is to use one of the ubiquitous “platform” services everyone pretty much has to use on the Internet. Things like email addresses, URLs, domain names, etc. all fall into this category. Email addresses are easy, but disposable, can be easily manipulated, and aren’t so hot from a Twitter-like syntax perspective when prefixed with “@”.

URLs are clunky and hard to remember, often break, get shortened and obfuscated, and require a server at the end of each URL.

Simple domain names, like chuck.shotton.com, would be an awesome alternative to service-specific usernames (e.g. @cshotton on Twitter) because they could be tied to a persistent portion of the Internet infrastructure. The problem is one of ease of use. DNS ju-ju is inscrutable to most folks. If it goes beyond setting up a home page on GoDaddy, it’s into the realm of grey-bearded Internet science projects when it comes to DNS, running a DNS server, and keeping one up to date.

Enter the idea of a REST interface to DNS. Something that makes it drop dead simple to add, change, remove, and otherwise operate the DNS system as an easy-to-use mapping between a simple identity and the location of some other resource. That’s exactly what DNS was designed to do. Unfortunately most people only know it as a way to map friendly domain names into gnarly TCP/IP addresses.

But suppose it mapped friendly domain names into some branch of your online identity? Instead of mapping to an IP address, it could map to a URL to your RSS feed. Instead of using some random, often-changing email address, you could map a fixed name that you kept forever into specific resources on the Internet that you were related to — web sites, Twitter users, RSS or OPML feeds, etc. Much better for the long term, since people can always find you, and you simply need to point your identity at a new resource. TXT records in the DNS system are perfect for this. But the semantics of the TXT strings needs to be defined for this to work.

In all fairness, the market has already produced a service that pretty much allows all this to happen as part of the coral reef. DNSimple.com gives a great set of APIs for managing domains and DNS records through simple Web interfaces that anyone able to tweak a little Javascript can call remotely. Whether it’s the right answer, or if there’s an open source alternative, is one of several questions to resolve.

Here’s what I think the pending questions are that need answers before this solution takes off:

• REST APIs for DNS – what is the best solution for widespread developer access and support?
• DNS TXT records – what standards, if any, already exist for using these DNS fields to map to URLs or other identity methods?
• If nothing exists, what should go in TXT records? (e.g. some XML fragment like <url=”http://shotton.com/feeds.opml” /> ?)

I’m interested in trying to leverage DNSimple or a comparable service to start providing a prototype of this identity mapping through DNS service. Anyone else want to play along?

In case people show up here looking for an old post on rssNimbus, the source code and links to documentation can be found here.

The following tweet yesterday fired off a lightbulb of sorts:

suelyn @doctorow Will digital publishing be the end of bricks & mortar libraries.

Let me put that in context. My buddy, Dave Winer, has been on a long time quest to find, foster, create, or instigate some way for there to be a long-lasting, secure, independent archive of our lifes’ digital works. Dave has a huge body of mostly-digital creations, but he’s also interested in providing a repository for his dad’s blog, photos, his uncle’s writings, family memorabilia, etc. It’s something everyone needs, from the simplest of bloggers to the most well-read A-List journalist — some place where your stuff can be read 100, 300, or a thousand years from now, without fear of being censored by a government, neglected by a corporation, or abused by cyber-viginantes.

Lots of the focus has centered on long-lived institutions like the Library of Congress or colleges and universities. But the tweet made me think of an equally long-lived set of institutions that are fundamentally aligned with the interests of a free and open society — local public libraries.

The local library exists because it is important to the community. They serve as focal points for community activities, with events and programs for toddlers, kids, and adults alike. Meeting rooms, multimedia, and computer labs all make up a part of the 21st century public library. Unfortunately, the writing is on the wall regarding their most fundamental service, lending physical books to their patrons.

In the absence of a compelling follow-on mission, some could be prompted to make a tweet such as the one above. Once we all have Kindles, who needs the library? Here’s the lightbulb.

There are literally thousands of public libraries in the US today (122,101 according to the American Library Association), and most are well-conneted to the Internet and have modern computing facilities for use by staff and patrons. And there are easily that many again scattered around the rest of the planet. A quarter of a million sites for information storage is a pretty compelling bit of redundancy! So, why not leverage a few simple technologies and create the long-lived, public digital archive that piggy-backs on the public library mission?

Imagine a large, distributed, redundant file system (there are lots to choose from) with a well-connected peer to peer application suite that archives, shares, replicates, and serves digital content across the planet. A botnet of sorts for our digital masterpieces. Each library only has to commit a relatively small amount of CPU, network, and data storage (probably just one dedicated PC with a few terabytes of storage) and we suddenly have petabytes of redundant digital archive that could last as long as our respective societies place a value on free access to information.

The implementation is left for another post, but the idea is certainly compelling. Librarians have shown themselves to be paragons when it comes to their mission of adopting and adapting information technology in a manner that serves the public. Let’s figure out how to give them their next mission and build the tools for them to do it!

This WikiLeaks crap bothers me. This morning, PayPal has shut them off, making the judgement that their activity is somehow illegal without any hint of due process. There is a story not getting told, which is all about who is pressuring Amazon, PayPal, DNS providers, and ISPs to turn off WikiLeaks.

Kiddie porn sites, illegal gambling sites, etc. all get a blind eye. Now that someone is pissed about WikiLeaks, these companies all suddenly take the moral high ground. Why isn’t anyone asking THEM why? Why now? Really asking them and not just taking their bogus PR as an answer. (It sure as hell isn’t that tool Joe Lieberman.)

Here’s the last best shot old school news organs have at leveraging their news desks and they are cowering. *I* want to know the answer, and short of a blogged leak from inside PayPal, we’ll never find out. Investigative journalism seems to be over.

“Do I really look like a guy with a plan? You know what I am? I’m a dog chasing cars. I wouldn’t know what to do with one if I caught it. You know, I just do things. The mob has plans, the cops have plans, Gordon’s got plans. You know, they’re schemers. Schemers trying to control their little worlds. I’m not a schemer. I try to show the schemers how pathetic their attempts to control things really are.”

—The Joker, The Dark Knight

(By way of explanation/disclaimer/credibility enhancer/armchair-quarterbacking, let me say that I was chairman of my town’s cable TV regulatory commission for 3 or 4 years and had the extreme pleasure of negotiating 2 franchise agreements with Comcast.)

This quick blog note is in response to the news that Comcast is charging L3 to deliver movies and other content that would normally be available to Comcast cable subscribers, but over Comcast’s data network.

IANAL, but it sure seems like Comcast has jumped the shark by asking that their data service to be considered the same as their CATV service as far as fees to deliver movies. If so, I’d say the FCC should declare the two Comcast networks identical and let localities tax and franchise the crap out of Comcast’s data pipes the same way they can regulate and tax the Comcast CATV network. IMO, Comcast just gave up common carrier status for their data network and if I was NYC or any other big city, I’d be knocking on their door with a new franchise agreement that taxed them for as much as they are charging L3.

Just my $.02. But since I have Verizon’s FiOS service for video and data, I really only marginally care out of anything more significant than latent spite and malice left over from many hours of wasted time hassling with their legal team over franchise terms. Good luck, L3. Don’t let ‘em get away with it!

Dave Winer infected me with the node.js meme the other day. I’m sure it was innocent on his part, but the results have been a bit like John Pinette at a chinese buffet. So much to try, so little time.

node.js has been around for a while. I remember looking at it almost a year ago and thinking that I liked my own Java version of poor man’s middleware, built out of Rhino, Derby, and my own home-grown object database, a lot better than the weird mishmash of pieces that seemed to be dumped into node.js.

Fast forward to last week, where the value proposition clicked into place, hard and fast. The missing piece was the Cloud9 IDE which is a completely browser-based IDE for developing and debugging Javascript apps for node.js or Chrome. Now I had a complete development environment that I can stick in the cloud, hosting it on one of Amazon’s new, essentially free EC2 “micro” server instances. A huge plus is that all the pieces run fine on Mac OS X as well.

And all the pieces are there to build just about any sort of Web app you can imagine now. Modules exist to tie together just about any popular database, web framework, CMS, or web service you can think of. And it’s all designed to be maximally scalable and take proper advantage of multi-core processors. It’s like someone dumped out a huge pile of all the Lego pieces in the world and sat me down in the middle of it.

Unfortunately, just like with real Legos, I now have to figure out what to make out of them.

I’ve just finished a weekend-long battle with Apple’s RAID software. In the end, it saved my bacon. But the story is longer than that and holds a couple of warnings about backup strategies. First some background.

I have a old PowerBook running OS X Server that plays the role of media server. It had 3 TB of drive space connected to it. 1 TB was set aside as a network-shared Time Machine back-up drive and the other 2 TB were in a mirrored RAID array. Because movie files, audio, photos, ebooks, etc. take up a lot of space, the decision was made to keep them on a RAID, rather than trying to keep them backed up with Time Machine or some sort of removable media.

Somewhere along the way, one of the two drives in the RAID array failed, and here comes the problem. Apple’s OS X Server product and their software-based RAID software give you no alarm or warning that a RAID array is degraded or failing. The only way I noticed the problem was because movies weren’t streaming properly off the server and I had to use the Disk Utility app to see why one drive wasn’t mounting.

It was at that point that I happened to click on the RAID tab and noticed a big red warning and the word “degraded” next to the RAID volume. Had I not bothered to investigate, it is entirely possible that the second drive in the RAID array could have failed or been corrupted by the first, completely defeating the purpose of the RAID in the first place.

So a few lessons learned:

1. Don’t put cheesy, low-end Western Digital hard drives in a RAID array. You’ll just end up having to constantly rebuild the array.

2. Don’t trust OS X Server to give you any sort of warning about your failing RAID array. Check early and often using Disk Utility.

3. Regardless of 1 and 2, a RAID array is a great way to avoid more expensive or time consuming back-up options like Time Machine or tons of removable media.

p.s. This is partially an exercise in trying out the WordPress -> Twitter functionality. Apologies if anyone finds this post boring.

I’ve been working with Dave Winer’s “feedhose” system for the past several days and have run into a problem that will likely face everyone trying to leverage this technology.

Since the RSS spec is fairly lenient, things like title and description tags can have all sorts of mark-up in them. (Don’t get me started on the issue of mark-up in RSS titles — it’s a major pet peeve of mine!) Anyway, it’s obviously possible to stick JSON syntax into the content of a RSS feed.

This makes interpreting the JSON data coming from Dave’s RSS-to-JSON conversion process a bit complicated. So here is the question(s) at hand:

When is the proper time to escape/encode JSON syntax that may be present in RSS, what specific tokens need to be escaped, and is it ever OK to send the feedhose JSON straight to Javascript’s eval() function?

The answers will help the feedhose stack properly and securely pass RSS up to clients for display, so please give freely of your painful past experience.

In response to Dave Winer’s post about bootstrapping a federated 140 character message system, I doodled up a spec for a hypothetical service called “F140″. I have a basic implementation of it working and am interested in hearing comments and feedback. The implementation is VERY simple and barely scratches the surface of the issues related to this sort of thing. But in the spirit of bootstrapping, please consider the F140 Proposal.

Please note, also, that this page is extracted from the running system, so ignore the links at the bottom of the document.