Journal tags: magnolia

Since Magnolia went down, taking everyone’s bookmarks with it, I’ve been through a mild Kübler-Ross cycle.

1. Denial. “It can’t be that all the data is gone. They’ll recover it.”
2. Anger. “I want my freaking bookmarks!”
3. Bargaining. “Isn’t there something I can do? Maybe there’s some API hacking that would help.”
4. Depression. “Why do I bother contributing to any social websites. Our data is doomed in the end.”
5. Acceptance. “C’est le Web.”

I also experienced déjà vu at every stage. The only difference between the end of Pownce and the end of Magnolia was that just one of those pieces of plug-pulling was planned. From the perspective of the people running those services, that’s a huge difference. From my perspective as an avid user of both services, it felt the same.

Actually, things turned out okay for my Magnolia data in the end. I was able to recover all my bookmarks …and it wasn’t down to any API hacking either. My bookmarks were saved by two messy, scrappy, plucky little technologies: RSS and microformats.

Google Reader caches RSS feeds aggressively. As long one person has ever subscribed to the RSS feed of your Magnolia links, you should be able to retrieve your links using Google’s Feed API—‘though for the life of me, I cannot understand why Google insists on marketing all these APIs as “Ajax” APIs, hiding server-side documentation under “Flash and other Non-Javascript Environments”.

If that doesn’t work, there’s always the regular HTML as archived by Google and the Internet Archive. Magnolia’s pages were marked up with the XFolk microformat. Using tools like Glenn’s UfXtract, this structured data can be converted into JSON or some other importable format. As Chris put it, Microformats are the vinyl of the web.

Magnolia’s bookmark recovery page uses a mixture of RSS and XFolk extraction tricks. I was able to recover my bookmarks and import them into Delicious.

But what’s the point of that? Swapping one third-party service for another. Well, believe me, I did a lot of soul searching before putting my links back in another silo. Really, I should be keeping my links here on adactio.com, maybe pinging Delicious or some other social bookmarking site as a back-up …what would Steven Pemberton do?

In the end, I decided to keep using Delicious partly out of convenience, but mostly because I can export my bookmarks quite easily; either through the API or as a hulking great hideous HTML bookmarks file (have you ever looked at the markup of those files that browsers import/export? Yeesh!)

But the mere presence of backup options isn’t enough. After all, Magnolia had a better API than Delicious but that didn’t help when the server came a crashin’. If I’m going to put data into a third-party site, I’m going to have to be self-disciplined and diligent about backing up regularly, just as I do with local data. So I’m getting myself into the habit of running a little PHP script every weekend that will extract all my bookmarks for safekeeping.

That’s my links taken care of. What about other data stores?

• Twitter. This PHP script should take care of backing up all my inane utterances.
• Flickr. I still have all the photos I’ve uploaded to Flickr so the photos themselves will be saved should anything happen to the site. But it would be a shame to lose the metadata that the pictures have accumulated. I should probably investigate how much metadata is maintained by backup services like QOOP.
• Dopplr. Well, the data about my trips isn’t really the important part of Dopplr; it’s the ancillary stuff like coincidences that makes it so handy. Still, with a little bit of hacking on the Dopplr API I could probably whip an export script together. Update: Tom writes to tell me that Dopplr does offer data export in the form of an .ics file.
• Last.fm. Again, like Dopplr, I’m not sure how valuable the data is outside the social context of the site. But again, like Dopplr, a bit of hacking on the Last.fm API might yield a reusable export script.
• Ffffound. I don’t use it to store anything useful or valuable. That’s what tools like LittleSnapper are for. Update: Hacker extraordinaire Paul Mison has whipped up a Ruby script to scrape ffffound and he points me in the direction of ddddownload.
• Facebook. It could fall off the face of the planet for all I care. I’ve never put any data into the site. I only keep a profile there as a communication hub for otherwise unconnected old friends.

As for my own sites—adactio, DOM Scripting, Principia Gastronomica, Salter Cane and of course The Session and Huffduffer—I’ve got local copies which are regularly backed up to an external hard drive and I’m doing database dumps once a week, which probably isn’t often enough. I worry sometimes that I’m not nearly as paranoid as I should be.

What happened to Magnolia was a real shame but, to put a positive spin on it, it’s been a learning experience not just for me, but for Larry too.

I’ve used del.icio.us for quite a while now. I’m storing 1159 bookmarks, each one of them tagged. It works just fine but it also feels a little, I don’t know …stale. There is supposedly a redesign in the works but I’m not sure that I want to wait around any longer to find out if they’re finally going to put some microformats in the markup.

Instead, I’m moving over to Magnolia. I’ve had a Magnolia account for years but I’ve never really used it. I didn’t see the point while I had a del.icio.us account. But whereas de.icio.us appears stagnated, Magnolia seems to be constantly innovating. Also, it uses microformats. There’s also the fact that I know Larry and I’ve briefly met Todd (lovely gents, both) but I don’t know Joshua Schachter. That shouldn’t matter but it kind of does.

Moving from del.icio.us to Magnolia is very straightforward. But that alone wasn’t going to be enough for me. I’m also accessing my del.icio.us bookmarks through the API. It turns out that Magnolia provides an ingenious way to ease my pain. As well as providing their own fully-featured API, Magnolia also provides a mirror of the infamously sucky del.icio.us API. All I had to do was change some URL endpoints and I had Adactio Elsewhere switched over in no time. Other services take note: providing mirrored versions of your competitors’ APIs eases the pain of migration.

I’ve updated my feedburner RSS feed to point it at my Magnolia links instead of my del.ious.us links. If you were subscribed to my del.icio.us feed separately, you’ll probably want to update your feedreader to point to my Magnolia links instead.

It remains to be seen whether I’ll stay at Magnolia. Even though it is functionally and cosmetically superior to del.icio.us, that might not be enough. After all, Jaiku is superior to Twitter in almost every way—design,markup, reliability—but Twitter still wins. That’s mostly because that’s where all my friends are. Right now my bookmarking friends are split fairly evenly between del.icio.us and Magnolia. Then again, I’ve never really made much use of the “social” part of “social bookmarking”.

So who knows? Maybe I’ll end up moving back to del.icio.us at some stage. It’s reassuring to know that moving my data around between these services is pretty straightforward: I can export from Magnolia and import into del.icio.us any time I want.

Here’s a list of websites on which I have an account and which involve some form of social networking. I’m listing them in order of how often I visit. I’m also listing how many contacts/buddies/friends/connections/people I have on each site.

My Social Networks

Website	Visits	Connections
Flickr	Daily	154
Twitter	Daily	205
Del.icio.us	Daily	4
Upcoming	Frequently	95
Last.fm	Frequently	66
Dopplr	Frequently	96
Jaiku	Weekly	34
Anobii	Weekly	2
Corkd	Infrequently	27
Pownce	Infrequently	22
Revish	Infrequenty	9
Ficlets	Infrequently	4
Newsvine	Infrequently	4
Facebook	Infrequently	59
Ma.gnolia	Rarely	7
Linked in	Rarely	90
Odeo	Rarely	10
Xing	Never	2
Digg	Never	0

This is just a snapshot of activity so some of the data may be slightly skewed. Pownce, for instance, is quite a new site so my visits may increase or decrease dramatically over time. Also, though I’ve listed Del.icio.us as a daily visit, it’s really just the bookmarklet or Adactio Elsewhere that I use every day—I hardly ever visit the site itself.

Other sites that I visit on a daily basis don’t have a social networking component: blogs, news sites, Technorati, The Session (hmmm… must do something about that).

In general, the more often I use a service, the more likely I am to have many connections there. But there are some glaring exceptions. I have hardly any connections on Del.icio.us because the social networking aspect is fairly tangential to the site’s main purpose.

More interestingly, there are some exceptions that run in the other direction. I have lots of connections on Linked in and Facebook but I don’t use them much at all. In the case of Linked in, that’s because I don’t really have any incentive. I’m sure it would be a different story if I were looking for a job.

As for Facebook, I really don’t like the way it tries to be a one-stop shop for everything. It feels like a walled garden to me. I much prefer services that choose to do one thing but do it really well:

• Flickr for photos,
• Dopplr for travel,
• Del.icio.us for bookmarks,
• Upcoming for events and
• Last.fm for music.

Mind you, there’s now some crossover in the events space when the events are musical in nature. The next Salter Cane concert is on Last.fm but it links off to the Upcoming event … which then loops back to Last.fm.

I haven’t settled on a book reading site yet. It’s a toss-up between Anobbii and Revish. It could go either way. One of the deciding factors will be how many of friends use each service. That’s the reason why I use Twitter more than Jaiku. Jaiku is superior in almost every way but more of my friends use Twitter. Inertia keeps me on Twitter. It’s probably just inertia that keeps me Del.icio.us rather than Ma.gnolia.

The sum total of all my connections on all these services comes to 890. But of course most of these are the same people showing up on different sites. I reckon the total amount of individual people doesn’t exceed 250. Of that, there’s probably a core of 50 people who I have connected to on at least 5 services. It’s for these people that I would really, really like to have portable social networks.

Each one of the services I’ve listed should follow these three steps. In order of difficulty:

1. Provide a publicly addressable list of my connections. Nearly all the sites listed already do this.
2. Mark up the list of connections with hCard and, where appropriate, XFN. Twitter, Flickr, Ma.gnolia, Pownce, Cork’d and Upcoming already do this.
3. Provide a form with a field to paste the URL of another service where I have suitably marked-up connections. Parse and attempt to import connections found there.

That last step is the tricky one. Dopplr is the first site to attempt this. That’s the way to do it. Other social networking sites, take note.

It’s time that social networking sites really made an effort to allow not just the free flow of data, but also the free flow of relationships.