Archiving Twitter

I’ve become a big Twitter fan recently. Before getting going earlier this year, I was a Twitter-sceptic for a long time – I actually signed up twice before and lasted all of, ooh, a day, before deciding that I couldn’t see the point and deleting my account(s). But this time I persevered for a few days, reached that critical mass of followers and people I wanted to follow, and here I am a few months later, still Twittering away.

Having decided early on that I was going to use my Twitter account primarily for work purposes, there are lots of work-related tweets in my account. Many of them are about digital preservation inititives I’ve come across. Whilst I wouldn’t go so far as to say they are gold dust, they’re not quite transient and I’d quite like to have a copy of them – for a while, at least. So I’ve started looking into tools to archive Twitter accounts.

There’s not much on the Twitter website about it, you have to do some rooting around. (Enter Google, stage left). In a couple of minutes, I found:

• Tweettake
• Tweetdumpr
• A Python script to grab a copy of your tweets
• A Twitter tool for making an archive of your account (though this seems to have been taken down)

I’ve used Tweettake to grab a copy of my tweets to date, along with a list of (and info about) people I follow, my followers, and private messages. This is all wrapped up in a nice and convenient *.csv file – fairly straightforward for preservation purposes. So it was a nice experiment and I’m happy that I can get a copy of my Twitter data that’s important to me (so far, anyway; the scalability of these apps is something that I’ll look at as my account grows).

Now, here’s the wider context. I know that many people out there couldn’t give a monkey’s about saving Twitter updates and account information. And for many Twitter acounts, I’d probably agree. But there are certainly instances where Twitter accounts are valuable and interesting and worthy of archiving. Think of all the authors and poets on Twitter, for example – their feeds could provide a fascinating insight into a social and professional aspect of their lives that would otherwise be lost to time. Who should be responsible for archiving them, if anyone? What of the Twitter novels – a new and emerging art form or an exercise in futility as you can’t write anything coherent if it’s forced into 140 character batches? Future generations won’t be able to make that decision if we don’t keep a copy. What of updates from government officials and institutions – should they be kept as  records? If not, why not?  If people rely on the information in them then they should be kept too, surely? (As an aside, how many archivists and records keepers have applied their retention schedules to tweets?)

If we do decide we want the data, then another issue is how on earth we get it? Tweetdeck required me to give my username and password, so it’s an okay way to get data from an account when we have that information, but what about when we don’t? And if we do figure out a way, is it ethical to go around grabbing copies of other people’s Twitter data for our archives, even if they have posted it online for the world to see?

Finally (at least for now), how do we validate that what we’ve got is what we think we’ve got? Issues of identity and authenticity are real problems in the Twittersphere. There are already numerous examples of fake twitter accounts that amass thousands of followers before they are sprung. There are probably many more still out there. And if we’re going to archive Twitter accounts then we need to be sure that we’re archiving the genuine item. Questions questions questions… . Tweets to @mopennock please!