twittermeterbeta
Twitter meter let's you query an index of all the words that have been sent to twitter's
public timeline since I started gathering them on 11/6/2007 and plot the number of times that word was used over time.
The fuzzy search option will search for any word that starts with what
you type in the box (so it would pick up both "twitter" and "twitter's").
To see if I could.
The twittermeter is a multi-step process utilizing Twitter's API, PHP, a MySQL Database and the fantastic timeplot javaScript goodness from Simile over at MIT
TwitterTwitter Publishes the public feed. Sadly, I wrote the whole thing to just scrape the public page and parse the resulting XHTML, but then thought to myself "hrm... I wonder if Twitter has an API." Well, I'm glad I thought it, because they do (Twitter API).
Most important is the API call that lets you ask for all the status messages occurring since a certain ID. All you have to do is pop the results into a database, check for the last status that made it in and then look for any status messages that have occurred since then.
topThe back end code is built to deal with the requests for word plots as well as parsing the Twitter results and getting them into the database - nothing too special going on there. To fill up the database the script rips along pinging the public timeline every 5 seconds. It plops the status messages into one table and parses the words into another table along with the id for their status. Every morning around 2am, a script runs to add the words from the day before to table of the word, day and count. The next itemm tacking onto the todo list is to have this table update every hour.
topThe data come from a MySQL database that's been filling up with results from the php parsing since November 6th, 2007. There will definitely be some nastiness to overcome when the database starts to get too big. As of right now, there are over a 2.1 million status messages and 14.5 million words.
topTimeplot comes from Simile at MIT. I found out about it while seeing a demo of another widget they've produced called Timeline. Sadly, it only works in Firefox and Safari for the time being, but they claim to be working on it. Plus, I'm only doing this for fun and dealing with IE's lack of Canvas support isn't fun.
top