Animation: 3 Days of Geotagged Tweets in NYC


What’s in a tweet?  A lot, when there’s a set of latitude and longitude coordinates attached to it.  Using the twitter streaming API, Rudin research assistant Chris Whong was able to compile three full days worth of geotagged tweets from around the New York City region, totaling more than 74,000 data points.  Instead of simply visualizing the location and time of individual tweets, we can “connect the dots” through time and space for a given user, showing a movement vector across the map.

Played back at one minute per frame, the video clearly shows the ebbs and flows of activity throughout the day.  The mass movement of people during rush hours is visible, as well as movement to and from several hotspots in the region.  (Keep an eye on Metlife Stadium in New Jersey during the first 20 seconds of the video – you’ll many people who tweeted during a Monday night football game moving back to their homes – JFK airport also stands out as a key destination)

A Day in the Life: How the Sept. 11 TweetMap Was Created


Yesterday we showed you Chris Whong’s tweet map from September 11th, 2012. Here’s how he did it:
A Day in the Life is a dump of 15,000 geocoded tweets, all from a single day, all from the five boroughs of New York City.  Created by NYU Urban Planning Student and civic techie Chris Whong, the map is labeled a social media experiment, a visualization of social media interactions that allows a user to freely explore the city and see who was tweeting what, and most interestingly, where they tweet from.  Our online social networks tend to mirror our real world networks, and A Day in the Life offers a peek into thousands of other networks that share the Urban Landscape, even if their many nodes and linkages don’t cross paths often (online or in real life).
The addition of latitude and longitude coordinates to the normal tweet data has some powerful implications, and adds a spatial element to the typical analysis of tweets by keyword or hashtag, and even see the movement of individual tweeters around the city over the course of the day (provided they tweet regularly of course).  A Day in the Life is meant more for exploration, but other static maps and visualizations of links and specific keywords can be produced from the same types of data sets. (Eric Fischer released a series of maps highlighting movement corridors through cities using geocoded tweets earlier this year)  The New York map is based on a similar one for Baltimore (http://www.charmcitynetworks.com/bmoretweets) that also features layers for Census data and Baltimore’s Vacant properties, giving the user some context for the location of the tweeter.
Interesting? Yes.  Entertaining? Of course!  Alarming? Sometimes (tweets about violence, drug use, truancy, etc can be seen here and there), but is this data really useful for drawing real conclusions about a city and effecting change?  Maybe.  It should be noted that this collection represents only a small sample of all tweets, 2-4% by some estimates.  While there is certainly a broad geographic representation, with no corner of the city left out, the only people on these maps are those who had location services on, and the picture might be very different if all tweets were considered.  Those who tweet their location, for whatever reason, may not be a representative sample of all tweeters.
The data source for these maps is Twitter’s streaming API, which allows a user to specify a bounding box.  Any geocoded tweets that occur within the box are sent in real-time, and can be stored in a database for future use.  The Baltimore Map was a result of impromptu civic hacktivism born on a Facebook group called Baltimore tech.  Dave Troy, a local tech entrepreneur and urbanist wrote a script to pull Baltimore tweets from the API, and then published a link to the data for any who could find something useful to do with it.  The results included animations of user movement overs time, aggregate tweet trail maps that highlight frequently traveled routes, word clouds that attempt to highlight themes, A Day in the Life, and more.  So, we used Facebook connections to do twitter data analysis.  Social Media begets Social Media.

PDF Hackathon


The Personal Democracy Forum:Applied Hackathon was held last weekend, an event that attracted dozens of participants from nonprofits, activist groups, hackers, developers and government agencies.  The event was a lead-in to PDF’s two-day conference, held at NYU on Monday and Tuesday, with themes focused around technology, politics, government and civic life.

Representatives from the MTA also attended the event, with a special treat for any hacker: The first sample of real-time data for the New York City Subway, which is set to be released in Fall 2012.  For our NYU Rudin Center rep at the event, the idea for a real-time visualization of this data, with animated trains moving along the screen and stopping at stations, evolved into a mobile web app called SeeTrain, by Rudin graduate research assistant Chris Whong, along with front-end developer Sam Richard and back-end developers Jeremy Baron and Graham Brooks formed a team to create an app that could make use of the real-time data.
The team faced challenges converting the data from Google’s GTFS-realtime format, an accepted standard for real time transit data, but not the most hacker-friendly.  With just under two days of development time, the team was able to create a simulation of what real-time subway visualization looks like, available for viewing at http://seetra.in.  The app includes animated icons for trains traveling in both directions on the 1-2-3 trains between 96th street and Chambers street.  Beyond this demo, the team hopes to add stop specific arrival times, trip planning and more.
Seetra.in tied for third place at the hackathon, earning the team the right to present their new app to an audience of 800 people at the Personal Democracy Forum.
View all applications from the event here; the other winners were:
1st: Pollwatch - a real-time reporting app for people to report mischief or other unfriendly conditions at polling places on election day

2nd: Open Up NYC – an app that automates FOIL requests for the NYC government, ensuring that they are in the right format, sent to the right agency, and tracked every step of the way.
3rd (tied): Crowdshift -  an app that allows protest participants to sign up for shifts, and allows organizers to know where/when they need more participants.
Congrats to Chris and all of the contest winners!

Geeks on a Train


Last Thursday, 25 programmers, developers, and entrepreneurs representing Baltimore, Maryland’s flourishing tech community boarded Amtrak trains in an effort to create unity among their fellow geeks in the northeast megalopolis.  The event, called “Geeks on a Train”, sat at the intersection of transportation and regional economic development, was dubbed a ”rolling tweetup.”, and fell on the anniversary of the first telegraph transmission (sent between D.C. and Baltimore, it also followed the route of the railroads). It was hosted by the Greater Baltimore Technology Council, an organization that encourages technological innovation and tech startup activity in Charm City with events and other resources.

The train numbers were advertised, and geeks were encouraged to board in their own city, wherever Amtrak’s Northeast Regional stops between D.C. and Boston.  A tour of The Hatchery, a New York  business incubator on 7th Avenue was planned as a lunch break.  From NYC, a second geektrain would carry the tweetup to Boston, where the group would crash an established weekly happy hour at the Cambridge Innovation Center’s Venture Café.

The original geektrain had an engine failure between D.C. in Baltimore.  D.C. geeks tweeted their frustrations from the stationary train while the Baltimore geeks made arrangements to change their tickets, noting the irony in Amtrak’s initial message that the original train was delayed due to computer issues.  The Baltimore geeks were switched to a Vermonter and continued to NYC without further delay, occupying the dining car.  Verizon Wireless donated several mobile hotspots for use during the event, as no self-respecting geek could be productive on Amtrak’s spotty wifi.

The Hatchery’s founder, Yao Huang, gave a guided tour of their new offices, complete with coworking spaces, conference rooms ranging from living-room to board-room style, and a “programmer’s den”, where developers can don headphones, tune out the world, and maximize efficiency.  Huang emphasized that good attitudes not only go a long way in their incubator, they are required.

Amtrak sorted out the engine troubles, allowing the D.C. geeks to arrive in New York just in time to link-up with the rest of the group and board the next train to Boston.   After arrival at South Station and a short ride on the T, the group was greeted by the Cambridge Innovation Center with ribs, an open bar and a great mix of entrepreneurs, developers, venture capitalists, and business coaches.  The Venture Café is a sort of high-tech happy hour, complete with its own web app that cycles through attendee bios on a big screen TV.  One of CIC’s recent startups, a web service that unites athletes and coaches, was in the spotlight, and had a chance to publicize their product and sing the praises of the incubator.

Geeks on a train accomplished its mission, showing Baltimore’s geeks what a wealth of resources for tech startups exist in their neighbor cities, and showing the rest of the corridor that there is a talented and vibrant tech scene just a few stops away in Baltimore.

More information is available at http://gb.tc.

- Written by Chris Whong