Iraq Deaths TwitterBot

We’ve posted an update to our Iraq Deaths agent at . Here I’m going to journal and document the work involved in making this actually work.

At a high level this agent makes a daily post to twitter to broadcast any deaths reported by the Iraq War Bodycount database. I should mention that without their efforts at keeping and surfacing these records none of this would be possible. This kind of work itself is emotionally challenging and I want to applaud them.

Design-wise this was a project that Paige and I thought up and were excited by. We did it in a morning while we were both supposed to be doing other work – but the fact is it resonated with work we both care about.  Speaking for myself at Meedan we’ve been looking for ways to help social networks bridge the barriers of language and I saw this as a way to contribute to a project that helped people keep attention on something that is normally invisible.  For me this was a rewarding project because I enjoy removing the technical boundary between design and implementation. Often I like to work with designers to put real muscle underneath their vision.

The technical implementation consists of three stages

  1. collection
  2. analysis
  3. publishing

In the collections stage we talk to the iraq war bodycount database – pulling the entire corpus of deaths and then plucking them out of their csv structured text – like so:

require 'net/http'
require 'rubygems'
require "fastercsv"
url = ""
data = Net::HTTP.get_response(URI.parse(url)).body
puts "fetched"
@results = FasterCSV.parse(data)
@deaths = []
inside_header = true
@results.each do |death|
 if death[0] == "IBC code"
   inside_header = false
 elsif inside_header == false
   @deaths << death

After this stage we go and add any new data to our own database. In this way we keep a running track of any changes and can act only on changes rather than on all data I first attempted to use sqlite3 but then ended up using datamapper - like so:

require 'rubygems'
require 'dm-core'
DataMapper.setup(:default, {
   :adapter  => 'postgres',
   :database => "endiraqwar",
   :username => 'endiraqwar',
   :host     => 'localhost'
class Death
 include DataMapper::Resource
 property :id,         Integer, :serial => true
 property :code,       String
 property :name,       Text
 property :age,        Text
 property :sex,        Text
 property :marital,    Text
 property :parental,   String
 property :earliest,   DateTime
 property :latest,     DateTime
 property :location,   Text
 property :created_at, DateTime
 property :posted,     DateTime, :default => nil
# DataMapper.auto_migrate!
@deaths.each do |death|
 if Death.first(:code => death[0] )
   puts "We already found this death #{death[1]} #{death[0]} so not saving"
 # take a second to convert the date phrase into a machine date
 death[6] = DateTime.parse(death[6])
 death[7] = DateTime.parse(death[7])
 record =
             :code => death[0],
             :name => death[1],
             :age => death[2],
             :sex => death[3],
             :marital => death[4],
             :parental => death[5],
             :earliest => death[6],
             :latest => death[7],
             :location => death[8]
 puts "recording the passing of #{} at #{record.earliest} and #{record.code}"

The last phase is to report the actual deaths. We rely on the twitter gem to do this - I find I am using this gem more and more and it is quite convenient - like so:

twittercap = 50 # twitter this many posts max
require 'twitter'
twitter ="iraqdeaths",secret_password)
@deaths = Death.all(:order => [:earliest.desc], :limit => twittercap)
@copyofdeaths = []
@deaths.each do |death|
 @copyofdeaths << death
@copyofdeaths.reverse.each do |death|
 # publish deaths that are new
 next if death.posted != nil
 result ="#{}, #{death.age}, #{}, #{death.marital}, #{death.parental} killed on #{death.earliest.strftime("%d %b %Y")} at L:#{death.location}")
 # remember that we already published this death
 death.posted =
 puts "posted the death of #{} #{death.code}"

I try to be very careful to never update my own understanding of the database record until I am absolutely certain that Twitter has been updated. Even if my agent crashes I want it to crash in a way that doesn't spray garbage all over twitterspace.

Overall, as you can see, the high aspiration of building something that makes a statement is connected to real metal underneath. We're grateful to the organization for making this possible.

Post a Comment

Your email is never published nor shared. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>