To catch up with my blog, I needed to extract data from a twitter feed. Usually I update the blog on a weekly basis. Since I didn’t find the time recently, I fell behind.
I was looking for a quick solution to download all the feed data at once. A bash script should be good enough.
The script that I use to extract the data from the twitter feed, expects a date argument in the form
mm/dd/yyyy
. The script will extract all data for the range [date - 7days, date]
Create the series of dates
I need to create a series of dates for several weeks. Given a start date, I’m going to produce new dates with a 7 day interval. The starting date is included in the series. Here is what I ended up with:
#!/usr/bin/env bash set -e # startdate in the form mm/dd/yyyy startDate=$1 weeks=$2 for ((i=0;i<${weeks};i++)); do offsetDays=$((${i}*7)) newDate=`date -j -v+${offsetDays}d -f "%m/%d/%Y" ${startDate} "+%m/%d/%Y"` startDateFilename=`echo ${newDate} | sed "s/\//-/g"` echo ${startDateFilename} # download the twitter data and redirect all output to that file exec &> weeklyTwitterData-${startDateFilename}.txt # call the script to extract the data from twitter stream coffee src/WeeklyStats.coffee -n ${startDate} done
Since I didn’t want to do all the calendar math myself, I rely on the magic of the date utility. The line defining
newDate
is where the magic happens. Parsing a date ${startDate}
in a given format %m/%d/%Y
, adding $offsetDays
to the parsed date and print out the new date in the format %m/%d/%Y
.
That’s it.
No comments:
Post a Comment