To catch up with my blog, I needed to extract data from a twitter feed. Usually I update the blog on a weekly basis. Since I didn’t find the time recently, I fell behind.
I was looking for a quick solution to download all the feed data at once. A bash script should be good enough.
The script that I use to extract the data from the twitter feed, expects a date argument in the form
mm/dd/yyyy. The script will extract all data for the range [date - 7days, date]Create the series of dates
I need to create a series of dates for several weeks. Given a start date, I’m going to produce new dates with a 7 day interval. The starting date is included in the series. Here is what I ended up with:
#!/usr/bin/env bash
set -e
# startdate in the form mm/dd/yyyy
startDate=$1
weeks=$2
for ((i=0;i<${weeks};i++)); do
offsetDays=$((${i}*7))
newDate=`date -j -v+${offsetDays}d -f "%m/%d/%Y" ${startDate} "+%m/%d/%Y"`
startDateFilename=`echo ${newDate} | sed "s/\//-/g"`
echo ${startDateFilename}
# download the twitter data and redirect all output to that file
exec &> weeklyTwitterData-${startDateFilename}.txt
# call the script to extract the data from twitter stream
coffee src/WeeklyStats.coffee -n ${startDate}
done
Since I didn’t want to do all the calendar math myself, I rely on the magic of the date utility. The line defining
newDate is where the magic happens. Parsing a date ${startDate} in a given format %m/%d/%Y, adding $offsetDays to the parsed date and print out the new date in the format %m/%d/%Y.
That’s it.
No comments:
Post a Comment