Create 1 file with all movement records in: msslml:shutter mcrw$ awk -f tidy.awk shutter_20061020_20071011.txt > intermediate/total.txt msslml:shutter mcrw$ awk -f tidy.awk shutter_20071011_20080802.txt >> intermediate/total.txt msslml:shutter mcrw$ awk -f tidy.awk shutter.txt >> intermediate/total.txt Sort the file and remove duplicates: msslml:shutter mcrw$ cd intermediate msslml:intermediate mcrw$ sort total.txt > sorted.txt msslml:intermediate mcrw$ uniq sorted.txt > uniq.txt Create a file with movement number by day (creates file daily_count.txt): msslml:intermediate mcrw$ ./by_day.sh Filter out blank and impossible dates: msslml:intermediate mcrw$ awk -f filter_total.awk daily_count.txt > proper_daily_count.txt Next 2 numbers (4448668) agree. This is the total shutter movements since launch: msslml:intermediate mcrw$ awk -f filter_total.awk proper_daily_count.txt | awk '{tot+=$3} END {print tot}' msslml:intermediate mcrw$ awk '{tot+=$3} END {print tot}' uniq.txt proper_daily_count.txt contains the number of movements per day where the record exists.