Friday, 30 October 2009

extracting uniq epoch timestamps

I have a large set of files (2 lac files) of which every line has fields separated by a Ctrl-B(^B) character. Every line starts with letter "z" , I want to extract 5th field which is timestamp and see the uniq day time stamps of each file. This is how i can do:

1)looping accross all the files in the directory and printing filename and uniq epoch timestamps into a file.


for file in `ls -tr` ; do echo $file >> /tmp/epochs; perl -nlaF"^B" -e 'if ($F[0] eq "z") { my @t= eval "$F[4]-($F[4] % 86400)" ; print @t; } ' $file |
sort -u >> /tmp/epochs ; done

or

for file in `ls -tr` ; do echo $file >> /tmp/epoch.two; perl -nlaF"^B" -e 'if ($F[0] eq "z") { my $t= eval "$F[4]-($F[4] % 86400)" ; print $t; } ' $file
| sort -u >> /tmp/epoch.two ; done

2)taking only the epochs from that file and converting them to readable format.


perl -wnl -e ' if ( length() == 10 ) { print $_;} ' /tmp/epochs | while read line ; do date --date "1970-01-01 UTC $line seconds" ; done

No comments:

Post a Comment

Tweets by @sriramperumalla