Rene Kita's weblog

Blog[0] About[1] RSS[2] Links[3]

ts benchmark

2023-10-29

I wrote a tool that reads lines from stdin and prepends them with a timestamp. Out of reasons I did so in C, awk, perl, python and shell. After I was done I found out, that there is already such a tool, ts from moreutils. ts from moreutils is a Perl script. I did a little test comparing for how long each version needs to process some dummy input.

The basic function looks like this in shell:

while read -r line
do
	d=$(date '+%Y%m%dT%H%M%S')
	printf "%s %s\n" "$d" "$line"
done

Let's generate a file with many lines:

% dd if=/dev/zero bs=1024 count=1024 2>/dev/null | od -v > t_s
% wc -l t_s
65537 t_s

And with the help of a little script we get results:

% ./t t_s
ts      t_s     real 0.190000 user 0.150000 sys 0.030000
C       t_s     real 0.020000 user 0.010000 sys 0.000000
awk     t_s     real 0.020000 user 0.020000 sys 0.000000
pl      t_s     real 0.080000 user 0.070000 sys 0.000000
py      t_s     real 0.180000 user 0.170000 sys 0.010000
sh      t_s     real 33.740000 user 27.780000 sys 9.250000
./t t_s  28.23s user 9.31s system 109% cpu 34.233 total

The first column identifies the version, where 'ts' is the version from moreutils. Awk is GNU awk here, because I needed the time functions which are not include in The One True AWK.

Let's see with a bigger file and without the shell version:

% dd if=/dev/zero bs=8192 count=8192 2>/dev/null | od -v > t_xl
% wc -l t_xl
4194305 t_xl

% ./t t_xl
ts      t_xl    real 10.360000 user 8.000000 sys 2.340000
C       t_xl    real 0.920000 user 0.830000 sys 0.090000
awk     t_xl    real 1.350000 user 1.260000 sys 0.080000
pl      t_xl    real 4.280000 user 4.200000 sys 0.060000
py      t_xl    real 10.600000 user 10.520000 sys 0.070000
./t t_xl  24.85s user 2.68s system 100% cpu 27.529 total

I'm really amazed how fast awk is. And it's very interesting to see the difference between my perl version and the original ts.

That's it for a sunday - back to real work!

Here is the code: the codez[4].

Last modified: 2023-10-29T12:32:36Z

E-mail: mail@rkta.de[5] IRC: #rkta[6] on Libera.Chat[7]

-- Response ended

-- Page fetched on Fri May 10 16:51:19 2024