Skip to content

Concern about some benchmarks

Hi Issac -

It looks like the regex-redux test for python 3 is not using pythons internal regex and instead loading and using an external C library giving a very misleading impression that Python out performs Rust on this test.  If the goal to compare language performance, it should not be from delegating the majority of the work to another language, because most languages can pull in C libraries and call them, but it is not really reflective of the actual language differences.

I think this started by programs loading GMP to optimize the pidigits implementation. I do think it is slightly different with GMP as there are bindings provided in language specific wrappers, but honestly I think even that gives a false impression of the languages actual performance.    I'm guilty of doing that myself, for pidigits by importing the gmp gem.  But honestly I'd like to see even GMP removed from all tests to see how a language actually performs (obviously it is different if gmp is compiled into the language as its implementation). It would be great to get the shootout back to a better objective comparison of different language performance.

Best Aaron Tavistock