Pages

Thursday, December 04, 2008

Who says Java is not fast?!?

While performance tests actually show that for even core numerical calculations Java is at par with C in terms of speeds, and sometimes even hits Fortran-like speeds, people keep think that Java is not fast. I only invite you to test that yourself.

Meanwhile, I would like to take the opportunity to advertise Noel's cinfony paper in CCJ (doi:10.1186/1752-153X-2-24) which features these speed measurements (from the paper, CC-BY license): I have to say that these numbers surprised me, as the CDK is hardly optimized for speed at al...

12 comments:

  1. I don't think Open Babel or RDKit have necessarily optimized for speed much either.

    One difference is that I think Open Babel "does more work" when "just" iterating through molecules. I suspect with a bit of work, all of the above packages could considerably improve performance.

    An interesting test for Noel might be to compare with OEChem or another commercial package. :-)

    ReplyDelete
  2. Geoff, I do not know what OB understands of reading an SD file, but I can guess it does some preliminary work aimed at file conversion.

    I also was thinking that Java might under the hood take advantage of the second core for caching file content...

    ReplyDelete
  3. Thanks for the advertisment. I note that the code is included for those that wish to perform the timing comparisons themselves, but see my comment over at Depth-First regarding the rationale behind the Table...

    ReplyDelete
  4. Egon,

    I think your title/comment about java speed is off target with regards to interpreting this table. It is rather questionable how much and what kind of work various programs do under the label of "iterating through SDF" as others have pointed out. If it is merely reading the file and storing the content in memory, that has very little to do with computation performance and a lot more to do with file IO performance. If they do various amount of extra work (charge calculation, protonation, etc.) that makes the speed comparison rather pointless. However, there is one thing you can conclude from the table for actual computation speed and that is the time spent in computation of molecular weight (subtracting the SDF iteration time). When you do that, you get ~16s for CDK Java, 12s for Babel and 1s for RDkit, which shows that C++ is 16X faster than Java... Hmmm quite the opposite conclusion than what the title says.

    ZZ

    ReplyDelete
  5. ZZ, the table is not about comparing performances of languages, and there is some intentional flame bait in the title...

    Like always, there is so much to comment... note my comment up front 'I invite you to test test yourself' for a reason...

    The paper also has an section on unique features of each toolkit, and it is surprising that when you ask for iterating over SDF entries, the toolkit actually automatically starts to do extra things... I have no idea whether that actually is happening (did not review the code involved), and others indeed seem to confirm that.

    I also find this kind of dangerous to have algorithms modify the content of entries on what it believes to be stated in the SD file... that's why I am happy the CDK does not do that... I'm sure that any toolkit will work well for what is correctly represented in the SD file, but I am less positive about the recover or even detect problems in the SDF entries...

    Good catch on the weight calculation, though... I overlooked that. That should not be so time consuming, and will actually do some benchmarking on that myself... it takes about as much time as reading an entry... that does not sound right...

    ReplyDelete
  6. ZZ and Egon, one reason for the apparently slow MW calculation could be that the Java test prints to System.out. In my experience this can dramatically slow things down, although I wish I understood why. The C++ test prints to cout, which may be faster.

    ReplyDelete
  7. Dleskov, good suggestion. I never quite mastered using gcj for creating native .so code for jar libraries...

    On the other hand, modern JIT compilers can, apparently, optimize code based on what data is actually processed... one would likely loose those powers when compiling into a native binary...

    ReplyDelete
  8. A couple of quick comments
    The RDKit certainly could be more highly optimized.
    Still, it's probably not as slow as this benchmark is making it out to be: when an SD file is parsed all of the chemistry perception is done. Geoff pointed out the same thing about Open Babel above.

    ReplyDelete
  9. Egon, Excelsior JET is much simpler to use than GCJ. Just watch this 5-minute Excelsior JET Getting Started tutorial and you'll be all set. Do not hesitate to contact our techsupport if you need help.

    ReplyDelete
  10. Hi,
    the (most) comprehensive benchmark comparison of computer languages (besides 99 bottles of beer) can be found on the The Computer Language Benchmarks Game. It compares scientific computations like binary trees, hashtables, streaming commands and others.

    So this is actually independent on the fact if this is a cheminformatics toolkit or a machine learning or mandelbrot algorithm.

    And if you look at some of the double precision (SSE/SSE2/SSE3) streaming applications, JAVA is in par with FORTRAN (same speed). And C++ can be faster than JAVA and JAVA can be faster than C++.

    All that stuff doesn't matter I guess. What matters are your language skills and how easy it is to create such examples (see python or ruby or perl).

    I am not impressed with the times in the paper, (I agree the paper is about doing some fast and elegant computations and not benchmarks), but a pure word count (WC) reads through the file in 1 second (on my 2 GHz Core Intel). I don't like any other Intel chips than core duo or intel quad, so the CPU in the paper is probably freakin slow NETBUST technology.

    I would rather challenge everybody to do the MW calculations in 5 seconds instead of 36.8 seconds. That can be done on a quad core or oct-core machine using parallelization techniques.

    Using cxcalc I could finish the molecular weight calculation for the same file
    as in the publication from ZINC DB
    3_p0.0.sdf
    in 20 seconds on a Dual Core 2 GHz. So using a quad core could be 10 sec or
    oct core could be 5 seconds, assuming the file is split in 4 or 8 pieces.

    If I convert it to SMILES instead of SDF (showing the overhead of SDF) I could finish the MW calculation in 8 seconds on a Dual Core (2 GHz). So I guess on a quad or oct core it can be finished in less than 5 seconds. And concurrency matters in this case. Its cheap and easy if you have a multi-core CPU.

    Cheers
    Tobias (FiehnLab)

    ReplyDelete