| Author |
Message |
|
Michael Pa
#1 / 71
|
 CMUCL performance in the Shootout Why did CMUCL do so poorly in the "Shootout" ? It is a few times slower than C for matrix multiplication. All types were declared. On my machine, when I test it from REPL, so there are no start-up costs, Lisp is 4-5 times slower than C. Why? http://www.*-*-*.com/ ~doug/shootout/bench/matrix/
P.S. I hope no one gets over-sensitive because of this question.
|
| Wed, 09 Nov 2005 10:10:22 GMT |
|
 |
|
Scott A Douglas
#2 / 71
|
CMUCL performance in the Shootout --On Friday, May 23, 2003 7:10 PM -0700 Michael Park
Quote:
> Why did CMUCL do so poorly in the "Shootout" ? It is a few times > slower than C for matrix multiplication. All types were declared. On > my machine, when I test it from REPL, so there are no start-up costs, > Lisp is 4-5 times slower than C. Why? > http://www.bagley.org/~doug/shootout/bench/matrix/
Why did C do so poorly in the "Shootout"? It is a few times slower that CMUCL for regular expression matching. Why? http://www.bagley.org/~doug/shootout/bench/regexmatch/
I'd hardly call 6th overall "doing so poorly"!
I guess when Doug states in his "Best of Show":
"I've dabbled with Lisp (e.g. CMUCL) on and off for many years, but the
shootout gave me an opportunity to look more at Common Lisp and at Scheme
(Bigloo) too, which I hadn't done before. This family of languages has so
much to offer, it's a shame that they are not more popular."
he simply didn't know what he was talking about.
Quit trying to get a rise out of the group...
|
| Wed, 09 Nov 2005 14:04:02 GMT |
|
|
|
Florian Weime
#3 / 71
|
CMUCL performance in the Shootout
Quote: > Why did C do so poorly in the "Shootout"? It is a few times slower that
> CMUCL for regular expression matching. Why?
In this case, Shootout does not measure C performance, but PCRE performance, 8-)
|
| Wed, 09 Nov 2005 21:13:25 GMT |
|
|
|
Rolf Weste
#4 / 71
|
CMUCL performance in the Shootout Quote:
> Why did CMUCL do so poorly in the "Shootout" ? It is a few times > slower than C for matrix multiplication. All types were declared. On > my machine, when I test it from REPL, so there are no start-up costs, > Lisp is 4-5 times slower than C. Why? > http://www.bagley.org/~doug/shootout/bench/matrix/
> P.S. I hope no one gets over-sensitive because of this question.
I have benchmarked a 300x300 matrix multiplication. The fastest CMUCL version was only 2 times slower than the C++ version using TNT::Array2D (Template Numerical Library). The CMUCL program still made array bound checks while the C++ program did not. So I think CMUCL is quite fast in this example although not as fast as C++ (a C program using raw C-arrays may be even faster). The main advantage of CMUCL for me is the combination of fast
interactive and dynamic developement and reasonable fast compiled code.
There is no other language/implementation that is comparable to CMUCL
in that respect.
Rolf Wester
|
| Wed, 09 Nov 2005 21:16:04 GMT |
|
|
|
Gareth McCaugha
#5 / 71
|
CMUCL performance in the Shootout Quote:
> Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> slower than C for matrix multiplication. All types were declared. On
> my machine, when I test it from REPL, so there are no start-up costs,
> Lisp is 4-5 times slower than C. Why?
Looking at the disassembled code from both versions (which I'll call "C" and "L" with the obvious meanings): - C has an inner loop 9 instructions long; L has an
inner loop 21 instructions long.
- C keeps the accumulated sum in a register
throughout the calculation; L spills it.
- C keeps the inner loop's counter in a register
throughout the calculation; L spills it.
- C moves one indexing operation out of the inner loop;
L doesn't.
Quote: > P.S. I hope no one gets over-sensitive because of this question.
If you want to avoid people getting over-sensitive, it might be advisable to phrase the question in a less provocative way. You found *one* test in which CMUCL does much worse than C; that doesn't constitute "doing poorly" on its own. (My vague recollection: there are a couple of tests on which it's substantially worse than C, and for the rest it's similar in speed.) --
.sig under construc
|
| Wed, 09 Nov 2005 21:54:33 GMT |
|
|
|
Scott McKa
#6 / 71
|
CMUCL performance in the Shootout
Quote: > --On Friday, May 23, 2003 7:10 PM -0700 Michael Park
> > Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> > slower than C for matrix multiplication. All types were declared. On
> > my machine, when I test it from REPL, so there are no start-up costs,
> > Lisp is 4-5 times slower than C. Why?
> > http://www.bagley.org/~doug/shootout/bench/matrix/
> Why did C do so poorly in the "Shootout"? It is a few times slower that
> CMUCL for regular expression matching. Why?
> http://www.bagley.org/~doug/shootout/bench/regexmatch/
Doesn't this graph really show that we should all be using OCaml? ;-)
|
| Wed, 09 Nov 2005 22:55:23 GMT |
|
|
|
Nikodemus Siivol
#7 / 71
|
CMUCL performance in the Shootout Quote:
> Doesn't this graph really show that we should all be using OCaml?
Btw -- there is somewhere an interesting variation on the shootout, with the *bytes* of code metric, using both plain and bzip2'edded sources. Can't seem to find it right now, though. IIRC Ocaml did a lot worse in those if any significant weight was given to
the source size.
What I'd like to see (or do, if I find the time) would be a version of the
shootout using various CL implementations -- both commercial and free --
plus maybe g++ and Sun javac to give a non-CL baseline. Also, a FP
intensive test would be interesting as well.
Cheers,
-- Nikodemus
|
| Wed, 09 Nov 2005 23:27:11 GMT |
|
|
|
Bruce Houl
#8 / 71
|
CMUCL performance in the Shootout
Quote: > Doesn't this graph really show that we should all be using OCaml?
I've played with OCaml and it certainly seems to be a very attractive language if you already know what you want before you start, but I never seem to. The OCaml guys have had some spectacular sucesses in the ICFP contest (e.g. the Ray Tracer in '00), but also some big failures (e.g. last year). Four weeks to the next one!
-- Bruce
sticking with Dylan, but hoping to see more CL entries
|
| Thu, 10 Nov 2005 08:55:43 GMT |
|
|
|
Neil Schemenaue
#9 / 71
|
CMUCL performance in the Shootout Quote:
> The OCaml guys have had some spectacular sucesses in the ICFP
> contest (e.g. the Ray Tracer in '00), but also some big
> failures (e.g. last year).
I think you mean two years ago. Last year first place was taken by a team using OCaml. Neil
|
| Fri, 11 Nov 2005 00:34:04 GMT |
|
|
|
Bruce Houl
#10 / 71
|
CMUCL performance in the Shootout
Quote:
> > The OCaml guys have had some spectacular sucesses in the ICFP > > contest (e.g. the Ray Tracer in '00), but also some big > > failures (e.g. last year). > I think you mean two years ago. Last year first place was taken
> by a team using OCaml.
Hmm ... you're correct. I was thinking of "the OCaml guys" at INRIA (Camls 'R Us), who came in
81st last year.
There were a number of other teams using OCaml who did better. Which is
of course a good reflection on the language.
-- Bruce
|
| Fri, 11 Nov 2005 05:35:59 GMT |
|
|
|
David Steube
#11 / 71
|
CMUCL performance in the Shootout I love trolls. My favorite troll is one that has been sautaid (ispell sucks) in a mushroom, garlic, butter sauce. hmmm mmm good! Just don't expose them to sunlight. Stone is much harder to eat.
--
(describe 'describe)
|
| Fri, 11 Nov 2005 06:30:49 GMT |
|
|
|
Marc Spitze
#12 / 71
|
CMUCL performance in the Shootout Quote:
> I love trolls. My favorite troll is one that has been sautaid (ispell > sucks) in a mushroom, garlic, butter sauce. hmmm mmm good! > Just don't expose them to sunlight. Stone is much harder to eat.
But then you get lawn trolls, get enough and you can start your own miniature golf course/theme park. marc
|
| Fri, 11 Nov 2005 06:50:51 GMT |
|
|
|
Dan Schmid
#13 / 71
|
CMUCL performance in the Shootout
|
|| | The OCaml guys have had some spectacular sucesses in the ICFP
|| | contest (e.g. the Ray Tracer in '00), but also some big
|| | failures (e.g. last year).
||
|| I think you mean two years ago. Last year first place was taken
|| by a team using OCaml.
|
| Hmm ... you're correct.
|
| I was thinking of "the OCaml guys" at INRIA (Camls 'R Us), who came in
| 81st last year.
|
| There were a number of other teams using OCaml who did better. Which is
| of course a good reflection on the language.
My team came in 24th out of 168, and the three of us probably had
around 10 man-hours of experience in OCaml before the contest started.
That slowed us down, but there was a lot about OCaml that sped us up
even more. (I do like Lisp too.)
--
http://www.dfan.org
|
| Fri, 11 Nov 2005 11:18:04 GMT |
|
|
|
Michael Pa
#14 / 71
|
CMUCL performance in the Shootout Quote:
> > Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> > slower than C for matrix multiplication. All types were declared. On
> > my machine, when I test it from REPL, so there are no start-up costs,
> > Lisp is 4-5 times slower than C. Why?
> Looking at the disassembled code from both versions
> (which I'll call "C" and "L" with the obvious meanings):
> - C has an inner loop 9 instructions long; L has an
> inner loop 21 instructions long.
> - C keeps the accumulated sum in a register
> throughout the calculation; L spills it.
> - C keeps the inner loop's counter in a register
> throughout the calculation; L spills it.
> - C moves one indexing operation out of the inner loop;
> L doesn't.
1.) Is there any way I can convince CMUCL to perform these particular optimizations? Or do some kind of inline assembly? "Silly loops" like these are kind of important in the code I tend to write. 2.) I always kind of envied people who could read assembly. With no
prior knowledge of x86 registers or assembly language, how much
learning would be required (in earth years) to learn this kind of
stuff? Any specific tutorials suggestions? Thanks
P.S. I don't think "bounds checking" necessarily needs to be
compromised. Bounds checks can probably be optimised away, as no one
resizes the matrices while they are being multiplied.
P.P.S. I want Lisp programs to run faster, I must be a troll. Thanks
for noticing.
|
| Fri, 11 Nov 2005 14:22:55 GMT |
|
|
|
Michael Pa
#15 / 71
|
CMUCL performance in the Shootout Quote:
> > Doesn't this graph really show that we should all be using OCaml?
> I've played with OCaml and it certainly seems to be a very attractive
> language if you already know what you want before you start, but I never
> seem to. The OCaml guys have had some spectacular sucesses in the ICFP
> contest (e.g. the Ray Tracer in '00), but also some big failures (e.g.
> last year).
> Four weeks to the next one!
Four weeks or four months? Quote: > sticking with Dylan, but hoping to see more CL entries
How does one submit a CL entry? As code only? As an image? As fasl? What if their version of CMUCL is different?
|
| Fri, 11 Nov 2005 14:59:40 GMT |
|
| |
