Now that everything is "final".... 
Author Message
 Now that everything is "final"....

Has anyone done any performance evaluations of C# Vs. Java Vs C++ ?


Wed, 07 Jul 2004 13:07:33 GMT  
 Now that everything is "final"....

Quote:
> Has anyone done any performance evaluations of C# Vs. Java Vs C++ ?

Warning: The benchmarking clause is still in the EULA.

Regards,
Mark Hurd, B.Sc.(Ma.) (Hons.)



Thu, 08 Jul 2004 08:52:55 GMT  
 Now that everything is "final"....

Quote:


> > Has anyone done any performance evaluations of C# Vs. Java Vs C++ ?

> Warning: The benchmarking clause is still in the EULA.

That seems a bit daft. One can't do effective programming without
benchmarking. What precisely are they trying to outlaw.

mule
--
"jewels and binoculars hang from the head of the mule"
                                             Bob Dylan



Fri, 09 Jul 2004 01:16:37 GMT  
 Now that everything is "final"....

Quote:




> > > Has anyone done any performance evaluations of C# Vs. Java Vs C++ ?

> > Warning: The benchmarking clause is still in the EULA.

> That seems a bit daft. One can't do effective programming without
> benchmarking. What precisely are they trying to outlaw.

Publishing your results without asking for permission.

Quote:

> mule

Regards,
Mark Hurd, B.Sc.(Ma.) (Hons.)


Fri, 09 Jul 2004 07:06:39 GMT  
 Now that everything is "final"....

Quote:

>> What precisely are they trying to outlaw.

>Publishing your results without asking for permission.

Now, that's sad!  I just prepared to port OO Bench to C#, but this clause
makes it effectively infeasible to discuss its results and review it in
the public...

I hope Microsoft eventually gets a clue!

Best regards,

Sven C. Koehler

--
http://www.sf.net/projects/oobench/



Thu, 15 Jul 2004 10:12:37 GMT  
 Now that everything is "final"....


Quote:
> Now, that's sad!  I just prepared to port OO Bench to C#, but this clause
> makes it effectively infeasible to discuss its results and review it in
> the public...

The problem is that you might get it wrong -- good performance analysis is
*incredibly* hard to do!

Maybe your results show that Some Other Language (SOL) is 3 times faster
than C#, but it was because you had some really bad C# code, or a bad SQL
Server setup, or whatever. A journalist from a high-circulation publication
picks this up, and the front-page news the next day is that SOL is 3 times
faster than C#. Not good.

Most companies have clauses that say you can't publish benchmarks for just
this reason. There is nothing stopping you from *doing* the benchmarks for
your own private use, but you can't *publish* them to the world.

Peter

--

Waiting for the Vengabus? http://www.microsoft.com/info/cpyright.htm
Please post all questions to the group. Thanks.



Fri, 16 Jul 2004 07:13:21 GMT  
 Now that everything is "final"....

<snip>

Quote:

> The problem is that you might get it wrong -- good performance analysis is
> *incredibly* hard to do!

This doesn't seem to stop MS from making claims about performance of
competitors.  A fine example of 'do as I say, not as I do'.  Shameful.

Quote:

> Maybe your results show that Some Other Language (SOL) is 3 times faster
> than C#, but it was because you had some really bad C# code, or a bad SQL
> Server setup, or whatever. A journalist from a high-circulation publication
> picks this up, and the front-page news the next day is that SOL is 3 times
> faster than C#. Not good.

Nonsense.  Utter nonsense.  Is this the best MS has to offer in terms of
spin?

Jim S.



Fri, 16 Jul 2004 22:57:42 GMT  
 Now that everything is "final"....

Quote:
> Nonsense.  Utter nonsense.  Is this the best MS has to offer in terms of
> spin?

Hi Jim,

I think you are mistaken when you refer to this as "spin" -- I'm about as
far away from marketing as you can get. I also think you are mistaken if you
think this is "nonsense".

But just to clear things up that was *my* take on the issue. There may be
other, completely different reasons why Microsoft doesn't want you to
publish benchmarks about ASP .NET in this particular case, but that's the
reason we don't like people publishing "benchmarks" about script. People are
often claiming that VBScript is N times slower (or faster) than JScript when
the problem was just that their code was very bad.

I've seen one website where the author went to a lot of trouble writing
various tests and generating lots of nice graphs to show that JScript was
much faster than VBScript. When we looked at the code (accidentally, of
course, because we were never made aware of the "benchmarks" until another
customer referenced them in some random newsgroup post) it turned out that
the VBScript code was horrible (not surprising -- this was a JScript
developer trying to prove a point) and that if you re-wrote the code
correctly, VBScript was actually about the same speed or faster.

Peter

--

Waiting for the Vengabus? http://www.microsoft.com/info/cpyright.htm
Please post all questions to the group. Thanks.



Sat, 17 Jul 2004 01:24:53 GMT  
 Now that everything is "final"....

<snip>

Quote:

> But just to clear things up that was *my* take on the issue. There may be
> other, completely different reasons why Microsoft doesn't want you to
> publish benchmarks about ASP .NET in this particular case, but that's the
> reason we don't like people publishing "benchmarks" about script. People are
> often claiming that VBScript is N times slower (or faster) than JScript when
> the problem was just that their code was very bad.

When did this become a discussion about ASP .NET?  The 'no-bechmarking'
nonsense applies to the *entire* .NET SDK.  People have been writing bad
benchmarks for years, and I don't recall anyone going out of business
because of it.

This is a convenient excuse to prevent the dissemination of the volumes
of good, impartial information which would result from benchmarks
performed by the people actually using the language/libraries to do real
work.

Jim S.



Sat, 17 Jul 2004 02:15:47 GMT  
 Now that everything is "final"....

Quote:
> This is a convenient excuse to prevent the dissemination of the volumes
> of good, impartial information which would result from benchmarks
> performed by the people actually using the language/libraries to do real
> work.

My whole point was that what you usually get is not "good, impartial
information". You tend to get bad, biased data (either accidentally or
maliciously). But again, I don't speak for Microsoft officially, and I'm not
suggesting that you as an individual would post bad data -- just that in
general it is what tends to happen.

There is nothing stopping you doing benchmarks in the privacy of your own
lab.

Peter

--

Waiting for the Vengabus? http://www.microsoft.com/info/cpyright.htm
Please post all questions to the group. Thanks.



Sat, 17 Jul 2004 02:58:46 GMT  
 Now that everything is "final"....
Peter, first of all: thanks for your answer!

Quote:



>> Now, that's sad!  I just prepared to port OO Bench to C#, but this clause
>> makes it effectively infeasible to discuss its results and review it in
>> the public...

>The problem is that you might get it wrong -- good performance analysis is
>*incredibly* hard to do!

> [Description of benchmarks that are biased towards a certain language
> by using bad code.]

I agree to this 100%.  In my experience, it's even worse; even, if one
writes a benchmark, and uses the language and its interfaces as intended
and advised by the creator, it doesn't need to make you more wise,
neccessarly.  I have yet identified the following additional risks
regarding the use of benchmarks:

1. Dynamic optimization, that depends on how components interact.  This
becomes even more true, if VMs enter the game, which traditionally can
make far more sophisticated optimizations as one could by using merely
static analysis.  Most simple micro benchmarks don't dignify this.

2. If certain kinds of benchmarks become popular, language creators tend
to optimize their languages to "tune" the results, without gaining any
performance for real use. (David A. Patterson et al. describe this IMO
very nicely in chapter 2 of "Computer Organization & Design" in the context
of CPU benchmarks.)

3. Benchmarks usually don't include the time one needs to write the code,
and maintain it.

Quote:
>Maybe your results show that Some Other Language (SOL) is 3 times faster
>than C#, but it was because you had some really bad C# code, or a bad SQL
>Server setup, or whatever. A journalist from a high-circulation publication
>picks this up, and the front-page news the next day is that SOL is 3 times
>faster than C#. Not good.

In my opinion, the only way to appropriately design benchmarks is to let
others review the results and the code that lead to them.  So, maybe I
should put OO Bench's benchmark results under a certain license that
forbids publishing benchmarks, except for the intend of finding bugs in
the benchmark itself, and only to a an appropriate audience--would I then
comply to MS's .NET EULA?

All the best,

Sven



Sat, 17 Jul 2004 04:26:13 GMT  
 Now that everything is "final"....


Quote:
> In my opinion, the only way to appropriately design benchmarks is to let
> others review the results and the code that lead to them.  So, maybe I
> should put OO Bench's benchmark results under a certain license that
> forbids publishing benchmarks, except for the intend of finding bugs in
> the benchmark itself, and only to a an appropriate audience--would I then
> comply to MS's .NET EULA?

> All the best,

> Sven

The thing that seems to be being missed (or perhaps it is assumed to be not
worth trying) is that it doesn't say you can't publish; it says you have to
get permission.

If your website is a well regarded benchmarking site, I'd expect that
Microsoft would want to work with you to get .NET benchmarks on your pages.

Regards,
Mark Hurd, B.Sc.(Ma.) (Hons.)



Sun, 18 Jul 2004 09:52:55 GMT  
 Now that everything is "final"....



Quote:

> >> What precisely are they trying to outlaw.

> >Publishing your results without asking for permission.

> Now, that's sad!  I just prepared to port OO Bench to C#, but this
clause
> makes it effectively infeasible to discuss its results and review it
in
> the public...

> I hope Microsoft eventually gets a clue!

The State of New York may give them a clue soon.

"N.Y. Accuses software maker of censorship"

http://news.com.com/2100-1023-832221.html?tag=cd_mh



Fri, 30 Jul 2004 11:25:32 GMT  
 Now that everything is "final"....

Quote:

>"N.Y. Accuses software maker of censorship"

>http://news.com.com/2100-1023-832221.html?tag=cd_mh

Thanks for the link!  At least there is some slight hope for a shift now.

Best regards,

Sven C. Koehler



Mon, 02 Aug 2004 07:52:26 GMT  
 
 [ 14 post ] 

 Relevant Pages 

1. ANNOUNCE: "Bit::Vector" 5.1 (final release)

2. I am going to study "C"

3. remove() vrs fopen("""w")

4. Displaying binary data as ascii "1"'s and "0"'s

5. Looking for "Shroud"/"Obfus"

6. ""help with TSR""

7. Parse trees and "("")"

8. Error "free"-ing "malloc"-ed memory

9. Displaying binary data as ascii "1"'s and "0"'s

10. Attention "C"/"C++" gurus and "C" newbees

11. merits of "#define", "const", and "enum" ???

12. why not "cout", "<<" , and "endl"??

 

 
Powered by phpBB® Forum Software