IO performance C# Versus C++ (For C# GURU's only) 
Author Message
 IO performance C# Versus C++ (For C# GURU's only)

The IO perfomance of C# is terrible, compared to the C++ routine.

I have recently ported a small C++ program which uses an ANSI C file routine. The program calculates a CRC32 for any given filename.

1. opens an existing file.
2. read one byte - perform bit manipulation - read one byte - repeat
3. display crc32 value
===========================
Performance times using a 32Megabyte file:
2         seconds for C++ routine
105     seconds for C## routine
===========================

Does anyone know how to increase this performance?

public void getcrc(string fullpath)
{
  byte buffer = 0xff;

  FileStream file = new FileStream(fullpath, FileMode.Open);

  uint crc;
  byte b1;
  crc = 0xFFFFFFFF;
  for(uint x = 0; x < file.Length; x++)
  {
    b1 = (byte)file.ReadByte();
    crc = ((crc>>8) & 0x00FFFFFF) ^ crcTable[ (crc^b1) & 0xFF ];
  }
    crc_32 = ( crc^0xFFFFFFFF );
  }

// Thanks for looking at this



Wed, 09 Jun 2004 14:57:36 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Quote:
>> Does anyone know how to increase this performance?

Oh yes, indeed!!!

worst was the line:
   for(uint x = 0; x < file.Length; x++)
file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow
Doing buffering and algorithm simplification gets some more speed.

with my code below, I'm faster then your C++!
.NET Beta2 release-build, XP, PIII/933 :  672 ms for 56 MByte file !

Don't forget, with great final .NET and good compiler/CLR knowledge,
you will get speed comparable to native C++!  ;-)

// ================================= Print CRC32 (ZIP compatible) =================================
using System;
using System.Threading;
using System.IO;
namespace Crc32SpeedUp
{
class TestCRC
 {
 static uint[] crcTable = new uint[] {
 0x000000000, 0x077073096, 0x0EE0E612C, 0x0990951BA, 0x0076DC419, 0x0706AF48F, 0x0E963A535, 0x09E6495A3,
 0x00EDB8832, 0x079DCB8A4, 0x0E0D5E91E, 0x097D2D988, 0x009B64C2B, 0x07EB17CBD, 0x0E7B82D07, 0x090BF1D91,
 0x01DB71064, 0x06AB020F2, 0x0F3B97148, 0x084BE41DE, 0x01ADAD47D, 0x06DDDE4EB, 0x0F4D4B551, 0x083D385C7,
 0x0136C9856, 0x0646BA8C0, 0x0FD62F97A, 0x08A65C9EC, 0x014015C4F, 0x063066CD9, 0x0FA0F3D63, 0x08D080DF5,
 0x03B6E20C8, 0x04C69105E, 0x0D56041E4, 0x0A2677172, 0x03C03E4D1, 0x04B04D447, 0x0D20D85FD, 0x0A50AB56B,
 0x035B5A8FA, 0x042B2986C, 0x0DBBBC9D6, 0x0ACBCF940, 0x032D86CE3, 0x045DF5C75, 0x0DCD60DCF, 0x0ABD13D59,
 0x026D930AC, 0x051DE003A, 0x0C8D75180, 0x0BFD06116, 0x021B4F4B5, 0x056B3C423, 0x0CFBA9599, 0x0B8BDA50F,
 0x02802B89E, 0x05F058808, 0x0C60CD9B2, 0x0B10BE924, 0x02F6F7C87, 0x058684C11, 0x0C1611DAB, 0x0B6662D3D,
 0x076DC4190, 0x001DB7106, 0x098D220BC, 0x0EFD5102A, 0x071B18589, 0x006B6B51F, 0x09FBFE4A5, 0x0E8B8D433,
 0x07807C9A2, 0x00F00F934, 0x09609A88E, 0x0E10E9818, 0x07F6A0DBB, 0x0086D3D2D, 0x091646C97, 0x0E6635C01,
 0x06B6B51F4, 0x01C6C6162, 0x0856530D8, 0x0F262004E, 0x06C0695ED, 0x01B01A57B, 0x08208F4C1, 0x0F50FC457,
 0x065B0D9C6, 0x012B7E950, 0x08BBEB8EA, 0x0FCB9887C, 0x062DD1DDF, 0x015DA2D49, 0x08CD37CF3, 0x0FBD44C65,
 0x04DB26158, 0x03AB551CE, 0x0A3BC0074, 0x0D4BB30E2, 0x04ADFA541, 0x03DD895D7, 0x0A4D1C46D, 0x0D3D6F4FB,
 0x04369E96A, 0x0346ED9FC, 0x0AD678846, 0x0DA60B8D0, 0x044042D73, 0x033031DE5, 0x0AA0A4C5F, 0x0DD0D7CC9,
 0x05005713C, 0x0270241AA, 0x0BE0B1010, 0x0C90C2086, 0x05768B525, 0x0206F85B3, 0x0B966D409, 0x0CE61E49F,
 0x05EDEF90E, 0x029D9C998, 0x0B0D09822, 0x0C7D7A8B4, 0x059B33D17, 0x02EB40D81, 0x0B7BD5C3B, 0x0C0BA6CAD,
 0x0EDB88320, 0x09ABFB3B6, 0x003B6E20C, 0x074B1D29A, 0x0EAD54739, 0x09DD277AF, 0x004DB2615, 0x073DC1683,
 0x0E3630B12, 0x094643B84, 0x00D6D6A3E, 0x07A6A5AA8, 0x0E40ECF0B, 0x09309FF9D, 0x00A00AE27, 0x07D079EB1,
 0x0F00F9344, 0x08708A3D2, 0x01E01F268, 0x06906C2FE, 0x0F762575D, 0x0806567CB, 0x0196C3671, 0x06E6B06E7,
 0x0FED41B76, 0x089D32BE0, 0x010DA7A5A, 0x067DD4ACC, 0x0F9B9DF6F, 0x08EBEEFF9, 0x017B7BE43, 0x060B08ED5,
 0x0D6D6A3E8, 0x0A1D1937E, 0x038D8C2C4, 0x04FDFF252, 0x0D1BB67F1, 0x0A6BC5767, 0x03FB506DD, 0x048B2364B,
 0x0D80D2BDA, 0x0AF0A1B4C, 0x036034AF6, 0x041047A60, 0x0DF60EFC3, 0x0A867DF55, 0x0316E8EEF, 0x04669BE79,
 0x0CB61B38C, 0x0BC66831A, 0x0256FD2A0, 0x05268E236, 0x0CC0C7795, 0x0BB0B4703, 0x0220216B9, 0x05505262F,
 0x0C5BA3BBE, 0x0B2BD0B28, 0x02BB45A92, 0x05CB36A04, 0x0C2D7FFA7, 0x0B5D0CF31, 0x02CD99E8B, 0x05BDEAE1D,
 0x09B64C2B0, 0x0EC63F226, 0x0756AA39C, 0x0026D930A, 0x09C0906A9, 0x0EB0E363F, 0x072076785, 0x005005713,
 0x095BF4A82, 0x0E2B87A14, 0x07BB12BAE, 0x00CB61B38, 0x092D28E9B, 0x0E5D5BE0D, 0x07CDCEFB7, 0x00BDBDF21,
 0x086D3D2D4, 0x0F1D4E242, 0x068DDB3F8, 0x01FDA836E, 0x081BE16CD, 0x0F6B9265B, 0x06FB077E1, 0x018B74777,
 0x088085AE6, 0x0FF0F6A70, 0x066063BCA, 0x011010B5C, 0x08F659EFF, 0x0F862AE69, 0x0616BFFD3, 0x0166CCF45,
 0x0A00AE278, 0x0D70DD2EE, 0x04E048354, 0x03903B3C2, 0x0A7672661, 0x0D06016F7, 0x04969474D, 0x03E6E77DB,
 0x0AED16A4A, 0x0D9D65ADC, 0x040DF0B66, 0x037D83BF0, 0x0A9BCAE53, 0x0DEBB9EC5, 0x047B2CF7F, 0x030B5FFE9,
 0x0BDBDF21C, 0x0CABAC28A, 0x053B39330, 0x024B4A3A6, 0x0BAD03605, 0x0CDD70693, 0x054DE5729, 0x023D967BF,
 0x0B3667A2E, 0x0C4614AB8, 0x05D681B02, 0x02A6F2B94, 0x0B40BBE37, 0x0C30C8EA1, 0x05A05DF1B, 0x02D02EF8D };

 static void Main(string[] args)
  {
  if( args.Length < 1 )
   { Console.WriteLine( "SYNTAX: TestCRC file" ); return; }
  Thread.Sleep( 200 );
  int s = Environment.TickCount;
  uint crc = getcrc( args[0] );
  int e = Environment.TickCount;
  Console.WriteLine( "crc32 = {0:x} in {1} ms", crc, e - s );
  }

 public static uint getcrc( string fullpath )
  {
  FileStream file = new FileStream( fullpath, FileMode.Open, FileAccess.Read, FileShare.Read, 16384 );
  long len = file.Length;
  byte[] adat = new byte[ 16384 ];
  int rr = 0;
  uint crc = 0xFFFFFFFF;
  do
   {
   rr = file.Read( adat, 0, 16384 );
   for( int i = 0; i < rr; i++ )
    {
    byte b1 = (byte) (crc ^ adat[i]);
    crc = (crc>>8) ^ crcTable[ b1 ];
    }
   }
  while( rr > 0 );
  return crc ^ 0xFFFFFFFF;
  }
 }

Quote:
}

// ===================================================================================

  Wonna my pentium-assembler version? sooner finished then started!   ;-)

  NETMaster
  http://www.cetus-links.org/oo_csharp.html

The IO perfomance of C# is terrible, compared to the C++ routine.
I have recently ported a small C++ program which uses an ANSI C file routine. The program calculates a CRC32 for any given filename.
1. opens an existing file.
2. read one byte - perform bit manipulation - read one byte - repeat
3. display crc32 value
===========================
Performance times using a 32Megabyte file:
2         seconds for C++ routine
105     seconds for C## routine
===========================
Does anyone know how to increase this performance?
public void getcrc(string fullpath)
{
  byte buffer = 0xff;
  FileStream file = new FileStream(fullpath, FileMode.Open);
  uint crc;
  byte b1;
  crc = 0xFFFFFFFF;
  for(uint x = 0; x < file.Length; x++)
  {
    b1 = (byte)file.ReadByte();
    crc = ((crc>>8) & 0x00FFFFFF) ^ crcTable[ (crc^b1) & 0xFF ];
  }
    crc_32 = ( crc^0xFFFFFFFF );
  }



Wed, 09 Jun 2004 20:23:39 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
So i Know that you first run a program in C# is slowest than you run in
second time , did you test it ???

Quote:
>>> >> Does anyone know how to increase this performance?
>>> Oh yes, indeed!!!

>>> worst was the line:
>>>    for(uint x = 0; x < file.Length; x++)
>>> file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow
>>> Doing buffering and algorithm simplification gets some more speed.

>>> with my code below, I'm faster then your C++!
>>> .NET Beta2 release-build, XP, PIII/933 :  672 ms for 56 MByte file !

>>> Don't forget, with great final .NET and good compiler/CLR knowledge,
>>> you will get speed comparable to native C++!  ;-)

>>> // ================================= Print CRC32 (ZIP compatible) =================================

--
Abra?os
----
Daniel Reis de Almeida Prado - SilverHawk
www.SilverHawk.com.br
Nextel : 1*14199


Thu, 10 Jun 2004 00:37:47 GMT  
 IO performance C# Versus C++ (For C# GURU's only)

Quote:

> >> Does anyone know how to increase this performance?
> Oh yes, indeed!!!

> worst was the line:
>    for(uint x = 0; x < file.Length; x++)
> file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow

Hi,
Absolutely right, although P/Invoke has some overhead but the main portion of the time is consumed by the 32 Million GetFileSize API
call's.
The lesson learned is 'be careful when using properties in for loops' especially when benchmarking :-).

Willy.



Thu, 10 Jun 2004 00:35:20 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Yippee! Your code runs like a lightning bolt, compared to my code! And...
thank you for taking the time to respond to my question.

This little routine (CRC32), is the keystone of a small utility which our
company uses, it was beginning to look like I was never going to get it
ported to C#.
I had been working on improving the performance for 4 days, I really didn't
expect anyone to rescue me!

You have no idea how happy you have made me.
1. Your coding efforts, have proven to me (without doubt), that C# is as
capable as C++/C language, at least for I/O performance.
2. Since I am just a beginner (an "advanced beginner" <grin>), I wanted to
take this new programming paradigm shift, from C++ to C#.
3. C++ , is a very powerful language, but as a beginner, the C++ language
makes me work too hard for small results.
4. I was very close to giving up on the C# language, returning to C++. I was
going to give C# another year to mature, this would have been a mistake.

What does this line of code do?     Thread.Sleep(200);     I have not seen
this before.

Once again, thank you for the nice "Christmas Present".


Quote:
> >> Does anyone know how to increase this performance?
> Oh yes, indeed!!!

> worst was the line:
>    for(uint x = 0; x < file.Length; x++)
> file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow
> Doing buffering and algorithm simplification gets some more speed.

> with my code below, I'm faster then your C++!
> .NET Beta2 release-build, XP, PIII/933 :  672 ms for 56 MByte file !

> Don't forget, with great final .NET and good compiler/CLR knowledge,
> you will get speed comparable to native C++!  ;-)

> // ================================= Print CRC32 (ZIP compatible)

=================================
Quote:
> using System;
> using System.Threading;
> using System.IO;
> namespace Crc32SpeedUp
> {
> class TestCRC
>  {
>  static uint[] crcTable = new uint[] {
>  0x000000000, 0x077073096, 0x0EE0E612C, 0x0990951BA, 0x0076DC419,

0x0706AF48F, 0x0E963A535, 0x09E6495A3,
Quote:
>  0x00EDB8832, 0x079DCB8A4, 0x0E0D5E91E, 0x097D2D988, 0x009B64C2B,

0x07EB17CBD, 0x0E7B82D07, 0x090BF1D91,
Quote:
>  0x01DB71064, 0x06AB020F2, 0x0F3B97148, 0x084BE41DE, 0x01ADAD47D,

0x06DDDE4EB, 0x0F4D4B551, 0x083D385C7,
Quote:
>  0x0136C9856, 0x0646BA8C0, 0x0FD62F97A, 0x08A65C9EC, 0x014015C4F,

0x063066CD9, 0x0FA0F3D63, 0x08D080DF5,
Quote:
>  0x03B6E20C8, 0x04C69105E, 0x0D56041E4, 0x0A2677172, 0x03C03E4D1,

0x04B04D447, 0x0D20D85FD, 0x0A50AB56B,
Quote:
>  0x035B5A8FA, 0x042B2986C, 0x0DBBBC9D6, 0x0ACBCF940, 0x032D86CE3,

0x045DF5C75, 0x0DCD60DCF, 0x0ABD13D59,
Quote:
>  0x026D930AC, 0x051DE003A, 0x0C8D75180, 0x0BFD06116, 0x021B4F4B5,

0x056B3C423, 0x0CFBA9599, 0x0B8BDA50F,
Quote:
>  0x02802B89E, 0x05F058808, 0x0C60CD9B2, 0x0B10BE924, 0x02F6F7C87,

0x058684C11, 0x0C1611DAB, 0x0B6662D3D,
Quote:
>  0x076DC4190, 0x001DB7106, 0x098D220BC, 0x0EFD5102A, 0x071B18589,

0x006B6B51F, 0x09FBFE4A5, 0x0E8B8D433,
Quote:
>  0x07807C9A2, 0x00F00F934, 0x09609A88E, 0x0E10E9818, 0x07F6A0DBB,

0x0086D3D2D, 0x091646C97, 0x0E6635C01,
Quote:
>  0x06B6B51F4, 0x01C6C6162, 0x0856530D8, 0x0F262004E, 0x06C0695ED,

0x01B01A57B, 0x08208F4C1, 0x0F50FC457,
Quote:
>  0x065B0D9C6, 0x012B7E950, 0x08BBEB8EA, 0x0FCB9887C, 0x062DD1DDF,

0x015DA2D49, 0x08CD37CF3, 0x0FBD44C65,
Quote:
>  0x04DB26158, 0x03AB551CE, 0x0A3BC0074, 0x0D4BB30E2, 0x04ADFA541,

0x03DD895D7, 0x0A4D1C46D, 0x0D3D6F4FB,
Quote:
>  0x04369E96A, 0x0346ED9FC, 0x0AD678846, 0x0DA60B8D0, 0x044042D73,

0x033031DE5, 0x0AA0A4C5F, 0x0DD0D7CC9,
Quote:
>  0x05005713C, 0x0270241AA, 0x0BE0B1010, 0x0C90C2086, 0x05768B525,

0x0206F85B3, 0x0B966D409, 0x0CE61E49F,
Quote:
>  0x05EDEF90E, 0x029D9C998, 0x0B0D09822, 0x0C7D7A8B4, 0x059B33D17,

0x02EB40D81, 0x0B7BD5C3B, 0x0C0BA6CAD,
Quote:
>  0x0EDB88320, 0x09ABFB3B6, 0x003B6E20C, 0x074B1D29A, 0x0EAD54739,

0x09DD277AF, 0x004DB2615, 0x073DC1683,
Quote:
>  0x0E3630B12, 0x094643B84, 0x00D6D6A3E, 0x07A6A5AA8, 0x0E40ECF0B,

0x09309FF9D, 0x00A00AE27, 0x07D079EB1,
Quote:
>  0x0F00F9344, 0x08708A3D2, 0x01E01F268, 0x06906C2FE, 0x0F762575D,

0x0806567CB, 0x0196C3671, 0x06E6B06E7,
Quote:
>  0x0FED41B76, 0x089D32BE0, 0x010DA7A5A, 0x067DD4ACC, 0x0F9B9DF6F,

0x08EBEEFF9, 0x017B7BE43, 0x060B08ED5,
Quote:
>  0x0D6D6A3E8, 0x0A1D1937E, 0x038D8C2C4, 0x04FDFF252, 0x0D1BB67F1,

0x0A6BC5767, 0x03FB506DD, 0x048B2364B,
Quote:
>  0x0D80D2BDA, 0x0AF0A1B4C, 0x036034AF6, 0x041047A60, 0x0DF60EFC3,

0x0A867DF55, 0x0316E8EEF, 0x04669BE79,
Quote:
>  0x0CB61B38C, 0x0BC66831A, 0x0256FD2A0, 0x05268E236, 0x0CC0C7795,

0x0BB0B4703, 0x0220216B9, 0x05505262F,
Quote:
>  0x0C5BA3BBE, 0x0B2BD0B28, 0x02BB45A92, 0x05CB36A04, 0x0C2D7FFA7,

0x0B5D0CF31, 0x02CD99E8B, 0x05BDEAE1D,
Quote:
>  0x09B64C2B0, 0x0EC63F226, 0x0756AA39C, 0x0026D930A, 0x09C0906A9,

0x0EB0E363F, 0x072076785, 0x005005713,
Quote:
>  0x095BF4A82, 0x0E2B87A14, 0x07BB12BAE, 0x00CB61B38, 0x092D28E9B,

0x0E5D5BE0D, 0x07CDCEFB7, 0x00BDBDF21,
Quote:
>  0x086D3D2D4, 0x0F1D4E242, 0x068DDB3F8, 0x01FDA836E, 0x081BE16CD,

0x0F6B9265B, 0x06FB077E1, 0x018B74777,
Quote:
>  0x088085AE6, 0x0FF0F6A70, 0x066063BCA, 0x011010B5C, 0x08F659EFF,

0x0F862AE69, 0x0616BFFD3, 0x0166CCF45,
Quote:
>  0x0A00AE278, 0x0D70DD2EE, 0x04E048354, 0x03903B3C2, 0x0A7672661,

0x0D06016F7, 0x04969474D, 0x03E6E77DB,
Quote:
>  0x0AED16A4A, 0x0D9D65ADC, 0x040DF0B66, 0x037D83BF0, 0x0A9BCAE53,

0x0DEBB9EC5, 0x047B2CF7F, 0x030B5FFE9,
Quote:
>  0x0BDBDF21C, 0x0CABAC28A, 0x053B39330, 0x024B4A3A6, 0x0BAD03605,

0x0CDD70693, 0x054DE5729, 0x023D967BF,
Quote:
>  0x0B3667A2E, 0x0C4614AB8, 0x05D681B02, 0x02A6F2B94, 0x0B40BBE37,

0x0C30C8EA1, 0x05A05DF1B, 0x02D02EF8D };
Quote:

>  static void Main(string[] args)
>   {
>   if( args.Length < 1 )
>    { Console.WriteLine( "SYNTAX: TestCRC file" ); return; }
>   Thread.Sleep( 200 );
>   int s = Environment.TickCount;
>   uint crc = getcrc( args[0] );
>   int e = Environment.TickCount;
>   Console.WriteLine( "crc32 = {0:x} in {1} ms", crc, e - s );
>   }

>  public static uint getcrc( string fullpath )
>   {
>   FileStream file = new FileStream( fullpath, FileMode.Open,

FileAccess.Read, FileShare.Read, 16384 );

- Show quoted text -

Quote:
>   long len = file.Length;
>   byte[] adat = new byte[ 16384 ];
>   int rr = 0;
>   uint crc = 0xFFFFFFFF;
>   do
>    {
>    rr = file.Read( adat, 0, 16384 );
>    for( int i = 0; i < rr; i++ )
>     {
>     byte b1 = (byte) (crc ^ adat[i]);
>     crc = (crc>>8) ^ crcTable[ b1 ];
>     }
>    }
>   while( rr > 0 );
>   return crc ^ 0xFFFFFFFF;
>   }
>  }
> }
> //

============================================================================
=======
Quote:

>   Wonna my pentium-assembler version? sooner finished then started!   ;-)

>   NETMaster
>   http://www.cetus-links.org/oo_csharp.html




Quote:
> The IO perfomance of C# is terrible, compared to the C++ routine.
> I have recently ported a small C++ program which uses an ANSI C file

routine. The program calculates a CRC32 for any given filename.

- Show quoted text -

Quote:
> 1. opens an existing file.
> 2. read one byte - perform bit manipulation - read one byte - repeat
> 3. display crc32 value
> ===========================
> Performance times using a 32Megabyte file:
> 2         seconds for C++ routine
> 105     seconds for C## routine
> ===========================
> Does anyone know how to increase this performance?
> public void getcrc(string fullpath)
> {
>   byte buffer = 0xff;
>   FileStream file = new FileStream(fullpath, FileMode.Open);
>   uint crc;
>   byte b1;
>   crc = 0xFFFFFFFF;
>   for(uint x = 0; x < file.Length; x++)
>   {
>     b1 = (byte)file.ReadByte();
>     crc = ((crc>>8) & 0x00FFFFFF) ^ crcTable[ (crc^b1) & 0xFF ];
>   }
>     crc_32 = ( crc^0xFFFFFFFF );
>   }



Thu, 10 Jun 2004 07:39:42 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Willy, thanks for the reply, nice to see quality assistance when needed.

Can you clarify your statement? I don't understand what you mean about
P/Invoke. I do understand your last point, about the GetFileSize API call

Quote:
>"Absolutely right, although P/Invoke has some overhead but the main portion

of the time is consumed by the 32 Million GetFileSize API call's."
.
I thought I was smart, when I used the "file.Length" sentinel. It looked so
clean to me, Wow... was this a mistake or what!
The reason I used this "file.Length" thing, is because I was struggling with
converting a C++ while statement to C#.
I kept running past the end of the stream using the while statement in C#,
so I used a for loop, and file.Length sentinel.

Quote:
>The lesson learned is 'be careful when using properties in for loops'

especially when benchmarking :-).
Wow! How right you are!

Merry Xmas!


Quote:




Quote:
> > >> Does anyone know how to increase this performance?
> > Oh yes, indeed!!!

> > worst was the line:
> >    for(uint x = 0; x < file.Length; x++)
> > file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow

> Hi,
> Absolutely right, although P/Invoke has some overhead but the main portion

of the time is consumed by the 32 Million GetFileSize API
Quote:
> call's.
> The lesson learned is 'be careful when using properties in for loops'

especially when benchmarking :-).
Quote:

> Willy.



Thu, 10 Jun 2004 07:57:55 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
I figured Willy has done his part (nice job BTW), so I'd save him some typing.

The runtime walks the stack for permissions before PInvoking (calling extern
function). There is some overhead in doing so.

Also, if the types you are marshalling to the external func aren't blittable,
conversions have to be made in/out for those types.

There's a bit more, but that's the gist of it.

Keep Smilin'
Ed Stegman


Quote:
> Willy, thanks for the reply, nice to see quality assistance when needed.

> Can you clarify your statement? I don't understand what you mean about
> P/Invoke. I do understand your last point, about the GetFileSize API call
> >"Absolutely right, although P/Invoke has some overhead but the main portion
> of the time is consumed by the 32 Million GetFileSize API call's."
> .
> I thought I was smart, when I used the "file.Length" sentinel. It looked so
> clean to me, Wow... was this a mistake or what!
> The reason I used this "file.Length" thing, is because I was struggling with
> converting a C++ while statement to C#.
> I kept running past the end of the stream using the while statement in C#,
> so I used a for loop, and file.Length sentinel.

> >The lesson learned is 'be careful when using properties in for loops'
> especially when benchmarking :-).
> Wow! How right you are!

> Merry Xmas!





> > > >> Does anyone know how to increase this performance?
> > > Oh yes, indeed!!!

> > > worst was the line:
> > >    for(uint x = 0; x < file.Length; x++)
> > > file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow

> > Hi,
> > Absolutely right, although P/Invoke has some overhead but the main portion
> of the time is consumed by the 32 Million GetFileSize API
> > call's.
> > The lesson learned is 'be careful when using properties in for loops'
> especially when benchmarking :-).

> > Willy.



Thu, 10 Jun 2004 09:50:41 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
it sends a thread to bed. When it is gettin tired, it may wanna sleep ;-)
Anyway, with this line of code, you kind of deactivate a thread for in this
case 200 milli seconds.
Imagine a "while(true){NOP;}" scenario. Even though nothing is done, the
thread would take all
the power of the processor. Extreme example, but you could use
"while(true){NOP; myThread.Sleep(200)}"
to enable other threads to do some work while this ridiculous thread is
doing its NOP's.(actually when the thread is in the Sleep "routine").

Regards,
Chris


Quote:
> Yippee! Your code runs like a lightning bolt, compared to my code! And...
> thank you for taking the time to respond to my question.

> This little routine (CRC32), is the keystone of a small utility which our
> company uses, it was beginning to look like I was never going to get it
> ported to C#.
> I had been working on improving the performance for 4 days, I really
didn't
> expect anyone to rescue me!

> You have no idea how happy you have made me.
> 1. Your coding efforts, have proven to me (without doubt), that C# is as
> capable as C++/C language, at least for I/O performance.
> 2. Since I am just a beginner (an "advanced beginner" <grin>), I wanted to
> take this new programming paradigm shift, from C++ to C#.
> 3. C++ , is a very powerful language, but as a beginner, the C++ language
> makes me work too hard for small results.
> 4. I was very close to giving up on the C# language, returning to C++. I
was
> going to give C# another year to mature, this would have been a mistake.

> What does this line of code do?     Thread.Sleep(200);     I have not seen
> this before.

> Once again, thank you for the nice "Christmas Present".



> > >> Does anyone know how to increase this performance?
> > Oh yes, indeed!!!

> > worst was the line:
> >    for(uint x = 0; x < file.Length; x++)
> > file.Length is implemented as a PInvoke to WIN32:GetFileSize => slow
> > Doing buffering and algorithm simplification gets some more speed.

> > with my code below, I'm faster then your C++!
> > .NET Beta2 release-build, XP, PIII/933 :  672 ms for 56 MByte file !

> > Don't forget, with great final .NET and good compiler/CLR knowledge,
> > you will get speed comparable to native C++!  ;-)

> > // ================================= Print CRC32 (ZIP compatible)
> =================================
> > using System;
> > using System.Threading;
> > using System.IO;
> > namespace Crc32SpeedUp
> > {
> > class TestCRC
> >  {
> >  static uint[] crcTable = new uint[] {
> >  0x000000000, 0x077073096, 0x0EE0E612C, 0x0990951BA, 0x0076DC419,
> 0x0706AF48F, 0x0E963A535, 0x09E6495A3,
> >  0x00EDB8832, 0x079DCB8A4, 0x0E0D5E91E, 0x097D2D988, 0x009B64C2B,
> 0x07EB17CBD, 0x0E7B82D07, 0x090BF1D91,
> >  0x01DB71064, 0x06AB020F2, 0x0F3B97148, 0x084BE41DE, 0x01ADAD47D,
> 0x06DDDE4EB, 0x0F4D4B551, 0x083D385C7,
> >  0x0136C9856, 0x0646BA8C0, 0x0FD62F97A, 0x08A65C9EC, 0x014015C4F,
> 0x063066CD9, 0x0FA0F3D63, 0x08D080DF5,
> >  0x03B6E20C8, 0x04C69105E, 0x0D56041E4, 0x0A2677172, 0x03C03E4D1,
> 0x04B04D447, 0x0D20D85FD, 0x0A50AB56B,
> >  0x035B5A8FA, 0x042B2986C, 0x0DBBBC9D6, 0x0ACBCF940, 0x032D86CE3,
> 0x045DF5C75, 0x0DCD60DCF, 0x0ABD13D59,
> >  0x026D930AC, 0x051DE003A, 0x0C8D75180, 0x0BFD06116, 0x021B4F4B5,
> 0x056B3C423, 0x0CFBA9599, 0x0B8BDA50F,
> >  0x02802B89E, 0x05F058808, 0x0C60CD9B2, 0x0B10BE924, 0x02F6F7C87,
> 0x058684C11, 0x0C1611DAB, 0x0B6662D3D,
> >  0x076DC4190, 0x001DB7106, 0x098D220BC, 0x0EFD5102A, 0x071B18589,
> 0x006B6B51F, 0x09FBFE4A5, 0x0E8B8D433,
> >  0x07807C9A2, 0x00F00F934, 0x09609A88E, 0x0E10E9818, 0x07F6A0DBB,
> 0x0086D3D2D, 0x091646C97, 0x0E6635C01,
> >  0x06B6B51F4, 0x01C6C6162, 0x0856530D8, 0x0F262004E, 0x06C0695ED,
> 0x01B01A57B, 0x08208F4C1, 0x0F50FC457,
> >  0x065B0D9C6, 0x012B7E950, 0x08BBEB8EA, 0x0FCB9887C, 0x062DD1DDF,
> 0x015DA2D49, 0x08CD37CF3, 0x0FBD44C65,
> >  0x04DB26158, 0x03AB551CE, 0x0A3BC0074, 0x0D4BB30E2, 0x04ADFA541,
> 0x03DD895D7, 0x0A4D1C46D, 0x0D3D6F4FB,
> >  0x04369E96A, 0x0346ED9FC, 0x0AD678846, 0x0DA60B8D0, 0x044042D73,
> 0x033031DE5, 0x0AA0A4C5F, 0x0DD0D7CC9,
> >  0x05005713C, 0x0270241AA, 0x0BE0B1010, 0x0C90C2086, 0x05768B525,
> 0x0206F85B3, 0x0B966D409, 0x0CE61E49F,
> >  0x05EDEF90E, 0x029D9C998, 0x0B0D09822, 0x0C7D7A8B4, 0x059B33D17,
> 0x02EB40D81, 0x0B7BD5C3B, 0x0C0BA6CAD,
> >  0x0EDB88320, 0x09ABFB3B6, 0x003B6E20C, 0x074B1D29A, 0x0EAD54739,
> 0x09DD277AF, 0x004DB2615, 0x073DC1683,
> >  0x0E3630B12, 0x094643B84, 0x00D6D6A3E, 0x07A6A5AA8, 0x0E40ECF0B,
> 0x09309FF9D, 0x00A00AE27, 0x07D079EB1,
> >  0x0F00F9344, 0x08708A3D2, 0x01E01F268, 0x06906C2FE, 0x0F762575D,
> 0x0806567CB, 0x0196C3671, 0x06E6B06E7,
> >  0x0FED41B76, 0x089D32BE0, 0x010DA7A5A, 0x067DD4ACC, 0x0F9B9DF6F,
> 0x08EBEEFF9, 0x017B7BE43, 0x060B08ED5,
> >  0x0D6D6A3E8, 0x0A1D1937E, 0x038D8C2C4, 0x04FDFF252, 0x0D1BB67F1,
> 0x0A6BC5767, 0x03FB506DD, 0x048B2364B,
> >  0x0D80D2BDA, 0x0AF0A1B4C, 0x036034AF6, 0x041047A60, 0x0DF60EFC3,
> 0x0A867DF55, 0x0316E8EEF, 0x04669BE79,
> >  0x0CB61B38C, 0x0BC66831A, 0x0256FD2A0, 0x05268E236, 0x0CC0C7795,
> 0x0BB0B4703, 0x0220216B9, 0x05505262F,
> >  0x0C5BA3BBE, 0x0B2BD0B28, 0x02BB45A92, 0x05CB36A04, 0x0C2D7FFA7,
> 0x0B5D0CF31, 0x02CD99E8B, 0x05BDEAE1D,
> >  0x09B64C2B0, 0x0EC63F226, 0x0756AA39C, 0x0026D930A, 0x09C0906A9,
> 0x0EB0E363F, 0x072076785, 0x005005713,
> >  0x095BF4A82, 0x0E2B87A14, 0x07BB12BAE, 0x00CB61B38, 0x092D28E9B,
> 0x0E5D5BE0D, 0x07CDCEFB7, 0x00BDBDF21,
> >  0x086D3D2D4, 0x0F1D4E242, 0x068DDB3F8, 0x01FDA836E, 0x081BE16CD,
> 0x0F6B9265B, 0x06FB077E1, 0x018B74777,
> >  0x088085AE6, 0x0FF0F6A70, 0x066063BCA, 0x011010B5C, 0x08F659EFF,
> 0x0F862AE69, 0x0616BFFD3, 0x0166CCF45,
> >  0x0A00AE278, 0x0D70DD2EE, 0x04E048354, 0x03903B3C2, 0x0A7672661,
> 0x0D06016F7, 0x04969474D, 0x03E6E77DB,
> >  0x0AED16A4A, 0x0D9D65ADC, 0x040DF0B66, 0x037D83BF0, 0x0A9BCAE53,
> 0x0DEBB9EC5, 0x047B2CF7F, 0x030B5FFE9,
> >  0x0BDBDF21C, 0x0CABAC28A, 0x053B39330, 0x024B4A3A6, 0x0BAD03605,
> 0x0CDD70693, 0x054DE5729, 0x023D967BF,
> >  0x0B3667A2E, 0x0C4614AB8, 0x05D681B02, 0x02A6F2B94, 0x0B40BBE37,
> 0x0C30C8EA1, 0x05A05DF1B, 0x02D02EF8D };

> >  static void Main(string[] args)
> >   {
> >   if( args.Length < 1 )
> >    { Console.WriteLine( "SYNTAX: TestCRC file" ); return; }
> >   Thread.Sleep( 200 );
> >   int s = Environment.TickCount;
> >   uint crc = getcrc( args[0] );
> >   int e = Environment.TickCount;
> >   Console.WriteLine( "crc32 = {0:x} in {1} ms", crc, e - s );
> >   }

> >  public static uint getcrc( string fullpath )
> >   {
> >   FileStream file = new FileStream( fullpath, FileMode.Open,
> FileAccess.Read, FileShare.Read, 16384 );
> >   long len = file.Length;
> >   byte[] adat = new byte[ 16384 ];
> >   int rr = 0;
> >   uint crc = 0xFFFFFFFF;
> >   do
> >    {
> >    rr = file.Read( adat, 0, 16384 );
> >    for( int i = 0; i < rr; i++ )
> >     {
> >     byte b1 = (byte) (crc ^ adat[i]);
> >     crc = (crc>>8) ^ crcTable[ b1 ];
> >     }
> >    }
> >   while( rr > 0 );
> >   return crc ^ 0xFFFFFFFF;
> >   }
> >  }
> > }
> > //

============================================================================

- Show quoted text -

Quote:
> =======

> >   Wonna my pentium-assembler version? sooner finished then started!
;-)

> >   NETMaster
> >   http://www.cetus-links.org/oo_csharp.html



> > The IO perfomance of C# is terrible, compared to the C++ routine.
> > I have recently ported a small C++ program which uses an ANSI C file
> routine. The program calculates a CRC32 for any given filename.
> > 1. opens an existing file.
> > 2. read one byte - perform bit manipulation - read one byte - repeat
> > 3. display crc32 value
> > ===========================
> > Performance times using a 32Megabyte file:
> > 2         seconds for C++ routine
> > 105     seconds for C## routine
> > ===========================
> > Does anyone know how to increase this performance?
> > public void getcrc(string fullpath)
> > {
> >   byte buffer = 0xff;
> >   FileStream file = new FileStream(fullpath, FileMode.Open);
> >   uint crc;
> >   byte b1;
> >   crc = 0xFFFFFFFF;
> >   for(uint x = 0; x < file.Length; x++)
> >   {
> >     b1 = (byte)file.ReadByte();
> >     crc = ((crc>>8) & 0x00FFFFFF) ^ crcTable[ (crc^b1) & 0xFF ];
> >   }
> >     crc_32 = ( crc^0xFFFFFFFF );
> >   }



Thu, 10 Jun 2004 11:34:04 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Quote:
>> You have no idea how happy you have made me.

 Glad I could help!

Quote:
>>that C# is as capable as C++/C language, at least for I/O performance.

 Note, I/O performance is NOT directly bound to C#,
 but to the .NET IO classes like 'FileStream'.
 FileStream.Read() actually is a 'wrapper' of the Win32 ReadFile API call,
 with some additional stream memory buffering/handling.
 So you will get I/O performance like any Win32-App does,
 but with a bit more CPU load.

Quote:
>> What does this line of code do? Thread.Sleep...

 The code sequence
   Thread.Sleep( 200 );
   int s = Environment.TickCount;
 is to 'calm' the system (due starting the TestCRC.exe) and doing
 timing with as few interruptions as possible from other (system-) tasks.
 Ok, no warranty, but it may make timing a little bit more reproducable
 if doing sub-second timing.
 [Please just remove Sleep()+TickCount for your final code!]

NOTE to my code:
line:
   long len = file.Length;
'len' is unused, so remove line!

  Regards,
  NETMaster
  http://www.cetus-links.org/oo_csharp.html

Quote:

> Yippee! Your code runs like a lightning bolt, compared to my code! And...
> thank you for taking the time to respond to my question.
> This little routine (CRC32), is the keystone of a small utility which our
> company uses, it was beginning to look like I was never going to get it
> ported to C#.
> I had been working on improving the performance for 4 days, I really didn't
> expect anyone to rescue me!
> You have no idea how happy you have made me.
> 1. Your coding efforts, have proven to me (without doubt), that C# is as
> capable as C++/C language, at least for I/O performance.
> 2. Since I am just a beginner (an "advanced beginner" <grin>), I wanted to
> take this new programming paradigm shift, from C++ to C#.
> 3. C++ , is a very powerful language, but as a beginner, the C++ language
> makes me work too hard for small results.
> 4. I was very close to giving up on the C# language, returning to C++. I was
> going to give C# another year to mature, this would have been a mistake.
> What does this line of code do?     Thread.Sleep(200);     I have not seen
> this before.



Thu, 10 Jun 2004 16:29:31 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Hi,

Thanks, but credits must be where credits are due. That was NETMaster who pointed you in the right direction.

Willy.

Quote:

> Willy, thanks for the reply, nice to see quality assistance when needed.



Thu, 10 Jun 2004 18:44:08 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Hi NetMaster, pmfji but how did you figure out that

   for(uint x = 0; x < file.Length; x++)

was the worst line.  Did you use some type of profiler.  Would be interested
to know


Quote:
> >> You have no idea how happy you have made me.
>  Glad I could help!

> >>that C# is as capable as C++/C language, at least for I/O performance.
>  Note, I/O performance is NOT directly bound to C#,
>  but to the .NET IO classes like 'FileStream'.
>  FileStream.Read() actually is a 'wrapper' of the Win32 ReadFile API call,
>  with some additional stream memory buffering/handling.
>  So you will get I/O performance like any Win32-App does,
>  but with a bit more CPU load.

> >> What does this line of code do? Thread.Sleep...
>  The code sequence
>    Thread.Sleep( 200 );
>    int s = Environment.TickCount;
>  is to 'calm' the system (due starting the TestCRC.exe) and doing
>  timing with as few interruptions as possible from other (system-) tasks.
>  Ok, no warranty, but it may make timing a little bit more reproducable
>  if doing sub-second timing.
>  [Please just remove Sleep()+TickCount for your final code!]

> NOTE to my code:
> line:
>    long len = file.Length;
> 'len' is unused, so remove line!

>   Regards,
>   NETMaster
>   http://www.cetus-links.org/oo_csharp.html




- Show quoted text -

Quote:
> > Yippee! Your code runs like a lightning bolt, compared to my code!
And...
> > thank you for taking the time to respond to my question.
> > This little routine (CRC32), is the keystone of a small utility which
our
> > company uses, it was beginning to look like I was never going to get it
> > ported to C#.
> > I had been working on improving the performance for 4 days, I really
didn't
> > expect anyone to rescue me!
> > You have no idea how happy you have made me.
> > 1. Your coding efforts, have proven to me (without doubt), that C# is as
> > capable as C++/C language, at least for I/O performance.
> > 2. Since I am just a beginner (an "advanced beginner" <grin>), I wanted
to
> > take this new programming paradigm shift, from C++ to C#.
> > 3. C++ , is a very powerful language, but as a beginner, the C++
language
> > makes me work too hard for small results.
> > 4. I was very close to giving up on the C# language, returning to C++. I
was
> > going to give C# another year to mature, this would have been a mistake.
> > What does this line of code do?     Thread.Sleep(200);     I have not
seen
> > this before.



Fri, 11 Jun 2004 13:26:29 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Actually, just from my first review, the usage of any
FileStream.* - Methods inside the loop looked suspect to me...
but in first place, I guessed FileStream.ReadByte() a larger waste.
(I learned later, FileStream does fairly efficient stream-buffering)

Note, I did some CRC32 coding before, even in pentium-assembler,
and I had the crcTable[] (was NOT included in original post!) used in other source.
So I knew the formula (EXOR and table-lookup) was correct,
and the (AND & 0x00ffffff) operation for a shifted unsigned>>8 value isn't needed.

I didn't use any profiler.
I just created a VS project, copy&paste of original code,
added my crcTable[] and timing logic, selected a huge file and did
some measures to reproduce the bad timing numbers.

Then I directly modified the code by removing
first .Length, then .ReadByte(), and got the superb execution timings.
To be sure, I inspected .NET .ReadByte() and .Length with ILDASM.
Finally, I tried some variants of formula (e.g. the byte sized indexer).

I think with some experience, a few lines of code could be optimized without profiler.
But for multiple complex functions or a complete app, a profiler is VERY useful.

  Regards,
  NETMaster
  http://www.cetus-links.org/oo_csharp.html

Quote:

> Hi NetMaster, pmfji but how did you figure out that

>    for(uint x = 0; x < file.Length; x++)

> was the worst line.  Did you use some type of profiler.  Would be interested to know


> > >>that C# is as capable as C++/C language, at least for I/O performance.
> >  Note, I/O performance is NOT directly bound to C#,
> >  but to the .NET IO classes like 'FileStream'.
> >  FileStream.Read() actually is a 'wrapper' of the Win32 ReadFile API call,
> >  with some additional stream memory buffering/handling.
> >  So you will get I/O performance like any Win32-App does,
> >  but with a bit more CPU load.



> > > Yippee! Your code runs like a lightning bolt, compared to my code!
> And...
> > > thank you for taking the time to respond to my question.
> > > This little routine (CRC32), is the keystone of a small utility which our
> > > company uses, it was beginning to look like I was never going to get it
> > > ported to C#.



Fri, 11 Jun 2004 16:41:46 GMT  
 IO performance C# Versus C++ (For C# GURU's only)

Quote:
>Hi NetMaster, pmfji but how did you figure out that

>   for(uint x = 0; x < file.Length; x++)

>was the worst line.

From the experience quoted, it seems like it took quite a bit of
effort to deduce.

Personally I think making Length appear as a property rather than a
method is a poor decision since I inherently (maybe incorrectly)
assume properties should be fast operations. If it were a method name
GetFileLength, then there is (in my mind) the perception that it's not
going to be lightning fast.

What do others think about this speed assumption regarding properties
and method designation?

Dave Lowndes
--
MVP VC++ FAQ: http://www.mvps.org/vcfaq
My address is altered to discourage junk mail.
Please post responses to the newsgroup thread,
there's no need for follow-up email copies.



Fri, 11 Jun 2004 17:09:43 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Hi David,

The purpose of Properties is to include logic.
This implies that there can be much code behind it
or just 1 line of code. But Properties are somehow nothing but specialized
methods.
So forget that properties are always fast - it depends what is done in the
Property-Method

Cheers,
Chris
And as you never know how much logic is in a property, but tha

Quote:
> >Hi NetMaster, pmfji but how did you figure out that

> >   for(uint x = 0; x < file.Length; x++)

> >was the worst line.

> From the experience quoted, it seems like it took quite a bit of
> effort to deduce.

> Personally I think making Length appear as a property rather than a
> method is a poor decision since I inherently (maybe incorrectly)
> assume properties should be fast operations. If it were a method name
> GetFileLength, then there is (in my mind) the perception that it's not
> going to be lightning fast.

> What do others think about this speed assumption regarding properties
> and method designation?

> Dave Lowndes
> --
> MVP VC++ FAQ: http://www.mvps.org/vcfaq
> My address is altered to discourage junk mail.
> Please post responses to the newsgroup thread,
> there's no need for follow-up email copies.



Fri, 11 Jun 2004 17:49:36 GMT  
 IO performance C# Versus C++ (For C# GURU's only)
Quote:
>> assume properties should be fast...

in C++, we used inline functions just to return a (private) member variable.
May be this is what you remember?

From inspecting (ILDASM) many .NET class properties,
I would state only few are 'light'.

SUMMARY:
 Properties only are a 'light by access SYNTAX' thing.

NOTE specific for FileStream.Length:
 I guess there is another problem:
 while reading from stream, may be the file could still grow
 (another task appending data to file: FileShare.ReadWrite)!

 If the implementation just would do a 'GetFileSize' at 'Open' event of FileStream
 and 'cache' the value to a member variable, this would be wrong.
 I think for any kind of 'Stream', reading up to 0/empty-return is MUCH cleaner...
 Some Streams even don't support (efficient) .Length or seek, => NotSupportedException

 I recomment use of explicit like  FileInfo.Length .

  NETMaster
  http://www.cetus-links.org/oo_csharp.html

Quote:

> >Hi NetMaster, pmfji but how did you figure out that
> >   for(uint x = 0; x < file.Length; x++)
> >was the worst line.
> From the experience quoted, it seems like it took quite a bit of
> effort to deduce.
> Personally I think making Length appear as a property rather than a
> method is a poor decision since I inherently (maybe incorrectly)
> assume properties should be fast operations. If it were a method name
> GetFileLength, then there is (in my mind) the perception that it's not
> going to be lightning fast.
> What do others think about this speed assumption regarding properties
> and method designation?



Fri, 11 Jun 2004 18:16:28 GMT  
 
 [ 19 post ]  Go to page: [1] [2]

 Relevant Pages 

1. c++ style IO completion ports in C#?

2. This is a biztalk question but probably an easy question for you C# guru's

3. Newbie: separate big .cs file into small .cs files

4. C++ vs C# or ( C++ + C#)?

5. Performance comparison between C# and C++

6. C# vs C++ .NET performance question

7. C# vs. C++ in performance

8. Data Access Performance of XML, versus SQL versus Text File

9. include another form(cs) in C#?

10. c versus c++ performance(an objective comparison) ?

11. C versus C++ (performance)

12. IO in C#

 

 
Powered by phpBB® Forum Software