How to write a C/C++ program that can accept input from a pipe 
Author Message
 How to write a C/C++ program that can accept input from a pipe

How to write a C/C++ program that can accept input from a pipe. I do
not want two programs one for pipe and
one for actual data file, but more uniformly. Pseudo code would be

(1) open stdin or infile depending on argc.
(2) figure out the size of the infile or stdin, using the fseek/ftell
or otherwise.
(3) allocate that amount of memory and dump the infile/stdio contents
into the dynamically allocated array.
(4) close the file pointer if it was infile.
(5) do whatever processing, searching, running around you needed in
this array.
(6) write the array to stdout

There are ideals to be striven for. Quick, dirty and limited solutions
are out there by the tons.

(1) I know that you could allocate and reallocate memory. Could any of
you see how this would be
a dirty process? It fuses the steps (2) and (3) above and renders them
inseparable. OK it can be made clean by making a module. But what
about computational overhead? Anyway, I want someone with facility
with malloc/realloc/free to fill the following function as neat as
possible.

// steps (2) and (3) - a short term solution when the collective
intelligence of the newsgroups fails to give a better one
void readFileToArrayAndAllocate(FILE* fp, unsignedCharArray*
arrayFound, int* sizeFound){

Quote:
}

Just wanted to point out that realloc reallocates a bigger array, then
copies the existing data there and then frees the
smaller one. The file could be meg, 10meg, 100meg?

(2) The other solution is to seak into the operating system and find
the size of the file using
lsString = system("ls full_path_name");
but the question is how do I get the full_path_name and can avoid the
parsing of the lsString
and write ifdefs for compilation in DOS/Linux/Unix?

(3) A relevant post is included and can elaborate it some what?
Note my OS has /proc/self/fd/0 but this command does not work while
the program works
in this way:

program inFile > outFile

(4) I really would like to avoid a linked list as it makes the
readibility of the program poor in C. The
problem is that C is really bad for linked list readibility due to
absence of classes. I want to stay
away from C++ as much but not the super C (the repaired C, where it is
useful to declare variables
exactly where they are needed rather than all mixed up on the top).

-------------------------------------------------------------------------------------------------------------------------------------
Subject: Re: How to read pipe instead of file in C
Date: 08/14/1999

On Fri, 6 Aug 1999 14:22:56 -0500,

Quote:

>I need to change a program which usually reads the file specified as
an
>argument so that it can read piped input.  How can I read piped input
(and
>not require a file name argument) or determine there is no pipe and
read the
>named file as usual?

There is no difference in reading from a file or reading from standard
input, like was already point
ed out.

But if your program requires a file name argument and cannot read from
stdin, you do not have to rew
rite the program. Instead, call the program like this:

cat stuff | program /proc/self/fd/0

Or if you are running on an inferior system where the proc-filesystem
is not available:

mknod /tmp/myfifo p
program /tmp/myfifo & cat stuff > /tmp/myfifo
rm /tmp/myfifo

Some shells like the zsh can do the same thing like this:

program =(cat stuff)

--

------------------------------------------------------------------------------------------------------------------------------
Hey C pros, who have objection to interdisciplinary C/OS question to
be in this group, give a solution if you know so much!



Thu, 30 Sep 2004 10:31:31 GMT  
 How to write a C/C++ program that can accept input from a pipe

Quote:
> How to write a C/C++ program that can accept input from a pipe. I do
> not want two programs one for pipe and
> one for actual data file, but more uniformly. Pseudo code would be

1.  There is no such language as 'C/C++'.
    C and C++ are two separate, distinct languages.

2. 'Piping' is a function of an operating system
   (if it supports it), not the language.

3. OS issues are off-topic in comp.langc and comp.lang.c++

Quote:

> (1) open stdin or infile depending on argc.
> (2) figure out the size of the infile or stdin, using the fseek/ftell
> or otherwise.

Find the 'size' of 'stdin'?  That would be a neat trick. :-)

Quote:
> (3) allocate that amount of memory and dump the infile/stdio contents
> into the dynamically allocated array.

Why{*filter*}around with arrays?  Why not just 'read a char,
write a char?'  Most OS's buffer i/o effectively enough
that you shouldn't notice a performance difference versus
reading and writing larger 'blocks'.

#include <stdio.h>
int main()

{
    int c = 0;
    while((c = getchar()) != EOF)
        if(putchar(c) == EOF)
            break;

    return 0;

Quote:
}
> (4) close the file pointer if it was infile.
> (5) do whatever processing, searching, running around you needed in
> this array.
> (6) write the array to stdout

<OT>
If you take input from stdin, and send output to stdout,
an OS that supports 'piping' should handle this just fine.
You will not be able to redirect i/o tied to a file, however.
</OT>

Quote:

> There are ideals to be striven for. Quick, dirty and limited solutions
> are out there by the tons.

> (1) I know that you could allocate and reallocate memory. Could any of
> you see how this would be
> a dirty process?

Yes.  And unnecessary in context of your query.

Quote:
>It fuses the steps (2) and (3) above and renders them
> inseparable. OK it can be made clean by making a module. But what
> about computational overhead?

What 'computation?'  Seems like you're just asking about i/o.

Quote:
>Anyway, I want someone with facility
> with malloc/realloc/free to fill the following function as neat as
> possible.

No need.

Quote:

> // steps (2) and (3) - a short term solution when the collective
> intelligence of the newsgroups fails to give a better one
> void readFileToArrayAndAllocate(FILE* fp, unsignedCharArray*
> arrayFound, int* sizeFound){
> }

> Just wanted to point out that realloc reallocates a bigger array,

It reallocates to the size specified by its argument.
This size could be smaller than the original, or the same.
It need not be larger.

Quote:
> then
> copies the existing data there and then frees the
> smaller one. The file could be meg, 10meg, 100meg?

File?  You were talking about memory.

Quote:

> (2) The other solution is to seak into the operating system and find
> the size of the file using
> lsString = system("ls full_path_name");
> but the question is how do I get the full_path_name and can avoid the
> parsing of the lsString
> and write ifdefs for compilation in DOS/Linux/Unix?

How to get a 'path name' from an OS (if it has them) will
be specific to that OS, and not part of standard C or C++,
thus offtopic in clc and clc++.  Ask in a group that discusses your OS.

Quote:

> (3) A relevant post is included and can elaborate it some what?
> Note my OS has /proc/self/fd/0 but this command does not work while
> the program works
> in this way:

> program inFile > outFile

<OT>
program < inFile > outFile
<OT>

Quote:

> (4) I really would like to avoid a linked list as it makes the
> readibility of the program poor in C.

No need for a linked list.

Quote:
>The
> problem is that C is really bad for linked list readibility due to
> absence of classes.

I've never had trouble reading well written linked-list
code in C.

Quote:
>I want to stay
> away from C++ as much but not the super C (the repaired C, where it is
> useful to declare variables
> exactly where they are needed rather than all mixed up on the top).

[snip]

Quote:
> Hey C pros, who have objection to interdisciplinary C/OS question to
> be in this group,

We object to off-topic material.  The only topic
of comp.lang.c is ISO standard C.  THe only topic
of comp.lang.c++ is ISO standard C++.  Operating system
issues are off topic in both groups.

Quote:
>give a solution if you know so much!

People who answer questions in a topical newsgroup are
implying  that they have knowledge of the group's topic(s).
They may or may not know about various other topics, but
answering questions about them where they're not topical
is a violation of Usenet etiquette.

-Mike



Thu, 30 Sep 2004 11:26:29 GMT  
 How to write a C/C++ program that can accept input from a pipe
I am reading this in comp.lang.c. I realise my code is wrong for C++, but
gnuist seems to want solutions in both languages.

Quote:

> How to write a C/C++ program that can accept input from a pipe. I do
> not want two programs one for pipe and
> one for actual data file, but more uniformly.

Sure, but be aware that there is no such thing as a "pipe" in this
newsgroup (comp.lang.c). Just say you want one program to be able to read
either from stdin or from a file.

Quote:
> Pseudo code would be

> (1) open stdin or infile depending on argc.

Sure.

Quote:
> (2) figure out the size of the infile or stdin, using the fseek/ftell
> or otherwise.

Wrong approach. Firstly, stdin has no defined size, secondly the
fseek/ftell solution is not portable and may be wrong anyway if the file
size changes while you're reading it.

Quote:
> (3) allocate that amount of memory and dump the infile/stdio contents
> into the dynamically allocated array.

Why not realloc your array as you read, that way will work for both
file-based and stdin input.

Quote:
> (4) close the file pointer if it was infile.

Ok.

Quote:
> (5) do whatever processing, searching, running around you needed in
> this array.

Ok. But if your algorithm can work one line at a time then I wouldn't waste
the memory reading the whole file in first.

Quote:
> (6) write the array to stdout

Ok.

Quote:
> There are ideals to be striven for. Quick, dirty and limited solutions
> are out there by the tons.

> (1) I know that you could allocate and reallocate memory. Could any of
> you see how this would be
> a dirty process? It fuses the steps (2) and (3) above and renders them
> inseparable. OK it can be made clean by making a module. But what
> about computational overhead? Anyway, I want someone with facility
> with malloc/realloc/free to fill the following function as neat as
> possible.

Unfortunately it's about the only solution that isn't limited in file size
(except by available memory) and is resilient to file size changes or
reading from stdin.

Quote:
> // steps (2) and (3) - a short term solution when the collective
> intelligence of the newsgroups fails to give a better one
> void readFileToArrayAndAllocate(FILE* fp, unsignedCharArray*
> arrayFound, int* sizeFound){
> }

What is an "unsignedCharArray*arrayFound"? Don't you want to pass in a
pointer to a pointer so you can fill it out with malloc/realloc?

/* Returns:
   0 for success, whole file read into contents;
   1 for fatal memory error, nothing read, contents is NULL;
   2 for file too big, read some of it, contents is valid;
   3 for error shrinking allocated space, whole file was read
         successfully but excess memory has been used.
*/
int readFile(FILE *fp, unsigned char **contents, size_t *size)
{
  unsigned char *tmp;
  size_t allocated = 1000;
  *size = 0;
  *contents = malloc(allocated);
  if(NULL == *contents) return 1;
  while(((*contents)[*size] = getc(fp)) != EOF)
  {
    ++*size;
    if(size == allocated)
    {
      allocated += allocated / 2;  /* Grow by 50% */
      tmp = realloc(*contents, allocated);
      if(NULL == tmp) return 2;
      *contents = tmp;
    }
  }
  tmp = realloc(*contents, *size); /* Shrink to real size */
  if(NULL == tmp) return 3;
  *contents = tmp;
  return 0;

Quote:
}

Then use it like:
{
  unsigned char *array;
  size_t size;
  int ret;
  FILE *fp = stdin;

  if(argc == 2) fp = fopen(argv[1], "rb");
  if(NULL == fp) return 1;
  ret = readFile(fp, &array, &size);
  ...

Quote:
}
> Just wanted to point out that realloc reallocates a bigger array, then
> copies the existing data there and then frees the
> smaller one. The file could be meg, 10meg, 100meg?

Yep, what's wrong with that? Disk access is so slow that this probably
isn't a problem. If you want to reduce the number of copies made then
increase the factor by which allocated is increased, to 2x or 3x even. So
long as you don't run out of memory this shouldn't be a problem, since
there is no wasted memory as the space is shrunk back to the real size at
the end of the function.

Quote:
> (2) The other solution is to seak into the operating system and find
> the size of the file using
> lsString = system("ls full_path_name");
> but the question is how do I get the full_path_name and can avoid the
> parsing of the lsString
> and write ifdefs for compilation in DOS/Linux/Unix?

Don't bother! Not only is it difficult, slow and non-portable, it could
well be wrong by the time you get to the end of the file.

[snip other unnecessary solutions]

--
Simon.



Thu, 30 Sep 2004 13:03:27 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. Some Expert level complications, Re: To accept input from PIPE by a program

2. To accept input from PIPE by a program

3. Newbie: separate big .cs file into small .cs files

4. Problem #30 on http://cs.nmu.edu/programming/c/problems.htm

5. SoftEng or CS Graduate Program Search

6. FS: Programming and CS Books --- CHEAP

7. 1995 UT IEEE CS National Programming Contest

8. Good CS Programs

9. IEEE CS National Programming Contest [Addendum3]

10. 1993 IEEE CS National Programming Contest Notice

11. Description of The 1991 IEEE CS National Programming Contest

12. 1993 IEEE CS National Programming Contest Addendum 2

 

 
Powered by phpBB® Forum Software