Awk Problem: Changing Input Data Format 
Author Message
 Awk Problem: Changing Input Data Format

I've been thinking {*filter*} this one but this is giving me some
headaches.

I need a script to process some files and its output looks like below.
  The file headers has to be created on the fly because it varies for
every file.   I can't just place in under BEGIN...   When processing
this file, the header for all times(00:00-23:00) is same.   But the
header in *other files might have more or less "LEP" or "PPC", thus
headers changes.

The file header is taken from field 5,7 & 8 for every time.  The
values to be taken based on the headers is field 9.

Thanks,
yen

-output.txt
TIME,CODE__,LEP_0_18,LEP_0_23,LEP_0_25,LEP_1_18,LEP_1_23,LEP_1_25,LEP_2_18,LEP_2_20,LEP_2_23,LEP_2_24,LEP_2_25,LEP_3_18,PMO_0_20,PMO_1_20,PPC_0_21,PPC_1_21,PPC_2_21,PPC_2_22,AMC__
00:00,20,13.06,21.73,3.04,9.45,17.11,16.58,14.74,6.99,23.38,9.44,14.87,13.18,2.84,2.25,17.44,19.18,8.89,17.90,10.00
00:10,21.00,12.06,20.44,5.14,8.34,27.02,14.47,14.22,8.33,23.38,18.14,14.83,11.13,3.32,2.35,16.17,19.28,9.38,11.00
.
.

-input1.txt
09/16/2001|00:00|9||AMC|9|||10.00%|600.00|
09/16/2001|00:00|9|1|CODE|1|||20.00%|600.00|
09/16/2001|00:00|9|1|LEP|17|0|18|13.06%|599.90|
09/16/2001|00:00|9|1|PMO|1|0|20|2.84%|600.19|
09/16/2001|00:00|9|1|PPC|6|0|21|17.44%|600.21|
09/16/2001|00:00|9|1|LEP|7|0|23|21.73%|599.94|
09/16/2001|00:00|9|1|LEP|1|0|25|3.04%|1030.05|
09/16/2001|00:00|9|1|LEP|18|1|18|9.45%|599.89|
09/16/2001|00:00|9|1|PMO|2|1|20|2.25%|600.20|
09/16/2001|00:00|9|1|PPC|1|1|21|19.18%|600.24|
09/16/2001|00:00|9|1|LEP|8|1|23|17.11%|599.81|
09/16/2001|00:00|9|1|LEP|2|1|25|16.58%|599.85|
09/16/2001|00:00|9|1|LEP|19|2|18|14.74%|599.82|
09/16/2001|00:00|9|1|LEP|13|2|20|6.99%|1117.17|
09/16/2001|00:00|9|1|PPC|2|2|21|8.89%|600.20|
09/16/2001|00:00|9|1|PPC|7|2|22|17.90%|600.23|
09/16/2001|00:00|9|1|LEP|9|2|23|23.38%|599.92|
09/16/2001|00:00|9|1|LEP|23|2|24|9.44%|599.84|
09/16/2001|00:00|9|1|LEP|3|2|25|14.87%|599.80|
09/16/2001|00:00|9|1|LEP|20|3|18|13.18%|599.96|
09/16/2001|00:10|9||AMC|9|||11.00%|600.00|
09/16/2001|00:10|9|1|CODE|1|||21.00%|600.00|
09/16/2001|00:10|9|1|LEP|17|0|18|12.06%|599.90|
09/16/2001|00:10|9|1|PMO|1|0|20|3.32%|600.19|
09/16/2001|00:10|9|1|PPC|6|0|21|16.17%|600.21|
09/16/2001|00:10|9|1|LEP|7|0|23|20.44%|599.94|
09/16/2001|00:10|9|1|LEP|1|0|25|5.14%|1030.05|
09/16/2001|00:10|9|1|LEP|18|1|18|8.34%|599.89|
09/16/2001|00:10|9|1|PMO|2|1|20|2.35%|600.20|
09/16/2001|00:10|9|1|PPC|1|1|21|19.28%|600.24|
09/16/2001|00:10|9|1|LEP|8|1|23|27.02%|599.81|
09/16/2001|00:10|9|1|LEP|2|1|25|14.47%|599.85|
09/16/2001|00:10|9|1|LEP|19|2|18|14.22%|599.82|
09/16/2001|00:10|9|1|LEP|13|2|20|8.33%|1117.17|
09/16/2001|00:10|9|1|PPC|2|2|21|9.38%|600.20|
09/16/2001|00:10|9|1|PPC|7|2|22|18.12%|600.23|
09/16/2001|00:10|9|1|LEP|9|2|23|23.38%|599.92|
09/16/2001|00:10|9|1|LEP|23|2|24|18.14%|599.84|
09/16/2001|00:10|9|1|LEP|3|2|25|14.83%|599.80|
09/16/2001|00:10|9|1|LEP|20|3|18|11.13%|599.96|
.
.
.



Fri, 05 Mar 2004 17:44:51 GMT  
 Awk Problem: Changing Input Data Format
In awk (perl or your favorite scripting language) this is fairly easy:

ot!=$2  { if(hdr="") next
          if(! cont) {
            hdr = "TIME" hdr
            print hdr
            cont = 1
          }
          printf("%s",ot)
          for(i=1;i<=nd;i++) printf(",%s",data[i])
          ot = $2
          nd = 0
        }
        { nd++
          if(! cont) hdr = "," hdr $5 "_" $7 "_" $8
          data[nd] = $9
        }

The above code is untested, but should approximate a solution to you
problem. The date is ignored and the file is  presumed sorted in time
order, with the same number and order of fields (records) for each time.

Quote:

> I've been thinking {*filter*} this one but this is giving me some
> headaches.

> I need a script to process some files and its output looks like below.
>   The file headers has to be created on the fly because it varies for
> every file.   I can't just place in under BEGIN...   When processing
> this file, the header for all times(00:00-23:00) is same.   But the
> header in *other files might have more or less "LEP" or "PPC", thus
> headers changes.

> The file header is taken from field 5,7 & 8 for every time.  The
> values to be taken based on the headers is field 9.

> Thanks,
> yen

> -output.txt
> TIME,CODE__,LEP_0_18,LEP_0_23,LEP_0_25,LEP_1_18,LEP_1_23,LEP_1_25,LEP_2_18,LEP_2_20,LEP_2_23,LEP_2_24,LEP_2_25,LEP_3_18,PMO_0_20,PMO_1_20,PPC_0_21,PPC_1_21,PPC_2_21,PPC_2_22,AMC__
> 00:00,20,13.06,21.73,3.04,9.45,17.11,16.58,14.74,6.99,23.38,9.44,14.87,13.18,2.84,2.25,17.44,19.18,8.89,17.90,10.00
> 00:10,21.00,12.06,20.44,5.14,8.34,27.02,14.47,14.22,8.33,23.38,18.14,14.83,11.13,3.32,2.35,16.17,19.28,9.38,11.00
> .
> .

> -input1.txt
> 09/16/2001|00:00|9||AMC|9|||10.00%|600.00|
> 09/16/2001|00:00|9|1|CODE|1|||20.00%|600.00|
> 09/16/2001|00:00|9|1|LEP|17|0|18|13.06%|599.90|
> 09/16/2001|00:00|9|1|PMO|1|0|20|2.84%|600.19|
> 09/16/2001|00:00|9|1|PPC|6|0|21|17.44%|600.21|
> 09/16/2001|00:00|9|1|LEP|7|0|23|21.73%|599.94|
> 09/16/2001|00:00|9|1|LEP|1|0|25|3.04%|1030.05|



Fri, 05 Mar 2004 21:58:26 GMT  
 
 [ 2 post ] 

 Relevant Pages 

1. Fortran Data Input Problem-sample data and READ stmt

2. input from data file into awk under WindowsNT

3. Need AWK to change format.

4. Changing input field picture format at runtime?

5. how can I use my parallel port for data input under LabVIEW (output is no problem)

6. Fortran data input problem

7. Similar Fortran data input problem

8. Problem with Data-Structures/Formats

9. data formatting problem

10. Problems with formatting incoming data from a pipe into a text widget

11. Problem reading data: unrelated variables change...

12. Data type change problem .

 

 
Powered by phpBB® Forum Software