Can I use awk within awk in UNIX?

Question

I have two files. One contains students and their average grade from the first semester and the second contains students and 5 grades from the second semester. I have to update the first file, so that it contains the students sorted after the yearly average grade. I'm new to unix and so far I've tried to use the awk command within an awk but I keep getting a syntax error. I have tried other ways but all I can manage to achieve is separating the file contents into lines, without direct access to the second word(the average grade) from each line.

input files:

firstsem:

Name1 6
Name2 8
Name3 10

secondsem:

Name1 5 6 5 6 5
Name2 10 10 10 10 10
Name3 2 2 2 2 2

result:

firstsem:

Name1  5.7
Name3 6
Name2 9

Code:

awk '{
        name = $1
        total=$2+$3+$4+$5+$6;
        avg=total/5;
        print name " " avg
        #sprintf("",nume,avg)
}' < secondsem > secondsemesteravg
# i didn't really want the secondsemesteravg file, but i didn't find another way
while read line; do
       # wanted to get the avg from firstsem
        echo $line | awk '{$2=avgfirstsem + avgsecondsem/2; print $0;}';
done < firstsem

Welcome to S.O., you will get a better response/answer if you posted the sample input and desired output, Also show what have you tried so far. — Jetchisel, Apr 07 '20 at 20:47
Like @Jetchisel said, please provide a [mre] for debugging questions. BTW check out the [tour], and [ask] if you want more advice. — wjandrea, Apr 07 '20 at 20:49
BTW the short answer is yes, [you could use `system()`](https://stackoverflow.com/q/14634349/4518341) to run an AWK command within an AWK script, or since you tagged [tag:bash], maybe you're trying to embed the output of one AWK command into another with [command substitution, `"$()"`](https://stackoverflow.com/q/4651437/4518341), which would also work but might be more fragile. — wjandrea, Apr 07 '20 at 20:53
This should be doable with only one `awk` process altogether. Remember you can pass just one copy of awk more than one file as an argument, and persist variables in memory between them. — Charles Duffy, Apr 07 '20 at 21:05
For that matter, I'm not sure there's even a compelling reason for the awk script to care about file boundaries. Maintain two associative arrays, one mapping name -> total_grade (across all semesters), and a second one mapping name to number of semesters observed. Every time you see a new line of input, increment the count for that name by one, and add the total of all the numbers in subsequent columns to the total grade for that name. When you hit the end of all inputs, iterate over the names, and divide the totals by the counts. Done -- with only one script. — Charles Duffy, Apr 07 '20 at 21:09
BTW, let me add a note of caution -- using `system()` in awk, or its equivalent pipe support, has serious security risks. When you're working with native shell variables in a shell script, expanding them doesn't risk unintentionally running them as code -- but it *does* have that effect when you substitute their values into a string that's evaluated as a new shell command. This makes any command that uses input, user parameters, etc. in generating a string to pass to a shell as code very, very dangerous. — Charles Duffy, Apr 07 '20 at 21:11

score 1 · Answer 1 · answered Apr 07 '20 at 22:37

1

I think this should work for more than 2 files as well but didn't test

 $ awk '{k=$1; sum=0; 
         for(i=2;i<=NF;i++) sum+=$i; 
         a[k]=(a[k]*c[k]+sum/(NF-1))/++c[k]} 
    END {for(k in a) print k,a[k]}' file{1,2}

Name1 5.7
Name2 9
Name3 6

answered Apr 07 '20 at 22:37

karakfa

62,998
7
34
47

score 1 · Accepted Answer · answered Apr 07 '20 at 22:40

1

I have written such script with comments that could help you:

awk '{
    # calculate avarage of all numbers greater then column 2
    total = 0;
    for (i = 2; i <= NF; ++i) {
        total += $i;
    }
    cur = total / (NF - 1);
}
FNR==NR {
    # first file
    avg[$1] = cur;
}
FNR != NR {
    # second file
    avg[$1] = ( avg[$1] + cur ) / 2
}
END {
    for (i in avg) {
        print i, avg[i]
    }
}' firstsem secondsem

answered Apr 07 '20 at 22:40

KamilCuk

69,546
5
27
60

Thank you so much. It is exactly what I've been looking for – Corina Holom Apr 08 '20 at 12:12

Can I use awk within awk in UNIX?

2 Answers2