Efficient File Manipulation and Selection with Awk Command: Step-by-Step Guide

[confi999]

Hello,
I have about 8000 files in a directory all having names like
file000001.txt
file000002.txt
file000003.txt
file000004.txt
----------
----------
file007999.txt
file008000.txt

Now I need to do 2 things:

1. I found out that I don't actually need the data from the files whose names are above 'file003000.txt'. So I want to delete the 5000 files from 'file003001.txt' through 'file008000.txt' using awk. (I don't know how to do this part) I will then take the values of column 3 of each of the remaining files if column 1 == 20 and put these in 'select.txt' file (I know how to do this part).

2. I believe it is possible (but I am not being able to do it) even without deleting those 5000 files to write an awk command that will only take files which has values less than / equal to 'file003000.txt' and do the latter part on them i.e. taking the values of column 3 of each file if column 1 == 20 and put these in 'select.txt' file. In other words it is something like if VARIABLE<=3000 in 'file00VARIABLE.txt' consider it otherwise ignore it.

How can I do both of the above using awk (one liner preferred than scripts).
Any help is highly appreciated. Thank you.

 

[D H]

1. I found out that I don't actually need the data from the files whose names are above 'file003000.txt'. So I want to delete the 5000 files from 'file003001.txt' through 'file008000.txt' using awk. (I don't know how to do this part)

Why use awk? Just use the command line. In csh/tcsh,
rm file00[3-8]?.txt

2. I believe it is possible (but I am not being able to do it) even without deleting those 5000 files to write an awk command that will only take files which has values less than / equal to 'file003000.txt'

Don't use awk to do the filtering that is easier done at the shell level.
awk -f script.awk file00[0-2]?.txt