Mercurial > repos > bgruening > text_processing
comparison awk.xml @ 0:5314e5d6f040 draft
Imported from capsule None
author | bgruening |
---|---|
date | Thu, 29 Jan 2015 07:53:17 -0500 |
parents | |
children | 20344ce0c811 |
comparison
equal
deleted
inserted
replaced
-1:000000000000 | 0:5314e5d6f040 |
---|---|
1 <tool id="tp_awk_tool" name="Text reformatting" version="@BASE_VERSION@.0"> | |
2 <description>with awk</description> | |
3 <macros> | |
4 <import>macros.xml</import> | |
5 </macros> | |
6 <expand macro="requirements"> | |
7 <requirement type="package" version="4.1.0">gnu_awk</requirement> | |
8 </expand> | |
9 <version_command>awk --version | head -n 1</version_command> | |
10 <command> | |
11 <![CDATA[ | |
12 awk | |
13 --sandbox | |
14 -v FS=' ' | |
15 -v OFS=' ' | |
16 --re-interval | |
17 -f "$awk_script" | |
18 "$infile" | |
19 > "$outfile" | |
20 ]]> | |
21 </command> | |
22 <inputs> | |
23 <param name="infile" format="txt" type="data" label="File to process" /> | |
24 <param name="code" type="text" area="true" size="5x35" label="AWK Program" help=""> | |
25 <sanitizer> | |
26 <valid initial="string.printable"> | |
27 <remove value="'"/> | |
28 </valid> | |
29 </sanitizer> | |
30 </param> | |
31 </inputs> | |
32 <configfiles> | |
33 <configfile name="awk_script">$code</configfile> | |
34 </configfiles> | |
35 <outputs> | |
36 <data name="outfile" format_source="infile" metadata_source="infile"/> | |
37 </outputs> | |
38 <tests> | |
39 <test> | |
40 <param name="infile" value="awk1.txt" /> | |
41 <!-- commas are not allowed in a value field. Values with comma will be splitted --> | |
42 <param name="code" value='$2>0.5 { print $2*9"\t"$1 }' /> | |
43 <output name="outfile" file="awk_results1.txt" /> | |
44 </test> | |
45 </tests> | |
46 <help> | |
47 <![CDATA[ | |
48 **What it does** | |
49 | |
50 This tool runs the unix **awk** command on the selected data file. | |
51 | |
52 .. class:: infomark | |
53 | |
54 **TIP:** | |
55 | |
56 This tool uses the **extended regular** expression syntax (not the perl syntax). | |
57 **\\d**, **\\w**, **\\s** etc. are **not** supported. | |
58 | |
59 | |
60 **Further reading** | |
61 | |
62 - Awk by Example (http://www.ibm.com/developerworks/linux/library/l-awk1.html) | |
63 - Long AWK tutorial (http://www.grymoire.com/Unix/Awk.html) | |
64 - Learn AWK in 1 hour (http://www.selectorweb.com/awk.html) | |
65 - awk cheat-sheet (http://cbi.med.harvard.edu/people/peshkin/sb302/awk_cheatsheets.pdf) | |
66 - Collection of useful awk one-liners (http://student.northpark.edu/pemente/awk/awk1line.txt) | |
67 | |
68 ----- | |
69 | |
70 **AWK programs** | |
71 | |
72 Most AWK programs consist of **patterns** (i.e. rules that match lines of text) and **actions** (i.e. commands to execute when a pattern matches a line). | |
73 | |
74 The basic form of AWK program is:: | |
75 | |
76 pattern { action 1; action 2; action 3; } | |
77 | |
78 | |
79 **Pattern Examples** | |
80 | |
81 - **$2 == "chr3"** will match lines whose second column is the string 'chr3' | |
82 - **$5-$4>23** will match lines that after subtracting the value of the fourth column from the value of the fifth column, gives value alrger than 23. | |
83 - **/AG..AG/** will match lines that contain the regular expression **AG..AG** (meaning the characeters AG followed by any two characeters followed by AG). (This is the way to specify regular expressions on the entire line, similar to GREP.) | |
84 - **$7 ~ /A{4}U/** will match lines whose seventh column contains 4 consecutive A's followed by a U. (This is the way to specify regular expressions on a specific field.) | |
85 - **10000 < $4 && $4 < 20000** will match lines whose fourth column value is larger than 10,000 but smaller than 20,000 | |
86 - If no pattern is specified, all lines match (meaning the **action** part will be executed on all lines). | |
87 | |
88 | |
89 **Action Examples** | |
90 | |
91 - **{ print }** or **{ print $0 }** will print the entire input line (the line that matched in **pattern**). **$0** is a special marker meaning 'the entire line'. | |
92 - **{ print $1, $4, $5 }** will print only the first, fourth and fifth fields of the input line. | |
93 - **{ print $4, $5-$4 }** will print the fourth column and the difference between the fifth and fourth column. (If the fourth column was start-position in the input file, and the fifth column was end-position - the output file will contain the start-position, and the length). | |
94 - If no action part is specified (not even the curly brackets) - the default action is to print the entire line. | |
95 | |
96 | |
97 **AWK's Regular Expression Syntax** | |
98 | |
99 The select tool searches the data for lines containing or not containing a match to the given pattern. A Regular Expression is a pattern descibing a certain amount of text. | |
100 | |
101 - **( ) { } [ ] . * ? + \ ^ $** are all special characters. **\\** can be used to "escape" a special character, allowing that special character to be searched for. | |
102 - **^** matches the beginning of a string(but not an internal line). | |
103 - **(** .. **)** groups a particular pattern. | |
104 - **{** n or n, or n,m **}** specifies an expected number of repetitions of the preceding pattern. | |
105 | |
106 - **{n}** The preceding item is matched exactly n times. | |
107 - **{n,}** The preceding item ismatched n or more times. | |
108 - **{n,m}** The preceding item is matched at least n times but not more than m times. | |
109 | |
110 - **[** ... **]** creates a character class. Within the brackets, single characters can be placed. A dash (-) may be used to indicate a range such as **a-z**. | |
111 - **.** Matches any single character except a newline. | |
112 - ***** The preceding item will be matched zero or more times. | |
113 - **?** The preceding item is optional and matched at most once. | |
114 - **+** The preceding item will be matched one or more times. | |
115 - **^** has two meaning: | |
116 - matches the beginning of a line or string. | |
117 - indicates negation in a character class. For example, [^...] matches every character except the ones inside brackets. | |
118 - **$** matches the end of a line or string. | |
119 - **\|** Separates alternate possibilities. | |
120 | |
121 @REFERENCES@ | |
122 ]]> | |
123 </help> | |
124 </tool> |