Mercurial > repos > bgruening > text_processing
diff awk.xml @ 0:5314e5d6f040 draft
Imported from capsule None
author | bgruening |
---|---|
date | Thu, 29 Jan 2015 07:53:17 -0500 |
parents | |
children | 20344ce0c811 |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/awk.xml Thu Jan 29 07:53:17 2015 -0500 @@ -0,0 +1,124 @@ +<tool id="tp_awk_tool" name="Text reformatting" version="@BASE_VERSION@.0"> + <description>with awk</description> + <macros> + <import>macros.xml</import> + </macros> + <expand macro="requirements"> + <requirement type="package" version="4.1.0">gnu_awk</requirement> + </expand> + <version_command>awk --version | head -n 1</version_command> + <command> +<![CDATA[ + awk + --sandbox + -v FS=' ' + -v OFS=' ' + --re-interval + -f "$awk_script" + "$infile" + > "$outfile" +]]> + </command> + <inputs> + <param name="infile" format="txt" type="data" label="File to process" /> + <param name="code" type="text" area="true" size="5x35" label="AWK Program" help=""> + <sanitizer> + <valid initial="string.printable"> + <remove value="'"/> + </valid> + </sanitizer> + </param> + </inputs> + <configfiles> + <configfile name="awk_script">$code</configfile> + </configfiles> + <outputs> + <data name="outfile" format_source="infile" metadata_source="infile"/> + </outputs> + <tests> + <test> + <param name="infile" value="awk1.txt" /> + <!-- commas are not allowed in a value field. Values with comma will be splitted --> + <param name="code" value='$2>0.5 { print $2*9"\t"$1 }' /> + <output name="outfile" file="awk_results1.txt" /> + </test> + </tests> + <help> +<![CDATA[ +**What it does** + +This tool runs the unix **awk** command on the selected data file. + +.. class:: infomark + +**TIP:** + +This tool uses the **extended regular** expression syntax (not the perl syntax). +**\\d**, **\\w**, **\\s** etc. are **not** supported. + + +**Further reading** + +- Awk by Example (http://www.ibm.com/developerworks/linux/library/l-awk1.html) +- Long AWK tutorial (http://www.grymoire.com/Unix/Awk.html) +- Learn AWK in 1 hour (http://www.selectorweb.com/awk.html) +- awk cheat-sheet (http://cbi.med.harvard.edu/people/peshkin/sb302/awk_cheatsheets.pdf) +- Collection of useful awk one-liners (http://student.northpark.edu/pemente/awk/awk1line.txt) + +----- + +**AWK programs** + +Most AWK programs consist of **patterns** (i.e. rules that match lines of text) and **actions** (i.e. commands to execute when a pattern matches a line). + +The basic form of AWK program is:: + + pattern { action 1; action 2; action 3; } + + +**Pattern Examples** + +- **$2 == "chr3"** will match lines whose second column is the string 'chr3' +- **$5-$4>23** will match lines that after subtracting the value of the fourth column from the value of the fifth column, gives value alrger than 23. +- **/AG..AG/** will match lines that contain the regular expression **AG..AG** (meaning the characeters AG followed by any two characeters followed by AG). (This is the way to specify regular expressions on the entire line, similar to GREP.) +- **$7 ~ /A{4}U/** will match lines whose seventh column contains 4 consecutive A's followed by a U. (This is the way to specify regular expressions on a specific field.) +- **10000 < $4 && $4 < 20000** will match lines whose fourth column value is larger than 10,000 but smaller than 20,000 +- If no pattern is specified, all lines match (meaning the **action** part will be executed on all lines). + + +**Action Examples** + +- **{ print }** or **{ print $0 }** will print the entire input line (the line that matched in **pattern**). **$0** is a special marker meaning 'the entire line'. +- **{ print $1, $4, $5 }** will print only the first, fourth and fifth fields of the input line. +- **{ print $4, $5-$4 }** will print the fourth column and the difference between the fifth and fourth column. (If the fourth column was start-position in the input file, and the fifth column was end-position - the output file will contain the start-position, and the length). +- If no action part is specified (not even the curly brackets) - the default action is to print the entire line. + + +**AWK's Regular Expression Syntax** + +The select tool searches the data for lines containing or not containing a match to the given pattern. A Regular Expression is a pattern descibing a certain amount of text. + +- **( ) { } [ ] . * ? + \ ^ $** are all special characters. **\\** can be used to "escape" a special character, allowing that special character to be searched for. +- **^** matches the beginning of a string(but not an internal line). +- **(** .. **)** groups a particular pattern. +- **{** n or n, or n,m **}** specifies an expected number of repetitions of the preceding pattern. + + - **{n}** The preceding item is matched exactly n times. + - **{n,}** The preceding item ismatched n or more times. + - **{n,m}** The preceding item is matched at least n times but not more than m times. + +- **[** ... **]** creates a character class. Within the brackets, single characters can be placed. A dash (-) may be used to indicate a range such as **a-z**. +- **.** Matches any single character except a newline. +- ***** The preceding item will be matched zero or more times. +- **?** The preceding item is optional and matched at most once. +- **+** The preceding item will be matched one or more times. +- **^** has two meaning: + - matches the beginning of a line or string. + - indicates negation in a character class. For example, [^...] matches every character except the ones inside brackets. +- **$** matches the end of a line or string. +- **\|** Separates alternate possibilities. + +@REFERENCES@ +]]> + </help> +</tool>