Showing posts with label sed and grep - mastering unix file processing. Show all posts
Showing posts with label sed and grep - mastering unix file processing. Show all posts

Wednesday, 7 November 2012

Print previous , present and next line of a line matching pattern

Here is the link from great jadu , my friend and master of sed,awk and bash scripting. 


http://unstableme.blogspot.in/2008/05/print-currentnextprevious-line-using.html    

 Q. Print present line and previous line to the pattern that matches  "Sriram" word below using awk:

 Exercise 1:
 cat sample.txt

 Jadu previous line
Sriram present line
Varun next line
 Gowda some line
 Prasahanth some line

Output should be :

 Jadu previous line
Sriram present line
Varun next line

Answer: You can do that using grep command, which supports  -A and -B options

grep -B1 -A1  -i  "sriram"  sample.txt

Refer to "sed" solution given by jadu  in the link above.

Sunday, 16 September 2012

Executable File Searching Techniques

Following are the steps to perform executable File Search if there is no Installer Evidence or known File Evidence found:


1)    Search for a String Recursively and Redirect it to File:
•    Recursively searching for a string in all the files and redirect it to a file.

Syntax:      
              grep -ir “<pattern>” <PATH>     >      <FileName>

Options:
•    i: Ignore case
•    r: recursive search
                                                                                                              
Example:

          grep -ir "bpa\|suite\|analysis"  *  >   filelist.tmp


2)    File and strings search:
•    Search for a string in contents of the file.

         Syntax:
                      grep -i  “<pattern>”  <filename>  | grep –v “<pattern>”

         Options:
•    v: ignore these strings

         Example: 

          grep -i "bpa\|suite\|analysis" filelist.tmp | grep -v ".jar\|.html\|.doc\|.xml\|man1\|txt\|.loc"

3)    Search for Executables from desired path:
•    Lists all the executables in the particular path.

         Syntax:
find <path> -type f -perm 0755  -a ! -name “<pattern>”

         Options:
•    type     : Type of file to be searched, a file or a directory.
•    perm    : Permissions of the file
•    a           : AND Condition
•    !            : Not Operator
•    name    : Name of the File.
  
         Example:

         find . -type f -perm 0755 -a ! -name "java"  -a ! -name "*.msb" -a ! -name "*.qm" -a ! -name "*.sh" -a ! -name "*.jar"

Tuesday, 4 September 2012

Sed: Insert a string after every line of a file

Input to sed :   

AccessMode.Local
Account
AcquisitionMode
AcquisitionMode.Leased
AcquisitionMode.Loaned
AcquisitionMode.Purchased
AcquisitionMode.Rented
Action


Required Output from sed : 

SELECT 'AccessMode.Local',
UNION ALL
SELECT 'Account',
UNION ALL
SELECT 'AcquisitionMode',
UNION ALL
SELECT 'AcquisitionMode.Leased',
UNION ALL
SELECT 'AcquisitionMode.Loaned',
UNION ALL
SELECT 'AcquisitionMode.Purchased',
UNION ALL
SELECT 'AcquisitionMode.Rented',
UNION ALL
SELECT 'Action',
UNION ALL


Here is the two liner Script:

sed -ne "s/\(.*\)/SELECT '\1',/p" ResourceString.txt  > ResourceString_SELECT.txt
sed -ne "s/$/\nUNION ALL/p" ResourceString_SELECT.txt  > ResourceString_SELECT_UNIONALL.txt   


Reference : http://www.computing.net/answers/unix/how-to-insert-single-quote-in-a-text-line/8570.html 






Saturday, 18 April 2009

searching for exact string

If i want to get the lines containing exact word ignoring the lines which consists of same word embedded as part of another word,i can use grep like this:

$inputline = "sriram sriraman raja rajaram"

sol : echo $inputline | grep '\&ltsriram\&gt' selects the line containing a separate word sriram, but not the line which contains words like sriraman sriramkumar etc..

Monday, 9 February 2009

extracting part of file and replacing string in that

suppose i have a file emp.txt containing the following lines.I want to extract lines beginning with sriram till rajesh with out knowing the line numbers and replace company1 with xyz company in those lines and rite to a file.

1|sriram|manager|company1|203
2|satish|director|company1|205
3|raghu|software|company2|305
4|ravi|designer|company3|203
.
.
.
98|rajesh|executive|company5|405
99|rajesh|engineer|company4|204

Sol : sed -ne '/.*|sriram|/,/.*|rajesh|/p' emp.txt | sed 's/|company1|/|xyz|/g'

First sed prints line containig sriram till first occurrence of line containing rajesh and replaces string company1 with xyz.

Wednesday, 14 January 2009

How to print a word next to the selected key word?

Say, I have a file like this :

cat >myfile
my name is sriram working in x company
my name is ramu working in y company
my name is psr working in z company
my name is rajesh working in global company
my name is ravi working in dash company

I want to pick all the names of the lines following the word "is" :

This is one way to do using sed out of millions of ways:

$mystr="is"; sed "s/.*$mystr \([^ ]*\) .*/\1/" myfile -- prints the following :

sriram
ramu
psr
rajesh
ravi

Sunday, 14 December 2008

sed - grouping multiple instructions

Grouping two or more commands in sed using { } braces.

Exp : For example, if i want to remove all blank lines and # type comments in my shell script.

I can use sed -e 's/#.*//' -e '/^$/ d' myscript.sh . (using -e to write two or more commands at the same time)

without -e option the above version can be written as :

sed -n '
{
s/#.*//
/^$/ d
p
}' test.sh > noblank.txt

The redirected noblank.txt file contains my test.sh lines w/o # type comments and blank lines.

head command using sed

An alternate way to head command using sed is q or quit command..

Exp : sed '10 q' prints first 10 lines of the file and quit.

remove recurring numbers

If i want to remove recurring numbers in a line of file1 like.. 123 456 789 and replace it with only 123

then do this,

sed 's/\([0-9]*\) \([0-9]*\) \([0-9]*\)/\1/' file1 > tmp

Explanation : I have used TRE (tagged regular expression) to remember the patterns and replaced them with a first group value through "\1".This way we can have up to "\9" values. The result file is put to a "tmp" file.

Note : Here after I would replace "Explanation" with "Exp" to minimize my effort.

sed slash delimiter

The character after the s in sed 's/' is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want, however. If you want to change a pathname that contains a slash - say /usr/local/bin to /common/bin - you could use the backslash to quote the slash:

sed 's/\/usr\/local\/bin/\/common\/bin/' new

Gulp. Some call this a 'Picket Fence' and it's ugly. It is easier to read if you use an underline instead of a slash as a delimiter:

sed 's_/usr/local/bin_/common/bin_' new

Some people use colons:

sed 's:/usr/local/bin:/common/bin:' new

Others use the "|" character.

sed 's|/usr/local/bin|/common/bin|' new

Pick one you like. As long as it's not in the string you are looking for, anything goes. And remember that you need three delimiters. If you get a "Unterminated `s' command" it's because you are missing one of them.

Tweets by @sriramperumalla