Ilovepdf Merged

8/10/2021 iON| Digital Learning
Operating Systems
The structure of a Computer system can be visualized as having four basic components:
1. Hardware – Provides basic computing resources - CPU, memory, I/O devices
2. Operating system – Controls and coordinates use of hardware among various applications and users
3. Application programs – Define the ways in which the system resources are used to solve the computing
problems of the users
4. Users – Anybody who desires work to be done by a computer system. People, machines, other computers
Types of OS
OS are classified into following different types depending on their capability of processing.

Single User and Single Tasking OS:
These are simple operating system designed to manage one task at a time ,for use by a single user for a
standalone single computer for performing a single task .
Single User and Multitasking OS:
These OS allow execution of more than one task or process concurrently by dividing the processor time amongst
different tasks.
Multi-programming OS:
These OS allow more than one programs to run at the same time .
Real Time OS:
These are designed to respond to an event within a predetermined time. These operating systems are used to
control processes
Embedded OS:
https://g91.tcsion.com/LX/contents/content_home?content_player=true&LaunchFrom=iHUB&newWindow=Y&User-Agent=Computer&usrid=14994006… 1/3
Embedded in a device in the ROM. They are specific to a device and are less resource intensive.
Function Of OS
The OS performs basic tasks such as controlling and allocating memory, prioritizing system requests, controlling
input and output devices, facilitating networking, and managing files
Introduction to UNIX OS
Unix is an OS for Programmers as shell(the command interpreter)provides the programming facility.

It provides an in-built security mechanism through the user name and password, combined with the
access rights associated with files
Developed by Ken Thompson and Ritchie originally in assembly, and later in C, thus making it portable
to other machines
Supports C, Fortran, Basic, Pascal, COBOL, Lisp, Prolog, Java, Ada compilers
Features of Unix Operating System
• Multi-user and Multitasking

• Everything is a file
• Configuration data stored in text
• Small, single purpose programs
• Ability to chain programs together to perform complex task
• Facility of background processing
Architecture of the UNIX System
The UNIX operating system has a layered architecture having three main components
• Kernel
• Shell
• Utilities
The Unix Kernel
Kernel is a collection of programs mostly written in C which allocate the system resources and coordinate all the
details of the computer's internals.
Functions of Kernel:
It allocates time and memory to programs and handles the file store and communications
Interacts directly with the hardware through device drivers
Provides sets of services to programs
Manages memory, controls access, maintains file system, handles interrupts, allocates resources of the
computer
System calls
The system calls are functions used in the kernel itself. UNIX system calls are used to manage the file system,
control processes, and to provide interpocess communication.
System calls can be categorized as:
File structure related calls -For example create, open,read, write,lseek,dup etc.
Process Related calls -For example fork,exec,wait,exit etc
Inter process related calls - For example pipe,msgget,msgsnd etc.
The Unix File System:
Unix File System is a hierarchical collection of 3 types of files:
Ordinary Files
Directory Files
Special Files (device, pipe, fifo, socket).
Characteristics of Unix Files:
UNIX file is featureless because it is simply an array of bytes.

Dominant file type in UNIX is the text file.
System related files are also stored in text form.
Separate device can be added by creating a file for it.
Root is the supremo and is represented by the ‘/’. Every sub-directory must have a parent.
File names can be up to 14 characters long, can contain both upper and lower case alphabets, digits, a dot,
hyphen (-), underscore (_) anywhere but should not have a blank or tab.
Files are accessed using path names. Path names are a sequence of directory names separated by ‘/’.
There are two types of path names in unix.
Absolute path name - file location is determined with respect to the root.
Relative path name - file location is determined with respect to the current directory.
Structure of Unix File System:
The Unix File System (UFS) looks hierarchical but it is actually a directed a-cyclic graph because files can be
shared.
Following are some directories available under root in UFS.
/home – It holds user’s home directories. In other UNIX systems, this can be /usr directory.

/bin – It holds many of the basic Linux programs; bin stands for binaries, files that are executable.
/usr – It holds many user-oriented directories:
/sbin – It holds system files that are usually run automatically.
/etc – It and its subdirectories hold many of Linux config files.
/dev – It holds device files. All info sent to /dev/null is thrown into trash. Your terminal is one of the /dev/tty
files.
Unix file System Organization:
Unix divides physical disks into logical disks called partition which is comprising of a set of consecutive
cylinders.
The UFS resides on a single partition.
Each file system contains four blocks :
a) boot block : This block is located in the first few sectors of a file system. It contains the initial
bootstrap program used to load the operating system.Typically, the first sector contains
a bootstrap program that reads in a larger bootstrap program from the next few sectors,
and so forth.
b) super block : This block describes the state of the file system such as the total size of the partition,
the block size, pointers to a list of free blocks, the inode number of the root directory,
magic number or file signature (the first few bytes of a file which are unique to a
particular file type), etc.
c) inode list : A linear array of inodes (index nodes). This is a data structure which describes the
attributes of a file. There is a one to one mapping of files to inodes and vice versa.
An inode is identified by it's "inode number". Users uses file names to refer to a file but
Unix represent files in terms of inodes.
d) data blocks : blocks containing the actual contents of files
Internal File Maintenance:
For each file created in the system, an inode is also created. Inode is a disk file record of 64 bytes that maintains
the permanent attributes of a file.
An inode is permanent and it exists until the corresponding file is removed from the system.
Sample details of an inode –
Owner and group identifiers

File type and file size
Number of links for this file
Times of file creation, last file access and modification, and last inode modification
List of access rights – read/write/execute permissions

Reference count showing number of times file is opened
Physical address of file on the disk: array of 13 pointers for data storage
Whenever a file is opened, its inode is brought into main memory. The active inode is kept there until the file is
closed and is used to locate the beginning of an open file on disk and to verify that every I/O request comes from
a valid user as per specified access permissions.
Users and Access Rights
Objective
This module is to have an understanding of different user types in UNIX and their access rights.
Course Content
User types
File access permissions
Changing file access permissions using symbolic mode
Changing file access permissions using octal mode
User Types in Unix
There are three types of accounts in a Unix system:

- Root Account
- System Account
- User Account
Managing users and groups
Unix supports a concept of Group which logically groups a number of accounts.

➢ Every account would be a part of any group.
➢ Unix groups plays important role in handling file permissions and process management.
➢ Grouping of users allows to grant and revoke file permissions collectively.
File Permission & Ownership level

File ownership is an important component of Unix that provides a secure method for storing files. Every file in Unix has the following
attributes:
File Permission Types (Mode)

Three type of permissions can be set for any ownership level:
Read
Write
Execute
These three modes or permissions have different meaning for a file and a directory;
Changing Permissions (chmod)

To change file or directory permissions, the chmod (change mode) command is used. There are two ways to use chmod:
Symbolic mode
Absolute mode
chmod command options:
Using chmod in Symbolic Mode:
With symbolic representation, permission set can be added, deleted, or specified using the operators listed in the following table:
Example:
Let us assume there is a file name "abc" , whose initial permissions are only read permission to user and group and others. We will
provide full permissions to all three types of users.
Let us verify the existing file permissions using ls -l command as below and observe the next steps, like changing the permissions and
verifying the changes.
In the above example,
chmod u+wx,g+wx abc , adds write and execute permission to user and group to the file abc
chmod g-x abc, removes execute permission from group from the file abc.
chmod o=r-- abc, provides read permission and removes write and execute permission from others for file abc
Using chmod in Absolute(octal) Mode
The second way to modify permissions with the chmod command is to use a number to specify each set of permissions for the file.
Each permission is assigned a value, as the following table shows, and the total of each set of permissions provides a number for that
set.
Examples: Observe the following sequence of command execution statements and analyze the output
1. Let us view the initial permissions of the file abc

ls -l abc
-rw-r--r-- 1 735873 oinstall 0 Feb 7 12:37 abc
If we observe the output, initially owner have read and write permissions and group and others have only read permission
2. Let us remove all the permission from all users for abc file, as below
chmod 0 abc
Let us verify the changed permissions
ls -l abc
---------- 1 735873 oinstall 0 Feb 7 12:37 abc
Yes all permissions are taken away from all users on the file abc
3. To provide all permissions to the user, read permission to group and write and execute permissions for Others,
use the below command

chmod 743 abc
Let us verify the changed permissions

ls -l abc
-rwxr---wx 1 735873 oinstall 0 Feb 7 12:37 abc
From the output all permissions to user, read permission to group and write and execute
permissions for Others, Use the below command
4. chmod 755 abc

The above command provides all permissions to the user, read and execute permissions to group and others
Let us verify ls -l abc
-rwxr-xr-x 1 735873 oinstall 0 Feb 7 12:37 abc
5. chmod 777 abc

The above command provides all permissions to all users . We can verify using below command
ls -l abc
-rwxrwxrwx 1 735873 oinstall 0 Feb 7 12:37 abc
Basic UNIX commands :

The basic Unix commands are:
echo
It displays a line text.
Echo the given string or strings to standard output.
Syntax:
echo [OPTION]... [STRING]...
Example:
echo "Good Morning"
This will display the following in the standard output
Good Morning
passwd command
The passwd command is used to change passwords for user accounts. A normal user may only
change the password for his or her own account, while the superuser may change the password for
any account.
Syntax : passwd [options] [LOGIN]
Options:
uname command
Print certain system information.
Syntax : uname [OPTION]...
Options:
Examples: To show the name of the OS
uname command:
Used to get the information about the current system (the name, version and other details)
$uname
To get the kernel name, you can use -s parameter.
$ uname -s
To get the kernel release, -r option is used
$ uname -r

who command:
who command prints information about all users who are currently logged in.
Syntax:
who [OPTION]... [ FILE ] [ am i ]
find command:
find is one of the powerful utility of Unix (or Linux). It can search
the entire file-system to locate files and directories according to the specific
search criteria. Other than searching files in can also perform actions(executing
commands) on searched files.

Syntax:
$ find [option] [action]
Examples :
1 . File Type based Search:
>> The following command finds the file named Abc in current(.) directory and
all its sub-directories
2. Wild-Card based Search : Wild-card characters like * and ? can be used:

>> To find all files where filename starts with Abc the command will be:
[390119@InGhyUnix ~]$ find . -name "Abc*"

./Abc
./Dir1/Abc
./Dir1/Abc.php
./Abc.txt
./Abc.dat
3 . Source Location based Search:

>>The command find can search in multiple source location :
[390119@InGhyUnix ~]$ find Dir1 Dir2 -name "Abc.*"

Dir1/Abc.php
Dir2/Abc.htm
Searches for the file Abc with any extension only in directories Dir1 and Dir2

4 . Size Based Search : Using find files can be searched based on its size .
>> To find the files with size more than 10Mb the command will be
[390119@InGhyUnix ~]$ find . -size +10M
./Abc.dat

5 . Access/Modification-Time Based Search : Using find files can be searched

based on the time of last access/modification :
>> To find the files which are accessed within last 3 days the command will be
[390119@InGhyUnix ~]$ find . -atime -3
./Abc
6 . Permission Based Search : Using find command files can be searched

based on the access permission they have
>> To find the files having read write and execute permission only for the owner
the command will be :
[390119@InGhyUnix ~]$ find . -perm 700

./Abc.txt
7 . Perform Action on the search result : Using exec option any action (command)
can be executed on the search result.
>> To remove all the files having extension .swp from the current directory the
command will be :
[390119@InGhyUnix ~]$ find . -name “*.swp” -exec rm {} \;
bc command:
bc is the command used for basic mathematical calculations.
It can be used either a mathematical scripting language or as an interactive mathematical shell .
In interactive mode it accepts input from the terminal and providing calculations on requested

arithmatic expression.
That means typing the command bc on a Unix command prompt and entering a mathematical
expression, such as (1 + 4) * 2 , whereupon 10 will be the output
While used with files , bc starts by processing code from all the files listed on the command line in
the order listed. After all files have been processed, bc reads from the standard input. All code is
executed as it is read.
Syntax: bc [ -hlwsqv ] [long-options] [ file ... ]
Example:
$ echo "2+5" | bc
Output will be:
tty command:
Prints the file name of the terminal connected in standard input.
Syntax : tty [OPTION]...
UNIX Text Editors

To write a program or a paper even to edit a mail message, you need to create, write,store, retrieve,
view, and edit files. In unix several text editors available for this .
Editors available unix are ed, node,NEdit, emacs and vi .
One of the first end-user programs hosted on the system and standardized in UNIX-based systems
ever since is the vi editor. ed is a line editor for the UNIX operating system.
vi Editor:
vi (pronounced "vee-eye") is short for "vi"sual editor. It is a visual editor used to create and edit
text files.
It displays a window into the file being edited that shows 24 lines of text. vi lets you add, change
and delete text, but does not provide formatting capabilities like centering lines or indenting
paragraphs.
vi works in three different modes:
1. Edit Mode -where any key is entered as text

2. Command Mode -where keys are used as commands
3. Ex Mode -ex commands can be entered in last line to act on text
Invoking vi with/without filename puts it in command mode:
vi []
Commands to be used in vi:
For cursor movement within the vi editor, we can not use the mouse . We have to use key
commands. To use a key command, user has to be in the command mode. To go to command mode,
press the Esc key.
Commands to Enter and Exit vi:
command Purpose of the command
vi filename edit filename starting at line 1
vi –r recover filename that was being edited when system crashed

filename
The new or modified file is saved when user exits from vi. But it is also possible to exit without
saving the file.
To use a command to exit from vi , after pressing the Esc key ,colon : is to be pressed.
The cursor moves to bottom of screen whenever a colon (:) is typed. This type of command is
completed by hitting the (or ) key.
command Purpose of the command
:wq quit vi, writing out modified file to file named in original

invocation
:q quit (or exit) vi
:q! quit vi even though latest changes have not been saved for
this vi call
:w write current contents to file named in original vi call
Commands to insert or add text:
Following are the commands to be used to insert or add text.
key Purpose of the command
u Undo the last command’s action
i
insert text before the cursor,until is hit

I
insert text at beginning of current line,
until hit

a append text after cursor, until hit
A append text to end of current line, until hit
o open and put text in a new line below current

line, until hit
O open and put text in a new line above current

line, until hit
Commands to modify texts:
r replace single character under cursor (no needed)
R replace characters, starting with current cursor position, until hit
cw change the current word with new text,

starting with the character under cursor, until hit
cNw change N words beginning with character under cursor, until hit;

e.g., c5w changes 5 words
C change (replace) the characters in the current line, until hit
cc change (replace) the entire current line, stopping when is hit
Ncc change (replace) the next N lines, starting with the current line,
or stopping when is hit
cNc
Commands to Delete texts:
x delete single character under cursor
Nx delete N characters, starting with character under cursor
cw delete the single word beginning with character under cursor
dw change N words beginning with character under cursor, until hit;

e.g., c5w changes 5 words
dNw delete N words beginning with character under cursor;

e.g., d5w deletes 5 words
D delete the remainder of the line, starting with current cursor position
dd delete entire current line
Ndd delete N lines, beginning with the current line;

or e.g., 5dd deletes 5 lines
dNd
Commands to copy and paste text:
yy copy (yank, cut) the current line into the buffer
Nyy copy (yank, cut) the next N lines, including the current line, into the buffer
p put (paste) the line(s) in the buffer into the text after the current line
Commands for searching text
/string search forward for occurrence of string in text
? search backward for occurrence of string in text

string

UNIX FILE COMMANDS

In this module following important file handling commands will be discussed:
touch
cat
cat > (single redirection)
cat >>(double redirection)
cp
mv
cmp
comm
diff
touch command
Touch command is used in Unix to create an empty file.
$ touch file1 file2

$ ls -l
-rw-r--r-- 1 user group 0 2014-02-08 7:14 file1
-rw-r--r-- 1 user group 0 2014-02-08 7:14 file2
$
Touch command is used to change the timestamps (access time and modification time of a file).
cat command
Use of cat command:
'cat' command is used to display a file content.
Syntax:
$ cat filename [ filename ]
$ cat data
This is an unix file
This is line two
$
cat > and cat >> command

Redirection operator ( > and >>) can be used with cat command to take input from standard input
device(keyboard) and store them in a file.
Syntax: $ cat > [filename]
$ cat > data

The file created using
cat > filename
^d
$
cat with single redirection(cat >filename) , creates a new file if no file with the given file name
already exists and overrites the file if a file with given file name already exists.
cat with double redirection (cat >>filename) appends the content given through standard input to the
file.
cp command
cp command is used to copy one file's content to another file.
Syntax: $ cp [source filename] [destination filename]
Switches:
1. cp -i [sourcefilename] [destination filename] This command copies the content of a file
interactively.
2.cp -r [source_directory] [destination_directory] This command copies the whole file hierarchy.
3. cp -v [sourcefilename] [destination filename] Copies in verbose mode, explains what is being

done
mv command
mv command is used for:
1. for renaming a file in same directory.
2. for moving a file to a different directory.
Syntax:
$ mv [sourcefilename] [destinationfilename]
$ mv [source_directory] [destination_directory]
Directory Commands:
Every item in the UNIX filesystem tree is either a file, or a directory. A directory is like a file folder.
A directory can contain files, and other directories. A directory contained within another is called
the child of the other. A directory in the filesystem tree may have many children, but it can only
have one parent. A file can hold information, but cannot contain other files, or directories.
The place in the file system tree where an user is located is called the current
working directory (which can be listed using pwd command).
As a user logs in, user is put in his/her home directory.
mkdir(make directory) command:

The mkdir command creates a directory with specified name in the present working directory or
specified path.
Syntax: $ mkdir [ ]
ls Command
Lists the contents of any directory (current directory if no directory specified).
Sorts entries without option
$ ls [option(s)] [filename]
File comparison commands:

cmp – This command is used to compare two files, and if they differ, tells the first byte and line
number where they differ. If input files are same , returns nothing.
Syntax :
cmp options file1 file2

To use cmp commands both the files need to be sorted.
comm- This command is used to compare two sorted files.
Syntax:
comm [options] file1 file2
One set of options allows selection of ‘columns’ to suppress.
-1: suppress lines unique to file1 (column 1)
-2: suppress lines unique to file2 (column 2)
-3: suppress lines common to file1 and file2 (column3)

Example: To show only lines common between file1 and file2, command is
$ comm -12 file1 file2
diff - This command is used to compare two files line by line. The output indicates how the lines in
each file are different, and the steps that will be involved in changing file1 to file2.
The change commands are in the format [range][acd][range]. The range on the left may be a line
number or a comma-separated range of line numbers referring to file1, and the range on the right
similarly refers to file2. The character in the middle indicates the action i.e. add, change or delete.
‘LaR’ – Add lines in range ‘R’ from file2 after line ‘L’ in file1.
‘FcT’ – Change lines in range ‘F’ of file1 to lines in range ‘T’ of file2.
‘RdL’ – Delete lines in range ‘R’ from file1 that would have appeared at line ‘L’ in file2
Syntax: diff [options] file1 file2
USAGE OF WinSCP
Introduction to WinSCP
WinSCP (Windows Secure CoPy) is a popular , free and open-source Secure File Transfer application (SFTP) for Windows computers .It is used to transfer files securely
between remote server and local computer and vice verse. It includes support for SCP file transfers, and has the ability to create secure tunnels via SSH. Secure FTP
(SFTP) is similar to FTP, but with SFTP the entire session is encrypted, so that passwords are never sent in the clear text , and are therefore much less vulnerable to
interception. That is ,for transferring files securely, we can use Secure copy or SCP protocol, it uses Secure Shell (SSH), which encrypts the content before transferring
data over network. It runs over TCP port 22 by default.
How to use WinSCP
Step I : Start WinSCP

To start with double click on WinSCP icon in the desktop or select it from Programs Group . The
first screen will look as below
Step II : Logging in to Server
Enter IP address of the server in Host Name field.

Unix login userid in the User name field
Unix login Password in Password field
Choose the protocol ( SFTP/ SCP / FTP)
For example
1) Host name used : 172.26.132.40 (For example)
2) User name used : 390119
3) Password used : ********
3) Protocol selected : SCP

Step III : Select/Navigate to working directory

1. We can select the working directory in both Local as well as remote machine from the drop
down menu.
2. Tool bar can be used to navigate to desired folder

Step IV : Transferring content
Content can be transferred in two ways

A . Downloading : Transferring content from Remote server to Local machine
B . Uploading : Transferring content from Local machine to Remote server
Following are the Steps to be performed for Downloading file/Directory from Server
Step - A1 : Select the file to transfer in the Remote pane,
Step - A2 : Right click → Select Copy or move option
Step - A3 : Select the destination folder in the local machine(can be changed by selecting Browse button )
Step - A4 : The file Unix_Recap.txt copied to desktop.
Following are the steps to be followed for uploading files to Remote Server
Step - B1 : Select the file to copy in Local machine(Left) pane.

Step - B2 : Drag it and drop it to the remote pane desired location.
Step - B3 : Select the desired location from drop down list and select copy
Saving sessions for future use:
All the session information can be saved for future use and faster access.
The information can be saved in following ways :
>> Server IP
>> User name,
>> Password(Not recommended)
>> Communication protocol
Filters
Filter is a programm in unix . It takes its input from another program, performs some operation on
that input, and writes the result to the standard output. Thus the common use of filters is to modify
or restructure output.
Some common filters in UNIX are:
uniq – Removes identical adjacent lines
head – displays first n lines of a file .
tail – displays last n lines of a file .
sort – sorts files by line (lexically or numerically)
cut – select portions of a line.
wc – word count (line count, character count)
tr – translate
grep, egrep – search files using regular expressions
head
This command list the beginning of a file to standard output. The default is 10 lines, but a different
number can be specified. The command has a number of options.
Syntax:
head [OPTION] [FILE]
Options:
-c Prints the first N bytes of file; with leading -, prints all but the last N bytes of the file.
-n Prints first N lines; with leading - print all but the last N lines of each file.

Example: To display the first 10 lines of the file myfile.txt. $head myfile.txt

To display the first 100 lines of the file myfile.txt.
$head -n100 myfile.txt
To print the first 5 bytes from the file
$ head -c5 myfile.txt
tail
List the (tail) end of a file to stdout. The default is 10 lines, but this can be changed with the -n
option. Commonly used to keep track of changes to a system log-file, using the -f option, which
outputs lines appended to the file.
Syntax:
tail [OPTION]... [FILE]...
Example:
To display the last 10 lines of the file myfile.txt.
$tail myfile
To display the last 100 lines of the file myfile.txt.
$ tail -100 myfile.txt
$tail –n 100 myfile.txt
more
more command allows to view text files or other output in a scrollable manner. When can command
is used to view a very long file, all the output scrolls off the top of your screen and only the last
page can be viewed. more command solves this problem by allowing the output of cat command
one screenful of data at a time.
Syntax:
more [option] filename
Options:
-num This option specifies an integer which is the screen size (in lines).
-d more will prompt the user with the message "[Press space to continue, 'q' to quit.]" and will
display "[Press 'h' for instructions.]" instead of ringing the bell when an illegal key is pressed.
-l more usually treats ^L (form feed) as a special character, and will pause after any line that
contains a form feed. The -l option will prevent this behavior.
-p Do not scroll. Instead, clear the whole screen and then display the text.
tr

tr command automatically translates or substitute characters.

Syntax:
tr [OPTION] set1 [set2]
Translate, squeeze, and/or delete characters from standard input, writing to standard output.
Options:
c. : complements the set of characters in string.
d. : deletes the characters in set1
s. : replaces repeated characters listed in the set1 with single occurrence
t. : truncates set1
Example: To replace any occurrence of a by x, b by y and c by z in a given string
$echo “about to call “|tr [abc] [xyz]
Output : xyout to zxll
Example: To replace non matching characters
$ echo "Hello"|tr -c e a
Output : aeaaaa
In the above example , except the character “e” other characters are replaced by a
Example: Squeez , we can squeeze more than one occurrence of continuous characters with single
occurrence.
$echo “about to call “|tr – s ‘ ‘
Output : about to call
Above example squeezes two or more blank spaces into one.
sort
sort command reorders the lines of a file in ascending or descending order.
The default order is ascending .
Syntax:
sort -t field_delemeter [OPTION] file1 [file 2]
Options:
-k n sort on the nth field of the line
-t char use char as the field delimiter
-n sort numerically
-r reverse order sort
-u removes repeated lines
-m list merge sorted files in list
Examples:
Below examples will help you to understand sort used with different options:
Example 1:
Consider a file named “list”, which has below data
1, Justin Timberlake, Title 545, Price $7.30
2, Lady Gaga, Title 118, Price $7.30
3, Johnny Cash, Title 482, Price $6.50
4, Elvis Presley, Title 335, Price $7.30
5, John Lennon, Title 271, Price $7.90
To sort on the 2nd field of file named “list” we have to use the below command:
$sort –t’,’ –k 2 list
Note: File list is comma separated file.
Output:
4, Elvis Presley, Title 335, Price $7.30
5, John Lennon, Title 271, Price $7.90
3, Johnny Cash, Title 482, Price $6.50
1, Justin Timberlake, Title 545, Price $7.30
2, Lady Gaga, Title 118, Price $7.30

Example 2: Numerically sorting:
To numerically sort data , option to be used is –n
Suppose list is the name of the file having following data:
19
20
49
00
If we sort it as below:
$sort list

Output is :
19
20
200
49
To get the expected output , the command will be
$sort –n list
Output:
19
20
49
200
Sort can sort multiple files also.
$sort file1 file2 file3 …
Example 3: Numerically sort in reverse order

$sort –nr list
Output :
200
49
20
19
Example 4: Sort the file list removing the repeated lines.
Syntax:
$sort –u filename
File list has following content:
Unix
Unix
Linux
Linux
Solaris
Axis
Axis
$sort –u list
Output:
Unix
Linux
Solaris
Axis

uniq
uniq command is used to suppress the duplicate lines from a file. It discards all the successive
identical lines except one from the input and writes the output.

Syntax:
uniq [option] filename

Options:
-u lists only the lines that are unique
lists only the lines that are duplicates
-c counts the frequency of occurrences

Suppress duplicate lines:
The default behavior of the uniq command is to suppress the duplicate line. Note that, you have to
pass sorted input to the uniq, as it compares only successive lines.
If the lines in the file are not in sorted order, then use the sort command and then pipe the output to
the uniq command.
Count of lines:
The -c option is used to find how many times each line occurs in the file. It prefixes each line with
the count.
Display only duplicate lines:
You can print only the lines that occur more than once in a file using the -d option. The -D option
prints all the duplicate lines.
Skip first N fields in comparison:
the -f option is used to skip the first N columns in comparison. Here the fields are delimited by the
space character.
cut
This command is used for text processing. You can use this command to extract portion of text from
a file by selecting columns.
Syntax:
cut –option filename
Select Column of Characters :
To extract only a desired column from a file use -c option.
The following example displays 2nd character from each line of a file test.txt.
$cut –c2 test.txt
Select Column of Characters using Range :

Range of characters can also be extracted from a file by specifying start and end position delimited
with -.
The following example extracts first 3 characters of each line from a file called test.txt
$cut –c 1-3 test.txt
Select Column of Characters using either Start or End Position :

Either start position or end position can be passed to cut command with -c option.
Following example extracts from 3rd character to end of each line from test.txt file.

$cut –c3- test.txt
To extract 8 characters from the beginning from the file test.txt,
$cut –c-8 test.txt

Select a Specific Field from a File :
Instead of selecting x number of characters you can combine option -f and –d to extract a whole
field.
The option -f specifies which field you want to extract,
The option -d specifies what delimiter that is used in the input file.
The following example displays only first field of each lines from /etc/passwd file using the field
delimiter: (colon). In this case, the 1st field is the username.
$ cut -d':' -f1 etc/passwd
paste
This is the command for merging together different files into a single, multi-column file. In
combination with cut, useful for creating system log files.
Syntax:
paste file1 file2

join
This utility allows merging two files in a meaningful fashion, which essentially creates a simple
version of a relational database.

The command join operates on exactly two files, but pastes together only those lines with a
common tagged field (usually a numerical label), and writes the result to standard output.

The files to be joined should be sorted according to the tagged field for the matchups to work
properly.
Example:
The content of two files file1 and file2 are as below,

$cat file1
100 Shoes
200 Laces
300 Socks
$cat file2
100 $40.0
200 $1.00
300 $2.00

The following command will join these two files.

$ join 1.data 2.data
100 Shoes $40.00
200 Laces $1.00
300 Socks $2.00
Pipe
In unix , you can connect two commands together so that the output from one program becomes the
input of the next program. Two or more commands connected in this way form a pipe. In shell the
symbol '|’ is used to represent pipe.

Purpose of Pipes :
Using pipe you can construct powerful unix command lines by combining basic unix commands.
UNIX commands are powerful; however by using pipe you can combine them together, to
accomplish complex tasks with ease.
Through the standard output of one command (the command to the left of the pipe) gets sent as
standard input to another command (the command to the right of the pipe). Pipe functions in a
similar manner like the output redirection in UNIX (using > symbol to redirect the standard output
of a command to a file. However, the pipe is different because it is used to pass the output of a
command to another command, not a file.
Example:
$ cat apple.txt | wc
3 4 21
In this example, the contents of the file apple.txt are sent through pipe to wc (word count)
command. The wc command then does its job and counts the lines, words, and characters in the file.
You can combine many commands with pipes on a single command line. Here's an example where
the characters, words, and lines of the file apple.txt is sent to wc and then the output of wc mailed to
[email protected] with the subject line "The count."
$ cat apple.txt | wc | mail -s "The count" [email protected]
awk
awk is a scripting language which is used for processing or analyzing text files.
awk is used for grouping of data based on either a column or field, or on a set of columns.
It derives its name from the first letter of the last name of its three authors namely Alfred V. Aho, Peter J.Weinberger
and Brian W. Kernighan.
awk can be used for reporting data in a useful manner. It searches one or more files to see if they contain lines that
match specified patterns and then perform associated actions. awk is an advanced filter.
Simple awk Filtering
Syntax of awk:
~$ awk 'pattern {action}' input-file

Let’s take a input file with the following data
~$cat awk_file
Name,Marks,Max_Marks
Peter,200,1000
Sam,500,1000
Greg,1000
Abharam,800,1000
Henry,600,1000
Peter,400,1000
Example: Default behavior of awk

Print all the lines from a file.
By default, awk prints all lines of a file, so to print every line of above created file , use below command:
~$ awk '{print}' awk_file
Name,Marks,Max_Marks
Peter,200,1000
Sam,500,1000
Greg,1000
Abharam,800,1000
Henry,600,1000
Peter,400,1000
Example 2: Print only specific field

Print 2nd & 3rd fileds
~$ awk -F”,” {print $2,$3;}' awk_file
Example: Pattern Matching

Print the lines which matches the pattern (lines which contains the word “Henry" or "Peter”)
~$ awk '/Henry|Peter/' awk_file
Peter,200,1000
Henry,600,1000
Peter,400,1000
Initialization and Final Action
BEGIN and END blocks are helpfull in displaying information before and after executing actual awk script.
BEGIN block is evaluated before awk starts processing the actual awk script; it’s an excellent place to initialize the
FS (field separator) variable, print a heading, or initialize other global variables.
BEGIN block Usages:
Declaring variables.
Initialization variables for doing increment/decrements operations in main AWK code.
Printing Headings/info before actual AWK code output.
END block is evaluated after all the lines in the awk script have been processed.
Typically, the END block is used to perform final calculations or print summaries that should appear at the end of the
output stream.
END block Usages:
Printing final results, after doing operations in main AWK block.

Printing Completion/info after actual AWK code output.
awk tool is mainly used for reporting some data in useful manner. Without these BEGIN and END
blocks the output will be meaningless.
Consider db.txt which contains below data:
Jones 2143 78 84 77
Gondrol 2321 56 58 45
RinRao 2122234 38 37
awk BEGIN block
This is a block of code which is executed before executing actual awk script.
BEGIN block Syntax
awk ‘BEGIN{awk initializing code}{actual AWK code}’ filename.txt
Example: Print a meaning full info before actual AWK output.
~$ awk ‘BEGIN{print “########################\nThis is the output of
filtered
data\n########################”}{print $0}’ db.txt
Output:
##########################
This is the output of filtered data
##########################
Jones 2143 78 84 77
Gondrol 2321 56 58 45
RinRao 2122234 38 37
Edwin 253734 87 97 95
Dayan 24155 30 47
awk END block
This is the block which is executed after executing all the awk code.
Example:
Print some meaning full info after processing awk code.

~$ awk ‘{print $0} END {print “#########################\n Completed
printing
filtered data\n########################”}’ db.txt
Output:
Jones 21 78 84 77
Gondrol 23 56 58 45
RinRao 25 21 38 37
Edwin 25 87 97 95
Dayan 24 55 30 47
#########################
Completed printing filtered data
#########################
Combining BEGIN and END block
Example:
~$ awk ‘BEGIN{print “##########################\n This is the output of filtered

data\n##########################”}{print $0}END{print
“########################\n Completed printing filtered
data\n########################”}’ db.txt
Output:
#########################
This is the output of filtered data
#########################
Jones 21 78 84 77
Gondrol 23 56 58 45
RinRao 25 21 38 37
Edwin 25 87 97 95
Dayan 24 55 30 47
########################
Completed printing filtered data
awk inbuilt variables
awk is supplied with good number of built-in variables which comes in handy when working with data files. We will
see usages of awk built-in variables with one or two examples . These variable are used to format the output of an
awk command.
List of built-in variables:
FS field separator character (default blank & tab)

OFS output field separator string (default blank)
RS input record separator character (default newline)
ORS output record separator string (default newline)
NF number of fields in input record
NR number of input record
FNR output number of lines
FILENAME name of current input file
Consider below db.txt as sample file.

~$ cat db.txt
John,29,MS,IBM,M,Married
Barbi,45,MD,JHH,F,Single
Mitch,33,BS,BofA,M,Single
Tim,39,Phd,DELL,M,Married
Lisa,22,BS,SmartDrive,F,Married
In order to make it simple we can divide above inbuilt variables in to groups on basis of their operations.
Group1: FS(input field separator), OFS(Output Field Separator)
Group2: RS(Row separator) and ORS(Output Record Separator)
Group3: NR, NF and FNR
Group4: FILENAME variable
FS (Input Field Separator)
This variable is useful in storing the input field separator. By default AWK can understand only spaces, tabs as input
and output separators. But if your file contains some other character as separator other than these mention one’s, awk
cannot understand them.
For example UNIX password file which contain ‘:’ as a separator. So in order to mention the input filed separator we
use this inbuilt variable. We will see what issue we face if we don’t mention the field separator for our db.txt.
Example: without using FS
Print first column data from db.txt file.

~$ awk ‘{print $1}’ db.txt
Output:
John,29,MS,IBM,M,Married
Barbi,45,MD,JHH,F,Single
Mitch,33,BS,BofA,M,Single
Tim,39,Phd,DELL,M,Married
Lisa,22,BS,SmartDrive,F,Married
OFS (Output Field Separator)

This variable is useful for defining the output field separator for the expected output data.
Example:
Display only 1st and 4th column and with $ as field separator for the output .
~$ awk ‘BEGIN{FS=”,”;OFS=” $ “}{print $1,$4}’ db.txt
Output:
John $ IBM
Barbi $ JHH
Mitch $ BofA
Tim $ DELL
Lisa $ SmartDrive
Note: Space is give before and after $ in OFS variable to show better output.
RS (Row separator)
Row Separator is helpful in defining separator between rows in a file. By default awk takes row separator as new line.
We can change this by using RS built-in variable.
Example:
Convert a sentence to a word per line. We can use RS variable for doing it.
~$ echo “This is how it works” | awk ‘BEGIN{RS=” ”}{print $0}’
Output:
This
is
how
it
Works
ORS (Output Record Separator)

This variable is useful for defining the record separator for the awk command output. By default ORS is set to new
line.
Example:
Print all the company names in single line which are in 4th column.
~$ awk -F’,’ ‘BEGIN{ORS=” “}{print $4}’ db.txt
Output:
IBM JHH BofA DELL SmartDrive
NF
This variable keeps information about total fields in a given row. The final
value of a row can be represented with $NF.
Example: Consider abc.txt which contains below data:

Jones 2143 78 84 77
Gondrol 2321 56 58 45
RinRao 2122234 38 37
Edwin 253734 87 97 95
Dayan 24155 30 47
Print number of fields in each row in abc.txt.
~$ awk ‘{print NF}’ abc.txt
Output:
5
5
4
5
4
NR
This variable keeps the value of present line number. This will come handy when you want to print line numbers in a
file.
Example:
Print line number for each line in a given file.
~$ awk ‘{print NR, $0}’ abc.txt
Output:
1 Jones 2143 78 84 77
2 Gondrol 2321 56 58 45
3 RinRao 2122234 38 37
4 Edwin 253734 87 97 95
5 Dayan 24155 30 47
This can be treated as cat command -n option for displaying line number for a file
FNR
This variable keeps count of number of lines present in a given file/data. This will come handy when
you want to print no of line present in a given file. This command is equivalent to wc -l command.
Example:
Print total number of lines in a given file.
~$ awk ‘END{print FNR}’ abc.txt
Output:
5
FILENAME
This variable contain file awk command is processing.
Example:
Print filename for each line in a given file.
~$ awk ‘{print FILENAME, NR, $0}’ abc.txt
Output:
abc.txt 1 Jones 2143 78 84 77
abc.txt 2 Gondrol 2321 56 58 45
abc.txt 3 RinRao 2122234 38 37
abc.txt 4 Edwin 253734 87 97 95
abc.txt 5 Dayan 24155 30 47
awk Built in Function
A function is a self-contained computation that accepts a number of arguments as input and returns some value. awk
has a number of built-in functions in two groups: arithmetic and string functions.
Arithmetic Functions
Nine of the built-in functions can be classified as arithmetic functions. Most of them take a numeric argument and
return a numeric value. Below table summarizes these arithmetic functions with some Examples.
awk Function Description

cos ( x ) Returns cosine of x (x is in radians).
exp ( x ) Returns e to the power x.
index (s1,s2) Position of string s2 in s1; returns 0 if not present
int ( x ) Returns truncated value of x.
log ( x ) Returns natural logarithm (base- e) of x.
sin ( x ) Returns sine of x (x is in radians)
sqrt ( x ) Returns square root of x.

atan2 ( y , x ) Returns arctangent of y / x in the range - to .
rand () Returns pseudo-random number r, where 0 <= r < 1.
sqrt(expr) Returns the square root of the expression or value given
Examples:
~$ awk 'BEGIN{
print sqrt(16);
print sqrt(0);
print sqrt(-12);
}'
Output:
4
0
nan
Here nan stands for not a valid number.
String Functions
The built-in string functions are much more significant and interesting than the numeric functions. Because awk is
essentially designed as a string-processing language, a lot of its power derives from these functions. Below table lists
the string functions found in awk. awk's Built-In String Functions
Shell Programming
The UNIX shell program interprets user commands, which are either directly entered by the user, or
which can be read from a file called the shell script or shell program. Shell scripts are interpreted,
not compiled. The shell reads commands from the script line by line and searches for those
commands on the system.
Shell script
Shell script is a file-containing list of commands to be executed in a particular order.
A good shell script will have comments, (a statement preceded by a pound sign, #) , describing the
purpose of the statement.
In a script we can use conditional tests, such as value A is greater than value B, loops or iterative
statements to execute some steps repetitively or to navigate through a list of data or records and do
processing. We can use files to read and store data. Can use variables to read and store data. A script
may include functions also.
When a script is executed, the shell reads the commands one by one and executes them .
We can create the simple shell script file by using vi editor or cat command like,
$ vi test.sh
$ cat > test.sh
Below mentioned shebang statement should be the first statement in the shell script as it tells the
system that the commands mentioned in the shell script are to be executed by the shell /bin/sh
#!/bin/sh
Consider the shell script with just two commands pwd & ls.
$cat test.sh
#!/bin/bash
pwd
ls
Importance of shell script
Shell scripts are basically used for automating processes that we repeat at the prompt .
Following are some activities we can do using shell scripts:
Automation of repetitive task
Creating our own power tools/utilities.
Automating command input or entry.
Customizing administrative tasks.
Creating simple applications.
Automating administration tasks such as adding new users, removing obsolete users etc
Some Practical examples where shell scripting can be actively used:
Monitoring your Linux system.
Data backup and creating snapshots.
Dumping Oracle or MySQL database for backup.
Creating email based alert system.
Find out what processes are eating up your system resources.
Find out available and free memory.
Find out all logged in users and what they are doing.
Find out if all necessary network services are running or not. For example if web server failed
then send an alert to system administrator via a pager or an email.
Find out all failed login attemp. If login attempt are continued repeatedly from same network
IP, automatically block all those IPs accessing your network/service via firewall.
User administration as per your own security policies.
Find out information about local or remote servers.
Configure server such as BIND (DNS server) to add zone entries.
Executing a shell script
Below two methods can be used to execute the shell script.
$ sh filename
Or
$ ./filename
In this case we have to modify the file access permissions of the shell script before execution.
To provide execute permission , following command is used.
$ chmod u+x filename

Basic Operators in shell scripting
Below operators are supported by shell.
Arithmetic Operators.
Relational Operators.
Boolean Operators.
String Operators.
File Test Operators.
Here is simple example to add two numbers:
Example:
#!/bin/sh
val=èxpr 2 + 2`
echo "Total value : $val"
Output:
$ Total value : 4
There are following points to note down:
There must be spaces between operators and expressions for example 2+2 is not correct,
where as it should be written as 2 + 2.
Complete expression should be enclosed between ``, called inverted commas to execute expr
command correctly.

Arithmetic Operators

Assume variable a holds 10 and variable b holds 20 then:
Operator Description Example

+ Addition èxpr $a + $b ` will give 30
- Substraction èxpr $a - $b ` will give -10

* Multiplication èxpr $a \* $b` will give 200
/ Division èxpr $b / $a` will give 2
% Modulus èxpr $a % $b` will give 0
!= Not equal [ $a != $b ] will give true
= assignment a=$b will assign value of b to a.
== Equality [ $a == $b ] will return false.
It is very important to note here that all the conditional expressions would be put inside square
braces with one spaces around them, for example [ $a == $b ] is correct where as [$a==$b] is
incorrect.

Relational Operators
Below are relational operators which are specific to numeric values. These operators would not
work for string values unless their value is numeric.
For example, following operators would work to check a relation between 10 and 20 as well as in
between "10" and "20" but not in between "ten" and "twenty".
Assume variable a holds 10 and variable b holds 20 then:

-eq Check if the values of 2 operands are equal or not, [ $a -eq $b ] is false
if yes then condition becomes true.
-ne Check if the values of 2 operands are equal or not, [ $a -eq $b ] is true
if values are not equal then condition becomes true.
-gt Check if the value of left operand is greater than [ $a -gt $b ] is false
the value of right operand, if yes then condition
becomes true.
-lt Check if the value of left operand is less than the [ $a -lt $b ] is true
value of right operand, if yes then condition
becomes true.
-ge Check if the value of left operand is greater than or [ $a -ge $b ] is false
equal to the value of right operand, if yes then
condition becomes true.
-le Check if the value of left operand is less than or [ $a -le $b ] is true
equal to the value of right operand, if yes then
It is very important to note here that all the conditional expressions would be put inside square
braces with one spaces around them, for example [ $a <= $b ] is correct where as [$a <= $b] is
incorrect.
Boolean Operators
Assume variable a holds 10 and variable b holds 20 then

! This is logical negation. This inverts a true [ !false ] is true
condition into false and vice versa.
-o This is logical OR. If one of the operands is [ $a -lt 20 -o $b -gt 100 ] is
true then condition would be true. true
-a This is logical AND. If both the operands [ $a -lt 20 -a $b -gt 100 ] is
are true then condition would be true false.
otherwise it would be false.
String Operators
These are string operators. Assume variable a holds "abc" and variable b holds "efg" then:

-z Check if the given string operand size is [ -z $a ] will return false.
zero. If it is zero length then it returns true.
-n Check if the given string operand size is [ -z $a ] will return true.
non- zero. If it is non-zero length then it
returns true.
= Check if the value of two operands is equal [ $a = $b ] will return false
or not, if yes then condition becomes true.
!= Check if the value of two operands is equal [ $a != $b ] will return true
or not, if the values are not equal then
str Check if the str is not the empty string. If it [ $a ] will return true
is empty then it returns false.
File Test Operators:
Assume a variable file holds an existing file name "test" whose size is 100 bytes and has read, write
and execute permission on:

-b file Returns true, if file is a block special file [ -b $file ] is false.
-c file Returns true, if file is a character special file [ -b $file ] is false.
-d file Returns true, Check if file is a directory [ -d $file ] is not true.
-f file Returns true, Check if file is an ordinary file [ -f $file ] is true.
or special file
-r file Returns true, Checks if file is readable [ -r $file ] is true.
-w file Returns true, Check if file is writable [ -w $file ] is true.
-x file Returns true, Check if file is execute [ -x $file ] is true.
-s file Returns true, Check if file has size greater [ -s $file ] is true.
than 0
-e file Returns true, Check if file exists [ -e $file ] is true.
Wild Card Characters
Symbol used to replace or represent one or more characters. Wildcards or wild characters are either
asterisk (*), which represent one or more characters or question mark (?), which represent a single
character.
Wild card
Meaning Examples
/Shorthand
Matches any string or group of $ ls * will show all files
*
characters.
$ ls a* will show all files whose first
name is starting with letter 'a'
$ ls *.c will show all files having
extension .c
$ ls ut*.c Will show all files having
extension .c but file name must
begin with 'ut'.
? Matches any single character. $ ls ? will show all files whose names
are 1 character long
$ ls fo? will show all files whose names
are 3 character long and file
name begin with fo
[...] Matches any one of the enclosed $ ls Will show all files beginning
characters [abc]* with letters a,b,c
Note: [..-..] A pair of characters separated by a minus sign denotes a range.
Example:
$ ls /bin/[a-c]*
Will show all File name beginning with letter a,b or c.
Output:
/bin/arch /bin/awk /bin/bsh /bin/chmod /bin/cp /bin/as /bin/basename /bin/cat
Shell Quoting Mechanism
The Metacharacters
Unix Shell provides various metacharacters which have special meaning while using them in any
Shell Script and causes termination of a word unless quoted.
Example:
? Matches with a single character while listing files in a directory and an * would match more than
one characters.
Here is a list of most of the shell special characters (also called metacharacters):
* ? [ ] ' " \ $ ; & ( ) | ^ < > new-line space tab
A character may be quoted (i.e., made to stand for itself) by preceding it with a \.
Example:
#!/bin/sh echo Hello; Word
This would produce following result.
Hello ./test.sh: line 2: Word: command not found shell returned 127
Now let us try using a quoted character:
#!/bin/sh echo Hello\; Word
This would produce following result:
Hello; Word
The $ sign is one of the metacharacters, so it must be quoted to avoid special handling by the shell:
#!/bin/sh echo "I have \$1200"
I have $1200
Quote Description
Single quote All special characters between these quotes lose their special meaning.
Double quote Most special characters between these quotes lose their special meaning
with these exceptions:
\$
\'
\"
\\
Backslash Any character immediately following the backslash loses its special
meaning.
Back Quote Anything in between back quotes would be treated as a command and
would be executed.
The Single Quotes
Consider an echo command that contains many special shell characters:
echo <-$1500.**>; (update?) [y|n]
Putting a backslash in front of each special character is tedious and makes the line difficult to read:
echo \<-\$1500.\*\*\>\; $update\?$ \[y\|n\]
There is an easy way to quote a large group of characters. Put a single quote ( ') at the beginning and
at the end of the string:
echo '<-$1500.**>; (update?) [y|n]'
Any characters within single quotes are quoted just as if a backslash is in front of each character. So
now this echo command displays properly.
If a single quote appears within a string to be output, you should not put the whole string within
single quotes instead you would precede that using a backslash (\) as follows:
echo 'It\'s Shell Programming
The Double Quotes:
Try to execute the following shell script. This shell script makes use of single quote:
VAR=ZARA
echo '$VAR owes <-$1500.**>; [ as of (`date +%m/%d`) ]'

$VAR owes <-$1500.**>; [ as of (`date +%m/%d`) ]
So this is not what you wanted to display. It is obvious that single quotes prevent variable
substitution. If you want to substitute variable values and to make invert commas work as expected
then you would need to put your commands in double quotes as follows:
VAR=ZARA
echo "$VAR owes <-\$1500.**>; [ as of (`date +%m/%d`) ]"
Now this would produce following result:
ZARA owes <-$1500.**>; [ as of (07/02) ]
Double quotes take away the special meaning of all characters except the following:
$ for parameter substitution.
Backquotes for command substitution.
\$ to enable literal dollar signs.
\` to enable literal backquotes.
\" to enable embedded double quotes.
\\ to enable embedded backslashes.
All other \ characters are literal (not special).
Back Quotes: Command Substitution
Putting any Shell command in between back quotes would execute the command
Syntax: var=`command`
Example:
Following would execute date command and produced result would be stored in DATA variable.
DATE=`date` echo "Current Date: $DATE"
Current Date: Thu Jul
Shell Decision Statement
While writing a shell script, there may be situations when you need to adopt one path out of many
available paths. In such cases you need to make use of conditional statements that allow your
program to make correct decisions and perform right actions.
Unix Shell supports conditional statements, which are used to perform different actions based on
different conditions.
Here we will explain following two decision-making statements

The if...else statement
The case...esac statement
If..else statements
We can use “if..else” statement which can be used as decision making statement to select an option
from a given set of options.
Unix Shell supports following forms of if..else statement:
if...fi statement
if...else...fi statement
if...elif...else...fi statement
Syntax:
if...fi statement
if [condition]
then
command(s)
fi
if...else...fi statement
if [ condition(s) ] then
command(s)
else
command(s)
fi
if...elif...else...fi statement
if [ condition(s) ]
then
command(s)
elif [ condition(s) ] # Many elif-then allowed
then
command(s)
else
command(s)
fi
test command in if condition
We can use test command as condition of if condition as used in the below script.
$cat if_test.sh

#!/bin/sh
echo “Do you want to quit (Y/y)? ”
read ans
if test $ans ==’y’ –o $ans==’Y’
then
exit
else
echo “ to exit enter N or n”
fi
case...esac Statement
We can use multiple if...elif statements to perform a multiway branch. However, this is not always
the best solution, especially when all of the branches depend on the value of a single variable.
Unix Shell supports case...esac statement which handles exactly this situation, and it does so more
efficiently than repeated if...elif statements.
The interpreter checks each case against the value of the expression until it finds a match. If nothing
matches, goes with the default condition.
Syntax:
case word in pattern1)
Statement(s) to be executed if pattern1 matches
;;
pattern2)
;;
pattern3)
;;
esac
Here string word is compared against every pattern until a match is found and the statement(s)
following the match pattern executes. If shell cannot find any match, the case statement exits
without performing any action. When statement(s) part is executed. The command ;; indicates
program flow should jump to the end of the entire case statement.
There is no maximum number of patterns, but the minimum is one.
Example:
#!/bin/sh
COURSE=”DB”
case “$COURSE” in
“Java”) echo “Java is a programming language”
;;
“Perl”)echo “Perl is scripting language”
;;
“DB”)echo “Oracle is a DB”
;;
esac

Output:
Oracle is a DB
Iterative Statements/Loop :
Loops are a powerful programming tool that enables you to execute a set of commands repeatedly.
while loop
for loop
until loop
while loop
Here condition is evaluated by shell at the beginning of each iteration. If the resulting value is true,
given command(s) are executed. If condition is false then no command would be executed and
program would jump to the next line after done statement.
Syntax:
while condition
do # executed as long as the condition is true
command(s)
done
Example:
a=0
while [ $a -lt 3 ]
do
echo $a
a=èxpr $a + 1`
done
Output:
until loop
Here condition is evaluated by shell. If the resulting value is false, given command(s) are executed.
If condition is true then no command would be executed and program would jump to the next line
after done statement.
Syntax:
until condition # complement of while
do
# executes the body as long as the condition is false
command(s)
done
Example:
a=0
until [ ! $a -lt 3 ] do
echo $a a=èxpr $a + 1`
done
Output:
for loop
For loop operate on lists of items. It repeats a set of commands for every item in a list.
Syntax:
for var in w1 w2 w3.. wn
do
command #Statement(s) to be executed for every w in the list
done
Example:
for var in 0 1 2 3 4 5
do
echo $var
done
Output:
5

Example:
for filename in `ls tempdir`
do
echo “Displaying contents of ${filename}”
cat ${filename}
done
We can use "break" and "continue" commands to control the loop.
Command "break" causes the termination of a loop and “continue” resumes execution at its top.
String Handling
String handling with test command:

test str Returns true if str is not null

test –n str Returns true if length of str is greater
than zero
test –z str Returns true if length of str is equal to
zero
String handling with expr command
The expr is quite handy for finding the length of a string and extracting a sub-string:
Length of the string:
$ str=”abcdefghijk” ;
$ n=èxpr "$str" : ‘.*’` ;
$ echo $n
11
expr gave how many times any character (.*) occurs. This feature is very useful in validating data
entry.
Extracting a sub-string:
$ expr “$str” : ‘……$..$’
gh
Note that there are 6 dots preceding the sequence $..$. This advanced regular expression signifies
that the first six characters of the string are to be ignored and extraction should start from the 7th
character. Two dots inside $..$ suggests that this extraction is limited to two characters only
(backslashes override the usual interpretation of ‘()’).
Extracting string from 3rd character to end of the string:
$ str="abcdefghijk"
$ expr "$str" : '..$.*$'
cdefghijk
Location of first occurrence of a character “d” inside string:
$ expr "$str" : '[^d]*d‘
Location of last occurrence if a character inside string:
Below will give the last occurrence of character 'a' from string str.
$str=”abc def abc”
$expr "$str" : '[û]*a'

9.

6. Command Line Arguments
To make a shell script a generalized script, we should avoid hard coding. User should be able to
provide the values required for processing as input while running the shell script. To facilitate this
we have to use command line arguments.
The statement we write in the command prompt starting with the command followed by list of
arguments is the command line argument.
Example:
$ ls dir1 dir2 dir3

A set of shell variables are there to represent the command line arguments.
$0 – represents the command name (first word on command line). For above example “ls”
$1 - the first argument of command (second word/symbol ).For the above example dir1
$2 – the second argument to the command. For the above example, dir2
In this way we can use up to $9
The command-line arguments $1, $2, $3...$9 are also called positional parameters, with $0 pointing
to the actual command/program/shell script and $1, $2, $3, ...$9 as the arguments to the command.
Consider below sample shell script “test.sh”
$cat test.sh

#!/bin/sh
echo “Program name is $0”

echo “First argument is $1”
echo “Number of arguments passed =$#”
echo “The arguments are $@”
Let’s execute the above shell script “test.sh”. Remember to change File access permission of script
to execute
$ sh test.sh arg1 arg2 arg3
Program name is test
First argument is arg1
Second argument is arg2
Number of arguments passed = 3
The arguments are arg1 arg2 arg3
There’s another way to execute the script using ./
$ ./test.sh arg1 arg2 arg3
Program name is test
First argument is arg1
Second argument is arg2
Number of arguments passed = 3
The arguments are arg1 arg2 arg3
User Input: read
To get input from the keyboard, you use the read command. The read command takes input from the
keyboard and assigns it to a variable.
Example: read.sh
#!/bin/sh
echo –n “Enter some text > “
read text
echo “You entered: $text”
Note that “-n” given to the echo command causes it to keep the cursor on the same line; i.e., it does
not output a carriage return at the end of the prompt.
Next, we invoke the read command with “text” as its argument. What this does is wait for the user
ro type something followed by a carriage return (the Enter key) and then assign whatever was typed
to the variable text.
Execution:
$sh read.sh
Enter some text > This is some text
You entered: This is some text
Validation of Command Line Arguments
Let us assume, we have one shell script which requires exactly 2 arguments to execute. Shell script
should throw proper error message in case user has not given exactly two arguments.
Let us have a small shell script "test.sh" to show how to implement validations.

#!/bin/sh
if [ $# -ne 2 ]
then
echo " Not sufficient arguments, please give exact 2 arguments"
else
echo "Continue with our task" fi

$ ./test.sh
Not sufficient arguments, please give exact 2 arguments
$ sh test.sh abc
$ ./test.sh abc def ghi
$ ./test.sh abc def 2
$ ./test.sh abc def
Continue with our task
For scripts where input expected is a file , validations can be incorporated for file tests like whether
the file given as input exists or not, whether it is empty or not, whether it is readable or not and so
on as per requirements.
Environment Variables:
An important concept in Unix is the environment, which is defined by values stored in a set of

predefined variables called environment variables.
Values of some of these variables are set by the system, some can be set by the user, some of them
by the shell, or any program that loads another program.
When a user logs in to the system, the shell undergoes a twostep process called initialization to set
up the environment. This process involves the shell reading two files −/etc/profile and .profile in the
home directory of the user.
The file /etc/profile is maintained by the system administrator of the Unix machine and contains
shell initialization information required by all users on a system.
The file .profile is under the control of the user. User can add as much shell customization
information as he/she wants to this file. The minimum set of information that a user need to
configure includes −
The type of terminal user is using.
A list of directories in which to locate the commands.
A list of variables affecting the look and feel of the terminal.
Environment around the UNIX Process:
When a process is created, UNIX opens 3 streams stdin/stdout/stderr for basic communication with
respect to the process control terminal. In addition, it knows the current working directory for
performing file I/O.
Each login shell maintains a description of the environment as a table of pointers to strings.
A global shell environment pointer called environ is maintained by UNIX kernel and it can be used
by a process to access its own table
The shell does not directly use this table, but it creates a child process and calls exec() system call to
execute a command program that uses the table inherited from the shell parent. Children inherit the
entire execution environment from the parent.
Some examples of the environment variables are the USER, LOGNAME, HOME, PATH, PS1,
PS2, TERM MAIL, etc.

The HOME Variable
It specifies an associated directory with every user in a UNIX system. If the HOME variable for the
user Sita contains /usr/sita/stores, every time Sita logs in, she is taken to the directory stores.
The variable HOME is referenced the same way:
$ echo ${HOME}

The PATH Variable
contains a list of all full path-names (separated by a colon) of directories that are to be searched for
an executable program. For example, the command
$PATH=.:/usr/bin:/bin specifies directories to be searched for any executable file or a command file
(current directory, /usr/bin and /bin, in that order).

The PS1 Variable
The system prompt may be changed by setting the value of this variable to the desired prompt:
$ PS1=“Hello>”
Hello> #can be changed only at the UNIX command line, not within a shell script.

The PS2 Variable: prompt string for continued command line (default ‘> ‘).

The LOGNAME Variable
contains user’s login name. Its contents cannot be changed by the user, but can be displayed:
echo “${LOGNAME}”

The TERM Variable
Names the kind of terminal you are using; setting it helps to manage your screen more effectively,
say,
$ TERM=vt100

The PWD Variable
The current working directory can be displayed:
echo “${PWD}”
In fact the whole environment table can be displayed.

$IFS: String of characters which are used as word separators in command line ( space, tab, newline
chars).

The MAIL Variable
Names the standard file where your mail is kept

The .profile File
Some of above variables like HOME and LOGNAME are set automatically each time the user logs
in. The others, however, have to be set. The .profile is used for this purpose as it is executed as soon
the user logs in. A sample .profile file would look like:
PATH=.:/bin:/usr/bin
export HOME PATH PS1 MAIL
The UNIX Process:
A process in UNIX is a program in execution with definite life-time and well-defined hierarchy.
The context of a process is a snapshot of its current run-time environment that consists of the current state
of the processor registers and
User program image - Process execution on user data structure that represents an application and is
manipulated by its own functions (user mode execution).
System image - Process execution on system’s data structures that represent the resources (memory, files,
devices) and supported by the kernel routines. Depending upon resource requirement and availability,
process’s states are controlled by executing the kernel routines accessed through system calls (system
mode execution).
The kernel maintains a process table to manage all processes. The two data structures per process are the
user structure and the process structure.
The kernel process is the first (root) process that comes into existence when the system is booted. Its
process_id and group_id are both 0.
In establishing a multi-programming environment, the kernel process creates the init process with its
process_id of 1 and group_id, 0, showing that process 1 is part of process 0. The init process creates and
manages terminal processes for active terminals in the system.
At the time of creation, each terminal process belongs to process group 0 and is made to execute a
program called getty. Once each terminal process (now called a getty process) establishes communication
with the associated terminal, it displays a login message and waits for a user to input a user name (Process
group of the init process).
When a user starts a dialogue, the getty process receives the user name and leaves the task of validating
the user’s password to another program called login. The same terminal process is now called a login
process.
The login process, after validating the user’s password, calls a command line program, the login shell to
run in the same process. The same terminal process is now called a shell process.
Each shell process now establishes a new process group and becomes ready to process the user
commands. A shell process is the initiating process by which each terminal maintains the user session.
While interpreting a command, the shell creates an execution thread and then assigns the requested
command program to this new process.
Both the shell and the new process proceed independently in separate execution threads. The parent shell
process normally waits until child process completes its execution.
A process require certain resources to accomplish its intended tasks. They are
>> CPU time
>> System memory
>> Disk Files
>> I/O devices.
Process control blocks (PCB)

The OS must know all the information about a specific process in order to manage and control it. The OS
maintains a table (an array of structures), called the process table, with one entry per process. These entries are
called process control blocks (PCB) - also known as task control block.This entry contains information about the
process' state, its program counter, stack pointer,memory allocation, the status of its open files, its accounting
and scheduling information, and everything else about the process that must be saved when the process is
switched from ready to running or blocked state so that it can be restarted later with the status it left off.
Process state:- The state may be new, ready, running, waiting, halted, and so on.
Program counter:- The counter indicates the address of the next instruction to be
executed for this process.
CPU registers:- The registers vary in number and type, depending on the computer
architecture. They include accumulators, index registers, stack pointers, and general-purpose registers, plus any
condition-code information.
CPU-scheduling information:- This information includes a process priority, pointers to scheduling queues, and
any other scheduling parameters.
Memory-management information:- This information may include such information as the value of the base
and limit registers, the page tables, or the segment tables, depending on the memory system used by the OS.
Accounting information:- This information includes the amount of CPU and real time used, time limits,
account numbers, job or process numbers, and so on.
I/O status information:- This information includes the list of I/O devices allocated to the process, a list of open
files, and so on.
Process state:
A process may present in one of the following state.
New:- The process is being created or just created.
Ready:- The process is waiting to be assigned/allocated to a processor time.
Running:- Instructions of the process are being executed by processor.
Waiting:- The process is waiting for some other event/process to execute.
Terminated:- The process has finished its execution.
ps command:
The ps (i.e., process status) command is used to provide information about the currently running processes in the
system.
When ps is used without any option four columns of information labeled PID,TTY, TIME and CMD for at least
two processes, the shell and ps will be visible.
PID :- The process are identified by a 5 digit number known as PID (Process Identification Number).
TTY :- Is the name of the console or terminal that the user logged into (Stands for terminal type now
but originally stood for teletype).
TIME :- Is the amount of CPU time in minutes and seconds that the process has been running.
CMD :- is the name of the command that launched the process.
The information that ps -aux command provides about each process are :
The user of the process,

PID of the process,
Percentage of CPU used by the process,
Percentage of memory used by the process,
VSZ (virtual size in kilobytes),
RSS (real memory size or resident set size in 1024 byte units),
STAT (the process state code, explained later),
Starting time of the process,
Length of time the process has been active
The command that initiated the process.
Types of process:
Parent and Child Process

A process can initiate a sub process, which is a called a child process, the initiating process is referred to as its
parent.The child processes, in turn create other child processes forming a tree of processes
( which can be displayed using ps command with –forest option)
Orphan Process
When a child process is killed, parent process gets the notification via a signal. Parent then, can continue other
task. However if the parent process is killed before, its child, is called an orphan process.
Zombie Process
When a process finished its execution and exit status not received by the parent ( or parent did not read the exit
status till now), the process state becomes zombie.
The process is dead (not to be scheduled for further execution) but cannot becompletely removed from process
table, until it has been determined that exit status is no longer needed.
Daemon Process
Some programs are not designed to be run with continuous user input and disconnect from the terminal when
task completed. For example, a web server responds to web requests, rather than user input. Mail servers are
another example of this type of application. These types of programs are known as daemons.
Starting a Process:
Process can be started in two ways:
In Foreground: By default every process starts in foreground, ie. Gets the input from keyboard and sends
the output in monitor. But in this case till the process completes its execution no other process can be
started in foreground.
In Background: To take the advantage multiprocessing environment, a process can be started in

background, so that other process can be started in the foreground without waiting for the previous process
to complete execution.
A process can be started in background by adding ampersand(&) after it.
[390119 @ INGNRILPORCL] $ $ ls | wc -l > file1 &
Switching process from foreground to background:
A process running in foreground can be send to background using the following steps:
Press + Z to suspend the job

bg command puts the job in background
nohup unattaches the job from the terminal*
$ cat > file

File content...
+z
[1] + Stopped
$ bg
[1] + cat > file &
Switching process from background to foreground
Process running in the background can be taken into foreground using the following steps:
>> Find the job id of the process by the command jobs
>> Use fg %to get the job to foreground
Stopping/Killing a Process
A process dies(terminates ) automatically when it completes the job it intended to.

A process can be killed abnormally with the command kill.
Use the ps command to find out the process-id for the process
Use the command kill to terminate it
Use the command kill -9 to kill a process forcefully.
Option settings for background process
While a process is executing/running , if the owner tries to log off the process ,process will get killed.
Sometimes a job or command takes a long time to complete and it is required the job to be completed without
interruption. This situation can be handled in two different ways:
Does not allow the use to log off

Continue execution in background even after the user logged off
This can be achieved using command nohup
PERIODICAL EXECUTION OF JOBS
Sometimes it may be required to execute certain Job or task on specific time. It may be not possible for any user to start or
execute the job physically on that particular time due to other business requirements. Again if similar task need to be executed
periodically on a regular interval, then the task becomes very hectic. Unix internally has solutions to handle this type of
situations using utilities such as cron and at.
Crontab
The word 'crontab' in UNIX stand for chron(chronograph) table, or time table for Unix system .
The crontab is a special table where it is possible to specify commands and time or interval to execute the command once or
repeatedly. Commands can be any executable programs, for example, a script can be written to take regular backup of all the
required tables in database. A crontab instruction can be created to execute the script on the specified time. The cron daemon
reads the table and executes the commands at the times specified.
Crontab Command Syntax
Syntax
Components of Crontab
The basic format of a crontab schedule consists of 6 fields, placed on a single line and separated by spaces, formatted as
follows:
The various fields and their utility are explained in the image below:
Example:
To executes an incremental backup shell script "takeBackup.sh" available in your home directory at 11:00 on every day the
cron entry will be as below:
00 11 * * * ~/takeBackup.sh
To schedule the same job for every minute, the cron entry will be as follows:
* * * * * ~/takeBackup.sh

Ilovepdf Merged

Uploaded by

Copyright:

Available Formats

Ilovepdf Merged

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ilovepdf Merged

Uploaded by

Copyright:

Available Formats

8/10/2021 iON| Digital Learning

Single User and Single Tasking OS:

Single User and Multitasking OS:

Real Time OS:

Unix is an OS for Programmers as shell(the command interpreter)provides the programming facility.

Features of Unix Operating System

• Multi-user and Multitasking

Architecture of the UNIX System

The Unix Kernel

System calls can be categorized as:

The Unix File System:

Unix File System is a hierarchical collection of 3 types of files:

Characteristics of Unix Files:

UNIX file is featureless because it is simply an array of bytes.

There are two types of path names in unix.

Structure of Unix File System:

Following are some directories available under root in UFS.

/usr – It holds many user-oriented directories:

/sbin – It holds system files that are usually run automatically.

/etc – It and its subdirectories hold many of Linux config files.

Unix file System Organization:

d) data blocks : blocks containing the actual contents of files

Internal File Maintenance:

Sample details of an inode –

Owner and group identifiers

List of access rights – read/write/execute permissions

Users and Access Rights

User Types in Unix

There are three types of accounts in a Unix system:

Managing users and groups

Unix supports a concept of Group which logically groups a number of accounts.

File Permission & Ownership level

File Permission Types (Mode)

Changing Permissions (chmod)

Using chmod in Symbolic Mode:

In the above example,

Using chmod in Absolute(octal) Mode

1. Let us view the initial permissions of the file abc

-rw-r--r-- 1 735873 oinstall 0 Feb 7 12:37 abc

use the below command

Let us verify the changed permissions

4. chmod 755 abc

5. chmod 777 abc

Basic UNIX commands :

It displays a line text.

Echo the given string or strings to standard output.

echo "Good Morning"

This will display the following in the standard output

Syntax : passwd [options] [LOGIN]

Print certain system information.

Syntax : uname [OPTION]...

Examples: To show the name of the OS

To get the kernel name, you can use -s parameter.

To get the kernel release, -r option is used

commands) on searched files.

2. Wild-Card based Search : Wild-card characters like * and ? can be used:

[390119@InGhyUnix ~]$ find . -name "Abc*"

3 . Source Location based Search:

[390119@InGhyUnix ~]$ find Dir1 Dir2 -name "Abc.*"

[390119@InGhyUnix ~]$ find . -size +10M