Work On the Command Line
Text searching
*- match 0 or any in filenames
?- Match single character in file name
[abcde] – Match any listed characters
[a-g] – Match range of characters
[!abcde] – Match any characters not listed
[!a-e] – Match any characters non in range
[cat,dog,car] – Match any word in list
[$] – lis file names ending preceding the $ character
Ex: ll xfs*
ll xfs[a-e]
Ex : count lines ll -l wc
Tab used to autocompletion.
Meta characters and control characters.
|| – Logical OR
&& – Logical &&
; – separate multiple command at single line
EX:
cd /usr/share; ll; df -h;
Can divide commands to multiple lines using \
ll /usr/share\
>\xml
is equal to
ll /usr/share/xml
Single Shell Command and one line command sequences
To print all environment varriables
env
To print specific environment varriable
echo $PATH
echo $UID
Get system information
uname -a
Get current user IDs
id
-i for ignore case
cat /var/log/messages | grep -i error
Get error counts
cat /var/log/messages | grep -i error | wc -l
Print Working Directory
pwd
Using and Modifying Environment Varriables
Print current user
echo $USER
Print current shell type
echo $SHELL
To go to home directory
cd ~
Have to set and export the varibale to be used in shell
TEST=Hi; export TEST
echo $TEST
Hi
Use and Edit command HIistory, and PATH
Show last 15 commands
history 15
Show last command
!!
Shows most recent command starting with string
!string
Show most recent command containing string
!?string?
Defining a PATH
PATH=$PATH:/my/folder; export PATH
Process Text Streams Using Filters
Send output of one command as an input to another command
ll | sort
ll | grep file1
Redirect out put, typically fo a file
ll | sort > out.txt
Note : single > create a new file, >> appends to existing data
Specify input
<
Output input stream or file to stdout
Eg: to check lines in a file
cat files.txt | wc -l
Display files in octal format
od
Dividing large files
Eg: Split file by line numbers
split -l 200 files.txt
Eg: Split based on chunk numbers
split -n 5 files.txt
Eg: Split based on file size
split -b 100M files.txt
Counts words of lines of the input stream
wc
Eg: count words in file
cat files.txt | wc -w
see man wc for more options
Display top or bottom of the file
head or tail
Convert spaces into tabs ot the reverse
expand unexpand
Change/convert case of character
Eg: ABCDEFGHIJKLMN in file texts.txt
cat texts.txt | tr “[A-G]” “[a-g]”
output : abcdefgHIJKLMN
Formats data for printing
pr file.txt|more
• The number lines (nl) command assigns a number to each line of output
Eg: Add lines to a file and create new file with numbers
nl files.txt > line_files.txt
• Text files can be reformatted to a specified width using the fmt command
Eg: Format contents of a file in the width of 40 characters
fmt -w 40 files.txt
• Numerical or lexigraphical sorting can be performed with the sort command
sort -n files_with_numbers_at_leading.txt
• The uniq command will remove duplicate entries from its input
to avoid duplicates on above -nu
• Filtering out specific columns from a file can be done with the cut command
Eg: to view specific(1, 3) columns of a comma separated or other delimiter files
cat files.txt | cut -d “,” -f 1,3
• Lines from multiple files can be joined with the paste command
merge files based on lines side by side
paste file1.txt file2.txt
• Multiple files can be assembled with the join command
Join command joins common fields in the output
join file1.txt file2.txt
join command asks for files to be sorted before joining. by tasing ‘no check’ could skip this
• The Stream EDitor (sed) uses regular expressions for modifying text
replace “this” with “that”
cat files.txt | sed -e “s/this/that/”
multiple replacement on same line also can be done
• The diff command allows comparison of two files
diff file2.txt file2.txt
Perform Basic File Management
Make directory/ parent directories as needed
mkdir -p /dir1/dir2/dir3
Create an empty file
touch file1
> file1
Copy file to my current directory
cp /tmp/mydir .
Note: ls -lrt shows more recent file at the bottom
Recursively Copy File/Direcory
cp -R /tmp/mydir /home/
Copy all directories(starts with ‘dir’) and contents into a directory named ‘pack’
cp -R dir* pack
Move and optionally rename files. -t for target directory
mv /tmp/mydir -t /home/hisdir
Remove files with rm
rm -rf /tmp/mydir
to remove parent directory
rmdir -p /tmp/mydir
to remove all subdirectories and files without prompting
rm -drf dir1
Use simple and advances wildcard specifications in commands
List files and directories with some charecters in name
ls /etc/pa*s*d
Negations
ls /tmp/*[!r].ser*
touch command can update a timestamp or create a new file
touch filename
touch -d 9am filename
touch -d “2 days ago 17:30” filename
touch -d “15 Jan” filename
The file command identifies the type of a file
file filename
file /sbin/*
find command helps to locate and act on the files base on type, size or time
Eg; find all configuration file
find / -name *.conf
Eg: find files begins with “system” in directory /tmp
find /tmp -name “system*”
Eg: Find regular files within /tmp
find /tmp -type f
Eg: Find directories within /tmp
find /tmp -type d
Eg; Find deepest files and folders in hierarchy
find /usr -depth
Eg: Find files greater than 100MB
find /usr -size +100M
Eg; find files accessed within last 3 days
find /usr -atime 3
Eg: find file in /usr those are owned by the user root
find /usr -user root
Eg: find file in /usr those used within last 3 days, and long list with -lrt
find /usr -atime 3 -exec ls -lrt “{}” \;
Eg: find recursively in current directory
find . -name “*.gz”
File Compression and Decompression
Using gzip to compress files and gunzip to decompress
Eg
gzip /tmp/myfile*
Better compression
gzip -9 /home/user/data.gz
Faster compressiont
gzip -1 /home/user/data.gz
View contents of a gz
gzip -l data.gz
Unzip
gunzip /home/user/data.gz
The bzip2 utility operates the same as gzip and gunzip. but uses different compression algorithm.
Zip
bzip2 Events_admin_2019-07-06_scnd.csv
Unzip
bzip2 -d Events_admin_2019-07-06_scnd.bz2
The xz utiliy is also available for compressing or decompressing files
xz file.txt
unxz file.txt
Usage of TAR, CPIO(Copy in and out) and DD
TAR
Archieve in TAR
Eg: Add all files begin with TLD to a tar
tar -cvf mytar.tar TLP*
c – create
v – verbose
f – flename
Inspect tar
tar -tvf mytar.tar
-t list
Extract
tar -xvf mytar.tar
CPIO
Find files stat with TLP and add to archieve
find . -name “TLP*” | cpio -ov > mycp.cpio
Re put files thos archieved earlier
cpio -iv < mycp.cpio
Eg; find everything start eith TLP and move to a new dir
find . -name “TLP*” | cpio -pvd ./test
-p pass through
-d directory
dd
convert and copy a file
dd if=/dev/zero of=zero.txt bs=10M count=10
if – input file
of – output file
bs – byte size
Read from from one device to another
dd if=/dev/sda1 of=/media/usb -bs=10M count=10
Use the output of one command as the input to another command
ls | xargs echo My Files:
ls | xargs ls -lia
Eg: find all text files and delete
find /tmp -name *.txt | xargs rm -f
Send output to both stdout and files
Eg; Redirect Error to a different file
ls /sbin/* > sbin.txt 2> sbin.err
Eg: translate output capitalize output of a file
tr ‘a-z’ ‘A-Z’ < sbin.txt
Tee command read from stdin and writes to stdout and files.
netstat -an | tee netstat.txt
ls /tmp | tee tmp1.txt tmp1.txt
Create Monitor and kill process
To run job at background, add & at the end of the command
sleep 100&
sleep 200&
sleep 500&
To view jobs running in background
jobs -l
Jobs are belongs to particular instance of a terminal.
When a bg job brought into foreground, it’s priority increases
fg 1
Ctrl + z stops the job
to re run a stoped job
bg 1
To run program ofter logout use nuhup command.
outputs saved to nohup.out by default
Eg: run a program after logout and save the errors to a different file
nohup myprog.sh> ~/meerr.err &
1> standard output
2> standard error
Monitor Active Processes
ps command is the direct command
ps -ef gives full programs being run and executed commands
Processes used by user root
ps -u root
process used by the group admin
ps -g admin
process used specific ports
ps -p 443 3433
find processes by name
ps -ef | grep systemd
search for the process id of a process
pidof systemd
search by name
pgrep sleep
List Column Names
UID PID PPID
Show each process related to bash
ps -ef | grep -i bash
top command give interact-ability
z – color change
b – bold high priority processes
c – order by cpu usage
m – memory
u -user
shft + l – highlight specific strings in process list
k – kill a process
r – renice a process
e – shows more usages
Kill Process
kill job number
kill %1
kill pid
kill 9939
kill with signal
kill -9 9939
kill -SIGTERM 9939
Modify Process Execution Priorities
Know the default priority of the job created
Run program with higher or lower priority than the default priority
Change the priority of the running process
Default priority for the most process is 20
ps -el shows priority in 8th column
run ‘nice’ command to check the current value
nice
nice -n -10 sleep 600&
nice -n 10 sleep 600&
top
shift + l to filter (enter sleep)
r to modify priorities
renice
renice 10 [pid]
Search Text Files Using Regular Expressions
grep
grep error /var/log/messages
— search for word error in messages log
grep -v error /var/log/messages
— invert above search. other than word ‘error’
grep -i error /var/log/messages
— ignore case
Regex Search
grep ^root /etc/passwd
— search for the word ‘root’ at the beginning of the line
egrep ‘^[a-d]’ /etc/passwd
— search the letter range a to be, at the beginning of the line
grep error –color /
— highlight searches
egrep ‘[0-9]{4}’ /var/log/messages
— find numbers with 4 digits
grep -F ‘nologin’ /etc/passwd
— search for fixed characters
grep -F -f users.txt passwd.txt
— searches words in users.txt exists in passwd.txt
Perform Basic File Editing Operations Using VI
gg – top of the file
VI Modes
Command Mode
— used to navigate the file
— search the file
— Replace
— Delete
— Copy and paste
— Send commands to execute mode
Editing Mode
Data is typed into editor
d twice to delete a line
5dd — to detele 5 lines
Shift + — Open a line above to insert
yy – copy
p – put/paste
5yy – copy five lines and ‘p’ to put/paste
Shift + a — insert mode at the end of the line
Shift + i — insert mode at the beginning of the line
