How to use grep command in Linux and Unix

The grep command is one of the most essential commands in Unix/Linux ecosystem, It is used to extract match and find the occurrence of string literals and patterns among the text files.

In this section we will focus on the important scenarios and provide the realtime explanation about the Various Grep command options offered in linux command line.

We now have a text file with Following content.

Below is the content of the file called intro.txt.

[vamshi@linuxcent grep]$ cat intro.txt
the first line is in lower case
THE SECOND LINE IS IN UPPER CASE


There following are Most popular Linux server Distributions:
1) Redhat Enterprise Linux
2) Ubuntu
3) Centos
4) Debian
5) SUSE Linux Enterprise Server
This is a Demonstration of Linux grep command

Let’s start with search of a string pattern in the given file

The grep command sample search syntax is

$ grep “string pattern” <filename>
$ cat < filename | * > | grep "string pattern"

Lets see the Demonstration

[vamshi@linuxcent grep]$ grep "Linux" intro.txt
There following are Most popular Linux server Distributions:
1) Redhat Enterprise Linux
5) SUSE Linux Enterprise Server
This is a Demonstration of Linux grep command

Searching for a pattern on multiple files using grep command

Lets see the Demonstration

[vamshi@linuxcent grep]$ grep "Linux" intro.txt intro-demo.txt
intro.txt:There following are Most popular Linux server Distributions:
intro.txt:1) Redhat Enterprise Linux
intro.txt:5) SUSE Linux Enterprise Server
intro.txt:This is a Demonstration of Linux grep command
intro-demo.txt:There following are Most popular Linux server Distributions:
intro-demo.txt:1) Redhat Enterprise Linux
intro-demo.txt:5) SUSE Linux Enterprise Server
intro-demo.txt:This is a Demonstration of Linux grep command

 

How to print the sequence number of matched pattern in file using Grep command in Linux?

Print the sequence number of lines that contain the matched string pattern can be achieved using -n Option:

[vamshi@linuxcent grep]$ grep -n "Linux" intro.txt
4:There following are Most popular Linux server Distributions:
5:1) Redhat Enterprise Linux
9:5) SUSE Linux Enterprise Server
11:This is a Demonstration of Linux grep command

The -b Option prints the subsequent starting offset index of the matching word from the lines within the given filename.

[vamshi@linuxcent grep]$ grep -b "Linux" intro.txt
66:There following are Most popular Linux server Distributions:
127:1) Redhat Enterprise Linux
184:5) SUSE Linux Enterprise Server
217:This is a Demonstration of Linux grep command

So far we have seen only the matching string pattern, but if you want to find the occurance of particular word then use grep -w Option

Lets us see the Demonstration below:

[vamshi@linuxcent grep]$ grep -w "Cent" intro.txt


And As expected here we have not seen any occurrence of the word Cent from the file although the word Centos was present, But it didnt match because we used -w option to match the word Cent.

But when we grep for the Word Linux then we can see the exact occurrence of the word and the matching lines are printed.

[vamshi@linuxcent grep]$ grep -w "Linux" intro.txt
There following are Most popular Linux server Distributions:
1) Redhat Enterprise Linux
5) SUSE Linux Enterprise Server
This is a Demonstration of Linux grep command

This can be used in conjunction with -o prints only completely matched word string pattern.

[vamshi@linuxcent grep]$ grep -w -o "Linux" intro.txt
Linux
Linux
Linux
Linux

How to use the Invert Option in Linux grep command

The -v option prints all the lines not matching the given string pattern.

[vamshi@linuxcent grep]$ grep -v "Linux" intro.txt
the first line is in lower case
THE SECOND LINE IS IN UPPER CASE


2) Ubuntu
3) Centos
4) Debian

 

How to print the names of all the files matching the string pattern using Linux Grep command.

Using the -l Option of grep command will print all the file names containing the matching string pattern/string literal

[vamshi@node02 grep]$ grep -l Linux *
intro-demo.txt
intro.txt

How to get the number of Count Matches using Grep command ?

In Linux Grep command we can use the Option -c to get the total occurance Count of the matching pattern/string

Lets see the Demonstration below

[vamshi@linuxcent grep]$ grep -c "Linux" intro.txt
4

How to Exclude specific Directories while searching with grep command ?

The --exclude-dir="Some Directory name" options skips the contents of the mention directory Lets see the Demonstration:

$ grep -w -E "data" * -R --exclude-dir="backup" --exclude-dir="adb-fastboot" --exclude-dir="IntelliJ-IDEA"

How to use Linux Grep commmand to ignore the case  OR perform Case Insensitive search ?

Linux Grep Command offers the -i to skip the case and perform the search.

[vamshi@linuxcent grep]$ grep -i Cent intro.txt
3)Centos

Lets search for the Work Upper from the file with -i option

[vamshi@linuxcent grep]$ grep Upper -i intro-demo.txt
THE SECOND LINE IS IN UPPER CASE

How to Grep for the lines before and After the pattern occurrence in a file

We can use the below options in Linux Grep command to extract the N number of lines matching the containing pattern before/After from the given file.. Lets look at the Options in detail

Generic Syntax:

grep -<A|B|C> “string pattern” filename*

Lets see the Demonstration as follows:

-A: Prints N number of lines After the pattern match and the line containing the match

[vamshi@linuxcent grep]$  grep -A2 docker /etc/passwd
dockerroot:x:996:992:Docker User:/var/lib/docker:/sbin/nologin
mysql:x:27:27:MariaDB Server:/var/lib/mysql:/sbin/nologin
ntp:x:38:38::/etc/ntp:/sbin/nologin

-B: Prints N number of lines Before the pattern match and the line containing the match

[vamshi@linuxcent grep]$  grep -B2 docker /etc/passwd
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
builduser1:x:1002:1002::/home/builduser1:/bin/bash
dockerroot:x:996:992:Docker User:/var/lib/docker:/sbin/nologin

-C: Prints the line Containing the pattern and the N number of lines Before and After

[vamshi@linuxcent grep]$ grep -C2 docker /etc/passwd
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
builduser1:x:1002:1002::/home/builduser1:/bin/bash
dockerroot:x:996:992:Docker User:/var/lib/docker:/sbin/nologin
mysql:x:27:27:MariaDB Server:/var/lib/mysql:/sbin/nologin
ntp:x:38:38::/etc/ntp:/sbin/nologin

Linux Grep command using the Regular Expression (regex) search pattern

How to use the Special Meta-characters in Grep Command

The Special Meta-Character ^ (caret) symbol to match expression at the start of a line.
Practical examples of grep command using meta-characters are as follows:

[vamshi@linuxcent grep]$ grep ^root /etc/services
rootd           1094/tcp                # ROOTD
rootd           1094/udp                # ROOTD

$ (dollar) symbol to match expression at the end of a line

[vamshi@linuxcent grep]$ grep "bash$" /etc/passwd
root:x:0:0:root:/root:/bin/bash
vagrant:x:1000:1000:vagrant:/home/vagrant:/bin/bash
vamshi:x:1001:1001::/home/vamshi:/bin/bash
jenkins:x:997:994:Jenkins Automation Server:/var/lib/jenkins:/bin/bash
builduser1:x:1002:1002::/home/builduser1:/bin/bash
linuxcent:x:1003:1004::/home/linuxcent:/bin/bash

Another Bonus regex for evading Empty lines from a text file using the Meta-characters ^ Carat and $ Dollar Symbols.

[vamshi@node02 grep]$ grep -Ev "^$" intro.txt
the first line is in lower case
THE SECOND LINE IS IN UPPER CASE
There following are Most popular Linux server Distributions:
1) Redhat Enterprise Linux
2) Ubuntu
3) Centos
4) Debian
5) SUSE Linux Enterprise Server
This is a Demonstration of Linux grep command

As the result of the above command observed Empty lines in our text file are now excluded with the inverted matching of Empty line pattern expression.

Digit Matching operation using Linux Grep Command

The Grep command Option -P offers the perl-regex pattern matching to perform some of the tricky pattern matching conditions, Lets see the Demonstration.

[vamshi@linuxcent grep]$ grep -P "[\W][\d][\d]/" /etc/services

Now we Demonstrate Numerical pattern matching in the following ways to match the exact occurrence of IPv4 address

[vamshi@linuxcent grep]$ ip a |grep -E "[0-9].[0-9].[0-9].[0-9]"
inet 127.0.0.1/8 scope host lo
inet 10.100.0.20/24 brd 10.100.0.255 scope global noprefixroute eth1

The similar result can be extracted using -P, Demonstrated as below:

[vamshi@linuxcent grep]$ ip a  | grep -P "[\d].[\d].[\d].[\d]"
inet 127.0.0.1/8 scope host lo
inet 10.100.0.20/24 brd 10.100.0.255 scope global noprefixroute eth1

Highlighting the Search patterns with Color codes

If you want to highlight the search pattern from the output we can make use of Color code Option is grep

[vamshi@linuxcent ~]$ ps -ef |grep java --color=yes

 

grep command and the Meta-characters with practical examples

In the Extended Regular Expression the metacharacters ?, \+, \{,  \|, \(, and \) enhances and takes the search string to manifest into a rich pattern

The square brackets [/code][ ][/code] includes a list of characters and it matches a single character in the list. Within the bracket, we can specify a  range, like [a-z] [0-9] [abcd], But it only matches a single character in the give list range.
The brackets can contain the carat ^ or the $ symbols and can be globbed with the wildcard characters for finding the repetitive pattern.

Here we shall see the demonstration of a few metacharacters conjunction into a single grep command

Using the Infix Operator | with the meta-characters

$ dmesg -H | egrep '(s|h)d[a-z]'

A quick practical look at the dmesg to find about the errors and warnings using grep command.

$ dmesg -H | grep -Ei “error|warn”

How to find the exact work match for the word error or err with a blank white space at the start of the word from dmesg?
$ dmesg -H | grep -Ei “(er)[r]{1,2}(or)”
$ dmesg -H | grep -Ei “(\We)[r]+(or)|(\Werr)”

Explanation: The grep search metacharacter regular expression matches the occurring of the string literals “error” with the minimum 1 time occurrence (error) of literal r and maximum 2 times.

The output only prints lines containing the word “error“ in a case insensitive grep search.

 

Another great practical example here we apply the same logic to enhance the metacharacter regex pattern to search for the occurrence of error with but without a trailing colon “:” symbol

dmesg -H | grep -Ei “(error)[:]{0}”

 

How to create symbolic Link or Softlinks in Linux and differentiate between Softlink vs Hardlink

The concept of Links in Linux/Unix based systems is Unique and gives a very deeper understanding of the Linux working internals at various levels.

The symbolic also known as Soft link is a special type of linking that acts as a pointer to the original file present on the disk.

The Softlink span across different filesystems extensively and widely used during software package installation and configuring

Lets look at the example of java command linking:

[root@node02 ]# which java
/bin/java
[root@linuxcent ]# ls -l /bin/java
lrwxrwxrwx. 1 root root 22 Apr 10 11:52 /bin/java -> /etc/alternatives/java
[root@node02 boot]# ls -l /etc/alternatives/java
lrwxrwxrwx. 1 root root 73 Apr 10 11:52 /etc/alternatives/java -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java
[root@linuxcent ]#

If you upgrade java on your system then the /bin/java command points out to a newer installed version of /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java

We will see the demonstration of a Symbolic link.

In Linux to make the links between files, the command-line utility is “ln”.

Softlink Creation syntax using the Linux command line utility ln -s option:

ln -s <source file|Directory > <destination Link file|Directory>

Below is an example

[vamshi@linuxcent html]$ ln -sf /var/www/html/index.html /tmp/index.html

[vamshi@linuxcent html]$ ls -l /tmp/index.html

lrwxrwxrwx. 1 vamshi vamshi 24 Apr  1 18:23 /tmp/index.html -> /var/www/html/index.html

 

The second file is called a symbolic ink to /tmp/index.html

Now the second file /tmp/index.html is called a Symbolic link to the First file on the disk /var/www/html/index.html

It means that the second file is just pointing to the first file’s location on disk without actually copying the contents of the file.

[vamshi@linuxcent html]$ cat /var/www/html/index.html
Welcome to LinuxCent.com
[vamshi@linuxcent html]$ cat /tmp/index.html
Welcome to LinuxCent.com

When you edit either of the file, the contents of the original file on disk are directly modified.

 

How to create Softlinks to Directories in Linux ?
The same logic applies to creating the soft links to directories, Lets see the Demonstration below:

[vamshi@linuxcent html]$  ln -sf /var/www/html/linuxcent/static/ /tmp/static
[vamshi@linuxcent html]$ ls -l /tmp/static
lrwxrwxrwx. 1 vamshi vamshi 32 Apr  8 18:37 /tmp/static -> /var/www/html/linuxcent/static/


 

How do we Remove the linking between the files is even simpler.

Simply run the unlink command on its Destination:

Sample command:

unlink <destination Link file | Directory>

Lets see the Demonstration

unlink /tmp/data-dir

 

Understanding Symbolic / Hard link concept better with improving knowledge on Inode.

To better understand the concept of Symbolic linking we need to understand the concept of Inode numbers present in Linux. I will give a brief overview of it in this section But for Detailed Review, Please see the What is Inode in Linux Section.

We can list out the inode number of any file on linux system using [/code]ls -i <filename>[/code]

Now The extreme left column indicates the system wide unique inode number:

[root@node02 ~]# ls -li /var/www/html/index.html /tmp/index.html 
67290702 lrwxrwxrwx. 1 root root 24 Apr 8 19:09 /tmp/index.html -> /var/www/html/index.html
33557754 -rw-r--r--. 1 root root 25 Apr 8 18:19 /var/www/html/index.html

We can see here the inode number for both the files have different, Because the second file is just a pointer to the original source file..

So why do we create the Symbolic Links ?

We have the advantages of the Softlinks

  1. One Advantage of Softlinks/symbolic links is they span across different filesystems
  2. Can have many softlinks to a single file/Directory.
  3. Can create Symbolic Links of Directories and Files respectively.
  4. Rsync program by default preserves the symbolic links.
  5. The softlinks become activated immediately if the source is recreated or recovered after any kinds of network outages.

What are the best practices when using Softlinks ?

The best practice while creating softlinks is to mention absolute path for source and destination links.

On the other hand you should also understand the disadvantages or shortcomings.

  1. It is Important to observe that If the source file is deleted then the symbolic link becomes useless.
  2. Incases of the Network filesystem issues leading to unavailability of softlinks.

It is also Essential to understand about the Hardlinks to get an overall understanding.

 

Creating a hard link is a simple operation using ln command with no options

Sample Command:

$ ln /path/to/source/ /path/to/HardLink/

So lets start of by creating a hard link of our file /var/www/html/index.html

[vamshi@linuxcent linuxcent]$ ln /var/www/html/index.html /tmp/index-hl.html

The command ls -il lists the inode number of the files.

[vamshi@linuxcent  ]$ ls -i
33557752 index-hl.html  33557752 index.html

So to conclude, The hard linking results in the same inode number, howmany ever times the hardlink of same file is created.
The data continues to persists in the same storage location on the filesystem even if one of the hardlink file is deleted.
As long as the inode number is identical on the files, no filename change matters to the filesystem.
There will be no change in Hardlink behaviour.

Let’s add some content to the hardlink file here and see the Demonstration

[vamshi@linuxcent ]$ echo "We are updating the file to check out the Hardlinks" >>  /tmp/index-hl.html
We are updating the file to check out the Hardlinks

The new line is added in the original file.

[vamshi@linuxcent linuxcent]$ cat index.html
Welcome to LinuxCent.com
We are updating the file to check out the Hardlinks

Content manipulations to either of the files will be treated as a same file.

How to Identify how many times a particular file was linked ?

Note that Linux Command ls -li provides the actual link aggregate count information in the fourth column represented by the number 2 here, which means that it has a second hardlink reference and both the files are interchangeable.
And It should be noted that a file has a link count of 1 by default as it references itself

$ls -li
33557752 -rw-rw-r--. 2 vamshi vamshi 59 Apr  8 19:48 index-hl.html
33557752 -rw-rw-r--. 2 vamshi vamshi 59 Apr  8 19:48 index.html

In case either one of the file is deleted, the other file survives and continues to function properly.

Lets see some hard facts on the Hardlinks.

  1. Hardlinks can’t be created to Directories.
  2. They do not span across filesystems.

Linux Copy File Command for Files and Directories – cp Command Examples

Linux copy files command: cp is generally used for organizing the data on the Linux operating system, It copies the files and directories.

We shall take a deeper look at Linux cp command-utility in the section

In order to copy files and directories, you must have read permissions on the source file(s) and write permissions on the destination directory

How do I copy files under Linux operating systems?

How do I make a 2nd copy of a file on a Linux bash shell?

How can I copies files and directories on a Linux

Linux Copy File command Syntax

cp sourcefile destinationfile
cp sourcefile DESTDIR
cp sourcefile1 sourcefile2 DESTDIR
cp [OPTION] SOURCE DESTFILE
cp [OPTION] SOURCE DESTDIR

How to Copy a Directory if the destination does not exist?

To achieve this we can make use of the following cp command options -R or -r: Copy directories recursively.

Linux cp command Syntax with -R option:

cp -R SOURCE DESTINATION

If the destination doesn’t exist, it will be created.

It can also be used to Copy the contents Recursively

Lets see the demonstration as follows:

[vamshi@linuxcent ]$ cp -R dir1/ dir1-copy
[vamshi@linuxcent ]$ ls -l 
total 0
drwxrwxr-x. 2 vamshi vamshi 6 Apr 11 06:35 dir1
drwxrwxr-x. 2 vamshi vamshi 6 Apr 11 06:37 dir1-Recursive

Using the verbose Option -v to print the copy activity information onto the output screen.

Let’s use the -v flag to print the verbose information onto the screen.

How to Preserve the Source file and Directory permission?

Linux Copy command Syntax with -p option:

-p option preserves the mode, ownership and timestamps from the source to the destination

cp -p file1 file1-copy

Lets us see the Demonstration as Below

[vamshi@node02 cp-command]$ cp -Rp dir1/ dir1-copy
[vamshi@node02 cp-command]$ ls -ld dir1*
drwxrwxr-x. 2 vamshi vamshi 6 Apr 11 06:35 dir1
drwxrwxr-x. 2 vamshi vamshi 6 Apr 11 06:35 dir1-copy
drwxrwxr-x. 2 vamshi vamshi 6 Apr 11 06:37 dir1-Recursive

From the out we can conclude the the Linux copy command with -p Option preserves the original timestamps information and copies it to the destination

Linux cp command with Force copy -f Option, It forcefully overwrites the destination content
Sample Syntax:

cp -f file1 file1-copy

How to Copy Multiple files at once ?

Asterisk / wildcard (*) character is used to copy files multiple files with same pattern.

[vamshi@linuxcent ]$ cp -varpf file* DEST/
‘file10.txt’ -> ‘DEST/file10.txt’
‘file1.txt’ -> ‘DEST/file1.txt’
‘file2.txt’ -> ‘DEST/file2.txt’
‘file3.txt’ -> ‘DEST/file3.txt’
‘file4.txt’ -> ‘DEST/file4.txt’
‘file5.txt’ -> ‘DEST/file5.txt’
‘file6.txt’ -> ‘DEST/file6.txt’
‘file7.txt’ -> ‘DEST/file7.txt’
‘file8.txt’ -> ‘DEST/file8.txt’
‘file9.txt’ -> ‘DEST/file9.txt’

The options -p or -d enables preserving the links and can be used in conjunction with -R option to copy contents Recursively from the source directory.

How to Copy Files and Folders on Linux Using the cp Command recursively to Destination Directory

How to preserve the links with cp command?

Using the Options -p preserves the links and -r Option copies the content recussively same as -R Option and -v prints the verbose information

[vamshi@node02 Linux-blog]$ cp -varpf Redhat-Distro/ /tmp/DEST
‘Redhat-Distro/’ -> ‘/tmp/DEST’
‘Redhat-Distro/Fedora’ -> ‘/tmp/DEST/Fedora’
‘Redhat-Distro/Fedora/fedora.txt’ -> ‘/tmp/DEST/Fedora/fedora.txt’
‘Redhat-Distro/Centos’ -> ‘/tmp/DEST/Centos’
‘Redhat-Distro/Centos/centos.txt’ -> ‘/tmp/DEST/Centos/centos.txt’
‘Redhat-Distro/Centos/CentOS-versions’ -> ‘/tmp/DEST/Centos/CentOS-versions’
‘Redhat-Distro/Centos/CentOS-versions/centos7.txt’ -> ‘/tmp/DEST/Centos/CentOS-versions/centos7.txt’
‘Redhat-Distro/Centos/CentOS-versions/centos6.1.txt’ -> ‘/tmp/DEST/Centos/CentOS-versions/centos6.1.txt’
‘Redhat-Distro/Centos/README-CentOS’ -> ‘/tmp/DEST/Centos/README-CentOS’
‘Redhat-Distro/README-Redhat-Distro’ -> ‘/tmp/DEST/README-Redhat-Distro’
‘Redhat-Distro/RHEL-Versions’ -> ‘/tmp/DEST/RHEL-Versions’
‘Redhat-Distro/RHEL-Versions/redhat5.txt’ -> ‘/tmp/DEST/RHEL-Versions/redhat5.txt’
‘Redhat-Distro/RHEL-Versions/redhat8.txt’ -> ‘/tmp/DEST/RHEL-Versions/redhat8.txt’
‘Redhat-Distro/redhat.txt’ -> ‘/tmp/DEST/redhat.txt’

How to make a symbolic link with Linux cp command to files ?

As we know that ln command us useful to create symboic links, But the Linux copy command Syntax can do that to files with -s Option which creates Symbolic links:

cp -s SOURCE DESTINATION

Linux copy command Syntax with Softlink with Demonstration:

[vamshi@linuxcent ~]$ ls -l total 0
-rw-rw-r--. 1 vamshi vamshi 0 Apr 11 06:39 file1.txt

lrwxrwxrwx. 1 vamshi vamshi 9 Apr 11 06:39 file2.txt -> file1.txt

Linux cp command with interactive prompt using -i option

Sample Syntax:

cp -i file1 file1-copy

Also you can make it a best practice to setup alias alias for cp command.
The best practice is enable options -av

cp -av SOURCE DESTINATION
export cp="cp -av"

How can i copy the hidden files ?

To Copy the hidden files we can use cp command with -a option,lets us see in a practical example.

$ cp -av source/ destination/
‘source/.config1’ -> ‘destination/source/.config1’
‘source/.config2’ -> ‘destination/source/.config2’
‘source/.config3’ -> ‘destination/source/.config3’

Generally the hidden files in Linux are prefixed with a dot . So we can also use the wildcard character *, and copy them, below is another pracctical example

[vamshi@linuxcent cp-command]$ cp -av source/.conf* destination/
‘source/.config1’ -> ‘destination/.config1’
‘source/.config2’ -> ‘destination/.config2’
‘source/.config3’ -> ‘destination/.config3’

How to Copy a File from One Location to Another With a Different Name on Linux Using the cp Command

Assuming we have a couple of users on our linux server called Alice and Bob

[alice@linuxcent ~]$ sudo cp -avrpf /home/alice/djangoproject1/ /home/bob/
‘djangoproject1/’ -> ‘/home/bob/djangoproject1’
‘djangoproject1/__init__.py’ -> ‘/home/bob/djangoproject1/__init__.py’
‘djangoproject1/asgi.py’ -> ‘/home/bob/djangoproject1/asgi.py’
‘djangoproject1/settings.py’ -> ‘/home/bob/djangoproject1/settings.py’
‘djangoproject1/urls.py’ -> ‘/home/bob/djangoproject1/urls.py’
‘djangoproject1/wsgi.py’ -> ‘/home/bob/djangoproject1/wsgi.py’
‘djangoproject1/__pycache__’ -> ‘/home/bob/djangoproject1/__pycache__’
‘djangoproject1/__pycache__/__init__.cpython-36.pyc’ -> ‘/home/bob/djangoproject1/__pycache__/__init__.cpython-36.pyc’
‘djangoproject1/__pycache__/settings.cpython-36.pyc’ -> ‘/home/bob/djangoproject1/__pycache__/settings.cpython-36.pyc’

How to backup files using cp command?

The linux cp command offer the option --backup to backup the data files, below is the command.

cp --backup source destination

Check the Open Ports in Linux

The Ports on a Linux OS are used for exchange and transfer of data on the network connected devices.

A Very high number of security exploitation happen due to no surveillance of in bound connections targeting specific ports. It is most essential to identify the underlying Linux process opening up specific ports for listening over a shared network.

Thus, It is important to identify which ports are open on your Linux machine.

Firstly for basic administration tasks, identifying the port and its correlating application, so that you are well aware of the Open sockets and Enhance the network security by preventing the network intrusion by writing the firewall rules.

In this tutorial we will look at some of the most popular Linux network tools and see how to gather information, and identify the web server like process Apache Httpd or Nginx running so that you don’t conflict when you are configuring them and troubleshooting.

We will discuss some of the popular tools and their general commands syntax.

Netstat Command and its Syntax

netstat

Netstat gives you multiple features and a must know tool if you are in your day to day activities.

It gives out the information about the Open ports in you Linux machine along with the Established connection, TimeWait and Closed state connections.

The netstat command goes by netstat -ntlp

ss

It stands for Socket Statistics, It is a command line utility which provides the information about Open ports, the corresponding process ID which opened the port.

ss command is the successor to netstat command in Linux and has the similar options as its predecessor, In Fact it necessarily is a better enhancement over the old netstat command,

First up lets run ss -tua, which lists all the TCP and UDP Sockets.

[vamshi@linuxcent ~]$ ss -tua

Netid  State     Recv-Q Send-Q                                  Local Address:Port                                                   Peer Address:Port

udp    UNCONN     0      0                                           127.0.0.1:domain                                                            *:*

udp    UNCONN     0      0                                           127.0.0.2:domain                                                            *:*

udp    UNCONN     0      0                                                   *:bootpc                                                            *:*

udp    UNCONN     0      0                                                   *:sunrpc                                                            *:*

udp    UNCONN     0      0                                           127.0.0.1:323                                                               *:*

udp    UNCONN     0      0                                                   *:lanserver                                                         *:*

udp    UNCONN     0      0                                                [::]:sunrpc                                                         [::]:*

udp    UNCONN     0      0                                               [::1]:323                                                            [::]:*

udp    UNCONN     0      0                                                [::]:lanserver                                                      [::]:*

tcp    LISTEN     0      100                                         127.0.0.1:smtp                                                              *:*

tcp    LISTEN     0      128                                                 *:sunrpc                                                            *:*

tcp    LISTEN     0      128                                                 *:ssh                                                               *:*

tcp    ESTAB      0      0                                         10.100.0.20:ssh                                                      10.100.0.1:45662

tcp    LISTEN     0      100                                             [::1]:smtp                                                           [::]:*

tcp    LISTEN     0      70                                               [::]:33060                                                          [::]:*

tcp    LISTEN     0      128                                              [::]:mysql                                                          [::]:*

tcp    LISTEN     0      128                                              [::]:sunrpc                                                         [::]:*

tcp    LISTEN     0      128                                              [::]:http                                                           [::]:*

tcp    LISTEN     0      128                                              [::]:ssh                                                            [::]:*

 

How to get the Established connection information using ss command in linux

$ ss -tua state established

 

[vamshi@node02 ~]$ ss -l sport = 80

The detailed options ss offers are as follows:

-a : Displays all Sockets:

-i : Displays internal TCP information

-t : Displays only TCP Sockets

-l : Displays Only Listening Sockets

-u : Displays only UDP Sockets

-r : Resolves host names

-n : Doesn’t Resolve the Hostnames

-p : Display process information using the Socket

 

ss command takes the following state option filters

established syn-sent |syn-recv |fin-wait-{1,2} |time-wait |closed |close-wait |last-ack |listen |closing

 

nmap

It is one of the most popular open source tools to explore networks and mainly used for security auditing.

NMAP is used extensively for Host Discovery , Port and Protocol Scanning such as ICMP, TCP and UDP Port Scanning.

The service and its version Detection, Operating system Detection.

It is one of the Intrusion prevention tool when used effectively and used for greater security Reporting.

run the command sudo nmap localhost

$ sudo nmap 10.100.0.0/24
$ vamshi@linuxcent: ~ $ nmap 10.100.0.20

Starting Nmap 7.70 ( https://nmap.org ) at 2020-04-10 13:45 GMT

Nmap scan report for 10.100.0.20

Host is up (0.00014s latency).

Not shown: 996 closed ports

PORT     STATE SERVICE

21/tcp   open ftp

22/tcp   open ssh

53/tcp   open domain

80/tcp   open http

111/tcp  open rpcbind

3306/tcp open  mysql

8009/tcp open  ajp13

MAC Address: 08:00:27:5A:26:BD (Oracle VirtualBox virtual NIC)


Nmap done: 1 IP address (1 host up) scanned in 1.51 seconds

It lists all the ports that are Open and even lists up the MAC Address of the Target Host.

How to scan a list of hosts from a file using nmap command

You can do it and run the command as demonstrated below:

$ nmap -iL /etc/hosts # where you have a list of ip/dns names

To increase the verbosity of nmap output use -v<n> where n is a number ranging from 0 – 9

This offer a lot of options to gather information passively and lets see some options listed below:

 

-sn : Lists and does Ping Scan on the network; Least aggressive

-sL : Lists hosts to scan on the network

-O : Enables Target Host OS detection.

-sS : Does the syn Scan, in stealth mode.

-sT : Performs TCP port Scan

-sU : Performs UDP port Scan.

-p : Scans only for Target Port Listed Eg: # sudo nmap -v  -sS -p80 10.100.0.0/24

-PA : TCP Ack Flag is set

-sV : Extracts the Service Version information and the Operating System information from Target Hosts.

 

lsof

lsof linux command gives the information on the list of open files on the system as the abbreviation says. It’s one of the valuable tools when troubleshooting under fire, Gives you the practical linux system behaviour

Start it by running lsof command, will print a bunch of information including the all the programs currently started and owned by you, It includes the block/filesystem files, network stream data/character files, virtual memory paging and temporary data files.

Listing the openfiles using lsof

Now run lsof -u <Your login username>

$ lsof -u vamshi

# Prints a bunch of information about the open files for a particular user.

Now lsof has a lot of options to extract relevant information

To list the total number of open files on the system

$ sudo lsof | wc -l
$ sudo lsof -i TCP :22
$ sudo sudo lsof -P -i:22
vamshi@linuxcent:~$ sudo lsof -i -P :22

COMMAND  PID USER   FD TYPE DEVICE SIZE/OFF NODE NAME

sshd     380   root    3u  IPv4  13672     0t0  TCP *:22 (LISTEN)

sshd     380   root    4u  IPv6  13683     0t0  TCP *:22 (LISTEN)

sshd    1295   root    3u  IPv4  20367     0t0  TCP 10.100.0.30:22->10.100.0.1:39054 (ESTABLISHED)

sshd    1307 vamshi    3u  IPv4  20367     0t0  TCP 10.100.0.30:22->10.100.0.1:39054 (ESTABLISHED)


Lists the information of open and Established connections of the given port

lsof offer a lot of options and filters, lets list some of the most commonly used ones below:

 

-u : takes the username as filter option, Lists openfiles caused by the given username

-i : Lists the open files belonging to the Service port numbers

-P : Inhibits the translation of service names based on port number and prints the port number for simplicity purpose.

 

Change file(s) and Directory/Folder ownership – chmod command in Linux

The Linux command utility chmod is used to set permissions on files and directories

The command chmod, or Change Mode, is widely used to modify the access permissions of files and directories, This facilitates the users to keep the data secure and properly organized.
There are three types the file permissions can be modified using chmod command:

1) Symblolic Notation

2) Octal Notation

3) Reference operator

The Owners, Groups and Others have different permissions to access a particular file.
Lets see the Sample syntax of chmod

$ chmod [option] [mode] < file | directory>

The Users categories are symbolically represented as

-u is used for the User/Owner

g is used for the Group

o is used for the Others

The Permission modes used to modify the Read, Write and Execute permissions respectively on a file or Directory and are symbolically represented as follows

r is for Read permission

w is for Write permission

x is for Execute permission

The Octal notation is simpler interms of representation and execution with r=4 w=2 x=1 total adding up to numerical aggregate value of 7 taking the place holders of Owner(-u) Group(-g) Others(-o) respectively.

Lets see the file permissions for a sample file with complete access to every user category i.e owner, group and others, Meaning that any person who has access to the file can modify,execute(run) or delete the newfile.txt

[vamshi@node02 linuxcent]$ ls -l file.txt
-rwxrwxrwx. 1 vamshi vamshi 3 Apr 10 19:13 file.txt

From the above listed permissions we would like to release the “others” from writing this file but only limit them to read it.

The command will be to delete the Write permissions from others:

[vamshi@linuxcent ~]$ $ chmod o-w file.txt
[vamshi@linuxcent ~]$ ls -l file.txt
-rwxrwxr-x. 1 vamshi vamshi 3 Apr 10 19:13 file.txt

We use the + and - operators to provide the Read,Write and Execute permissions to user categories, It’s Demonstrated as follows:

[vamshi@node02 linuxcent]$ chmod og-rw file.txt
[vamshi@node02 linuxcent]$ ls -l file.txt
-rwx------. 1 vamshi vamshi 3 Apr 10 20:23 file.txt

Combining options such read operation for other and group, write for group and execute permission being removed for owner.

[vamshi@node02 linuxcent]$ chmod og+r,g+w,u-x file.txt
[vamshi@node02 linuxcent]$ ls -l file.txt
-rw-rw-r--. 1 vamshi vamshi 3 Apr 10 20:23 file.txt

Creating a new file newfile.txt and working with it

[vamshi@node02 linuxcent]$ echo "Welcome to LinuxCent" > newfile.txt

Here are the file’s permissions when it’s newly created. It is resultant of the umask

Lets look at a Linux practical example to setup only execute permission for all the user categories

$ chmod +x  /etc/profile.d/java-path.sh

Listing out the System defined Default umask in Symbolic notation.

The Default Umask permissions can be demonstrated as below:

[vamshi@node02 linuxcent]$ umask -S
u=rwx,g=rwx,o=rx

Umask in Octal Notation as below:

[vamshi@node02 linuxcent]$ umask
0002

Now the new lets see the file permissions:

[vamshi@node02 linuxcent]$ ls -l newfile.txt
-rw-rw-r--. 1 vamshi vamshi 21 Apr 10 19:49 newfile.txt

This is the inverse notation of the umask Octal value(002) set at the system wide and our file has the 664 Octal Notation.

The Octal mode of Permissions

These permissions carry a numerical value all adding up to a sum of 4 and categorized as follows:

r=4 w=2 x=1

For Example chmod 754 file1.txt

[vamshi@node02 linuxcent]$ ls -lth newfile.txt
-rwxr-xr--. 1 vamshi vamshi 3 Apr 10 19:13 newfile.txt

How to apply reference permissions in Linux using chmod ?

We shall be using the option --reference which takes the reference from the existing file and applies the permission

Lets see the demonstration below:

vamshi@linuxcent ~]$ ls -l anotherfile.txt
-rwxrwx---. 1 vamshi vamshi  33 Apr 8 19:52 anotherfile.txt
vamshi@linuxcent ~]$ chmod --reference=anotherfile.txt newfile.txt
[vamshi@linuxcent ~ ]$ ls -l newfile.txt 
-rwxrwx---. 1 vamshi vamshi 21 Apr 10 19:49 newfile.txt

With this reference operator, the nexfile.txt permissions are modified identical to anotherfile.txt

 

Docker container volumes

The concept of docker is to run a compressed image into a container which servers the purpose and then container can be removed, leaving no trace of the data generated during the course of its runtime. This exact case is referred as ephemeral container.
The docker is traditionally non-persistent data storage and retains only the data originally from its image build creation, It provides the facility to integrate the volume mounts to a running container for data storage and manages the persistence issue to a certain extent.
We have realized the ability to store the data on volumes and then make them available to the container runtime environment to satisfy the needs.

As per the practice the docker volumes are mounted to docker image in runtime via the command line arguments which are demonstrated as show below.

This implementation of the docker volumes provides the running container a volume binding capability and this is method in docker volumes can be broadly categorized into two abilities which are listed as follows.

(1) The volume mapping from the host to the target container, This is like directory mapping between the Host and the container which happens during its runtime.
(2) A permanent volume name that can be shared among container and it even persists if the container is deleted and the same volume can be again mounted to another docker container.

The storage options offered by docker are for persistent storage of files and ensures it in cases of docker container restarts and removal of the container.

The example for the host bind volume mapping is as follows:

# docker run -v  src:target --name=container-name  docker-image -d

The syntax of docker directory mount:

# docker run -it -v /usr/local/bin:/target/local --name=docker-container-with-vol  ubuntu:latest /bin/bash

We will invoke a container by mounting the host path to an target container path and mount the host directory to nginx-html /usr/share/nginx/html.

[vamshi@node01 ~]$ docker run -d --name my-nginx -v /home/vamshi/nginx-html:/usr/share/nginx/html -p 80:80 nginx:1.17.2-alpine
34797e6d8939e42bc8cfe36eed4b60521355edadc2fa6c74a26fe4172384575c

Now we log into the container and verify the contents

[vamshi@node01 ~]$ docker exec -it my-nginx1 sh
~ # hostname
34797e6d8939
~ # df -h /usr/share/nginx/html
Filesystem                Size      Used Available Use% Mounted on
/dev/sda1                40.0G     16.1G     23.9G  40% /usr/share/nginx/html

From the df -h output from the container shows the path /usr/share/nginx/html as mount point.

Checking the contents of the webroot directory at /usr/share/nginx/html inside the container.

~ # cat /usr/share/nginx/html/index.html
<H1>Hello from LinuxCent.com</H1>

This is the same file which we have on our host machine and it is shared through the volume mount.
We verify using the curl command as follows

~ # curl localhost
<H1>Hello from LinuxCent.com</H1>

We can also mount the same directory with as many containers as possible on our system and it can be an effective way of updating the static content being utilized within our containers.

This is a bind operation offered by docker to mount the directory to a container.. This information can be identified by inspecting the docker command as follows:

Here is the extract snippet from the docker inspect container my-nginx1 :

            {
                "Type": "bind",
                "Source": "/home/vamshi/nginx/nginx-html",
                "Destination": "/usr/share/nginx/html",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }

As we can see the Type of mount is depicted as a bind here.

The formatter can also used with the filtering options in json format

[vamshi@node01 ~]$ docker container inspect my-nginx1 -f="{{.Mounts}}"
[{bind /home/vamshi/nginx-html /usr/share/nginx/html true rprivate}]

The Dockerfile VOLUME expression

Using the Persistent docker volumes.

We now focus our attention to the Docker volume mounts, which are isolated storage resources in docker and are a persistent storage which can be reused and mounted to the containers.

The VOLUME can be used while writing a Dockerfile, it creates a docker image with the volume settings and then mounts to the container when it is startedup.
We use the expression in dockerfile volume.

VOLUME [ "my-volume01" ]

We now build the image and lets observe the output below.

Step 1/4 : FROM nginx-linuxcent
 ---> 55ceb2abad47
Step 2/4 : COPY nginx-html/index.html /usr/share/nginx/html/
 ---> c482aa15da5a
Removing intermediate container a621e114a01d
Step 3/4 : VOLUME my-volume01
 ---> Running in ac523d6a02f0
 ---> 72423fe5f27d
Removing intermediate container ac523d6a02f0
Step 4/4 : RUN ls /tmp
 ---> Running in 8fc1fbc0f0bb
 ---> 0f453e3cfff1
Removing intermediate container 8fc1fbc0f0bb

Here the volume is mounted at the path /my-volume01 based on absolute path from /.

Here my-volume01 will be mounted to /my-volume01 inside the container path

/ # df -Th /my-volume01/
Filesystem Type Size Used Available Use% Mounted on
/dev/sda1 xfs 40.0G 15.1G 24.9G 38% /my-volume01

The information can be extracted by inspecting the container as follows:

# docker container inspect nginx-with-vol -f="{{.Mounts}}"
[{volume 7d6d92cffac1d216ca062032c99eb105b120d769331a2008d8cad1a2c086ad19 /var/lib/docker/volumes/7d6d92cffac1d216ca062032c99eb105b120d769331a2008d8cad1a2c086ad19/_data my-volume01 local true }]

 

If you would like the volume to be mounted to some other path then you can declare that in Dockerfile VOLUME as below:

VOLUME [ "/mnt/my-volume" ]

The information can be extracted from the docker image by inspecting for the volumes.

            "Image": "sha256:72423fe5f27de1a495e5e875aec83fd5084abc6e1636c09d510b19eb711424cc",
            "Volumes": {
                "/mnt/my-volume": {},
                 }

The volumes defined in the Dockerfile VOLUME expression will not be visible with the docker volume ls as they are present within the scope of specific docker container. But will be displayed with anonymous hash tags in the output, But they can be shared among other docker containers. We will look at sharing the docker volumes in the following section.It is also important to understand that once the volume is explicitly deleted then its data cannot be recovered again.

Using the container volume from one container and accessing them in another container.

This option is beneficial during the times when a container access and the debug tools are disabled, and you need to view the logs of the container and run analysis on it.

Using the volume from the container and mounting it to another container for auditing purposes.
We have now created a container called view-logs which uses the volumes from another container called ngnix-with-vol

docker run -d --name view-logs --volumes-from nginx-with-vol degug-tools

Best Practices:
The view-logs container can have a set of debug and troubleshooting tools to view the logs of other app containers.

Creating a Volume from the docker commandline:

The docker volume resource has to be initialized first and can be done as follows:
Command to create a docker volume:
# docker volume create my-data-vol
[vamshi@node01 ~]$ docker volume inspect my-data-vol

[
    {
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/my-data-vol/_data",
        "Name": "my-data-vol",
        "Options": {},
        "Scope": "local"
    }
]

We can use this volume and then mount this to a container as we mounted the host volume in earlier sections.

We will be using the persistent volume name and then mounting the docker volume to a container bind volume name to the target container path..
These options have a specific changes in syntax and has to be specified exactly while running the mount operation.

Below is the syntax:
docker run –volume :</path/to/mountpoint/> image-name.

We have now the docker volume available and mounting it to the target container path /data as follows:
# docker run -d –name data-vol –volume my-data-vol:/data nginx-linuxcent:v4
2dfa965bbbc79a522e9c109ef8eee20bf47e2b61062f3b3df61d4eb677de4506

Verification:

The docker volume can be listed by the following command

Check the mount information with docker volume inspect.
Here is an extract from the docker inspect container

"Mounts": [
            {
                "Type": "volume",
                "Name": "my-data-vol",
                "Source": "/var/lib/docker/volumes/my-data-vol/_data",
                "Destination": "/var/log/nginx",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
             }

As we can see the Type of mount is depucted as a Volume here.

Conclustion:
We have seen the practices of mounting the hostpath to to target container path is a bind operation and the dependency it creates is the host affinity, which is binded to particular host and has to be avoided if you are dealing with more dynamix data exchange between containers over a network. But it is very useful if you have host-container data exchange.
The Option with Volumes is very dynamic and has less binding dependency on the host machines, They can be declared and used in two ways as demonstrated in the before sections. !st being the explicit volume creation and another is the Volume creation from within the Docker build.
The explicit docker creation and then binding provides the scopt of choosing a mount point inside the container after the image is built.

The Dockerfile’s VOLUME expression can be used to automatically define the volume name and the mount point desired and has all the same techniques of volume sharing for data exchange in run time.

What is docker container volume?

Docker volumes are file systems mounted on Docker containers to preserve data generated by the running container. The data doesn’t persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it. … The data cannot be easily moveable somewhere else.

How do I find the volume of a docker container?

We can find out where the volume lives on the host by using the docker inspect command on the host (open a new terminal and leave the previous container running if you’re following along): docker inspect -f “{{json . Mounts}}” vol-test | jq.

What is the purpose of docker volume?

docker volume create creates a volume without having to define a Dockerfile and build an image and run a container. It is used to quickly allow other containers to mount said volume.

How many volumes are there in docker?

Docker volumes are used to persist data from within a Docker container. There are a few different types of Docker volumes: host, anonymous, and, named. Knowing what the difference is and when to use each type can be difficult, but hopefully, I can ease that pain here.

How do I create a docker volume?

docker volume create

  1. Description. Create a volume. …
  2. Usage. $ docker volume create [OPTIONS] [VOLUME]
  3. Extended description. Creates a new volume that containers can consume and store data in. …
  4. Options. Name, shorthand. …
  5. Examples. Create a volume and then configure the container to use it: …
  6. Parent command. Command. …
  7. Related commands.

What is Docker desktop volume?

Docker volumes on Windows are always created in the path of the graph driver, which is where Docker stores all image layers, writeable container layers and volumes. By default the root of the graph driver in Windows is C:\ProgramData\docker , but you can mount a volume to a specific directory when you run a container.

How do I add volume to a running docker container?

Step 1 – Copy. If path, where we are going to add volume, is not empty, then make sure to copy the content to the host system, As adding a volume will overwrite container data at that location. …
Step 2 – Create new Image. …
Step 3 – Delete container. …
Step 4 – Create a new container.

How do I know my container size?

To view the approximate size of a running container, you can use the command docker container ls -s . Running docker image ls shows the sizes of your images.

Can we attach volume to running container?

To attach a volume into a running container, we are going to: use nsenter to mount the whole filesystem containing this volume on a temporary mountpoint; create a bind mount from the specific directory that we want to use as the volume, to the right location of this volume; umount the temporary mountpoint.

Linux rsync command

Linux Command Utility [/code]rsync[/code] is a very robust, fast content copy command which can be used within the same linux host and over a connected network between 2 linux hosts. It is a special program which has intelligence in terms of not copying data repetitively if the destination has the same copy as source based on file checksum calculations.

We shall explore some of the practical rsync command features and demonstrate them

Syntax of command

$ rsync [OPTIONS] /source/path/ /dest/path/

Running rsync on the same host?

[vamshi@linuxcent ~]$ rsync -avx newfile.txt /tmp/
sending incremental file list
newfile.txt

sent 133 bytes  received 35 bytes  336.00 bytes/sec
total size is 21  speedup is 0.12

Running rsync between two hosts in a network

$ rsync [OPTIONS] host:/source/path/ /dest/path/

Run rsync in Dry-run mode by using [/code]-n[/code] option

$ rsync -avn Source_host:/source/path Destination_host:/dest/path

This generally runs over the SSH protocol and you are required to enter the login credentials appropriately.

How to invoke SSH remote shell in rsync?

In case you are using a SSH keys then you have to invoke the remote shell to authenticate to the remote server with your private keypair. This is Demonstrated as follows:

$ rsync -avxn --rsh="ssh -i ~/.ssh/vamshi_id_rsa" vamshi@<Your.Source.IP.DNS>:"/<Source_Path>" "/<Destination_Path>"

For more information about the SSH key setup, Please refer to our SSH keys section

How to persist Hard links on the system using rsync. Following is the Demonstration

Flag : -H. Using this option enables to preserve the HardLinks over the destination copy of the data.

$ rsync -avHx /path/to/source/ /path/to/destination/

The most practical example of working with rsync comes in replicating mission critical data or transferring Database dumps within the DB servers etc.,

How to exclude certain directories in rsync in linux ?

using --exclude filter option is demonstrated as follows:

$ rsync -avx /source/path/to/backup-v31/ /dest/databackups/backup-v31/ --exclude="DontTouchMyData/"

Using the delete option, enables us to delete the directories from the source upon completion of the operation.

Note: This operation has the same effects as the mv command on linux but performed over the network between source and destination hosts.

$ rsync -avx --delete /source/path/to/backup-v31/ /dest/databackups/backup-v31/ --exclude="data/" --exclude="data/board" --exclude="cache/apt" --exclude="opt"

Redirect the rsync output to a file by appending output redirection symbol to a file on current location

$rsync -avx --delete /source/path/to/backup-v31/ /dest/databackups/backup-v31/ --exclude="data/" --exclude="data/board" --exclude="cache/apt" --exclude="opt"  >>/tmp/rsync.log

What is rsync command Linux?

rsync or remote synchronization is a software utility for Unix-Like systems that efficiently sync files and directories between two hosts or machines. … Copying/syncing to/from another host over any remote shell like ssh, rsh.

How do I use rsync in Linux?

Syntax of rsync command:

  • -v, u2013verbose Verbose output.
  • -q, u2013quiet suppress message output.
  • -a, u2013archive archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
  • -r, u2013recursive sync files and directories recursively.
  • -b, u2013backup take the backup during synchronization.

What is rsync in bash?

rsync is a fast and versatile command-line utility for synchronizing files and directories between two locations over a remote shell, or from/to a remote Rsync daemon. … Rsync can be used for mirroring data, incremental backups, copying files between systems, and as a replacement for scp , sftp , and cp commands.

How do I transfer files using rsync?

You can use SecureShell (SSH) or Remote Sync (Rsync) to transfer files to a remote server. Secure Copy (SCP) uses SSH to copy only the files or directories that you select. On first use, Rsync copies all files and directories and then it copies only the files and directories that you have changed.

What is rsync command do?

Rsync is typically used for synchronizing files and directories between two different systems. For example, if the command rsync local-file user@remote-host:remote-file is run, rsync will use SSH to connect as user to remote-host.

Why do we use rsync?

Syntax of rsync command:

  • -v, u2013verbose Verbose output.
  • -q, u2013quiet suppress message output.
  • -a, u2013archive archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
  • -r, u2013recursive sync files and directories recursively.
  • -b, u2013backup take the backup during synchronization.

What is rsync in RHEL?

Rsync can be used to quickly move large amounts of data to both local and remote destinations. For this reason, rsync is often used to copy data, make backups, migrate hosts, and bridge the gap between site staging and production environments.

How does rsync work in Linux?

An rsync process operates by communicating with another rsync process, a sender and a receiver. At startup, an rsync client connects to a peer process. If the transfer is local (that is, between file systems mounted on the same host) the peer can be created with fork, after setting up suitable pipes for the connection.

How do I rsync a file in Linux?

Copy a single file locally If you want to copy a file from one location to another within your system, you can do so by typing rsync followed by the source file name and the destination directory. Note: Instead of u201c/home/tin/file1. txtu201d, we can also type u201cfile1u201d as we are currently working in the home directory.

How do I use rsync?

You can use secure shell (SSH) or Remote Sync (Rsync) to transfer files to a remote server. Secure Copy (SCP) uses SSH to copy only the files or directories that you select. On first use, Rsync copies all files and directories and then it copies only the files and directories that you have changed.

How to setup RabbitMQ Queues monitoring using Nagios

Installing the Nagios RabbitMQ plugin.

 

The RabbitMQ plugin here we are going to use is validated listed under monitoring section by https://www.rabbitmq.com/monitoring.html

https://github.com/nagios-plugins-rabbitmq/nagios-plugins-rabbitmq

The installation is done as follows:

 

The Perl library can be downloaded and installed as follows:

cd /tmp/

wget http://search.cpan.org/CPAN/authors/id/T/TO/TONVOON/Nagios-Plugin-0.36.tar.gz

tar xvf Nagios-Plugin-0.36.tar.gz

cd Nagios-Plugin-0.36

perl Makefile.PL

make

make test

make install

 

In case you do not have the make, Please install it using following command to complete the make install of Nagios Plugin:

ap install make

Ofcourse the primary requirement is to have the nrpe setup installed

This can be done as follows in a ubuntu system:

# apt install nagios-nrpe-server nagios-plugins

 

Now we can go ahead and start the setup of rabbitMQ plugin.

 

cd /usr/local/nagios/libexec/

clone the repository.

git clone https://github.com/nagios-plugins-rabbitmq/nagios-plugins-rabbitmq.git

The repository already contains the executables in script directory which we will use for monitoring after completion.

We now have to provide the required perl libraries to support the rabbitmq-nagios plugins and execute them.

Lets us now proceed and organize the directory structure and move directory nagios-plugins-rabbitmq to /usr/local/nagios/libexec/

# mv nagios-plugins-rabbitmq /usr/local/nagios/libexec/

Change the ownership to nagios user, This step is important and must to done as a prerequisite,

ensure that you have the desired nagios user already present by running the below command

# id nagios

Change the ownership to nagios user and group.

# sudo chown nagios:nagios nagios-plugins-rabbitmq -R

The files and directories now should be owned by the nagios user and group

Now we can go ahead and install the required modules from cpan.

Run the cpan command to fetch and install perl dependencies

 

On some systems you might face some problems while trying to install ” install Monitoring::Plugin ”

cpan[1]> install Monitoring::Plugin

You may try this

cpan[1]> install Nagios::Monitoring::Plugin

After installing it, Next install LWP

cpan[2]> install LWP

After installing it, Next install JSON

cpan[3]> install JSON

Type exit to exit out of the cpan prompt.

cpan[4]> exit

Terminal does not support GetHistory.
Lockfile removed.

 

Alternatively you may also run the cpanm and avoid entering into the cpan subshell and issue the perl install commands

# apt install cpanminus

You may also use this to install Nagios::Plugin

After the installation is completed without any errors you would have successfully installed the RabbitMQ nagios plugins.

Now check the installation by going to scripts directory

# /usr/local/nagios/libexec/nagios-plugins-rabbitmq/scripts

Lets look at what plugins are provided for rabbitmq

/usr/local/nagios/libexec/nagios-plugins-rabbitmq/scripts# ls
check_rabbitmq_aliveness check_rabbitmq_connections check_rabbitmq_objects check_rabbitmq_partition check_rabbitmq_server check_rabbitmq_watermark check_rabbitmq_cluster check_rabbitmq_exchange check_rabbitmq_overview check_rabbitmq_queue check_rabbitmq_shovels

As most of them are set with executable bit, run any check by giving absolute path or relative prefixing ./check_rabbitmq_aliveness

You can pass -h to print the help menu

The commands shall take the mandatory option of Hostname Port followed by the login credentials -u username and -p password

[email protected]:/usr/local/nagios/libexec/nagios-plugins-rabbitmq/scripts# sudo ./check_rabbitmq_server -H “RabbitMQ1.linuxcent.com” -u “guest” -p “guest”

Output

RABBITMQ_SERVER OK – Memory OK (2.63%) Process OK (0.08%) FD OK (0.27%) Sockets OK (0.10%) | Memory=2.63%;80;90 Process=0.08%;80;90 FD=0.27%;80;90 Sockets=0.10%;80;90

 

Similarly the other important plugin is to get the queue status.

This plugin provides various checks on the RabbitMQ

The most useful checks are monitoring and alerting the queue names on features such as the Messages in Queue, Messages in Ready State, Un-Acknowledged messages and the number of Consumers.

The order for the Warning and Critical are

messages ,messages_ready ,messages_unacknowledged ,consumers

Here we are monitoring the RabbitMQ queue download_queue, for the messages_ready if exceeding 0 then, warning notification will be sent.

[email protected]:/usr/local/nagios/libexec/nagios-plugins-rabbitmq/scripts# ./check_rabbitmq_queue -H “RabbitMQ1.linuxcent.com” -u “guest” -p “guest” –queue “download_queue” -w -1,0,-1,-1

Output

RABBITMQ_QUEUE CRITICAL – messages_ready CRITICAL (4219), messages OK (4269) messages_unacknowledged OK (50) consumers OK (2) | messages=4269;; messages_ready=4219;;0 messages_unacknowledged=50;; consumers=2;;

 

Linux Commands Cheat Sheet

Do you want to learn the Linux commands? If yes, then you are in the right place. There are uncountable commands in Linux. Generally, we use some of the Linux commands from the Linux commands cheat sheet regularly to perform the common tasks.

pythonarray.com is a free Python tutorial website for people who want to learn Python, fast.

You can also learn Google Sheets Tips – New Google Spreadsheet Hacks, Tricks with Examples

Types of Linux Commands

All the Linux commands are categorized into 7 different types. They are Basic Linux commands, File permission commands, Environment variables commands, User management commands, Networking commands, Process commands, VI Editing commands, Other Commands.

We are giving a detailed explanation for each and every Linux command with their examples for easy learning in the below sections. Have a look at them and follow them. Check the actual description of each Linux command in their manual page.

Basic Linux Commands

  • ls command:

It is used to list all files and directories of a directory. The ‘-l’ option gives the long listing format. Its syntax is $ ls [options] [file|dir]

Example: $ ls -l samplefile

  • ls-R command:

It gives the lists files in subdirectories. List recursively directory tree and syntax is $ ls -R [options] [file|dir]

Example: $ ls -R samplefile

  • ls -a Command:

It lists all files including hidden files starting with’.’ Its syntax is $ ls -a [options] [file|dir]

  • ls -al Command:

It lists all files details and directories with detailed information like permissions, size, owner, etc. Syntax is $ ls -al [options] [file|dir]

  • cd or cd ~ Command:

This command will navigate to the home directory or change the directory. Its syntax is $ cd [directory]

Example: $ cd Directory1

  • cd .. Command:

It moves one level up. Syntax is $ cd . . [directory]

  • cd Command:

Use this command to change to a particular directory. Its syntax is $ cd [directory]

Example: $ cd directory1

  • cd / Command:

This command will move to the root directory. Syntax is $ cd /

  • cat filename Command:

Create a new file using this command in Linux. Its syntax is $ cat filename.

Example: $ cat filename1

  • cat filename Command:

It displays the content in that particular file. The syntax is $ cat filename.

Example: $ cat filename1.

  • cat file1 file2 file3 Command:

It combines two files namely file1 and file2 and stores the output in a new file called file3. Its syntax is $ cat file1 file2 file3.

Example: $ cat add multiply arithmetic.

  • mv file “new file path” Command:

It moves the files to the new location. Syntax is $ mv file “new file path”.

Example: $ mv add “E:/Arithmeticopeartions/”.

  • mv filename new_file_name Command:

It renames or changes the file name to a new file name. Syntax is $ mv filename new_file_name.

Example: $ mv add sum

  • sudo Command:

sudo command allows a permitted user to run a command as root or another user. It runs commands in superuser mode. The syntax is $ sudo filename option

Example: $ sudo apt update

  • rm Command:

It is used to remove the files or directories. Syntax is $ rm filename or $ rm directory

Examples: $ rm sum and $ rm mydrive

  • man Command:

It is used to view the online reference manual pages for the commands or programs. It provides the help information of any commands. Its syntax is $ man commandname

Example: $ man cd

  • history Command:

It produces the list of previously used commands and gets the all past commands types in the current terminal session. The syntax is $ history

  • clear Command:

clear command is helpful to clear the terminal screen. The syntax is $ clear

  • mkdir directoryname Command:

It creates a new directory in the present working directory or at any specific path. Syntax is $ mkdir directoryname

Example: $ mkdir new_directory

$ mkdir -p new_directory: Here -p option is used to override the already available directory.

  • rmdir Command:

It helps to remove/ delete the empty directories. Syntax is $ rmdir [directory_name]

Example: $ rmdir new_directory

  • mv Command:

It is helpful to rename files or directories. mv command also moves a file or a directory to another location in the directory structure. Its syntax is $ mv file1 file2

Example: $ mv add sum

  • pr -x Command:

It divides the file into x columns. Syntax is $ pr -x filename.

Example: $ pr -5 add

  • pr -h Command:

It is useful to assign a header to the file. Syntax is $ pr -h filename

Example: $ pr -h sum

  • pr -n Command:

This command denotes the file with the line numbers. Its syntax is $ pr -n file

Example: $ pr -n sum

  • lp -nc , lpr c Command:

It prints the c number of copies of a file. lp stands for a line printer. Its syntax is $ lpr c filename

Example: $ lpr 5 sum

  • lp -d lp -P Command:

It specifies the name of the printer. The syntax is $ lp -P

  • apt-get Command:

It is a powerful and free front end package manager for Debian/Ubuntu systems. It is used to install new software packages, remove available software packages, upgrade existing software packages, and upgrade the entire operating system. Its syntax is $ apt-get package

Example: $ sudo apt-get update

  • mail -s ‘subject’ -c ‘cc-address’ -b ‘bcc-address’ ‘to-address’ Command:

It is a linux command used to send mail. Syntax is $ mail -s ‘subject’ -c ‘cc-address’ -b ‘bcc-address’ ‘to-address’

Example: $ mail -s ‘regarding_job’ -c ‘[email protected]’ -b ‘[email protected]’ ‘[email protected]

  • mail -s “Subject” to-address < Filename Command:

It is used to send an email with an attachment. Its syntax is $ mail -s “Subject” to-address < Filename

Example: $ mail -s “Addition of Numbers” [email protected] < Addition

File Permission Commands

  • ls-l Command:

Use this command to show file type and access permission of any particular file. Syntax is $ ls-l filename

Example: $ ls-l sum

  • r Command:

It is used to give read permission to a file. The syntax is $ r filename.

Example: $ r sum

  • w Command:

It gives write permission to the file. The syntax is $ w filename

Example: $ w sum

  • x Command:

It is useful to provide execute permission to the file. Its syntax is $ x filename

Example: $ x sum

  • -= Command:

This command is used when you want to give no permission to a file. Syntax is $ -= filename

Example: $ -= file1

  • Chown user Command:

It is helpful to change the ownership of a file/directory. The syntax is $ chown user filename

Example: $ chown user file1

  • Chown user:group filename Command:

It changes the user and group for a file or directory. Syntax is $ chown user:group filename

Example: $ chown user:group sum

Environment Variables Command

  • echo $VARIABLE Command:

It displays the value of a variable or prints a text of the line. Its syntax is echo [option] [string]

Example: $ echo “Print this line” or $ echo $VARIABLE var1

  • env Command:

It lists all environment variables. The syntax is $ env

  • VARIABLE_NAME= variable_value Command:

It creates a new variable and assigns it a value. Syntax is $ VARIABLE_NAME= variable_value

Example: $ My_Variable= 10

  • Unset Command:

It removes a variable. Syntax is $ Unset VARIABLE_NAME

Example: $ Unset My_Variable

  • export Variable=value Command:

It is helpful to set the value of an environment variable. Syntax is $ export Variable=value

Example: $ export Variable1=90

User Management Commands of Linux

  • sudo adduser username Command:

It is used to create a username. The syntax is $ sudo adduser username

Example: $ sudo dduser user1

  • sudo passwd -l ‘username’ Command:

It is helpful to change the user password. Syntax is $ sudo passwd -l ‘username’

Example: $ sudo passwd -l ‘user1’

  • sudo userdel -r ‘username’ Command:

This command is used to add and delete users on Linux. Syntax is $ sudo userdel -r ‘username’

Example: $ sudo userdel -r ‘user1’

  • finger Command:

It is a user information lookup command which provides details of all users logged in. The Syntax $ finger

  • finger username Command:

It gives the login details of that particular user. Syntax $ finger username

Example: $ finger user1

Networking Commands

  • SSH username@ip-address or hostname Command:

You can log in to a remote Linux machine using SSH. The syntax is $ SSH username@ip-address or hostname

Example: $ SSH user1@https:11//1001

  • Ping hostname=”” or =”” Command:

It is used for analyzing the network and host connections. Syntax is $ Ping hostname=”” or =””

Example: $ Ping hostname=”host1″

  • dir Command:

It displays all files in the current directory of a remote computer. The syntax is $ dir

  • cd “dirname” Command:

It changes the current directory to “dirname” on a remote computer. Syntax is $ cd “dirname”

Example: $ cd “E”

  • put file Command:

It is helpful to upload the file from local to the remote computer. The syntax is $ put file

Example: $ put sum

  • get file Command:

It is used to download the file from remote to the local computer. The syntax is $ get file

Example: $ get add

  • quit Command:

It is used to logout from that network. The syntax is $ quit

Process Commands

  • bg Command:

You can send a process to the background with this command. The syntax is $ bg processname

Example: $ bg process1

  • fg Command:

To run a stopped process in the foreground. The syntax is $ fg processname.

Example: $ fg process1

  • top Command:

It gives the details of all active processes in the directory. The syntax is $ top

  • ps Command:

It produces the status of processes running for a user. The syntax is $ ps username

Example: $ ps user1

  • ps PID Command:

This command gives the status of a particular process. The syntax is $ ps PID processname

Example: $ ps PID process1

  • pidof Command:

It gives the process id of a process. The syntax is $ pidof processname.

Example: $ pidof process1

  • kill PID Command:

It kills a process. The syntax is $ kill PID processname

Example: $ kill PID process1

  • nice Command:

It is used to start a process with the given priority. The syntax is $ nice processname.

Example: $ nice process

  • renice Command:

It is used to change the priority of an already running process. The syntax is $ renice processname

Example: $ renice process2

  • df Command:

It provides free hard disk space on your system. The syntax is $ df

  • free Command:

It gives free RAM space on your system. The syntax is $ free

VI Editing Commands

  • i Command:

It inserts at the cursor i.e goes to insert mode. The syntax is $ i

  • a Command:

It is used to write after the cursor in insert mode. The syntax is $ a

  • A Command:

This command is used to write at the end of the line in insert mode. The syntax is $ A

  • ESC Command:

It is helpful to terminate insert mode. The syntax is $ ESC.

  • u Command:

It undoes the last change. The syntax is $ u.

  • U Command:

It undoes all changes to the entire line. The syntax is $ U

  • o Command:

It opens a new line in insert mode. The syntax is $ o

  • dd Command:

It deletes that particular line. The syntax is $ dd line number

Example: $ dd 25

  • 3dd Command:

This command deletes 3 lines. The syntax is $ 3dd line number

Example: $ 3dd 25

  • D Command:

It deletes the contents of a line after the cursor. The syntax is $ D

  • C Command:

It is used to delete the contents of a line after the cursor and insert new text. Press the ESC key to end the insertion. Syntax is $ C

  • dw Command:

Delete a word using this command. The syntax is $ dw

  • 4dw Command:

It is used to delete 4 words. The syntax is $ 4dw

  • cw Command:

It is used to change a word. Syntax is $ cw

  • x Command:

It deletes the character at the cursor. Syntax is $ x

  • r Command:

Replace the character. The syntax is $ r

  • R Command:

It overwrites characters from cursor onward. The syntax is $ R

  • s Command:

Substitute one character under the cursor and continue to insert. The syntax is $ s

  • S Command:

Substitutes the entire line and starts to insert at the beginning of the line. The syntax is $ S

  • ~ Command:

It is used to change the case of the individual character. Syntax is $ ~

Other Linux Commands

  • adduser/addgroup Command:

adduser adds a user and addgroup adds groups to the system. The syntax is $ adduser username

Example: $ sudo adduser tech

  • agetty Command:

agetty is a program that manages physical or virtual terminals and is invoked by init. It is the substitute for Linux getty.

Example: $ agetty -L 9600 ttyS1 vt100

  • alias Command:

It is a useful shell built-in command for creating shortcuts to a Linux command on a system.

Example: $ alias home’cd /home/drive/file1′

  • apropos Command:

This is used to search and display a short man page description of a program/ command. The syntax is $ apropos commandname

Example: $ apropos rm

  • aptitude Command:

It is a powerful text-based interface to the Debian Linux/GNU package management system. The syntax is $ aptitude package

Example: $ sudo aptitude update

  • arch Command:

This command is used to display the machine architecture or hardware name. The syntax is $ arch

  • arp Command:

ARP stands for Address Resolution Protocol that maps IP network addresses of a network neighbor with the hardware (MAC) addresses in an IPv4 network.

Example: $ sudo arp-scan –interface=enp2s0 –localnet

  • at Command:

It is used to schedule tasks to run in the forthcoming time. Syntax is $ at

Example: $ sudo echo “shutdown -h now” | at -m 15:20

  • atq Command:

This command is used to view jobs in at command queue. Syntax is $ atq

  • atrm Command:

It is helpful to delete/remove jobs from the queue. $ atrm 2

  • awk Command:

Awk is a programming language created for text processing and used as a data extraction and reporting tool.

Example: $ awk ‘//{print}’/etc/hosts

  • batch Command:

It is similar to at command which is useful to schedule tasks to run in a future time. Syntax is $ batch

  • basename Command:

It helps to print the name of a file stripping of directories in absolute path. Syntax is $ basename filename

Example: $ basename samplefile

  • bzip2 Command:

It is useful to compress or decompress the files. Syntax is $ bzip2 filename

Example: $ bzip2 -z filename #Compress

$ bzip2 -d filename.bz2 #Decompress

  • cal Command:

This command is used to print a calendar on the standard output. Syntax is $ cal

  • chgrp Command:

It is used to change the group ownership of a file. Syntax is $ chgrp new_group_name filename

Example: $ chgrp Product Multiplication.txt

  • chmod Command:

It is used to change/ update file access permissions. Syntax is $ chmod permission filename

Example: $ chmod +x product.txt

  • cksum Command:

This command is used to provide the checksum and byte count of the given input file. Syntax is $ cksum filename

Example: $ cksum product.txt

  • cmp Command:

It compares byte by byte comparison of two files. Syntax is $ cmp file1 file2

Example: $ cmp sum add

  • comm Command:

It is used to compare two sorted files line by line. Syntax is $ comm file1 file2

Example: $ comm product multiplication

  • cp Command:

It copies files and directories from one location to another. Syntax is $ cp location1 location2

Example: $ cp /home/arithmetic/add /home/arithmetic/product/

  • date Command:

It displays/ sets the system date and time. Syntax is $ date

Example: $ date –set=”20 Oct 2020 10:35:05″

  • diff Command:

The double advantage of this command is used to compare two files line by line and used to find the difference between two directories. Syntax is $ diff file1 file2

Example: $ diff sum add

  • dmidecode Command:

It retrieves the hardware information of any linux system. Syntax is $ dmidecode

Example: $ sudo dmidecode –type system

  • du Command:

It shows the disk space usage of files present in the directory as well as its subdirectories. The syntax is $ du directory

Example: $ du /home/arithmetic

  • eject Command:

eject command is used to eject removable media such as CD/DVD/ROM/floppy disk from the system. The syntax is $ eject disklocation

Example: $ eject /dev/cdrom

  • exit Command:

It is used to exit a shell. The syntax is $ exit

  • expr Command:

It is used to calculate an expression. The syntax is $ expr expression

Example: $ expr 15 * 5

  • factor Command:

It is used to display the prime factors of a number. The syntax is $ factor number

Example: $ factor 18

  • find Command:

It searches for files in the directory and its subdirectories. The syntax is $ find directory filename

Example: $ find /home/add/ -name product.txt

  • grep Command:

It searches for a specified pattern in the file and displays the output as lines containing that pattern. The syntax is $ grep

Example: $ grep ‘linuxcent’ domain-list.txt

  • groups Command:

Shows all names of groups of a user. The syntax is $ groups

  • gzip Command:

It helps to compress a file and replaces it with one having a .gz extension. The syntax is gzip filename

Example: $ gzip passwds.txt

  • gunzip Command:

It expands or restores files compressed with the gzip command. The syntax is $ gunzip gzfilename

Example: $ gunzip sum.gz

  • head Command:

It is helpful to display the first 10 lines of the specified file or stdin to the screen. The syntax is $ head

  • hostname Command:

This command is useful to print or set the system hostname. The syntax is $ hostname

Example: $ hostname New_hostname

  • id Command:

It shows user & group information for the current user or specified user. The syntax is $ id username

Example: $ id linuxcent

  • ifconfig Command:

It is used to configure Linux system network interfaces. The syntax is $ ifconfig

  • ionice Command:

This command is used to set or view process I/O scheduling class and priority of the specified process. The syntax is $ ionice process

Example: $ ionice -c 3 rm /var/logs/syslog

  • iostat Command:

iostat command is helpful to show the CPU and i/o statistics for devices and partitions. The syntax is $ iostat

  • ip Command:

It is used to display or manage devices, routing, policy routing, and tunnels. The syntax is $ ip addr

  • iw Command:

This is used to manage wireless devices and their configuration. The syntax is $ iw

Example: $ iw list

  • killall Command:

It kills a process with the help of its name. The syntax is $ killall processname

Example: $ killall process1

  • kmod Command:

It is useful to manage Linux kernel modules. To list all currently loaded modules, type. Syntax is $ kmod

Example: $ kmod list

  • last Command:

It displays a list of last logged in users. The syntax is $ last

  • ln Command:

This is used to create a soft link between files using -s flag. Syntax is $ ln -s file1 file2

Example: $ ln -s product multiplication

  • locate Command:

It is used to find a file with its name. The syntax is $ locate filename

Example: $ locate -b ‘add.txt’

  • lshw Command:

It gives detailed information on the hardware configuration of the machine. The syntax is $ lshw

  • lscpu Command:

This Linux command displays the system’s CPU architecture information. The syntax is $ lscpu

  • lsof Command:

It displays information related to files opened by processes. The syntax is $ lsof processname

Example: $ lsof -u add

  • lsusb Command:

It shows information: about USB buses in the system and devices connected to them. The syntax is $ lsusb

  • md5sum Command:

This command is used to compute and print the MD5 message digest of the file.

  • more Command:

It enables to view through relatively lengthy text files one screenful at a time. The syntax is $ more filename

Example: $ more add

  • nano Command:

You can open a file using this nano command. The syntax is $ nano filename

Example: $ nano sum

  • netstat Command:

It displays useful information such as routing tables, network connections, and others about Linux networking subsystems. Syntax is $ netstat

Example: $ netstat -a | more

  • nproc Command:

It shows the number of processing units present to the current process. The syntax is $ nproc

  • passwd Command:

This command is used to create/update passwords for user accounts. The syntax is $ passwd username

Example: $ passwd user1

  • pstree Command:

It displays running processes as a tree that is rooted at either PID or init if PID is omitted. The syntax is $ pstree

  • reboot Command:

This command is used to power-off, halt, or reboot a system. The syntax is $ reboot

  • scp Command:

It enables you to securely copy files between hosts on a network. Syntax is $ scp

Example: $ scp ~/names.txt [email protected]:/root/names.txt

  • shutdown Command:

It schedules a time for the system to be powered down. Its syntax is $ shutdown

Example: $ shutdown –poweroff

  • sleep Command:

This command is used to pause or delay for a specific amount of time. The syntax is $ sleep

Example: $ check.sh; sleep 5; sudo apt update

  • sort Command:

This is used to sort lines of text in the specified files from stdin. The syntax is $ sort

Example: $ sort add.txt

  • split Command:

It is used to split a large file into smaller pieces. The syntax is $ split filename

Example: $ split add

  • stat Command:

It is helpful to know the file system status. The syntax is $ state filename

Example: $ stat -f add.txt

  • su Command:

It is used to switch to another user ID or become root during a login session. The syntax is $ su

  • sum Command:

This command shows the checksum and blocks counts for all files on the command line. The syntax is $ sum output filename

Example: $ sum output add.txt

  • tac Command:

It concentrates and displays files in reverse. It prints each file to the standard output, shows the last line first. The syntax is $ tac filename

Example: $ tac sum.txt

  • tail Command:

It displays the last 10 lines of each file. The syntax is $ tail filename

Example: $ tail sum.txt

  • talk Command:

talk is used to talk to another network user/ system. Use a login name to talk to the user on the same machine and use ‘use@host’ to talk to a user on another machine. The syntax is $ talk

Example: $ talk person [ttyname] or $ talk ‘user@host’ [ttyname]

  • tee Command:

This is used to read from standard input & prints to standard output and files.

Example: $ echo “Testing how tee command works” | tee file1

  • time Command:

This command runs programs and summarizes system resource usage. The syntax is $ time

  • touch Command:

It changes file timestamps and it is used to create a file. The syntax is $ touch filename

  • uname Command:

It displays system information such as operating system, network node hostname kernel name, version, and release, etc. Syntax is $ uname

  • uniq Command:

It displays or omits repeated lines from input. The syntax is $ uniq

  • uptime Command:

It shows how long the system has been running, the number of logged on users, and system load averages. The syntax is $ uptime

  • wall Command:

This command is used to send/display a message to all users on the system. The syntax is $ wall “message”

Example: $ wall “This is Linux operating system”

  • wc Command:

This command displays the word, newline, and byte counts for each file. The syntax is $ wc filename

Example: $ wc add.txt

  • whatis Command:

It searches & shows a short or one-line manual page description of the provided command name. The syntax is $ whatis wget

  • who Command:

It shows details about users who are currently logged in. The syntax is $ who

  • which Command:

This command is used to get the absolute path of the file which would be executed in the current environment. The syntax is $ which who

  • whereis Command:

It helps to locate binary, source and manual files for commands. The syntax is $ whereis cat

  • yes Command:

This command is used to display a string repeatedly until when terminated or killed. The syntax is 4 yes

Example: $ yes “This is LinuxCent – Linux HowTos”

  • zcmp/zdiff Command:

zcmp and zdiff minimal utilities used to compare compressed files. Syntax is $ zcmp filesnames, $ zdiff filesnames

Example: $ zcmp domain-list.txt.zip basic_passwords.txt.zip

$ zdiff domain-list.txt.zip basic_passwords.txt.zip

  • zip Command:

It is simple & easy to use utility used to package & compress files. Syntax is $ zip packagename

Example: $ zip inarchive.zip foo.c bar.c –out outarchive.zip

We have mentioned all the Linux commands in the Linux Commands Cheatsheet. Make use of these countless commands that you might feel helpful.

What are 5 Linux commands?

Linux File commands

  • touch Command. The touch command is used to create empty files. …
  • cat Command. The cat command is a multi-purpose utility in the Linux system. …
  • rm Command. The rm command is used to remove a file.
  • cp Command. The cp command is used to copy a file or directory.
  • mv Command. …
  • rename Command.

How do I see all commands in Linux?

In Linux, there is a very useful command to show you all of the last commands that have been recently used. The command is simply called history, but can also be accessed by looking at your . bash_history in your home folder. By default, the history command will show you the last five hundred commands you have entered.

What is Linux cheat sheet?
Basic Linux commands

What is Linux cheat sheet?

How many commands are in Linux?

There are well over 100 Unix commands shared by the Linux kernel and other Unix-like operating systems.

What does F do in Linux?

Many Linux commands have an -f option, which stands for, you guessed it, force! Sometimes when you execute a command, it fails or prompts you for additional input. This may be an effort to protect the files you are trying to change or inform the user that a device is busy or a file already exists.

How do I get a list of commands?

You can open the Command Prompt by pressing ⊞ Win + R to open the Run box and typing cmd. Windows 8 users can also press ⊞ Win + X and select Command Prompt from the menu. Retrieve the list of commands.

How do you list all commands in Terminal?

Just tap the Tab key twice ( Tab Tab ). You’ll be prompted if you want to see all possible commands. Tap y and you’ll be presented with a list. You can do that same thing for individual commands to see all options for that specific command.

How do you write commands in Linux?

Linux operating system allows users to create commands and execute them over the command line. To create a command in Linux, the first step is to create a bash script for the command. The second step is to make the command executable. Here, bashrc means run the Bash file.

What does Linux dd command do?

dd is a command-line utility for Unix and Unix-like operating systems, the primary purpose of which is to convert and copy files. … As a result, dd can be used for tasks such as backing up the boot sector of a hard drive, and obtaining a fixed amount of random data.

What is Yum in Linux?

YUM is the primary package management tool for installing, updating, removing, and managing software packages in Red Hat Enterprise Linux. YUM performs dependency resolution when installing, updating, and removing software packages. YUM can manage packages from installed repositories in the system or from

What is Sudo in Linux?

Sudo stands for either “substitute user do” or “super user do” and it allows you to elevate your current user account to have root privileges temporarily. This is different from “su” which is not temporary.

How use Linux command line?

On many systems, you can open a command window by pressing the Ctrl+Alt+t keys at the same time. You will also find yourself on the command line if you log into a Linux system using a tool like PuTTY. Once you get your command line window, you’ll find yourself sitting at a prompt.

What is Ubuntu command?

The latest LTS release is 20.04; it will be available till 2025. Ubuntu supports both Command Line Interface (CLI) and Graphical User Interface (GUI) to perform various tasks on the OS. CLI is the basic way to interact with systems hardware (processor/memory); you can perform all the tasks that GUI can perform.

Where is Linux command?

The whereis command in Linux is used to locate the binary, source, and manual page files for a command. This command searches for files in a restricted set of locations (binary file directories, man page directories, and library directories).

How To Restart Jenkins Safely

Jenkins provides the Frontend User interface and the API to access the jenkins servers and API calls also can be sent from the URL

Process to restart jenkins server safely

Here is our jenkins server hosted on our url: http://jenkins.linuxcent.com:8080

And the API request to restart Jenkins safely is to run http://YourJenkins-url-or-ip/safeRestart
http://jenkins.linuxcent.com:8080/safeRestart
See the below screenshot for more information.

This option is reliable as the restart operation will wait for the currently running jobs to complete and then proceed with restart

Safe Restart jenkins from UI API

Force restart option in jenkins

http://jenkins.linuxcent.com:8080/restart
This option will restart the Jenkins forcefully and the currently running jobs will be subjected for force termination.
Forcefully Restart jenkins from UI API

Restart jenkins server from commandline

Through the command you can initiate the restart command, but this will be a forceful restart of Jenkins server.

It will be stopping and starting the jenkins server from commandline, although you can run the stop and then start with same results.

[vamshi@jenkins jenkins]$ sudo systemctl restart jenkins

On older systemv servers you can also initiate the restart using service command

[vamshi@jenkins jenkins]$ sudo service jenkins restart

How do you restart Jenkins?

Go to the Jenkins installation, open the cmd and run:

  • To stop: jenkins.exe stop.
  • To start: jenkins.exe start.
  • To restart: jenkins.exe restart.

Is the command used to restart Jenkins manually?

To restart Jenkins manually, you can use either of the following commands (by entering their URL in a browser). jenkins_url/safeRestart – Allows all running jobs to complete. … jenkins_url/restart – Forces a restart without waiting for builds to complete.

What is the command to restart Jenkins service on Windows?

  • To stop: jenkins.exe stop.
  • To start: jenkins.exe start.
  • To restart: jenkins.exe restart.

How long does it take to restart Jenkins?

I also restarted the Jenkins service and it worked. It did take 3-4 minutes after I restarted the service for the page to load up, though. So make sure you’re patient before moving on to something else.

How do I restart Jenkins in Kubernetes?

Just kubectl delete pods -l run=jenkins-ci – Will delete all pods with this label (your jenkins containers). Since they are under Deployment, it will re-create the containers. Network routing will be adjusted automatically (again because of the label selector).

How do I set Jenkins to restart itself?

Jenkins -> Manage Jenkins -> Manage Plugins -> Search for Safe Restart -> Install it. Then Restart Safely appear on the Dashboard.

How do I start Jenkins on port 8080?

  • Go to the directory where you installed Jenkins (by default, it’s under Program Files/Jenkins)
  • Open the Jenkins.xml configuration file.
  • Search –httpPort=8080 and replace the 8080 with the new port number that you wish.
  • Restart Jenkins for changes to take effect.

How do I start Jenkins on Mac?

Terminal and Start / Stop daemon
Start Jenkins: sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist.
Stop Jenkins: sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist.

How do I run Jenkins job daily?

The steps for schedule jobs in Jenkins:

  • click on “Configure” of the job requirement.
  • scroll down to “Build Triggers” – subtitle.
  • Click on the checkBox of Build periodically.

How do I open Jenkins?

To start Jenkins from command line

  1. Open command prompt.
  2. Go to the directory where your war file is placed and run the following command: java -jar jenkins.war.

How to get the docker container ip ?

The metadata information from the docker containers can be extracted using the docker inspect command.
We see the demonstration as follows:

The docker engine api is based around the golang templates and the commands use extensive formatting around the json function definitions.

[vamshi@node01 ~]$ docker inspect <container-name | container-id> -f '{{ .NetworkSettings.IPAddress }}'
172.17.0.2
[vamshi@node01 ~]$ docker inspect my-container --format='{{ .NetworkSettings.IPAddress }}'
172.17.0.2