Uncategorized

In this tutorial we will quickly setup java on linux centos,

We will be using the yum command to download the openjdk 1.8 and install

[vamshi@node01 ~]$ sudo yum install java-1.8.0-openjdk.x86_64

We have installed the java openjdk 1.8 and we can check the version using java -version

[vamshi@node01 ~]$ java -version
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

 

We make use of the alternatives command in centos which lists if we have any other version of java installed on the machine, and then enabling the default java version on the system wide.

[vamshi@node01 ~]$ alternatives --list | grep java
java auto /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/java
jre_openjdk auto /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre
jre_1.8.0 auto /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre
jre_1.7.0 auto /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.261-2.6.22.2.el7_8.x86_64/jre

[vamshi@node01 ~]$ sudo alternatives --config java

There are 2 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
*  1           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/java)
 + 2           java-1.7.0-openjdk.x86_64 (/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.261-2.6.22.2.el7_8.x86_64/jre/bin/java)

Enter to keep the current selection[+], or type selection number: 1

This enabled openjdk1.8 to be the default version of java.

Setting JAVA_HOME path
In order to set the system JAVA_HOME path on the system we need to export this variable, for the obvious reasons of other programs and users using the classpath such as while using maven or a servlet container.

Now there are two levels we can setup the visibility of JAVA_HOME environment variable.
1. Setup JAVA_HOME for single user profile
We need to update the changes to the ~/.bash_profile

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/

PATH=$PATH:$JAVA_HOME

export PATH

Now we need enforce the changes with reloading the .bash_profile with a simple logout and then login into the system or we can source the file ~/.bash_profile as follows:

[vamshi@node01 ~]$ source .bash_profile

Verifying the changes:

[vamshi@node01 ~]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/vamshi/.local/bin:/home/vamshi/bin:/home/vamshi/.local/bin:/home/vamshi/bin:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/

2. Setup JAVA_HOME for the system wide profile and available to all the users.

[vamshi@node01 ~]$ sudo sh -c "echo -e 'export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/' > /etc/profile.d/java.sh"

This echo command writes the JAVA_HOME path to the system profile.d and creates a file java.sh which is read system wide level.

Ensure the changes are written to /etc/profile.d/java.sh

[vamshi@node01 ~]$ cat /etc/profile.d/java.sh
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/

Now source to apply the changes immediately to the file /etc/profile.d/java.sh as follows

[vamshi@node01 ~]$ sudo sh -c ' source /etc/profile.d/java.sh '

Or login to the root account and run the source command

Ensure to run the env command

[vamshi@node01 ~]$ env  | grep JAVA_HOME
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/bin/

How do I download and install Java on CentOS?

Install Java On CentOS

1. Install OpenJDK 11. Update the package repository to ensure you download the latest software: sudo yum update. …
2. Install OpenJRE 11. Java Runtime Environment 11 (Open JRE 11) is a subset of OpenJDK. …
3. Install Oracle Java 11. …
4. Install JDK 8. …
5. Install JRE 8. …
6. Install Oracle Java 12.

Is Java installed on CentOS?

OpenJDK, the open-source implementation of the Java Platform, is the default Java development and runtime in CentOS 7. The installation is simple and straightforward.

How do I install Java on Linux?

• Java for Linux Platforms
• Change to the directory in which you want to install. Type: cd directory_path_name. …
• Move the . tar. gz archive binary to the current directory.
• Unpack the tarball and install Java. tar zxvf jre-8u73-linux-i586.tar.gz. The Java files are installed in a directory called jre1. …
• Delete the . tar.

How do I install latest version of Java on CentOS?

To install OpenJDK 8 JRE using yum, run this command: sudo yum install java-1.8. 0-openjdk.

Where is java path on CentOS?

They usually reside in /usr/lib/jvm . You can list them via ll /usr/lib/jvm . The value you need to enter in the field JAVA_HOME in jenkins is /usr/lib/jvm/java-1.8.

How do I know if java is installed on CentOS 7?

• To check the Java version on Linux Ubuntu/Debian/CentOS:
• Open a terminal window.
• Run the following command: java -version.
• The output should display the version of the Java package installed on your system. In the example below, OpenJDK version 11 is installed.

Where is java path set in Linux?

Steps

• Change to your home directory. cd $HOME.
• Open the . bashrc file.
• Add the following line to the file. Replace the JDK directory with the name of your java installation directory. export PATH=/usr/java/<JDK Directory>/bin:$PATH.
• Save the file and exit. Use the source command to force Linux to reload the .

How do I install java 14 on Linux?

Installing OpenJDK 14

• Step 1: Update APT. …
• Step 2: Download and Install JDK Kit. …
• Step 3: Check Installed JDK Framework. …
• Step 4: Update Path to JDK (Optional) …
• Step 6: Set Up Environment Variable. …
• Step 7: Open Environment File. …
• Step 8: Save Your Changes.

How do I know where java is installed on Linux?

This depends a bit from your package system … if the java command works, you can type readlink -f $(which java) to find the location of the java command. On the OpenSUSE system I’m on now it returns /usr/lib64/jvm/java-1.6. 0-openjdk-1.6. 0/jre/bin/java (but this is not a system which uses apt-get ).

How do I install java 11 on Linux?

Installing the 64-Bit JDK 11 on Linux Platforms

1. Download the required file: For Linux x64 systems: jdk-11. interim. …
2. Change the directory to the location where you want to install the JDK, then move the . tar. …
3. Unpack the tarball and install the downloaded JDK: $ tar zxvf jdk-11. …
4. Delete the . tar.

Facing an issue with pulling the repository while dealing with an expired SSL certificate.

[vamshi@workstation ~]$ git pull https://gitlab.linuxcent.com/linuxcent/pipeline-101.git
fatal: unable to access 'https://gitlab.linuxcent.com/linuxcent/pipeline-101.git/': Peer's Certificate has expired.
[vamshi@workstation ~]$

SSL error while cloning git URL

If you have faced the error, then we can work around it by ignoring SSL certificate check and continue working with the git repo.

[vamshi@workstation ~]$ git clone https://gitlab.linuxcent.com/linuxcent/pipeline-101.git
Cloning into 'pipeline-101'...
fatal: unable to access 'https://gitlab.linuxcent.com/linuxcent/pipeline-101.git/': Peer's Certificate has expired.

It doesn’t allow the clone or pull or push to the gitlab website as its certificate is not valid, and the certificate is unsigned by a Valid CA. In most cases, we will have the corporate gitlab repo in our internal network and not publicly exposed.
We therefore trust the gitlab server as we have a bunch of our code on it.. Why not, I say?
We have to ensure to disable the check for the SSL certificate verification

Set the Variable GIT_SSL_NO_VERIFY=1 or GIT_SSL_NO_VERIFY=false and try to execute your previous command.

[vamshi@workstation ~]$ GIT_SSL_NO_VERIFY=1 git clone https://gitlab.linuxcent.com/linuxcent/pipeline-101.git
Cloning into 'pipeline-101'...
Username for 'https://gitlab.linuxcent.com': vamshi
Password for 'https://[email protected]': 
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.

Make a permanent entry to system wide user level profiles as below. The following change works at the system level

[vamshi@workstation ~]$ sudo bash -c "echo -e export GIT_SSL_NO_VERIFY=1 > /etc/profile.d/gitconfig.sh "

[vamshi@workstation ~]$ cat /etc/profile.d/gitconfig.sh
export GIT_SSL_NO_VERIFY=false

The practical use case of setting the environment variable can be made while building container images, using  the GIT_SSL_NO_VERIFY false as an environment variable in Dockerfile and building an image.

[vamshi@workstation ~]$ cat Dockerfile
FROM jetty:latest
-- CONTENT TRUNCATED --
env GIT_SSL_NO_VERIFY 1
-- CONTENT TRUNCATED --

We can also setup the container build agent with Jenkins Pipeline code with similar configuration to fetch a gitrepo in our next sessions.

Why is git clone not working?

If you have a problem cloning a repository, or using it once it has been created, check the following: Make sure that the path in the git clone call is correct. … If you have an authorization error, have an administrator check the ACLs in Administration > Repositories > <repoName> > Access.

How do I fix fatal unable to access?

How to resolve “git pull,fatal: unable to access ‘https://github.com… \’: Empty reply from server”

1. If you have configured your proxy for a VPN, you need to login to your VPN to use the proxy.
2. to use it outside the VPN use the unset command: git config –global –unset http.proxy.

How do I bypass SSL certificate in git?

Prepend GIT_SSL_NO_VERIFY=true before every git command run to skip SSL verification. This is particularly useful if you haven’t checked out the repository yet. Run git config http. sslVerify false to disable SSL verification if you’re working with a checked out repository already.

How do I open a cloned git repository?

Clone Your Github Repository

• Open Git Bash. If Git is not already installed, it is super simple. …
• Go to the current directory where you want the cloned directory to be added. …
• Go to the page of the repository that you want to clone.
• Click on “Clone or download” and copy the URL.

Can not clone from GitHub?

If you’re unable to clone a repository, check that:
You can connect using HTTPS. For more information, see “HTTPS cloning errors.”
You have permission to access the repository you want to clone. For more information, see “Error: Repository not found.”
The default branch you want to clone still exists.

Do I need git for GitLab?

To install GitLab on a Linux server, you first need Git software. We explain how to install Git on a server in our Git tutorial. Next, you should download the GitLab omnibus package from the official GitLab website.

How do I clone a project from GitHub?

Cloning a repository

• In the File menu, click Clone Repository.
• Click the tab that corresponds to the location of the repository you want to clone. …
• Choose the repository you want to clone from the list.
• Click Choose… and navigate to a local path where you want to clone the repository.
• Click Clone.

How do I push code to GitHub?

Using Command line to PUSH to GitHub

• Creating a new repository. …
• Open your Git Bash. …
• Create your local project in your desktop directed towards a current working directory. …
• Initialize the git repository. …
• Add the file to the new local repository. …
• Commit the files staged in your local repository by writing a commit message.

The Linux tar command abbreviation is “tar archive” released under POSIX standards initially and It now follows the GNU standards

Often when you download the files from the internet they are in the .tar, tar.gz. Or tar.bz2 compressed format using either bz2 or gz compression algorithm.

If you are familiar with the Opensource then there’s a good chance that you would have come across the extensions like Package-name.tar, tar.gz, tar.bz, tar.xz which are standard.

Well most open source software use tarballs to distribute programs/source codes in this format as it offers efficient compression and better organized way of grouping files.

It supports a vast range of compression programs such as gzip, bzip2, xz, lzip, lzma, lzop.

In the following tutorial we will show how to Compress/Extract the files into tar.gz or tgz.

How to specify the format POSIX Or GNU while using tar Command?

[vamshi@linuxcent ]$ tar --format=posix -cf posix.tar *
[vamshi@linuxcent ]$  tar --format=gnu -cf gnu.tar *
[vamshi@linuxcent ]$  file posix.tar gnu.tar 
posix.tar: POSIX tar archive
gnu.tar: POSIX tar archive (GNU)

GNU is based on an older POSIX format, So that’s why it says POSIX for both.

To print the contents information of an archived directory, use the directory name as a file name argument in conjunction with --list (-t). To find out file attributes, include the --verbose (-v) option.

Firstly start off by long listing ls in the present directory we have here:

vamshi@LinuxCent:~/Linux> ls
Debian-Distro/  OpenSuse-Distro/ README  Redhat-Distro/

We will now compress the present working directory using Linux GNU’s tar command line utility.. Creating a tar of the current directory mentioned by Asterisk *. The Options -c creates, -v: Verbose mode, -z: Uses GZIP algorithm
Demonstration shown below:
Either one of the following commands can be used

vamshi@LinuxCent:~/Linux> tar --gzip -cvf Linux-flavours.tar.gz *
vamshi@LinuxCent:~/Linux> tar -cvzf Linux-flavours.tar.gz *
Debian-Distro/
Debian-Distro/ubuntu.txt
Debian-Distro/debian.txt
Debian-Distro/README-Debian-Distro
Linux-flavours.tar.gz
OpenSuse-Distro/
OpenSuse-Distro/README-Opensuse-Distro
OpenSuse-Distro/opensuse.txt
README
Redhat-Distro/
Redhat-Distro/Fedora/
Redhat-Distro/Fedora/fedora.txt
Redhat-Distro/Centos/
Redhat-Distro/Centos/centos.txt
Redhat-Distro/Centos/CentOS-versions/
Redhat-Distro/Centos/CentOS-versions/centos7.txt
Redhat-Distro/Centos/CentOS-versions/centos5.5.txt
Redhat-Distro/Centos/CentOS-versions/centos6.9.txt
Redhat-Distro/Centos/CentOS-versions/centos5.8.txt
Redhat-Distro/Centos/CentOS-versions/centos6.1.txt
Redhat-Distro/Centos/README-CentOS
Redhat-Distro/README-Redhat-Distro
Redhat-Distro/RHEL-Versions/
Redhat-Distro/RHEL-Versions/redhat5.txt
Redhat-Distro/RHEL-Versions/redhat7.txt
Redhat-Distro/RHEL-Versions/redhat6.txt
Redhat-Distro/RHEL-Versions/redhat8.txt
Redhat-Distro/redhat.txt

How to List/view Archives

The option-t does a Dry-run of extract operation but only to print the contents of the mentioned compression.

vamshi@LinuxCent:~/Linux> tar -tvf Linux-flavours.tar.gz
Debian-Distro/
Debian-Distro/ubuntu.txt
Debian-Distro/debian.txt
Debian-Distro/README-Debian-Distro
…

How to Extracting tar.gz File?

Extracting the tar file using the options -x works out just fine as -x option chooses the tye of decompression based on the compression file type, and the content will be extracted to current working directory.

Here are various extraction options:

$ tar -xvf Linux-flavours.tar.gz
$ tar -zxvf Linux-flavours.tar.gz
$ tar --gzip -xvf Linux-flavours.tar.gz

The Filter / Options for compression types:
-z or --gzip : Used for Archival operation for .gzip type
-j -r --bzip2: Used for archival operation for .bzip2 type
[/code]-J[/code] or --xz: User for Archival for .xz type

How to Extract Only Specific Files from a compressed tar archive (tar) File

This doesn’t require any special option but you have to name the exact file and directory that you want to extract

[vamshi@linuxcent ~]$ tar -zxvf redmine-4.0.6.tar.gz redmine-4.0.6/bin/about
redmine-4.0.6/bin/about

Successful in extracting only the redmine-4.0.6/bin/about file from the redmine-4.0.6.tar.gz archive.

[vamshi@linuxcent ~]$ ls -l redmine-4.0.6/bin/about 
-rwxrwxr-x. 1 vamshi vamshi 167 Dec 20 11:46 redmine-4.0.6/bin/about

How to Extract specific files or Directories from the archive

vamshi@LinuxCent:/tmp/linux-test> tar -zxvf Linux-flavours.tar.gz Redhat-Distro/Centos/
Redhat-Distro/Centos/
Redhat-Distro/Centos/centos.txt
Redhat-Distro/Centos/CentOS-versions/
Redhat-Distro/Centos/CentOS-versions/centos7.txt
Redhat-Distro/Centos/CentOS-versions/centos5.5.txt
Redhat-Distro/Centos/CentOS-versions/centos6.9.txt
Redhat-Distro/Centos/CentOS-versions/centos5.8.txt
Redhat-Distro/Centos/CentOS-versions/centos6.1.txt
Redhat-Distro/Centos/README-CentOS

 

This way we are able to extract only the specific directories pertaining to Centos from Redhat-Distro.
Now we would do the extraction of the contents on Redhat-Distro and its sub directories.
Extracting Sub-directories inside the compressed tar.gz

How to Extract TAR file Contents to a New directory?

The tar program by default, extracts the archive contents to the present working directory.
By specifying the option --directory (-C) You can extract archive files in a Target directory.

vamshi.santhapuri@LinuxCent:/tmp/linux-test1> tar -xzvf ~/Linux/jdk-8u101-linux-x64.tar.gz  -C /usr/local/

The above operation extracts the java bundle to /usr/local/ directory.

How to extract only the specific directory from the Compression to a Target directory?

vamshi.santhapuri@LinuxCent:/tmp/linux-test1> tar -xzvf ~/Linux/Linux-flavours.tar.gz Redhat-Distro/RHEL-Versions/ -C /tmp/linux-test
Redhat-Distro/RHEL-Versions/
Redhat-Distro/RHEL-Versions/redhat5.txt
Redhat-Distro/RHEL-Versions/redhat7.txt
Redhat-Distro/RHEL-Versions/redhat6.txt
Redhat-Distro/RHEL-Versions/redhat8.txt

How to tar compress the specific Files and Directory and Sub-directories / Multiple Directories using tar command?

Below is the Demonstration of compression program using gz compression.

vamshi@LinuxCent:~/Linux> tar -cvzf Redhat-Distro/ Linux-flavours.tar.gz centos.txt fedora.txt opensuse.txt redhat.txt ubuntu.txt
centos.txt
fedora.txt
opensuse.txt
redhat.txt
Redhat-Distro/centos.txt
Redhat-Distro/fedora.txt
Redhat-Distro/redhat.txt
Redhat-Distro/Redhat-Versions/
Redhat-Distro/Redhat-Versions/redhat5.txt
Redhat-Distro/Redhat-Versions/redhat7.txt
Redhat-Distro/Redhat-Versions/redhat6.txt
Redhat-Distro/Redhat-Versions/redhat8.txt

You can also compress multiple directories and files into a single tar.gz file as demonstrated below

How to exclude particular directories and file from the compression using tar command?

We can make use of the --exclude="DontIncludethisPath" in Linux tar command, where in the base directory of mentioned pattern are excluded.. Lets run the tar command and see the results

vamshi@linuxCent:~/Linux/OSes> tar -czvf exclude-flavours.tar.gz --exclude="Redhat-Distro" .
./
./OpenSuse-Distro/
./OpenSuse-Distro/README-Opensuse-Distro
./OpenSuse-Distro/opensuse.txt
./Debian-Distro/
./Debian-Distro/ubuntu.txt
./Debian-Distro/debian.txt
./Debian-Distro/README-Debian-Distro
tar: .: file changed as we read it

The compression successfully completed and now lets list out the files

vamshi@linuxCent:~/Linux/OSes> ls -lthr
total 16K
drwxr-xr-x 2 vamshi users 4.0K Apr  8 14:09 OpenSuse-Distro
drwxr-xr-x 2 vamshi users 4.0K Apr  8 14:09 Debian-Distro
drwxr-xr-x 5 vamshi users 4.0K Apr  8 14:09 Redhat-Distro
-rw-r--r-- 1 vamshi users  718 Apr 8 19:18 exclude-flavours.tar.gz

From the dry-run extract output of the compression does not contain the Directory Redhat-Distro

vamshi@linuxCent:~/Linux/OSes> tar -tvf exclude-flavours.tar.gz
drwxr-xr-x vamshi/users 0 2020-04-08 19:18 ./
drwxr-xr-x vamshi/users 0 2020-04-08 14:09 ./OpenSuse-Distro/
-rw-r--r-- vamshi/users 0 2020-04-08 14:09 ./OpenSuse-Distro/README-Opensuse-Distro
-rw-r--r-- vamshi/users 9 2020-04-08 13:22 ./OpenSuse-Distro/opensuse.txt
drwxr-xr-x vamshi/users 0 2020-04-08 14:09 ./Debian-Distro/
-rw-r--r-- vamshi/users 7 2020-04-08 13:22 ./Debian-Distro/ubuntu.txt
-rw-r--r-- vamshi/users 7 2020-04-08 13:50 ./Debian-Distro/debian.txt
-rw-r--r-- vamshi/users 0 2020-04-08 14:09 ./Debian-Distro/README-Debian-Distro

How do I open a tar file in Linux?

tar Utility

1. Let’s consider that we want to extract and open a doc file, and then we can use the below command to unzip the
2. file on Linux:
3. tar –xvzf doc.tar.gz. Remember that the tar. …
4. tar –cvzf docs.tar.gz ~/Documents. …
5. tar -cvf documents.tar ~/Documents. …
6. tar –xvf docs.tar. …
7. gzip xyz.txt. …
8. gunzip test.txt. …
9. gzip *.txt.

How do I view the contents of a tar file in Unix?

tar File Content. Use -t switch with tar command to list content of a archive. tar file without actually extracting.

How do I view a tar file?

How to open TAR files

• Download and save the TAR file to your computer. …
• Launch WinZip and open the compressed file by clicking File > Open. …
• Select all of the files in the compressed folder or select only the files you want to extract by holding the CTRL key and left-clicking on them.

Where can I find tar files in Linux?

Combining find and tar commands so that we can find and tar files into a tarball

1. -name “*. doc” : Find file as per given pattern/criteria. In this case find all *. doc files in $HOME.
2. -exec tar … : Execute tar command on all files found by the find command.

How do I extract a tar file?

To extract (unzip) a tar. gz file simply right-click on the file you want to extract and select “Extract”. Windows users will need a tool named 7zip to extract tar.

How do I unzip a tar file in Terminal?

The most common uses of the tar command are to create and extract a tar archive. To extract an archive, use the tar -xf command followed by the archive name, and to create a new one use tar -czf followed by the archive name and the files and directories you want to add to the archive.

How do I extract a tar file from a directory in Linux?

Syntax For Tar Command To Extract Tar Files To a Different Directory

1. x : Extract files.
2. f : Tar archive name.
3. –directory : Set directory name to extract files.
4. -C : Set dir name to extract files.
5. -z : Work on . tar. …
6. -j : Work on . tar. …
7. -J (capital J ) : Work on . tar. …
8. -v : Verbose output i.e. show progress on screen.

Can 7 Zip open tar files?

7-Zip can also be used to unpack many other formats and to create tar files (amongst others). Download and install 7-Zip from 7-zip.org. … Move the tar file to the directory you wish to unpack into (usually the tar file will put everything into a directory inside this directory).

What is tar file in Linux?

The Linux ‘tar’ stands for tape archive, is used to create Archive and extract the Archive files. tar command in Linux is one of the important command which provides archiving functionality in Linux. We can use Linux tar command to create compressed or uncompressed Archive files and also maintain and modify them.

The Linux command timedatectl is a systemd linux command that enables the Linux system admin to configure the date and time effectively.

The timedatectl is an important and a handy tool in linux to configure the time and date of the system, especially in a cluster setup where the synchronization is essential.

[vamshi@node01 .ssh]$ sudo timedatectl
Local time: Wed 2020-04-01 13:53:03 UTC
Universal time: Wed 2020-04-01 13:53:03 UTC
RTC time: Sun 2020-04-26 14:26:09
Time zone: UTC (UTC, +0000)
NTP enabled: yes
NTP synchronized: no
RTC in local TZ: no
DST active: n/a
List the timezones from commandline

[vamshi@node01 ~]$ sudo timedatectl list-timezones

How to sync the server to with ntp time sync using timedatectl ?

You have to ensure that the ntp in installed and running and then automatically the time sync will be enforced with the following command.

# sudo timedatectl set-ntp true

The manual time set can be performed using timedatectl as follows:

[vamshi@node01 ~]$ sudo timedatectl set-time 17:53:03
[vamshi@node01 ~]$ date
Wed Apr 1 17:53:04 UTC 2020

The best practice and practical approach is to set the timesync on with ntpd process enabled and in running state as show below:

# sudo systemctl enable ntpd --now

What is Buildah? Is it a better Docker alternate?

Used to build container images as per OCI(Open Container Initiative) standard which provides API to build Images in docker format using golang.

The Buildah project provides a command line tool that be used to create an OCI or traditional Docker image format image and to then build a working container from the image. The container can be mounted and modified and then an image can be saved based on the updated container.

Podman – Used to manage the pulling, tagging the images similar to Docker based on OCI standard.
To be used for Production grade deployments. It also allows you to create, run, and maintain containers created from those images.

Buildah Concepts:

• Buildah’s commands replicate all of the commands that are found in a Dockerfile.
• Also has the feasibility to build docker images with Dockerfile while not requiring root privileges while providing lowerlevel coreutils interface to build images
• Buildah uses simple fork-exec model and does not run as a daemon but it is based on a comprehensive API in golang.
• Buildah uses runcrun commands when buildah run used, or when buildah build-using-dockerfile encounters a RUN instruction. And it is namespace dependent on the Host System

• Buildah relies on the CNI library and plugins to set up interfaces and routing for network namespaces
• Buildah on Red Hat Enterprise Linux or CentOS, version 7.4 or higher is required. On other Linux distributions Buildah requires a kernel version of 4.0 or higher in order to support the OverlayFS filesystem – https://github.com/containers/buildah/blob/master/install.md
• The dependencies required as btree filesystem, containernetworking-cni, golang and bz2 and other common-container tools.

Buildah Configuration Files and its Directory Structure:

# /usr/share/containers/

# registries.conf
# mounts.conf
# seccomp.json
# policy.json

How to login to the docker through commandline?

The docker exec is similar to running a tty on a linux machine. the only difference is it can accept many more exec connections, Although you can enable ssh to the docker but it will only be possible while running an openssh server inside the running container.

You can exec to connect to a running container much like the ssh into the running machine, Here we use the special options -i or --interactive to provide interaction and -t or --tty which allocates a pseudo tty terminal of the type of shell that is followed
The syntax of docker exec is as below:

docker exec -it <container-id | container-name> bash

[vamshi@node01 ~]$ docker exec -it b511234ebe31 bash
jenkins@b511234ebe31:/$ id
uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins)

We know that the docker exec command is used opens up a tty terminal to connect to the running docker container, but It will connect us with the Default USER that was activated during it docker build.

We can use the -u flag to connect to the container with the username that is enabled for that image, We see the below example

[vamshi@node01 ~]# docker exec -it  -u root b511234ebe31 bash
root@b511234ebe31:/# pwd
/
root@b511234ebe31:/# id
uid=0(root) gid=0(root) groups=0(root)

 

 

We have seen the docker runtime environment takes up the overall available system resources on the system and tends to impact the base system.

To better utilize the containers, we can avail the resource cap and define the metric limits on specific containers while starting up the respective docker images..

The general syntax goes as follows:

# docker run --cpus ="x.x" --memory=x[M|G] docker-image

Here we see the demonstration

[root@node01 ~]# docker run --cpus="0.2" --memory="200M" jenkins:latest

/usr/bin/docker-current: Error response from daemon: Minimum memory limit allowed is 4MB.

It should be noticed that the minimum Memory limit allowed for the docker container to run is 4MB and the minimum CPU cores is at 0.01, any thing lower that this means the container runtime fails to allocate sufficient resources. These limits will be efficient when running on some test and debug scenarios.

How to make a file or Folder/Directory un-deletable on Linux?

The linux operating as we know if famous for the phrase “Everything is a file”, In such circumstances it is interesting to explore the possibilities of making a file undeletable, even by the owner of the file and for that matter even the root user, In the Linux Ecosystem the root is the poweruser.

This section we will see the potential of such feature.

As we have already seen the section on deleting files on Linux (removing the files in Linux).

We will now demonstrate the power of Linux where you can restrict the deletion of a file on Linux.

Linux offers a chattr commandline utility which generally modifies the file attributes as the name suggests, but the practical use is to make a file undeletable.

Sample command syntax:

[vamshi@linuxcent ~]$ chattr +i <samplefile>

vamshi@linuxcent delete-dir]$ sudo chattr +i samplefile2.txt

Now we do ls -l samplefile2.txt

[vamshi@linuxcent ~]$ sudo chattr +i samplefile2.txt
[vamshi@linuxcent ~]$ ls -l samplefile2.txt
-rw-rw-r--. 1 vamshi vamshi 4 Apr 8 15:42 samplefile2.txt

Now we shall try to write some content to this file and see no change in the basic file permissions(see changing ownership of files).

[vamshi@linuxcent delete-dir]$ echo "New content" > samplefile2.txt
-bash: samplefile2.txt: Permission denied

Deleting file forcefully with the --force option ?

[vamshi@linuxcent delete-dir]$ sudo /bin/rm -f samplefile2.txt

/bin/rm: cannot remove ‘samplefile2.txt’: Operation not permitted

Linux command lsattr offers the ability to view the permissions set by the chattr command.
The current File attributes can be listed using lsattr followed by the filename [/code]samplefile2.txt[/code] as below

[vamshi@linuxcent delete-dir]$ lsattr samplefile2.txt
----i----------- samplefile2.txt

Even the root user on the host is unable to delete the file or modify its contents.

The file can be deleted only when the attributes are unset, It is demonstrated as follows:

[vamshi@linuxcent delete-dir]$ sudo chattr -i samplefile2.txt
[vamshi@linuxcent delete-dir]$ lsattr samplefile2.txt
---------------- samplefile2.txt

As we can see the lsattr doesn’t hold true anymore attributes on our file samplefile2.txt and is now being treated as any other normal file with basic file attributes.
The - operation removes the special linux file attributes on the mentioned file.

The chattr / lsattr linux commandline utilities currently supports the popular filesystems such as ext3,ext4,xfs, btrfs etc,.

The concept of Links in Linux/Unix based systems is Unique and gives a very deeper understanding of the Linux working internals at various levels.

The symbolic also known as Soft link is a special type of linking that acts as a pointer to the original file present on the disk.

The Softlink span across different filesystems extensively and widely used during software package installation and configuring

Lets look at the example of java command linking:

[root@node02 ]# which java
/bin/java
[root@linuxcent ]# ls -l /bin/java
lrwxrwxrwx. 1 root root 22 Apr 10 11:52 /bin/java -> /etc/alternatives/java
[root@node02 boot]# ls -l /etc/alternatives/java
lrwxrwxrwx. 1 root root 73 Apr 10 11:52 /etc/alternatives/java -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java
[root@linuxcent ]#

If you upgrade java on your system then the /bin/java command points out to a newer installed version of /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java

We will see the demonstration of a Symbolic link.

In Linux to make the links between files, the command-line utility is “ln”.

Softlink Creation syntax using the Linux command line utility ln -s option:

ln -s <source file|Directory > <destination Link file|Directory>

Below is an example

[vamshi@linuxcent html]$ ln -sf /var/www/html/index.html /tmp/index.html

[vamshi@linuxcent html]$ ls -l /tmp/index.html

lrwxrwxrwx. 1 vamshi vamshi 24 Apr  1 18:23 /tmp/index.html -> /var/www/html/index.html

 

The second file is called a symbolic ink to /tmp/index.html

Now the second file /tmp/index.html is called a Symbolic link to the First file on the disk /var/www/html/index.html

It means that the second file is just pointing to the first file’s location on disk without actually copying the contents of the file.

[vamshi@linuxcent html]$ cat /var/www/html/index.html
Welcome to LinuxCent.com
[vamshi@linuxcent html]$ cat /tmp/index.html
Welcome to LinuxCent.com

When you edit either of the file, the contents of the original file on disk are directly modified.

 

How to create Softlinks to Directories in Linux ?
The same logic applies to creating the soft links to directories, Lets see the Demonstration below:

[vamshi@linuxcent html]$  ln -sf /var/www/html/linuxcent/static/ /tmp/static

[vamshi@linuxcent html]$ ls -l /tmp/static
lrwxrwxrwx. 1 vamshi vamshi 32 Apr  8 18:37 /tmp/static -> /var/www/html/linuxcent/static/

 

How do we Remove the linking between the files is even simpler.

Simply run the unlink command on its Destination:

Sample command:

unlink <destination Link file | Directory>

Lets see the Demonstration

unlink /tmp/data-dir

 

Understanding Symbolic / Hard link concept better with improving knowledge on Inode.

To better understand the concept of Symbolic linking we need to understand the concept of Inode numbers present in Linux. I will give a brief overview of it in this section But for Detailed Review, Please see the What is Inode in Linux Section.

We can list out the inode number of any file on linux system using [/code]ls -i <filename>[/code]

Now The extreme left column indicates the system wide unique inode number:

[root@node02 ~]# ls -li /var/www/html/index.html /tmp/index.html 
67290702 lrwxrwxrwx. 1 root root 24 Apr 8 19:09 /tmp/index.html -> /var/www/html/index.html
33557754 -rw-r--r--. 1 root root 25 Apr 8 18:19 /var/www/html/index.html

We can see here the inode number for both the files have different, Because the second file is just a pointer to the original source file..

So why do we create the Symbolic Links ?

We have the advantages of the Softlinks

1. One Advantage of Softlinks/symbolic links is they span across different filesystems
2. Can have many softlinks to a single file/Directory.
3. Can create Symbolic Links of Directories and Files respectively.
4. Rsync program by default preserves the symbolic links.
5. The softlinks become activated immediately if the source is recreated or recovered after any kinds of network outages.

What are the best practices when using Softlinks ?

The best practice while creating softlinks is to mention absolute path for source and destination links.

On the other hand you should also understand the disadvantages or shortcomings.

1. It is Important to observe that If the source file is deleted then the symbolic link becomes useless.
2. Incases of the Network filesystem issues leading to unavailability of softlinks.

It is also Essential to understand about the Hardlinks to get an overall understanding.

 

Creating a hard link is a simple operation using ln command with no options

Sample Command:

$ ln /path/to/source/ /path/to/HardLink/

So lets start of by creating a hard link of our file /var/www/html/index.html

[vamshi@linuxcent linuxcent]$ ln /var/www/html/index.html /tmp/index-hl.html

The command ls -il lists the inode number of the files.

[vamshi@linuxcent  ]$ ls -i
33557752 index-hl.html  33557752 index.html

So to conclude, The hard linking results in the same inode number, howmany ever times the hardlink of same file is created.
The data continues to persists in the same storage location on the filesystem even if one of the hardlink file is deleted.
As long as the inode number is identical on the files, no filename change matters to the filesystem.
There will be no change in Hardlink behaviour.

Let’s add some content to the hardlink file here and see the Demonstration

[vamshi@linuxcent ]$ echo "We are updating the file to check out the Hardlinks" >>  /tmp/index-hl.html

We are updating the file to check out the Hardlinks

The new line is added in the original file.

[vamshi@linuxcent linuxcent]$ cat index.html
Welcome to LinuxCent.com
We are updating the file to check out the Hardlinks

Content manipulations to either of the files will be treated as a same file.

How to Identify how many times a particular file was linked ?

Note that Linux Command ls -li provides the actual link aggregate count information in the fourth column represented by the number 2 here, which means that it has a second hardlink reference and both the files are interchangeable.
And It should be noted that a file has a link count of 1 by default as it references itself

$ls -li
33557752 -rw-rw-r--. 2 vamshi vamshi 59 Apr  8 19:48 index-hl.html
33557752 -rw-rw-r--. 2 vamshi vamshi 59 Apr  8 19:48 index.html

In case either one of the file is deleted, the other file survives and continues to function properly.

Lets see some hard facts on the Hardlinks.

1. Hardlinks can’t be created to Directories.
2. They do not span across filesystems.

The Ports on a Linux OS are used for exchange and transfer of data on the network connected devices.

A Very high number of security exploitation happen due to no surveillance of in bound connections targeting specific ports. It is most essential to identify the underlying Linux process opening up specific ports for listening over a shared network.

Thus, It is important to identify which ports are open on your Linux machine.

Firstly for basic administration tasks, identifying the port and its correlating application, so that you are well aware of the Open sockets and Enhance the network security by preventing the network intrusion by writing the firewall rules.

In this tutorial we will look at some of the most popular Linux network tools and see how to gather information, and identify the web server like process Apache Httpd or Nginx running so that you don’t conflict when you are configuring them and troubleshooting.

We will discuss some of the popular tools and their general commands syntax.

Netstat Command and its Syntax

netstat

Netstat gives you multiple features and a must know tool if you are in your day to day activities.

It gives out the information about the Open ports in you Linux machine along with the Established connection, TimeWait and Closed state connections.

The netstat command goes by netstat -ntlp

ss

It stands for Socket Statistics, It is a command line utility which provides the information about Open ports, the corresponding process ID which opened the port.

ss command is the successor to netstat command in Linux and has the similar options as its predecessor, In Fact it necessarily is a better enhancement over the old netstat command,

First up lets run ss -tua, which lists all the TCP and UDP Sockets.

[vamshi@linuxcent ~]$ ss -tua

Netid  State     Recv-Q Send-Q                                  Local Address:Port                                                   Peer Address:Port

udp    UNCONN     0      0                                           127.0.0.1:domain                                                            *:*

udp    UNCONN     0      0                                           127.0.0.2:domain                                                            *:*

udp    UNCONN     0      0                                                   *:bootpc                                                            *:*

udp    UNCONN     0      0                                                   *:sunrpc                                                            *:*

udp    UNCONN     0      0                                           127.0.0.1:323                                                               *:*

udp    UNCONN     0      0                                                   *:lanserver                                                         *:*

udp    UNCONN     0      0                                                [::]:sunrpc                                                         [::]:*

udp    UNCONN     0      0                                               [::1]:323                                                            [::]:*

udp    UNCONN     0      0                                                [::]:lanserver                                                      [::]:*

tcp    LISTEN     0      100                                         127.0.0.1:smtp                                                              *:*

tcp    LISTEN     0      128                                                 *:sunrpc                                                            *:*

tcp    LISTEN     0      128                                                 *:ssh                                                               *:*

tcp    ESTAB      0      0                                         10.100.0.20:ssh                                                      10.100.0.1:45662

tcp    LISTEN     0      100                                             [::1]:smtp                                                           [::]:*

tcp    LISTEN     0      70                                               [::]:33060                                                          [::]:*

tcp    LISTEN     0      128                                              [::]:mysql                                                          [::]:*

tcp    LISTEN     0      128                                              [::]:sunrpc                                                         [::]:*

tcp    LISTEN     0      128                                              [::]:http                                                           [::]:*

tcp    LISTEN     0      128                                              [::]:ssh                                                            [::]:*

 

How to get the Established connection information using ss command in linux

$ ss -tua state established

 

[vamshi@node02 ~]$ ss -l sport = 80

The detailed options ss offers are as follows:

-a : Displays all Sockets:

-i : Displays internal TCP information

-t : Displays only TCP Sockets

-l : Displays Only Listening Sockets

-u : Displays only UDP Sockets

-r : Resolves host names

-n : Doesn’t Resolve the Hostnames

-p : Display process information using the Socket

 

ss command takes the following state option filters

established syn-sent |syn-recv |fin-wait-{1,2} |time-wait |closed |close-wait |last-ack |listen |closing

 

nmap

It is one of the most popular open source tools to explore networks and mainly used for security auditing.

NMAP is used extensively for Host Discovery , Port and Protocol Scanning such as ICMP, TCP and UDP Port Scanning.

The service and its version Detection, Operating system Detection.

It is one of the Intrusion prevention tool when used effectively and used for greater security Reporting.

run the command sudo nmap localhost

$ sudo nmap 10.100.0.0/24

$ vamshi@linuxcent: ~ $ nmap 10.100.0.20

Starting Nmap 7.70 ( https://nmap.org ) at 2020-04-10 13:45 GMT

Nmap scan report for 10.100.0.20

Host is up (0.00014s latency).

Not shown: 996 closed ports

PORT     STATE SERVICE

21/tcp   open ftp

22/tcp   open ssh

53/tcp   open domain

80/tcp   open http

111/tcp  open rpcbind

3306/tcp open  mysql

8009/tcp open  ajp13

MAC Address: 08:00:27:5A:26:BD (Oracle VirtualBox virtual NIC)


Nmap done: 1 IP address (1 host up) scanned in 1.51 seconds

It lists all the ports that are Open and even lists up the MAC Address of the Target Host.

How to scan a list of hosts from a file using nmap command

You can do it and run the command as demonstrated below:

$ nmap -iL /etc/hosts # where you have a list of ip/dns names

To increase the verbosity of nmap output use -v<n> where n is a number ranging from 0 – 9

This offer a lot of options to gather information passively and lets see some options listed below:

 

-sn : Lists and does Ping Scan on the network; Least aggressive

-sL : Lists hosts to scan on the network

-O : Enables Target Host OS detection.

-sS : Does the syn Scan, in stealth mode.

-sT : Performs TCP port Scan

-sU : Performs UDP port Scan.

-p : Scans only for Target Port Listed Eg: # sudo nmap -v  -sS -p80 10.100.0.0/24

-PA : TCP Ack Flag is set

-sV : Extracts the Service Version information and the Operating System information from Target Hosts.

 

lsof

lsof linux command gives the information on the list of open files on the system as the abbreviation says. It’s one of the valuable tools when troubleshooting under fire, Gives you the practical linux system behaviour

Start it by running lsof command, will print a bunch of information including the all the programs currently started and owned by you, It includes the block/filesystem files, network stream data/character files, virtual memory paging and temporary data files.

Listing the openfiles using lsof

Now run lsof -u <Your login username>

$ lsof -u vamshi

# Prints a bunch of information about the open files for a particular user.

Now lsof has a lot of options to extract relevant information

To list the total number of open files on the system

$ sudo lsof | wc -l

$ sudo lsof -i TCP :22
$ sudo sudo lsof -P -i:22

vamshi@linuxcent:~$ sudo lsof -i -P :22

COMMAND  PID USER   FD TYPE DEVICE SIZE/OFF NODE NAME

sshd     380   root    3u  IPv4  13672     0t0  TCP *:22 (LISTEN)

sshd     380   root    4u  IPv6  13683     0t0  TCP *:22 (LISTEN)

sshd    1295   root    3u  IPv4  20367     0t0  TCP 10.100.0.30:22->10.100.0.1:39054 (ESTABLISHED)

sshd    1307 vamshi    3u  IPv4  20367     0t0  TCP 10.100.0.30:22->10.100.0.1:39054 (ESTABLISHED)

Lists the information of open and Established connections of the given port

lsof offer a lot of options and filters, lets list some of the most commonly used ones below:

 

-u : takes the username as filter option, Lists openfiles caused by the given username

-i : Lists the open files belonging to the Service port numbers

-P : Inhibits the translation of service names based on port number and prints the port number for simplicity purpose.

 

Linux Command Utility [/code]rsync[/code] is a very robust, fast content copy command which can be used within the same linux host and over a connected network between 2 linux hosts. It is a special program which has intelligence in terms of not copying data repetitively if the destination has the same copy as source based on file checksum calculations.

We shall explore some of the practical rsync command features and demonstrate them

Syntax of command

$ rsync [OPTIONS] /source/path/ /dest/path/

Running rsync on the same host?

[vamshi@linuxcent ~]$ rsync -avx newfile.txt /tmp/
sending incremental file list
newfile.txt

sent 133 bytes  received 35 bytes  336.00 bytes/sec
total size is 21  speedup is 0.12

Running rsync between two hosts in a network

$ rsync [OPTIONS] host:/source/path/ /dest/path/

Run rsync in Dry-run mode by using [/code]-n[/code] option

$ rsync -avn Source_host:/source/path Destination_host:/dest/path

This generally runs over the SSH protocol and you are required to enter the login credentials appropriately.

How to invoke SSH remote shell in rsync?

In case you are using a SSH keys then you have to invoke the remote shell to authenticate to the remote server with your private keypair. This is Demonstrated as follows:

$ rsync -avxn --rsh="ssh -i ~/.ssh/vamshi_id_rsa" vamshi@<Your.Source.IP.DNS>:"/<Source_Path>" "/<Destination_Path>"

For more information about the SSH key setup, Please refer to our SSH keys section

How to persist Hard links on the system using rsync. Following is the Demonstration

Flag : -H. Using this option enables to preserve the HardLinks over the destination copy of the data.

$ rsync -avHx /path/to/source/ /path/to/destination/

The most practical example of working with rsync comes in replicating mission critical data or transferring Database dumps within the DB servers etc.,

How to exclude certain directories in rsync in linux ?

using --exclude filter option is demonstrated as follows:

$ rsync -avx /source/path/to/backup-v31/ /dest/databackups/backup-v31/ --exclude="DontTouchMyData/"

Using the delete option, enables us to delete the directories from the source upon completion of the operation.

Note: This operation has the same effects as the mv command on linux but performed over the network between source and destination hosts.

$ rsync -avx --delete /source/path/to/backup-v31/ /dest/databackups/backup-v31/ --exclude="data/" --exclude="data/board" --exclude="cache/apt" --exclude="opt"

Redirect the rsync output to a file by appending output redirection symbol to a file on current location

$rsync -avx --delete /source/path/to/backup-v31/ /dest/databackups/backup-v31/ --exclude="data/" --exclude="data/board" --exclude="cache/apt" --exclude="opt"  >>/tmp/rsync.log

What is rsync command Linux?

rsync or remote synchronization is a software utility for Unix-Like systems that efficiently sync files and directories between two hosts or machines. … Copying/syncing to/from another host over any remote shell like ssh, rsh.

How do I use rsync in Linux?

Syntax of rsync command:

• -v, u2013verbose Verbose output.
• -q, u2013quiet suppress message output.
• -a, u2013archive archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
• -r, u2013recursive sync files and directories recursively.
• -b, u2013backup take the backup during synchronization.

What is rsync in bash?

rsync is a fast and versatile command-line utility for synchronizing files and directories between two locations over a remote shell, or from/to a remote Rsync daemon. … Rsync can be used for mirroring data, incremental backups, copying files between systems, and as a replacement for scp , sftp , and cp commands.

How do I transfer files using rsync?

You can use SecureShell (SSH) or Remote Sync (Rsync) to transfer files to a remote server. Secure Copy (SCP) uses SSH to copy only the files or directories that you select. On first use, Rsync copies all files and directories and then it copies only the files and directories that you have changed.

What is rsync command do?

Rsync is typically used for synchronizing files and directories between two different systems. For example, if the command rsync local-file user@remote-host:remote-file is run, rsync will use SSH to connect as user to remote-host.

Why do we use rsync?

Syntax of rsync command:

• -v, u2013verbose Verbose output.
• -q, u2013quiet suppress message output.
• -a, u2013archive archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
• -r, u2013recursive sync files and directories recursively.
• -b, u2013backup take the backup during synchronization.

What is rsync in RHEL?

Rsync can be used to quickly move large amounts of data to both local and remote destinations. For this reason, rsync is often used to copy data, make backups, migrate hosts, and bridge the gap between site staging and production environments.

How does rsync work in Linux?

An rsync process operates by communicating with another rsync process, a sender and a receiver. At startup, an rsync client connects to a peer process. If the transfer is local (that is, between file systems mounted on the same host) the peer can be created with fork, after setting up suitable pipes for the connection.

How do I rsync a file in Linux?

Copy a single file locally If you want to copy a file from one location to another within your system, you can do so by typing rsync followed by the source file name and the destination directory. Note: Instead of u201c/home/tin/file1. txtu201d, we can also type u201cfile1u201d as we are currently working in the home directory.

How do I use rsync?

You can use secure shell (SSH) or Remote Sync (Rsync) to transfer files to a remote server. Secure Copy (SCP) uses SSH to copy only the files or directories that you select. On first use, Rsync copies all files and directories and then it copies only the files and directories that you have changed.