Linux cut command with practical examples

by
Reading Time: 5 minutes

The Linux cut command, extracts the sections of text from the input lines of files.

The general syntax

cut [OPTION] [FILE]

The cut command offers special characters like - and , to extract the range and selection of data from input line or stream of lines read from a file.

The cut command has some most useful and quick options to extract data from the input lines..

First such option is -b, --bytes=LIST from the Specified LIST

Using the -b option we shall be able to extract the specific byte characters.

$ echo -e "Exploring the practical use of cut command"

For getting the specific byte index from input stream list:

echo -e "Exploring the practical use of cut command"  | cut -b 3
p

Extracting the range of bytes from input stream list:

[vamshi@linuxcent ~]$ echo -e "Exploring the practical use of cut command"  | cut -b 3-6
plor

Extracting the bytes from starting till the given index range

[vamshi@linuxcent ~]$ echo -e "Exploring the practical use of cut command"  | cut -b -6
Explor

Extracting the bytes from given index range till the end of the input line

[vamshi@linuxcent ~]$ echo -e "Exploring the practical use of cut command"  | cut -b 3-
ploring the practical use of cut command

Combining the , and – operations and extracting the bytes from the input lines:

[vamshi@node02 Linux-blog]$ echo -e "Exploring the practical use of cut command"  | cut -b 1,2-6,9
Explorg

First such option is -c, --characters=LIST from the Specified LIST

Extracting the specific character from the input string:

[vamshi@node02 Linux-blog]$ echo -e "Exploring the practical use of cut command"  | cut -c 11
t

Extracting the range of bytes from input stream list:

[vamshi@linuxcent ~]$ echo -e "Exploring the practical use of cut command"  | cut -c 11-24
the practical

Getting the range of the characters from given index till the end of the input line using -

[vamshi@node02 Linux-blog]$ echo -e "Exploring the practical use of cut command"  | cut -c 11-

the practical use of cut command

Using - to get the characters from starting till the given index range

[vamshi@node02 Linux-blog]$ echo -e "Exploring the practical use of cut command"  | cut -c -9

Exploring

We have a Text file called README with the following content.

The following are Most popular Linux server Distributions and their curent versions:

Distribution Versions
----------- --------
Redhat : 8.0
Ubuntu : 18.04
Centos : 8.0
Debian : 10.1
SUSE : 15.1

Extracting the fields from the given data file using -f, separated by the tabs which is a default delimited.

Lets try and extract the field from README datafile.

$ cat README | cut -f1
[vamshi@linuxcent ~]$ cut -f1 README
The following are Most popular Linux server Distributions and their current versions:

Distribution
-----------
Redhat
Ubuntu
Centos
Debian
SUSE

The options and ranges can also be mentioned in the field option to extract relevant data.
Extracting the fields indexed by numbers from given file separated by tabs

$ cat README | cut -f1,2,3
$ cut -f1,2,3 README

Extracting the first field through 3rd field from the given file separated by tabs

$ cat README | cut -f1-3 
$ cut -f1-3 README

A lot of combinations can be used to extract the ranges as shows below:

$ cat README | cut -f 3-

We will also like to shed some light on the --complement option, As the option name hints, it will print only the inverse of the operation, It can be used in conjunction with -b , -c and -f respectively:

For example

[vamshi@linuxcent Linux-blog]$ echo -e "Exploring the practical use of cut command" | cut -c1 --complement
xploring the practical use of cut command

The first character E is excluded and then remaining characters are printed because of –complement option being used in conjunction.

Delimiter Option -d, --bytes=DELIM from the Specified LIST
Unlike the field extractor option which only set to default single tab space as Delimiter, The Delimiter option provides the flexibility to setup a custom Delimiter to any character. Widely used for setting a Delimiter other than a Tab.

Lets take an example of the line following input line:

dockerroot:x:996:992:Docker User:/var/lib/docker:/sbin/nologin

The delimit option -dcan be set to :  and have the combination of fields and characters/bytes to highly customize and refine the extract text.

[vamshi@linuxcent Linux-blog]$ cat /etc/passwd | grep docker | cut -d":" -f1
dockerroot
[vamshi@linuxcent Linux-blog]$ cat /etc/passwd | grep docker | cut -d"/" -f1-5
dockerroot:x:996:992:Docker User:/var/lib/docker:/sbin

The same logic of obtaining and extracting the ranges and specific field extractions is applied commonly in cut command.

A very special mentioning of the operator -s or --only-delimited, Which prints only the lines modified by the cut command and delimited by the delimit operator.

[vamshi@linuxcent Linux-blog]$ cat README | cut -d":" -f1 -s
The following are Most popular Linux server Distributions and their curent versions
Redhat 
Ubuntu 
Centos 
Debian 
SUSE

The option --output-delimiter="STRING" takes a string as its input and then substitutes it with the actual delimiter:

Lets look at the below practical example:

[vamshi@node02 Linux-blog]$ cat /etc/passwd | grep docker | cut -d":" -f1- --output-delimiter=' '
dockerroot x 996 992 Docker User /var/lib/docker /sbin/nologin

Format the Output to print newline using cut command:

[vamshi@linuxcent  ~]$ cat /etc/passwd | grep docker | cut -d":" -f1- --output-delimiter=$'\n'
dockerroot
x
996
992
Docker User
/var/lib/docker
/sbin/nologin

The output-delimiter formatting the output by replacing the delimiter with a newline character using $'\n' thus printing the output separated by the newline

How use cut in Linux?

It can be used to cut parts of a line by byte position, character and field. Basically the cut command slices a line and extracts the text. It is necessary to specify option with command otherwise it gives error. If more than one file name is provided then data from each file is not precedes by its file name.

What does the cut command do?

The cut command is a command-line utility for cutting sections from each line of a file. It writes the result to the standard output.

How do I cut a word in Linux?

CUT
Most important Options:

  1. -b, –bytes=LIST # select only these bytes.
  2. -c, –characters=LIST # select only these characters.
  3. -d, –delimiter=DELIM # use DELIM instead of TAB for field delimiter.

What is cut in bash?

The cut command is used to extract the specific portion of text in a file. Many options can be added to the command to exclude unwanted items. It is mandatory to specify an option in the command otherwise it shows an error. In this article, we will throw light on each option of the cut command.

How do you cut on the keyboard?

Keyboard shortcuts

  1. Copy: Ctrl+C.
  2. Cut: Ctrl+X.
  3. Paste: Ctrl+V.

How do you trim in Unix?

Example-2: Trim string data using `sed` command

Use sed ‘s/^ *//g’, to remove the leading white spaces. There is another way to remove whitespaces using `sed` command. The following commands removed the spaces from the variable, $Var by using `sed` command and [[:space:]].

What does cut do Unix?

In computing, cut is a command line utility on Unix and Unix-like operating systems which is used to extract sections from each line of input — usually from a file.

How do you cut a column in Unix?

The Linux cut command allows you to cut data by character, by field, or by column. if used correctly along with sed, find, or grep in UNIX, the cut can do lots of reporting stuff. For example, you can extract columns from a comma-separated file or a pipe or colon-delimited file using cut command.

How do I trim a word in bash?

${var// /} removes all space characters. There’s no way to trim just leading and trailing whitespace with only this construct. Bash “patterns” are regular expressions, only with a different syntax than “POSIX (extended) regular expressions”.

How do I cut a word in bash?

In Linux try Ctrl+k to delete from where the cursor is to the end of the word. There are few other shortcuts listed below(working in Linux): Ctrl+e -> Takes cursor at the end of the word.

Bash How to Wait Seconds

by
Reading Time: < 1 minute

Bash How to Wait Seconds

How can we in the script wait for it until the system completes some tasks? The answer is to use sleep. This command suspends the script so that the script is low almost no system resources. Timing is sufficient.

Do you want to wait some seconds? In the next example, we will wait one second:

sleep 1

To turn off the script only for a split second? You can. This example shows the ingested sleep 100ms:

sleep 0.1

In this example, we will wait 20 minutes:

sleep 20m

In the next example, we will wait 8 hours:

sleep 8h

Do you want to wait several days? This is possible if you use a parameter d. However, consider using a cron scheduler. It is robust. Can you schedule to run it in your scripts at specified times and periodic runs?

In the last example, we will wait 7 days:

sleep 7d

Bash provides command wait to wait for the process/processes given as arguments:

wait $PID

PID is processing ID. ‘wait ${!}’ waits until the last background process is completed.

date command formatting with practical examples in Linux / Unix

by
Reading Time: 5 minutes

Date Command in Linux is very extensive and dynamic, provides very rich date formatting and is greatly customizable for working with scripts which depend on time based invocations.

Linux date command can also be used to set the system date and it requires the root permission.

Lets run date command and examine the output.

[vamshi@node02 log]$ date
Wed Apr 1 13:52:21 UTC 2020

Now lets examine some of the most useful options that comes with the date command.

Firstly date command along with -s or --set option can take for following format to set the new system time and date.

How to set the system date in Linux using date command?

[vamshi@node02 log]$ sudo date -s 'Apr 01 2020 13:52:59 UTC'
Wed Apr 1 13:52:59 UTC 2020

The date can also be setup in shot hand notation as follows,but it is more cryptic

[vamshi@node02 log]$ sudo date 040113522020.50
Wed Apr 1 13:52:50 UTC 2020
$ sudo date mmddHHMMyyyy.SS

The format is month of the Year(mm),day of the month(dd),Hour of the day(HH),minute of the Hour(MM) and the Year(yyyy),and the Seconds of the minute(.SS)
Now, Lets dive deep and get to know the date options and Demonstration practical examples in this tutorial:

Another Important Option is -d or –date=”String” which can display the time described
Lets see some examples as follows:

By running the date command, we get an elaborate time and date format along with the TimeZone information.
To covert the Epoc time to human readable date, we can use date command as follows:

[vamshi@node02 log]$ date -d"@1585749164"
Wed Apr 1 13:52:44 UTC 2020

If you want to get a future date then use:

[vamshi@linuxcent ~]$ date -d "+130 days"
Sun Aug 16 02:07:35 UTC 2020

Date command offers a great flexibility to extract past and future dates as we will show below:

$ date "+ %F" -d “+30 days”
$ date "+ %F" --date “+30 days”

To get the date in history; go back to a date some days ago in Linux

[vamshi@node02 log]$ date -d "17 days ago"
Sun Mar 15 13:52:45 UTC 2020

Here we present some of the more useful Format options:

Date Format Command Explanation Result
date +”%a” Prints the Abbreviated Day of the Week Sat-Sun Wed
date +”%A” Prints the Day of the Week Saturday-Sunday Wednesday
date +”%b” Prints Abbreviated Month Jan-Dec Apr
date +”%B” Print un-abbreviated month January-December April
date +”%c” Prints Full Current Date and time format Wed Apr 1 13:52:43 UTC 2020
date +”%D” Prints dd/mm/yy date format 04/01/2020
date +”%d” Prints day of the month (01-31) 01
date +”%D” Prints Date in MM/DD/YY 04/01/20
date +”%e” Prints the Day of the month 01
date +”%F” Prints only the Full date as YYYY-MM-DD 2020-04-01
date +”%H” Prints the hour 00-23 13
date +”%I” Prints the hour in 00-12 01
date +”%j” Prints Julian day of the Year(001-366) 092
date +”%M” Prints the Minute of the hour 00-59 52
date +”%m” Prints the month of the year 01-12 04
date +”%n” Prints the newline character Newline/Empty line
date +”%N” Prints the nanoseconds counts 036416306
date +”%P” Prints AM/PM in the day PM
date +”%r” Get only time in AM/PM notation 13:52:43 PM
date +”%S” Get the current seconds count in the minute (00-60) 43
date +”%s” Get the number of seconds since 1st January 1970 (Epoch time) 1585749164
date +”%T” Time in 24 Hour format HH:MM:YY 13:52:43
date +”%u” Get  current day of the week
1-7		 3 for Wednesday
date +”%U” Get the current week of the Year considering Sunday as first week 13
date +”%V” Get the current week of the Year considering Monday as first week 14
date +”%y” or date +”%g” Prints only the last two digits of Year 20
Date +“%Y” or date +”%F” Prints Year in YYYY format 2020
Date +“%z” Prints the current Timezone difference from UTC 00 – for UTC
date +”%Z” Prints Alphabetic time zone abbreviation UTC

How to write the current system time to the Machine’s hardware clock ?

The command hwclock can do that for us.
[/code] # sudo hwclock [OPTIONS][/code]

Lets see a practical example where our Hardware clock was 1 hour and 13 mins behind the actual system time .

[vamshi@node02 ~]$ sudo hwclock
Wed 01 Apr 2020 07:35:05 AM UTC -0.454139 seconds
[vamshi@node02 ~]$ date
Wed Apr 01 08:43:13 UTC 2020

Setting the hardware clock time to system time with option -w or --systohc as seen below.

[vamshi@node02 ~]$ sudo hwclock -w

Confirm it with hwclock command as follows:

[vamshi@node02 ~]$ sudo hwclock
Wed 01 Apr 2020 08:44:05 AM UTC -0.538163 seconds

Most of the times the hardware clock will be out of sync with the system time and its a good practice to set the hardware clock in sync and comes in real handy during the system reboots.

What is the date format in Unix?

Below is a list of common date format options with examples output. It works with the Linux date command line and the mac/Unix date command line.

Date Format Option Meaning Example Output
date +%m-%d-%Y MM-DD-YYYY date format 05-09-2020
date +%D MM/DD/YY date format 05/09/20

How do I format a date in Linux?

These are the most common formatting characters for the date command:
  1. %D – Display date as mm/dd/yy.
  2. %Y – Year (e.g., 2020)
  3. %m – Month (01-12)
  4. %B – Long month name (e.g., November)
  5. %b – Short month name (e.g., Nov)
  6. %d – Day of month (e.g., 01)
  7. %j – Day of year (001-366)
  8. %u – Day of week (1-7)

What %D format in date command does?

%D: Display date as mm/dd/yy.

%d: Display the day of the month (01 to 31). %a: Displays the abbreviated name for weekdays (Sun to Sat). %A: Displays full weekdays (Sunday to Saturday).

How do you change the date in Unix?

The basic way to alter the system’s date in Unix/Linux through the command line environment is by using “date” command. Using date command with no options just displays the current date and time. By using the date command with the additional options, you can set date and time.

What is the date format?

Date Format Types
Format Date order Description
1 MM/DD/YY Month-Day-Year with leading zeros (02/17/2009)
2 DD/MM/YY Day-Month-Year with leading zeros (17/02/2009)
3 YY/MM/DD Year-Month-Day with leading zeros (2009/02/17)
4 Month D, Yr Month name-Day-Year with no leading zeros (February 17, 2009)

How can I get yesterday date in Unix?

  1. Use perl: perl -e ‘@T=localtime(time-86400);printf(“%02d/%02d/%02d”,$T[4]+1,$T[3],$T[5]+1900)’
  2. Install GNU date (it’s in the sh_utils package if I remember correctly) date –date yesterday “+%a %d/%m/%Y” | read dt echo ${dt}
  3. Not sure if this works, but you might be able to use a negative timezone.

How do I display yesterday’s date in Linux?

Yesterday date YES_DAT=$(date –date=’ 1 days ago’ ‘+%Y%d%m’)
Day before yesterdays date DAY_YES_DAT=$(date –date=’ 2 days ago’ ‘+%Y%d%m’)

Which command is used for displaying date and calendar in Unix?

Which command is used for displaying date and calendar in UNIX? Explanation: date command is used for displaying the current system date and time while cal command is used to see the calendar of any specific month/year.

Which command is used for displaying date in the format dd mm yyyy?

To use the date in MM-YYYY format, we can use the command date +%m-%Y. To use the date in Weekday DD-Month, YYYY format, we can use the command date +%A %d-%B, %Y.

What does df command do in Linux?

Display Information of /home File System. To see the information of only device /home file systems in human-readable format use the following command.

Docker ADD vs COPY statement

by
Reading Time: 4 minutes

The contents of the docker container image are created using personalized build artifacts and the configuration code which is copied from the build workspace during the docker build process, To achieve this purpose we use certain Docker DSL statements like ADD, COPY to dump the content to the docker image during the build process

It is good to throw some light into understanding the subtle differences between these statements.

It is important to send the up to date content to the Dockerfile and perform the build successfully, Lets see some practical case study of the Docker ADD vs COPY commands below:

Docker ADD statement

Docker COPY statement
Syntax:

ADD </host/Relative/path/to/source/> <Container/image/path/to/Destination/>
ADD [ "/host/path/source1/","/host/path/source2",.. "/container/path/target/" ]
 Syntax:

COPY </host/Relative/path/to/source/> <container/image/path/to/Destination/>
COPY ["/host/path/source1/","/host/path/source2/",.. "/container/path/target/"  ]
 
ADD [source1,source2],,. /path/to/dest/

With multiple source files, the target container path must end with a /

 
COPY [source1,source2],,. /path/to/dest/

With multiple source files, the target container path must end with a /
The Destination path inside container image can be Absolute or Relative to WORKDIR The Destination path inside container image can be Absolute or Relative to WORKDIR
The Destination path will be automatically created on target container with 0755 permissions The Destination path will be automatically created on target container with 0755 permissions 
ADD default.conf /tmp/default.conf

Creates the new files with the default 0644 permission

 
COPY default.conf /tmp/default.conf

Creates the new files with the default 0644 permission
The Destination content will be owned by the root user and the root group with uid & gid as 0 The Destination content will be owned by the root user and the root group with uid & gid as 0 
ADD directory /app
 
COPY directory /app
ADD Directory Explanation: The whole directory will be copied from the source host to target container with directory permission as 0755 COPY Directory Explanation: The whole directory will be copied from the source host to target container with directory permission as 0755 
ADD Portal.tar.gz /tmp/portal1/

COPY Portal.tar.gz /tmp/portal2/
Add Compressed file Explanation: The ADD command will extract the tar file and the extracted will be placed at the target container, with directory permissions as 0755 COPY Compressed file Explanation: The COPY command will NOT extract the tar files and places them as it is.. at the destination target container path as a single compressed tar file.
URL file as Source:

ADD https://ftp.gnu.org/gnu/wget/wget-1.6.tar.gz.sig /tmp/test-add/
 URL file as Source – Not possible with COPY command

Source can't be a URL for COPY
ADD URL Explanation: With ADD the URL download and archive unpacking features cannot be used together Meaning it will be not extract the compressed tar.bz2 or tar.gz formats when it downloads from a URL inside the target container path. But it works just like RUN wget command and downloads the compressed package. Explanation:
COPY command will not be able treat source as a URL and Hence its not a valid command

 

Conclusion:
It is better to use ADD command if you have the source archive files like tar.gz, tar.bz2 and want to send them into your container image and extract them, ADD command does it automatically whilst the COPY command sends it as it is at source.

Both the ADD and COPY commands cannot access the source content which are outside of its current relative context path.
Eg:

ADD ../source-code /data
COPY ../source-code /data

You should always keep this in mind whilst writing Docker files.

Feel free to comment and share your experiences with the COPY / ADD commands.

Whats the difference between ADD and COPY Docker?

COPY only supports the basic copying of local files into the container, while ADD has some features (like local-only tar extraction and remote URL support) that are not immediately obvious. Consequently, the best use for ADD is local tar file auto-extraction into the image, as in ADD rootfs. tar.

What is the difference between COPY and add?

COPY copies a file/directory from your host to your image. ADD copies a file/directory from your host to your image, but can also fetch remote URLs, extract TAR files, etc

What is Docker add?

The ADD command is used to copy files/directories into a Docker image. It can copy data in three ways: Copy files from the local storage to a destination in the Docker image. Copy a tarball from the local storage and extract it automatically inside a destination in the Docker image.

What does COPY mean in Docker?

Dockerfiles can contain several different instructions, one of which is COPY. The COPY instruction lets us copy a file (or files) from the host system into the image. This means the files become a part of every container that is created from that image.

When should I use Docker COPY?

If you are copying local files to your Docker image, always use COPY because it’s more explicit. While functionality is similar, the ADD directive is more powerful in two ways: It can handle remote URLs. It can also auto-extract tar files.

Is Docker a recursive COPY?

Docker provides two commands for copying files from the host to the Docker image when building it: COPY and ADD . … COPY — copies local files recursively, given explicit source and destination files or directories.

Does Docker COPY overwrite?

It seems that docker build won’t overwrite a file it has previously copied. I have a dockerfile with several copy instructions, and files touched in earlier COPY directives don’t get overwritten by later ones.

How do I COPY a Docker container?

Solution

  1. To copy a file from the local file system to a container, run the command for Docker container or Kubernetes
  2. pod, respectively: docker cp <src-path> <container>:<dest-path> …
    To copy a file from the container to the local file system, use: docker cp <container>:<src-path> <local-dest-path>

Does Docker COPY create directory?

From the dockerfile reference about COPY and ADD , it says Note: The directory itself is not copied, just its contents. , so you have to specify a dest directory explicitly.

How do I copy a file in Dockerfile?

To do so follow the below steps:

  • Step 1: Create a Directory to Copy. …
  • Step 2: Edit the Dockerfile. …
  • Step 3: Build the Docker Image. …
  • Step 4: Verifying the Docker Image. …
  • Step 5: Running the Docker Container. …
  • Step 6: Verify the Copying of the Directory.

How does copy work in Dockerfile?

COPY and ADD are both Dockerfile instructions that serve similar purposes. They let you copy files from a specific location into a Docker image. COPY takes in a src and destination. It only lets you copy in a local file or directory from your host (the machine building the Docker image) into the Docker image itself.

What are Docker layers?

Basically, a layer, or image layer is a change on an image, or an intermediate image. Every command you specify ( FROM , RUN , COPY , etc.) in your Dockerfile causes the previous image to change, thus creating a new layer.

How do I copy a docker image?

To export your image to a tar file, run the docker save command, specifying a name for the . tar file, and the docker image name. This will save the docker image locally.

SVN commands

by
Reading Time: 4 minutes

SVN is Version controlling system and predates git, It was one of the most widely used version control system and served the community and still going on.. It had its own share advantages and shortcomings.

In this tutorial we will shed some light on practical usecases of SVN and day-to-day activities.

How to take a dump of live svn repo by loading it to another Repo on the fly.

# svnadmin dump /opt/svn/ProjectCode | svnadmin load /opt/svn/ProjectCode

SVN commands to take the dump from repo to a .dump file

# svnadmin dump /opt/svn/PST > test.dump

Using svn comands svnadmin to take the dump for specific revision number range from commandline.

# svnadmin dump -r 41:3601 /opt/svn/ProjectCode > test.dump

SVN Commands to split svn directories to respective svn repositories:

We have the folder structure of the ProjectCode and want to create a individual repository of all the folders with their own SVN repos, We can use the svndumpfilter to achieve the task.

$ cat projectslist
./project1./project2
./project3
./project4

$ for i in `cat projectslist`; do sudo svndumpfilter --drop-empty-revs --renumber-revs exclude `cat projectslist | grep -v $i` < SVN.dump > `echo $i | sed -e 's/\///g'`.dump; done

How to setup up a SVN repo from linux commandline.

Creating Dir for new SVN

# mkdir /opt/svn/ProjectCode

Initialize a repo in the newly created directory

#svnadmin create /opt/svn/ProjectCode

Load the repo from dump

#svnadmin load --force-uuid /opt/svn/ProjectCode < newProjectCode.dump

Loading repo from dump and redirecting output to file

#svnadmin load --force-uuid /opt/svn/ProjectCode < newProjectCode.dump >> /root/pst.out &

Linux Commands To delete a repo; Make sure the svnserve process is in stopper state and not active while removing the repo.

# rm -rf /opt/svn/PST

While taking SVN repo dump How to Exclude some paths from existing dump and create a new dump
We will be using the exclude option and passing the exclude pattern

# svndumpfilter exclude `cat ProjectCode-exclude-somerolders` < ProjectCode.dump > new-exclude.dump

Below is a sample exclude file with specific file names and directory paths

# cat ProjectCode-excludelist
/path/to/code/not/required
/Some/more/paths/and/specific-filenames.css
/code/static/test/

How to check the logs from SVN repo from commandline

# svn log file:///opt/svn/Project

How to setup SVN repository with configuration ?

# svnadmin create /opt/svn

Below is the example snippet from svnserve.conf

[general]
anon-access = none
auth-access = write
password-db = /opt/svn/conf/passwd
authz-db = /opt/svn/conf/authz

Here is the sample config snipet from authz

[/]
svamshi = rw

How to start SVN server from linux commandline

# svnserve --listen-host ip-address/project-repo -r /opt/svn/my-projectcode/ -d

How to checkout a SVN repo from linux commandline

# svn co --username=svamshi --password=mypassword svn://ip-address/project-repo

Adding the new created files to svn and then commit to svn.

# svn add linuxcent.html
# svn commit -am "updated with linuxcent.html"

How to serve the SVN repository through Apache Httpd server.

Example [/code]/opt/svn[/code]The svn module has to be downloaded

Below is the sample configuration of virtualhost informantion

subversion-apache-httpd-set-up-configuration

What are the svn commands?

Here are the basic SVN commands that every developer and admin should know.
svn admincreate. The svn admincreate command creates a new, empty repository.

  • svn import. …
  • svn checkout. …
  • svn commit. …
  • svn add. …
  • svn delete. …
  • svn list. …
  • svn diff.

How do I use svn in Linux?

  1. Connect via SSH. In order to install SVN, connect to your Linux VPS via SSH. …
  2. Update the OS Packages and Install SVN in Linux. If you are using an Ubuntu VPS, update the OS packages and install SVN on your server using the commands below: sudo apt-get update sudo apt-get install svn. …
  3. Check SVN Version

What is svn up command?

The svn update command lets you refresh your locally checked out repository with any changes in the repository HEAD on the server. It also tells you what has been changed, added, deleted. If a change has been made to a file you have also changed locally, svn will try to merge those changes.

How do I run svn?

  1. Open windows explorer.
  2. Create a folder where you will store project files.
  3. Right-click on the folder you created and select “SVN Checkout” (see image below).
  4. When prompted, enter your username and password.
  5. If everything worked, you now have a copy of the repository in your directory.

Which is Better Git or svn?

Why SVN Is Better Than Git

SVN is better than Git for architecture performance, binary files, and usability. And it may be better for access control and auditability, based on your needs.

Where are svn commands executed?

If you want to run Subversion commands from the command prompt, you should run the svn.exe command line client. TortoiseSVN 1.6.

How do I use shelve in svn?

To shelve your local changes, select your working copy and use Context Menu → Shelve The following dialog allows you to select the files you want to shelve and give a name under which you want to store them. If you select an existing shelf, then a new version is created for that shelf.

Does svn work on Linux?

SVN Installation

Subversion is a popular open-source version control tool. It is open-source and available for free over the internet. It comes by default with most of the GNU/Linux distributions, so it might be already installed on your system.

How configure svn server in Linux?

How to Install SVN Server on Ubuntu 18.04 & 16.04 LTS

  • Step 1 – Install Apache. …
  • Step 2 – Install SVN Server. …
  • Step 3 – Create First SVN Repository. …
  • Step 4 – Create Users for Subversion. …
  • Step 5 – Configure Apache with Subversion. …
  • Step 6 – Access Repository in Browser.

Where is svn installed on Linux?

Did you try whereis svn ? Executables are usually in /usr/bin or /usr/local/bin .

How do you svn add all files?

To add an existing file to a Subversion repository and put it under revision control, change to the directory with its working copy and run the following command: svn add file… Similarly, to add a directory and all files that are in it, type: svn add directory

How to start Jenkins on a different port

by
Reading Time: 3 minutes

The jenkins Build and Integration application based on java programming language, tends to have its roots stuck to the typical Tomcat web server
Out of the box the jenkins starts up on the default port 8080 and this can be customized on the system or a new proxy routing can be setup which might add a slight overhead on the jenkins inbound requests,

How to change the Jenkins default Port?

We have the Jenkins default home located at /var/lib/jenkins all the configuration files are present in this location.
The important configuration although is /etc/sysconfig/jenkins which is present outside $JENKINS_HOME
The Jenkins Default JENKINS_PORT parameter is set to 8080

[root@node02 jenkins]# grep JENKINS_PORT /etc/sysconfig/jenkins
JENKINS_PORT="8080"

If you are on Debian or Ubuntu linux, then the relevant file will be <span class="st">/etc/default/jenkins
We will be now changing the variable $JENKINS_HOME to listen to the port 9090 and restarting the jenkins service.

[root@node02 jenkins]# systemctl restart jenkins.service

You can check for the port information from the ps -ef | grep jenkins output as follows or checkout from the netstat command

[root@node02 jenkins]# ps -ef |grep jenkins
jenkins 5110 1 90 23:04 ? 00:00:02 /etc/alternatives/java -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --daemon --httpPort=9090 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20

How do I change the default Jenkins port in Linux?

  1. First, run this command to open jenkins configurations: sudo nano /etc/default/jenkins.
  2. The only part you need to change is: #port for HTTP connector (default 8080; disable with -1) Http_port = 8080. …
  3. Finally, Restart Jenkins service by running this command: sudo service jenkins restart.

How do I change my local Jenkins port?

  1. Go to the directory where you installed Jenkins (by default, it’s under Program Files/Jenkins)
  2. Open the Jenkins.xml configuration file.
  3. Search –httpPort=8080 and replace the 8080 with the new port number that you wish.
  4. Restart Jenkins for changes to take effect.

How do I change my Jenkins port from 8080 to 80?

  1. Go to /etc/default folder –> Open the file “jenkins”
  2. Modify the line HTTP_PORT=8080 as HTTP_PORT=80.
  3. Start jenkins as root by using the command: sudo /etc/init.d/jenkins start.
  4. Open a browser and browse as localhost:80.

What port is Jenkins running on?

The default Jenkins installation runs on ports 8080 and 8443. Typically, HTTP/HTTPS servers run on ports 80 and 443, respectively. But these ports are considered privileged on Unix/Linux systems, and the process using them must be owned by root.

How do I start Jenkins on a different port?

How to change the default port in Jenkins

  1. Go to C:\Program Files (x86)\Jenkins (I’m using Windows Server 2012 and assuming it’s installed to default location)
  2. Open Jenkins.xml.
  3. Edit the –httpPort argument (you may need to edit default permissions)
  4. Restart the Jenkins service.
  5. Now Jenkins will permanently use the new port.

How do I run a Jenkins war on another port?

All you need to do:

  1. Goto Jenkins folder present in C:\Program Files (x86)
  2. Open a notepad or text pad and run them as administrator and then try opening the jenkins. xml file present in the jenkins folder.
  3. Change the port number as below: <arguments>-Xrs -Xmx256m -Dhudson. lifecycle=hudson. lifecycle.

How do I change my Jenkins URL?

Fixing a root url

  1. Go to Jenkins > Manage Jenkins > Configure System, and locate the section titled “Jenkins Location”. You should see the warning here as well.
  2. Replace “localhost” with a valid hostname.
  3. Click Save.

How do I start Jenkins manually in Linux?

Go to the Jenkins installation, open the cmd and run:

  1. To stop: jenkins.exe stop.
  2. To start: jenkins.exe start.
  3. To restart: jenkins.exe restart.

What is Jenkins port 50000?

Jenkins runs on Tomcat, which uses port 8080 as the default. -p 5000:5000 required to attach slave servers; port 50000 is used to communicate between master and slaves.

How do I run Jenkins on port 443?

Basic solution (complete):

  1. You need a “Java keystore” of the SSL-certificate you want to use. …
  2. Copy the certificate, private key and (if present) intermediate CAs to your Jenkins host. …
  3. Convert the certificate-files to one single-filed PKCS12 container. …
  4. Make sure, that the Java “keystore”-command is present.

Bash How to Print Array

by
Reading Time: 3 minutes

Bash How to Print Array

Arrays are collection of elements, The Arrays in bash are indexed from 0 (zero-based).
Below is the definition on an Array in Bash
my_array=(zero one two three four)

Now our array is defined.
Here is exactly how the my_array is stored on BASH:

my_array=([0]="zero" [1]="one" [2]="two" [3]="three" [4]="four")

You can explicit define an array:

declare -a MY_ARRAY

You can view the declarations along with other environment variables using the declare command.

declare -p my_array
declare -a my_array=([0]="zero" [1]="one" [2]="two" [3]="three" [4]="four")

Now if you try to print the array:

my_array=(zero one two three four)
echo $my_array
zero

By default only the first element value is printed which belongs to the 0 index.

To print the first element of the array using the indexing:

my_array=(zero one two three four)
echo ${my_array[0]}
zero

The change we noticed here is the use of the Curly Braces ‘{}’, its used to refer to the value of an item in the array. The curly braces are required to avoid issues with path name expansion.

To read all elements of the array use the symbols “@” or “*”.

echo ${my_array[@]}
zero one two three four

echo ${my_array[*]}
zero one two three four

The difference between “$@” and “$*” is “$@” expands each element as a separate argument, however “$*” expand to the arguments merged into one argument.

To prove this, print the index elements followed by $@ or $* format.

echo ${my_array[$*0]}
zero
echo ${my_array[$@1]}
one
echo ${my_array[$*2]}
two

Getting the Length of the Array

If you need to get the length of the array uses the symbol “#” before the name of the array:

echo "${#my_array[*]}"
5

How do I print an array in bash?

Print Bash Array

We can use the keyword ‘declare’ with a ‘-p’ option to print all the elements of a Bash Array with all the indexes and details. The syntax to print the Bash Array can be defined as: declare -p ARRAY_NAME.

How do I print in bash?

After typing in this program in your Bash file, you need to save it by pressing Ctrl +S and then close it. In this program, the echo command and the printf command is used to print the output on the console.

How do you print an array element in a new line in Shell?

To print each word on a new line, we need to use the keys “%s’\n”. ‘%s’ is to read the string till the end. At the same time, ‘\n’ moves the words to the next line. To display the content of the array, we will not use the “#” sign.

How do I display all array elements at once?

Program:
  1. public class PrintArray {
  2. public static void main(String[] args) {
  3. //Initialize array.
  4. int [] arr = new int [] {1, 2, 3, 4, 5};
  5. System. out. println(“Elements of given array: “);
  6. //Loop through the array by incrementing value of i.
  7. for (int i = 0; i < arr. length; i++) {
  8. System. out. print(arr[i] + ” “);

How do I create an array in bash?

  1. To declare your array, follow these steps:
    Give your array a name.
  2. Follow that variable name with an equal sign. The equal sign should not have any spaces around it.
  3. Enclose the array in parentheses (not brackets like in JavaScript)
  4. Type your strings using quotes, but with no commas between them.

How do you create an array in bash?

Define An Array in Bash

You have two ways to create a new array in bash script. The first one is to use declare command to define an Array. This command will define an associative array named test_array. In another way, you can simply create Array by assigning elements.

How does printf work in bash?

What Is the Bash printf Function? As the name suggests, printf is a function that prints formatted strings of text. That means you can write a string structure (the format) and later fill it in with values (the arguments). If you’re familiar with the C/C++ programming languages, you might already know how printf works.

How do you pass an array to a function in bash?

10 Answers

  1. Expanding an array without an index only gives the first element, use copyFiles “${array[@]}” instead of copyFiles $array.
  2. Use a she-bang #!/bin/bash.
  3. Use the correct function syntax. Valid variants are function copyFiles {… …
  4. Use the right syntax to get the array parameter arr=(“$@”) instead of arr=”$1″

How do you create an empty array in bash?

To declare an empty array, the simplest method is given here. It contains the keyword “declare” following a constant “-a” and the array name. The name of the array is assigned with empty parenthesis.

How do I get the size of an array in bash?

To get the length of an array, we can use the {#array[@]} syntax in bash. The # in the above syntax calculates the array size, without hash # it just returns all elements in the array.

git config system vs global vs local

by
Reading Time: 4 minutes

Setting up your environment while working with GIT is essential as part of user authentication to the remote code base.

The git config can be applied at 3 levels:

  • First is at system level scope:

git config --system : Resulting in the file /etc/gitconfig getting modified.

  • Second is at the Global level scope:

git config --global : The resultant configuration is present at /home/<username>/.gitconfig

  • Third is the local level scope

git config --local : The resultant configuration is present under .git/config of the current repo workspace

Lets examine the changes on various levels with the git config.

The git config at system level requires the root level permissions and provides the unique user identify to the whole system.
Lets see the demonstration as below:

[vamshi@linuxcent ~]$ git config --system user.name vamshi
error: could not lock config file /etc/gitconfig: Permission denied

We need to run the git config –system with sudo permissions.

[vamshi@linuxcent ~]$ sudo git config --system user.name vamshi
[vamshi@linuxcent ~]$ sudo git config --system user.email [email protected]
[vamshi@linuxcent ~]$ cat /etc/gitconfig 
[user]
	name = vamshi
	email = [email protected]

In the case of –global, The config is limited to your home directory and not on a system level and It is best advised as if you have multiple users using the system.

git config --global 
[vamshi@linuxcent ~]$  git config --global user.name vamshi
[vamshi@linuxcent ~]$  git config --global user.email [email protected]

To ensure the changes are made at the user level on the home directory of a user, permanently updates users’ /home/username/.gitconfig file as below:

With this the configuration at global level in the user home directory is generated and stored into the file ~/.gitconfig as shown below:

[vamshi@linuxcent ~]$ cat ~/.gitconfig 
[user]
name = vamshi
email = [email protected]

The other entries can be added to the git config to accept connections from Insecure https site as shown below.

# git config --global http.sslVerify false

Resulting in modification of the global ~/.gitconfig file

[vamshi@workstation ~]$ cat ~/.gitconfig
[user]
	name = vamshi
	email = [email protected]
[http]
	sslVerify = false

With these settings in place now we should be well placed and would have no issues in accessing our gitlab repos.

Adding the changes to with git config –local which only reflect the specific git repository, we will see a practical demonstration as below.

[vamshi@node02 pipeline-101]$ git config --local user.name vamshi
[vamshi@node02 pipeline-101]$ git config --local user.email [email protected]
[vamshi@node02 pipeline-101]$ cat .git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = https://gitlab.linuxcent.com/linuxcent/pipeline-101.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
[user]
	name = vamshi
	email = [email protected]

Could not lock config file config file exist?

This will happen, if the user home directory is not writable by the user. git config –global needs to create a “lock” file ( ~/. gitconfig. … If a user has no permission of creating this file, You must check and change permissions for the home directory.

Where is Gitconfig?

The system level configuration file lives in a gitconfig file off the system root path. $(prefix)/etc/gitconfig on unix systems. On windows this file can be found at C:\Documents and Settings\All Users\Application Data\Git\config on Windows XP, and in C:\ProgramData\Git\config on Windows Vista and newer.

How do I change .gitconfig location?

Changing . gitconfig location on Windows

  1. Move your . gitconfig from user home directory, to the directory where you want.
  2. Run command line as Administrator.
  3. Go to your user home directory.
  4. Enter mklink .gitconfig \PathForNewLocationOfConfig.gitconfig.

How do I add a config file to github?

In the working directory of your git repo, you would do the following:

  • Create a version of the config file called “config-sample. …
  • Create a symbolic link between “config. …
  • Update your .gitignore to prevent the “config.ini” from being stored. …
    (Optional, but highly recommended) Create a . …
  • Do some coding and deploy…

How do I open Gitconfig?

Open a terminal of your choice. You can also use the shortcut key Ctrl+Alt+T to open a terminal.
In your terminal type : git config –global –edit It will open your global configuration file of git in your default editor.
Change the Settings that you want.

What is Gitconfig file?

gitconfig is used to store a per-user configuration as fallback values for the . git/config file. The file /etc/gitconfig can be used to store a system-wide default configuration. The configuration variables are used by both the Git plumbing and the porcelains.

How do I change my git config file?

The global git config is simply a text file, so it can be edited with whatever text editor you choose. Open, edit global git config, save and close, and the changes will take effect the next time you issue a git command. It’s that easy.

Where is my Gitconfig file Mac?

The global Git configuration file is stored at $HOME/. gitconfig on all platforms. However, you can simply open a terminal and execute git config , which will write the appropriate changes to this file.

How do I change git config in Windows?

gitconfig on my Windows machine and found this neat trick using git. Do a: git config –global -e and then, if you are lucky, you will get a text editor loaded with your global . gitconfig file.

How do I change my git config name?

  1. In your terminal, navigate to the repo you want to make the changes in.
  2. Execute git config –list to check current username & email in your local repo.
  3. Change username & email as desired. Make it a global change or specific to the local repo: git config [–global] –replace-all user.name “Full Name” …
  4. Done!

Jenkins admin password reset

by
Reading Time: 3 minutes

The config file contains the XML tags to enforce the security, you have to disable this option by changing it to false from true.

If you happen to forget your admin account jenkins password, Then you need to have the shell acccess to your jenkins setup
and modify config.xml and restart the jenkins service..
The passwords in jenkins are one way hash and they are encrypted using jbcrypt.

Lets look at a couple of ways to reset the password, first is disable to global security on jenkins server by modifying the /var/lib/jenkins/config.xml as shown below:
First method is to completely disable the security on Jenkins so that you can access the Jenkins without any password prompt, which means any body on the network can do so if they have the knowledge about it. Please advise caution.

[vamshi@linuxcent jenkins]$ sudo grep useSecurity -C1 /var/lib/jenkins/config.xml
<mode>NORMAL</mode>
<useSecurity>false</useSecurity>
<authorizationStrategy class="hudson.security.FullControlOnceLoggedInAuthorizationStrategy">

Proceed to restart the jenkins service from commandline.

Login to the GUI and then update the password. These files contain the jenkins login information.

How to reset the user password of jenkins ?

The second method being directly modifying the jbcrypt hashed password present in the file /var/lib/jenkins/users/<Your UserName>/config.xml
For example:

[root@linuxcent jenkins]# grep password /var/lib/jenkins/users/admin_353942241645223362/config.xml
<passwordHash>#jbcrypt:$2a$10$razdYOUR-OLD-HASH.IVrFydsxkcQCcLmujmFQz12345678</passwordHash>

Now replace the jbcrypt hash code line with this
<passwordHash>#jbcrypt:$2a$10$uEbq9yc/UErlbO0BjqcaNutxTXueFcTkbad1jR9iOeOiNx5koRFi6</passwordHash>
and save the file. The decrypted password is admin and use this password for the next time login after restating jenkins service.

This process will come in real handy in case you forgot your jenkins login password
Alternately you can use this hash
#jbcrypt:$2a$10$razd3L1aXndFfBNHO95aj.IVrFydsxkcQCcLmujmFQzll3hcUrY7S — for a decrypted password called test

NOTE: This is only a hash is a simple temporary password to login to the jenkins and needs to be change to a stronger password

Now save the file /var/lib/jenkins/users/admin_353942241645223362/config.xml and restart the Jenkins server:

$ sudo systemctl restart jenkins

Now navigate to your Jenkins UI, enter your username and the password “admin” to login.
Once you are logged in make sure to navigate to http://jenkins.linuxcent.com:8080/user/admin/configure
eg: http://jenkins.linuxcent.com:8080/user/<Your Username>/configure

Changing admin password in Jenkinsand Now enter a preferred new strong password and save the changes, You will be automatically logged out and then logback in with the new password.

How do I find my Jenkins admin password?

For this the Username is admin. Password should be located in: $JENKINS_HOME/secrets/initialAdminPassword.
You can view the password using: cat /var/lib/jenkins/secrets/initialAdminPassword.
cat $JENKINS_HOME/secrets/initialAdminPassword.

What is Jenkins default admin password?

Initializing Jenkins. The first time you start Jenkins, the configuration is created along with the administrator user and password. The default login is admin/password.

How do I find my username and password for Jenkins Windows?

  • Copy the initialAdminPassword in Specified path.
  • Login with following Credentials User Name : admin Password : <da12906084fd405090a9fabfd66342f0>
  • Once you login into the jenkins application you can click on admin profile and reset the password.

How do I log into Jenkins without a password?

How to Reset Jenkins Admin User Password

  • Enable “Enable Security’ option.
  • In Security Realm option select “Jenkins own database”
  • Make sure to uncheck “Allow users to sign up” option under the “Jenkins own database” option.
  • Save Changes – Save all changes made above.

How do I change my Jenkins UI password?

Reset Jenkins Admin Password

Select the “Security Realm” (e.g. “Jenkins’ own user database” ) and click on “Save” Go to “People” -> Click on a username for which you want to change the password (e.g. admin ) -> “Configure” -> Enter a new password in the “Password” and “Confirm password” fields and click on “Save”

How do I create a Jenkins user and password?

  • Steps to create a new user
  • Login into Jenkins.
  • Go to Manage Jenkins.
  • Go to Create Users.
  • Enter all the details – Username, Password, Confirm Pwd, FullName, Email.
  • Select Create User.

How do I reset Jenkins to default?

  • In config. xml , set disableSignup to false .
  • Restart Jenkins.
  • Go to the Jenkins web page and sign up with a new user.
  • In config. …
  • If it’s a private server, set disableSignup back to true in config. …
  • Restart Jenkins.
  • Go to the Jenkins web page and log in as the new user.
  • Reset the password of the original user.

How do I restart Jenkins?

  • Go to the Jenkins installation, open the cmd and run:
  • To stop: jenkins.exe stop.
  • To start: jenkins.exe start.
  • To restart: jenkins.exe restart.

How do I unlock Jenkins?

  • Unlocking Jenkins
  • From the Jenkins console log output, copy the automatically-generated alphanumeric password (or) open the file located in /var/Jenkins_home/secrets/initialAdminPassword .
  • On the Unlock Jenkins page, paste this password into the Administrator password field and click Continue.

How do I reset my Jenkins password on Mac?

  • To Reset individual password:
  • Go to /opt/bitnami/apps/jenkins/jenkins_home/users/gmhawash.
  • Edit config. xml file.
  • Remove the passwordHash tag (you can now log in without password)

How to access Tar files in Linux/Unix

by
Reading Time: 5 minutes

The Linux tar command abbreviation is “tar archive” released under POSIX standards initially and It now follows the GNU standards

Often when you download the files from the internet they are in the .tar, tar.gz. Or tar.bz2 compressed format using either bz2 or gz compression algorithm.

If you are familiar with the Opensource then there’s a good chance that you would have come across the extensions like Package-name.tar, tar.gz, tar.bz, tar.xz which are standard.

Well most open source software use tarballs to distribute programs/source codes in this format as it offers efficient compression and better organized way of grouping files.

It supports a vast range of compression programs such as gzip, bzip2, xz, lzip, lzma, lzop.

In the following tutorial we will show how to Compress/Extract the files into tar.gz or tgz.

How to specify the format POSIX Or GNU while using tar Command?

[vamshi@linuxcent ]$ tar --format=posix -cf posix.tar *
[vamshi@linuxcent ]$  tar --format=gnu -cf gnu.tar *
[vamshi@linuxcent ]$  file posix.tar gnu.tar 
posix.tar: POSIX tar archive
gnu.tar: POSIX tar archive (GNU)

GNU is based on an older POSIX format, So that’s why it says POSIX for both.

To print the contents information of an archived directory, use the directory name as a file name argument in conjunction with --list (-t). To find out file attributes, include the --verbose (-v) option.

Firstly start off by long listing ls in the present directory we have here:

vamshi@LinuxCent:~/Linux> ls
Debian-Distro/  OpenSuse-Distro/ README  Redhat-Distro/

We will now compress the present working directory using Linux GNU’s tar command line utility.. Creating a tar of the current directory mentioned by Asterisk *. The Options -c creates, -v: Verbose mode, -z: Uses GZIP algorithm
Demonstration shown below:
Either one of the following commands can be used

vamshi@LinuxCent:~/Linux> tar --gzip -cvf Linux-flavours.tar.gz *
vamshi@LinuxCent:~/Linux> tar -cvzf Linux-flavours.tar.gz *
Debian-Distro/
Debian-Distro/ubuntu.txt
Debian-Distro/debian.txt
Debian-Distro/README-Debian-Distro
Linux-flavours.tar.gz
OpenSuse-Distro/
OpenSuse-Distro/README-Opensuse-Distro
OpenSuse-Distro/opensuse.txt
README
Redhat-Distro/
Redhat-Distro/Fedora/
Redhat-Distro/Fedora/fedora.txt
Redhat-Distro/Centos/
Redhat-Distro/Centos/centos.txt
Redhat-Distro/Centos/CentOS-versions/
Redhat-Distro/Centos/CentOS-versions/centos7.txt
Redhat-Distro/Centos/CentOS-versions/centos5.5.txt
Redhat-Distro/Centos/CentOS-versions/centos6.9.txt
Redhat-Distro/Centos/CentOS-versions/centos5.8.txt
Redhat-Distro/Centos/CentOS-versions/centos6.1.txt
Redhat-Distro/Centos/README-CentOS
Redhat-Distro/README-Redhat-Distro
Redhat-Distro/RHEL-Versions/
Redhat-Distro/RHEL-Versions/redhat5.txt
Redhat-Distro/RHEL-Versions/redhat7.txt
Redhat-Distro/RHEL-Versions/redhat6.txt
Redhat-Distro/RHEL-Versions/redhat8.txt
Redhat-Distro/redhat.txt

How to List/view Archives

The option-t does a Dry-run of extract operation but only to print the contents of the mentioned compression.

vamshi@LinuxCent:~/Linux> tar -tvf Linux-flavours.tar.gz
Debian-Distro/
Debian-Distro/ubuntu.txt
Debian-Distro/debian.txt
Debian-Distro/README-Debian-Distro
…

How to Extracting tar.gz File?

Extracting the tar file using the options -x works out just fine as -x option chooses the tye of decompression based on the compression file type, and the content will be extracted to current working directory.

Here are various extraction options:

$ tar -xvf Linux-flavours.tar.gz
$ tar -zxvf Linux-flavours.tar.gz
$ tar --gzip -xvf Linux-flavours.tar.gz

The Filter / Options for compression types:
-z or --gzip : Used for Archival operation for .gzip type
-j -r --bzip2: Used for archival operation for .bzip2 type
[/code]-J[/code] or --xz: User for Archival for .xz type

How to Extract Only Specific Files from a compressed tar archive (tar) File

This doesn’t require any special option but you have to name the exact file and directory that you want to extract

[vamshi@linuxcent ~]$ tar -zxvf redmine-4.0.6.tar.gz redmine-4.0.6/bin/about
redmine-4.0.6/bin/about

Successful in extracting only the redmine-4.0.6/bin/about file from the redmine-4.0.6.tar.gz archive.

[vamshi@linuxcent ~]$ ls -l redmine-4.0.6/bin/about 
-rwxrwxr-x. 1 vamshi vamshi 167 Dec 20 11:46 redmine-4.0.6/bin/about

How to Extract specific files or Directories from the archive

vamshi@LinuxCent:/tmp/linux-test> tar -zxvf Linux-flavours.tar.gz Redhat-Distro/Centos/
Redhat-Distro/Centos/
Redhat-Distro/Centos/centos.txt
Redhat-Distro/Centos/CentOS-versions/
Redhat-Distro/Centos/CentOS-versions/centos7.txt
Redhat-Distro/Centos/CentOS-versions/centos5.5.txt
Redhat-Distro/Centos/CentOS-versions/centos6.9.txt
Redhat-Distro/Centos/CentOS-versions/centos5.8.txt
Redhat-Distro/Centos/CentOS-versions/centos6.1.txt
Redhat-Distro/Centos/README-CentOS

 

This way we are able to extract only the specific directories pertaining to Centos from Redhat-Distro.
Now we would do the extraction of the contents on Redhat-Distro and its sub directories.
Extracting Sub-directories inside the compressed tar.gz

How to Extract TAR file Contents to a New directory?

The tar program by default, extracts the archive contents to the present working directory.
By specifying the option --directory (-C) You can extract archive files in a Target directory.

vamshi.santhapuri@LinuxCent:/tmp/linux-test1> tar -xzvf ~/Linux/jdk-8u101-linux-x64.tar.gz  -C /usr/local/

The above operation extracts the java bundle to /usr/local/ directory.

How to extract only the specific directory from the Compression to a Target directory?

vamshi.santhapuri@LinuxCent:/tmp/linux-test1> tar -xzvf ~/Linux/Linux-flavours.tar.gz Redhat-Distro/RHEL-Versions/ -C /tmp/linux-test
Redhat-Distro/RHEL-Versions/
Redhat-Distro/RHEL-Versions/redhat5.txt
Redhat-Distro/RHEL-Versions/redhat7.txt
Redhat-Distro/RHEL-Versions/redhat6.txt
Redhat-Distro/RHEL-Versions/redhat8.txt

How to tar compress the specific Files and Directory and Sub-directories / Multiple Directories using tar command?

Below is the Demonstration of compression program using gz compression.

vamshi@LinuxCent:~/Linux> tar -cvzf Redhat-Distro/ Linux-flavours.tar.gz centos.txt fedora.txt opensuse.txt redhat.txt ubuntu.txt
centos.txt
fedora.txt
opensuse.txt
redhat.txt
Redhat-Distro/centos.txt
Redhat-Distro/fedora.txt
Redhat-Distro/redhat.txt
Redhat-Distro/Redhat-Versions/
Redhat-Distro/Redhat-Versions/redhat5.txt
Redhat-Distro/Redhat-Versions/redhat7.txt
Redhat-Distro/Redhat-Versions/redhat6.txt
Redhat-Distro/Redhat-Versions/redhat8.txt

You can also compress multiple directories and files into a single tar.gz file as demonstrated below

How to exclude particular directories and file from the compression using tar command?

We can make use of the --exclude="DontIncludethisPath" in Linux tar command, where in the base directory of mentioned pattern are excluded.. Lets run the tar command and see the results

vamshi@linuxCent:~/Linux/OSes> tar -czvf exclude-flavours.tar.gz --exclude="Redhat-Distro" .
./
./OpenSuse-Distro/
./OpenSuse-Distro/README-Opensuse-Distro
./OpenSuse-Distro/opensuse.txt
./Debian-Distro/
./Debian-Distro/ubuntu.txt
./Debian-Distro/debian.txt
./Debian-Distro/README-Debian-Distro
tar: .: file changed as we read it

The compression successfully completed and now lets list out the files

vamshi@linuxCent:~/Linux/OSes> ls -lthr
total 16K
drwxr-xr-x 2 vamshi users 4.0K Apr  8 14:09 OpenSuse-Distro
drwxr-xr-x 2 vamshi users 4.0K Apr  8 14:09 Debian-Distro
drwxr-xr-x 5 vamshi users 4.0K Apr  8 14:09 Redhat-Distro
-rw-r--r-- 1 vamshi users  718 Apr 8 19:18 exclude-flavours.tar.gz

From the dry-run extract output of the compression does not contain the Directory Redhat-Distro

vamshi@linuxCent:~/Linux/OSes> tar -tvf exclude-flavours.tar.gz
drwxr-xr-x vamshi/users 0 2020-04-08 19:18 ./
drwxr-xr-x vamshi/users 0 2020-04-08 14:09 ./OpenSuse-Distro/
-rw-r--r-- vamshi/users 0 2020-04-08 14:09 ./OpenSuse-Distro/README-Opensuse-Distro
-rw-r--r-- vamshi/users 9 2020-04-08 13:22 ./OpenSuse-Distro/opensuse.txt
drwxr-xr-x vamshi/users 0 2020-04-08 14:09 ./Debian-Distro/
-rw-r--r-- vamshi/users 7 2020-04-08 13:22 ./Debian-Distro/ubuntu.txt
-rw-r--r-- vamshi/users 7 2020-04-08 13:50 ./Debian-Distro/debian.txt
-rw-r--r-- vamshi/users 0 2020-04-08 14:09 ./Debian-Distro/README-Debian-Distro

How do I open a tar file in Linux?

tar Utility

  1. Let’s consider that we want to extract and open a doc file, and then we can use the below command to unzip the
  2. file on Linux:
  3. tar –xvzf doc.tar.gz. Remember that the tar. …
  4. tar –cvzf docs.tar.gz ~/Documents. …
  5. tar -cvf documents.tar ~/Documents. …
  6. tar –xvf docs.tar. …
  7. gzip xyz.txt. …
  8. gunzip test.txt. …
  9. gzip *.txt.

How do I view the contents of a tar file in Unix?

tar File Content. Use -t switch with tar command to list content of a archive. tar file without actually extracting.

How do I view a tar file?

How to open TAR files

  • Download and save the TAR file to your computer. …
  • Launch WinZip and open the compressed file by clicking File > Open. …
  • Select all of the files in the compressed folder or select only the files you want to extract by holding the CTRL key and left-clicking on them.

Where can I find tar files in Linux?

Combining find and tar commands so that we can find and tar files into a tarball

  1. -name “*. doc” : Find file as per given pattern/criteria. In this case find all *. doc files in $HOME.
  2. -exec tar … : Execute tar command on all files found by the find command.

How do I extract a tar file?

To extract (unzip) a tar. gz file simply right-click on the file you want to extract and select “Extract”. Windows users will need a tool named 7zip to extract tar.

How do I unzip a tar file in Terminal?

The most common uses of the tar command are to create and extract a tar archive. To extract an archive, use the tar -xf command followed by the archive name, and to create a new one use tar -czf followed by the archive name and the files and directories you want to add to the archive.

How do I extract a tar file from a directory in Linux?

Syntax For Tar Command To Extract Tar Files To a Different Directory

  1. x : Extract files.
  2. f : Tar archive name.
  3. –directory : Set directory name to extract files.
  4. -C : Set dir name to extract files.
  5. -z : Work on . tar. …
  6. -j : Work on . tar. …
  7. -J (capital J ) : Work on . tar. …
  8. -v : Verbose output i.e. show progress on screen.

Can 7 Zip open tar files?

7-Zip can also be used to unpack many other formats and to create tar files (amongst others). Download and install 7-Zip from 7-zip.org. … Move the tar file to the directory you wish to unpack into (usually the tar file will put everything into a directory inside this directory).

What is tar file in Linux?

The Linux ‘tar’ stands for tape archive, is used to create Archive and extract the Archive files. tar command in Linux is one of the important command which provides archiving functionality in Linux. We can use Linux tar command to create compressed or uncompressed Archive files and also maintain and modify them.

Build context in Dockerfile; Best practices

by
Reading Time: 5 minutes

Best practices while building the Dockerfile.

The context in Dockerfile is relative to the current working directory of the Dockerfile and that the location where Dockerfile is present becomes its context.

Which means we can create a Directory with some content and place our Dockerfile inside it and then traverse a number of directories away from the directory and can still execute the build command

Here is an example of out general approach to building an image from a Dockerfile with . context:

# docker build --tag nginx-linuxcent .

And we list the image as follows:

[vamshi@docker01 ~]$ docker images
REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE
nginx-linuxcent                              latest              0b0a4ea4d48a        3 minutes ago      210.1 MB

The build context is a . dot and the Dockerfile is present in the same directory.
If You are working locally you don’t really need a repository name and specifying just the image name is sufficient and then adding a tag is considered optional, in such cases a latest tag is appended to the end of the newly build image

As a standard practice that the Dockerfile doesn’t traverse back from the current working directory. Lets see an demonstration of of building a Dockerfile by giving the relative path from its Dockerfile.

Example given:

[vamshi@node01 ~]$ ls nginx/
default.conf Dockerfile-nginx index.html nginx.conf Portal.tar.gz

Here is our nginx/Dockerfile-nginx.

cat Dockerfile-nginx
FROM nginx:1.17.2-alpine
COPY index.html /usr/share/nginx/html/
ADD Portal.tar.gz /tmp/new1/portal
CMD ["/usr/sbin/nginx"]

Our command to build this Dockerfile-nginx now becomes:

[vamshi@docker01 ~]$ docker build -t nginx-linuxcent -f nginx/Dockerfile-nginx nginx/
Sending build context to Docker daemon 155.4 MB
Step 1/4 : FROM nginx:1.17.2-alpine
---> 55ceb2abad47
Step 2/4 : COPY index.html /usr/share/nginx/html/
---> cc652d0fc2b7
Removing intermediate container 11f195a0e2ac
Step 3/4 : ADD Portal.tar.gz /tmp/new1/portal1/
---> b18a86545c47
Removing intermediate container 1e1849be08b4
Step 4/4 : CMD /usr/sbin/nginx
---> Running in fdac087b636b
---> 02e2795eab12
Removing intermediate container fdac087b636b
Successfully built 02e2795eab12

Or you can also mention the absolute path as shown below.

[vamshi@docker01 ~]$  # docker build -t nginx-linuxcent -f /home/vamshi/nginx/Dockerfile-nginx /home/vamshi/nginx/

The above example successfully builds a docker image. The Directory nginx/ is its build context as nginx/Dockerfile-nginx is the relative path of the input Dockerfile-nginx to docker build command.

Dockerfile and the context being different

Placing the Dockerfile-nginx inside the nginx directory and context placed one directory above the Dockerfile-nginx.

We now need to modify and carefully place the ADD/COPY commands relative to its directory in order for it to work properly, The context being one directory ahead, they should be prefixed with the directory name as we see below:

FROM nginx:1.17.2-alpine
COPY nginx/index.html /usr/share/nginx/html/
ADD nginx/Portal.tar.gz /tmp/new1/portal1/
CMD ["/usr/sbin/nginx"]

Now our docker build command takes the following syntax:

[vamshi@docker01 ~]$ docker build -t nginx-linuxcent -f nginx/Dockerfile-nginx .
Sending build context to Docker daemon 155.4 MB
Step 1/4 : FROM nginx:1.17.2-alpine
 ---> 55ceb2abad47
Step 2/4 : COPY nginx/index.html /usr/share/nginx/html/
 ---> Using cache
 ---> cc652d0fc2b7
Step 3/4 : ADD nginx/Portal.tar.gz /tmp/new1/portal1/
 ---> Using cache
 ---> b18a86545c47
Step 4/4 : CMD /usr/sbin/nginx
 ---> Using cache
 ---> 02e2795eab12
Successfully built 02e2795eab12

Here the context remains outside the directory and the Dockerfile is present inside the subdirectory, the ADD/COPY commands are prefixed with the relative path of the dubirectory

Common errors Encountered with context mismatch:

unable to prepare context: The Dockerfile must be within the build context

How to tag a Docker image with a repository name during build process?

You can name your Dockerfile anything and it doesnt matter to the build process as long as you refer it with the -f

The standard naming convention is as shown below.

# docker build -t <DOCKER_IMAGE-NAME>:<TAG> -f Dockerfile .

Syntax:

# docker build -t <REPOSITORY/REGISTRY NAME>/<DOCKER_IMAGE-NAME>:<TAG> -f Dockerfile .
# docker build --tag mydocker-registry-name/nginx-linuxcent:version1.0 -f Dockerfile .

Here the Dockerfile need not be explicitly mentioned with -f as the name of the file is Dockerfile and the context being .

# docker build --tag mydocker-registry-name/nginx-linuxcent:version1.0 - .

The Build context . at the end is important because it signifies the current context and the context cannot span backward.

The tag name is a must for best practices and helps in identifying the newly build images and tagging enables visible versioning and better identification of images.

Docker build with no-cache

Creating Docker images with the --no-cache option when you do not use cache when building the image, The default option for this is set to false and can be used explicitly to enforce no-cache..

It can be at times important when building container images which are dependent upon downloading latest libraries from the internet or practically from your on-premise code repository which contains the freshly compiled code artifacts.

Build the Docker image with no cache:

# docker build --no-cache -t mydocker-registry-name/nginx:version0.1 -f Dockerfile .

Once the docker container is successfully built, we can take a look at the newly created image as below.

# docker images
REPOSITORY                                       TAG                 IMAGE ID            CREATED             SIZE
mydocker-registry-name/nginx:                    version0.1          bcdd25553d01        3 minutes ago       298 MB

Conclusion:

The docker build context becomes the present path of the Dockerfile.The docker image build is a simple process if things are neatly organized and the context can be quiet tricky if you are managing multiple Docker builds. You have the flexibility to give the absolute of relative path to the docker build.
Its always advised to implement the relative path and use the . dot as context being in the same directory where your Dockerfile is present to run the Docker builds.

Ensure to use the no-cache option

And have a proper tagging in place to enable better version identity of your docker images.

If at all you need to build an image being in different context then always write the Dockerfile relative to the directory path of current context

What is build context Docker daemon?

The build context is the set of files located at the specified PATH or URL. Those files are sent to the Docker daemon during the build so it can use them in the filesystem of the image.

How do I put files outside Docker build context?

The best way to work around this is to specify the Dockerfile independently of the build context, using -f. For instance, this command will give the ADD command access to anything in your current directory. docker build -f docker-files/Dockerfile .

What is Docker context file?

The docker context command makes it easy to export and import contexts on different machines with the Docker client installed. You can use the docker context export command to export an existing context to a file. This file can later be imported on another machine that has the docker client installed.

What is Docker compose context?

Either a path to a directory containing a Dockerfile, or a url to a git repository. When the value supplied is a relative path, it is interpreted as relative to the location of the Compose file. This directory is also the build context that is sent to the Docker daemon.

What can you do with Docker compose Yml?

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

How do I launch a Docker daemon?

On MacOS go to the whale in the taskbar > Preferences > Daemon > Advanced. You can also start the Docker daemon manually and configure it using flags. This can be useful for troubleshooting problems. Many specific configuration options are discussed throughout the Docker documentation.

Can you have multiple Dockerfiles?

Docker Compose is the most common way to build an application that uses multiple Dockerfiles. This requires a YAML file to create the container based on a series of commands.

What is difference between ADD and copy in Dockerfile?

COPY takes in a src and destruction. It only lets you copy in a local or directory from your host (the machine-building the Docker image) into the Docker image itself. ADD lets you do that too, but it also supports 2 other sources. First, you can use a URL instead of a local file/directory.

Does Docker copy follow symlinks?

That is not possible and will not be implemented. Please have a look at the discussion on github issue #1676: A symlink on your machine is the not the same as my machine and the same Dockerfile would produce two different results.

What is the Docker daemon?

The Docker daemon ( dockerd ) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. A daemon can also communicate with other daemons to manage Docker services.