Linux Cent

In this section we will see some important housekeeping tasks while maintaining gitlab server.

We will list out some of the important configuration services present in gitlab and best housekeeping tips.
The important configuration file for gitlab is: /opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml
Once the changes are made to this file it should be followed up running the reconfigure to apply the changes.
The chef server reads the config files and then updates the system wide configuration resources.

Reconfiguring the gitlab

[vamshi@gitlab ~]$ sudo gitlab-ctl reconfigure

This command reloads the configuration from the configuration files and also migrated if any changes regiuired to the Postgresql database.

View all the current logs run the gitlab-ctl tail command.

[vamshi@node01 ~]$ sudo gitlab-ctl tail
==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==

==> /var/log/gitlab/gitlab-rails/sidekiq.log <==
2020-04-o2_13:30:24.29475 2020-04-02T13:30:24.293Z 8872 TID-or0k2ve5w PagesDomainVerificationCronWorker JID-e46ed1d452563ca9b7d4adca INFO: done: 0.357 sec
-- OUTPUT TRUNCATED --

This in one of the first command you need when troubleshooting your gitlab server..
This prints the logs from various components present in gitlab and you as an administrator will be able to effectively identify the issues.
To start the interactive ruby console and debug mostly used for Database interaction operations, it takes -e as argument and takes you to the production environment. -s takes you to the sandbox environment.

gitlab-rails console <-e|-s> [environment]

For example we can get the complete gitlab information using the following gitlab:env:info:

[vamshi@gitlab ~]$ sudo gitlab-rails gitlab:env:info
[sudo] password for vamshi:

System information
System: 
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.5.3p105
Gem Version: 2.7.6
Bundler Version:1.17.3
Rake Version: 12.3.2
Redis Version: 3.2.12
Git Version: 2.18.1
GitLab information
Version: 11.10.4-ee
Revision: 88a3c791734

GitLab Shell
Version: 9.0.0

-- OUTPUT TRUNCATED --

Some more important commands for gitlab Houseeping:

[vamshi@gitlab ~]$ sudo gitlab-rails gitlab:check
[sudo] password for vamshi:
Checking GitLab subtasks ...

Checking GitLab Shell ...

-- OUTPUT TRUNCATED --

How to restart the gitlab serer from command line ?

[vamshi@gitlab ~]$ sudo gitlab-rails restart

How to clear the gitlab logs by truncating them to Zero bytes?

[vamshi@gitlab ~]$ sudo gitlab-rails log:clear

How To print the high level stats of gitlab server?

[vamshi@gitlab ~]$ sudo gitlab-rails stats

What is GitLab housekeeping?

GitLab supports and automates housekeeping tasks within your current repository such as: Compressing Git objects. Removing unreachable objects.

What is GitLab housekeeping?

How do I clean up GitLab repository?

• To clean up a repository:
• Go to the project for the repository.
• Navigate to Settings > Repository.
• Upload a list of objects. For example, a commit-map file created by git filter-repo which is located in the filter-
• repo directory. …
• Click Start cleanup.

What is Git repack?

DESCRIPTION. This command is used to combine all objects that do not currently reside in a “pack”, into a pack. It can also be used to re-organize existing packs into a single, more efficient pack.

What does Git prune do?

The git prune command is an internal housekeeping utility that cleans up unreachable or “orphaned” Git objects. Unreachable objects are those that are inaccessible by any refs. Any commit that cannot be accessed through a branch or tag is considered unreachable.

How do I know if GitLab is running?

When a user goes to your GitLab URL, they will be shown an arbitrary Deploy in progress page. To remove the page, you simply run sudo gitlab-ctl deploy-page down . You can also check the status of the deploy page with sudo gitlab-ctl deploy-page status.

How do I maintain a Git repository?

Manually reviewing large files in your repository

• Download the script to your local system.
• Put it in a well known location accessible to your Git repository.
• Make the script an executable: …
• Clone the repository to your local system.
• Change directory to your repository root.
• Run the Git garbage collector manually.

How do I purge Gitlab?

uninstall Gitlab:

1. Stop the gitlab service. Command : sudo gitlab-ctl stop.
2. Start commonde of service facility. command : sudo gitlab-ctl uninstall.
3. Delete the package of gitlab. command : Locat gitlab.
4. restart machine.

How do I empty a Gitlab project?

Delete a project

1. On the top bar, select Menu > Projects and find your project.
2. On the left sidebar, select Settings > General.
3. Expand Advanced.
4. In the “Delete project” section, select Delete project.
5. Confirm the action when asked to.

Are git repos compressed?

Thirdly, Git stores its data as compressed objects, whereas SVN stores them as uncompressed copies. Go into any . svn/text-base directory, and you’ll find uncompressed copies of the (base) files.

Wouldn’t it be great when we are using kubernetes and just tab to get a list of possible subcommands?

We have the extended facility to automatically tab and get the kubectl command suggestions, This feature is inbuilt into the kubectl and kubeadmin, All we need to just extract the script and enable it to the .bash_profile and source it. Lets do it as show below

[vamshi@workstation ~]$ kubectl completion bash > ~/.kube/k8s_bash_completion.sh

[vamshi@workstation ~]$ echo -e "\n#kubectl shell completion\nsource '$HOME/.kube/k8s_bash_completion.sh'\n" >> $HOME/.bash_profile

[vamshi@workstation ~]$ source $HOME/.bash_profile

Once you have successfully sourced the .bash_profie file.. type kubectl commands and keep tabbing all the way to get the suitable suggestions.

What is kubectl bash completion?

The kubectl completion script for Bash can be generated with the command kubectl completion bash . … However, the completion script depends on bash-completion, which means that you have to install this software first (you can test if you have bash-completion already installed by running type _init_completion ).

How do I create a completion in bash?

How to add bash auto completion in Ubuntu Linux

1. Open the terminal application.
2. Refresh package database on Ubuntu by running: sudo apt update.
3. Install bash-completion package on Ubuntu by running: sudo apt install bash-completion.
4. Log out and log in to verify that bash auto completion in Ubuntu Linux working properly.

What is kubectl auto completion?

Shell command-line completion allows you to quickly build your command without having to type every character. … The Kubernetes documentation provides great instructions on how to set it up for your development environment.

How do you use kubectl autocomplete?

To use shell autocompletion with kubectl simply press tab while writing out a command. For example we can type g and then press tab to autocomplete to get .

What is difference between Docker and Kubernetes?

A fundamental difference between Kubernetes and Docker is that Kubernetes is meant to run across a cluster while Docker runs on a single node. … Kubernetes pods—scheduling units that can contain one or more containers in the Kubernetes ecosystem—are distributed among nodes to provide high availability.

Is zsh better than bash?

It has many features like Bash but some features of Zsh make it better and improved than Bash, such as spelling correction, cd automation, better theme, and plugin support, etc. Linux users don’t need to install the Bash shell because it is installed by default with Linux distribution.

How does bash completion work?

The programmable completion feature in Bash permits typing a partial command, then pressing the [Tab] key to auto-complete the command sequence. [1] If multiple completions are possible, then [Tab] lists them all. Let’s see how it works. Tab completion also works for variables and path names.

How do I know if bash completion is installed?

If the autocomplete results contain directories only (no files), then Bash Completion is installed. If the autocomplete results include files, then Bash Completion is not installed.

What is complete command?

complete is a bash command used to perform the auto-complete action when the user hit the TAB key in a terminal. Calling just complete will list all the functions registered for auto-completion of commands or services options.

What is the difference between Minikube and Kubernetes?

Kubernetes is an open source orchestration system for Docker containers. … On the other hand, minikube is detailed as “Local Kubernetes engine”. It implements a local Kubernetes cluster on macOS, Linux, and Windows.

How do I know if kubectl is installed?

Install the kubectl Command Line

1. Check that kubectl is correctly installed and configured by running the kubectl cluster-info command: kubectl cluster-info. …
2. You can also verify the cluster by checking the nodes. …
3. To get complete information on each node, run the following: kubectl describe node.

The docker networking comprises of a overlay network and enabled communication with the outside resources using it.

There are following main types of built in connectivity networking drivers namely the bridged, host, macvlan, overlay network and the null driver with no network.

The docker container networking Model CNM architecture manages the networking for Docker container.

IPAM which stands for the IP address management works in a single docker node, and aids in Enabling the network connectivity among the doccker containers. Its primary responsibility is to allocate the IP address space for the subnets, allocation of the IP addresses to the endpoints and the network etc,.

The networking in docker is essentially an isolated sandbox environment, The isolation of the networking resources is possible by the namespaces

The overlay network enables the communication enabled the network spanning across many docker nodes on an environment like the Docker swarm network, The same networking logic is evident in a bridge networking but it is ony limited to a single docker host unlike the overlay network.

Here’s the output snippet from the docker info command; Listing the available network drivers.

# docker info
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay

The container networking enables connectivity inbetween the docker containers and also the host machine to docker container and vice-versa.

Listing the default networks in docker:

[vamshi@node01 nginx]$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
68b2ffd36e8f        bridge              bridge              local
c1aca4c87a2b        host                host                local
d5e48683def8        none                null                local

When a container is created by default it connects to the bridge network unless an extra arguments are specified.

When you install the docker by default a docker0 virtual interface is created which behaves as a bridge between the docker containers and the host system.

[vamshi@node01 nginx]$ brctl show
bridge name	bridge id		STP enabled	interfaces
docker0		8000.0242654b42ef	no

For brctl command we need to Install the bridge-util package.

We now examine the docker networks with docker network inspect.

Inspecting the various docker networks:

Inspecting the bridge network:

The bridge networking enables the network connectivity over the dockers in a single docker server host.

[vamshi@node01 nginx]$ docker network inspect bridge
[
    {
        "Name": "bridge",
        "Id": "68b2ffd36e8fcdc0c3b170dfdbdbc93bb58351d1b2c011abc80709928463f809",
        "Created": "2020-05-23T10:28:27.206979057Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

This bridge is shown with the ip addr command as follows:

# ip addr show docker0 
   docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:65:4b:42:ef brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:65ff:fe4b:42ef/64 scope link 
       valid_lft forever preferred_lft forever

Inspecting the host network.

[vamshi@node01 ~]$ docker network inspect host 
[
    {
        "Name": "host",
        "Id": "c1aca4c87a2b3e7db4661e0cdedc97245cd5dfdc8aa2c9e6fa4ff1d5ecf9f3c1",
        "Created": "2019-05-16T18:46:19.485377974Z",
        "Scope": "local",
        "Driver": "host",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": []
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

Inspecting the null driver network

[vamshi@node01 ~]$ docker network inspect none
[
    {
        "Name": "none",
        "Id": "d5e48683def80b2e739b3be95e58fb11abc580ce29a33ba0df679a7a3972f532",
        "Created": "2019-05-16T18:46:19.477155061Z",
        "Scope": "local",
        "Driver": "null",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": []
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

The following are special networking architectures to span across multihost docker servers enabling network connectivity among the docker containers.

1.Overlay network

2 macvlan network.

Let us inspect the multi host networking:

The core components of the docker interhost network consists of

Inspecting the overlay network:

[vamshi@docker-master ~]$ docker network inspect overlay-linuxcent 
[
    {
        "Name": "overlay-linuxcent",
        "Id": "qz5ucx9hthyva53cydei0y8yv",
        "Created": "2020-05-25T13:22:35.087032198Z",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.255.0.0/16",
                    "Gateway": "10.255.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {
            "ingress-sbox": {
                "Name": "overlay-linuxcent-endpoint",
                "EndpointID": "165beb97b22c2857e3637119016ef88e462a05d3b3251c4f66aa0fc9176cfe67",
                "MacAddress": "02:42:0a:ff:00:03",
                "IPv4Address": "10.255.0.3/16",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4096"
        },
        "Labels": {},
        "Peers": [
            {
                "Name": "node01.linuxcent.com-95ad856b6f56",
                "IP": "10.100.0.20"
            }
        ]
    }
]

The endpoint is the Virtual IP addressing that routes the traffic to the respective containers running on individual docker nodes.

Inspecting the macvlan network:

vamshi@docker-master ~]$ docker network inspect macvlan-linuxcent 
[
    {
        "Name": "macvlan-linuxcent",
        "Id": "99c6a20bd4029ce5a37139c6e6792ec4f8a075c94b5f3e71efc32d92d41f3f89",
        "Created": "2020-05-25T14:20:00.655299409Z",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.20.0.0/16",
                    "Gateway": "172.20.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

What is Docker networking?

Docker networking is primarily used to establish communication between Docker containers and the outside world via the host machine where the Docker daemon is running. … You can run hundreds of containers on a single-node Docker host, so it’s required that the host can support networking at this scale.

How does networking work with Docker?

Docker secures the network by managing rules that block connectivity between different Docker networks. Behind the scenes, the Docker Engine creates the necessary Linux bridges, internal interfaces, iptables rules, and host routes to make this connectivity possible.

How do I connect to a Docker network?

Connect a container to a network when it starts

You can also use the docker run –network= option to start a container and immediately connect it to a network.

Can a Docker container have multiple networks?

You can create multiple networks with Docker and add containers to one or more networks. Containers can communicate within networks but not across networks. A container with attachments to multiple networks can connect with all of the containers on all of those networks.

Why is docker network needed?

Some of the major benefits of using Docker Networking are: They share a single operating system and maintain containers in an isolated environment. It requires fewer OS instances to run the workload. It helps in the fast delivery of software.

What are the types of docker networks?

There are three common Docker network types – bridge networks, used within a single host, overlay networks, for multi-host communication, and macvlan networks which are used to connect Docker containers directly to host network interfaces.

How do I ping a Docker container?

Ping the IP address of the container from the shell prompt of your Docker host by running ping -c5 . Remember to use the IP of the container in your environment. The replies above show that the Docker host can ping the container over the bridge network.

What does Docker network create do?

When you install Docker Engine it creates a bridge network automatically. This network corresponds to the docker0 bridge that Engine has traditionally relied on. When you launch a new container with docker run it automatically connects to this bridge network.

Does Docker offer support for IPv6?

Before you can use IPv6 in Docker containers or swarm services, you need to enable IPv6 support in the Docker daemon. Afterward, you can choose to use either IPv4 or IPv6 (or both) with any container, service, or network. Note: IPv6 networking is only supported on Docker daemons running on Linux hosts.

How do I run a docker on a local network?

This article discusses four ways to make a Docker container appear on a local network.
…

• Using NAT
• It will create a veth interface pair.
• Connect one end to the docker0 bridge.
• Place the other inside the container namespace as eth0 .
• Assign an ip address from the network used by the docker0 bridge.

Can a docker container have its own ip address?

The answer is: you can configure it. Create the container with –network host and it will use the host ip.

The ENTRYPOINT or CMD usually determine the single and main startup process inside the running container, The ENTRYPOINT and the CMD are the final Docker DSL invocation statements in a Dockerfile.A Dockerfile can not be complete without either one of them.

Both the ENTRYPOINT and CMD will execute a /bin/sh -c , but have significant differences.

ENTRYPOINT usually is straight forward with the and is often the statement that enables the  docker image as a runnable container. The ENTRYPOINT statement doesnot let the run time arguments get overwritten during the container runtime, instead the commands are passed on as arguments to the ENTRYPOINT, for Example when the container is started as docker run my-image -d, then the resultant -d argument is passed on as an argument to the ENTRYPOINT.
It means that only last ENTRYPOINT statement is valid in the Dockerfile

The ENTRYPOINT takes CMD as its arguments.

ENTRYPOINT command arg1 arg2

OR

ENTRYPOINT command
CMD ["arg1", "arg2"]

ENTRYPOINT ["java","-jar","app.jar"]
CMD ["-Dserver.port=8080"]

CMD on the other hand is subversive to the ENTRYPOINT.
CMD start statement can be overwritten with container arguments during the container runtime. The CMD does not execute anything at the build time.

CMD ["command","arg1", "arg2"]
CMD ["java","-Djava.security.egd=file:/dev/urandom","-jar","-Dserver.port=8080","app.jar"]

The major practical use with CMD is that the you will be able to overwrite the docker runtime command line arguments but you cannot override with you have ENTRYPOINT.

The CMD will be very beneficial in cases which require sending arguments to containers in runtime which in development and QA stages and is great for debugging environments.

The [] are not mandatory for ENTRYPOINT and CMD instruction statements inside Dockerfile but can be given for better readability when you have lot of arguments

What is the difference between CMD and run in Dockerfile?

RUN is an image build step, the state of the container after a RUN command will be committed to the container image. A Dockerfile can have many RUN steps that layer on top of one another to build the image. CMD is the command the container executes by default when you launch the built image.

What is the use of CMD in docker?

The CMD command​ specifies the instruction that is to be executed when a Docker container starts. This CMD command is not really necessary for the container to work, as the echo command can be called in a RUN statement as well. The main purpose of the CMD command is to launch the software required in a container.

What is docker ENTRYPOINT script?

entrypoint scripts are plugins that can be written in any language. inherited images can add to existing entrypoint scripts by simply adding to the /docker-entrypoint.

What is the purpose of ENTRYPOINT in Dockerfile?

In Dockerfiles, an ENTRYPOINT instruction is used to set executables that will always run when the container is initiated. Unlike CMD commands, ENTRYPOINT commands cannot be ignored or overridden—even when the container runs with command line arguments stated.

Can Dockerfile have multiple ENTRYPOINT?

According to the documentation however, there must be only one ENTRYPOINT in a Dockerfile.

What is Docker ENTRYPOINT Initdb?

3. 3. /docker-entrypoint-initdb. d/init. sql is executed the moment your database container starts running, while your entrypoint.sh is executed the moment your web container starts running.

Does docker start run ENTRYPOINT?

So yes, the ‘ CMD ‘ commands are executed after a ‘ docker start ‘.

Can Dockerfile have multiple CMD?

You are right, the second Dockerfile will overwrite the CMD command of the first one. Docker will always run a single command, not more. So at the end of your Dockerfile, you can specify one command to run.

What is in docker Run command?

The docker run command creates a container from a given image and starts the container using a given command. It is one of the first commands you should become familiar with when starting to work with Docker.

What is difference between CMD and entrypoint?

In a nutshell: CMD sets default command and/or parameters, which can be overwritten from command line when docker container runs. ENTRYPOINT command and parameters will not be overwritten from command line. Instead, all command line arguments will be added after ENTRYPOINT parameters.

What is the default entrypoint for docker?

/bin/sh -c
Docker defaults the entrypoint to /bin/sh -c . This means you’ll end up in a shell session when you start the container.

Login to the Jenkins and navigate to the Manage Jenkins -> Manage Users. Now you will be able to see the current users on the Jenkins.

You should be able to land on the URL similar to this: http://jenkins.linuxcent.com:8080/securityRealm/

On the Left hand side menu Click on Create User and follow the process. .

With this process you will be able to create a new user using the Jenkins’s own user database.

You can also ensure the Jenkins system user details from the jenkins server by navigating to the path $JENKINS_HOME and from the file users/users.xml

[root@node02 jenkins]# grep -w builduser * users/users.xml 2>/dev/null 
users/users.xml: <string>builduser</string>

How do I give user access to Jenkins?

Essentially you do this:

• Go to Jenkins -> Manage Jenkins -> Configure Global Security.
• Check “Enable security”.
• Set “Jenkins own user database” as security realm.
• Check “Allow users to sign up”
• Choose “Matrix based security”
• Check “Overall read” on Anonymous.
• Add your admin account in the matrix, check every box.

How do I make the first admin in Jenkins?

Jenkins – Security

• Step 1 − Click on Manage Jenkins and choose the ‘Configure Global Security’ option.
• Step 2 − Click on Enable Security option. …
• Step 3 − You will be prompted to add your first user. …
• Step 4 − It’s now time to setup your users in the system.

How get manage users in Jenkins?

How do I change users in Jenkins?

How do I create multiple users in Jenkins?

What user runs Jenkins?