Migrating to Docker on Black Duck Hub

Migrating to Docker on Black Duck Hub

Before Black Duck began leveraging Docker, customers utilized the App Manager Install Method to deploy the Hub. The Hub now deploys as a set of containers, so customers need to install Docker to take advantage of updates to the application. By the end of this guide, you'll have a basic understanding of how to migrate the Hub to a containerized environment, as well as the benefits of using containers. 

Docker is a packaging technology that allows you to bundle applications with its software dependencies—eliminating the need to configure infrastructure before deployment. It is built once and can run anywhere as a single unit. These units on their own are called images, but when they are being run as applications we call them containers.  

The Docker architecture brings a number of significant improvements to the Hub:

  • Simplifies installation and updates
  • Takes up less space than a comparable Virtual Machine (VM)
  • Scales to customer environment
  • Improves performance

The Black Duck Hub is deployed as a set of Docker containers, which together comprise the application. Each container fulfills a different role. For example, the Web App container processes UI requests while the Solr container acts as the enterprise search platform component. A detailed description of each container’s role is available in the Black Duck Hub Docker Install Guide on the Black Duck Customer Success Community.

Please ensure that you meet all of the hardware and software requirements before getting started:

Hardware (min. specs)

Software

Preferred Operating System

64-bit 5 core processor

Docker 17.03.x or Docker 17.06.x (CE or EE)

CentOS 7.3

20 GB of RAM

Orchestration tool such as Docker Swarm**, Docker Compose, Docker Run, or Kubernetes.

Ubuntu 16.04.x

250 GB of free space for the database and other Hub containers

For additional information on supported orchestration techniques, consult the Docker Installation Guide.

Oracle Enterprise Linux 7.3

Free space for database backups

 

Red Hat Enterprise Linux Server 7.3

 

 

SUSE Linux Enterprise Server 12.x

 

 

Linux Operating Systems that support Docker 17.03.x

Setting up the Docker Repository on a New Host Machine 

Note that the following steps depend on the operating system (OS). Also, while other methods are supported, this example uses the Docker Swarm tool to orchestrate the Hub container deployment and installation on a CentOS / Docker CE platform. To begin migrating to a containerized environment, we begin by installing Docker, which is comprised of five steps

Step 1: Install required packages and set up the stable repository. 

  • To install required utility packages, issue command: yum install -y yum-utils device-mapper-persistent-data lvm2 
  • To set up the stable repository, issue command: yum-config-manager\
    --add-repo \
     
    https://download.docker.com/linux/centos/docker-ce.repo
      

Step 2: Update the yum package index and install the latest version of Docker available. 

  • Issue command: yum makecache fast  
  • To install Docker, issue command: yum install docker-ce  

Step 3:  Verify installation with hello-world image, which runs a test image in a container. When it runs, Docker prints an informational message and then exits.  

  • To start Docker, issue command: systemctl start docker  
  • To test an image in a container, issue command: Docker run hello-world  

Post installation (optional) 

Step 4: Manage Docker as a non-root user so you don’t have to issue a sudo command every time you want to run a Docker command. Start by creating a group called Docker. 

  • To add a group, issue command: groupadd docker 
  • To display groups to ensure your group was created, issue command: getent group 
  • Then add yourself to the group. To add yourself to the group, issue command: usermod – aG docker $USER 
  • Test that this was successful by running the hello-world image without writing “sudo” before it. Issue command: docker run hello-world 

Step 5: Configure Docker to automatically start when your system reboots. 

  • To have Docker start automatically, issue command: systemctl enable docker 

Here are links to more info on installing Docker and Docker Swarm. Now that the Docker architecture has been set up, let's install the Hub.   

Begin Migrating to Docker 

We now begin moving your previous data to a Dockerized Hub. The Hub migration can be broken down into four major steps: 1) preserve configuration and open source data from the existing version of the Hub before uninstalling it, 2) uninstall the old version of the Hub, 3) obtain configuration and orchestration files for Dockerized hub, and 4) restore your Hub data. 

Step 1: If you want to retain your existing data on your open source components, you must first back up your database. Begin by logging onto the Hub server as a Black Duck user. After you’re logged on, dump your data to a compressed file with three commands: 

  • Issue command: export PATH=SPATH:/opt/blackduck/hub/postgresql/bin 
  • Issue command: export PGPORT=55436 
  • Issue command: pg_dump -Fc -f /tmp/bds_hub.dump bds_hub 

Be sure to save the bds_hub.dump file onto another system or offline!  

Step 2: Now that the database is backed up, we can uninstall the old Hub server. Start by navigating to the app manager bin directory and issue a command to stop all bin processes: 

  • Issue command: cd /opt/blackduck/hub/appmgr/bin 
  • To stop all processes, issue command: ./StopAllProcesses.sh 

Now we are ready to uninstall the application: 

  • Issue command: ./uninstall.sh 

Step 3: Download the orchestration and configuration files. You need these files to install the Hub and launch the Docker images. 

  • Go to Black Duck’s space on Github and select “blackducksoftware/hub”
  • Download the latest version of the Hub by scrolling down and copying the first link under “Hub Docker Orchestration Files/Documentation” 
    • Issue command: wget *paste link above*
      Example:
       wget https://github.com/blackducksoftware/hub/archive/v4.4.0.tar.gz

Unpack .tar file and configure Web server and, if needed, Proxy Server settings. 

  • Unpack the .tar file on a server 
    • Issue command: tar xvf -hub docker-<current version of hub (ex. 4.0.0)>.tar  
    • Within bin directory, there are two environment files (hub-proxy.env AND hub-webserver.env), which we use to configure the Web server and, if needed, proxy server 

Now that we have the environment files, let’s configure the Web server and, if needed, proxy server settings. 

  • Edit the Web server environment file 
    • Issue command: vi hub-webserver.env 
    • The environment variable has the service name set as default value. Change this to the host name so that the certificate host name matches. 
  • Configure the proxy settings using the proxy environment file. This must be configured if you require proxy for eternal internet access. Specifics for using an external PostgreSQL instance and configuring proxy environment file are available in the Black Duck Hub Docker Install Guide. 

Step 4: Now that the old version of the Hub is uninstalled and you have the configuration and orchestration files, we can run the migration script to restore your open source data. This transfers the data from the existing database dump file. Begin by downloading and starting three of the containers and volumes needed to migrate the database: 

  • Issue command: docker stack deploy -c docker-compose.dbmigrate.yml hub 

Ignore the error messages if you have any, as they do not affect the data restoration. Now we are ready to run the migration script to restore your Hub data: 

  • Issue command: ./bin/hub_db_migrate.sh <path to dumpfile (ex. /tmp/bds hub.dump)> 

Install Hub on Containers 

Now we are ready to install the Hub. Keep in mind that the following instructions are for those using the Docker Swarm orchestration tool. If you do not use Docker Swarm, please reference the Black Duck Hub Docker Install Guide. 

Step 1: Install the Dockerized Hub with the PostgreSQL database instance. Begin by initializing Docker Swarm, creating a single node swarm. 

  • Issue command: docker swarm init 

Now, install the Hub with the PostSQL database instance; this creates all the containers that comprise the Hub. 

  • Issue command: docker stack deploy -c docker-compose.yml hub 

Docker implementation of the Hub enables scalability by allowing admins to choose the number of active job runner containers. 

  • To do this through the command shell, issue command: docker service scale hub_jobrunner=2   

Verify Installation Completion 

Step 1: Confirm Installation was successful by running the docker ps commands 

  • Ensure all container statuses are listed as “healthy”
  • To do this, issue command: docker ps 
  • The web application for the Hub is exposed on port 443 to the docker host. You can access the Hub by entering the following: https://fillinyourhostnameorIPaddresshere
  • Log into the web interface using the default sysadmin account. The username is sysadmin; the password is blackduck. 
  • Change the password so that your web server is secure. To do this, click on the sysadmin in the upper right-hand corner, and select “My profile.”

Black Duck Academy

0 Comments
Sorry we missed you! We close comments for older posts, but we still want to hear from you. Tweet @black_duck_sw to continue the discussion.
0 Comments

MORE BY THIS AUTHOR

The Best Way to Secure Applications in 2018? Learn from 2017

| Feb 7, 2018

2017—a Turbulent Year in Application Security From breaches making headlines to exciting new technologies, 2017 was abuzz with conversation around securing applications and the implications of access to personal data. We saw what can happen when sensitive data is not properly secured, providing a

| MORE >

An Introduction to Installing the Black Duck Hub

| Jan 17, 2018

Black Duck Academy hosts a series of videos that help our customers deploy, manage, and use our products. To help our customers see value from the Black Duck Hub immediately after their purchase, this post supplements our video on installing the Hub. Think of this post as a quick way to get you

| MORE >