Scan Nirvana: Hub Detect for All Native Build & CI Tools

Scan Nirvana: Hub Detect for All Native Build & CI Tools

When you’re trying to secure and manage the open source code in your applications, the first step is to accurately discover all the open source in your systems. Simply put, if you don’t know which open source components you’re using, you can’t protect yourself from vulnerabilities in those components. That’s why Black Duck uses a multi-factor discovery method, leveraging the power of both package manager declarations and file signature scanning for the most accurate results.

 With Hub 4.1, we introduced a new umbrella implementation that envelops all our scanning technologies, making for an easier out-of-the-box scan experience and the most accurate open source scans. Hub Detect is an umbrella implementation for all existing native build and CI tools. 

Detect encapsulates all the package managers and the Cis into one implementation

Let's look at the different scanning technologies supported by Hub Detect. Like all our CI/Build Tool plugins, Detect continues to work as a post-build step. It starts by first looking at package manager information and then signature scanning data.

Package Management Information

Detect plugs into existing build processes to look at the flow of dependencies within the build job of your CI of choice. It captures both declared and transitive dependencies and provides a comprehensive 360° view into the project’s dependencies without having to make any changes to the native build environment. (Pretty cool, right?) The plugin has been designed to recursively check for formal dependency management files (For example: pom.xml, setup.py, gemfile.lock), which help it understand the environment the project is getting built in so it can invoke the right configuration. Detect currently supports the following package managers: Maven, Gradle, SBT, Ruby Gems, NPM, NodeJS, Cocopods, PyPi, Pear, Packagist, CPAN, Go, CRAN & Nuget. Hub Detect also supports functionality to handle major Linux package managers like apk, dpkg and rpms for major Linux operating systems via the Docker-inspector functionality.

Although Hub Detect is invoked as a post-build step, it monitors and injects itself within the build to capture the dependency information above. After Package Management inspection, the baton is handed over to the Black Duck Signature Scanner to scan the built artifact for file matches. 

Black Duck Signature Scanner

Detect now invokes the Black Duck Signature Scanner to look for file/directory matches and identify components that may have been missed the package manager inspection above. This culminates in a comprehensive, more accurate bill of materials (BoM) that can also be checked for policy violations, with the optional setting to fail builds.

Black Duck Signature Scanner identifies components missed by package manager

Axis 1.4 in the example below was found declared as a dependency and also as an exact match in multiple directories; thus providing multiple pieces of evidence to corroborate a result. 

Multiple pieces of evidence corroborate a result using Hub Detect

Hub Detect also offers a PDF risk report that gets added to the local workspace of the CI tool. This risk report can be shared with users outside of the native build environments.

Hub Detect also offers a PDF risk report

Automagically, all these processes can be configured to run from just one post-build step! 

Hub Detect processes can be configured to run from one post-build step

This is a drastic improvement from the previous approach of having multiple tools to handle multiple techniques.

Hub Detect does not add significant additional time to your build jobs. Its fast execution technique was designed to meet the needs of high velocity enterprise teams running multiple CI builds.  

In summary, Hub Detect:

  • What: A shell based post-build configuration that envelops functionality for package management and signature scanning
  • Pre-requisites: Detect should be plugged into an environment that can build the project, that is, an environment that has the native build environment (can be local/CI tool). Internet connection required.
  • Benefits: All-in-one, easy to setup, minimal one-time configuration, no install and upgrade required.

Download the implementation, which is hosted on GitHub as always

The documentation is available on our Black Duck Public Wiki.

 

Learn about Hub Detect, a new open source discovery tool

0 Comments
Sorry we missed you! We close comments for older posts, but we still want to hear from you. Tweet @black_duck_sw to continue the discussion.
0 Comments

MORE BY THIS AUTHOR

Hub Detect: Comprehensive Open Source Scanning

| Aug 22, 2017

As a product manager at Black Duck, I drive our priorities with integrations. This means, of course, that I listen to our customers a lot — what integrations are working for them, what’s missing, and what new features would help them. Based on customer feedback, our team has been improving our

| MORE >

Why Binary Risk Management is Similar to Managing Your Wardrobe

| Jan 3, 2017

As we bid adieu to 2016 and welcome 2017, I'm thinking about the shift from the Continuous Integration (CI)/Build step to the binary repository space as a new control point within the software development cycle. Such dramatic changes aren't new in the software world, but what suprises me most

| MORE >