Agent Usage

SourceClear Software Composition Analysis

Requirements

Scanning a repository which utilizes Java and one of its build/package managers requires the ability to build the code within the environment you intend to scan the project in. This includes the following requirements based on the various build/package managers:

  • When scanning with the SourceClear Command Line Interface, we require Java 1.8 or higher
  • macOS or 64-bit Linux, or Windows 7+ and PowerShell 3+
  • Ensure outbound connections to api.sourceclear.io
  • Git 1.9.3 or higher
  • If your code repository is not a Git based repository, you will need to setup the appropriate environment variables as described in the non-git scanning document.

Commands

The following are the primary commands which can be used with the SourceClear agent, along with arguments which can be added in order to alter each command’s functionality.

activate

Description: Activates your SourceClear agent with an initialization token and installs an agent.yml file to a directory of your choosing.

Example: $ srcclr activate

Multiple agents can be activated on a single machine. In the instance where you are activating a second agent, you will be asked to enter a profile name. This is a way for the user to identify what agent key they are scanning with. A suggestion for a profile name is the Workspace name that the results will appear in.

scan

Description: Performs a scan of a supported code repository, outputting vulnerability information for libraries in usage and optionally uploads information to SourceClear.

  • --ws= - Scan using an organization-level agent into a specific workspace. For more information see here
  • --json - Output the scan results in JSON format. See the SourceClear agent JSON Schema page for details on output.
  • --allow-dirty - Allows scans on repositories which contain differences that are not committed to the project.
  • --install-first - Run the install target of your project before analyzing
  • --unmatched - Show coordinates for all unmatched libraries (typically proprietary and unreleased versions of libraries)
  • --skip-compile - JAVA ONLY. Do NOT compile the project before analyzing
  • --loud - Shows raw build output.
  • --no-upload - Prevents scan results from being uploaded to the SourceClear Web interface
  • --do-not-collect - Specify which information to not collect from git commits logs. Currently only supports ‘emails’. Defaults to ‘emails’ if this option is specified without any arguments. (default: emails)
  • --quick - Performs a quick scan which doesn’t require build tools. (Lockfile required)
  • --url - Specify a remote Git URL to scan
  • --ref - Use –url and this to specify a branch or tag name to scan the specified branch or tag.

    • Example:
      $ srcclr scan --url https://github.com/srcclr/test-java-maven --ref
                1.0.0
  • --scan-collectors - Specify which build types to scan, rather than scanning all discovered build systems by default. Configuration options are: pip, maven, gradle, npm, gem, ant, bower, jar, yarn, cocoapods, sbt, "go get", godep, govendor, glide, ivy, trash, composer, MSBuildDotNet.

    • Example:
      $ srcclr scan /my/project --scan-collectors “maven,
              gradle”
  • --skip-collectors - Specify which build types to skip during the scan, rather than scanning all discovered build systems by default. Configuration options are: pip, maven, gradle, npm, gem, ant, bower, jar, yarn, cocoapods, sbt, "go get", godep, govendor, glide, ivy, trash, composer, MSBuildDotNet.

    • Example: $ srcclr scan /my/project --skip-collectors "ant, pip"
  • --skip-vms - Specify whether to skip vulnerable methods analysis.

    • Example: $ srcclr scan /my/project --skip-vms

debug

Description: Enables the debug mode on the SourceClear agent. This outputs additional information for identifying issues encountered when scanning

Example: $ srcclr scan --debug

test

Description: Tests the environment in which the agent is installed and verifies that everything required to complete a scan for the specified package manager is present.

Options:

  • --ant - Perform a ANT based test clone and scan operation
  • --bower - Perform a BOWER based test clone and scan operation
  • --cocoapods - Perform a COCOAPODS based test clone and scan operation
  • --composer - Perform a COMPOSER based test clone and scan operation
  • --gem - Perform a GEM based test clone and scan operation
  • --glide - Perform a GLIDE based test clone and scan operation
  • --go - Perform a GO GET based test clone and scan operation
  • --godep - Perform a GODEP based test clone and scan operation
  • --govendor - Perform a GOVENDOR based test clone and scan operation
  • --gradle - Perform a GRADLE based test clone and scan operation
  • --ivy - Perform a IVY based test clone and scan operation
  • --maven - Perform a MAVEN based test clone and scan operation
  • --npm - Perform a NPM based test clone and scan operation
  • --pip - Perform a PIP based test clone and scan operation
  • --sbt - Perform a SBT based test clone and scan operation
  • --trash - Perform a TRASH based test clone and scan operation
  • --yarn - Perform a YARN based test clone and scan operation
  • --nuget - Perform a NUGET based test clone and scan operation

Example: $ srcclr test --npm

config

Description: Allows the user to create, install, and list values from their agent.yml file.

Options:

  • --get - Display a single config value
  • --list - Display the config in its entirety

Example: $ srcclr config --install user ~/agent.yml

lookup

Description: Look up the release and vulnerability information found in the SourceClear Vulnerability Database for a single library with the agent

  • --json (mandatory) - Output the lookup results in JSON format. Currently this is the only available output format of the “lookup” verb. See the SourceClear agent JSON Schema page for details on output.
  • --coord1 - The primary library name in all cases except for type=maven, and in that case the first coordinate identifies the groupId in Maven nomenclature. The case sensitivity of the coord1 will depend on whether the underlying registry distinguishes packages by case. PyPI, for example, is not case sensitive but Maven is. In the case of Go libraries, coord1 should be the domain and top-level package name e.g. github.com/docker/docker
  • --coord2 (optional except for maven) - When the type=maven, this specifies the artifactId of the library to lookup.
  • --platform - If needed, one can specify the target platform of a library that one would find in the registry, such as freebsd or py3, depending on the kind of package and the kind of registry.
  • --type - The type of registry that contains the library one is going to specify using the “coord” options. Acceptable values are displayed in lookup --help and are as follows:

    • gem to identify a Ruby Gem dependency as one would find on rubygems.org
    • maven to identify a Maven dependency as one would find on repo.maven.apache.org
    • npm to identify a JavaScript dependency as one would find on npmjs.com
    • pypi to identify a Python dependency as one would find on pypi.python.org
    • cocoapods to identify a CocoaPods dependency as one would find on cocoapods.org
    • go to identify a Go dependency as one would find on the common Go repositories such as github.com, bitbucket.org, gopkg.in, golang.org
    • packagist to identify a PHP dependency as one would find on packgist.org
    • nuget to identify a .NET dependency as one would find on nuget.org
  • --version - The version number, as specified in the registry identified by --type, of the library to lookup. For Go, version has to be a released tag in the library’s repository. Looking up by commit hash is currently not supported.

Example:

$ srcclr lookup --type=maven --coord1=org.springframework --coord2=spring-core \
                --version=4.1.0.RELEASE --json

Global Options

Option: --help

Description: Displays a command summary

Example: $ srcclr --help

Option: --profile=

Description: If multiple agents are activated on one machine, you will be prompted for a profile name. This is an identifier you have assigned to this agent. Using the profile command you can choose which profile is used. Must be placed before the scan command.

Example: $ srcclr --profile=Engineering scan

Option: --config=CONFIG-FILENAME

Description: Read in an additional config file that overrides the default one, described in CONFIG-FILENAME

Example:$ srcclr --config=$HOME/.srcclr/agent-test.yml scan ~/Projects/test-java-maven

Option: --insecure

Description:Suppress the file permission check for every config file provided, which includes the default one.

Example: $ srcclr --insecure scan --url https://github.com/srcclr/test-java-maven

Option: --version

Description: Print the current version and exit.

Example: $ srcclr --version

Agent Configuration Values

The following key names are configured in the agent.yml file which is created upon activation of the SourceClear agent.

  • agentAuthorization - (Required) Bearer token used to authenticate SourceClear API calls

  • customMavenOptions - Specifies custom Maven commands when scanning repositories using the Maven build system

  • maxErrorLines - The maximum number of lines of build error messages that the CLI should transmit to the SourceClear API when reporting a build failure.

  • proxyType - One of the strings HTTP or SOCKS which will indicate the kind of proxy the Agent should use when connecting to proxy- Host and proxyPort.

  • proxyHost - The IP or hostname that the Agent should use for proxy communication.

  • proxyPort - The integer value of the port on the IP or hostname specified in proxyHost.

  • proxyUsername - The username that the Agent will use to authenticate against the proxy specified by proxyHost and proxyPort.

  • proxyPassword - The password that the Agent will use to authenticate against the proxy specified by proxyHost and proxyPort.

  • skipVersionCheck - A boolean which determines whether or not the agent will attempt to check for the latest version.

  • scmType - One of the following supported source code management systems: GITHUB, GITHUBENTERPRISE, GIT, STASH

  • scmToken - A GitHub or GitHub Enterprise API token which may also be used to clone private repos

  • scmUrl - The endpoint used to communicate with your source code management system’s API.

  • scmUsername - Used to authenticate the scmUrl above.

  • scmPassword - Password associated with the scmUsername

Environment Variables

The SourceClear agent can read from these environment variables instead of variables set in the agent.yml file.

  • SRCCLR_API_TOKEN - Provides an alternate means of supplying the agentAuthorizationtoken required to use the SourceClear API. If present, this supersedes the configuration file.

  • JAVA_OPTS - Permits altering the behavior (or system properties) of the underlying Java® runtime system that is used by the srcclr command.

  • SRCCLR_CONFIG - Provides an explicit means of specifying the SourceClear configuration file location. If this variable is populated, the program will use that path in addition to the system and user locations, but is still subject to override by the --config command line flag. If it is populated but points to an invalid path, the program will halt in error.

  • https_proxy or http_proxy - If either of these values are set, and contain a URL that points to a proxy which speaks the HTTP proxy protocol, the Agent will use those values for outbound HTTP requests, just as curl and git behave. Also like the other programs, the Agent will accept inline credentials in the URL, such as http://myUser:S3cr3t@proxy.example.com. If the URL does not contain an explicit port, then the traditional ports for the protocol of the URL will be implicitly inserted: 80 for http:// and 443 for https://. Unlike those other programs, the Agent will accept either environment variable name (https_proxy or http_proxy) and will use that proxy information for all HTTP requests. Be aware that proxy values in any config file provided to the Agent, the default location or values provided by –config, will supersede any proxy specification in these environment variables.

Scan Directives

The SourceClear Agent also supports scans that include per-scan directives (known as scan directives). These can be specified on a per-project basis by placing a srcclr.yml file at the root of the scan.

Here is an example of a srcclr.yml:

    scope: testCompile
    gradle_location: ../gradlew
    gradle_tasks: clean classes
    skip_collectors: gem

This indicates that we only wish to include dependencies that are in the ‘testCompile’ scope, or a scope that ‘testCompile’ inherits from. It’s specifies that the gradlew file exists in the parent folder, and that we should execute using the ‘clean’ and ‘classes’ tasks. Last, it specifies that even though a Gemfile might exist in the repository directory, the agent should skip that particular build system. For details on all of the various scan directives, check out the scan directive documentation.