Using the Nexus Repository Manager as well as the tools for software supply chain automation using the Nexus IQ Server and associated tools of the Nexus platform requires an understanding of a few concepts and terms like Component, Repository, Repository Format and others. This chapter provides you with all the necessary background and knowledge as well as an idea of a progression in your usage of the tools from the Nexus platform.
The Nexus platform with Nexus Repository Manager Pro, Nexus Repository Manager OSS and Nexus IQ Server is all about working with components and repositories.
So what are components? A component is a resource like a library or a framework that is used as part of your software application at run-time, integration or unit test execution time or required as part of your build process. It could be an entire application or a static resource like an image.
Typically these components are archives of a large variety of files including
The archives are using numerous formats such as
Components provide all the building blocks and features that allow a development team to create powerful applications by assembling them and adding their own business related components to create a full-fledged, powerful application.
In different tool-chains components are called artifact, package, bundle, archive and other terms. The concept and idea remains the same and component is used as the independent, generic term.
Components are identified by a set of specific values - the coordinates. A generic set of these coordinates is the usage of group, name and version. The names and the usage of these coordinates changes with the tool-chains used. Components can also be the anchor for further metadata.
Assets. Assets are the material addition to all this metadata. The actual archive file is an asset associated with the component. Many formats have a one-to-one mapping for component to asset.
More complex formats however have numerous assets associated with a component. For example a typical JAR component in a Maven repository is defined at least by the POM and the JAR files - both of which constitute separate assets belonging to the same components. Additional files such as JavaDoc or Sources JAR files are assets that belong to the same component.
The Docker format, on the other hand, gives assets unique identifiers and calls them Docker layers. These assets can be reused for different components - the Docker images. A Docker layer, for example, could be a specific operating system referenced by multiple Docker images.
Components in Repositories. A wide variety of components exists and more are continuously created by the open source community as well as proprietary vendors. There are libraries and frameworks written in various languages on different platforms that are used for application development every day. It has become a default pattern to build applications by combining the features of multiple components with your own custom components containing your application code to create an application for a specific domain.
In order to ease the consumption and usage of components, they are aggregated into collections of components. These are called a repository and are typically available on the internet as a service. On different platforms terms such as registry and others are used for the same concept.
Example for such repositories are
and a number of others. Components in these repositories are accessed by numerous tools including
and many, many others.
Repositories have Formats. The different repositories use different technologies to store and expose the components in them to client tools. This defines a repository format and as such is closely related to the tools interacting with the repository.
E.g. the Maven repository format relies on a specific directory structure defined by the identifiers of the components and a number of XML formatted files for metadata. Component interaction is performed via plain HTTP commands and some additional custom interaction with the XML files.
Other repository formats use databases for storage and REST API interactions, or different directory structures with format specific files for the metadata.
Maven developers are familiar with the concept of a repository, since repositories are used by default. The primary type of a binary component in a Maven format repository is a JAR file containing Java byte-code. This is due to the Java background of Maven and the fact that the default component type is a JAR. Practically however, there is no limit to what type of component can be stored in a Maven repository. For example, you can easily deploy WAR or EAR files, source archives, Flash libraries and applications, Android archives or applications or Ruby libraries to a Maven repository.
Every software component is described by an XML document called a Project Object Model (POM). This POM contains information that describes a project and lists a project’s dependencies — the binary software components, which a given component depends upon for successful compilation or execution.
When Maven downloads a component like a dependency or a plugin from a repository, it also downloads that component’s POM. Given a component’s POM, Maven can then download any other components that are required by that component.
Maven and other tools, such as Ivy or Gradle, which interact with a Maven repository to search for binary software components, model the projects they manage and retrieve software components on-demand from a repository.
The Central Repository. When you download and install Maven without any customization, it retrieves components from the Central Repository. It serves millions of Maven users every single day. It is the default, built-in repository using the Maven repository format and is managed by Sonatype. Statistics about the size of the Central Repository are available at http://search.maven.org/#stats.
The Central Repository is the largest repository for Java-based components. It can be easily used from other build tools as well. You can look at the Central Repository as an example of how Maven repositories operate and how they are assembled. Here are some of the properties of release repositories such as the Central Repository:
In addition to the Central Repository, there are a number of major organizations, such as Red Hat, Oracle or the Apache Software foundation, which maintain separate, additional repositories. Best practice to facilitate these available repositories is to install Nexus Repository Manager OSS or Nexus Repository Manager Pro and use it to proxy and cache the contents on your own network.
Component Coordinates and the Repository Format. Component coordinates create a unique identifier for a component. Maven coordinates use the following values: groupId, artifactId, version, and packaging. This set of coordinates is often referred to as a GAV coordinate, which is short for Group, Artifact, Version coordinate. The GAV coordinate standard is the foundation for Maven’s ability to manage dependencies. Four elements of this coordinate system are described below:
Tools designed to interact Maven repositories translate component coordinates into a URL which corresponds to a
location in a Maven repository. If a tool such as Maven is looking for version
1.2.0 of the
in the group
org.apache.commons, this request is translated into:
Maven also downloads the corresponding POM for commons-lang 1.2.0 from:
This POM may contain references to other components, which are then retrieved from the same repository using the same URL patterns.
Release and Snapshot Repositories. A Maven repository stores two types of components: releases and snapshots. Release repositories are for stable, static release components. Snapshot repositories are frequently updated repositories that store binary software components from projects under constant development.
While it is possible to create a repository which serves both release and snapshot components, repositories are usually segmented into release or snapshot repositories serving different consumers and maintaining different standards and procedures for deploying components. Much like the difference between a production network and a staging network, a release repository is considered a production network and a snapshot repository is more like a development or a testing network. While there is a higher level of procedure and ceremony associated with deploying to a release repository, snapshot components can be deployed and changed frequently without regard for stability and repeatability concerns.
The two types of components managed by a repository manager are:
1.2.0release of the
commons-langlibrary stored in the Central Repository. This release component,
commons-lang-1.2.0.jar, and the associated POM,
commons-lang-1.2.0.pom, are static objects which will never change in the Central Repository. Released components are considered to be solid, stable, and perpetual in order to guarantee that builds which depend upon them are repeatable over time. The released JAR component is associated with a PGP signature, an MD5 and SHA check-sum which can be used to verify both the authenticity and integrity of the binary software component.
1.3and a time-stamp in its name. For example, a snapshot component for
commons-lang 1.3.0might have the name
commons-lang-1.3.0-20090314.182342-1.jarthe associated POM, MD5 and SHA hashes would also have a similar name. To facilitate collaboration during the development of software components, Maven and other clients that know how to consume snapshot components from a repository also know how to interrogate the metadata associated with a Snapshot component to retrieve the latest version of a Snapshot dependency from a repository.
A project under active development produces snapshot components that change over time. A release is comprised of components which will remain unchanged over time.
Looking at the Maven repository format and associated concepts and ideas allowed you grasp some of the details and intricacies involved with different tools and repository formats, that will help you appreciate the need for repository management.
The proliferation of different repository formats and tools accessing them as well as the emergence of more publicly available repositories has triggered the need to manage access and usage of these repositories and the components they contain.
In addition, hosting your own private repositories for internal components has proven to be a very efficient methodology to exchange components during all phases of the software development life cycle. It is considered a best practice at this stage.
The task of managing all the repositories your development teams interact with can be supported by the use of a dedicated server application - a repository manager.
Put simply, a repository manager provides two core features:
Just as Source Code Management (SCM) tools are designed to manage source code, repository managers have been designed to manage and track external dependencies and components generated by your build.
Repository managers are an essential part of any enterprise or open-source software development effort, and they enable greater collaboration between developers and wider distribution of software, by facilitating the exchange and usage of binary components.
Once you start to rely on repositories, you realize how easy it is to add a dependency on an open source software library available in a public repository, and you might start to wonder how you can provide a similar level of convenience for your own developers. When you install a repository manager, you are bringing the power of a repository like the Central Repository into your organization. You can use it to proxy the Central Repositories and other repositories, and host your own repositories for internal and external use.
Capabilities of a Repository Manager. In addition to these two core features, a repository manager can support the following use cases:
Advantages of Using a Repository Manager. Using a repository manager provides a number of benefits including:
Once you adopting a repository manager as a central point of of storage and exchange for all component usage, the next step is expand its use in your efforts to automate and manage the software supply chain throughout your software development life-cycle.
Modern software development practices have shifted dramatically from large efforts of writing new code to the usage of components to assemble applications. This approach limits the amount of code authorship to the business-specific aspects of your software.
A large number of open source components in the form of libraries, reusable widgets or whole applications, application servers and others are now available featuring very high levels of quality and feature sets that could not be implemented as a side effect of your business application development. For example creating a new web application framework and business work-flow system just to create a website with a publishing work-flow would be extremely inefficient.
Development starts with the selection of suitable components for your projects based on comprehensive information about the components and their characteristics e.g., in terms of licenses used or known security vulnerabilities available in Nexus Repository Manager Pro. Besides focusing on being a repository manager it includes features, such as the display of security vulnerabilities as well as license analysis results within search results and the Repository Health Check reports for a proxy repository.
Software supply chain automation progresses through your daily development efforts, your continuous integration builds and your release processes all the way to your applications deployed in production environments at your clients or your own infrastructure.
Nexus IQ Server provides a number of tools to improve your component usage in your software supply chain allowing you to automate your processes to ensure high quality output, while increasing your development speed towards continuous deployment procedures. These include:
Nexus IQ Server enables you to ensure the integrity of the modern software supply chain, amplifying the benefits of modern development facilitating component usage, while reducing associated risks.