Repository Management with Nexus
2.7. Adopting a Repository Manager
This section talks about the stages of moving to a repository manager. Adopting a repository manager is not an all or nothing proposition, and there are various levels (or stages) of adoption that can be distinguished when approaching repository management. On one end of the adoption spectrum is the organization that installs a repository manager just to control and consolidate access to a set of remote repositories. On the other end of the spectrum is the organization which has integrated the repository manager into an efficient software development life-cycle, using it to facilitate decision points in the life-cycle, encouraging more efficient collaboration throughout the enterprise, and keeping detailed records to increase visibility into the software development process.
While this isn’t a stage of adoption, Stage Zero is a description of the way software builds work in the absence of a repository manager. When a developer decides that a he needs a particular open source software component, he will download it from the component’s web site, read the documentation, and find the additional software that his components rely on (referred to as "dependencies"). Once he has manually assembled a collection of dependencies from various open source project web sites and proprietary vendors, he will place all these components somewhere on the network so that he, his team members, the build script, the QA team, and the production support team can find it. At any time, other developers may bring in other components, sometimes with overlapping dependencies, placing them in different network locations. The instructions to bring all of these ad-hoc, developer-managed components libraries together in a software build process can become very complicated and hard to maintain.
Maven was introduced to improve this build process by introducing the concept of structured repositories from which the build scripts can retrieve the software components. In Maven language, these software components or dependencies are referred to as "artifacts", a term which can refer to any generic software artifact including components, libraries, frameworks, containers, etc. Maven can identify artifacts in repositories, understand their dependencies, retrieve all that are needed for a successful build, and deploy its output back to repositories when done.
Developers using Maven without a repository manager find most of their software artifacts and dependencies in Maven Central. If they happen to use another remote repository or if they need to add a custom artifact, the solution, in Stage Zero, is to manually manipulate the files in a local repository and share this local repository with multiple developers. While this approach may yield a working build for a small team, managing a shared local repository doesn’t allow an organization to scale a development effort. There is no inherent control over who can set up a local repository, who can add to them or change or delete from them, nor are there tools to protect the integrity of these repositories.
That is, until Repository Managers were introduced.
This is the easiest stage to understand both in terms of benefits to an organization and action required to complete this stage. All you need to do to start proxying a remote repository is to deploy Nexus and start the server with the default configuration. Configure your Maven clients to read from the Nexus public repository group, and Nexus will automatically retrieve artifacts from remote repositories, such as Maven Central, caching them locally.
Without a repository manager, your organization might have hundreds of developers independently downloading the same artifacts from public, remote repositories. With a repository manager, these artifacts can be downloaded once and stored locally. After Stage One, your builds run considerably faster than they did when you relied upon the Maven Central repository.
Once you’ve installed Nexus and you’ve configured all of your organization’s clients to use it as a single point of access to remote repositories, you begin to realize that it now provides you with a central configuration point for the artifacts used throughout your organization. Once you’ve started to proxy, you can start to think about using Nexus as a tool to control policy and what dependencies are allowed to be used in your organization. Nexus Professional provides a procurement plugin which allows for fine-grained control over which artifacts can be accessed from a remote repository. This procurement feature is described in more detail in the section which deals with Life-cycle Integration.
Once you have started to proxy remote repositories and you are using Nexus as a single, consolidated access point for remote repositories, you can start to deploy your own artifacts to Nexus hosted repositories. Most people approach repository management to find a solution for proxying remote repositories, and while proxying is the most obvious and immediate benefit of installing a repository manager, hosting internally generated artifacts tends to be the stage that has the most impact on collaboration within an organization.
To understand the benefits of hosting an internal repository, you have to understand the concept of managing binary software artifacts. Software development teams are very familiar with the idea of a source code repository or a source code management tool. Version control systems such as Subversion, Clearcase, Git, and CVS provide solid tools for managing the various source artifacts that comprise a complex enterprise application, and developers are comfortable checking source out from source control to build enterprise applications. However, past a certain point in the software development life-cycle, source artifacts are no longer relevant. A QA department trying to test an application or an Operations team attempting to deploy an application to a production network no longer needs access to the source artifacts. QA and Operations are more interested in the compiled end-product of the software development life-cycle: the binary software artifacts. A repository manager allows you to version, store, search, archive, and release binary software artifacts derived from the source artifacts stored in a source control system. A repository manager allows you to apply the same systematic operations on binary software artifacts which you currently apply to your source code.
When your build system starts to deploy artifacts to an internal repository, it changes the way that developers and development groups can interact with one another in an enterprise. Developers in one development group can code and release a stable version of an internal library, deploy this library to an internal Nexus release repository, and so share this binary artifact with another group or department. Without a repository manager managing internal artifacts, you have ad-hoc solutions and the organizational equivalent of "duct tape". How does the infrastructure group send a new library to the applications group without Nexus? Someone copies a file to a shared directory, and sends an email to the team lead. Organizations without repository managers are full of these ad-hoc processes that get in the way of efficient development and deployment.
With a repository manager, every developer and every development group within the enterprise understands and interacts with a common collaborative structure: the repository manager. Do you need to interact with the Commerce team’s new API? Just add a dependency to your project and Maven will retrieve the library from Nexus automatically.
One of the other direct benefits of deploying your own artifacts to a repository such as Nexus is the ability to quickly search the metadata and contents of those artifacts both via a web UI and through IDE integration tools such as m2eclipse. When you start to deploy internal artifacts you can synchronize all development groups to a common version and naming standard, and you can use the highly configurable authentication and role-based access controls to control which developers and which development groups can deploy artifacts to specific repositories or paths within a repository.
Developing this collaborative model further, if your application is being continuously built and deployed using a tool like Hudson, a developer can checkout a specific module from a large multi-module build and not have to constantly deal with the entire source tree at any given time. This allows a software development effort to scale efficiently. If every developer working on a complex enterprise application needs to checkout the entire source tree every time he or she needs to make a simple change to a small component, you are quickly going to find that building the entire application becomes a burdensome bottleneck to progress. The larger your enterprise grows, the more complex your application becomes, the larger the collective burden of wasted time and missed opportunities. A slow enterprise build prevents the quick turnaround or quick feedback loop that helps your developers maintain focus during a development cycle.
Once you are building with Maven, sharing binary artifacts with Nexus, continuously testing and deploying with Hudson, and generating reports and metrics with tools like Sonar, your entire organization gains a collaborative "central nervous system" that enables a more agile approach to software development.
Once you’ve configured a repository manager to proxy remote repositories and you are using a repository manager as an integration point between developers and departments, you start to think about the various ways your repository manager can be used to support the decisions that go into software development. You can start using the repository manager to stage releases and supporting the work-flow associated with a managed release, and you can use the procurement features of a tool like Nexus Professional to give management more visibility into the origins, characteristics and open source licenses of the artifacts used during the creation of an enterprise application.
Nexus Professional enables organizations to integrate the management of software artifacts tightly with the software development life-cycle: Provisioning, Compliance, Procurement, Enterprise Security, Staging and other capabilities that support the work-flow that surrounds a modern software development effort.
Using Nexus Professional’s Maven Settings management feature and integrated security features you can configure a developer’s Maven settings by running a single, convenient Maven goal and downloading customized settings for a particular developer. When you use Maven and Nexus Professional together, developers can get up and running quickly, collaborating on projects that share common conventions without having to manually install dependencies in local repositories.
- Using Nexus as an integration point between Engineering and Operations means that Engineering can be responsible for delivering solid, tested artifacts to Quality Assurance and Operations via a standard repository format. Often development teams are roped into the production deployment story and become responsible for building entire production environments within a build system. This conflates software engineering with system administration and blurs the line between Engineering and Operations. If you use Nexus as a end-point for releases from Engineering, Operations can then retrieve, assemble, and configure an application from tested components in the Nexus repository.
- Procurement, staging, and audit logs are all features which increase the visibility into who and what is involved with your software development effort. Using Nexus Professional, Engineering can create the reports and documents which can be used to facilitate discussions about oversight. Organizations subject to various regulations often need to produce a list of components involved in a software release. Legal departments often require a list of open source licenses being used in a particular software component, and managers often lack critical visibility into the software development process.
- The ease with which today’s developer can add a dependency on a new open source library and download this library from a Central repository has a downside. Organizations large and small are constantly wondering what open source libraries are being used in applications, and whether these libraries have acceptable open source licenses for distribution. The Procurement features of Nexus Professional give architects and management more oversight over the artifacts which are allowed into an organization. Using the Procurement features, a Nexus administrator or Procurement manager can allow or deny specific artifacts by group, version, or path. You can use the procurement manager as a firewall between your own organization’s development environment and the 95,000 artifacts available on the Maven Central repository.
- Enterprise Security
- Nexus' LDAP integration allows an enterprise to map existing LDAP groups to Nexus roles and provides Nexus administrators with a highly configurable interface to control which individuals or groups have access to a fine-grained set of Nexus permissions.
- Nexus Professional adds an important step to the software release work-flow, adding the concept of a managed (or staged) release to a hosted repository. When a developer needs to perform a production release, Nexus Professional can isolate the artifacts involved in a release in a staged repository which can then be certified and tested. A manager or a quality assurance tester can then promote or discard a release. The staging feature allows you to specify the individuals that are allowed to promote a release and keeps an audit of who was responsible for testing, promoting, or discarding a software release.