SKM IT World

Just another blog about IT


1 Comment

Generate P2 Repository From Maven Artifacts In 2017

Some years ago, I wrote a blog post about how to generate a P2 repository based on Maven artifacts. That described approach is obsolete nowadays and I’d like to show a new approach that is based on the p2-maven-plugin that was created to solve exactly this problem.

P2-Maven-Plugin Integration in Maven Build Life Cycle

First at all, we bind the p2-maven-plugin’s goal site to the Maven’s life cycle phase package. This goal is responsible for the generation of the P2 repository.

<plugin>
  <groupId>org.reficio</groupId>
  <artifactId>p2-maven-plugin</artifactId>
  <version>1.3.0</version>
  <executions>
    <execution>
      <id>default-cli</id>
      <phase>package</phase>
      <goals>
        <goal>site</goal>
      </goals>
      <!--... -->
    </execution>
  </executions>
</plugin>

Generating P2 Repository

Now, we can define which Maven artifacts should be a part of the new P2 repository. It is irrelevant for the p2-maven-pluging if the defined artifacts have already a OSGi manifest or not. If no OSGi manifest exists, the plugin will generate one.


<execution>
<!-- ... -->
<configuration>
  <artifacts>
    <!-- specify your dependencies here -->
    <!-- groupId:artifactId:version -->
    <artifact>
      <id>com.google.guava:guava:jar:23.0</id>
      <!-- Artifact with existing OSGi-Manifest-->
    </artifact>
    <artifact>
      <id>commons-io:commons-io:1.3</id>
      <!-- Artifact without existing OSGi-Manifest-->
    </artifact>
  </artifacts>
</configuration>
</execution>

The artifacts are specified by the pattern groupId:artifactId:version. If you want to save some typing, use the Buildr tab on MVN repository website for copying the right dependency declaration format.

This sample configuration creates a P2 repository that look like the following one:


target/repository
├── artifacts.jar
├── category.xml
├── content.jar
└── plugins
    ├── com.google.code.findbugs.jsr305_1.3.9.jar
    ├── com.google.errorprone.error_prone_annotations_2.0.18.jar
    ├── com.google.guava_23.0.0.jar
    ├── com.google.j2objc.annotations_1.1.0.jar
    ├── commons-io_1.3.0.jar
    └── org.codehaus.mojo.animal-sniffer-annotations_1.14.0.jar

1 directory, 9 files

 

The default behavior of the plugin is, that all transitive dependencies of the defined artifact are also downloaded and packed into the P2 repository. If you don’t want it, then you have to set the option transitive to false in the corresponded artifact declaration. If you need the sources (if they exist in the Maven repository) of the defined artifact in the P2 repository, then you have to set the option source to true in the corresponded artifact declaration.

<!-- ... -->
<artifact>
  <id>com.google.guava:guava:jar:23.0</id>
  <transitive>false</transitive>
  <source>true</source>
</artifact>
<!-- ... -->

Then the generated P2 repository looks like the following one:


target/repository
├── artifacts.jar
├── category.xml
├── content.jar
└── plugins
    ├── com.google.guava.source_23.0.0.jar
    ├── com.google.guava_23.0.0.jar
    └── commons-io_1.3.0.jar

1 directory, 6 files

Generating P2 Repository With Grouped Artifacts

In some situations, you want to group artifacts in so-called feature. p2-maven-plugin provides an option that allows to group the Maven artifact directly into features. The definition of the artifacts is the same like above. The difference is that it has to be inside the corresponded feature. Then, the feature definition needs some meta data information like feature ID, feature version, description etc.


<!-- ...-->
<configuration>
  <featureDefinitions>
    <feature>
      <!-- Generate a feature including artifacts that are listed below inside the feature element-->
      <id>spring.feature</id>
      <version>4.3.11</version>
      <label>Spring Framework 4.3.11 Feature</label>
      <providerName>A provider</providerName>
      <description>${project.description}</description>
      <copyright>A copyright</copyright>
      <license>A licence</license>
      <artifacts>
        <artifact>
          <id>org.springframework:spring-core:jar:4.3.11.RELEASE</id>id>
        </artifact>
        <artifact>
          <id>org.springframework:spring-context:jar:4.3.11.RELEASE</id>id>
          <source>true</source>
        </artifact>
      </artifacts>
    </feature>
    <!--...-->
  </featureDefinitions>
  <!-- ... -->
<configuration>

Then the generated P2 repository looks like the following one:


target/repository
├── artifacts.jar
├── category.xml
├── content.jar
├── features
│   └── spring.feature_4.3.11.jar
└── plugins
    ├── org.apache.commons.logging_1.2.0.jar
    ├── org.springframework.spring-aop.source_4.3.11.RELEASE.jar
    ├── org.springframework.spring-aop_4.3.11.RELEASE.jar
    ├── org.springframework.spring-beans.source_4.3.11.RELEASE.jar
    ├── org.springframework.spring-beans_4.3.11.RELEASE.jar
    ├── org.springframework.spring-context.source_4.3.11.RELEASE.jar
    ├── org.springframework.spring-context_4.3.11.RELEASE.jar
    ├── org.springframework.spring-core_4.3.11.RELEASE.jar
    ├── org.springframework.spring-expression.source_4.3.11.RELEASE.jar
    └── org.springframework.spring-expression_4.3.11.RELEASE.jar

2 directories, 14 files

Of course both options (generating p2 repository with feature and only with plugins) can be mixed.

p2-maven-plugin provides more options like excluding specific transitive dependencies, referencing to other eclipse features and so on. For more information, please look at the p2-maven-plugin homepage.

Now, we can generate P2 repositories from Maven artifacts. We lacks of how to deploy this P2 repository to a Repository manager like Artifactory or Sonatype Nexus. Both repository manager supports P2 repositories, Artifactory in the Professional variant (cost money) and Sonatype Nexus in OSS variant (free). For Nexus, it’s important that you use the version 2.x. The newest version, 3.x, doesn’t yet support P2 repositories.

Deploying P2 Repository to a Repository Manager

First at all, we want that our generated P2 repository is packed into a zip file. Therefore, we add the tycho-p2-repository-plugin to the Maven build life cycle:


<plugin>
  <groupId>org.eclipse.tycho</groupId>
  <artifactId>tycho-p2-repository-plugin</artifactId>
  <version>1.0.0</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>archive-repository</goal>
      </goals>
    </execution>
  </executions>
</plugin>


Then, we have to mark this zip file, so that Maven recognize that it has to deploy it during the deploy phase to a repository manager. For this, we add the build-helper-maven-plugin to the Maven build life cycle.

<!-- Attach zipped P2 repository to be installed and deployed in the Maven repository during the deploy phase. -->
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <version>3.0.0</version>
  <executions>
    <execution>
      <goals>
        <goal>attach-artifact</goal>
      </goals>
      <configuration>
        <artifacts>
          <artifact>
            <file>target/${project.artifactId}-${project.version}.zip</file>
            <type>zip</type>
          </artifact>
        </artifacts>
      </configuration>
    </execution>
  </executions>
</plugin>

Now, the generated P2 repository can be addressed by other projects. For more information about how to address the P2 repository, please have a look on the documentation of your repository manager.

A whole pom.xml sample can be found on Github.

Links


2 Comments

Generate P2 Repository From Maven Dependencies

A new version of this blog post is published under the title “Generate P2 Repository From Maven Artifacts in 2017

Motivation

If you use Maven Tycho to build your Eclipse RCP application, you can use Maven dependencies as you know it from Maven (for more information see [1]). I notice that Tycho takes many times for computing the target platform from the Maven dependencies during the build. I expect a speed up of build time if I use a P2 repository instead of Maven dependencies directly. Therefore, the task is how do I get the Maven dependencies to a P2 repository.

Procedure for generating a P2 repository from Maven dependencies

We generate the P2 repository with the help of some Maven’s plugin and some Maven Tycho’s plugin:

  • Builder Helper Maven Plugin
  • Maven Dependency Plugin
  • Maven Tycho’s P2 Extra Plugin
  • Maven Tycho’s P2 Plugin
  • Maven Tycho’s P2 Repository Plugin

Maven Tycho’s Plugins use Eclipse standard tools internally. How the Eclipse standard tools work are described well on this blog post [2]. Further useful information about Tycho’s plugins can be found in [3] and [4].

We create a POM project and configure plugins mentioned above, so that the following procedure can work:

  1. Define the Maven dependencies, that should be add to the P2 repository, in the <dependencies> section.
  2. Copy these defined dependencies to the source location of the Feature and Bundle Publisher with Maven Dependency Plugin.
  3. Generate P2 repository with P2 Extra Plugin.
  4. Add categories to the P2 metadata with P2 Plugin, so that you can see your P2 repository in Eclipse Target Platform Wizard.
  5. Zip P2 repository with P2 Repository Plugin.
  6. Attach zipped P2 repository to be installed and deployed in the Maven repository during the deploy phase with Builder Helper Plugin.

You can find the whole POM configuration in [5].

How to use zipped P2 Repository from Maven Repository

When you use Nexus as Maven Repository, you can use the Nexus Unzip Plugin ([6]).

Links

[1] Maven Tycho How to – Dependency on pom-first artifacts
[2] Blog Post about Generation P2 Repository with Eclipse Standard Tool
[3] P2 Extra Plugin Project Site
[4] Sonatype Wiki Page about P2 Extra Plugin
[5] POM Project on GitHub
[6] Nexus Unzip Plugin Documentation


Leave a comment

Automated Build of RCP Artifacts with Maven Tycho – A field report (Part 2)

In my last post of this series, I described how to solve the dependency problem. In this post, I will describe how to build Eclipse Plugin and Eclipse Feature with Maven Tycho and which problems I have.

POM Definition

The procedure for both Eclipse artifacts is similar. In both, you have to insert a pom.xml and fill it with the Maven coordinate (group id, artifact id and version) and the packaging type. But there exists some rules about the content.

The packaging type is depended on what Eclipse artifact should be built:

Eclipse ArtifactPackaging Type
Eclipse Plugineclipse-plugin
Eclipse Featureeclipse-feature

The version specification in the POM is depended on the version specification in the manifest (in case of Eclipse plugin) and by the version specification in the feature.xml (in case of Eclipse feature) respectively:

MANIFEST.MF / feature.xmlpom.xmlDescription
1.5.1.qualifier1.5.1-SNAPSHOTDevelopment version
1.5.11.5.1Release version

\The specification of the artifact id in the POM must be equal like the bundle symbolic name in the manifest (in case of Eclipse plugin) and by the specification of the id in the feature.xml (in case of Eclipse feature) respectively. That’s all you need to define the pom.xml. Then call only mvn clean install and the Eclipse artifact is built by Maven Tycho.

Troubleshooting

I met two problems during the introduction of Maven Tycho for the building of Eclipse plugins. The first problem was that Maven Tycho sometimes throws a NullPointerException during the read-out of the manifest file. The reason is that a blank must be between the colon, that follows after the manifest header, and the command.

# Don't
Import-Package:com.library.*
# Do
Import-Package: com.library.*

The second problem was that Maven Tycho throws compiler error although everything is alright in Eclipse. The analyze of the error message shows that the command for the Import-Package in the manifest is not completed. A second analyze shows that package names of used Eclipse RCP artifacts are missing. After adding these missing package names in the manifest, Maven Tycho builds without error. But this also means that Eclipse does not generate the manifest osgi-compliantly.


Leave a comment

Automated Build of RCP Artifacts with Maven Tycho – A field report (Part 1)

This post series should not be a tutorial for Maven Tycho. You can find many tutorials on the web (see for example [2] and [3]). This post series should be a field report about the problems during the introduction of Maven Tycho.

Why Maven Tycho?

One requirement is that we can use Maven artifacts. So it seems obvious to search after a solution based on Maven. After a search on the web, two Maven plugins are noticed:

  • PDE Maven Plugin ([7]): This plugin builds the RCP artifacts with the help of a generated Ant script by Eclipse. But this generated Ant script is error-prone.
  • Maven Tycho Plugin ([1]): Very young plugin to build RCP artifacts (current version 0.13.0), but the community is very active. It tries to build RCP artifacts in the similar way than Eclipse.

The Proceeding

During working the tutorials out the impression occurs that the main problem is to solve the dependency problem. So this is the first task for the introduction of Maven Tycho.

Solution of the dependency problem

In Maven the dependencies are declared in the POM file. In Eclipse the dependencies are declared by a target definition file. The goal is to merge these both worlds. Maven Tycho enables Maven to build against a pre-defined target definition file.

The next question is where the dependencies come from. Maven’s solution is the Maven Repository. Eclipse has its own solution, P2 repository. In Eclipse’s target definition file you can specify a URL to a P2 repository and Eclipse searches after the needed dependencies, automatically.

A further requirement is that the Maven repository can still use. The reason is that our server artifacts are stored in a Maven repository and the communication between client and server is via Hessian, so some artifacts are needed by client and server. Furthermore, many third party libraries are only available in a Maven repository.

All examples in the tutorial work only with P2 repositories. But the introducing of a P2 repository is not an option, because then two repositories have to be administered.

First Approach – Define a Target Definition File with the Type ‘Directory’

The idea of this approach is that you can define a folder with jars (that are the dependencies) in the target definition file. So it seems that the problem can be reduced to how I get the dependencies to this folder.

This problem can be solved by the Maven Dependency Plugin ([9]). It has the goal copy-dependencies. This goal copies all dependencies that are declared in the <dependencies>-element of the pom file to a declared output directory. The configuration of the dependency plugin could look like the following one:

<?xml version="1.0" encoding="UTF-8"?>
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <version>2.3</version>
  <executions>
    <execution>
      <id>copy-dependencies</id>
      <phase>process-resources</phase>
      <goals>
        <goal>copy-dependencies</goal>
      </goals>
      <configuration>
        <excludeTransitive>true</excludeTransitive>
        <outputDirectory>${project.build.directory}/plugins</outputDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

This solution works well in Eclipse, but not in with Maven’s mvn clean install, because Maven Tycho doesn’t support target type ‘Directory’, only the target type ‘Software Site’. So this solution doesn’t work for us.

Second Approach – A solution based on Maven’s ‘pom-first’ approach

With Maven Tycho, you can define that the RCP artifacts should be built against the dependencies that are defined in the POM (old-known Maven way).

<?xml version="1.0" encoding="UTF-8"?>
<plugin>
  <groupId>org.eclipse.tycho</groupId>
  <artifactId>target-platform-configuration</artifactId>
  <version>0.13.0</version>
  <configuration>
    <pomDependencies>consider</pomDependencies>
  </configuration>
</plugin>

With Maven Tycho, the build works fine. But in Eclipse you cannot work in Eclipse-known way, because it has not the needed dependencies. I thought I could solve this problem with the Maven Eclipse Plugin and its goal eclipse:eclipse. This plugin generates all needed Eclipse files but the dependencies are generated in the Java Build Path named Referenced Library. But Eclipse needs the dependencies for the RCP plugin development in the Java Build Path named Plug-in Dependencies. Next try!

Third Approach – Nexus P2 Repository Plugin

A further research results that Nexus OSS should get a new plugin that can simulate the access to a Maven repository whether it would be a P2 repository (P2 proxy) (see [4]). This plugin is an experimental feature at the moment, so you can only use the SNAPSHOT version of this plugin. Unfortunately, the SNAPSHOT version (mid December 2011) that I used did not work with Nexus OSS version 1.9.2.2. But I think this would be a good idea for this problem.

Fourth Approach – Generation of a Local P2 Repository

A hint in the Tycho user mailing list (see [5]) helps me find this approach. It exits a Maven Tycho plugin (see [6]), that can generate a P2 repository from a directory with a bunch of bundles. In the first approach, we used the Maven Dependency Plugin to copy all needed dependencies in a directory. For this approach I combine these two plugins with two another plugins to build a local P2 repository.

By the Maven Dependency Plugin all dependencies are copied to a directory named plugins. The Tycho P2 Extra Plugin can generate from this directory a local P2 repository. Unfortunately, this plugin does not generate the needed feature.jar. But without this jar Eclipse does not recognize that we have here a P2 repository (an introduction to the P2 repository you can find in [10]). So we have to build this jar.

We create a new Maven project with the packaging option jar. In this project, there is only one file: feature.xml. In this xml file the name of the bundles of the plugins directory are stored. The next code snippet shows an example for this entry:

<?xml version="1.0" encoding="UTF-8"?>
<plugin  id="org.eclipse.core.contenttype"  download-size="0"  install-size="0"  version="3.4.100.v20110423-0524"  unpack="false"/>

Then this new generated feature.jar are copied by the Maven Dependency Plugin to the directory features. Now with these two directories (features and plugins), the Tycho P2 Extra Plugin can generate a local P2 repository that are recognized by Eclipse.

The next problem is that the target definition file accepts only absolute path for the target type ‘Software Site’. But we want a target definition file that is independent of the platform. So we save a target definition file with a token for the location of the local P2 repository in src/main/resources. During the Maven build this token are replaced by absolute platform-dependent path to the location of the local P2 repository. This replacement is done by the Maven Replacer Plugin ([11]). The next code snippet shows the whole plugin configuration for the above described solution:

<?xml version="1.0" encoding="UTF-8"?>
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <version>2.3</version>
  <executions>
    <execution>
      <id>copy-dependencies</id>
      <phase>process-resources</phase>
      <goals>
        <goal>copy-dependencies</goal>
      </goals>
      <configuration>
        <excludeTransitive>true</excludeTransitive>
        <outputDirectory>${project.build.directory}/source/plugins</outputDirectory>
      </configuration>
    </execution>
    <execution>
      <id>copy-feature</id>
      <phase>process-resources</phase>
      <goals>
        <goal>copy</goal>
      </goals>
      <configuration>
        <artifactItems>
          <artifactItem>
            <groupId>skm.p2</groupId>
            <artifactId>repository.feature</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <type>jar</type>
            <overWrite>true</overWrite>
            <outputDirectory>${project.build.directory}/source/features</outputDirectory>
          </artifactItem>
        </artifactItems>
      </configuration>
    </execution>
  </executions>
</plugin>
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-resources-plugin</artifactId>
  <version>2.5</version>
  <executions>
    <execution>
      <id>copy-target-definition</id>
      <phase>process-resources</phase>
      <goals>
        <goal>copy-resources</goal>
      </goals>
      <configuration>
        <resources>
          <resource>
            <directory>src/main/resources</directory>
            <includes>
              <include>${target.definition}</include>
            </includes>
          </resource>
        </resources>
        <outputDirectory>${project.build.directory}</outputDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>
<plugin>
  <groupId>com.google.code.maven-replacer-plugin</groupId>
  <artifactId>maven-replacer-plugin</artifactId>
  <version>1.4.0</version>
  <executions>
    <execution>
      <phase>prepare-package</phase>
      <goals>
        <goal>replace</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <ignoreMissingFile>false</ignoreMissingFile>
    <file>target/${target.definition}</file>
    <regex>false</regex>
    <replacements>
      <replacement>
        <token>$repository.location$</token>
        <value>${project.build.directory}/repository/</value>
      </replacement>
      <replacement>
        <token>
        </token>
        <value>/</value>
      </replacement>
    </replacements>
  </configuration>
</plugin>
<plugin>
  <groupId>org.eclipse.tycho.extras</groupId>
  <artifactId>tycho-p2-extras-plugin</artifactId>
  <version>0.13.0</version>
  <executions>
    <execution>
      <phase>prepare-package</phase>
      <goals>
        <goal>publish-features-and-bundles</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <compress>false</compress>
  </configuration>
</plugin>
<plugin>
  <groupId>org.eclipse.tycho</groupId>
  <artifactId>tycho-p2-repository-plugin</artifactId>
  <version>0.13.0</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>archive-repository</goal>
      </goals>
    </execution>
  </executions>
</plugin>
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <version>1.7</version>
  <executions>
    <execution>
      <id>attach-artifacts</id>
      <phase>package</phase>
      <goals>
        <goal>attach-artifact</goal>
      </goals>
      <configuration>
        <artifacts>
          <artifact>
            <file>target/${target.definition}</file>
            <type>target</type>
            <classifier>indigo</classifier>
          </artifact>
        </artifacts>
      </configuration>
    </execution>
  </executions>
</plugin>

In Eclipse, this generated target definition file works fine, but in Maven Tycho it does not work: Before Maven Tycho builds the local P2 repository and the target definition file, it tries to build the RCP plugins against the not yet existing target definition file and the build failed.

Fifth Approach and Final Solution – Combination of Dependency in the POM and Target Definition

We know Maven works with in the POM defined dependencies, perfectly and Eclipse works with the target platform, perfectly. So the idea is to combine this two approaches, so that every tool can work in its perfect way. So I combine the first both approaches that are described in the above chapters.

All dependencies that are used by all RCP plugins are defined in the root project POM, so that every sub module that defines one RCP plugin is built against these dependencies and one sub module is for the building of the target definition file (this build is described in the first approach). Maven Tycho builds the RCP artifacts in the known Maven way and in Eclipse the RCP artifacts are developed with the generated target definition file.

Links

[1] Maven Tycho Home Page
[2] Tutorial by Matthias Holmqvist – Good for the first introduction
[3] Tutorial with example (in German)
[4] Nexus P2 Repository Plugin (Experimental Feature)
[5] Post on Tycho User Mailing List – How to build local P2 Repository
[6] Introduction to the Tycho-P2-Extra-Plugin
[7] PDE Maven Plugin
[8] Hessian Protocol
[9] Maven Dependency Plugin
[10] Introduction to the format of P2 repositoy
[11] Maven Replacer Plugin