Discussion:
Tycho: Eclipse plugin dependency resolution from local repository
(too old to reply)
m***@springdot.org
2008-10-17 23:53:53 UTC
Permalink
I have a MANIFEST.MF declaring this dependency:
Require-Bundle: org.apache.commons.httpclient

I'm trying to stick the httpclient Jar into my local repository so that
Maven/Tycho can resolve this dependency.

I tried several combinations of groupId, artifactId and version when
installing
my artifact into the local repository, but I am unable to make this error go
away:

Resolution errors:
Bundle mybundle - Missing Constraint: Require-Bundle:
org.apache.commons.httpclient; bundle-version="0.0.0"

What would be the correct coordinate?

Is Maven/Tycho able to resolve Eclipse plugin dependencies from the local
repository?

Thanks!
-Max
Igor Fedorenko
2008-10-20 17:25:40 UTC
Permalink
Sorry for the late response, somehow missed this email on Friday.

Our initial focus for tycho was (and still is) building eclipse plugins
developed with PDE. As such, tycho requires all dependencies to be OSGi
bundles and uses build target platform to resolve dependencies. Next dev
build will have initial target platform management support, however, and
can use bundles coming from maven repository.

As far as I know, httpclient jar does not have necessary OSGi manifest
attributes, and cannot be directly used in a tycho build. Our current
approach is to "wrap" such artifacts into OSGi bundles (see, for example,
how we include maven embedder into m2e) but I am open for suggestions
about how to make consumption of plain JARs in tycho builds easier.
Post by m***@springdot.org
Require-Bundle: org.apache.commons.httpclient
I'm trying to stick the httpclient Jar into my local repository so that
Maven/Tycho can resolve this dependency.
I tried several combinations of groupId, artifactId and version when
installing
my artifact into the local repository, but I am unable to make this error go
org.apache.commons.httpclient; bundle-version="0.0.0"
What would be the correct coordinate?
Is Maven/Tycho able to resolve Eclipse plugin dependencies from the local
repository?
Thanks!
-Max
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email
Eugene Kuleshov
2008-10-20 17:28:51 UTC
Permalink
Igor,

If I am not mistaken, Eclipse has httpclient as well as bunch of other
OSGi wrappers published on their Ganymede update site as enabling
features. Also there is Maven (and probably OSGi) repository hosted by
SpringSource that may have this bundle too.

regards,
Eugene
Post by Igor Fedorenko
Sorry for the late response, somehow missed this email on Friday.
Our initial focus for tycho was (and still is) building eclipse plugins
developed with PDE. As such, tycho requires all dependencies to be OSGi
bundles and uses build target platform to resolve dependencies. Next dev
build will have initial target platform management support, however, and
can use bundles coming from maven repository.
As far as I know, httpclient jar does not have necessary OSGi manifest
attributes, and cannot be directly used in a tycho build. Our current
approach is to "wrap" such artifacts into OSGi bundles (see, for example,
how we include maven embedder into m2e) but I am open for suggestions
about how to make consumption of plain JARs in tycho builds easier.
Post by m***@springdot.org
Require-Bundle: org.apache.commons.httpclient
I'm trying to stick the httpclient Jar into my local repository so that
Maven/Tycho can resolve this dependency.
I tried several combinations of groupId, artifactId and version when
installing
my artifact into the local repository, but I am unable to make this error go
org.apache.commons.httpclient; bundle-version="0.0.0"
What would be the correct coordinate?
Is Maven/Tycho able to resolve Eclipse plugin dependencies from the local
repository?
Thanks!
-Max
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email
m***@springdot.org
2008-10-20 17:58:12 UTC
Permalink
Igor,

thanks for the response.
Post by Igor Fedorenko
Sorry for the late response, somehow missed this email on Friday.
No problem :)
Post by Igor Fedorenko
Our initial focus for tycho was (and still is) building eclipse plugins
developed with PDE.
That's exactly my focus too.
Post by Igor Fedorenko
As such, tycho requires all dependencies to be OSGi
bundles and uses build target platform to resolve dependencies. Next dev
build will have initial target platform management support, however, and
can use bundles coming from maven repository.
I'm looking forward to trying this out.
Post by Igor Fedorenko
As far as I know, httpclient jar does not have necessary OSGi manifest
attributes, and cannot be directly used in a tycho build. Our current
approach is to "wrap" such artifacts into OSGi bundles (see, for example,
how we include maven embedder into m2e)
Let me clarify that httpclient is just an example.
Making it and my other 3rd party artifacts to be OSGi bundles is the easy
part.
I've planned to deploy them then to my group repository.

However, for my internally developed artifacts it's a different story.
All of them are Eclipse plugins.
I want to build them through a CI framework (probably Continuum) and
deploy them also to the group repository.
Then, members of my team can work on individual plugins and the plugins'
OSGi dependencies
(as expressed in MANIFEST.MF) are resolved from the group repository.

That's my plan in a nutshell. Having a target platform in this picture
where a base set of plugins are resolved from is fine, too.
Post by Igor Fedorenko
but I am open for suggestions
about how to make consumption of plain JARs in tycho builds easier.
I wouldn't cater to consuming plain Jars in general with Tycho.
I would target OSGi bundles only instead.
If there was a way to map an OSGi dependency into a Maven coordinate,
and Maven/Tycho was able to resolve the OSGi dependency from a Maven
repository,
that might make the consumption of OSGi bundles easier.

-Max
Post by Igor Fedorenko
Post by m***@springdot.org
Require-Bundle: org.apache.commons.httpclient
I'm trying to stick the httpclient Jar into my local repository so that
Maven/Tycho can resolve this dependency.
I tried several combinations of groupId, artifactId and version when
installing
my artifact into the local repository, but I am unable to make this error
go
org.apache.commons.httpclient; bundle-version="0.0.0"
What would be the correct coordinate?
Is Maven/Tycho able to resolve Eclipse plugin dependencies from the local
repository?
Thanks!
-Max
Igor Fedorenko
2008-10-21 17:12:06 UTC
Permalink
Post by m***@springdot.org
Let me clarify that httpclient is just an example.
Making it and my other 3rd party artifacts to be OSGi bundles is the easy
part.
I've planned to deploy them then to my group repository.
However, for my internally developed artifacts it's a different story.
All of them are Eclipse plugins.
I want to build them through a CI framework (probably Continuum) and
deploy them also to the group repository.
Then, members of my team can work on individual plugins and the plugins'
OSGi dependencies
(as expressed in MANIFEST.MF) are resolved from the group repository.
That's my plan in a nutshell. Having a target platform in this picture
where a base set of plugins are resolved from is fine, too.
Actually, this is the primary use case for "target platform management"
feature we're adding to tycho and I would be very interested to know how
well (or not so well) it works for your projects. There is no user
documentation yet, but here is relatively detailed description of how
this works.

In the just announced tycho 0.3.0-DEV-1756, build target platform can be
specified using dependencyManagement pom.xml section. It is possible to
specify path to local eclipse installation using dependency with system
scope, but more interestingly, it is also possible to specify
eclipse-feature and eclipse-plugin artifacts using maven
groupId/artifactId/version coordinates.

At build time, tycho will resolve all eclipse-feature artifacts
(transitively) and eclipse-plugin artifacts (non-transitively) from
maven repositories and target platform used during the build will
include features/bundles both from local eclipse-installation and from
maven repositories. Build results can be installed/deployed into maven
local/remote repositories by invoking regular "install" or "deploy"
build phases.

So in your scenario, CI framework can run "tycho/bin/mvn deploy" for set
of related projects and everything is expected to work "the maven way"
assuming dependencyManagement is configured properly. We are using
Hudson, btw, but I guess Continuum should work too.

You can see this in action in integration test [1]. The test first runs
build01 to install a feature and couple of bundles into local maven
repository. Then it runs build02 which uses the feature and bundles from
the first build. You can see specification of build02 target platform in
pom.xml [2].

Please note that -Dtycho.targetPlatform OVERRIDES target platform in
pom.xml. In other words, target platform specified in
dependencyManagement sections will be ignored if -Dtycho.targetPlatform
is specified on command line or via settings.xml/pom.xml. (I should have
added a warning, I guess)

Also note that there is no PDE integration yet. We can talk about using
tycho build target platform in PDE if you're happy with how it works on
command line.

[1]
http://svn.sonatype.org/m2eclipse/tycho/trunk/tycho-its/projects/tycho164/
[2]
http://svn.sonatype.org/m2eclipse/tycho/trunk/tycho-its/projects/tycho164/build02/pom.xml
Post by m***@springdot.org
Post by Igor Fedorenko
but I am open for suggestions
about how to make consumption of plain JARs in tycho builds easier.
I wouldn't cater to consuming plain Jars in general with Tycho.
I would target OSGi bundles only instead.
If there was a way to map an OSGi dependency into a Maven coordinate,
and Maven/Tycho was able to resolve the OSGi dependency from a Maven
repository,
that might make the consumption of OSGi bundles easier.
I think I know what you mean and I might have some ideas how to
implement it, although not necessary exactly as you describe it here. I
can't be more specific at the moment, but stay tuned ;-).

--
Regards,
Igor
m***@springdot.org
2008-10-22 02:08:13 UTC
Permalink
Hi Igor,

it works: I was able to satisfy the httpclient dependency with an artifact
coming from the local repository, following the example from the integration
test.

However: Now I have to "declare" all of my *own* plugins in the
dependencyManagement section in order to build a plugin X, install/deploy X
into the local/group repository, so that I can have the dependency of another
plugin Y onto X get resolved from the repository.

Having to add dependencies into the dependencyManagement section would be OK
for 3rd party plugins, as I want to have this single point of control over
all
versions, but for my internal plugins, this would not be practical.

Besides, having those dependencies in the dependencyManagement after I
wipe out
my local repository gives me errors, because Maven/Tycho tries to download
them
from the group repository, where they don't exist either.

So, if there's a way to automatically derive the Maven coordinate from a
plugin
reference, that would solve the problem :-)

Thanks!
-Max
Igor Fedorenko
2008-10-23 05:26:52 UTC
Permalink
Post by m***@springdot.org
Hi Igor,
it works: I was able to satisfy the httpclient dependency with an artifact
coming from the local repository, following the example from the integration
test.
However: Now I have to "declare" all of my *own* plugins in the
dependencyManagement section in order to build a plugin X, install/deploy X
into the local/group repository, so that I can have the dependency of another
plugin Y onto X get resolved from the repository.
Having to add dependencies into the dependencyManagement section would be OK
for 3rd party plugins, as I want to have this single point of control over
all
versions, but for my internal plugins, this would not be practical.
Somehow I managed to convince myself this would be acceptable/reasonable.

First, this is more or less how dependencyManagement works for regular
maven projects, i.e. the same dependency is listed both in
<dependencyManagement/> and <dependencies/> sections.

Second, from what I saw, most of the projects have one or very few
top-level features that include all bundles/features produced by the
build. It is enough to only include such top-level feature into
<dependencyManagement/> and this will implicitly add everything included
in the feature to the build target platform.

And lastly, I assumed that target platform specification is going to be
useful by itself, especially because it is now possible to import
<dependencyManagement/> section from one pom to another using scope=import.

Does this make any sense or you still think this problem absolutely has
to be addressed for target platform management to be useful?
Post by m***@springdot.org
Besides, having those dependencies in the dependencyManagement after I
wipe out
my local repository gives me errors, because Maven/Tycho tries to download
them
from the group repository, where they don't exist either.
I am not sure I understand the problem. Where these dependencies are
supposed to come from if they are not available from anywhere? Or you
think the build should fail at a later stage, during OSGi dependency
resolution?
Post by m***@springdot.org
So, if there's a way to automatically derive the Maven coordinate from a
plugin
reference, that would solve the problem :-)
Very good question! I've been struggling with this problem for some time
but could not come up with a good generic solution. In my opinion,
usable solution would have to deal with at least this problems/usage
scenarios.

1. There are many more ways to express dependencies in OSGi compared to
maven, Import-Package, DynamicImport-Package, Import-Service and
Eclipse-GenericRequire cannot be expressed in maven terms, afaict.

2. Even for Require-Bundle, it is common to either not specify version
at all or specify open version range which makes it impossible to map
such dependencies to maven, or in fact resolve them without context of
specific target platform.

3. I am not sure how common this is, but at least for m2e, we want to be
able to build/test with several eclipse versions (i.e., e32, e33,
e34...). Again, this can only be achieved when target platform is
defined separately.

So I see only three possible solutions that address all three scenarios.
One is to specify build target platform explicitly. Another one is to
only support Require-Bundle with specific version. And the last one is
to push target platform management from the build to some external
entity, like repository manager (imagine if you could point your build
to a virtual repository that only contains needed artifacts and nothing
else, for example).

Do you see any other options?

--
Regards,
Igor
Eugene Kuleshov
2008-10-23 15:18:43 UTC
Permalink
I would like to clarify couple things.

Forst of all, in my understanding it is not necessary to declare
dependencies in <dependencyManagement> if they all came from the same
reactor build (i.e. when build is self-contained)

You can use features to aggregate several plugins into one unit, so
such features and plugins included in it would be installed to Maven
repository by a separate build

Igor, is that correct?

regards,
Eugene
Post by Igor Fedorenko
First, this is more or less how dependencyManagement works for regular
maven projects, i.e. the same dependency is listed both in
<dependencyManagement/> and <dependencies/> sections.
Second, from what I saw, most of the projects have one or very few
top-level features that include all bundles/features produced by the
build. It is enough to only include such top-level feature into
<dependencyManagement/> and this will implicitly add everything
included in the feature to the build target platform.
And lastly, I assumed that target platform specification is going to
be useful by itself, especially because it is now possible to import
<dependencyManagement/> section from one pom to another using
scope=import.
Does this make any sense or you still think this problem absolutely
has to be addressed for target platform management to be useful?
Igor Fedorenko
2008-10-23 15:45:27 UTC
Permalink
Post by Eugene Kuleshov
I would like to clarify couple things.
Forst of all, in my understanding it is not necessary to declare
dependencies in <dependencyManagement> if they all came from the same
reactor build (i.e. when build is self-contained)
Right, there is no need to list reactor build projects in
<dependencyManagement/>. I was talking about scenario when a team uses
repository to share binary artifacts among separate builds.
Post by Eugene Kuleshov
You can use features to aggregate several plugins into one unit, so
such features and plugins included in it would be installed to Maven
repository by a separate build
Not quite sure what you want to say or ask here... Have a look at tycho164
integration test [1]. There are two separate builds there. build01 builds
and installs couple of bundles and a feature into maven local repository.
build02 includes feature from the first build, and this implicitly adds
the bundle included into the feature.

[1] http://svn.sonatype.org/m2eclipse/tycho/trunk/tycho-its/projects/tycho164
Post by Eugene Kuleshov
Igor, is that correct?
regards,
Eugene
Post by Igor Fedorenko
First, this is more or less how dependencyManagement works for regular
maven projects, i.e. the same dependency is listed both in
<dependencyManagement/> and <dependencies/> sections.
Second, from what I saw, most of the projects have one or very few
top-level features that include all bundles/features produced by the
build. It is enough to only include such top-level feature into
<dependencyManagement/> and this will implicitly add everything
included in the feature to the build target platform.
And lastly, I assumed that target platform specification is going to
be useful by itself, especially because it is now possible to import
<dependencyManagement/> section from one pom to another using
scope=import.
Does this make any sense or you still think this problem absolutely
has to be addressed for target platform management to be useful?
m***@springdot.org
2008-10-23 20:52:16 UTC
Permalink
Post by Igor Fedorenko
Post by m***@springdot.org
Hi Igor,
it works: I was able to satisfy the httpclient dependency with an
artifact
coming from the local repository, following the example from the
integration
test.
However: Now I have to "declare" all of my *own* plugins in the
dependencyManagement section in order to build a plugin X,
install/deploy X
into the local/group repository, so that I can have the dependency of
another
plugin Y onto X get resolved from the repository.
Having to add dependencies into the dependencyManagement section would
be OK
for 3rd party plugins, as I want to have this single point of control
over
all
versions, but for my internal plugins, this would not be practical.
Somehow I managed to convince myself this would be acceptable/reasonable.
First, this is more or less how dependencyManagement works for regular
maven projects, i.e. the same dependency is listed both in
<dependencyManagement/> and <dependencies/> sections.
Second, from what I saw, most of the projects have one or very few
top-level features that include all bundles/features produced by the
build. It is enough to only include such top-level feature into
<dependencyManagement/> and this will implicitly add everything included
in the feature to the build target platform.
And lastly, I assumed that target platform specification is going to be
useful by itself, especially because it is now possible to import
<dependencyManagement/> section from one pom to another using scope=import.
Does this make any sense or you still think this problem absolutely has
to be addressed for target platform management to be useful?
Let me give you a little more context. I'm trying to introduce
Tycho/Maven/m2eclipse to a team of Eclipse plugin developers who are
currently
using a "monolithic" PDE build and are managing the dependencies manually,
i.e. they have to open all plugins as Eclipse projects to resolve all
dependencies to get to a compilable set of projects.

I want the ability to easily scale in terms of number of components and
developers. I.e. having each developer to build the full set of
components is
exactly what I'm trying to avoid. A developer should only need to open her
component from the SCM in Eclipse and all of that component's dependencies
are
getting resolved from the group repository which in turn gets populated by a
CI-driven build and a nightly build which does a full build from scratch.

Now, the important point is that I would like developers to be able to create
new components without having to "register" them in a dependencyManagement
section when another component wants to make of the new component. This is
highly desirable for me, though not an absolute requirement.
Post by Igor Fedorenko
Post by m***@springdot.org
Besides, having those dependencies in the dependencyManagement after I
wipe out
my local repository gives me errors, because Maven/Tycho tries to
download
them
from the group repository, where they don't exist either.
I am not sure I understand the problem. Where these dependencies are
supposed to come from if they are not available from anywhere? Or you
think the build should fail at a later stage, during OSGi dependency
resolution?
I have two plugins X and Y which do not have any dependencies itself, but are
used by other plugins. In order for the other plugins to find X and Y, I
declare two dependencies in the dependencyManagent section of my parent POM
(applicable to all of my plugins).

Now, I wipe out my local repository and try to re-install X. Because of the
dependency in the dependencyManagement section for Y, I get this error:

| Downloading:
http://repository.sonatype.org/content/groups/public/group/id/Y/1.0.0/Y-1.0.0.jar
| org.apache.maven.wagon.ResourceDoesNotExistException: Unable to locate
resource in repository
| at
org.apache.maven.wagon.providers.http.LightweightHttpWagon.fillInputData(LightweightHttpWagon.java:132)
| at
org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:116)
| at
org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88)
| at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61)
| at
org.apache.maven.artifact.manager.DefaultWagonManager.getRemoteFile(DefaultWagonManager.java:600)
| at
org.apache.maven.artifact.manager.DefaultWagonManager.getArtifact(DefaultWagonManager.java:479)
| at
org.apache.maven.artifact.manager.DefaultWagonManager.getArtifact(DefaultWagonManager.java:357)
| at
org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:167)
| at
org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:82)
| at
org.codehaus.tycho.osgitools.targetplatform.EclipseTargetPlatformFactory.resolveArtifact(EclipseTargetPlatformFactory.java:132)
| at
org.codehaus.tycho.osgitools.targetplatform.EclipseTargetPlatformFactory.resolvePlugin(EclipseTargetPlatformFactory.java:127)
| at
org.codehaus.tycho.osgitools.targetplatform.EclipseTargetPlatformFactory.createTargetPlatform(EclipseTargetPlatformFactory.java:67)
| at
org.codehaus.tycho.maven.EclipseMaven.resolveOSGiState(EclipseMaven.java:52)
| at
org.codehaus.tycho.maven.EclipseMaven.getProjects(EclipseMaven.java:35)
| at
org.apache.maven.DefaultMaven.createReactorManager(DefaultMaven.java:98)
| at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
| at
org.apache.maven.embedder.MavenEmbedder.execute(MavenEmbedder.java:899)
| at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:176)
| at org.apache.maven.cli.MavenCli.main(MavenCli.java:63)
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
| at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
| at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:597)
| at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
| at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
| at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:408)
| at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:351)
| Caused by: java.io.FileNotFoundException:
http://repository.sonatype.org/content/groups/public/group/id/Y/1.0.0/Y-1.0.0.jar
| at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1239)
| at
org.apache.maven.wagon.providers.http.LightweightHttpWagon.fillInputData(LightweightHttpWagon.java:115)
| ... 26 more

This happens also when I try to build all of my plugins all together using a
multi-module POM.
Post by Igor Fedorenko
Post by m***@springdot.org
So, if there's a way to automatically derive the Maven coordinate from a
plugin
reference, that would solve the problem :-)
Very good question! I've been struggling with this problem for some time
but could not come up with a good generic solution. In my opinion,
usable solution would have to deal with at least this problems/usage
scenarios.
1. There are many more ways to express dependencies in OSGi compared to
maven, Import-Package, DynamicImport-Package, Import-Service and
Eclipse-GenericRequire cannot be expressed in maven terms, afaict.
When mapping the referenced plugin name onto artifactId, the plugin version
onto the Maven version, the missing piece is the groupId. I'd enhance the
dependency element (in a dependencyManagement section) to allow for
wildcards,
then one could say something like

<dependency>
<groupId>org.myorg</groupId>
<artifactId>*</artifactId>
<version>*</version>
<type>eclipse-plugin</type>
</dependency>

Which would say that each actual, matching dependency would use
"org.myorg" as
groupId.

Maybe to not go into full regexps, one could use a slightly simpler approach,
allowing to omit certain elements from a dependency, having the same
semantics
as above:

<dependency>
<groupId>org.myorg</groupId>
<type>eclipse-plugin</type>
</dependency>

I could image that such a generalization of the dependencyManagement section
can be useful even outside of Tycho, whereever one wants to define dependency
details for many more concrete dependencies.
Post by Igor Fedorenko
2. Even for Require-Bundle, it is common to either not specify version
at all or specify open version range which makes it impossible to map
such dependencies to maven, or in fact resolve them without context of
specific target platform.
The suggested approach could also here be used to give the missing
information.
Post by Igor Fedorenko
3. I am not sure how common this is, but at least for m2e, we want to be
able to build/test with several eclipse versions (i.e., e32, e33,
e34...). Again, this can only be achieved when target platform is
defined separately.
Could this be achieved by having different POMs and the right one gets
selected
through a property?
Post by Igor Fedorenko
So I see only three possible solutions that address all three scenarios.
One is to specify build target platform explicitly. Another one is to
only support Require-Bundle with specific version. And the last one is
to push target platform management from the build to some external
entity, like repository manager (imagine if you could point your build
to a virtual repository that only contains needed artifacts and nothing
else, for example).
Do you see any other options?
I still think being able to map OSGi references onto Maven coordinates
would be
a powerful feature.

Thanks!
-Max
Igor Fedorenko
2008-10-27 15:11:53 UTC
Permalink
Post by m***@springdot.org
Post by Igor Fedorenko
Post by m***@springdot.org
Hi Igor,
it works: I was able to satisfy the httpclient dependency with an
artifact
coming from the local repository, following the example from the
integration
test.
However: Now I have to "declare" all of my *own* plugins in the
dependencyManagement section in order to build a plugin X,
install/deploy X
into the local/group repository, so that I can have the dependency of
another
plugin Y onto X get resolved from the repository.
Having to add dependencies into the dependencyManagement section would
be OK
for 3rd party plugins, as I want to have this single point of control
over
all
versions, but for my internal plugins, this would not be practical.
Somehow I managed to convince myself this would be acceptable/reasonable.
First, this is more or less how dependencyManagement works for regular
maven projects, i.e. the same dependency is listed both in
<dependencyManagement/> and <dependencies/> sections.
Second, from what I saw, most of the projects have one or very few
top-level features that include all bundles/features produced by the
build. It is enough to only include such top-level feature into
<dependencyManagement/> and this will implicitly add everything included
in the feature to the build target platform.
And lastly, I assumed that target platform specification is going to be
useful by itself, especially because it is now possible to import
<dependencyManagement/> section from one pom to another using scope=import.
Does this make any sense or you still think this problem absolutely has
to be addressed for target platform management to be useful?
Let me give you a little more context. I'm trying to introduce
Tycho/Maven/m2eclipse to a team of Eclipse plugin developers who are
currently
using a "monolithic" PDE build and are managing the dependencies manually,
i.e. they have to open all plugins as Eclipse projects to resolve all
dependencies to get to a compilable set of projects.
I want the ability to easily scale in terms of number of components and
developers. I.e. having each developer to build the full set of
components is
exactly what I'm trying to avoid. A developer should only need to open her
component from the SCM in Eclipse and all of that component's dependencies
are
getting resolved from the group repository which in turn gets populated by a
CI-driven build and a nightly build which does a full build from scratch.
Now, the important point is that I would like developers to be able to create
new components without having to "register" them in a dependencyManagement
section when another component wants to make of the new component. This is
highly desirable for me, though not an absolute requirement.
Yes, I understand your scenario and this is exactly the reason we're
adding target platform management logic to Tycho.

I also understand that it is desirable not to explicitly register
inter-component dependencies in dependencyManagement, but unfortunately
I do not see how to implement this without dropping features that I
believe are important. I hope managing dependencyManagement section is
not going to be too difficult for real projects and teams, but I will
certainly revisit this if I am mistaken.
Post by m***@springdot.org
Post by Igor Fedorenko
Post by m***@springdot.org
Besides, having those dependencies in the dependencyManagement after I
wipe out
my local repository gives me errors, because Maven/Tycho tries to
download
them
from the group repository, where they don't exist either.
I am not sure I understand the problem. Where these dependencies are
supposed to come from if they are not available from anywhere? Or you
think the build should fail at a later stage, during OSGi dependency
resolution?
I have two plugins X and Y which do not have any dependencies itself, but are
used by other plugins. In order for the other plugins to find X and Y, I
declare two dependencies in the dependencyManagent section of my parent POM
(applicable to all of my plugins).
Now, I wipe out my local repository and try to re-install X. Because of the
http://repository.sonatype.org/content/groups/public/group/id/Y/1.0.0/Y-1.0.0.jar
| org.apache.maven.wagon.ResourceDoesNotExistException: Unable to locate
resource in repository
| at
[stack trace was here]
Post by m***@springdot.org
This happens also when I try to build all of my plugins all together using a
multi-module POM.
This should be fixed now... well, almost ;-) Tycho will still print a
warning about target platform resolution problems, but will not fail the
build, assuming all project dependencies can be satisfied.
Post by m***@springdot.org
Post by Igor Fedorenko
Post by m***@springdot.org
So, if there's a way to automatically derive the Maven coordinate from a
plugin
reference, that would solve the problem :-)
Very good question! I've been struggling with this problem for some time
but could not come up with a good generic solution. In my opinion,
usable solution would have to deal with at least this problems/usage
scenarios.
1. There are many more ways to express dependencies in OSGi compared to
maven, Import-Package, DynamicImport-Package, Import-Service and
Eclipse-GenericRequire cannot be expressed in maven terms, afaict.
When mapping the referenced plugin name onto artifactId, the plugin version
onto the Maven version, the missing piece is the groupId. I'd enhance the
dependency element (in a dependencyManagement section) to allow for
wildcards,
then one could say something like
<dependency>
<groupId>org.myorg</groupId>
<artifactId>*</artifactId>
<version>*</version>
<type>eclipse-plugin</type>
</dependency>
Which would say that each actual, matching dependency would use
"org.myorg" as
groupId.
Maybe to not go into full regexps, one could use a slightly simpler approach,
allowing to omit certain elements from a dependency, having the same
semantics
<dependency>
<groupId>org.myorg</groupId>
<type>eclipse-plugin</type>
</dependency>
I could image that such a generalization of the dependencyManagement section
can be useful even outside of Tycho, whereever one wants to define dependency
details for many more concrete dependencies.
Post by Igor Fedorenko
2. Even for Require-Bundle, it is common to either not specify version
at all or specify open version range which makes it impossible to map
such dependencies to maven, or in fact resolve them without context of
specific target platform.
The suggested approach could also here be used to give the missing
information.
Post by Igor Fedorenko
3. I am not sure how common this is, but at least for m2e, we want to be
able to build/test with several eclipse versions (i.e., e32, e33,
e34...). Again, this can only be achieved when target platform is
defined separately.
Could this be achieved by having different POMs and the right one gets
selected
through a property?
Post by Igor Fedorenko
So I see only three possible solutions that address all three scenarios.
One is to specify build target platform explicitly. Another one is to
only support Require-Bundle with specific version. And the last one is
to push target platform management from the build to some external
entity, like repository manager (imagine if you could point your build
to a virtual repository that only contains needed artifacts and nothing
else, for example).
Do you see any other options?
I still think being able to map OSGi references onto Maven coordinates
would be
a powerful feature.
To make sure I understand your suggestions.

Currently, there are two resolution phases during tycho build. First,
tycho resolves build target platform using maven coordinates from maven
artifact repositories (feature dependnecies are transitive, bundle
dependencies are not transitive). The result of target platform
resolution is then fed to (real) OSGi resolver, that deals with various
Eclipse/OSGi dependencies, like Import-Package or Eclipse-GenericRequire.

Do you think this is a reasonable approach?

You suggestions appear to apply to target platform resolution phase only
(specifically, Eclipse/OSGi metadata is not checked at all). Is this
correct?


--
Regards,
Igor
m***@springdot.org
2008-10-24 00:50:52 UTC
Permalink
Hi Igor,

here's another suggestion for providing the missing groupId in case of
packageing is eclipse-plugin:

Simply use the POM's groupId for any dependency of type eclipse-plugin with
missing groupId.

Since the dependencies do come from the MANIFEST.MF, they don't have a
groupId
and are implicitly of type eclipse-plugin.

What do you think?
-Max
Igor Fedorenko
2008-10-27 15:28:23 UTC
Permalink
Post by m***@springdot.org
Hi Igor,
here's another suggestion for providing the missing groupId in case of
Simply use the POM's groupId for any dependency of type eclipse-plugin with
missing groupId.
Since the dependencies do come from the MANIFEST.MF, they don't have a
groupId
and are implicitly of type eclipse-plugin.
What do you think?
-Max
Import-Package cannot be mapped into maven coordinates. This is why we
have two phase dependency resolution. As I explained in another reply,
tycho first resolves build target platform to determine set of features
and bundles to use by "real" OSGi resolver.

It is theoretically possible to instrument maven repository with
Eclipse/OSGi metadata, effectively turning maven repository into p2
repository. We may add this functionality to our Nexus repository
manager some time in the future, but I can't tell when and in what form
this is going to happen. You may want to create enhancement request in
Nexus JIRA [1] if you are interested in this functionality.

[1] http://issues.sonatype.org/browse/NEXUS

--
Regards,
Igor
m***@springdot.org
2008-10-28 17:55:10 UTC
Permalink
...
Post by Igor Fedorenko
I also understand that it is desirable not to explicitly register
inter-component dependencies in dependencyManagement, but unfortunately
I do not see how to implement this without dropping features that I
believe are important. I hope managing dependencyManagement section is
not going to be too difficult for real projects and teams, but I will
certainly revisit this if I am mistaken.
As I said earlier, having to use dependencyManagement for inter-component
dependencies is OK for me, though somewhat inconvenient.

...
Post by Igor Fedorenko
Post by m***@springdot.org
Now, I wipe out my local repository and try to re-install X. Because
of the
http://repository.sonatype.org/content/groups/public/group/id/Y/1.0.0/Y-1.0.0.jar
Post by Igor Fedorenko
Post by m***@springdot.org
| org.apache.maven.wagon.ResourceDoesNotExistException: Unable to
locate resource in repository
Post by Igor Fedorenko
Post by m***@springdot.org
| at
[stack trace was here]
Post by m***@springdot.org
This happens also when I try to build all of my plugins all together
using a
Post by Igor Fedorenko
Post by m***@springdot.org
multi-module POM.
This should be fixed now... well, almost ;-) Tycho will still print a
warning about target platform resolution problems, but will not fail the
build, assuming all project dependencies can be satisfied.
This change does work for me. Just tried it out. Thanks!
Post by Igor Fedorenko
Post by m***@springdot.org
Post by Igor Fedorenko
Post by m***@springdot.org
So, if there's a way to automatically derive the Maven coordinate
from a
plugin
reference, that would solve the problem :-)
Very good question! I've been struggling with this problem for some time
but could not come up with a good generic solution. In my opinion,
usable solution would have to deal with at least this problems/usage
scenarios.
1. There are many more ways to express dependencies in OSGi compared to
maven, Import-Package, DynamicImport-Package, Import-Service and
Eclipse-GenericRequire cannot be expressed in maven terms, afaict.
When mapping the referenced plugin name onto artifactId, the plugin
version
onto the Maven version, the missing piece is the groupId. I'd enhance
the
dependency element (in a dependencyManagement section) to allow for
wildcards,
then one could say something like
<dependency>
<groupId>org.myorg</groupId>
<artifactId>*</artifactId>
<version>*</version>
<type>eclipse-plugin</type>
</dependency>
Which would say that each actual, matching dependency would use
"org.myorg" as
groupId.
Maybe to not go into full regexps, one could use a slightly simpler
approach,
allowing to omit certain elements from a dependency, having the same
semantics
<dependency>
<groupId>org.myorg</groupId>
<type>eclipse-plugin</type>
</dependency>
I could image that such a generalization of the dependencyManagement
section
can be useful even outside of Tycho, whereever one wants to define
dependency
details for many more concrete dependencies.
Post by Igor Fedorenko
2. Even for Require-Bundle, it is common to either not specify version
at all or specify open version range which makes it impossible to map
such dependencies to maven, or in fact resolve them without context of
specific target platform.
The suggested approach could also here be used to give the missing
information.
Post by Igor Fedorenko
3. I am not sure how common this is, but at least for m2e, we want to be
able to build/test with several eclipse versions (i.e., e32, e33,
e34...). Again, this can only be achieved when target platform is
defined separately.
Could this be achieved by having different POMs and the right one gets
selected
through a property?
Post by Igor Fedorenko
So I see only three possible solutions that address all three scenarios.
One is to specify build target platform explicitly. Another one is to
only support Require-Bundle with specific version. And the last one is
to push target platform management from the build to some external
entity, like repository manager (imagine if you could point your build
to a virtual repository that only contains needed artifacts and nothing
else, for example).
Do you see any other options?
I still think being able to map OSGi references onto Maven coordinates
would be
a powerful feature.
To make sure I understand your suggestions.
Currently, there are two resolution phases during tycho build. First,
tycho resolves build target platform using maven coordinates from maven
artifact repositories (feature dependnecies are transitive, bundle
dependencies are not transitive). The result of target platform
resolution is then fed to (real) OSGi resolver, that deals with various
Eclipse/OSGi dependencies, like Import-Package or Eclipse-GenericRequire.
Do you think this is a reasonable approach?
I have to admit that I'm not absolutely clear at this point in time, how I
will
be using these mechanisms.

On one hand, I like to keep my target platform as minimal as possible and be
able to have even 3rd party plugins in my Maven repository. For sure I
want my
inhouse plugins to reside in the Maven repository, while operating in
MANIFEST-first mode at least in the beginning to confuse my developers as
little as possible.

On the other hand, I can imagine to later switch to a POM-first mode where I
anticipate dependency resolution to be clearer.

Anyhow. The way it works right now (i.e. the 20081027.200229 snapshot) is OK
for me to start with.
Post by Igor Fedorenko
You suggestions appear to apply to target platform resolution phase only
(specifically, Eclipse/OSGi metadata is not checked at all). Is this
correct?
Not really. I respond on this topic in a separate thread.

Regards,
-Max

Continue reading on narkive:
Loading...