[MNG-2205] "provided" scope dependencies must be transitive

David Boden opened MNG-2205 and commented

A provided scope dependency can also be thought of as "compile-only".

Project A requires Sybase JConnect on the runtime classpath. Project A declares a "provided" dependency on Sybase JConnect.
Project B depends upon Project A. Project B declares a "compile" dependency on Project A.
Project C depends upon Project B. Project C declares a "compile" dependency on Project B.

C
| - compile dependency
B
| - compile dependency
A
| - provided dependency
Sybase JConnect

So, does Project C transitively depend on Sybase JConnect. Yes, of course! The "provided" dependency needs to be transitive.

Ultimately, when Project C gets deployed, Sybase JConnect needs to be somewhere on the runtime classpath in order for the application to function. It's valid for Project C to assume that Sybase JConnect is available and use JDBC all over the Project C code. Project C is safe to do this because it can happily deduce that Sybase JConnect will be there in the runtime environment because Project A NEEDS IT.

I've got Use Cases all over my aggregated build which make it absolutely critical and common sense that provided scope dependencies are transitive. For the (very rare) odd case where you don't want to inherit provided dependencies, you can <exclude/> them.

Attachments:

transitivetest.zip (1.61 kB)

Issue Links:

MNG-4746 A provided scoped transitive dependency of a provided scoped direct dependency will be omitted.
("is duplicated by")
MNG-1978 "Provided" scope transitive dependencies required + exclude dependencies for runtime scope only
("is duplicated by")
MNG-5048 Optionally include "provided" transitive dependencies on classpath
("is duplicated by")
MNG-2611 test scope should inherit provided scope
("is duplicated by")
MENFORCER-385 Enforce that transitive provided dependencies are contained in the runtime Maven classpath
("causes")
MNG-3890 Transitive dependencies override explicitly set scope.
MNG-2317 Provided scope dependencies are based in target app server, current approach doesn't work very well
MNG-2589 Inherit transitive dependencies with runtime scope

48 votes, 62 watchers

Andrew Williams commented

Your point may be correct, but your use case is wrong.

if C wants to use Sybase JConnect then it must declare this as a dependency. A could at any time change it's dependencies and "break" this assumption of C's.
It is wrong to use a dependency that you do not declare.

David Boden commented

I agree with you in some sense. Sybase JConnect isn't a great example.

However, your "any time change its dependencies" is not at all accurate. A project depends on a released version of, say Apache DBCP. Once it's released, its dependencies are set in stone. The whole point of transitive dependencies is that you benefit from not having to redundantly redeclare you dependencies at each level. When you're upgrading your dependencies, you have time to make a good decision on whether the new dependency tree meets your needs.

Is the logical conclusion of your point that we shouldn't bother with transitive dependencies because we always need the safety of declaring them in each project?

Joerg Schaible commented

No, you're using "provided" in a wrong manner. A provided means that the artifact is available in the target environment i.e. if you build an enterprise application for a J2EE server you can safely assume that javamail or activation is available, since it is part of the JEE stack. In your case you are introducing a runtime dependency and you should use therefore scope "runtime".

David Boden commented

That's why JConnect is a terrible example.

A better example of this sort of dependency is XML Beans. XML Beans will not work based on a "runtime" dependency because your code will require it on the compilation classpath. If all your products depend on XML Beans, you want to be able to declare it as a base dependency and have all your modules inherit that dependency.

Paul Gier commented

If you make provided transitive, then what is the difference between the "provided" and "compile" scopes? Doesn't the compile scope already give the transitive behaviour that you're looking for?

From your original example above, I don't see where the problem is. If project C is using JDBC with a JConnect driver, and JConnect is being provided on the runtime classpath, by the app server or whatever, then I would think your app would run fine.

The XML beans example sounds like it should be compile scope. It is required on the compile classpath, and should be inherited transitively. Why not use the default compile scope for this?

Ramon Grunder commented

The difference from PROVIDED to the COMPILE dependency would be, that in the PROVIDED scope, the dependent artifact is not packaged in the target artifact (since its provided by the runtime environment), whereas the COMPILE depedency is alwas packaged into the target artifact which is not always what you want. think of an example like a jdbc-driver, which is installed in your tomcat or jboss; you do not want to package this one in your WAR or EAR artifact too, since its PROVIDED....

Paul Gier commented

Sorry I think I'm still missing something. So then what is wrong with PROVIDED the way it currently functions? The JConnect dependency would not be included in the war because it is provided by the container, and project C would be able to use the JConnect driver at runtime because it is provided in the classpath by the app server, right?

David Boden commented

The problem is that project C is not able to use JConnect classes directly. It's not on project C's compile classpath.

Why not? Why not indeed. It should be! That's what this bug is aiming to fix.

JConnect is a bad example, because you typically don't access JConnect classes directly, you're more likely to use the JDBC API and hide the implementation details.

Eclipse RCP is a better example (it's a real world example for me). Project A is an Eclipse RCP component. It's used by Project B which is in turn used by Project C. Now, because of the way we package the application, the Eclipse RCP (big) jar files are downloaded separately and are provided for the distribution of my application. The problem is that why should I explicitly declare a dependency from Project C to Eclipse RCP? It should be inherited transitively. The default behaviour should be that I can use the Eclipse RCP API in Project C.

syvalta commented

I think distinguishing packaging and compiling would make sense here. I use compile scope as an example:

for direct dependencies it's clear that they are included in you compile time classpath and also included when packaged to a war for example.
for transitive dependencies, it's clear they should be packaged but the question is if they should be included in the compile time classpath.

Ok, let's have an example:

1. Current behavior:
A -> B (compile) -> C (compile)
A -> B (compile) -> D (provided)

C: included in A's compile classpath, included in packaging
D: not included in A's compile classpath, not packaged

2. Requested in this issue:
A -> B (compile) -> C (compile)
A -> B (compile) -> D (provided)

C: included in A's compile classpath, included in packaging
D: included in A's compile classpath, not packaged

3. Preferable solution in my opinion:
A -> B (compile) -> C (compile)
A -> B (compile) -> D (provided)

C: not included in A's compile classpath, included in packaging
D: not included in A's compile classpath, not packaged

If I remember correctly there were some talk in the mailing list that maven 2.1 would implement option 3, but I might remember wrong.

edit: this seem to be bit more complicated than that: how about classpath for compiling and running tests etc.
As a quick thought, classpath for compiling tests could be just the direct dependencies and for running tests same as for packaging (but how about transitive test dependencies)?

Alexandre Sauve commented

I think that 'provided' transitive dependencies are important! The case that comes to my mind is OSGi and Eclipse development. You need a certain class/package in order to compile your code; however that same package will be available on the runtime environment as a bundle. You don't want to start adding this jar as an embed library into your bundle! You want to embed only those that are runtime/compile dependencies.

However since 'provided' is NOT transitive you would have to list all the dependencies this bundle you depend on has in your POM! Not exactly a step forward from Maven 1. You would like to pull in these bundle dependencies just for the need of compilation (and actually you would what them in the manifest file too).

So in the end for OSGi and Eclipse transitive provided dependencies would be a really nice thing to have!

Maarten Billemont commented

What I'd like to do is create a POM-type artifact which is basically just a grouping of dependencies used by other artifacts.

<dependencies>
    <!-- jboss/server/default/lib -->
    <dependency>
        <groupId>commons-codec</groupId>
        <artifactId>commons-codec</artifactId>
        <version>1.3</version>
        <scope>provided</scope>
    </dependency>
    ....

Then I would add this artifact as a provided dependency to all artifacts.
The desired result would be an inclusion of all dependencies from this container artifact in my other artifacts, marked as provided.

You may wonder why I'm not just marking them as compile-scoped in the container artifact; here's why:
I have an artifact that creates a ZIP file containing the JBoss AS. This artifact basically just downloads JBoss, extracts it, messes around with its configuration, and packages it as a ZIP file. Additionally, all artifacts from the container artifact above that are not marked as provided (because they are not already provided by JBoss) are added to the jboss/server/default/lib/ directory just before zipping it up. That means I can specify two types of provided dependencies in the container POM above:

Dependencies provided by the JBoss AS default configuration (<scope>provided</scope> in the container POM).
Dependencies that I add to my JBoss' server/default/lib directory (<scope>compile</scope> in the container POM).

Every artifact that depends on the container POM and sets that dependency to a <scope>provided</scope> should have all dependencies in the container pom, be they provided or compile, added to the artifact as provided. This is exactly as described by the documentation here:
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
With one single exception: I cannot yet mark provided dependencies in the container POM as transitive.

The table on the link above says "provided" in the intersection of "provided" on the left side and "provided" on the top. Reading the text above the table you will see this implies that:
If a dependency is set to [provided], transitive dependencies of that dependency with the scope [provided] will result in a dependency in the main project with the scope [provided]. If no scope is listed, it means the dependency will be omitted.

I would've thought that the author would've just put a "-" at the intersection of provided with provided seeing as there currently is no such thing as transitive provided dependencies. Or is there?

Roman Kalukiewicz commented

There is another very important reason why provided dependencies should be transitive. This reason is tests.

If you have some class that needs (lets say) servlet-api and servlet-api is provided, then you cannot use this class in another project, because when you write test, you are not able to instantiate your original class because it imports javax.servlet.* that is not available.

You also cannot extend any created servlet class in any different project, because it will not compile if Serlvet interface is not available on classpath.
This is similar to the note you have just under the table at
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope

I believe that the most typical situation is to scope servlet-api as provided and it shows, that it doesn't work as it should.

Timothy Twelves commented

Hi,

In the context of a "war" a dependency that is "provided" can mean 1 of two things:

It is provided within the application ear
OR
It is provided by the environment.

The solution for (1) is easy because we could have "provided" being transitive and have the ear-plugin package the "provided" dependencies.

The solution for (2) is tricky because (as in the servlets example) we need it for compilation and we do NOT want it packaged by the ear-plugin since it is provided by the appserver or jvm path. I considered "runtime" but this would make it unavailable during compilation and with the servlet -api example we need those dependencies during compilation. "optional" for (2) works fine however the reasons why it works are not obvious to the layman.

The solution is to introduce a new state called "environmental" and duplicate functionality around "optional".
Make a "provided" transitive and have all class-packaging plugins (maven-ear-plugin, maven-war-plugin) support the correct packaging of "provided" dependencies.

From my understanding of the maven code base (i've been digging) this is relatively do-able and the semantic changes are not going to be too bad considering the way people work around the problem of creating wars. I have only one problem which is the need to rewrite the manifest within jar/war/etc to support bundled libraries with different jar locations and signed jar/wars but again i believe this can be managed in the context of how people work around skinny jars at the moment.

-Tim

Timothy Twelves commented

Just to clarify around "optional" i am referring to a "compile" dependencies that are marked optional. This means that "environmental" dependencies will be there for compilation, tests, execution within maven but excluded from packaging.

Alternatively you could leave "provided" the way it is and add a new dependency type called "application" that is intended to be provided and packaged by the deliverable application component (war, ear, etc). In the context of a war an dependency marked "application" must be packaged by the ear.

-Tim

Denis Kramer commented

I believe the right behavior is postulated in the Maven Scope Matrix (http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html).

There a combination of provided/provided (for dependency and a dependency in the dependency) leads to a provided scope for the secondary dependency (as it should be).

I'm not entirely sure if a xxx/provided dependency should be transitive. Thinking about OSGi environments, this might lead to version issues.

Anyway, maven (as of 2.2.1) does not even handle the provided/provided case as given in the Maven Scope Matrix.

Michael Osipov commented

As a side note: if you introduce model version 5.0.0, we could easily fix the provided scope and leave it in 4.0.0 as-is.

Jason van Zyl commented

I still think this would be confusing to have one behaviour with the 4.0.0 version of the POM and another behaviour with the 5.0.0 version of the model. I honestly think this would be confusing and after several years with this behaviour making a new scope name for the new behaviour is the way to go.

Tibor Digana commented

I agree with Jason that the model version should not change.
I agree with other commiters saying that XML schema should not be changed.
Therefore a new scope should be introduced and we can call it something like "nonrunnable".
Thus the behavior of scope "provided" should not change in spec.

Paul Benedict commented

If this conversation is really about controlling the transitive nature of a dependency, I think we should introduce a <transitive> boolean tag. That won't need a version update since it's a new tag (no one has used it before). Then instead of inventing new scopes, we can just let programmers determine it for themselves based on their own needs.

Tibor Digana commented

@Paul
We shouldn't introduce a new XML element like <transitive/>.
The worst and still possible we can do is to introduce new scope.

Candid Dauth commented

The documentation even says about transitive compile dependencies: “it is intended that this should be runtime scope instead, so that all compile dependencies must be explicitly listed - however, there is the case where the library you depend on extends a class from another library, forcing you to have available at compile time. For this reason, compile time dependencies remain as compile scope even when they are transitive.”

For the same reason, transitive provided dependencies should remain provided. Imagine project A has a compile/provided dependency on project B, which has a provided dependency on project C. Project A has a class that extends a class from project B. The class from project B extends a class from project C. In order to compile the class from project A, the class from project C needs to be available (at compile time!).

Christian Schulte commented

Reading the comments on this issue I really fail to see the issue at hand. The only reason people are asking for the 'provided' scope to be transitive is because they want such artifacts to not get packaged in the final distribution artifact automatically. That's the only reason people are not using 'compile' scope. You can use 'dependencyManagement' in the final distribution artifact and manage scopes to 'provided' there or you can setup the plugin performing the packaging to exclude artifacts or to only include specific artifacts. Another option is to just add all 'provided' dependencies to the 'dependencies' section of a common parent so they are inherited to all modules with scope 'provided'. Introducing a new scope behaving exactly the same way the 'compile' scope behaves only to have a way to automatically exclude artifacts from getting packaged is even more effort then just using 'compile' (default) and configuring the packaging accordingly.

Christian Schulte commented

This can easily be solved in multiple ways. You may also want to consider the optional flag.

compile+optional  == provided
runtime+optional  == runtime-provided

Chris Povirk commented

I think that most of the reasons given here are bad reasons. Still, I think there are good reasons to want a new "transitive compile-time" scope.

Primarily, it's about annotations: A project may want to depend on some annotations at compile time but might not use them at runtime. In the simplest case, the annotations may have only source retention, so they're provably unusable at runtime.

Yet they may still be needed when compiling transitive users: If you build against a library that uses annotations but the annotations aren't present, javac sometimes issues warnings. And if you're the kind of person who uses -Xlint:all -Werror, your build breaks. (And I don't think it's possible to suppress that kind of warning without also suppressing others.) For example:

Cannot find annotation method 'value()' in type 'com.google.errorprone.annotations.CompatibleWith': class file for com.google.errorprone.annotations.CompatibleWith not found

This means that optional and provided are both insufficient. As a result, we include the dependencies at runtime, too. Users then have to manually exclude them to avoid the extra classes.

Another use case: Static analysis tools (e.g., Error Prone) may expect transitive dependencies to be present at compile-time. If the dependencies aren't there, the tool may crash. In the case of Error Prone, the tool owners have been willing to work around these problems, but the cost is that the tool may have to skip files and thus miss errors.

The background here is that we've had exactly these problems with Guava. See #2721, #2824, and this discussion on our latest release.

Chris Povirk commented

I just learned that JUnit stopped using provided/optional scope for their annotation dependencies for similar reasons: Users were reporting problems because the annotations weren't visible to downstream projects (e.g., 1, 2).

Thomas Broyer commented

Fwiw, Gradle is adding a compileOnlyApi configuration for cases like these: it's only used at compile time, and it's transitive. Not even used at test time in the same project (used to compile tests, but not to run them).

When publishing to a Maven repository, those dependencies appear with compile scope in the published POM.

https://docs.gradle.org/6.7-rc-1/release-notes.html#compile-only-api

Navigation Menu

Repository navigation

[MNG-2205] "provided" scope dependencies must be transitive #4265

12 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[MNG-2205] "provided" scope dependencies must be transitive #4265

Description

Activity

jira-importer commented on Mar 12, 2007

jira-importer commented on Mar 12, 2007

jira-importer commented on Mar 13, 2007

jira-importer commented on Mar 13, 2007

jira-importer commented on Jul 11, 2007

jira-importer commented on Jul 11, 2007

jira-importer commented on Jul 11, 2007

jira-importer commented on Jul 11, 2007

jira-importer commented on Apr 17, 2008

jira-importer commented on Jul 8, 2008

jira-importer commented on Aug 7, 2008

jira-importer commented on Aug 27, 2008

jira-importer commented on Jan 22, 2009

jira-importer commented on Jan 22, 2009

jira-importer commented on Jan 7, 2010

12 remaining items

jira-importer commented on Apr 27, 2014

jira-importer commented on May 7, 2014

jira-importer commented on May 14, 2014

jira-importer commented on May 14, 2014

jira-importer commented on May 16, 2014

jira-importer commented on Oct 17, 2015

jira-importer commented on Feb 8, 2016

jira-importer commented on Feb 17, 2016

jira-importer commented on Nov 27, 2017

jira-importer commented on Jan 16, 2018

jira-importer commented on Sep 24, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions