SpringSource Application Platform Manifest Headers

Glyn Normington

The SpringSource Application Platform is constructed from OSGi bundles and supports applications which are also constructed from OSGi bundles. The Platform supports the standard features of OSGi, but it also supports some additional manifest headers. Several people have asked Why did SpringSource add proprietary headers? and What are the semantics of the new headers?, so this post explains the background motivation and the semantics of Import-Library and Import-Bundle.

Standard OSGi Bundle Support

The Platform is built on the OSGi R4.1 standard, or JSR 291 if you prefer, and uses Equinox as its OSGi implementation. The result is that you can develop standard OSGi bundles using the Platform's tooling and deploy those bundles on the Platform, as a number of users have been doing since the Platform's launch.

So OSGi savvy developers can use the Platform as a standard OSGi container and benefit from Platform features such as:

  • the ability to deploy bundles using the Admin Console or by dropping bundles in the Platform's pickup directory,
  • diagnostics such as resolution failure diagnosis, application specific trace, and automatic deadlock detection,
  • strong integration with Spring and Spring Dynamic Modules, for developers who want to use these frameworks, and
  • automatic provisioning of dependencies from a repository.

However, the Platform also aims to make it easy for enterprise application developers with little or no prior exposure to OSGi to benefit from OSGi, which places some extra requirements on the Platform.

Additional Requirements of Enterprise Applications

As Sam's recent blog on the Platform's deployment options explains, you can deploy existing monolithic WAR files on the Platform with no need to understand OSGi – the Platform takes care of everything for you. But to benefit from shared libraries, shared services and, ultimately, PAR file scoping, it is necessary to break monolithic WAR files into OSGi bundles. How hard can that be?

Well, some steps in the process are relatively easy, especially if good software engineering practices have been followed and the code has been organised into service, domain, and infrastructure components. These components can be converted into bundles and the dependencies between them expressed using standard OSGi Import-Package and Export-Package headers in META-INF/MANIFEST.MF.

A more difficult step is expressing dependencies on enterprise frameworks such as Spring and Hibernate. It is entirely possible to express these dependencies using standard OSGi Import-Package and Require-Bundle headers, and this is exactly what you should do if your aim is to create OSGi bundles which will run in other OSGi containers, but this approach has some hidden costs.

Firstly, the developer has to decide precisely which packages comprise a given framework. It isn't sufficient merely to import the packages the application code uses, as several enterprise frameworks weave further dependencies into the bytecode of the application when the application is loaded. The developer has to discover, probably by trial and error, which additional implementation packages to import to ensure correct behaviour of the woven application.

Then there is the chore of migrating from one version of a framework to the next where the precise set of packages comprising the framework has changed. The additional packages required for weaving are typically not defined by a public contract and so are subject to change.

Additionally, the resultant package imports don't properly capture the design intent, which makes maintaining or extending the application more difficult in the future.

We really don't want to impose these burdens on our users, so we created some additional SpringSource Application Platform specific manifest headers, Import-Library and Import-Bundle, as convenient ways of expressing dependencies on enterprise frameworks. As you'll see below, these headers are really just syntactic sugar which are expressed in terms of standard OSGi package imports.

Import-Library

The basic syntax is similar to that of other manifest headers:

    Import-Library: <librarySymbolicName>;version=<versionRange>

where <librarySymbolicName> is the symbolic name of the library and <versionRange> is a range of acceptable versions of the library using OSGi version range notation. A library definition specifies the library's symbolic name and version and these together uniquely identify the library to the Platform.

If you are unfamiliar with OSGi version range notation, by far the most commonly used forms are minimum version ranges such as 2, meaning version 2 or later, and half-open ranges such as [2.2.1,2.2.2), meaning any version between 2.2.1 inclusive and 2.2.2 exclusive. If version=<versionRange> is omitted, together with the semicolon delimiter of course, then the default range includes all versions.

For each library import, the Platform selects the library with the given symbolic name and the highest version in the given version range available in the Platform's repository. The Platform then replaces the library import with a set of package imports which match all the packages exported by the bundles of the library. The Platform detects the situation where a bundle imports two or more libraries which export a common package, issues an appropriate log message, and fails to install the importing bundle.

So, for example, the following header imports some version of the Spring Framework library between 2.5.4 inclusive and 2.5.5 exclusive:

    Import-Library: org.springframework.spring;version="[2.5.4,2.5.5)"

Optional Library Import

You can indicate that a library import is optional using the following syntax. Note the special separator := which indicates a directive that modifies the semantics of the manifest header, as opposed to the separator = which indicates a matching attribute, like version.

    Import-Library: <librarySymbolicName>;version=<versionRange>;resolution:=optional

If resolution is not specified, or is specified as mandatory, the bundle containing the import library header will fail to install if there is no library with the given symbolic name and a version in the given range. But if resolution:=optional is specified, the library import will be ignored if no suitable library is available.

So, for example, the following header imports some version of the Spring Framework library from 2.5 onwards, but is ignored if no suitable library is available:

    Import-Library: org.springframework.spring;version="2.5";resolution:=optional

Importing More than One Library

If you need to import more than one library, then specify a comma-separated list of library imports in a single Import-Library manifest header as in the following example:

    Import-Library: org.foo.p;version="[1,2)",org.bar.q;version="[2,3)"

Import-Bundle

Import-Bundle is a further convenience for cases where a library would consist of only a single bundle and a library definition is inconvenient to create. The syntax is very similar to that of Import-Library except that it refers to a bundle's symbolic name and version instead of those of a library.

As you would expect, for each imported bundle, the Platform selects the bundle with the given symbolic name and the highest version in the given version range available in the Platform's repository. The Platform then replaces the bundle import with a set of package imports matching the packages exported by the bundle.

So, for example, the following header imports the Hibernate Object-Relational Mapper bundle:

    Import-Bundle: com.springsource.org.hibernate;version="[3.2.6,3.2.7)"

Why not Overload Require-Bundle?

If you are familiar with OSGi, you may be asking yourself why we didn't overload Require-Bundle instead of introducing Import-Bundle.

Well, we wanted Require-Bundle to retain its standard semantics, including the ability to marry together pieces of a split package. But we wanted Import-Library and Import-Bundle to have the same underlying semantics as Import-Package which avoid the complexities of split packages.

We also anticipate that, as the Platform evolves over time, we'll need to add further directives to Import-Library and Import-Bundle which would not be appropriate to add to Require-Bundle.

What Next?

The Platform beta program is in progress and we'll be listening to all feedback on Platform features, including the new manifest headers.

For users who want to take advantage of the Platform's headers, but who need to produce bundles which will run on other OSGi containers, we plan to produce a tool which will replace the Import-Library and Import-Bundle syntactic sugar with equivalent standard package imports.

We'll also be discussing the new manifest headers with our colleagues in the OSGi Alliance to determine if there are general requirements which would be appropriate for OSGi to address in the future.

Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • TwitThis
 

12 responses


  1. Hi Glynn,

    Thanks for the clarification, I can see the motivation behind these changes but as an architect already working with OSGi in the enterprise and involved with the OSGi enterprise expert group I still wonder why they're necessary.

    Given these headings are purely syntactic sugar, surely they don't need to be included in the bundle manifest at all. Instead it seems simpler to always resolve these links at build time so you always produce standard bundles? It strikes me that the tool to resolve these links should have been the starting point vs the end point of this problem.

    Of course resolving these statements at build time limits the ability for SpringSource to add further directives outside of the functionality provided by Import-Package and Requires-Bundle. But then if you continue down this path, with further embedded non-standard directives, aren't you then making these headers more than just syntactic sugar? As it would not necessarily be possible to turn them back into standard osgi bundles to be used on other platforms.

    Finally ignoring the issue of adding manifest headers outside of the spec, why do this at runtime at all? Using the build level conversion the problems of multiple imports from different libraries with different exports can readily be fixed when the developer builds the code vs getting nasty surprises when he deploys his/her work in a live environment.

    Just all seems backwards to me…but often the case with complex graph problems ;)

    Regards,

    Dave
    ——————————
    http://www.paremus.com
    http://newton.codecauldron.org
    http://sigil.codecauldon.org
    ——————————


  2. Hi Dave

    [quote post="331"]Thanks for the clarification, I can see the motivation behind these changes but as an architect already working with OSGi in the enterprise and involved with the OSGi enterprise expert group I still wonder why they're necessary.[/quote]

    It seems we agree on the problem, but not necessarily on the solution, if I've understood you correctly.

    [quote post="331"]Given these headings are purely syntactic sugar, surely they don't need to be included in the bundle manifest at all. Instead it seems simpler to always resolve these links at build time so you always produce standard bundles? It strikes me that the tool to resolve these links should have been the starting point vs the end point of this problem.[/quote]

    I think it would be messy for the user to have to deal with source bundles and built bundles, so the Platform performs the transformation during deployment before installing the bundles into OSGi.

    [quote post="331"]Of course resolving these statements at build time limits the ability for SpringSource to add further directives outside of the functionality provided by Import-Package and Requires-Bundle. But then if you continue down this path, with further embedded non-standard directives, aren't you then making these headers more than just syntactic sugar? As it would not necessarily be possible to turn them back into standard osgi bundles to be used on other platforms.[/quote]

    Let's see. ;-)

    [quote post="331"]Finally ignoring the issue of adding manifest headers outside of the spec, why do this at runtime at all? Using the build level conversion the problems of multiple imports from different libraries with different exports can readily be fixed when the developer builds the code vs getting nasty surprises when he deploys his/her work in a live environment.[/quote]

    I agree the tooling should give early warning of such problems. A further advantage of processing the headers at runtime is that it allows the Platform to provide diagnostics and other feedback in terms of the developer's inputs.

    [quote post="331"]Just all seems backwards to me…but often the case with complex graph problems ;) [/quote]

    I hope further discussion will make things clearer.

    Glyn


  3. I'm not convinced of the difference between Require-Bundle and Import-Bundle. I especially don't see the need of Import-Library, either.

    You could easily have a single bundle which represented hibernate-as-a-whole and solely existed to provide wiring for importing packages (exported to the client). A bundle like:

    Bundle-Name: hibernate
    Bundle-Version: 1.2.3
    Require-Bundle: org.hibernate.foo;visibility:=reexport

    would have the same ability to have the dependencies on the bundle generally. The org.hibernate.foo could import packages as appropriate and then export them if needed. You still get the loose coupling about what is in the 'hibernate' package, and can get the packages from there.

    "Any problem can be solved by an extra level of indirection"

    Alex


  4. Hi Glynn,

    Yep agree there is a problem, but not on the solution – makings of a fun discussion ;)

    [quote post="331"]I agree the tooling should give early warning of such problems. A further advantage of processing the headers at runtime is that it allows the Platform to provide diagnostics and other feedback in terms of the developer's inputs.[/quote]

    Are you suggesting the application server can modify your bundle to resolve any problems at deployment time? If so I'm not sure how this works with version control systems – if the system gets restarted do you need to manually refix each bundle each time? It seems to me build or development time tools could still have feedback but generate standard osgi bundles in the end and do so in a way that worked with version control.

    Regards,

    Dave


  5. Hi Alex

    [quote comment=\"105679\"]I\'m not convinced of the difference between Require-Bundle and Import-Bundle.[/quote]

    Then I guess you\'ve never had to grapple with a split package. ;-)

    [quote comment=\"105679\"]I especially don\'t see the need of Import-Library, either.

    You could easily have a single bundle which represented hibernate-as-a-whole and solely existed to provide wiring for importing packages (exported to the client). A bundle like:

    Bundle-Name: hibernate
    Bundle-Version: 1.2.3
    Require-Bundle: org.hibernate.foo;visibility:=reexport

    would have the same ability to have the dependencies on the bundle generally. The org.hibernate.foo could import packages as appropriate and then export them if needed. You still get the loose coupling about what is in the \'hibernate\' package, and can get the packages from there.

    \"Any problem can be solved by an extra level of indirection\"[/quote]

    Yes, it is certainly possible to use a \"visibility:=reexport\" façade as a way of implementing libraries, but the design intent gets lost along the way and you have to resort to some careful naming conventions if the runtime is to \"understand\" libraries and manage them in any meaningful way.

    Consider a situation where a library is deployed into the Platform and then some applications are deployed that import the library. Later an administrator comes along and is considering upgrading the library and wants to know which applications currently depend upon it to assess the impact. The runtime needs to be able to report the dependencies at the level of applications and libraries.

    Glyn


  6. [quote comment="105683"]Hi Glynn,

    Yep agree there is a problem, but not on the solution – makings of a fun discussion ;)

    [quote post="331"]I agree the tooling should give early warning of such problems. A further advantage of processing the headers at runtime is that it allows the Platform to provide diagnostics and other feedback in terms of the developer's inputs.[/quote]

    Are you suggesting the application server can modify your bundle to resolve any problems at deployment time? If so I'm not sure how this works with version control systems – if the system gets restarted do you need to manually refix each bundle each time? It seems to me build or development time tools could still have feedback but generate standard osgi bundles in the end and do so in a way that worked with version control.

    Regards,

    Dave[/quote]

    I'm suggesting no such thing, so I'm sorry that I somehow gave you that impression!

    As I said in the post, the Platform detects the situation where a bundle imports two or more libraries which export a common package, issues an appropriate log message, and fails to install the importing bundle.

    I was simply making the point that the tooling could provide prior warning of that situation and prompt the developer to fix it before deploying to the Platform. This would prevent the "nasty surprise" you alluded to.

    Glyn


  7. Hi Glyn,

    Ok, understood. Sorry for misunderstanding.

    Agree split packages and can be a major pain with Require-Bundle, so I can see the intent of Import-Bundle.

    It just seems the real value of statements like Import-Bundle would be at build time. Certainly validating at runtime is useful but there wouldn't need to be validation if the extension headers didn't exist?

    I'll stop hogging the list, seems we think there are different solutions which I'm happy to agree on :)

    Regards,

    Dave


  8. [quote comment="105690"]Hi Glyn,

    Ok, understood. Sorry for misunderstanding.

    Agree split packages and can be a major pain with Require-Bundle, so I can see the intent of Import-Bundle.

    It just seems the real value of statements like Import-Bundle would be at build time. Certainly validating at runtime is useful but there wouldn't need to be validation if the extension headers didn't exist?

    I'll stop hogging the list, seems we think there are different solutions which I'm happy to agree on :)

    Regards,

    Dave[/quote]

    Thanks Dave.

    Glyn


  9. Glyn,

    I've written up my thoughts at http://alblue.blogspot.com/2008/05/springsource-app-platform-and-bundle.html and in particular, refute your thought that the dependency information can't be found:

    [quote post="331"]Consider a situation where a library is deployed into the Platform and then some applications are deployed that import the library. Later an administrator comes along and is considering upgrading the library and wants to know which applications currently depend upon it to assess the impact. The runtime needs to be able to report the dependencies at the level of applications and libraries.
    [/quote]

    That's trivial to do if the library is itself a bundle, because you can use the PlatformAdmin to find out what bundles you depend on. One can thus easily find all the applications depend on that 'library' bundle, and it doesn't need any non-standard wiring headers to make that happen. If using naming conventions isn't enough (e.g. hibernate-library etc.) then it's easy enough to add a marker header into the manifest that identifies that as a library bundle – though one might also do that by other means (for example, the way that Knopflerfish identifies library bundles is by listing those without a Bundle-Activator). My main point is that this is all trivially possible to do in the current dependency environment without needing to create additional dependency headers, and that's what I think should have been done here.


  10. Hi Alex

    The disadvantages of using a bundle façade as a way of implementing a library are as follows. Firstly, the design intent of the library producer and the library consumers is obscured. Libraries and bundles really are distinct concepts. Secondly, dependency management requires a definitive way of identifying libraries. Using the absence of a bundle activator is too weak as subsidiary bundles of an application do not need a bundle activator. Indeed, when Spring DM is used, application bundles do not need bundle activators at all. So we'd need to introduce a non-standard header to indicate libraries. Thirdly, there's the usage model. A client of a bundle façade would need to access it using Require-Bundle (to avoid the headache of maintaining the list of packages if they used Import-Package) and we are trying to avoid promoting the use of Require-Bundle because of its semantic rough edges, primarily split packages. Plus it would not be possible to tell if the intention in using Require-Bundle was to access a library or a bundle, which is important if you accept that libraries and bundles are distinct concepts. Fourthly, split packages would become a significant risk when customers and third parties define their own libraries. The Platform completely avoids this risk by building on Import-Package semantics.

    So the main objection to the use of non-standard headers seems to be that they are non-standard. ;-) But consider how some of the OSGi R4 facilities such as Require-Bundle and fragment bundles came into existence. They were implemented in Eclipse as non-standard extensions of OSGi R3 (without any special Eclipse prefix on the header names) and then adopted into OSGi R4 through participation of several Eclipse developers in the OSGi standards work. SpringSource is adopting a similar approach when we see requirements of customers of the Platform that are not satisfied by OSGi R4.1. We're hosting the next face to face meeting of the Core Platform and Enterprise Expert Groups next week. Apart from debating the approach taken by the Platform, we'll be explaining the requirements and rationale. If there really are missing facilities in the OSGi standard, then I would expect suitable generalisations of our solutions to find their way into a future version of the standard.

    This is a healthy way for the standard to evolve: proving certain features in the field before adopting them for standardisation. For instance, I fully expect some of our non-standard facilities will remain non-standard and only of interest to Spring users.

    Finally, I would like to correct your statement "The only problem with the repository is the use of non-standard headers (which many don't like)". All bundles in the SpringSource Enterprise Bundle Repository use only OSGi standard headers as a matter of policy. We even take care to expunge non-standard headers left over from tools such as bnd.

    Note that this provides a further justification for not using bundle façades to represent libraries. We include various library definitions in the repository. If we included bundle façades then these would need to include non-standard headers to clearly identify them as libraries, as reasoned above, and then we would deviate from our own policy of only including OSGi standard headers in bundles in the repository.

    Regards,
    Glyn


  11. Glyn,

    Thanks for the correction; I've updated the blog entry. And indeed, the objections are based upon the headers being non-standard, and as I noted in my post, I hope that they make it into OSGi 4.2 or 5.0 or whatever. I also appreciate that Equinox has used some non-standard ones headers in the past, but at least they've been prefixed Equinox- to clearly delineate ownership of the headers (and the migration path to, say, Bundle-ActivationPolicy has been made now that they are the standard). So on the whole, I think we're agreeing about where it should be going, just not necessarily about the way that it's been done.

    [quote post="331"] Firstly, the design intent of the library producer and the library consumers is obscured. Libraries and bundles really are distinct concepts. Secondly, dependency management requires a definitive way of identifying libraries.[/quote]

    This is a circular argument. You are asserting that libraries and bundles are really distinct concepts, then using that distinction to claim that you can't use the bundle's dependency management to determine the dependencies. Based on the lemma that libraries and bundles are different, I'd agree with the circular reasoning.

    However, I do not believe the lemma holds. What is a library, if not a large bundle? Why should I care how they split it down? If I'm depending on RCP to build my Eclipse application, should I care how it's broken down, or should I just say "I want to depend on RCP"? The Eclipse RCP feature is nothing more than a default grouping of bundles, and there's no reason why a facade bundle cannot be created to represent that (and indeed, hold extra meta-information about the suite of bundles as a whole such as who signed them, where the update site is, and so on).

    In fact, your arguments boil down to "We don't like Require-Bundle":

    [quote post="331"] A client of a bundle façade would need to access it using Require-Bundle (to avoid the headache of maintaining the list of packages if they used Import-Package) and we are trying to avoid promoting the use of Require-Bundle because of its semantic rough edges, primarily split packages.[/quote]

    There may be valid reasons for you not wanting to use Require-Bundle in this way, but claiming that it cannot be used to do this is again circular reasoning. It's perfectly possible to build a facade bundle in this way and use it by clients. Arguably, it protects you against the split-package issues that you can get when one package is separated across many bundles, because the facade can take care of that and not bother the client where they came from (or how they are combined). In fact, this is what is done in a number of cases in Equinox where split packages are common; each split package exports a tagged package, and then one of them imports all the tagged packes and re-exports the package as a whole. This is precisely the behaviour that a facade would do on top of existing bundles.

    [quote post="331"]It would not be possible to tell if the intention in using Require-Bundle was to access a library or a bundle, which is important if you accept that libraries and bundles are distinct concepts[/quote]

    The point is, I don't. It's a lemma you have taken for granted in your entire post, and has been used as the supporting argument in most of what you say. What I'm challenging you is to describe why they are distinct concepts. After all, modularity is as much about composition as it is about components; why can't we combine components together and treat them as a larger component? The J2EE spec is merely an aggregation of other specs, but is treated as a (large) component in its own right, albeit with optional dependencies. Similarly, people can write an application that depends on Spring DM, but that doesn't mean that they are using every component that is part of the Spring DM system.

    Why must a bundle only be the lowest-level granular component, and why can't it be aggregated? Why invent an artificial distinction and talk about 'library' and 'bundle'? Is Log4J a bundle or a library? What about Xerces? What about Tomcat? What about JBoss? Why and how do you decide to transition between one term and another in that sequence of ever-increasing sized systems, and importantly, as the person who uses those systems and doesn't care about the internal implementation details, why do I care about a difference in terminology when 'my app uses Log4j or Xerces or Tomcat or JBoss'. I don't care whether it's implemented in one big blob or two smaller blobs or fifteen smaller blobs; I depend on the aggregation, not each individually.


  12. [quote comment="106177"]What is a library, if not a large bundle?[/quote]

    A library is a named, versioned collection of bundles that an application programmer wants to deal with as a single entity, similar to an Eclipse feature, but visible in the runtime as a type supporting management and dependency operations.

    A suitably tagged façade bundle could be used, as we have already agreed, to represent a library, but that doesn't equate the concepts of library and (large) bundle.

Leave a Reply