[daisy] New Daisy repository runtime platform

[Bruno Dumon]

Hi,

I'm currently looking at creating an alternative for our current
repository runtime platform, Avalon Merlin (which is a sort of "IoC"
component container).

Merlin has brought some good things, like isolation of component
containers and maven-repository-based classloaders. However, since
Merlin development stopped some time after we started using it (a couple
of years ago), it has not been further improved and we didn't invest
into using it more extensively.

The closest, most popular alternative to Merlin will probably be
provided by spring-osgi. However, this would be a few steps too many to
take in one time, therefore I'm now looking into establishing a more
gradual/realistic short-term solution.

In summary, it consists of a set of isolated spring containers which can
have custom classloaders (still maven repository-based) and which can
share selected services between them.

The following diagram illustrates this:

+-------------+                    +---------------------+
|Daisy Runtime| import services -> | Spring container 1  |
|             |                    |                     |
|manages set  | <- export services +---------------------+
|of isolated  |                    | classloader         |
|spring       |                    +---------------------+
|containers   |
|             |                    +---------------------+
|             | import services -> | Spring container 2  |
|             |                    |                     |
|             | <- export services +---------------------+
|             |                    | classloader         |
|             |                    +---------.-----------+
|             |                              |
|             |   ... more containers        |
|             |                              |
|             |                              |
|             |                   +----------+-----------+
+-------------+                   |Maven (1/2) repository|
                                  +----------------------+


Each container will be packaged as a jar containing spring XML bean
container configurations, and a classloader specification.

The runtime is booted by some configuration that specifies the list of
such jars to load. We can support wildcards to allow easy extensibility
of e.g. jars located in <datadir>/ext.

There will be one global classloader for the Daisy runtime as a whole,
and one for each container. The classloader specifications in the
container configuration can specify artifacts to be added to the global
classloader (e.g. for API jars), or artifacts which may be shared if
used by multiple containers, and artifacts which cannot be shared (the
private implementation).

This is a very skimpy description, I'll write up a more extensive
document later on, once I've validated the core ideas. If you have
questions feel free to ask them.

The runtime platform change will bring us lots of good stuff:

 * a container which is more suited for fine-grained objects (Merlin was
more oriented towards larger-scale components)
 * a container (spring) familiar to many people
 * easily adding custom additions (authentication schemes, repo
extensions, pre-save hooks, etc.) without needing to modify a central
configuration file (block.xml) or needing to learn about obsolete avalon
interfaces.
 * no more java-1.4 limitation for component classes (which is today the
case when using the merlin maven plugin)
 * no more maven 1 dependency as far as merlin is concerned, so we can
start thinking about moving to maven 2
 * initial Daisy source build will become easier (no more merlin
download & maven plugin install needed)
 * easier embedding of the repository, "new DaisyRuntime(config)" should
do it
 * and still leaves us with these feature:
    * maven-based repositories: allows to run the repository right after
build (no further copying of jars necessary), and allows for easier
dependency management
    * isolation of core repository from extensions

Some of the things that will need to be done to achieve all this are:

 * creating the runtime system
 * adjusting components to work in the new system, mainly:
    - get dependencies and do initialisation in constructor
    - configuration will be accessible by depending on a
ConfigurationManager service (in a first step, still using the avalon
Configuration object)
    - logging using commons-logging or slf4j
 * updating the artifacter tool
 * updating the distribution build / default setup (log configuration,
component configuration)
 * updating repository testcase system
 * docs (description of the runtime system, how-to of how to add
extensions)

Development will happen in the branch BRANCH_NEWRUNTIME. Right now I
have a very premature runtime implementation, and I'm working on
adjusting components.

Since we've got a lot of other work going on, I'm not sure by when this
will be finished, but this is definitely post-2.1 stuff.

WDYT?

-- 
Bruno Dumon                             http://outerthought.org/
Outerthought - Open Source, Java & XML Competence Support Center
bruno at outerthought.org                          bruno at apache.org