[daisy] Daisy Architecture Questions

[Paul Focke]

Hi Shon

Interesting stuff there.

Having quickly gone over your mail I must say that I'd opt for option 1.
However I think doing the whole import export document per document
might be overkill. You could just listen to the JMS events concerning
documents and the content of such an event is nearly enough to create a
document. I think that all you are missing is the part content. This
content can be retrieved from the master/write repository using one of
the apis (http, java).
So what you would have is 
 1. the write repo, 
 2. a bunch of read repos (each with their own db & data-store), 
 3. an jms event listener component that is configured to catch jms
events from the write repository and replicate them on the configured
read repositories.

How would you do the load balancing itself? I would think of using the
load balancing feature in apache for example. You would have 1 instance
of your flex frontend per read repository and in front of the frontend
your load balancing proxy. You could place the load balancer in between
the frontend & the repositories but then the bottle neck would be in the
frontend. 

Note to you, dear reader : I haven't ever worked with load balancers &
fail over stuff so I'm just writing down my gut feeling about this.


Paul

On Thu, 2008-01-24 at 13:54 -0600, Shon Schetnan wrote:
> Hello,
> 
> We are planning to use Daisy as a repository to an application that is
> primarily a Flash/Flex application fronted by a web container providing
> content to the flex application via XML.  The Daisy repository will hold
> content that we will be presenting in the Flex application.
> 
> We have requirements that the Daisy deployment include a fail-over component
> as well as load balancing.  We have identified four different possibilities
> for a Daisy deployment architecture
> 
> 1.  Master Daisy repository behind a farm of read-only Daisy repositories -
> We would deploy a master repository where all edits and content management
> would take place.  An exporter service would monitor Daisy JMS events and
> fire off a Daisy export process and a corresponding export event.  The
> read-only Daisy servers would be fronted by import listeners that would
> catch export events.  They would then import the zip file provided by the
> export event into their repositories.  This approach would require some
> development work by us, but could give somewhat of a "Daisy Appliance" type
> deployment where we would get fail-over plus load balancing of
> Daisy-Repository servers plus MySQL and file system.
> 
> Question:  I am assuming that if we implemented export/import services using
> the Daisy API that we could run imports on the read-only servers while users
> are reading from the repository.  True?
> 
> 2. 1 Master Daisy writer instance, 1 or more Daisy read-only instances. The
> Daisy writer points to its own datastore and database. The Daisy readers
> share a separate datastore and database.  The MySQL database of the master
> would be replicated to the MySQL instance of the shared reader Daisy's.  The
> Master "DaisyData"  would be replicated to the shared readers'  file system.
> 
> Concerns:  What happens if the timing of MySQL and file system repositories
> are slightly out of sync timing wise?  Would this open us up to strange
> sided effects and exceptions?  Any downtime required?
> 
> 3.  Similar to previous approach except that there is only a single active
> reader instance. Additional reader instances are configured as passive
> nodes. The reader's datastore lives on network storage to ensure that all
> files are available to the passive nodes at the time their made active.
> This would not satisfy our load balancing requirement, but could potentially
> satisfy failover.
> 
> 
> 4.  1 Daisy writer instance, 1 or more Daisy read-only instances. All Daisy
> instances point to the same datastore and MySQL database.  This would
> provide failover and some level of load balancing at least for the
> Daisy-Repository servers.  It wouldn't provide full balancing of file-stores
> and MySQL databases.
> 
> Concerns:  Does a shared file-system and MySQL database pose any problems?
> 
> 
> I am interested in feedback on the feasibility of either of these
> approaches.  I am currently leaning toward option #1 or option #4, because
> they seem to fit our needs best, but I could use some feedback around
> possible problems with either approach that maybe we have not thought of.
> 
> Thanks for any feedback!
> 
> Shon
> 
> 
> This message is intended only for the use of the individual or entity to whom it is addressed, and may contain information that is PROPRIETARY and CONFIDENTIAL. If you are not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the above named sender at the above telephone number.
> _______________________________________________
> daisy community mailing list
> Professional Daisy support: http://outerthought.org/en/services/daisy/support.html
> mail to: daisy at lists.cocoondev.org
> list information: http://lists.cocoondev.org/mailman/listinfo/daisy