gistfile1.txt

16:39 bobmcw: miclorb: hey dude
16:39 miclorb: hey
16:39 bobmcw: around for a design discussion?  I've got a mental block.
16:39 bobmcw: torquebox now supports config/queues.yml and config/topics.yml to deploy a-la-carte queues and topics
16:39 miclorb: have a call in 15 mins with john graham and kurt - they wanted to talk about productisatino stuff (drools)
16:39 miclorb: ok
16:40 miclorb: nice re yml
16:40 bobmcw: I still like my app/queues/foo_queue.rb for easy async task stuff
16:40 bobmcw: those, though, define their own queue, as an impl detail
16:40 bobmcw: thinking about moving that to app/tasks/foo.rb
16:40 bobmcw: for stuff like sending emails async
16:40 bobmcw: it just happens to use MQ underneath to accomplish it
16:41 bobmcw: but, we *also* want to be able to basically have a MessageListener in ruby attached to any random queue/topic
16:41 miclorb: I agree - I think async stuff that usues Queues underneath is an impl concern
16:41 bobmcw: a different use-case, more generalized, than tasks
16:41 miclorb: don't expose it directly, but call it a "background task" etc.
16:41 bobmcw: this use-case is more "expose JMS"
16:41 miclorb: but queues are for messaing
16:41 miclorb: yes
16:41 bobmcw: right, so I've got the bg task happy
16:41 miclorb: queues often involve muldiple nodes by design
16:41 miclorb: tasks, not always
16:41 bobmcw: but I still need to figure a nice way to say "attach this class to queue/topic X"
16:42 bobmcw: as a MessageListener
16:42 miclorb: hmmm....
16:42 bobmcw: like a ruby MDB
16:42 miclorb: acts_on_message :whatever ?
16:42 bobmcw: attached to either/or, with more flexibility than bg tasks
16:42 bobmcw: mostly for interacting with queues into other systems etc
16:42 miclorb: yes
16:42 bobmcw: plus I need to roll the similar client end, to send to arbitrary queues not necessarily backed by ruby consumers
16:43 miclorb: ok
16:43 bobmcw: so, app/consumers/foo.rb ?  class Foo ; acts_on_message 'queue_name'
16:43 bobmcw: let me back up and give you marek's use-case
16:44 bobmcw: for boxgrinder
16:44 bobmcw: the -rest server will define a queue for each architecture it supports
16:44 bobmcw: i386, x86_64, etc
16:44 bobmcw: when you request an appliance for i386, it'll drop the request into the queue, typical stuff
16:44 bobmcw: durable queue
16:45 bobmcw: a builder node may fire up on EC2, connect to queue, and build your appliance
16:45 bobmcw: or maybe we have 10 nodes connected capable of building i386 already
16:45 miclorb: right yes
16:45 bobmcw: so, just a way to farm out tasks
16:45 bobmcw: since appliance-spinning isn't cheap
16:45 bobmcw: s/cheap/fast/
16:45 miclorb: right - yeah I like that
16:45 miclorb: where would you set up how many parallel consumers there are etc?
16:45 bobmcw: so, server defines queues.yml to set up the queues, easily enough.  no matching .rb class specific to the queues.
16:46 bobmcw: at this point, boot up N build nodes manually
16:46 bobmcw: they query uname -a, and set up a message-listener against the correct queue deployed on the server
16:46 bobmcw: ie, figure out what platform they support, and become a consumer to the right channel
16:46 bobmcw: at some point -rest itself might use deltacloud to boot up build nodes if necessary
16:47 miclorb: yes when it needs more
16:47 miclorb: queues make sense for that
16:47 bobmcw: right now, build nodes are probably not torquebox
16:47 bobmcw: jruby, but just a simple client, not TB
16:47 bobmcw: might use TB gems, of course
16:47 miclorb: so what goes in the yml?
16:47 bobmcw: but no reason for the build node to fire up an entire freakin' AS to simply deploy a MDB
16:47 bobmcw: .yml right now is just a list of queues (or topics) to create
16:47 miclorb: yeah you can just use horney queue on java
16:48 bobmcw: they default to durable and clustered
16:48 bobmcw: right, so build nodes will be hornetq-jms clients
16:48 bobmcw: I'm not sure what my question is, to be honest
16:48 miclorb: bobmcw: does it need to be in yml? as if they are the names, they aren't really config items? or do you think there will be additional config needed?
16:48 bobmcw: just needed to ramble
16:48 miclorb: yeah, it gets like that
16:48 miclorb: could the config be in rb itself?
16:48 bobmcw: well, figured expose more config
16:48 miclorb: just thinking aloud - what rails and other things do
16:48 bobmcw: myQueue:
16:48 bobmcw:  durable: false
16:48 bobmcw:  attach:
16:49 bobmcw:    SomeListener
16:49 miclorb: fair enough
16:49 bobmcw: but yah, I've also thought maybe queues.rb similar to routes.rb
16:49 miclorb: that is fairly standard for messaging middleware
16:49 bobmcw: instead of shoe-horning into a datastructure
16:49 miclorb: yes
16:49 bobmcw: queue( "myqueue" ) do
16:49 bobmcw:  durable
16:49 miclorb: much of a much-ness from users point of view
16:49 miclorb: yeah
16:49 bobmcw:  attach( Whatever )
16:49 bobmcw: being .rb, it could include marek's logic of figuring out ARCH or PLATFORM of the build node itself, also
16:50 bobmcw: I've also got to allow build nodes to be able to configure JNDI lookups specifically back against the server
16:50 bobmcw: anyhow, back to rails, let's say...
16:51 bobmcw: if my app connects to some queue deployed either by my app, or outside of my app...
16:51 bobmcw: app/consumers/ ?  app/listeners/ ?
16:51 bobmcw: do I have a magic directory for MDB-alikes?
16:51 bobmcw: s/do/should/
16:51 bobmcw: and still have queues.rb to align a listener to a queue or topic?
16:52 bobmcw: how do I differentiate between queues.rb where I'm deploying a queue, and queues.rb where I'm attaching to a pre-deployed queue?
16:52 bobmcw: create-if-needed, I reckon
16:52 bobmcw: so, app/what/foo.rb
16:52 bobmcw: I need a value for 'what'
16:53 bobmcw: to accurately describe a class that can respond to messages, from a topic or queue
16:53 bobmcw: or should we have app/queues/ and app/topics/ and make a distinction at the dir level?
16:53 bobmcw: recampbell_: thoughts? :)
16:54 recampbell_: heh, still looking for my intro to ruby book ;-)
16:55 bobmcw: miclorb: so, rails is split, depending on complexity
16:55 bobmcw: database.yml
16:55 bobmcw: but it's ERB, so you can punt to <%= weird_stuff() %>
16:56 bobmcw: and config/routes.rb is a ruby DSL
16:56 bobmcw: and a lot of config/environment** stuff is .rb, as are initializers
16:56 miclorb: bobmcw: I think if I was lookign at a project, if there were listeners to react to events - I would want ONE place to look - either a directory, or some .rb or .yaml config file to bounce off from
16:56 bobmcw: and I'm also thinking I don't want to restrict all of this to "deploy" time only
16:56 bobmcw: I think a run-time API for setting up a new listener could be good
16:56 miclorb: yeah - well the important thing is one place.. hmm....
16:57 bobmcw: need to check out dmlloyd's threads component thing
16:57 miclorb: I think the closest things is the routes in rails right?
16:57 miclorb: as in a sense - rails is "reacting" to styuff happening on a URL
16:57 bobmcw: yah, I think fairly analogous to routes
16:57 miclorb: to dispatch to the controllers
16:57 bobmcw: connecting an input to some code
16:57 miclorb: so perhaps listeners.rb
16:57 miclorb: exactly
16:57 bobmcw: URLs to controllers; destinations to listeners
16:57 bobmcw: 'k, queues.yml to deploy some queues, if you like.  listeners.rb to wire up code to queues/topics, regardless of deployment origin
16:58 bobmcw: and app/tasks/**.rb async bg tasks manage their own queues
16:58 bobmcw: simply as an impl detail
16:58 bobmcw: thanks!