Architecture thoughts v 0.1.1

architecture-thoughts.markdown

Architecture Thoughts

A quick writeup of architecture ideas for developing and deploying ruby applications.

(NOTE: I'm still semi-new to ruby so I'm mainly compiling untested ideas I hear about).

TODO

• Mocking
• Unit Testing
• Integration Testing
• Deployment (Capistrano?)
• Sample Project
• Integrate additions from http://backstage.soundcloud.com/2012/08/evolution-of-soundclouds-architecture/
• PubSub/Fanout queues (something like RabbitMQ or qdis+redis)

Version

• 0.1.1 - moved to rails.
• 0.1 - initial version

Links

• http://www.shopify.com/technology/3383012-why-developers-should-be-force-fed-state-machines
• http://backstage.soundcloud.com/2012/08/evolution-of-soundclouds-architecture/

architecture.markdown

Architecture

Code

• Rails base.
• Integrating with an outside system is done through a resque worker.
• Scheduling jobs is done by creating a resque job from within cron.
• Core logic can be modeled inside the ActiveRecord model using state_machine.

Servers

• "N" Web application and REST api nodes deployed and load balanced behind nginx.

  • installed with foreman.
  • supervised by upstart.
  • running on unicorn.

• "M" worker nodes running long running processes.

  • installed with foreman
  • supervise by upstart

• "P" mysql or postgresql servers.

• "Q" AMQP servers.

development.markdown

Development

Core development libraries

• Web Applications: Rails.
• Long Running/Blocking Processes: minion
• Business Logic: ActiveRecord, StateMachine, state_machine-audit_trail
• Authentication: ?
• Server-Side HTML Binding/Generation: ERB
• Client-Side HTML Binding/Generation: knockoutjs? mustache? angularjs?
• UI: foundation
• Search: ransack
• Code Management: git.
• Standout gems:
  • Layout
    • zurb_foundation - foundation layout system.
    • simple_form - form building for foundation

Development tools

• Running:

  • foreman, for describing the programs run in development mode.

• Code Reloading:

  • shotgun, for reloading Rack applications on code changes.
  • guard, for reloading of non-Rack applications (workers and non rack daemons).

pubsub.markdown

These are the potential worker libraries I'm looking at:

• resque, which is redis backed.
• minion, which is RabbitMQ backed.

Neither one of these support pubsub. Should we then use resque for doing delayed jobs and standard AMQP or handrolled for pubsub? Or use minion for delayed jobs and standard AMQP for events? Or patch minion to support pubsub as specified here: orionz/minion#7

There is also something like faye: http://faye.jcoglan.com/ruby.html with redis backend. but I'm not sure faye can do multi-subscriber pub/sub.

some options (unless we want to run both RabbitMQ and redis at the same time):

• minion + minion-pubsub patche(s)
• resque + faye with redis backend.
• minion + a custom AMQP backed pubsub.

redis supports pub-sub but resque does not =( https://gist.github.com/348262

even the pubsub resque plugin doesn't use the pubsub support in redis, it uses persistence. Time for a handrolled redis based pubsub system.