Jobs (2 kinds)

Does anyone know of a good open source queuing system for database processes? I’ve heard a few suggestions but I could use some links and documentation. I’m looking for a tool to abstract and manage what I’m currently doing with a database table and a cron job.

Also, I’m looking for a great web developer to help me with Pownce. I’m looking for someone who loves working on web applications and is smarter and/or more experienced than me. There’s so many interesting problems to work on at Pownce and we have a small, enthusiastic and friendly team. The best part is the amount of freedom, flexibility, and influence Pownce has as a new web application. Please read the description and email me if you’re interested.

Meta

This entry was written by Leah, posted on October 12, 2007 at 9:32 am.

Filed under Uncategorized.

Bookmark the permalink.

Follow any comments here with the RSS feed for this post.

Both comments and trackbacks are currently closed.

9 Comments

  1. Matt Erkkila

    Posted October 12, 2007 at 9:53 am | Permalink

    Take a look at Active MQ. It is a JMS based queue/messaging system that you can use to manage and process any type of message, be it database, email, etc. It has great redundancy and load balancing features and is a apache project so it has a great user base. The system itself is java (it has all sorts of libraries to access it, even python) so it can be a resource hog and sometimes hard to configure but once you get it working it works well. Hope this helps.

  2. Dion Almaer

    Posted October 12, 2007 at 11:16 am | Permalink

    Yah check out ActiveMQ and Stomp. Python-y help is available: http://stomp.codehaus.org/Python

  3. Alex

    Posted October 12, 2007 at 12:10 pm | Permalink

    Have you considered Amazon SQS ? Since you’re using S3 already it would seem like a good fit. I’d be interested in your thoughts on Amazon SQS and other queuing systems once you’ve made a decision.

  4. Chris Milton

    Posted October 12, 2007 at 10:41 pm | Permalink

    I could probably write a daemon in Perl with something like Queue::Dir [ http://search.cpan.org/~luismunoz/Queue-Dir-0.01/ ].

  5. Cam

    Posted October 14, 2007 at 5:49 am | Permalink

    Can’t tell from the post: Are you looking for a message queue, or something to queue batch processes?

  6. john

    Posted October 14, 2007 at 11:13 pm | Permalink

    an open source queuing system for database processes? i’d offer a suggestion, but i have no idea wtf that is. and from the other comments, it sounds like the others don’t either.

  7. Mason

    Posted October 15, 2007 at 11:05 pm | Permalink

    Looks promising:

    http://code.google.com/p/django-queue-service/

    Pax,
    M

  8. Chris Milton

    Posted October 16, 2007 at 4:49 pm | Permalink

    My previous comment is still awaiting moderation, but that Django Queue Service looks cool.

  9. Brian Aker

    Posted October 19, 2007 at 12:32 pm | Permalink

    Hi!

    Ask Blaine of Twitter. They were using Innodb under MySQL for quite some time, and then they moved to some sort of memcached hacked up server (which they have not open sourced). There are some attempts to come up with a queue engine for MySQL, but at the moment I would just stick with Innodb.

    Cheers,
    -Brian