Deadlock with queue representation

deadlock-queue.xqy

(: The basic problem I'm trying to solve is that of a queue that can also updated 
   for priority overrides. I thought about using map:map, but this approach seemed 
   easier for a first cut. :)

(: If I call queue-lib:retrieve-queue() first, everything works fine. If I try to 
   call queue-lib:retrieve-queue() from within queue-lib:enqueue(), I'm creating 
   an update within an update condition, as I understand it, and I end up with a 
   deadlock condition. I haven't resolved this yet, so any suggestions are 
   welcome! :)

xquery version "1.0-ml";
module namespace queue-lib = "http://me.com/queue-lib";
declare namespace queue = "http://me.com/queue";

declare private variable $QUEUE-URI := "/queue/queue.xml";

(: Retrieves the entire queue. :)
declare function queue-lib:retrieve-queue() as element(queue:queue) {
    let $queue-document := queue-lib:initialize()
    
    return $queue-document/queue:queue
};

(: Adds a new queue:item if it does not already exist, which is checked by id. :)
declare function queue-lib:enqueue($item as element(queue:item)) {
    (: REVIEW: I'm not clear why re-using retrieve-queue() is a transactional problem when no queue exists ahead of time. :)
    (: But if I can't do this, I *have* to initialize the queue first, then call this second. :)
    (:let $queue-document := queue-lib:retrieve-queue():)

    (: This won't work if I don't call queue-lib:retrieve-queue() beforehand, outside of this function. :)
    let $queue-document := fn:doc($QUEUE-URI)/element() 
    let $id := $item/queue:id/text()
    
    return 
        if( fn:exists($queue-document/descendant::queue:item[queue:id = $id]) ) then
            fn:error( xs:QName("ERROR"), fn:concat($id, " already exists in the queue. ") )        
        else 
            xdmp:node-insert-child( $queue-document, $item )
};

(: If a queue document does not already exist, create one. Return the existing or newly created document. :)
declare private function queue-lib:initialize() {
    (: Transactions + functions http://developer.marklogic.com/pipermail/general/2010-July/005852.html :)
    if(fn:not(fn:exists(fn:doc($QUEUE-URI)))) then 
        xdmp:eval(
            fn:concat(
                (: REVIEW: Seems like I can't pass an element in as a variable because of concat(). :)
                'xdmp:document-insert("' , $QUEUE-URI , '", <queue xmlns="http://me.com/queue"/>, (), "queue")'
            ),
            (),
            <options xmlns="xdmp:eval">
                <isolation>different-transaction</isolation>
                <prevent-deadlocks>true</prevent-deadlocks>
            </options>            
        ) 
    else ()
    ,
    xdmp:eval(
        'fn:doc("/queue/queue.xml")',
        (),
        <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>true</prevent-deadlocks>
        </options>
    )
};

This is a pretty typical problem. The short answer is to use a directory of many queue-item documents rather than a single queue-document avoid lock contention on the single queue document.

What's happening is that MarkLogic uses standard Readers/Writer locks. So any transaction can have many readers locks or one writer lock. This access pattern: [get a resource, update that resource] creates contention when two transactions come in. T1 and T2 both get read locks (no problem). T1 then tries to upgrade to a write lock and blocks because T2 also has a read lock. T2 similarly blocks waiting on T1. This is a detectable deadlock so MarkLogic rolls back and retries one transaction. You'll see these deadlocks in ErrorLog.txt if your logging level is at Debug or lower.

Using evals inside updates that lock the same document can also create an undetectable deadlock, as described in MarkLogic documentation.

Ah, I wish I had thought of directories. If my thinking is correct, I can even use the search API. Excellent. Thanks so much for the suggestion!