Skip to content

Instantly share code, notes, and snippets.

@abloom

abloom/incident.rb

Created October 2, 2011 18:40
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save abloom/1257749 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save abloom/1257749 to your computer and use it in GitHub Desktop.
Download ZIP
Mongo DB Geo Spacial Map Reduce Error
Raw
incident.rb
class Incident
include MongoMapper::Document
key :case_number, Integer
key :date, Date
key :charge, String
key :description, String
key :arrest, Boolean
key :domestic, Boolean
key :beat, Integer
key :ward, Integer
key :location, Array
ensure_index :charge
ensure_index [[:location, "2d"]] # geo-spacial index
end
Raw
mongo.log
Sun Oct 2 14:53:30 [initandlisten] connection accepted from 127.0.0.1:59331 #5
Sun Oct 2 14:53:30 [conn5] run command admin.$cmd { whatsmyuri: 1 }
Sun Oct 2 14:53:30 [conn5] command admin.$cmd command: { whatsmyuri: 1 } ntoreturn:1 reslen:62 0ms
Sun Oct 2 14:53:30 [conn5] run command crime_maps_development.$cmd { count: "incidents", query: { location: { $within: { $box: [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } }, fields: {} }
Sun Oct 2 14:53:31 [conn5] command crime_maps_development.$cmd command: { count: "incidents", query: { location: { $within: { $box: [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } }, fields: {} } ntoreturn:1 reslen:48 767ms
Sun Oct 2 14:53:31 [conn5] run command crime_maps_development.$cmd { mapReduce: "incidents", map: function () {
emit(this.charge, 1);
}, reduce: function (k, values) {
var results = 0, i;
for (i = 0; i < val..., out: "console", query: { location: { $within: { $box: [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } } }
Sun Oct 2 14:53:31 [conn5] mr ns: crime_maps_development.incidents
Sun Oct 2 14:53:31 [conn5] query crime_maps_development.system.js reslen:20 0ms
Sun Oct 2 14:53:31 [conn5] run command crime_maps_development.$cmd { drop: "tmp.mr.incidents_2_inc" }
Sun Oct 2 14:53:31 [conn5] CMD: drop crime_maps_development.tmp.mr.incidents_2_inc
Sun Oct 2 14:53:31 [conn5] command crime_maps_development.$cmd command: { drop: "tmp.mr.incidents_2_inc" } ntoreturn:1 reslen:62 0ms
Sun Oct 2 14:53:31 [conn5] create collection crime_maps_development.tmp.mr.incidents_2_inc { autoIndexId: 0 }
Sun Oct 2 14:53:31 [conn5] New namespace: crime_maps_development.tmp.mr.incidents_2_inc
Sun Oct 2 14:53:31 [conn5] build index crime_maps_development.tmp.mr.incidents_2_inc { 0: 1 }
Sun Oct 2 14:53:31 [conn5] external sort root: /usr/local/var/mongodb/_tmp/esort.1317585211.897270579/
Sun Oct 2 14:53:31 [conn5] external sort used : 0 files in 0 secs
Sun Oct 2 14:53:31 [conn5] New namespace: crime_maps_development.tmp.mr.incidents_2_inc.$0_1
Sun Oct 2 14:53:31 [conn5] done building bottom layer, going to commit
Sun Oct 2 14:53:31 [conn5] fastBuildIndex dupsToDrop:0
Sun Oct 2 14:53:31 [conn5] build index done 0 records 0.001 secs
Sun Oct 2 14:53:31 [conn5] insert crime_maps_development.system.indexes 193ms
Sun Oct 2 14:53:31 [conn5] run command crime_maps_development.$cmd { drop: "tmp.mr.incidents_2" }
Sun Oct 2 14:53:31 [conn5] CMD: drop crime_maps_development.tmp.mr.incidents_2
Sun Oct 2 14:53:31 [conn5] command crime_maps_development.$cmd command: { drop: "tmp.mr.incidents_2" } ntoreturn:1 reslen:62 0ms
Sun Oct 2 14:53:31 [conn5] create collection crime_maps_development.tmp.mr.incidents_2 {}
Sun Oct 2 14:53:31 [conn5] New namespace: crime_maps_development.tmp.mr.incidents_2
Sun Oct 2 14:53:31 [conn5] adding _id index for collection crime_maps_development.tmp.mr.incidents_2
Sun Oct 2 14:53:31 [conn5] build index crime_maps_development.tmp.mr.incidents_2 { _id: 1 }
Sun Oct 2 14:53:31 [conn5] external sort root: /usr/local/var/mongodb/_tmp/esort.1317585211.796452019/
Sun Oct 2 14:53:31 [conn5] external sort used : 0 files in 0 secs
Sun Oct 2 14:53:31 [conn5] New namespace: crime_maps_development.tmp.mr.incidents_2.$_id_
Sun Oct 2 14:53:31 [conn5] done building bottom layer, going to commit
Sun Oct 2 14:53:31 [conn5] fastBuildIndex dupsToDrop:0
Sun Oct 2 14:53:31 [conn5] build index done 0 records 0.001 secs
Sun Oct 2 14:53:31 [conn5] query crime_maps_development.system.indexes reslen:20 0ms
Sun Oct 2 14:53:31 [journal] lsn set 2325444
Sun Oct 2 14:53:32 [conn5] Assertion: 10293:internal error: locks are not upgradeable: { "opid" : 31, "active" : true, "lockType" : "read", "waitingForLock" : false, "secs_running" : 0, "op" : "query", "ns" : "crime_maps_development.incidents", "query" : { "mapReduce" : "incidents", "map" : function () {
emit(this.charge, 1);
}, "reduce" : function (k, values) {
var results = 0, i;
for (i = 0; i < values.length; i++) {
results += values[i];
}
return results;
}, "out" : "console", "query" : { "location" : { "$within" : { "$box" : [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } } }, "client" : "127.0.0.1:59331", "desc" : "conn", "threadId" : "0x10320a000", "connectionId" : 5, "msg" : "m/r: (1/3) emit phase 999/119230 0%", "progress" : { "done" : 999, "total" : 119230 }, "numYields" : 0 }
0x1000898dd 0x100295741 0x100519a86 0x10052054c 0x10052b0b7 0x1004e0a43 0x1004e2397 0x1004a03f4 0x1004a1a34 0x1003d5e0f 0x1003e298a 0x100603394 0x100148d8c 0x10060a554 0x7fff89a23fd6 0x7fff89a23e89
0 mongod 0x00000001000898dd _ZN5mongo11msgassertedEiPKc + 333
1 mongod 0x0000000100295741 _ZN5mongo10MongoMutex4lockEv + 1009
2 mongod 0x0000000100519a86 _ZN5mongo2mr5State14reduceInMemoryEv + 438
3 mongod 0x000000010052054c _ZN5mongo2mr5State9checkSizeEv + 92
4 mongod 0x000000010052b0b7 _ZN5mongo2mr16MapReduceCommand3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 2327
5 mongod 0x00000001004e0a43 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 1107
6 mongod 0x00000001004e2397 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 2599
7 mongod 0x00000001004a03f4 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 52
8 mongod 0x00000001004a1a34 _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 4836
9 mongod 0x00000001003d5e0f _ZN5mongo13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 623
10 mongod 0x00000001003e298a _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1402
11 mongod 0x0000000100603394 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 148
12 mongod 0x0000000100148d8c _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 700
13 mongod 0x000000010060a554 thread_proxy + 132
14 libSystem.B.dylib 0x00007fff89a23fd6 _pthread_start + 331
15 libSystem.B.dylib 0x00007fff89a23e89 thread_start + 13
Sun Oct 2 14:53:32 [conn5] mr failed, removing collection :: caused by :: 10293 internal error: locks are not upgradeable: { "opid" : 31, "active" : true, "lockType" : "read", "waitingForLock" : false, "secs_running" : 0, "op" : "query", "ns" : "crime_maps_development.incidents", "query" : { "mapReduce" : "incidents", "map" : function () {
emit(this.charge, 1);
}, "reduce" : function (k, values) {
var results = 0, i;
for (i = 0; i < values.length; i++) {
results += values[i];
}
return results;
}, "out" : "console", "query" : { "location" : { "$within" : { "$box" : [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } } }, "client" : "127.0.0.1:59331", "desc" : "conn", "threadId" : "0x10320a000", "connectionId" : 5, "msg" : "m/r: (1/3) emit phase 999/119230 0%", "progress" : { "done" : 999, "total" : 119230 }, "numYields" : 0 }
Sun Oct 2 14:53:32 [conn5] run command crime_maps_development.$cmd { drop: "tmp.mr.incidents_2" }
Sun Oct 2 14:53:32 [conn5] CMD: drop crime_maps_development.tmp.mr.incidents_2
Sun Oct 2 14:53:32 [conn5] dropCollection: crime_maps_development.tmp.mr.incidents_2
Sun Oct 2 14:53:32 [conn5] dropIndexes done
Sun Oct 2 14:53:32 [conn5] command crime_maps_development.$cmd command: { drop: "tmp.mr.incidents_2" } ntoreturn:1 reslen:148 1ms
Sun Oct 2 14:53:32 [conn5] run command crime_maps_development.$cmd { drop: "tmp.mr.incidents_2_inc" }
Sun Oct 2 14:53:32 [conn5] CMD: drop crime_maps_development.tmp.mr.incidents_2_inc
Sun Oct 2 14:53:32 [conn5] dropCollection: crime_maps_development.tmp.mr.incidents_2_inc
Sun Oct 2 14:53:32 [conn5] dropIndexes done
Sun Oct 2 14:53:32 [conn5] command crime_maps_development.$cmd command: { drop: "tmp.mr.incidents_2_inc" } ntoreturn:1 reslen:152 0ms
Sun Oct 2 14:53:32 [conn5] command crime_maps_development.$cmd command: { mapReduce: "incidents", map: function () {
emit(this.charge, 1);
}, reduce: function (k, values) {
var results = 0, i;
for (i = 0; i < val..., out: "console", query: { location: { $within: { $box: [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } } } ntoreturn:1 exception: internal error: locks are not upgradeable: { "opid" : 31, "active" : true, "lockType" : "read", "waitingForLock" : false, "secs_running" : 0, "op" : "query", "ns" : "crime_maps_development.incidents", "query" : { "mapReduce" : "incidents", "map" : function () {
emit(this.charge, 1);
}, "reduce" : function (k, values) {
var results = 0, i;
for (i = 0; i < values.length; i++) {
results += values[i];
}
return results;
}, "out" : "console", "query" : { "location" : { "$within" : { "$box" : [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } } }, "client" : "127.0.0.1:59331", "desc" : "conn", "threadId" : "0x10320a000", "connectionId" : 5, "msg" : "m/r: (1/3) emit phase 999/119230 0%", "progress" : { "done" : 999, "total" : 119230 }, "numYields" : 0 } code:10293 reslen:930 987ms
Sun Oct 2 14:53:32 [conn5] end connection 127.0.0.1:59331
Sun Oct 2 14:53:35 [PeriodicTask::Runner] task: DBConnectionPool-cleaner took: 0ms
Sun Oct 2 14:53:35 [PeriodicTask::Runner] task: DBConnectionPool-cleaner took: 0ms
Sun Oct 2 14:53:35 [PeriodicTask::Runner] task: WriteBackManager::cleaner took: 0ms
Sun Oct 2 14:53:35 [DataFileSync] flushing mmap took 356ms for 7 files
Raw
script.js
geoQuery = {
location: {
$within: {
$box: [
[ 41.98518042784104, -87.68835532024703 ],
[ 42.02302490811244, -87.64736368746483 ]
]
}
}
};
map = function() {
emit(this.charge, 1);
};
reduce = function(k, values) {
var results = 0, i;
for(i = 0; i < values.length; i++) {
results += values[i];
}
return results;
};
count = db.incidents.count(geoQuery);
print(count); // => 119230
results = db.runCommand({
mapReduce: "incidents",
map: map,
reduce: reduce,
out: "console",
query: geoQuery
});
printjson(results);
/** output
{
"assertion" : "internal error: locks are not upgradeable: { \"opid\" : 17, \"active\" : true, \"lockType\" : \"read\", \"waitingForLock\" : false, \"secs_running\" : 0, \"op\" : \"query\", \"ns\" : \"crime_maps_development.incidents\", \"query\" : { \"mapReduce\" : \"incidents\", \"map\" : function () {\n emit(this.charge, 1);\n}, \"reduce\" : function (k, values) {\n var results = 0, i;\n for (i = 0; i < values.length; i++) {\n results += values[i];\n }\n return results;\n}, \"out\" : \"console\", \"query\" : { \"location\" : { \"$within\" : { \"$box\" : [ [ 41.98518042784104, -87.68835532024703 ], [ 42.02302490811244, -87.64736368746483 ] ] } } } }, \"client\" : \"127.0.0.1:58690\", \"desc\" : \"conn\", \"threadId\" : \"0x10320a000\", \"connectionId\" : 3, \"msg\" : \"m/r: (1/3) emit phase 999/119230 0%\", \"progress\" : { \"done\" : 999, \"total\" : 119230 }, \"numYields\" : 0 }",
"assertionCode" : 10293,
"errmsg" : "db assertion failure",
"ok" : 0
}
**/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment