Skip to content

Instantly share code, notes, and snippets.

@jcoyne

jcoyne/out.log

Created March 19, 2019 15:47
Show Gist options
  • Save jcoyne/9633c00f6464a429ed75c420566a2b9e to your computer and use it in GitHub Desktop.
Save jcoyne/9633c00f6464a429ed75c420566a2b9e to your computer and use it in GitHub Desktop.
Download ZIP
workflow getting slow then stopping
Raw
out.log
```
App 20285 stdout: [17b17870-b711-404b-a729-cafa047e3281] Processing by WorkflowQueuesController#show as XML
App 20285 stdout: [17b17870-b711-404b-a729-cafa047e3281] Parameters: {"completed"=>"dor:gisAssemblyWF:wrangle-data", "lane-id"=>"default", "limit"=>"100", "waiting"=>"dor:gisAssemblyWF:package-data"}
App 20233 stdout: [78bea94a-70a7-4473-8e1d-946c4a5dd2bb] Rendering workflow_queues/show.xml.builder
App 20233 stdout: [78bea94a-70a7-4473-8e1d-946c4a5dd2bb] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20233 stdout: [78bea94a-70a7-4473-8e1d-946c4a5dd2bb] Completed 200 OK in 185ms (Views: 0.6ms | ActiveRecord: 182.8ms)
App 20233 stdout: [95fab39f-c36d-448b-9070-c8890909d3ac] Started GET "/workflow_queue/lane_ids?step=dor%3AgisAssemblyWF%3Aextract-boundingbox" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [95fab39f-c36d-448b-9070-c8890909d3ac] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [95fab39f-c36d-448b-9070-c8890909d3ac] Parameters: {"step"=>"dor:gisAssemblyWF:extract-boundingbox"}
App 20233 stdout: [95fab39f-c36d-448b-9070-c8890909d3ac] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [95fab39f-c36d-448b-9070-c8890909d3ac] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [95fab39f-c36d-448b-9070-c8890909d3ac] Completed 200 OK in 2ms (Views: 0.5ms | ActiveRecord: 0.6ms)
App 20233 stdout: [a7b60f16-19fe-4e65-b9a4-697dfe742e7e] Started GET "/workflow_queue/lane_ids?step=dor%3AgisAssemblyWF%3Afinish-data" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [a7b60f16-19fe-4e65-b9a4-697dfe742e7e] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [a7b60f16-19fe-4e65-b9a4-697dfe742e7e] Parameters: {"step"=>"dor:gisAssemblyWF:finish-data"}
App 20233 stdout: [a7b60f16-19fe-4e65-b9a4-697dfe742e7e] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [a7b60f16-19fe-4e65-b9a4-697dfe742e7e] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [a7b60f16-19fe-4e65-b9a4-697dfe742e7e] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [70570ba6-62ac-41de-a656-b25dafd73893] Started GET "/workflow_queue/lane_ids?step=dor%3AdisseminationWF%3Acleanup" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [70570ba6-62ac-41de-a656-b25dafd73893] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [70570ba6-62ac-41de-a656-b25dafd73893] Parameters: {"step"=>"dor:disseminationWF:cleanup"}
App 20233 stdout: [70570ba6-62ac-41de-a656-b25dafd73893] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [70570ba6-62ac-41de-a656-b25dafd73893] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [70570ba6-62ac-41de-a656-b25dafd73893] Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.4ms)
App 20233 stdout: [a6dc3043-eba5-49a0-aefb-729fe462e733] Started GET "/workflow_queue/lane_ids?step=dor%3AwasSeedDisseminationWF%3Aupdate-thumbnail-generator" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [a6dc3043-eba5-49a0-aefb-729fe462e733] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [a6dc3043-eba5-49a0-aefb-729fe462e733] Parameters: {"step"=>"dor:wasSeedDisseminationWF:update-thumbnail-generator"}
App 20233 stdout: [a6dc3043-eba5-49a0-aefb-729fe462e733] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [a6dc3043-eba5-49a0-aefb-729fe462e733] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [a6dc3043-eba5-49a0-aefb-729fe462e733] Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.4ms)
App 20233 stdout: [16312658-f5e5-41b0-9148-669153c39168] Started GET "/workflow_queue/lane_ids?step=dor%3AdigitizationWF%3Astart-accession" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [16312658-f5e5-41b0-9148-669153c39168] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [16312658-f5e5-41b0-9148-669153c39168] Parameters: {"step"=>"dor:digitizationWF:start-accession"}
App 20233 stdout: [16312658-f5e5-41b0-9148-669153c39168] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [16312658-f5e5-41b0-9148-669153c39168] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [16312658-f5e5-41b0-9148-669153c39168] Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.4ms)
App 20233 stdout: [37d894fb-f161-4a82-a41d-5ba734f17151] Started GET "/workflow_queue/lane_ids?step=dor%3AdigitizationWF%3Adigitize" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [37d894fb-f161-4a82-a41d-5ba734f17151] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [37d894fb-f161-4a82-a41d-5ba734f17151] Parameters: {"step"=>"dor:digitizationWF:digitize"}
App 20233 stdout: [37d894fb-f161-4a82-a41d-5ba734f17151] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [37d894fb-f161-4a82-a41d-5ba734f17151] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [37d894fb-f161-4a82-a41d-5ba734f17151] Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.3ms)
App 20233 stdout: [b8fcf7e9-73b0-49fc-90e5-eb382b62bd93] Started GET "/workflow_queue/lane_ids?step=dor%3AreleaseWF%3Arelease-publish" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [b8fcf7e9-73b0-49fc-90e5-eb382b62bd93] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [b8fcf7e9-73b0-49fc-90e5-eb382b62bd93] Parameters: {"step"=>"dor:releaseWF:release-publish"}
App 20233 stdout: [b8fcf7e9-73b0-49fc-90e5-eb382b62bd93] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [b8fcf7e9-73b0-49fc-90e5-eb382b62bd93] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [b8fcf7e9-73b0-49fc-90e5-eb382b62bd93] Completed 200 OK in 3ms (Views: 0.3ms | ActiveRecord: 0.4ms)
App 20233 stdout: [9a28947c-40d7-4d67-be4b-e28891a76063] Started GET "/workflow_queue/lane_ids?step=dor%3AreleaseWF%3Aupdate-marc" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [9a28947c-40d7-4d67-be4b-e28891a76063] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [9a28947c-40d7-4d67-be4b-e28891a76063] Parameters: {"step"=>"dor:releaseWF:update-marc"}
App 20233 stdout: [9a28947c-40d7-4d67-be4b-e28891a76063] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [9a28947c-40d7-4d67-be4b-e28891a76063] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [9a28947c-40d7-4d67-be4b-e28891a76063] Completed 200 OK in 2ms (Views: 0.3ms | ActiveRecord: 0.8ms)
App 20233 stdout: [b3169eb5-e1ea-41f5-a47b-5ffd70a9a44c] Started GET "/workflow_queue/lane_ids?step=dor%3AreleaseWF%3Arelease-members" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [b3169eb5-e1ea-41f5-a47b-5ffd70a9a44c] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [b3169eb5-e1ea-41f5-a47b-5ffd70a9a44c] Parameters: {"step"=>"dor:releaseWF:release-members"}
App 20233 stdout: [b3169eb5-e1ea-41f5-a47b-5ffd70a9a44c] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [b3169eb5-e1ea-41f5-a47b-5ffd70a9a44c] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [b3169eb5-e1ea-41f5-a47b-5ffd70a9a44c] Completed 200 OK in 4ms (Views: 0.3ms | ActiveRecord: 0.4ms)
App 20233 stdout: [4cc8a48e-7096-43e9-ba4c-38b44aefef66] Started GET "/workflow_queue/lane_ids?step=dor%3AgisAssemblyWF%3Agenerate-content-metadata" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [4cc8a48e-7096-43e9-ba4c-38b44aefef66] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [4cc8a48e-7096-43e9-ba4c-38b44aefef66] Parameters: {"step"=>"dor:gisAssemblyWF:generate-content-metadata"}
App 20233 stdout: [4cc8a48e-7096-43e9-ba4c-38b44aefef66] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [4cc8a48e-7096-43e9-ba4c-38b44aefef66] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [4cc8a48e-7096-43e9-ba4c-38b44aefef66] Completed 200 OK in 2ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [347dc5a3-eddb-49a7-bccb-87f1cff60d02] Started GET "/workflow_queue/lane_ids?step=sdr%3AsdrIngestWF%3Aregister-sdr" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [347dc5a3-eddb-49a7-bccb-87f1cff60d02] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [347dc5a3-eddb-49a7-bccb-87f1cff60d02] Parameters: {"step"=>"sdr:sdrIngestWF:register-sdr"}
App 20233 stdout: [347dc5a3-eddb-49a7-bccb-87f1cff60d02] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [347dc5a3-eddb-49a7-bccb-87f1cff60d02] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [347dc5a3-eddb-49a7-bccb-87f1cff60d02] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [f27e5e9d-a7f0-442a-8789-741662a23dec] Started GET "/workflow_queue/lane_ids?step=sdr%3AsdrIngestWF%3Avalidate-bag" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [f27e5e9d-a7f0-442a-8789-741662a23dec] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [f27e5e9d-a7f0-442a-8789-741662a23dec] Parameters: {"step"=>"sdr:sdrIngestWF:validate-bag"}
App 20233 stdout: [f27e5e9d-a7f0-442a-8789-741662a23dec] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [f27e5e9d-a7f0-442a-8789-741662a23dec] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [f27e5e9d-a7f0-442a-8789-741662a23dec] Completed 200 OK in 2ms (Views: 0.9ms | ActiveRecord: 0.3ms)
App 20233 stdout: [b78df601-3562-4c90-ac2b-238b0d1451e5] Started GET "/workflow_queue/lane_ids?step=sdr%3AsdrIngestWF%3Atransfer-object" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [b78df601-3562-4c90-ac2b-238b0d1451e5] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [b78df601-3562-4c90-ac2b-238b0d1451e5] Parameters: {"step"=>"sdr:sdrIngestWF:transfer-object"}
App 20233 stdout: [b78df601-3562-4c90-ac2b-238b0d1451e5] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [b78df601-3562-4c90-ac2b-238b0d1451e5] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [b78df601-3562-4c90-ac2b-238b0d1451e5] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [e5b07d59-5998-4610-940b-24a1894b10eb] Started GET "/workflow_queue/lane_ids?step=sdr%3AsdrIngestWF%3Averify-agreement" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [e5b07d59-5998-4610-940b-24a1894b10eb] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [e5b07d59-5998-4610-940b-24a1894b10eb] Parameters: {"step"=>"sdr:sdrIngestWF:verify-agreement"}
App 20233 stdout: [e5b07d59-5998-4610-940b-24a1894b10eb] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [e5b07d59-5998-4610-940b-24a1894b10eb] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [e5b07d59-5998-4610-940b-24a1894b10eb] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.3ms)
App 20233 stdout: [2fd7621b-ba28-4347-95db-06a57a9eae26] Started GET "/workflow_queue/lane_ids?step=sdr%3AsdrIngestWF%3Aingest-cleanup" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [2fd7621b-ba28-4347-95db-06a57a9eae26] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [2fd7621b-ba28-4347-95db-06a57a9eae26] Parameters: {"step"=>"sdr:sdrIngestWF:ingest-cleanup"}
App 20233 stdout: [2fd7621b-ba28-4347-95db-06a57a9eae26] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [2fd7621b-ba28-4347-95db-06a57a9eae26] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [2fd7621b-ba28-4347-95db-06a57a9eae26] Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.3ms)
App 20233 stdout: [da5128d1-eb2e-4f3d-b7ea-9c5b138f81b5] Started GET "/workflow_queue/lane_ids?step=sdr%3AsdrIngestWF%3Acomplete-deposit" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [da5128d1-eb2e-4f3d-b7ea-9c5b138f81b5] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [da5128d1-eb2e-4f3d-b7ea-9c5b138f81b5] Parameters: {"step"=>"sdr:sdrIngestWF:complete-deposit"}
App 20233 stdout: [da5128d1-eb2e-4f3d-b7ea-9c5b138f81b5] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [da5128d1-eb2e-4f3d-b7ea-9c5b138f81b5] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [da5128d1-eb2e-4f3d-b7ea-9c5b138f81b5] Completed 200 OK in 4ms (Views: 0.3ms | ActiveRecord: 0.4ms)
App 20233 stdout: [96a37f45-a437-40de-bd6b-af173567ad17] Started GET "/workflow_queue/lane_ids?step=dor%3AetdSubmitWF%3Aregister-object" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [96a37f45-a437-40de-bd6b-af173567ad17] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [96a37f45-a437-40de-bd6b-af173567ad17] Parameters: {"step"=>"dor:etdSubmitWF:register-object"}
App 20233 stdout: [96a37f45-a437-40de-bd6b-af173567ad17] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [96a37f45-a437-40de-bd6b-af173567ad17] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [96a37f45-a437-40de-bd6b-af173567ad17] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [e41c27fc-f826-4a38-a046-e932d32c2537] Started GET "/workflow_queue/lane_ids?step=dor%3AetdSubmitWF%3Aother-metadata" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [e41c27fc-f826-4a38-a046-e932d32c2537] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [e41c27fc-f826-4a38-a046-e932d32c2537] Parameters: {"step"=>"dor:etdSubmitWF:other-metadata"}
App 20233 stdout: [e41c27fc-f826-4a38-a046-e932d32c2537] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [e41c27fc-f826-4a38-a046-e932d32c2537] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [e41c27fc-f826-4a38-a046-e932d32c2537] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [e15c613f-f8f3-4bb8-9099-f7c66adec258] Started GET "/workflow_queue/lane_ids?step=dor%3AetdSubmitWF%3Aregistrar-approval" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [e15c613f-f8f3-4bb8-9099-f7c66adec258] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [e15c613f-f8f3-4bb8-9099-f7c66adec258] Parameters: {"step"=>"dor:etdSubmitWF:registrar-approval"}
App 20233 stdout: [e15c613f-f8f3-4bb8-9099-f7c66adec258] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [e15c613f-f8f3-4bb8-9099-f7c66adec258] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [e15c613f-f8f3-4bb8-9099-f7c66adec258] Completed 200 OK in 2ms (Views: 0.4ms | ActiveRecord: 0.5ms)
App 20233 stdout: [6dc4f665-bc0c-4a38-a5b6-3e735e46d2cf] Started GET "/workflow_queue/lane_ids?step=dor%3AetdSubmitWF%3Asubmit" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [6dc4f665-bc0c-4a38-a5b6-3e735e46d2cf] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [6dc4f665-bc0c-4a38-a5b6-3e735e46d2cf] Parameters: {"step"=>"dor:etdSubmitWF:submit"}
App 20233 stdout: [6dc4f665-bc0c-4a38-a5b6-3e735e46d2cf] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [6dc4f665-bc0c-4a38-a5b6-3e735e46d2cf] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [6dc4f665-bc0c-4a38-a5b6-3e735e46d2cf] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.5ms)
App 20233 stdout: [8f6023c9-cd50-4328-8d41-87faecf90946] Started GET "/workflow_queue/lane_ids?step=dor%3AetdSubmitWF%3Areader-approval" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [8f6023c9-cd50-4328-8d41-87faecf90946] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [8f6023c9-cd50-4328-8d41-87faecf90946] Parameters: {"step"=>"dor:etdSubmitWF:reader-approval"}
App 20233 stdout: [8f6023c9-cd50-4328-8d41-87faecf90946] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [8f6023c9-cd50-4328-8d41-87faecf90946] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [8f6023c9-cd50-4328-8d41-87faecf90946] Completed 200 OK in 2ms (Views: 0.4ms | ActiveRecord: 0.9ms)
App 20233 stdout: [5a60ca09-2e8e-4d49-870f-8891f179ab00] Started GET "/workflow_queue/lane_ids?step=dor%3AetdSubmitWF%3Astart-accession" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [5a60ca09-2e8e-4d49-870f-8891f179ab00] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [5a60ca09-2e8e-4d49-870f-8891f179ab00] Parameters: {"step"=>"dor:etdSubmitWF:start-accession"}
App 20233 stdout: [5a60ca09-2e8e-4d49-870f-8891f179ab00] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [5a60ca09-2e8e-4d49-870f-8891f179ab00] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [5a60ca09-2e8e-4d49-870f-8891f179ab00] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [3873249b-618b-4a76-a296-bfda39bb9f17] Started GET "/workflow_queue/lane_ids?step=dor%3AgisAssemblyWF%3Aload-geo-metadata" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [3873249b-618b-4a76-a296-bfda39bb9f17] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [3873249b-618b-4a76-a296-bfda39bb9f17] Parameters: {"step"=>"dor:gisAssemblyWF:load-geo-metadata"}
App 20233 stdout: [3873249b-618b-4a76-a296-bfda39bb9f17] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [3873249b-618b-4a76-a296-bfda39bb9f17] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [3873249b-618b-4a76-a296-bfda39bb9f17] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [ecd8f9c0-2589-4f23-83c9-51ddbfbee9bf] Started GET "/workflow_queue/lane_ids?step=dor%3AwasDisseminationWF%3Astart-special-dissemination" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [ecd8f9c0-2589-4f23-83c9-51ddbfbee9bf] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [ecd8f9c0-2589-4f23-83c9-51ddbfbee9bf] Parameters: {"step"=>"dor:wasDisseminationWF:start-special-dissemination"}
App 20233 stdout: [ecd8f9c0-2589-4f23-83c9-51ddbfbee9bf] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [ecd8f9c0-2589-4f23-83c9-51ddbfbee9bf] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [ecd8f9c0-2589-4f23-83c9-51ddbfbee9bf] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.3ms)
App 20233 stdout: [83ceb227-84e4-4e18-a890-cef11ebb7821] Started GET "/workflow_queue/lane_ids?step=dor%3AgisAssemblyWF%3Afinish-gis-assembly-workflow" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [83ceb227-84e4-4e18-a890-cef11ebb7821] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [83ceb227-84e4-4e18-a890-cef11ebb7821] Parameters: {"step"=>"dor:gisAssemblyWF:finish-gis-assembly-workflow"}
App 20233 stdout: [83ceb227-84e4-4e18-a890-cef11ebb7821] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [83ceb227-84e4-4e18-a890-cef11ebb7821] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [83ceb227-84e4-4e18-a890-cef11ebb7821] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.3ms)
App 20233 stdout: [2d459494-402e-4a49-8742-88bb18dba9f6] Started GET "/workflow_queue/lane_ids?step=dor%3AgisDiscoveryWF%3Aload-geoblacklight" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [2d459494-402e-4a49-8742-88bb18dba9f6] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [2d459494-402e-4a49-8742-88bb18dba9f6] Parameters: {"step"=>"dor:gisDiscoveryWF:load-geoblacklight"}
App 20233 stdout: [2d459494-402e-4a49-8742-88bb18dba9f6] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [2d459494-402e-4a49-8742-88bb18dba9f6] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [2d459494-402e-4a49-8742-88bb18dba9f6] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.3ms)
App 20233 stdout: [f8c0d511-1a90-476e-b6d8-2f60c6403d83] Started GET "/workflow_queue/lane_ids?step=dor%3AgisDiscoveryWF%3Aexport-opengeometadata" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [f8c0d511-1a90-476e-b6d8-2f60c6403d83] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [f8c0d511-1a90-476e-b6d8-2f60c6403d83] Parameters: {"step"=>"dor:gisDiscoveryWF:export-opengeometadata"}
App 20233 stdout: [f8c0d511-1a90-476e-b6d8-2f60c6403d83] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [f8c0d511-1a90-476e-b6d8-2f60c6403d83] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [f8c0d511-1a90-476e-b6d8-2f60c6403d83] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.4ms)
App 20233 stdout: [99991a0e-72ab-4630-bc65-1a8862fd51df] Started GET "/workflow_queue/lane_ids?step=dor%3AgisDiscoveryWF%3Afinish-gis-discovery-workflow" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [99991a0e-72ab-4630-bc65-1a8862fd51df] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [99991a0e-72ab-4630-bc65-1a8862fd51df] Parameters: {"step"=>"dor:gisDiscoveryWF:finish-gis-discovery-workflow"}
App 20233 stdout: [99991a0e-72ab-4630-bc65-1a8862fd51df] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [99991a0e-72ab-4630-bc65-1a8862fd51df] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [99991a0e-72ab-4630-bc65-1a8862fd51df] Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.3ms)
App 20233 stdout: [6f460610-998d-45f7-a508-2dff50843548] Started GET "/workflow_queue/lane_ids?step=dor%3AgisDiscoveryWF%3Agenerate-geoblacklight" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [6f460610-998d-45f7-a508-2dff50843548] Processing by WorkflowQueuesController#lane_ids as XML
App 20233 stdout: [6f460610-998d-45f7-a508-2dff50843548] Parameters: {"step"=>"dor:gisDiscoveryWF:generate-geoblacklight"}
App 20233 stdout: [6f460610-998d-45f7-a508-2dff50843548] Rendering workflow_queues/lane_ids.xml.builder
App 20233 stdout: [6f460610-998d-45f7-a508-2dff50843548] Rendered workflow_queues/lane_ids.xml.builder (0.1ms)
App 20233 stdout: [6f460610-998d-45f7-a508-2dff50843548] Completed 200 OK in 2ms (Views: 0.3ms | ActiveRecord: 0.3ms)
App 20233 stdout: [b159fe48-a60a-4449-a076-087a7ca3d484] Started GET "/workflow_queue?completed=sdr%3ApreservationIngestWF%3Aupdate-moab&lane-id=default&limit=100&waiting=sdr%3ApreservationIngestWF%3Aupdate-catalog" for 171.67.35.38 at 2019-03-19 08:43:40 -0700
App 20233 stdout: [b159fe48-a60a-4449-a076-087a7ca3d484] Processing by WorkflowQueuesController#show as XML
App 20233 stdout: [b159fe48-a60a-4449-a076-087a7ca3d484] Parameters: {"completed"=>"sdr:preservationIngestWF:update-moab", "lane-id"=>"default", "limit"=>"100", "waiting"=>"sdr:preservationIngestWF:update-catalog"}
App 20285 stdout: [17b17870-b711-404b-a729-cafa047e3281] Rendering workflow_queues/show.xml.builder
App 20285 stdout: [17b17870-b711-404b-a729-cafa047e3281] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20285 stdout: [17b17870-b711-404b-a729-cafa047e3281] Completed 200 OK in 1853ms (Views: 0.4ms | ActiveRecord: 1850.7ms)
App 20285 stdout: [af55375d-34b1-49d7-bd8c-a5a70c3cc23c] Started GET "/workflow_queue?completed=sdr%3ApreservationIngestWF%3Atransfer-object&lane-id=default&limit=100&waiting=sdr%3ApreservationIngestWF%3Avalidate-bag" for 171.67.35.38 at 2019-03-19 08:43:42 -0700
App 20285 stdout: [af55375d-34b1-49d7-bd8c-a5a70c3cc23c] Processing by WorkflowQueuesController#show as XML
App 20285 stdout: [af55375d-34b1-49d7-bd8c-a5a70c3cc23c] Parameters: {"completed"=>"sdr:preservationIngestWF:transfer-object", "lane-id"=>"default", "limit"=>"100", "waiting"=>"sdr:preservationIngestWF:validate-bag"}
App 20277 stdout: [fdadf4eb-da8a-42df-9658-26d87ff52eb8] Rendering workflow_queues/show.xml.builder
App 20277 stdout: [fdadf4eb-da8a-42df-9658-26d87ff52eb8] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20277 stdout: [fdadf4eb-da8a-42df-9658-26d87ff52eb8] Completed 200 OK in 127915ms (Views: 0.5ms | ActiveRecord: 127912.6ms)
App 20277 stdout: [df3fb96f-fea8-4f4b-8416-5b2446c20fd9] Started GET "/workflow_queue?completed=sdr%3ApreservationIngestWF%3Avalidate-bag&lane-id=default&limit=100&waiting=sdr%3ApreservationIngestWF%3Averify-apo" for 171.67.35.38 at 2019-03-19 08:44:45 -0700
App 20277 stdout: [df3fb96f-fea8-4f4b-8416-5b2446c20fd9] Processing by WorkflowQueuesController#show as XML
App 20277 stdout: [df3fb96f-fea8-4f4b-8416-5b2446c20fd9] Parameters: {"completed"=>"sdr:preservationIngestWF:validate-bag", "lane-id"=>"default", "limit"=>"100", "waiting"=>"sdr:preservationIngestWF:verify-apo"}
App 20241 stdout: [02ae7743-8435-4c97-b720-034cf84fb3c4] Rendering workflow_queues/show.xml.builder
App 20241 stdout: [02ae7743-8435-4c97-b720-034cf84fb3c4] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20241 stdout: [02ae7743-8435-4c97-b720-034cf84fb3c4] Completed 200 OK in 128307ms (Views: 0.5ms | ActiveRecord: 128304.8ms)
App 20241 stdout: [f9ad9435-82ba-40ef-816a-c7f1e9354ea6] Started GET "/workflow_queue?completed=sdr%3ApreservationIngestWF%3Aupdate-catalog&lane-id=default&limit=100&waiting=sdr%3ApreservationIngestWF%3Acomplete-ingest" for 171.67.35.38 at 2019-03-19 08:44:45 -0700
App 20241 stdout: [f9ad9435-82ba-40ef-816a-c7f1e9354ea6] Processing by WorkflowQueuesController#show as XML
App 20241 stdout: [f9ad9435-82ba-40ef-816a-c7f1e9354ea6] Parameters: {"completed"=>"sdr:preservationIngestWF:update-catalog", "lane-id"=>"default", "limit"=>"100", "waiting"=>"sdr:preservationIngestWF:complete-ingest"}
App 20575 stdout: [8565f17d-1f9a-49d6-b94f-e4dc7828ab1f] Rendering workflow_queues/show.xml.builder
App 20575 stdout: [8565f17d-1f9a-49d6-b94f-e4dc7828ab1f] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20575 stdout: [8565f17d-1f9a-49d6-b94f-e4dc7828ab1f] Completed 200 OK in 128343ms (Views: 0.5ms | ActiveRecord: 128341.2ms)
App 20575 stdout: [23869c3f-4d6c-4f4e-9210-77178833d3cf] Started GET "/workflow_queue?completed=sdr%3ApreservationIngestWF%3Averify-apo&lane-id=default&limit=100&waiting=sdr%3ApreservationIngestWF%3Aupdate-moab" for 171.67.35.38 at 2019-03-19 08:44:45 -0700
App 20575 stdout: [23869c3f-4d6c-4f4e-9210-77178833d3cf] Processing by WorkflowQueuesController#show as XML
App 20575 stdout: [23869c3f-4d6c-4f4e-9210-77178833d3cf] Parameters: {"completed"=>"sdr:preservationIngestWF:verify-apo", "lane-id"=>"default", "limit"=>"100", "waiting"=>"sdr:preservationIngestWF:update-moab"}
App 20249 stdout: [10e998eb-0ecc-42fc-8d11-92d9504e7b0d] Rendering workflow_queues/show.xml.builder
App 20249 stdout: [10e998eb-0ecc-42fc-8d11-92d9504e7b0d] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20249 stdout: [10e998eb-0ecc-42fc-8d11-92d9504e7b0d] Completed 200 OK in 129048ms (Views: 0.5ms | ActiveRecord: 129046.1ms)
App 20249 stdout: [c315fbfb-5c16-4356-85b9-c057d3eaec62] Started GET "/workflow_queue?completed=dor%3AassemblyWF%3Achecksum-compute&lane-id=default&limit=100&waiting=dor%3AassemblyWF%3Aexif-collect" for 171.67.35.38 at 2019-03-19 08:44:46 -0700
App 20249 stdout: [c315fbfb-5c16-4356-85b9-c057d3eaec62] Processing by WorkflowQueuesController#show as XML
App 20249 stdout: [c315fbfb-5c16-4356-85b9-c057d3eaec62] Parameters: {"completed"=>"dor:assemblyWF:checksum-compute", "lane-id"=>"default", "limit"=>"100", "waiting"=>"dor:assemblyWF:exif-collect"}
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207] Rendering workflow_queues/show.xml.builder
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207] Rendered workflow_queues/show.xml.builder (0.1ms)
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207] Completed 200 OK in 128715ms (Views: 0.5ms | ActiveRecord: 128712.4ms)
App 20336 stdout: [2a925e58-7c29-4b1d-b828-6817cfd4a1aa] Started GET "/workflow_queue?completed=dor%3AassemblyWF%3Acontent-metadata-create&lane-id=default&limit=100&waiting=dor%3AassemblyWF%3Ajp2-create" for 171.67.35.38 at 2019-03-19 08:44:46 -0700
App 20336 stdout: [2a925e58-7c29-4b1d-b828-6817cfd4a1aa] Processing by WorkflowQueuesController#show as XML
App 20336 stdout: [2a925e58-7c29-4b1d-b828-6817cfd4a1aa] Parameters: {"completed"=>"dor:assemblyWF:content-metadata-create", "lane-id"=>"default", "limit"=>"100", "waiting"=>"dor:assemblyWF:jp2-create"}
[ 2019-03-19 08:44:57.6503 20027/7f6ace01c700 age/Cor/Con/CheckoutSession.cpp:269 ]: [Client 3-633] Returning HTTP 503 due to: Request queue full (configured max. size: 100)
[ 2019-03-19 08:44:57.6531 20027/7f6acd59a700 age/Cor/Con/CheckoutSession.cpp:269 ]: [Client 4-633] Returning HTTP 503 due to: Request queue full (configured max. size: 100)
```
@jcoyne
Copy link
Author

jcoyne commented Mar 19, 2019

here's a pathological request

grep 6da57231-8da7-4e94-9622-2078ef6d2207 /var/log/httpd/error_log
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207] Started GET "/workflow_queue?completed=sdr%3ApreservationIngestWF%3Aupdate-catalog&lane-id=default&limit=100&waiting=sdr%3ApreservationIngestWF%3Acomplete-ingest" for 171.67.35.38 at 2019-03-19 08:42:37 -0700
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207] Processing by WorkflowQueuesController#show as XML
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207]   Parameters: {"completed"=>"sdr:preservationIngestWF:update-catalog", "lane-id"=>"default", "limit"=>"100", "waiting"=>"sdr:preservationIngestWF:complete-ingest"}
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207]   Rendering workflow_queues/show.xml.builder
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207]   Rendered workflow_queues/show.xml.builder (0.1ms)
App 20336 stdout: [6da57231-8da7-4e94-9622-2078ef6d2207] Completed 200 OK in 128715ms (Views: 0.5ms | ActiveRecord: 128712.4ms)

@jcoyne
Copy link
Author

jcoyne commented Mar 19, 2019 

waiting_scope = WorkflowStep.waiting.where(repository: 'sdr', workflow: 'preservationIngestWF', process: 'complete-ingest').where(lane_id: 'default').select(:druid, :version)
completed_scope = WorkflowStep.complete.where(repository: 'sdr', workflow: 'preservationIngestWF', process: 'update-catalog').where(lane_id: 'default').group(:druid, :version).having('version = MAX(version)').select(:druid, :version)
scopes = [waiting_scope, completed_scope]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment