Skip to content

Instantly share code, notes, and snippets.

@iperdomo
Created July 8, 2015 13:11
Show Gist options
  • Save iperdomo/23ef4efec9b102d5e87f to your computer and use it in GitHub Desktop.
Save iperdomo/23ef4efec9b102d5e87f to your computer and use it in GitHub Desktop.
CartoDB geojson import
2015-07-08 12:55:42 UTC: Running on server cartodb.akvotest.org with PID: 1372
2015-07-08 12:55:42 UTC: new_importer()
2015-07-08 12:55:42 UTC: Fetching datasource public_url metadata for item id /home/cartodb/cartodb/public/uploads/26e5519ad5a171f9b690/output.geojson
2015-07-08 12:55:42 UTC: File will be downloaded from /home/cartodb/cartodb/public/uploads/26e5519ad5a171f9b690/output.geojson
2015-07-08 12:55:42 UTC: Before importer run
2015-07-08 12:55:42 UTC: Starting import for /home/cartodb/cartodb/public/uploads/26e5519ad5a171f9b690/output.geojson
2015-07-08 12:55:42 UTC: Unpacking /home/cartodb/cartodb/public/uploads/26e5519ad5a171f9b690/output.geojson
2015-07-08 12:55:42 UTC: Filename: /tmp/imports/20150708-1372-1cwrblc/output.geojson Size (bytes): 17439
2015-07-08 12:55:42 UTC: Importing data from /tmp/imports/20150708-1372-1cwrblc/output.geojson
2015-07-08 12:55:42 UTC: File-based import load
2015-07-08 12:55:42 UTC: Detected encoding UTF-8
2015-07-08 12:55:42 UTC: Using database connection with {:adapter=>"postgres"
:encoding=>"unicode"
:host=>"localhost"
:port=>5432
:database=>"cartodb_user_7de093b8-c0d9-42b7-b80e-dd79853ad893_db"
:username=>"postgres"
:conn_validator_timeout=>900
:user=>"cartodb_user_7de093b8-c0d9-42b7-b80e-dd79853ad893"}
2015-07-08 12:55:44 UTC: ogr2ogr call: OSM_USE_CUSTOM_INDEXING=NO PG_USE_COPY=YES PGCLIENTENCODING=UTF-8 /usr/bin/ogr2ogr -f PostgreSQL PG:"host=localhost port=5432 user=cartodb_user_7de093b8-c0d9-42b7-b80e-dd79853ad893 dbname=cartodb_user_7de093b8-c0d9-42b7-b80e-dd79853ad893_db password=9df3bcaebebdfc9cffdd161bee812fe71c276a23cartodb_user_7de093b8-c0d9-42b7-b80e-dd79853ad893" -t_srs EPSG:4326 -lco DIM=2 -lco PRECISION=NO /tmp/imports/20150708-1372-1cwrblc/output.geojson -nln cdb_importer.importer_a3ecd37e257011e58a4bda0009609cfd -nlt PROMOTE_TO_MULTI
2015-07-08 12:55:44 UTC: ogr2ogr output:
2015-07-08 12:55:44 UTC: ogr2ogr exit code: 0
2015-07-08 12:55:44 UTC: Georeferencing...
2015-07-08 12:55:44 UTC: Disabling autovacuum for "cdb_importer"."importer_a3ecd37e257011e58a4bda0009609cfd"
2015-07-08 12:55:44 UTC: Creating the_geom from wkb_geometry column
2015-07-08 12:55:44 UTC: replace empty strings by nulls?
2015-07-08 12:55:44 UTC: no string column found
nothing replaced
2015-07-08 12:55:44 UTC: geometrifying
2015-07-08 12:55:44 UTC: casting wkb_geometry to geometry
2015-07-08 12:55:44 UTC: Converting to 2D point
2015-07-08 12:55:44 UTC: Total processed: 34
2015-07-08 12:55:44 UTC: Total processed: 34
2015-07-08 12:55:44 UTC: FINISHED: Converting to 2D point
2015-07-08 12:55:44 UTC: geometrified
2015-07-08 12:55:44 UTC: Renaming column wkb_geometry TO the_geom
2015-07-08 12:55:44 UTC: Enabling autovacuum for "cdb_importer"."importer_a3ecd37e257011e58a4bda0009609cfd"
2015-07-08 12:55:44 UTC: Georeferenced
2015-07-08 12:55:44 UTC: Finished importing data from /tmp/imports/20150708-1372-1cwrblc/output.geojson
2015-07-08 12:55:44 UTC: Proceeding to register
2015-07-08 12:55:44 UTC: Before renaming from importer_a3ecd37e257011e58a4bda0009609cfd to output
2015-07-08 12:55:44 UTC: Before moving schema 'output_2' from cdb_importer to public
2015-07-08 12:55:44 UTC: Before persisting metadata 'output_2' data_import_id: 4a109cb3-d844-4046-a668-2be1dcaaff31
2015-07-08 13:00:45 UTC: Exception: PG::Error: ERROR: canceling statement due to statement timeout
CONTEXT: SQL statement "SELECT public.cdb_invalidate_varnish('output_2')"
PL/pgSQL function _cdb_tablemetadata_updated() line 40 at EXECUTE statement
SQL statement "WITH upsert AS (
UPDATE cartodb.cdb_tablemetadata
SET updated_at = NOW()
WHERE tabname = tablename
RETURNING *
)
INSERT INTO cartodb.cdb_tablemetadata (tabname
updated_at)
SELECT tablename
NOW()
WHERE NOT EXISTS (SELECT * FROM upsert)"
PL/pgSQL function cdb_tablemetadatatouch(regclass) line 3 at SQL statement
2015-07-08 13:00:45 UTC: ["/home/cartodb/cartodb/app/connectors/importer.rb:72:in `rescue in register'"
"/home/cartodb/cartodb/app/connectors/importer.rb:58:in `register'"
"/home/cartodb/cartodb/app/connectors/importer.rb:50:in `block in run'"
"/home/cartodb/cartodb/app/connectors/importer.rb:49:in `each'"
"/home/cartodb/cartodb/app/connectors/importer.rb:49:in `run'"
"/home/cartodb/cartodb/app/models/data_import.rb:599:in `new_importer'"
"/home/cartodb/cartodb/app/models/data_import.rb:342:in `dispatch'"
"/home/cartodb/cartodb/app/models/data_import.rb:159:in `run_import!'"
"/home/cartodb/cartodb/lib/resque/importer_jobs.rb:9:in `block in perform'"
"/home/cartodb/cartodb/lib/resque/base_job.rb:21:in `call'"
"/home/cartodb/cartodb/lib/resque/base_job.rb:21:in `run_action'"
"/home/cartodb/cartodb/lib/resque/importer_jobs.rb:9:in `perform'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/job.rb:240:in `block (3 levels) in perform'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-metrics-0.1.1/lib/resque/metrics.rb:353:in `around_perform_metrics'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/job.rb:239:in `block (2 levels) in perform'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/job.rb:247:in `call'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/job.rb:247:in `perform'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/worker.rb:250:in `perform'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/worker.rb:189:in `block in work'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/worker.rb:166:in `loop'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/worker.rb:166:in `work'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/resque-1.25.2/lib/resque/tasks.rb:41:in `block (2 levels) in <top (required)>'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:205:in `call'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:205:in `block in execute'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:200:in `each'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:200:in `execute'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:158:in `block in invoke_with_call_chain'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:151:in `invoke_with_call_chain'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/task.rb:144:in `invoke'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:116:in `invoke_task'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:94:in `block (2 levels) in top_level'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:94:in `each'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:94:in `block in top_level'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:88:in `top_level'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:66:in `block in run'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/lib/ruby/1.9.1/rake/application.rb:63:in `run'"
"/home/cartodb/.rbenv/versions/1.9.3-p551/bin/rake:32:in `<main>'"]
2015-07-08 13:00:46 UTC: ERROR!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment