output of "python3 test.py" after fresh install of openaddresses/machine "local development" machine

test-1480543543.log

/usr/local/lib/python3.4/dist-packages/pq/utils.py:132: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(notice)
...........2016-11-30 22:05:46,936   ERROR: Failed to check merged status of openaddresses/openaddresses gobbledygook: 'base_commit'
..........................2016-11-30 22:06:06,566 WARNING: Skipping file with unknown conform: test
2016-11-30 22:06:06,567 WARNING: Skipping file with unknown conform: test
..................2016-11-30 22:06:06,605 WARNING: Conform named bar.txt as file but we could not find it.
..2016-11-30 22:06:06,618 WARNING: No GDB found in ['nope.txt']
2016-11-30 22:06:06,621 WARNING: Multiple GDBs found, but source has no file attribute.
.2016-11-30 22:06:06,626 WARNING: No JSON found in ['nope.txt']
2016-11-30 22:06:06,627 WARNING: Found more than one JSON file in source, can't pick one
.2016-11-30 22:06:06,629 WARNING: No shapefiles found in ['nope.txt']
2016-11-30 22:06:06,630 WARNING: Multiple shapefiles found, but source has no file attribute.
2016-11-30 22:06:06,631 WARNING: Unknown source conform type broken
.....................................2016-11-30 22:06:07,401 WARNING: Found http://s3.amazonaws.com/data-testpreviews.openaddresses.io/runs/10/preview.png in an existing comment: https://api.github.com/repos/openaddresses/hooked-on-sources/issues/comments/259845585
...2016-11-30 22:06:07,559 WARNING: Mismatched /hook signatures: sha1=79c627d0a3f18ffb913fbbca70f6d9d983aa7c83 vs. sha1=80c4b24a90baa10a63811bd05f396b658a5759bc, sha1=94c05fcb02b3790f99dd17248070863c63ac685c
.Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 444, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 725, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 632, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (botocore 1.4.80 (/usr/local/lib/python3.4/dist-packages), Requirement.parse('botocore==1.4.79'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/openaddr-process-one", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: botocore==1.4.79
2016-11-30 22:06:12,941   ERROR: Failed to check merged status of openaddresses/hooked-on-sources badjson20f08890960f50f863626e1062f922522: 'base_commit'
...2016-11-30 22:06:13,185 WARNING: No files
...2016-11-30 22:06:13,379 WARNING: No /hook signature provided
.2016-11-30 22:06:13,545   ERROR: Failed to check merged status of openaddresses/hooked-on-sources e91fbc420f08890960f50f863626e1062f922522: Expecting ',' delimiter or '}': line 1 column 64956 (char 64955)
...2016-11-30 22:06:13,844   ERROR: Oops
Traceback (most recent call last):
  File "/vagrant/openaddr/ci/__init__.py", line 117, in process_github_payload
    is_rerun, owner, repo, status_url, comments_url)
  File "/vagrant/openaddr/ci/__init__.py", line 735, in create_queued_job
    task_files = add_files_to_queue(queue, job_id, job_url, files, commit_sha, rerun)
  File "/usr/local/lib/python3.4/dist-packages/mock/mock.py", line 1062, in __call__
    return _mock_self._mock_call(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/mock/mock.py", line 1128, in _mock_call
    ret_val = effect(*args, **kwargs)
  File "/vagrant/openaddr/tests/ci.py", line 1651, in job_queue_fails
    raise Exception('Nope')
Exception: Nope
.2016-11-30 22:06:14,034   ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
2016-11-30 22:06:14,078   ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
.2016-11-30 22:06:14,251   ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
2016-11-30 22:06:14,286   ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
...2016-11-30 22:06:17,197 WARNING: Source is missing a conform object
2016-11-30 22:06:17,198 WARNING: Error doing conform; skipping
Traceback (most recent call last):
  File "/vagrant/openaddr/__init__.py", line 185, in conform
    csv_path, addr_count = task4.convert(data, decompressed_paths, workdir)
  File "/vagrant/openaddr/conform.py", line 521, in convert
    source_path = find_source_path(source_definition, source_paths)
  File "/vagrant/openaddr/conform.py", line 404, in find_source_path
    conform = source_definition["conform"]
KeyError: 'conform'
2016-11-30 22:06:17,212 WARNING: Nothing processed
.2016-11-30 22:06:17,418 WARNING: Could not download source data
2016-11-30 22:06:17,419 WARNING: Error in process_one.process()
Traceback (most recent call last):
  File "/vagrant/openaddr/process_one.py", line 56, in process
    cache_result = cache(temp_src, temp_dir, extras)
  File "/vagrant/openaddr/__init__.py", line 109, in cache
    downloaded_files = task.download(source_urls, workdir, data.get('conform'))
  File "/vagrant/openaddr/cache.py", line 280, in download
    raise DownloadError('{} response from {}'.format(resp.status_code, source_url))
openaddr.cache.DownloadError: 404 response from http://www.ci.berkeley.ca.us/uploadedFiles/IT/GIS/No-Parcels.zip
.2016-11-30 22:06:17,876 WARNING: Nothing previewed
..2016-11-30 22:06:18,222 WARNING: Nothing previewed
.2016-11-30 22:06:18,387 WARNING: Nothing previewed
.2016-11-30 22:06:18,548 WARNING: Nothing previewed
.E2016-11-30 22:06:18,939 WARNING: Nothing previewed
.2016-11-30 22:06:19,088 WARNING: Nothing previewed
.2016-11-30 22:06:19,239 WARNING: Error doing conform; skipping
Traceback (most recent call last):
  File "/vagrant/openaddr/__init__.py", line 185, in conform
    csv_path, addr_count = task4.convert(data, decompressed_paths, workdir)
  File "/vagrant/openaddr/conform.py", line 525, in convert
    rc = conform_cli(source_definition, source_path, dest_path)
  File "/vagrant/openaddr/conform.py", line 1131, in conform_cli
    extract_to_source_csv(source_definition, source_path, extract_path)
  File "/vagrant/openaddr/conform.py", line 1080, in extract_to_source_csv
    csv_source_to_csv(source_definition, source_path, extract_path)
  File "/vagrant/openaddr/conform.py", line 684, in csv_source_to_csv
    with csvopen(source_path, 'r', encoding=enc) as source_fp:
  File "/vagrant/openaddr/compat.py", line 125, in csvopen
    return io.open(filename, mode=mode, encoding=encoding)
IsADirectoryError: [Errno 21] Is a directory: '/tmp/testOA-vtpocf_o/process_one-bnusyii4/conform-1hahq6qo/unzipped/07000-12.0a'
2016-11-30 22:06:19,247 WARNING: Nothing processed
.2016-11-30 22:06:19,420 WARNING: Nothing previewed
.2016-11-30 22:06:19,592 WARNING: Nothing previewed
.2016-11-30 22:06:19,799 WARNING: Nothing previewed
.2016-11-30 22:06:19,975 WARNING: Nothing previewed
.2016-11-30 22:06:20,123 WARNING: Nothing previewed
.2016-11-30 22:06:20,290 WARNING: Nothing previewed
.2016-11-30 22:06:20,461 WARNING: Unknown source conform type broken
2016-11-30 22:06:20,463 WARNING: Nothing processed
../vagrant/openaddr/conform.py:611: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  _L.warn("No encoding given and OGR couldn't guess. Trying ISO-8859-1, YOLO!")
2016-11-30 22:06:20,756 WARNING: No encoding given and OGR couldn't guess. Trying ISO-8859-1, YOLO!
2016-11-30 22:06:20,930 WARNING: Nothing previewed
.2016-11-30 22:06:21,172 WARNING: Nothing previewed
.2016-11-30 22:06:21,327 WARNING: Nothing previewed
.2016-11-30 22:06:21,871 WARNING: Nothing previewed
.2016-11-30 22:06:22,146 WARNING: Nothing previewed
..2016-11-30 22:06:24,607 WARNING: Error doing excerpt; skipping
Traceback (most recent call last):
  File "/vagrant/openaddr/__init__.py", line 176, in conform
    data_sample, geometry_type = task3.excerpt(decompressed_paths, workdir, conform)
  File "/vagrant/openaddr/conform.py", line 280, in excerpt
    return ExcerptDataTask._excerpt_csv_file(data_path, encoding, csvsplit)
  File "/vagrant/openaddr/conform.py", line 357, in _excerpt_csv_file
    data_sample = [row for (row, _) in zip(input, range(6))]
  File "/vagrant/openaddr/conform.py", line 357, in <listcomp>
    data_sample = [row for (row, _) in zip(input, range(6))]
_csv.Error: field larger than field limit (1)
2016-11-30 22:06:24,612 WARNING: Error doing conform; skipping
Traceback (most recent call last):
  File "/vagrant/openaddr/__init__.py", line 185, in conform
    csv_path, addr_count = task4.convert(data, decompressed_paths, workdir)
  File "/vagrant/openaddr/conform.py", line 525, in convert
    rc = conform_cli(source_definition, source_path, dest_path)
  File "/vagrant/openaddr/conform.py", line 1131, in conform_cli
    extract_to_source_csv(source_definition, source_path, extract_path)
  File "/vagrant/openaddr/conform.py", line 1085, in extract_to_source_csv
    csv_source_to_csv(source_definition, source_path, extract_path)
  File "/vagrant/openaddr/conform.py", line 713, in csv_source_to_csv
    num_fields = len(reader.fieldnames)
  File "/usr/lib/python3.4/csv.py", line 96, in fieldnames
    self._fieldnames = next(self.reader)
_csv.Error: field larger than field limit (1)
2016-11-30 22:06:24,619 WARNING: Nothing processed
2016-11-30 22:06:24,642 WARNING: Nothing previewed
.2016-11-30 22:06:24,792 WARNING: Nothing previewed
.2016-11-30 22:06:24,971 WARNING: Nothing previewed
.2016-11-30 22:06:25,110 WARNING: Nothing previewed
....................................2016-11-30 22:06:25,160   ERROR: Failed to download http://s3.amazonaws.com/openaddresses/000.csv
...../vagrant/openaddr/parcels/utils.py:73: ResourceWarning: unclosed file <_io.TextIOWrapper name='/vagrant/openaddr/tests/parcels/sources/us/ca/berkeley.json' mode='r' encoding='UTF-8'>
  source_json = json.loads(open('{}/sources/{}'.format(config.openaddr_dir, source)).read())
..../vagrant/openaddr/parcels/utils.py:88: ResourceWarning: unclosed file <_io.TextIOWrapper name='/vagrant/openaddr/tests/parcels/sources/us/ca/berkeley.json' mode='r' encoding='UTF-8'>
  source_json = json.loads(open('{}/sources/{}'.format(config.openaddr_dir, source)).read())
............................../usr/lib/python3.4/csv.py:118: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/output-kpwhhwqq/us/ca/berkeley.csv' mode='w' encoding='UTF-8'>
  d = dict(zip(self.fieldnames, row))
......2016-11-30 22:06:48,535 WARNING: Got an error from ftp02.portlandoregon.gov: 'NoneType' does not support the buffer interface
2016-11-30 22:06:48,537 WARNING: Got an error from ftp02.portlandoregon.gov: 'NoneType' does not support the buffer interface
........
======================================================================
ERROR: test_single_fr_lareunion (openaddr.tests.TestOA)
Test complete process_one.process on data that uses non-UTF8 encoding (issue #136)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant/openaddr/tests/__init__.py", line 810, in test_single_fr_lareunion
    state_path = process_one.process(source, self.testdir, False)
  File "/vagrant/openaddr/process_one.py", line 41, in process
    copy(source, temp_src)
  File "/usr/lib/python3.4/shutil.py", line 229, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.4/shutil.py", line 108, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/testOA-yx1arta6/sources/fr/la-réunion.json'

----------------------------------------------------------------------
Ran 232 tests in 64.476s

FAILED (errors=1)
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip