Skip to content

Instantly share code, notes, and snippets.

View twobraids's full-sized avatar

K Lars Lohn twobraids

View GitHub Profile
@twobraids
twobraids / gist:6e65f212cda90787c489
Created July 24, 2015 14:56
submitter as processor
Jul 24 14:55:41 i-aaee0d5c systemd: socorro-submitter.service: main process exited, code=exited, status=12/n/a
Jul 24 14:55:41 i-aaee0d5c systemd: Stopped Socorro Processor.
Jul 24 14:55:41 i-aaee0d5c systemd: Unit socorro-submitter.service entered failed state.
@twobraids
twobraids / reprocessing
Created August 24, 2015 22:54
push directly into reprocessing queue
$ . /data/socorro/socorro-virtualenv/bin/activate
$ envconsul -prefix socorro/common -prefix socorro/processor socorro submitter --destination.crashstorage_class=socorro.external.rabbitmq.crashstorage.RabbitMQCrashStorage --destination.routing_key=socorro.reprocessing --source.temporary_file_system_storage_path=./--new_crash_source.new_crash_source_class=socorro.collector.submitter_app.DBSamplingCrashSource --new_crash_source.sql="select 'ce8410a1-b080-4c77-8d80-e64fb2150817'"
@twobraids
twobraids / gist:091323605cf713068345
Created September 2, 2015 19:27
this looks reasonable
import datetime
import mock
op = mock.Mock()
today = datetime.datetime.utcnow().date()
for i in range(28, 0, -1):
then = today - datetime.timedelta(days=i)
till = then + datetime.timedelta(days=1)
@twobraids
twobraids / gist:13169bf340ef3152ff97
Created September 2, 2015 21:34
how could this possibly be true?
breakpad=> select uuid, position('\u0000' in raw_crash::text) from raw_crashes_20150831 where raw_crash::text like '\u0000%';
uuid | position
------+----------
(0 rows)
breakpad=> select uuid, position('\u0000' in raw_crash::text) from raw_crashes_20150831 where raw_crash::text like '%\u0000%';
uuid | position
--------------------------------------+----------
808ed71a-499b-45d9-ae86-7d9642150831 | 0
@twobraids
twobraids / gist:82496fcb2a52ccaaf1d2
Created September 2, 2015 22:08
this is even stranger. a return value of 0 from the position function apparently means "not found"
breakpad=> select uuid, position('\u0000' in raw_crash::text) from raw_crashes_20150824 where raw_crash::text like '%\u0000%';
uuid | position
--------------------------------------+----------
77c9e17c-39b8-409e-be5b-b63982150825 | 0
e3e9f28b-eff4-48fb-b4aa-bb8822150826 | 0
05a5a22e-21d7-4b85-aef0-3de3a2150828 | 0
b1b54310-402a-4bff-8cc2-f2c9b2150829 | 0
(4 rows)
@twobraids
twobraids / ERROR on master
Created September 24, 2015 15:03
unit test failure on master
======================================================================
ERROR: test_get_with_zero (socorro.unittest.external.es.test_supersearch.IntegrationTestSuperSearch)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lars/socorro/socorro/unittest/external/es/test_supersearch.py", line 1457, in test_get_with_zero
_results_number=0,
File "/home/lars/socorro/socorro/external/es/supersearch.py", line 503, in get
results = search.execute()
File "/home/lars/socorro/socorro-virtualenv/local/lib/python2.7/site-packages/elasticsearch_dsl/search.py", line 543, in execute
**self._params
@twobraids
twobraids / socorro_app_as_crontabber_app.py
Created October 12, 2015 21:43
socorro apps as crontabber apps
from crontabber.base import BaseCronApp
from crontabber.mixins import as_backfill_cron_app
from configman import Namespace, class_converter
@as_backfill_cron_app
class SocorroAppAsCrontabberApp(BaseCronApp):
required_config = Namespace()
required_config.add_option(
@twobraids
twobraids / argument collapsing
Created October 16, 2015 00:08
config change for arguments in signatures
consulate kv set socorro/processor/resource.signature.collapse_arguments True
#==============================================================================
class ConnectionContextWithHost(S3ConnectionContext):
"""an alternative base class that specific implementations of Boto
connection can derive. It adds "host" to the configuration"""
required_config = Namespace()
required_config.add_option(
'host',
doc="The hostname",
default="",
@twobraids
twobraids / example in use on the command line
Created February 19, 2016 23:39
an example of getting configuration information into a file
$ python t0.py --help
usage:
t0.py [OPTIONS]...
OPTIONS:
--admin.conf
the pathname of the config file (path/filename)
--admin.dump_conf