Skip to content

Instantly share code, notes, and snippets.

View geekpete's full-sized avatar
💭
Equilibrious

Peter Dyson geekpete

💭
Equilibrious
56 followers · 72 following
View GitHub Profile
@geekpete
geekpete / gist:7329085
Last active May 27, 2018 18:28
One line python command to generate a SHA512 password hash for use in linux shadow file including a randomised salt value. Change MySecretPassword to the password of your choice. Useful for generating password hashes to be used in puppet user manifests. This script is mildly idiotic. Bash will prevent any non-escaped characters being fed into py…
python -c "import random,string,crypt,getpass,pwd; randomsalt = ''.join(random.sample(string.ascii_letters,8)); print crypt.crypt('MySecretPassword', '\$6\$%s))\$' % randomsalt)"
@geekpete
geekpete / gist:7350382
Created November 7, 2013 07:15
Set and forget command for wget on slow links. Tell wget to retry if no data comes through for x seconds, to resume the file and to retry infinitely until it gets the file.
wget http://<whateverurl>/<whateverfile> -O <localdestinationfile> --continue --read-timeout=5 --tries=0
@geekpete
geekpete / 20130416-todo.md
Created July 14, 2014 04:21 — forked from mrflip/20130416-todo.md

Next Steps

  • Measure time spend on index, flush, refresh, merge, query, etc. (TD - done)
  • Take hot threads snapshots under read+write, read-only, write-only (TD - done)
  • Adjust refresh time to 10s (from 1s) and see how load changes (TD)
  • Measure time of a rolling restart doing disable_flush and disable_recovery (TD)
  • Specify routing on query -- make it choose same node for each shard each time (MD)
  • GC new generation size (TD)
  • Warmers
  • measure before/after of client query time with and without warmers (MD)
@geekpete
geekpete / gist:12baad19da2da2cc24d4
Last active August 29, 2015 14:06
Download JDK8 tarball with a script by building your own cookie to accept the license with wget
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-i586.tar.gz"
@geekpete
geekpete / gist:c702e9e443a8690501c7
Last active August 29, 2015 14:10
List expiry dates of all certificates in our Amazon AWS account with the aws cli
aws iam list-server-certificates --output text --query 'ServerCertificateMetadataList[*].[Expiration,ServerCertificateName]'|sort
@geekpete
geekpete / nested_aggs.sh
Last active August 29, 2015 14:13 — forked from dptesta/nested_aggs.sh
curl -XDELETE localhost:9200/nested_aggs
curl -XPUT localhost:9200/nested_aggs
curl -XPUT localhost:9200/nested_aggs/user/_mapping -d '
{
"_id" : {"index": "not_analyzed", "path" : "userId"},
"properties": {
"userId": {"type": "string", "index": "not_analyzed"},
"groups": {
"type": "nested",
"properties": {
@geekpete
geekpete / 20130416-todo.md
Last active August 29, 2015 14:13 — forked from mrflip/20130416-todo.md

Next Steps

  • Measure time spend on index, flush, refresh, merge, query, etc. (TD - done)
  • Take hot threads snapshots under read+write, read-only, write-only (TD - done)
  • Adjust refresh time to 10s (from 1s) and see how load changes (TD)
  • Measure time of a rolling restart doing disable_flush and disable_recovery (TD)
  • Specify routing on query -- make it choose same node for each shard each time (MD)
  • GC new generation size (TD)
  • Warmers
  • measure before/after of client query time with and without warmers (MD)
@geekpete
geekpete / gist:cd6d2fc166d329d902a7
Last active August 29, 2015 14:14 — forked from duydo/elasticsearch_best_practices.txt
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:
- Use create in the index API (assuming you can).
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
- Increase the memory allocated to elasticsearch node. By default its 1g.
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
- Increase the number of machines you have so
@geekpete
geekpete / gist:dbbf6e89035f3b41d93c
Created October 29, 2015 06:25 — forked from garnaat/gist:1443559
Using get_all_instance_status method in boto
>>> import boto
>>> ec2 = boto.connect_ec2()
>>> stats = ec2.get_all_instance_status()
>>> stats
[InstanceStatus:i-67c81e0c]
>>> stat = stats[0]
>>> stat
InstanceStatus:i-67c81e0c
>>> stat.id
u'i-67c81e0c'
@geekpete
geekpete / install-comodo-ssl-cert-for-nginx.rst
Created February 12, 2016 04:04 — forked from bradmontgomery/install-comodo-ssl-cert-for-nginx.rst
Steps to install a Comodo PositiveSSL certificate with Nginx.

Setting up a SSL Cert from Comodo

I use Namecheap.com as a registrar, and they resale SSL Certs from a number of other companies, including Comodo.

These are the steps I went through to set up an SSL cert.

Purchase the cert