Skip to content

Instantly share code, notes, and snippets.

View okram999's full-sized avatar

Niris Okram okram999

View GitHub Profile
@okram999
okram999 / ElasticSearch Notes
Last active October 31, 2018 15:44
ElasticSearch Notes
```
Index is like a database
Index can be splited into shared shared accross multiple nodes (it can have replicas)
A node can have more than 1 shards
Each shards is an instance of lucene - and they hence scales horizontally
Schema of the documents are defined by -- "the Type"
Inverted indices - quickly maps the search
@okram999
okram999 / Lamba
Last active November 6, 2018 19:32
Lamba
502 serverless
503 serverless -version
504 serverless config credentials --provider aws --key XXXXXXX --secret XXXXX --profile serverlessAdmin
### AWS profile for sls cli
You must specify the aws profile | region | timeout | memorySize in the serverless.yml
provider:
name: aws
runtime: python3.6
profile: serverlessAdmin
@okram999
okram999 / Security
Created November 6, 2018 21:32
Security
1. Disable 80 listener on ALB Web Listeners (FE UI, BE API, Micro) or confirm they result in 443 redirections
> Done
2. Confirm Port 8080 on the ALB target groups are configured for SSL only
> Primitive approach. Refer this: https://aws.amazon.com/blogs/aws/elastic-load-balancer-support-for-ssl-termination/
3. Confirm that SSL is enabled for the PGSQL listeners on TCP 5432 for AVA Backend and AVA Central Station db’s
> Require changes in the app as well. https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.SSL
4. Limit BackEnd service access to FrontEndWeb UI servers
@okram999
okram999 / aws
Last active December 5, 2018 16:31
aws
Some questions on EFS. (Mostly checking concurrent mounting from multiple nodes)
Some on ElasticCache
A lot from S3 and Cloudfront
Good understanding of the Autoscaling
Network security or routing based on security grps
Kenesis (Streams and Firehouse) capacity questions
SQS
VPC Endpoints
Some questions on solution RDS MultiAZ - to check if we know RDS MultiAZ supports mutli region or not
Dynamo db - with very low latency and multi region read and writes
@okram999
okram999 / AR - pain
Created December 5, 2018 17:47
AR - pain
Issue with some customFields deployment where the tags were missed and reports were disabled
Merges involving Huge objects (e.g: Opportunity, Account). No assistance from the tool to either help the merge operation / validate the file post merge
Metadata back-up issue
Data backup not available
No sandbox clone/refresh out of the box from the tool
No room to customise the CI/Automated jobs
Unstable platform
@okram999
okram999 / List of kafka commands
Created December 18, 2018 16:47
List of kafka commands
#List the consumer grps
kafka-consumer-groups.bat --bootstrap-server x.x.x.x:9092 -list
@okram999
okram999 / Things to do better
Created January 11, 2019 20:10
Things to do better
Cloudformation Templates:
- Improve security
Cloudform the below:
1. CW Alerts
2. Elastic Search
3.
RDS:
@okram999
okram999 / Enforce SSL in RDS Postgres DB instance.md
Last active February 15, 2019 15:19
Enforce SSL in RDS Postgres DB instance

Requiring SSL in AWS RDS Postgres DB instance

SSL Certificate:

  1. Obtain the root cert from: https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem . This cert will be used for connecting to the db instance when SSL is enforced

Create the new parameter group:

  1. Create a new parameter grp based on the DB engine version, as all the db instances are in default parameter grp [Default parameter grps cannot be modified]
  2. Change the rds.force_ssl to 1 from 0
@okram999
okram999 / Check if a fields is referenced anywhere in a sandbox.md
Last active January 16, 2019 21:00
Check if a fields is referenced anywhere in a sandbox

Check if a fields is referenced anywhere in a sandbox

Tools:

  • Ant
  • bash
  • DOS2Unix converter
  • IDE (e.g. VSCode/Atom)
  1. Create a directory structure as

Lamda

More like an annonymous function

even_num = lambda x: x%2 == 0

Map

Takes a functions/lambda and an iterable, and returns a map object. So convert to list

square = map(lambda x: x*2, [1,2,3,4,5,6]) #[2, 4, 6, 8, 10, 12]

Filters