deeTEEcee

How to onboard a new engineer?

• Show general architecture through diagrams and concrete examples.
• There should be clear enough instructions that a new onboarding engineer can just look through a document and follow that to setup a local development environment and be able to set breakpoints and play around with the environment. (Even startups with smaller code bases will fail to follow up on this because it's not "priority" but it definitely should be.)

deeTEEcee

List of Difficult Bugs

The Gradle bug Case: I ran a basic gradle command ./gradlew help to test if the configuration and setup is done right. After I added a new plugin, I get an error saying a java method doesn't exist. How long did it take to solve: 1-2 days Reason it was unsolveable: A poor implementation of gradle has their dependencies leaking from one place to the other but it's really difficult to make that connection. Post-mortem: Nothing I really could've done here to speed up my effort. You need a deeper understanding of gradle internals (or to have used buildSrc before)

deeTEEcee

Dependency management

With any new projects using gradle, this is a key point to start. Try using the 'com.palantir.consistent-versions' plugin.

Gradle helper methods

I ran into an issue where a gradle plugin had stated they had a method called 'getVersion' but I couldn't use it. This was the solution to retrieve that method that the plugin had supported:

val packageGetVersion : groovy.lang.Closure<String> = project.ext["getVersion"] as groovy.lang.Closure<String>
doLast {

deeTEEcee

package com.sightmachine.doclet;

import java.lang.reflect.Array;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.List;

deeTEEcee

import java.lang.reflect.Array;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.List;

import com.sun.javadoc.Doc;
import com.sun.javadoc.DocErrorReporter;

deeTEEcee

/*

Regex is interesting because I found out the following:
* It's pretty easy to translate patterns into actual variables.
* Even if you have repeated patterns, those can easily go into variables. (e.g: `(<some_pattern>)+` can be translated into an infinite number of variables)
* Some languages have a multiline mode that allow you to apply regex line-by-line. That has the benefit
of giving you more easily readable options.

To see a better example, look at the following code.

deeTEEcee

### Databases
* One does not need to add database indexes for every field. I think one way of thinking about it is use the ones that people are querying that don't also already have indexes. If there is a multi-field query that's commonly used and there's already indexes, you probably don't need to add another index for a new field you add since that query already narrows it down for you. (There's a lot more to read to understand this) 

#### Internal
* SSD's vs Hard Disk
  * 
  * Writes have problems on both sides.  
* Access Patterns - Sequential vs. Random

Links

deeTEEcee

# Created these after looking at unit tests. Couldn't really find any other online examples. 
from io import BytesIO

from amqp import Connection
import pprint
import pickle

connection = Connection()
channel = connection.channel()
message = channel.basic_get(queue='gegroby/ma.model.cycle.finalcal/analytics_finalcal/')

deeTEEcee

Missing Indexes

db.getCollection('system.profile').find({'ns': 'db.collection_name', 'planSummary': {'$not': /IXSCAN/}})

deeTEEcee

• ngrok # for opening local ports to web
• daff # tool for csv file comparisons. (a bit limited in csv file size)
• xsv # tool for csv file parsing. (filtering, selecting certain columns, works well with massive sizes)