deeTEEcee

List of Difficult Bugs

The Gradle bug Case: I ran a basic gradle command ./gradlew help to test if the configuration and setup is done right. After I added a new plugin, I get an error saying a java method doesn't exist. How long did it take to solve: 1-2 days Reason it was unsolveable: A poor implementation of gradle has their dependencies leaking from one place to the other but it's really difficult to make that connection. Post-mortem: Nothing I really could've done here to speed up my effort. You need a deeper understanding of gradle internals (or to have used buildSrc before)

deeTEEcee

Dependency management

With any new projects using gradle, this is a key point to start. Try using the 'com.palantir.consistent-versions' plugin.

Gradle helper methods

I ran into an issue where a gradle plugin had stated they had a method called 'getVersion' but I couldn't use it. This was the solution to retrieve that method that the plugin had supported:

val packageGetVersion : groovy.lang.Closure<String> = project.ext["getVersion"] as groovy.lang.Closure<String>
doLast {

deeTEEcee

package com.sightmachine.doclet;

import java.lang.reflect.Array;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.List;

deeTEEcee

import java.lang.reflect.Array;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.List;

import com.sun.javadoc.Doc;
import com.sun.javadoc.DocErrorReporter;

deeTEEcee

/*

Regex is interesting because I found out the following:
* It's pretty easy to translate patterns into actual variables.
* Even if you have repeated patterns, those can easily go into variables. (e.g: `(<some_pattern>)+` can be translated into an infinite number of variables)
* Some languages have a multiline mode that allow you to apply regex line-by-line. That has the benefit
of giving you more easily readable options.

To see a better example, look at the following code.

deeTEEcee

### Databases
* One does not need to add database indexes for every field. I think one way of thinking about it is use the ones that people are querying that don't also already have indexes. If there is a multi-field query that's commonly used and there's already indexes, you probably don't need to add another index for a new field you add since that query already narrows it down for you. (There's a lot more to read to understand this) 

#### Internal
* SSD's vs Hard Disk
  * 
  * Writes have problems on both sides.  
* Access Patterns - Sequential vs. Random

Links

deeTEEcee

# Created these after looking at unit tests. Couldn't really find any other online examples. 
from io import BytesIO

from amqp import Connection
import pprint
import pickle

connection = Connection()
channel = connection.channel()
message = channel.basic_get(queue='gegroby/ma.model.cycle.finalcal/analytics_finalcal/')

deeTEEcee

Missing Indexes

db.getCollection('system.profile').find({'ns': 'db.collection_name', 'planSummary': {'$not': /IXSCAN/}})

deeTEEcee

• ngrok # for opening local ports to web
• daff # tool for csv file comparisons. (a bit limited in csv file size)
• xsv # tool for csv file parsing. (filtering, selecting certain columns, works well with massive sizes)

deeTEEcee

# surprised that python does not have something like this in the iterator class or something.
def get_subset_from_dict(dict_obj, *keys):
    return dict((k, dict_obj[k]) for k in keys)