titovanton

Repo review

Repo is located by the following link.

README

Local setup

I've started to write this, then I found, that the developer uses Docker, inattentive me... So you can skip it, or read for the detailed explanation purpose.

There is no a reference to some sort of environment, which may cause incompatibility between developers machines: some of them may work on Win(like me), some of them may work on MacOS(most popular as I get it). The old school python way to make an environment to work in is python virtual environment package, so you can switch between different project on 1 machine and install needed packages corresponding to a particular project.

titovanton

BACK-END

Deploy

The first thing I see is the empty README.md. At least we would need here a small description and for sure - how to deploy to a local machine and to a production service/server. That process must be cross-platform, or at least as closer as possible, so we reached several goals: 1) minimize differences between developers machines, it may stack you for a day, and involve all team forces, to run the project on a machine of a new guy; 2) minimize differences between local deployment and stage/production, I realize, that the environment would be different, but for that purpose you need .env(good manners) file. But python packages versions(from requirements.txt), DB version, webserver version and other 3d party servers, tools and libs must be synchronized to prevent issues and raising code errors...

For that purpose, you may use something like Vagrant, but I suggest you to use Docker - this is modern and well documented/maintained project with a big community. To migrate that to a prod

titovanton

import cs_igc
import datetime
import hadoopSecureCheck
import hashlib
import logging
import logging.handlers
import os
import re
import select
import shutil

titovanton

#!/usr/bin/env bash

LRED='\033[1;31m'
LGREEN='\033[1;32m'
NC='\033[0m' # No Color

__border() {
  local COLOR=$1
  local title="║ ${@:2} ║"
  local edge=$(echo $title | sed -E "s/%%|##//g" | sed 's/./═/g' | sed 's/^./╔/' | sed 's/.$/╗/')

titovanton

#!/usr/bin/env bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source $DIR/border.sh

if [ "$0" = "$BASH_SOURCE" ]; then
    error "Script must be sourced!"
    exit 1
fi

titovanton

#!/usr/bin/env bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
source $DIR/border.sh

test_env=$(cat /proc/1/cgroup | head -1 | cut -d ':' -f3 | cut -d '/' -f2)

if [ "${test_env}" != "docker" ]; then
    error "You need to be inside docker container!"
    return 1

titovanton

import numpy as np


def csv_strip(reader):
    '''
    Removes all rows and columns, which have only blank cells.
    Takes csv.reader instance.
    Returns numpy array of lists
    '''
    data = np.array(list(reader))

titovanton

from decimal import Decimal

from django.db import connection
from django.utils.decorators import ContextDecorator


class CountQueries(ContextDecorator):
    '''
    Either logs(if logger instance has passed) or prints number of queries.

titovanton

class FileUploadSerializer(serializers.Serializer):
    attachment = serializers.FileField()
    
    
class FileUploadView(views.APIView):
    '''
    FileUploadParser - is some sort of retardnes, 
    it takes entier http body and parses it as a file 0_0.
    Very useful! Specially trash headers, prepended to the text file...
    So, just don't use it ;-> Use MultiPartParser instead.

titovanton

class BiDict(object):
    '''
    Maps both directions: key->value and value->key

    Requires both keys and values are unique. If you pass duplication,
    then key/value be overwritten and you get not what expected.
    '''
    origin = {}  # counters the {key->key} case
    bidict = {}