ShahBinoy

2022-01-27 19:10:02  INFO i.a.w.w.WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.35.11-alpha
2022-01-27 19:10:03  INFO i.a.w.t.TemporalAttemptExecution(get):120 - Cloud storage job log path: /workspace/129/0/logs.log
2022-01-27 19:10:31  destination > SLF4J: Class path contains multiple SLF4J bindings.
2022-01-27 19:10:03  INFO i.a.w.t.TemporalAttemptExecution(get):123 - Executing worker wrapper. Airbyte version: 0.35.11-alpha
2022-01-27 19:10:03 WARN i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):65 - Waiting for database to become available...
2022-01-27 19:10:03  INFO i.a.d.i.j.JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready...
2022-01-27 19:10:03  INFO i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):90 - Database available!
2022-01-27 19:10:03  INFO i.a.d.Databases(createPostgresDatabaseWithRetry):48 - Database available!
2022-01-27 19:10:03 DEBUG i.a.c.h.LogClientSingleton(setJobMdc):137 - Setting kube job mdc
2022-01-27 19:10:03 WARN c.n.s

ShahBinoy

def s3_to_pandas(client, bucket, key, header=None):

    # get key using boto3 client
    obj = client.get_object(Bucket=bucket, Key=key)
    gz = gzip.GzipFile(fileobj=obj['Body'])
    
    # load stream directly to DF
    return pd.read_csv(gz, header=header, dtype=str)

def s3_to_pandas_with_processing(client, bucket, key, header=None):

ShahBinoy

evolutionplugin = disabled
# the value below matches the config tag in line 10
play.db.default = "oracle"

jpa {
  default = devPersistenceUnit # should match the value of persistence-unit#name in persistence.xml
}

db {
# the value below should match the value of property "play.db.default"

ShahBinoy

#!/bin/bash

while [[ $# -gt 1 ]]
do
key="$1"

case $key in
    -s|--subnet)
    SUBNETS="$2"
    shift # past argument

ShahBinoy

#!/bin/sh
#
# redis - this script starts and stops the redis-server daemon
#
# chkconfig:   - 85 15 
# description:  Redis is a persistent key-value database
# processname: redis-server
# config:      /etc/redis/redis.conf
# config:      /etc/sysconfig/redis
# pidfile:     /var/run/redis/redis.pid

ShahBinoy

#!/bin/bash
# init script for Cassandra.
# chkconfig: 2345 90 10
# description: Cassandra
# script slightly modified from 
# http://blog.milford.io/2010/06/installing-apache-cassandra-on-centos/
 
. /etc/rc.d/init.d/functions