samukasmk

Pandas examples of read_parquet files from s3

Credentials (from enviroment variables)

import os
import pandas as pd

# define aws credentials by os enviroment variables
os.environ['AWS_ACCESS_KEY_ID'] = '...'
os.environ['AWS_SECRET_ACCESS_KEY'] = '...'

samukasmk

grep_matcher.sh

#!/bin/bash
#
# grep matcher: wait to match first case and exit

grep_expression="$1"
grep_file="$2"
__matched=1

samukasmk

Object-oriented style

File: workflow_by_chain_of_responsability_oo.py

from abc import ABC, abstractmethod

#
# Abstract classes
#

samukasmk

class Parent1:
    def parent1_method(self):
        return "Method from Parent1 class"

class Parent2:
    def parent2_method(self):
        return "Method from Parent2 class"

class Child:
    def __init__(self, parent_class):

samukasmk

from typing import Any


class WorkflowSkip(Exception):
    ...


class WorkflowProcess:
    workflow_steps = ()

samukasmk

import re

if re.match(r'^/tmp/(?!.*/$).*$', file_path):
    os.remove(file_path)

samukasmk

echo -e "#!/bin/bash
#

echo 'This is my script'" > my-script

sudo install my-script /usr/local/bin

samukasmk

Xargs examples

Executing linux commands in parallel with xargs command

Xargs arguments

xargs --help

Usage: xargs [OPTION]... COMMAND [INITIAL-ARGS]...
Run COMMAND with arguments INITIAL-ARGS and more arguments read from input.

samukasmk

How to enable swap memory on Linux

Always I forget how to set Linux swap so there it is

Steps:

1. check swap
2. check disk size
3. create swap file (with 4GB)
4. fix permissions for root
5. create swap area
6. enable swap in linux using the new file

samukasmk

#!/bin/bash

# Variables to store argument values
ARG1=""
ARG2=""

# Function to display the error message and exit the script
function usage_error {
    echo "Usage: $0 --argument_1=\"<text>\" --argument_2=\"<text>\""
    echo "Error: $1"