Skip to content

Instantly share code, notes, and snippets.

View shamilnabiyev's full-sized avatar

Shamil Nabiyev shamilnabiyev

View GitHub Profile
@shamilnabiyev
shamilnabiyev / copy_paste_augmentation_yolo.py
Created April 17, 2025 21:11
Simple copy paste augmentation for YOLO object detection task
from PIL import Image
import random
def paste_image(source_path, target_path, position=None):
"""
Pastes a small source image onto a larger target image.
Parameters:
- source_path: Path to the small source image.
- target_path: Path to the larger target image.
@shamilnabiyev
shamilnabiyev / ms-oauth.py
Created March 30, 2025 12:31
Microsoft identity platform and OAuth 2.0
# Microsoft Authentication Library (MSAL) for Python: pip install requests msal
import requests
import msal
## 1. Redirect User to Microsoft Login:
# Replace these variables with your actual values
client_id = "your_client_id"
client_secret = "your_client_secret"
authority = "https://login.microsoftonline.com/your_tenant_id"
@shamilnabiyev
shamilnabiyev / pyspark-leading-zeros.py
Created December 2, 2024 21:01
PySpark DataFrame - add leading zero if the value is a single digit between 0 and 9
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, when, regexp_extract, length, lpad
# Create a SparkSession
spark = SparkSession.builder.appName("TupleToDataFrame").getOrCreate()
# List of tuples
arr = [("a", "0a1x"), ("x", "3"), ("cc", "11"), ("h", "5")]
# Create a DataFrame
@shamilnabiyev
shamilnabiyev / aws-glue-job-read-based-on-date.py
Created September 30, 2024 18:26
Read files from an AWS S3 bucket based on their creation date using AWS Glue
import boto3
from datetime import datetime
from awsglue.context import GlueContext
from pyspark.context import SparkContext
# First, use boto3 to list and filter objects
def list_s3_files_after_date(bucket_name, prefix='', start_date=None):
s3 = boto3.client('s3')
@shamilnabiyev
shamilnabiyev / aws-glue-job-read-deleted-files.py
Last active September 30, 2024 18:23
Read files marked as deleted from an AWS S3 bucket into a DynamicFrame using boto3 and AWS Glue
import boto3
from awsglue.context import GlueContext
from pyspark.context import SparkContext
from awsglue.dynamicframe import DynamicFrame
def read_deleted_files_to_dynamicframe(bucket_name, prefix=''):
# Initialize Glue context
sc = SparkContext()
glueContext = GlueContext(sc)
@shamilnabiyev
shamilnabiyev / efi-boot-manager.sh
Last active August 15, 2024 20:08
Update EFI Boot Manager. Add new bootloaders for Windows OS and Linux Mint OS
# Print available disks (drives) and partitions
lsblk -o name,UUID,partuuid,mountpoint
# Assumption
## Windows OS bootloader is installed on /dev/nvme0n1
## Linux OS bootloader is installed on /dev/nvme1n1
# Add Windows bootloader entry
sudo efibootmgr -c -d /dev/nvme0n1 -p 1 -l \\EFI\\Microsoft\\Boot\\bootmgfw.efi -L "Windows Boot Manager"
@shamilnabiyev
shamilnabiyev / pipx-setup.md
Last active May 30, 2024 12:05
Install pipx on Windows
  1. Run commands in Git Bash
    • env PIP_REQUIRE_VIRTUALENV=0 python -m pip install --user pipx
    • python -m pipx ensurepath --force
  2. Open a new Git Bash terminal
    • pipx --version
  3. If getting an error during the installation of packages using pipx
    • No Python at 'C:\Users\<username>\AppData\Local\Programs\Python\PythonX\python.exe'
    • Then remove the following folder
    • C:\Users\<username>\.local\pipx
  4. If still having the same issue
@shamilnabiyev
shamilnabiyev / vbb-sbahn-berlin.py
Last active May 5, 2024 19:34
Get Berlin S-Bahn Lines and Station IDs from VBB Verkehrsverbund Berlin-Brandenburg
import json
import requests
import pandas as pd
from itertools import chain
def get_sbahn_list(product='suburban', operator='1'):
"""
Get S-Bahn Line list from VBB Transport API
S-Bahn: product=suburban
@shamilnabiyev
shamilnabiyev / pipeline.py
Created April 9, 2023 20:14
Transfer learning with VGG and Keras [for image classification]
# Credits:
# Author: Gabriel Cassimiro
# Blog post: https://towardsdatascience.com/transfer-learning-with-vgg16-and-keras-50ea161580b4
# GitHub Repo: https://github.com/gabrielcassimiro17/object-detection
#
import tensorflow_datasets as tfds
from tensorflow.keras import layers, models
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.callbacks import EarlyStopping
@shamilnabiyev
shamilnabiyev / hyp.custom-scratch-low.yaml
Created December 12, 2022 22:21
YOLOv5 Custom hyperparameters
# YOLOv5 🚀 by Ultralytics, GPL-3.0 license
# Hyperparameters for low-augmentation COCO training from scratch
# python train.py --batch 64 --cfg yolov5n6.yaml --weights '' --data coco.yaml --img 640 --epochs 300 --linear
# See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3)
lrf: 0.01 # final OneCycleLR learning rate (lr0 * lrf)
momentum: 0.937 # SGD momentum/Adam beta1
weight_decay: 0.0005 # optimizer weight decay 5e-4
warmup_epochs: 3.0 # warmup epochs (fractions ok)