StudioEtrange

A note about using gmail as a relay
Gmail by default does not allow email clients that don't use OAUTH 2 for authentication (like Thunderbird or Outlook). 
First you need to enable access to "Less secure apps" on your google settings.


Serveur de courrier entrant (IMAP)	imap.gmail.com

SSL requis : oui
Port : 993

StudioEtrange

Copy FOLDER_1 into PATH_B and preserve all attributes

sudo rsync -avihXP --info=progress2 --stats /PATH_A/FOLDER_1/ /PATH_B/


 * use --dry-run to test
 * for encoding char problems between Mac/Linux : https://serverfault.com/questions/638316/rsync-iconv-option-on-mac-not-working-sync-from-remote-linux-server-to-local
 ** rsync initiated from Mac : --iconv=utf-8-mac,utf-8
 ** rsync initiated from Linux : --iconv=utf-8,utf-8-mac

StudioEtrange

#https://stackoverflow.com/a/10965292

[ -n "$var" ] && echo "var is set and not empty"
[ -z "$var" ] && echo "var is unset or empty"
[ "${var+x}" = "x" ] && echo "var is set"  # may or may not be empty
[ -n "${var+x}" ] && echo "var is set"  # may or may not be empty
[ -z "${var+x}" ] && echo "var is unset"
[ -z "${var-x}" ] && echo "var is set and empty"

StudioEtrange

To invoke JupyterLab with Spark capabilities there are two ways. An ad hoc method is to just state on the command line that JupyterLab should use pyspark as kernel. For instance starting JupyterLab with Python 3.6 (needs to be consistent with your Spark distribution), 20 executors each having 5 cores might look like this:

PYSPARK_PYTHON=python3.6 PYSPARK_DRIVER_PYTHON="jupyter" PYSPARK_DRIVER_PYTHON_OPTS="notebook --no-browser --port=8899" /usr/bin/pyspark2 --master yarn --deploy-mode client --num-executors 20 --executor-memory 10g --executor-cores 5 --conf spark.dynamicAllocation.enabled=false

In order to be able to create notebooks with a specific PySpark kernel directly from JupyterLab, just create a file ~/.local/share/jupyter/kernels/pyspark/kernel.json holding:

{
 "display_name": "PySpark",
 "language": "python",

StudioEtrange

1. SSH key based authentification

1.1. Summary

• Autorize SSH connection from a local machine to a distant SSH machine without using a password
• Configure VSCode to use this SSH configuration
• DOC : https://code.visualstudio.com/docs/remote/troubleshooting
• NOTE : how to install OpenSSH Server on windows : https://learn.microsoft.com/fr-fr/windows-server/administration/openssh/openssh_install_firstuse?tabs=gui

1.2. Table of contents

StudioEtrange

# Solutions from https://stackoverflow.com/a/20855353




# SOLUTION 1 : analyse at the current level of execution (do not step into code)

# get https://www.f-hauri.ch/vrac/cours_truc-et-astuces_2012-04-03/add/elap.bash-v2

StudioEtrange

traefik:
    image: traefik:v2.1.6
    container_name: traefik
    volumes:
      - "traefik.enable=true"
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik-conf.yml:/etc/traefik/traefik.yml
      - ./traefik-services.yml:/etc/traefik/services.yml
    labels:
      - "traefik.http.routers.blog.entrypoints=web_main"

StudioEtrange

# launch a shell with an empty env 
# https://unix.stackexchange.com/a/291913

 env -i bash --noprofile --norc
 
# source var file and show a specific var
 env -i bash --noprofile --norc -c ". somefile.sh; echo $VAR;"

# backup/restore env
# https://unix.stackexchange.com/questions/284382/export-save-and-import-source-shell-variables

StudioEtrange

# count number of active task inside of all user crontab
# sample with excluding a string containing "str"

for c in $(sudo ls  /var/spool/cron); do sudo crontab -u $c -l 2>/dev/null | grep -v -e '^$' | grep -v -e '^#' | grep  -v "crontab;" | grep -v "str"; done | wc -l

StudioEtrange

#!/bin/bash
# This script is designed to work only with recent bash implementations, not ash/dash etc.

# @file      show-cron-jobs.sh
#
#             published at:
#               https://gist.github.com/myshkin-uk/d667116d3e2d689f23f18f6cd3c71107
#
#             @NOTE DGC 2-Sep-2019
#                      There is one comment there that the script doesn't run as expected.