Skip to content

Instantly share code, notes, and snippets.

View fuyi's full-sized avatar
🎯
Focusing

YI FU fuyi

🎯
Focusing
  • Stockholm, Sweden
View GitHub Profile
@fuyi
fuyi / qingmaio.py
Created July 5, 2021 13:56
qingmiao_zhang_python
INPUT = [
1,
2,
3,
4,
5,
11111,
9,
8,
7,
@fuyi
fuyi / cp_archi.md
Last active May 26, 2021 07:56
CP archtecture questions

General

  • What AEM and Asset Link stand for?

Diagram 3.2

  • How do we manage authentication between API gateway and web browser.
  • should Retrieves Asset Link and Sends Reward Info seem need to be switched?
  • Prediction-to-reward matching DB, where can I find the table schema definition?
@fuyi
fuyi / airflow-mysql.md
Created December 10, 2020 14:18
Airflow MySQL Meta store performance tuning

Login MySQL with this shell script

 #!/bin/bash
SECRET_VALUE=$(tink-kubectl -n airflow get secrets airflow-credentials -o json)
PASSWORD=$(echo $SECRET_VALUE | jq ."data" | jq ."rootPassword")
CORRECT_PASSWORD=$(echo "$PASSWORD" | sed -e 's/^"//' -e 's/"$//' | base64 -d)
tink-kubectl -n airflow exec -it airflow-mysql-0 bash
mysql -h 127.0.0.1 -P 3306 -u root -p $CORRECT_PASSWORD
use airflow;
@fuyi
fuyi / bq_hackday.md
Last active October 21, 2020 09:31
BQ+ Data studio hack day

Questions to answer

  • Heart beat Event distribution in the last 24 hours, per hour resolution.
  • Distribution of All events from Oxford Production in the last 48 hours.
  • Total number vs Actual number of categorization events on Rugby production per hour in the last 24 hours.

Learning:

  • Data studio doesn't support alert yet.
@fuyi
fuyi / dag.yaml
Created September 15, 2020 19:26
dag.yaml
_global: &globals
owner: test_owner
schedule_interval: "* * * * *"
retries: 3
retry_delay: 30
connection: some_connection
operator: BigQueryOperator
slack:
enabled: true
slack_connection_id: slack
'--include', 'users*.avro'
@fuyi
fuyi / yi-tmp.yaml
Last active July 5, 2019 08:48
Template file to Create a tmp pod in kubernates
apiVersion: v1
kind: Pod
metadata:
name: tmp-shell-yi
namespace: event-producer-service
spec:
securityContext:
runAsNonRoot: true
runAsUser: 65535
runAsGroup: 65535
cdebc563d04db6f86b8cbfc7ea9c93cf26e0fca8
import { Batch } from 'redux/modules/batch/helpers';
import batchStarter from 'redux/modules/batch/actions/batchStart';
import { bindActionCreators } from 'redux';
import fetchToken from 'redux/modules/user/actions/fetchToken';
export default (store) => (nextState, replace, cb) => {
const { batchStart } = bindActionCreators({ batchStart: batchStarter }, store.dispatch);
const batch = new Batch(batchStart);
batch.subBatch({ fetchToken: fetchToken() });
batch.subBatch({ routing: () => cb() });
def _check_relational_existence(list_of_values, root_limetype):
"""
check if the last element of the array is a valid lime property,
other elements must be valid lime type and has BelongTo relationship to next element
input:
list_of_values: an array contains lime type chain
root_limetype: the limetype to start property checking
return:
raise Exception if not valid limetype detect, otherwise return True
"""