YI FU fuyi

Code to write to Feature Store

@beam.typehints.with_input_types(Tuple[User, List[str]])
class WriteFeatureRPC(beam.DoFn):
    def __init__(
        self,
        api_endpoint,
        project_id,

Create custom window strategy to fulfil sequencer model feature requirement

Background

To train our neural sequencer model properly, we need to get the ordered sequence of user interaction with articles in a certain period of time. To formulate the requirements, the input dataset for the model training can be described as:

The last X articles the user has interacted with in the last Y days in chronological order.

X: variable with integer value between [0, 20] inclusive

	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="utf-8">
	<title>heavy_marts Gantt</title>
	<style>
	body { font-family: monospace; background: #1a1a2e; color: #eee; margin: 20px; }
	h2 { color: #a0c4ff; }
	.row { display: flex; align-items: center; height: 22px; margin: 2px 0; }
	.label { width: 380px; font-size: 11px; text-align: right; padding-right: 8px; flex-shrink: 0; overflow: hidden; white-space: nowrap; color: #ccc; }

	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="utf-8">
	<title>simple_marts Gantt</title>
	<style>
	body { font-family: monospace; background: #1a1a2e; color: #eee; margin: 20px; }
	h2 { color: #a0c4ff; }
	.row { display: flex; align-items: center; height: 22px; margin: 2px 0; }
	.label { width: 330px; font-size: 11px; text-align: right; padding-right: 8px; flex-shrink: 0; overflow: hidden; white-space: nowrap; color: #ccc; }

	import boto3
	import struct

	def validate_wav_header_from_s3(bucket_name, object_key):
	"""
	Validate WAV file by reading only the header (first 44 bytes)
	"""
	session = boto3.Session(profile_name='product-admin') # use the underlying aws credential
	s3 = session.client('s3')

	# use C-a, since it's on the home row and easier to hit than C-b
	set-option -g prefix C-a
	unbind-key C-a
	bind-key C-a send-prefix
	set -g base-index 1

	# Easy config reload
	bind-key R source-file ~/.tmux.conf \; display-message "tmux.conf reloaded."

	# vi is good

	FROM public.ecr.aws/lambda/python:3.8
	RUN yum -y install java-1.8.0-openjdk wget curl
	COPY requirements.txt requirements.txt
	RUN pip install -r requirements.txt
	ENV JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.amzn2.0.1.x86_64/jre"
	ENV PATH=${PATH}:${JAVA_HOME}/bin
	ENV SPARK_HOME="/var/lang/lib/python3.8/site-packages/pyspark"
	ENV PATH=$PATH:$SPARK_HOME/bin
	ENV PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-0.10.9-src.zip:$PYTHONPATH
	ENV PATH=$SPARK_HOME/python:$PATH

	{
	"parameters":[
	{
	"name":"touchpoint",
	"stringValue":"android"
	},
	{
	"name":"market",
	"stringValue":"gb"
	},

	@beam.typehints.with_input_types(Tuple[User, List[EventAction]])
	class WriteFeatureRPC(beam.DoFn):
	def __init__(
	self,
	api_endpoint,
	project_id,
	location,
	featurestore_id,
	entity_type_id,
	feature_name,

	{
	"parameters": [
	{
	"name": "touchpoint",
	"value_string": "DESKTOP"
	},
	{
	"name": "market",
	"value_string": "SE"
	},