Skip to content

Instantly share code, notes, and snippets.

@ntakouris
Created July 29, 2020 13:19
Show Gist options
  • Select an option

  • Save ntakouris/22c933e45c655414bf00e772b74c7a00 to your computer and use it in GitHub Desktop.

Select an option

Save ntakouris/22c933e45c655414bf00e772b74c7a00 to your computer and use it in GitHub Desktop.
def _gzip_reader_fn(filenames):
return tf.data.TFRecordDataset(filenames, compression_type='GZIP')
def _input_fn(file_pattern: List[Text],
tf_transform_output: tft.TFTransformOutput,
batch_size: int = 32) -> tf.data.Dataset:
transformed_feature_spec = (
tf_transform_output.transformed_feature_spec().copy())
dataset = tf.data.experimental.make_batched_features_dataset(
file_pattern=file_pattern,
batch_size=batch_size,
features=transformed_feature_spec,
reader=_gzip_reader_fn,
label_key=LABEL_KEY)
return dataset
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment