README.md

Shortlink: goo.gl/wSuuS9

Supplementary Materials for Generating Wikipedia by Summarizing Long Sequences

The github repository can be found at https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/wikisum

I'm also really looking forward to trying it out! Thx a lot for the good work!

Best,
Simon

I want to know when the code is released

Any news on this? We'll be discussing this paper next Tuesday, so it would be great if the source code was available by then.

Thanks for the interest! We're actively preparing the code for release and I think it's almost ready for consumption. It might not be available by Tuesday though.

Thanks for letting me know.

Great work!

Hi peterjliu, any news about the progress ?

Eagerly awaiting the dataset, want to apply other techinques to compare with the results.

Hi peterjliu, I was wondering if there were any updates on the dataset availability? I'm hoping to do some reproducibility tests. Thanks!

Is this the data set that @lukaszkaiser is referencing in the Google Developers video here? https://youtu.be/O2UvKxaOH7c?t=637
If he's going to boast about it, it would seem to make sense to actually have it be available. It sounds like a great application of @commoncrawl data.

Hi folks, check out the updated link now. Thanks for the patience.

@rafaelbou @sai-prasanna @vedant @SHohentanner @leisurehippo @coventry @cyberandy @Diego999 @Legalamb77 @tfmorris

Mentioning folks who specifically expressed interest here.

@peterjliu

Thank you for the share. I was wondering if it would be possible to store the preprocessed datasets on a local computer (after the preprocessing on the cloud) of it is too large ? Do you have an estimate of the necessary space ? 10 GB ? 100 GB ? 1 TB ?

Thank you for your help !

This looks really useful. I noticed that the pre-processed vocabs seem to be available in the gs://tensor2tensor-data/ bucket too (vocab.wikisum_commoncrawl.32768 and vocab.wikisum_web.32768)

The TODO says you release the hparams_set, which would be great, but can I request a pre-trained model release too?

Dear all,

Is there any available pre-trained model released for this wikisum problem? If there is, may I have the link to that pre-trained model?

Thank you so much

Thanks for linking that, @peterjliu. Am I reading the README.md correctly, here, that training uses a full transformer architecture, rather than a decoder-only architecture with memory-compressed attention?

---

Training

TODO(rsepassi): Put actual results achieved on wikisum_web and/or
wikisum_commoncrawl and with what hparams_set.

PROBLEM=wikisum_web  # or wikisum_commoncrawl
t2t-trainer \
  --problem=$PROBLEM \
  --model=transformer \
  --hparams_set=transformer_base \
  --train_steps=250000 \
  --eval_steps=100 \
  --data_dir=$DATA_DIR \
  --output_dir=$TRAIN_DIR

Does anyone have processed training examples (i.e., the output of step 3 here) available to share? I'm having trouble getting GCP to release IP addresses for data generation, so I'm hoping to be able to bypass this for the time being...

Also, as @nlothian and @hoang-ho have asked, are pre-trained model weights available anywhere?