Skip to content

Instantly share code, notes, and snippets.

@zomux

zomux/tmux.txt

Created October 28, 2019 06:01
Show Gist options
  • Save zomux/c6d0ed10edebe80dbfe3610e5903edb9 to your computer and use it in GitHub Desktop.
Save zomux/c6d0ed10edebe80dbfe3610e5903edb9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
tmux.txt
Your job 1199571 ("run_single.sh") has been submitted
➜ jobs qstatjob-ID prior name user state submit/start at queue
jclass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1199571 0.25586 run_single acb11190vf r 10/05/2019 23:25:46 gpu@g0022
10
➜ jobs qsub -g gcb50169 -l rt_F=4 ./run.sh python $HOME/lanmt/run.py --opt_dtok wmt14_ende --opt_disti
ll --opt_annealbudget --opt_finetune --opt_sgdft --opt_Trefine_steps 1 --train --test --evaluate
Your job 1199576 ("run.sh") has been submitted
➜ jobs vi run.py
➜ jobs vi run.py
➜ jobs vi run.sh
➜ jobs qsub -g gcb50169 -l rt_F=4 -l h_rt=20:00:00 -m e ./run.sh python $HOME/lanmt/run.py --opt_dtok
wmt14_ende --opt_distill --opt_annealbudget --opt_finetune --opt_sgdft --opt_Trefine_steps 1 --train --
test --evaluate
➜ jobs vi run.sh
➜ jobs vi run.sh
➜ jobs qsub -g gcb50169 -l rt_F=4 ./run.sh python $HOME/lanmt/run.py --opt_dtok wmt14_ende --opt_disti
ll --opt_annealbudget --opt_finetune --opt_sgdft --opt_Trefine_steps 1 --train --test --evaluate
➜ jobs qsub -g gcb50169 -l rt_F=4 -l h_rt=20:00:00 -m e ./run.sh python $HOME/lanmt/run.py --opt_dtok
wmt14_ende --opt_distill --opt_annealbudget --opt_finetune --opt_sgdft --opt_Trefine_steps 1 --train --
test --evaluate
Your job 1199594 ("run.sh") has been submitted
➜ jobs qstatjob-ID prior name user state submit/start at queue
jclass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1199576 0.25586 run.sh acb11190vf r 10/05/2019 23:35:04 gpu@g0376
320
1199594 0.25586 run.sh acb11190vf r 10/06/2019 00:15:20 gpu@g0381
320
➜ jobs kill 1199576
kill: kill 1199576 failed: no such process
➜ jobs qdel 1199576
acb11190vf has registered the job 1199576 for deletion
➜ jobs
➜ jobs
➜ jobs qdel 1199576
➜ jobs qstat
job-ID prior name user state submit/start at queue jcl
ass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1199594 0.25586 run.sh acb11190vf r 10/06/2019 00:15:20 gpu@g0381
320
➜ jobs qdel 1199594
acb11190vf has registered the job 1199594 for deletion
➜ jobs show_point
Group Disk Used Point Used%
gcb50169 0 16.0488 2,000 0
`- acb11190vf - 15.9042 - 0
➜ jobs show_point
Group Disk Used Point Used%
gcb50169 0 16.0488 2,000 0
`- acb11190vf - 15.9042 - 0
➜ jobs cd ..
➜ ~ ls
apps job3.sh.o1199486 lanmt run.sh.e1199567 run.sh.o1199568 ru
n_single.sh.o1199570
base job3.sh.o1199487 nvidia-smi.sh.e1198810 run.sh.e1199568 run.sh.o1199576 ru
n_single.sh.o1199571
job3.sh.e1199486 job3.sh.o1199491 nvidia-smi.sh.o1198810 run.sh.e1199576 run.sh.o1199594
job3.sh.e1199487 jobs raphpack run.sh.e1199594 run_single.sh.e1199570
job3.sh.e1199491 known_hosts research run.sh.o1199567 run_single.sh.e1199571
➜ ~ cd lanmt
➜ lanmt git:(master) ✗ ls
datasets.py lib_lanmt_model.py lib_rescoring.py mydata README.md
hvd_test.py lib_lanmt_modules.py lib_vae.py __pycache__ run.py
➜ lanmt git:(master) ✗ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: run.py
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# hvd_test.py
no changes added to commit (use "git add" and/or "git commit -a")
➜ lanmt git:(master) ✗ git checkout -- run.py
➜ lanmt git:(master) ✗ git status
# On branch master
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# hvd_test.py
nothing added to commit but untracked files present (use "git add" to track)
➜ lanmt git:(master) ✗ git pull
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 13 (delta 9), reused 10 (delta 6), pack-reused 0
Unpacking objects: 100% (13/13), done.
From https://github.com/zomux/lanmt
6626083..c831e41 master -> origin/master
Updating 6626083..c831e41
Fast-forward
README.md | 42 ++++++++++++++++++++++--------------------
lib_lanmt_modules.py | 7 ++++++-
run.py | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+-
3 files changed, 105 insertions(+), 22 deletions(-)
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_F=4 -l h_rt=20:00:00 -m e ./run.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_finetune --opt_sgdft --opt_Trefine_st
eps 1 --train --test --evaluate
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 ./run_single.sh python $HOME/lanmt/run.py --op
t_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_finetune --opt_Trefine_steps 1 --test --evalua
te
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --opt_distill -│ `- acb11190vf - 15.9042
- 0
bquote> -opt_annealbudget --opt_finetune --opt_Trefine_steps 1 --test --evaluate
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --opt_distill -│ `- acb11190vf - 15.9042
- 0
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1199760 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ vi run.py
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1199761 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --opt_Tbatch_size 500 --batch_test --evaluate
Your job 1199762 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --opt_Tbatch_size 100 --batch_test --evaluate
Your job 1199763 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --opt_Tbatch_size 80 --batch_test --evaluate
Your job 1199764 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --opt_Tbatch_size 80 --batch_test --evaluate
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_F=1 -l h_rt=20:00:00 -m e ./run.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_fp16 --train Un
able to read script file because of error: error opening ./run.sh: No such file or directory
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_F=1 -l h_rt=20:00:00 -m e $HOME/jobs/run.sh python $HOME
/lanmt/run.py --opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_fp16 --train
Your job 1200326 ("run.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_F=1 -l h_rt=20:00:00 -m e $HOME/jobs/run.sh python $HOME
/lanmt/run.py --opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_fp16 --train
Your job 1200329 ("run.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_F=1 -l h_rt=20:00:00 -m e $HOME/jobs/run.sh python $HOME
/lanmt/run.py --opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_fp16 --train
➜ lanmt git:(master) ✗ qstatjob-ID prior name user state submit/start at queue
jclass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1200329 0.25586 run.sh acb11190vf r 10/06/2019 16:33:47 gpu@g0265
80
➜ lanmt git:(master) ✗ qdel 1200329
acb11190vf has registered the job 1200329 for deletion
➜ lanmt git:(master) ✗ qrsh -g gcb50169 -l rt_F=4
➜ lanmt git:(master) ✗ qdel 1200329
➜ lanmt git:(master) ✗ qrsh -g gcb50169 -l rt_G.small=1ca
➜ lanmt git:(master) ✗ cat job2.sh
➜ lanmt git:(master) ✗ qstat
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --opt_Tbatch_size 80 --batch_test --evaluate
Your job 1201389 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
➜ lanmt git:(master) ✗ git pull remote: Enumerating objects: 16, done.
remote: Counting objects: 100% (16/16), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 11 (delta 8), reused 8 (delta 5), pack-reused 0
Unpacking objects: 100% (11/11), done.
From https://github.com/zomux/lanmt
83792d9..6b2f048 master -> origin/master
Updating 83792d9..6b2f048
error: Your local changes to the following files would be overwritten by merge:
lib_lanmt_model.py
run.py
Please, commit your changes or stash them before you can merge.
Aborting
➜ lanmt git:(master) ✗ git checkout -- run.py
➜ lanmt git:(master) ✗ git checkout -- lib_lanmt_model.py
➜ lanmt git:(master) ✗ git status
# On branch master
# Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded.
# (use "git pull" to update your local branch)
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# hvd_test.py
nothing added to commit but untracked files present (use "git add" to track)
➜ lanmt git:(master) ✗ git pull
Updating 83792d9..6b2f048
Fast-forward
README.md | 8 +++++++-
lib_lanmt_model.py | 55 ++++++++++++++++++++++++++++++++++++--------------
lib_lanmt_modules.py | 9 ++++++++-
run.py | 81 ++++++++++++++++++++++++++++++++++++++++++++++----------------------------
4 files changed, 106 insertions(+), 47 deletions(-)
➜ lanmt git:(master) ✗ vi run.py
➜ lanmt git:(master) ✗ python run.py --opt_dtok wmt14_ende --opt_annealbudget --opt_distill --opt_fp16
--train
➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/lanmt/ru
n.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1230462 ("run_single.sh") has been submitted
➜ lanmt git:(master) ✗ pip install -U nmtlab Collecting nmtlab
Downloading https://files.pythonhosted.org/packages/c5/98/0b34f4ee6cb3815fc713fde3a392f042e47c379de17
331b5f1f162aa6b28/nmtlab-0.7.0.tar.gz (42kB)
100% |████████████████████████████████| 51kB 4.0MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-_IG2Uh/nmtlab/setup.py", line 4, in <module>
import nmtlab
File "nmtlab/__init__.py", line 4, in <module>
from .trainers import MTTrainer
File "nmtlab/trainers/__init__.py", line 4, in <module>
from .base import TrainerKit
File "nmtlab/trainers/base.py", line 16, in <module>
import torch
ImportError: No module named torch
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-_IG2Uh/nmtlab/
You are using pip version 8.1.2, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
➜ lanmt git:(master) ✗ pip install -U nmtlab==0.7.0
Collecting nmtlab==0.7.0
Using cached https://files.pythonhosted.org/packages/c5/98/0b34f4ee6cb3815fc713fde3a392f042e47c379de1
7331b5f1f162aa6b28/nmtlab-0.7.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-aQI4X3/nmtlab/setup.py", line 4, in <module>
import nmtlab
File "nmtlab/__init__.py", line 4, in <module>
from .trainers import MTTrainer
File "nmtlab/trainers/__init__.py", line 4, in <module>
from .base import TrainerKit
File "nmtlab/trainers/base.py", line 16, in <module>
import torch
ImportError: No module named torch
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-aQI4X3/nmtlab/
You are using pip version 8.1.2, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
➜ lanmt git:(master) ✗ source $HOME/base/bin/activate
(base) ➜ lanmt git:(master) ✗ pip install -U nmtlab==0.7.0
/home/acb11190vf/base/bin/python3: error while loading shared libraries: libpython3.6m.so.1.0: cannot o
pen shared object file: No such file or directory
(base) ➜ lanmt git:(master) ✗ module load cuda/10.0/10.0.130 cudnn/7.6/7.6.4 nccl/2.4/2.4.8-1 python/3
.6/3.6.5 openmpi/2.1.6
(base) ➜ lanmt git:(master) ✗ pip install -U nmtlab==0.7.0 Collecting nmtlab==0.7.0
Downloading https://files.pythonhosted.org/packages/5d/e8/8a27292952f1de4fa3f8fef40538a5c2b2ce6a16eeb
508b50763ae47a00e/nmtlab-0.7.0-py3-none-any.whl (53kB)
100% |████████████████████████████████| 61kB 4.3MB/s
Requirement already up-to-date: numpy in /home/acb11190vf/base/lib/python3.6/site-packages (from nmtlab
==0.7.0)
Requirement already up-to-date: sacrebleu in /home/acb11190vf/base/lib/python3.6/site-packages (from nm
tlab==0.7.0)
Collecting torch (from nmtlab==0.7.0)
Downloading https://files.pythonhosted.org/packages/ae/05/50a05de5337f7a924bb8bd70c6936230642233e424d
6a9747ef1cfbde353/torch-1.3.0-cp36-cp36m-manylinux1_x86_64.whl (773.1MB)
99% |████████████████████████████████| 773.1MB 106.4MB/s eta 0:00:01
100% |████████████████████████████████| 773.1MB 2.2kB/s
Requirement already up-to-date: typing in /home/acb11190vf/base/lib/python3.6/site-packages (from sacre
bleu->nmtlab==0.7.0)
Requirement already up-to-date: portalocker in /home/acb11190vf/base/lib/python3.6/site-packages (from
sacrebleu->nmtlab==0.7.0)
Installing collected packages: torch, nmtlab
Found existing installation: torch 1.2.0
Uninstalling torch-1.2.0:
Successfully uninstalled torch-1.2.0
Found existing installation: nmtlab 0.5.1
Uninstalling nmtlab-0.5.1:
Successfully uninstalled nmtlab-0.5.1
Successfully installed nmtlab-0.7.0 torch-1.3.0
You are using pip version 9.0.3, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(base) ➜ lanmt git:(master) ✗
(base) ➜ lanmt git:(master) ✗
(base) ➜ lanmt git:(master) ✗
(base) ➜ lanmt git:(master) ✗
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1230465 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ vi run.py
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1230466 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ pip3 install torch==1.2.0
Collecting torch==1.2.0
Using cached https://files.pythonhosted.org/packages/30/57/d5cceb0799c06733eefce80c395459f28970ebb9e8
96846ce96ab579a3f1/torch-1.2.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting numpy (from torch==1.2.0)
Using cached https://files.pythonhosted.org/packages/e5/e6/c3fdc53aed9fa19d6ff3abf97dfad768ae3afce1b7
431f7500000816bda5/numpy-1.17.2-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: numpy, torch
Exception:
Traceback (most recent call last):
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_whee
l_files
isolated=self.isolated,
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber
ensure_dir(destdir)
File "/apps/python/3.6.5/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
os.makedirs(path)
File "/apps/python/3.6.5/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/apps/python/3.6.5/lib/python3.6/site-packages/numpy-1.
17.2.dist-info'
You are using pip version 9.0.3, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(base) ➜ lanmt git:(master) ✗ pip install torch==1.2.0 Collecting torch==1.2.0
Using cached https://files.pythonhosted.org/packages/30/57/d5cceb0799c06733eefce80c395459f28970ebb9e8
96846ce96ab579a3f1/torch-1.2.0-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied: numpy in /home/acb11190vf/base/lib/python3.6/site-packages (from torch==
1.2.0)
Installing collected packages: torch
Found existing installation: torch 1.3.0
Uninstalling torch-1.3.0:
Successfully uninstalled torch-1.3.0
Successfully installed torch-1.2.0
You are using pip version 9.0.3, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1230470 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1230472 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ clear
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --batch_test --evaluate
Your job 1230474 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --batch_test --evaluate
Your job 1230475 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --batch_test
Your job 1230476 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
Your job 1230801 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --batch_test
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluate
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --batch_test
Your job 1232156 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --opt_Tbatch_size 16000 --batch_test
Your job 1232263 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok aspec_jaen --use_pretrain --opt_Tbatch_size 16000 --batch_test
(base) ➜ lanmt git:(master) ✗ qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/l
anmt/run.py --opt_dtok wmt14_ende --use_pretrain --batch_test --evaluatewd
Your job 1375327 ("run_single.sh") has been submitted
(base) ➜ lanmt git:(master) ✗ pwd
/home/acb11190vf/lanmt
(base) ➜ lanmt git:(master) ✗ cd ..
(base) ➜ ~ cd research
(base) ➜ research ls
deepy exp2019 lanmt nmtlab nonauto_mt repos tree2code
exp2018 github-backup neuralmt nnlab platoon theano wmt
(base) ➜ research cd exp201
cd: no such file or directory: exp201
(base) ➜ research cd exp2018
(base) ➜ exp2018 ls
fadernmt imdb mcgen namt opennmt stnmt tools
gram_detect lanamt model1 nnlab_baseline robot_code t2t torch_nmt
(base) ➜ exp2018 cd ..
(base) ➜ research cd ..
(base) ➜ ~ ls
apps run_single.sh.e1230475 run_single.sh.o1230472
base run_single.sh.e1230476 run_single.sh.o1230474
data run_single.sh.e1230801 run_single.sh.o1230475
jobs run_single.sh.e1232156 run_single.sh.o1230476
known_hosts run_single.sh.e1232263 run_single.sh.o1230801
lanmt run_single.sh.e1233322 run_single.sh.o1232156
raphpack run_single.sh.e1233324 run_single.sh.o1232263
research run_single.sh.e1233375 run_single.sh.o1233322
run_single.sh.e1201389 run_single.sh.e1233615 run_single.sh.o1233324
run_single.sh.e1230462 run_single.sh.e1375327 run_single.sh.o1233375
run_single.sh.e1230465 run_single.sh.o1201389 run_single.sh.o1233615
run_single.sh.e1230466 run_single.sh.o1230462 run_single.sh.o1375327
run_single.sh.e1230470 run_single.sh.o1230465 sdata
run_single.sh.e1230472 run_single.sh.o1230466
run_single.sh.e1230474 run_single.sh.o1230470
(base) ➜ ~ rm run_single.sh.*
(base) ➜ ~ ls
apps base data jobs known_hosts lanmt raphpack research sdata
(base) ➜ ~ cd base
(base) ➜ base ls
bin include lib lib64 pip-selfcheck.json pyvenv.cfg
(base) ➜ base cd ..
(base) ➜ ~ ls
apps base data jobs known_hosts lanmt raphpack research sdata
(base) ➜ ~ cd lanmt
(base) ➜ lanmt git:(master) ✗ ls
datasets.py lib_lanmt_model.py lib_rescoring.py mydata README.md
hvd_test.py lib_lanmt_modules.py lib_vae.py __pycache__ run.py
(base) ➜ lanmt git:(master) ✗ cd ..
(base) ➜ ~ cd research
(base) ➜ research ls
deepy exp2019 lanmt nmtlab nonauto_mt repos tree2code
exp2018 github-backup neuralmt nnlab platoon theano wmt
(base) ➜ research ls
deepy exp2019 lanmt nmtlab nonauto_mt repos tree2code
exp2018 github-backup neuralmt nnlab platoon theano wmt
(base) ➜ research cd mcgen cd: no such file or directory: mcgen
(base) ➜ research ls
deepy exp2019 lanmt nmtlab nonauto_mt repos tree2code
exp2018 github-backup neuralmt nnlab platoon theano wmt
(base) ➜ research cd
(base) ➜ ~ ls
apps base data jobs known_hosts lanmt raphpack research sdata
(base) ➜ ~ cd research
(base) ➜ research ls
deepy exp2019 lanmt nmtlab nonauto_mt repos tree2code
exp2018 github-backup neuralmt nnlab platoon theano wmt
(base) ➜ research cd stnmt
cd: no such file or directory: stnmt
(base) ➜ research ls
deepy exp2019 lanmt nmtlab nonauto_mt repos tree2code
exp2018 github-backup neuralmt nnlab platoon theano wmt
(base) ➜ research cd exp2018
(base) ➜ exp2018 ls
fadernmt imdb mcgen namt opennmt stnmt tools
gram_detect lanamt model1 nnlab_baseline robot_code t2t torch_nmt
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6 --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dto
k-codeplan_aspec
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 141, in <module>
assert os.path.exists(train_src_corpus)
AssertionError
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6 --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dto
k-codeplan_aspec
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 142, in <module>
assert os.path.exists(train_tgt_corpus)
AssertionError
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6 --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dto
k-codeplan_aspec
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 142, in <module>
assert os.path.exists(train_tgt_corpus)
AssertionError
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6 --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dto
k-codeplan_aspec
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 142, in <module>
assert os.path.exists(train_tgt_corpus)
AssertionError
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_dtok-codep
lan_aspec
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 142, in <module>
assert os.path.exists(train_tgt_corpus)
AssertionError
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_dtok-codep
lan_aspec
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 506, in <module>
assert os.path.exists(OPTS.model_path)
AssertionError
(base) ➜ exp2018 python stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads
8 --diversity_generate
[OPTS] Model tag: codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dto
k-codeplan_aspec_encl-6_gpus-8_heads-8
^[start generating results...
/home/acb11190vf/data/torch_nmt/aspec/aspec_test.en-ja.bpe40k.tsv
Traceback (most recent call last):
File "stnmt/exp2_treecode.py", line 519, in <module>
with open(result_path, "w") as outf:
FileNotFoundError: [Errno 2] No such file or directory: '/home/acb11190vf/data/stnmt/results/stnmt2_cod
ename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dtok-codeplan_aspec_encl
-6_gpus-8_heads-8.result.diversity'
^C^C
(base) ➜ exp2018
(base) ➜ exp2018 ls
fadernmt imdb mcgen namt opennmt stnmt tools
gram_detect lanamt model1 nnlab_baseline robot_code t2t torch_nmt
(base) ➜ exp2018 qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/research/exp20
18/stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-codelearn_aspec_gpus-8_t
inydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads 8 --diversity_generate
Your job 1375440 ("run_single.sh") has been submitted
(base) ➜ exp2018 vi $HOME/jobs/run_single.sh
(base) ➜ exp2018 qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/research/exp20
18/stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-codelearn_aspec_gpus-8_t
inydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads 8 --diversity_generate
Your job 1375442 ("run_single.sh") has been submitted
(base) ➜ exp2018 qstat
job-ID prior name user state submit/start at queue jcl
ass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1375442 0.25586 run_single acb11190vf r 10/28/2019 00:56:03 gpu@g0001
10
(base) ➜ exp2018 qstat
job-ID prior name user state submit/start at queue jcl
ass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1375442 0.25586 run_single acb11190vf r 10/28/2019 00:56:03 gpu@g0001
10
(base) ➜ exp2018 vi $HOME/jobs/run_single.sh
(base) ➜ exp2018 qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/research/exp20
18/stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-codelearn_aspec_gpus-8_t
inydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads 8 --diversity_generate
Your job 1375444 ("run_single.sh") has been submitted
(base) ➜ exp2018 vi $HOME/jobs/run_single.sh
(base) ➜ exp2018 qsub -g gcb50169 -l rt_G.small=1 $HOME/jobs/run_single.sh python $HOME/research/exp20
18/stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-codelearn_aspec_gpus-8_t
inydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads 8 --diversity_generate
Your job 1375445 ("run_single.sh") has been submitted
(base) ➜ exp2018 qstat
(base) ➜ exp2018 bash job2.sh
(base) ➜ exp2018 qsub -g gcb50169 -l rt_F=1 -l h_rt=20:00:00 -m e $HOME/jobs/run.sh python $HOME/lanmt
/run.py --opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_fp16 --train
(base) ➜ exp2018 less stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrai
n_decl-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
(base) ➜ exp2018 qsub -g gcb50169 -l rt_G.small=1 -l h_rt=20:00:00 $HOME/jobs/run_single.sh python $H
OME/research/exp2018/stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-codele
arn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads 8 --
diversity_generate
(base) ➜ exp2018 qsub
^CUnable to read script file because of error: no input read from stdin
(base) ➜ exp2018 qsub -h
Usage: qsub [options]
Options:
[-a date_time] request a start time
[-adds list_attr_name attr_key attr_valu add additional sublist parameter
[-ac context_list] add context variable(s)
[-ar ar_id] bind job to advance reservation
[-b y[es]|n[o]] handle command as binary
[-clear] skip previous definitions for job
[-clearp attr_name] clear/delete a parameter
[-clears list_attr_name attr_key] clear/delete sublist parameter
[-cwd] use current working directory
[-C directive_prefix] define command prefix for job script
[-dc simple_context_list] delete context variable(s)
[-e path_list] specify standard error stream path(s)
[-g group_name] specify abci group(command line only)
[-h] place user hold on job
[-hard] consider following requests "hard"
[-help] print this help
[-hold_jid job_identifier_list] define jobnet interdependencies
[-hold_jid_ad job_identifier_list] define jobnet array interdependencies
[-i file_list] specify standard input stream file(s)
[-inherit] inherit existing job environment JOB_ID for rsh(qrsh only)
[-j y[es]|n[o]] merge stdout and stderr stream of job
[-jsv jsv_url] job submission verification script to be used
[-l resource_list] request the given resources
[-m mail_options] define mail notification events
[-mods list_attr_name attr_key attr_valu modify sublist parameter
[-notify] notify job before killing/suspending it
[-now y[es]|n[o]] start job immediately or not at all
[-M mail_list] notify these e-mail addresses
[-N name] specify job name
[-o path_list] specify standard output stream path(s)
[-p priority] define job's relative priority
[-pty y[es]|n[o]] start job in a pty
[-R y[es]|n[o]] reservation desired
[-rou reporting variable list] write online usage of job to the reporting database
[-sc context_list] set job context (replaces old context)
[-shell y[es]|n[o]] start command with or without wrapping <loginshell> -c
[-soft] consider following requests as soft
[-sync sync_options] define sync notification events
[-S path_list] command interpreter to be used
[-t task_id_range] create a job-array with these tasks
[-tc max_running_tasks] throttle the number of concurrent tasks
[-tcon y[es]|n[o]] run all tasks concurrently
[-terse] tersed output, print only the job-id
[-v variable_list] export these environment variables
[-verify] do not submit just verify
[-V] export all environment variables
[-wd working_directory] use working_directory
[-@ file] read commandline input from file
[{command|-} [command_args]]
attr_key key of a sublist element
attr_value (new) value for a sublist element
attr_name name of a job attribute
context_list variable[=value][,variable[=value],...]
date_time [[CC]YY]MMDDhhmm[.SS]
job_identifier_list {job_id|job_name|reg_exp}[,{job_id|job_name|reg_exp},...]
jsv_url [script:][username@]path
mail_address username[@host]
mail_list mail_address[,mail_address,...]
mail_options `e' `b' `a' `n' `s'
list_attr_name name of a list based job attribute
working_directory path
path_list [host:]path[,[host:]path,...]
file_list [host:]file[,[host:]file,...]
priority -1023 - 1024
resource_list resource[=value][,resource[=value],...]
reporting_variable_list variable[,variable,...]
simple_context_list variable[,variable,...]
task_id_range task_id['-'task_id[':'step]]
variable_list variable[=value][,variable[=value],...]
ar_id advance reservation id
max_running_tasks maximum number of simultaneously running tasks
sync_options `y' `n' `r' `l'
Report bugs to <[email protected]>
(base) ➜ exp2018 mv stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_
decl-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity stnmt2_codename-stnmt2_dtok-codelearn
_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.di
versity.bak
(base) ➜ exp2018 qsub -g gcb50169 -l rt_F=1 -l h_rt=20:00:00 -m e ./run.sh python $HOME/lanmt/run.py -
-opt_dtok wmt14_ende --opt_distill --opt_annealbudget --opt_fp16 --train
(base) ➜ exp2018 qsub -g gcb50169 -l rt_G.small=1 -l h_rt=20:00:00 -m e $HOME/jobs/run_single.sh pytho
n $HOME/research/exp2018/stnmt/exp2_treecode.py --opt_dtok codeplan_aspec --opt_codename stnmt2_dtok-co
delearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain --opt_encl 6 --opt_decl 6 --opt_gpus 8 --opt_heads
8 --diversity_generate
Your job 1375589 ("run_single.sh") has been submitted
(base) ➜ exp2018 qstat
job-ID prior name user state submit/start at queue jcl
ass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1375589 0.25586 run_single acb11190vf r 10/28/2019 07:13:18 gpu@g0001
10
(base) ➜ exp2018 qstat
job-ID prior name user state submit/start at queue jcl
ass slots ja-task-ID
-------------------------------------------------------------------------------------------------------
-----------------------------------------
1375589 0.25586 run_single acb11190vf r 10/28/2019 07:13:18 gpu@g0001
10
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dtok-codeplan_
aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 34, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dtok-codeplan_
aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 33, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dtok-codeplan_
aspec_encl-6_gpus-8_heads-8.result.diversity
/home/acb11190vf/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl
-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 34, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl-6_dtok-codeplan_
aspec_encl-6_gpus-8_heads-8.result.diversity
/home/acb11190vf/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl
-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 33, in <module>
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
/home/acb11190vf/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl
-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 32, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
/home/acb11190vf/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_decl
-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 32, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
/home/acb11190vf/stnmt/results/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w
_pretrain_decl-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 32, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
/home/acb11190vf/stnmt/results/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w
_pretrain_decl-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 32, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
/home/acb11190vf/stnmt/results/stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w
_pretrain_decl-6_dtok-codeplan_aspec_encl-6_gpus-8_heads-8.result.diversity
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 32, in <module>
raise SystemError
SystemError
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 74, in <module>
tokens = sent.split()
AttributeError: 'list' object has no attribute 'split'
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 49.4038350266527
bleus: 17.940755090997268
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 49.701313000385575
bleus: 18.09870667456805
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 49.01721412284288
bleus: 18.260397849765493
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 49.31555238578651
bleus: 18.22939743552324
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 49.30044362549329
bleus: 18.155653616029557
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(59)<module>()
-> for i in range(len(sents)):
(Pdb++) ll
1 #!/usr/bin/env python
2 # -*- coding: utf-8 -*-
3
4 from __future__ import absolute_import
5 from __future__ import print_function
6
7 import os
8 from collections import defaultdict
9 from nmtlab.utils.bleu import bleu
10 import numpy as np
11 import random
12 from argparse import ArgumentParser
13
14 ROOT = "{}/data/wmt/results".format(os.environ.get("HOME"))
15 HOME_ROOT = os.getenv("HOME")
16
17 fname = "stnmt2_codename-stnmt2_dtok-codelearn_aspec_gpus-8_tinydata_w_8bitcode_w_pretrain_dec
18 if "aspec" in fname:
19 ref_path = "{}/data/torch_nmt/aspec/aspec_test.case.en".format(os.getenv("HOME"))
20 elif "wmt14" in fname:
21 ref_path = "{}/data/wmt14_ende/raw/wmt14_deen_test.en".format(os.getenv("HOME"))
22
23 ap = ArgumentParser()
24 ap.add_argument("--bleu", action="store_true")
25 ap.add_argument("--diversity", action="store_true")
26 args = ap.parse_args()
27
28 # Store candidates
29 fp = "{}/data/stnmt/results/{}".format(HOME_ROOT, fname)
30 if not os.path.exists(fp):
31 print(fp)
32 raise SystemError
33
34 # Collection translation results
35 result_map = defaultdict(list)
36 for line in open(fp):
37 if "\t" not in line:
38 continue
39 i, sentence = line.split("\t")
40 i = int(i)
41 if "<eoc>" in sentence:
42 sentence = sentence.split("<eoc>")[1].strip()
43 if "▁" in sentence:
44 sentence = "".join(sentence.split()).replace("▁", " ").strip()
45 result_map[i].append(sentence)
46
47 # Crreate reference
48 references = list(map(str.strip, open(ref_path).readlines()))
49
50 for _ in range(1):
51 # Compute diversities
52 diversities = []
53 quality_scores = []
54 for k in sorted(result_map.keys()):
55 sents = result_map[k]
56 # sents = random.sample(sents, 3)
57 sents = sents[:3]
58 import pdb;pdb.set_trace()
59 -> for i in range(len(sents)):
60 tokens = sents[i].split()
61 # sents[i] = [x[1] for x in nltk.pos_tag(tokens)]
62 sents[i] = tokens
63 bleus = []
64 for i in range(len(sents)):
65 for j in range(len(sents)):
66 if i != j:
67 b = bleu(sents[i], sents[j])
68 bleus.append(100 - b)
69 if not bleus:
70 continue
71 diversities.append(np.mean(bleus))
72 # Compute BLEUs
73 bleus = []
74 ref_tokens = references[k].split()
75 for tokens in sents:
76 b = bleu(tokens, ref_tokens)
77 bleus.append(b)
78 quality_scores.append(np.mean(bleus))
79 print("mean discrepancy:", np.mean(diversities))
80 print("bleus:", np.mean(quality_scores))
(Pdb++) sents
["the dose rate of the `` Fugen '' can be calculated in detail using the DERS software .\n", "the dose
rate of the `` Fugen '' can be calculated in detail using the DERS software .\n", "the dose rate of the
`` Fugen '' is calculated in detail using the DERS software .\n"]
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(60)<module>()
-> tokens = sents[i].split()
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(62)<module>()
-> sents[i] = tokens
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(59)<module>()
-> for i in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(60)<module>()
-> tokens = sents[i].split()
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(62)<module>()
-> sents[i] = tokens
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(59)<module>()
-> for i in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(60)<module>()
-> tokens = sents[i].split()
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(62)<module>()
-> sents[i] = tokens
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(59)<module>()
-> for i in range(len(sents)):
(Pdb++) sents
[['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can', 'be', 'calculated', 'in', 'detail', '
using', 'the', 'DERS', 'software', '.'], ['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can
', 'be', 'calculated', 'in', 'detail', 'using', 'the', 'DERS', 'software', '.'], ['the', 'dose', 'rate'
, 'of', 'the', '``', 'Fugen', "''", 'is', 'calculated', 'in', 'detail', 'using', 'the', 'DERS', 'softwa
re', '.']]
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(63)<module>()
-> bleus = []
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(64)<module>()
-> for i in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) b = bleu(sents[i], sents[j])
*** IndentationError: unexpected indent
(Pdb++) b = bleu(sents[i], sents[j])
(Pdb++) b
100.0
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(67)<module>()
-> b = bleu(sents[i], sents[j])
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(68)<module>()
-> bleus.append(100 - b)
(Pdb++) v
*** NameError: name 'v' is not defined
(Pdb++) b
100.0
(Pdb++) sents[i]
['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can', 'be', 'calculated', 'in', 'detail', 'u
sing', 'the', 'DERS', 'software', '.']
(Pdb++) sents[j]
['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can', 'be', 'calculated', 'in', 'detail', 'u
sing', 'the', 'DERS', 'software', '.']
(Pdb++) i
0
(Pdb++) j
1
(Pdb++) sents
[['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can', 'be', 'calculated', 'in', 'detail', '
using', 'the', 'DERS', 'software', '.'], ['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can
', 'be', 'calculated', 'in', 'detail', 'using', 'the', 'DERS', 'software', '.'], ['the', 'dose', 'rate'
, 'of', 'the', '``', 'Fugen', "''", 'is', 'calculated', 'in', 'detail', 'using', 'the', 'DERS', 'softwa
re', '.']]
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(67)<module>()
-> b = bleu(sents[i], sents[j])
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(68)<module>()
-> bleus.append(100 - b)
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(64)<module>()
-> for i in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(67)<module>()
-> b = bleu(sents[i], sents[j])
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(68)<module>()
-> bleus.append(100 - b)
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(67)<module>()
-> b = bleu(sents[i], sents[j])
(Pdb++) bleus
[0.0, 22.218887769457808, 0.0]
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(68)<module>()
-> bleus.append(100 - b)
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(64)<module>()
-> for i in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(67)<module>()
-> b = bleu(sents[i], sents[j])
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(68)<module>()
-> bleus.append(100 - b)
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(67)<module>()
-> b = bleu(sents[i], sents[j])
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(68)<module>()
-> bleus.append(100 - b)
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(66)<module>()
-> if i != j:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(65)<module>()
-> for j in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(64)<module>()
-> for i in range(len(sents)):
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(69)<module>()
-> if not bleus:
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(71)<module>()
-> diversities.append(np.mean(bleus))
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(73)<module>()
-> bleus = []
(Pdb++) diversities
[14.708543826907729]
(Pdb++) c
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(58)<module>()
-> sents = sents[:3]
(Pdb++)
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 58, in <module>
sents = sents[:3]
File "stnmt/random_diversity_bleu.py", line 58, in <module>
sents = sents[:3]
File "/apps/python/3.6.5/lib/python3.6/bdb.py", line 51, in trace_dispatch
return self.dispatch_line(frame)
File "/apps/python/3.6.5/lib/python3.6/bdb.py", line 70, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 39.82948695047773
bleus: 20.70490720882358
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
[14.708543826907729, 25.715588104552243, 61.23460559016948, 60.09783027966574, 27.88671979181844, 24.24
273225090592, 64.36688862866446, 19.29454005251101, 7.525872901771365, 51.01632681845707, 22.6381754621
29343, 11.713369184947473, 41.091139202363486, 39.32421947075627, 33.27077894744988, 38.779901624309765
, 28.84445435939988, 5.18934914802152, 57.162532738473196, 40.63914259839619, 31.72306004690112, 27.834
989677345693, 77.97890454657461, 41.02220905807824, 19.43286774149692, 51.5104371093424, 58.70528913362
9016, 66.66666666666667, 29.385599245528837, 82.73323511473474, 19.5461390273006, 19.367607389987658, 8
.959796316388614, 0.0, 61.14213426242952, 30.212077265668267, 15.05608754611692, 20.428762965222862, 38
.856753673104286, 66.66666666666667, 15.286872583731451, 19.22418679137597, 59.79131960698328, 32.41676
0173661636, 36.606742617669546, 11.186304115076581, 31.749379626743263, 27.123192213586993, 0.0, 14.280
956991977556, 42.11831491705504, 85.50930426850027, 49.302195742560365, 75.85937747856238, 47.210529308
504526, 33.19089469406086, 50.821161868281195, 18.056745155528716, 39.78418889295106, 26.33438125734216
, 56.480923614575914, 18.078223914704783, 60.09942586107129, 42.57888648704286, 64.73916531137023, 30.1
19105662644213, 33.218310185047336, 68.3591163800646, 53.75571400867559, 38.90907370073733, 27.78076079
9545746, 30.752243380252395, 8.804366299682338, 44.84253812659153, 64.17944549885887, 56.01179827557811
6, 19.284980731904966, 36.28018791264939, 60.30796957390461, 28.234034271104907, 59.08567991704132, 8.9
67302072516105, 9.927058878984658, 55.34605447197547, 70.15489562527576, 67.41572476275645, 16.70512276
424571, 48.385527493213736, 45.39522097315261, 75.52592959531033, 40.19064875593653, 58.40481446500087,
0.0, 39.59459089146498, 23.054414024261643, 60.970860493267445, 53.25090874976002, 15.069436704471556,
25.82699940753464, 50.026757070042315, 85.23292148628173, 66.66666666666667, 27.683338023613945, 57.06
983552314919, 58.95572794204483, 71.24861534997346, 40.25991438205664, 47.913414449614926, 48.929188111
37498, 83.07585528148724, 34.35249326053972, 56.21554436263546, 45.384419732882016, 39.74717822570515,
32.043895014173756, 64.32175771137246, 55.31498320183758, 50.66201146833516, 68.41865582992972, 29.5522
60059418824, 55.1365327705335, 22.815761489258325, 74.22655680469519, 35.84844148313132, 22.06610283172
2674, 45.6000968026656, 55.64044108684829, 59.12272917346424, 27.625010199502327, 27.179779519845013, 4
9.49454566245753, 71.96287996202793, 10.009119534046723, 43.70170998749629, 13.03844236680061, 12.53690
1542777622, 33.39873879141662, 25.54504561840646, 40.553935416416564, 59.427253987571966, 46.5049030903
7796, 32.96413559925846, 29.96882688497423, 6.08966926222475, 52.54290878419628, 35.40657432662729, 58.
70538908174128, 21.784248802179093, 33.06810532308447, 56.521259224186544, 32.84481243987682, 24.070189
31187102, 33.030771476239146, 100.0, 21.312535959555944, 15.110217196652584, 46.258032709815865, 76.422
48628400934, 41.74004463431119, 72.40867370498866, 25.422791113526866, 83.81694094308922, 23.7789092428
1495, 56.032855058551, 69.57089599641472, 36.00676097650049, 36.91876395928583, 13.428033027944034, 19.
11908247931352, 47.789926488655034, 30.30479722800547, 30.170525338662316, 52.925892284059216, 13.82228
9821072408, 30.92655659804898, 34.23059619668178, 37.923749993713905, 45.72006144694893, 20.32776176887
8503, 56.4344811687558, 50.7374206519159, 73.12616749858306, 24.59329851111096, 17.48671639738771, 54.2
7224096617349, 82.34074042375306, 53.1524537660516, 8.476372066606032, 22.974772026481617, 63.374044339
76186, 35.25193426829903, 47.58643277781163, 24.454677735327124, 36.044806338540624, 47.18212472378021,
50.36352838052642, 80.09037862731255, 31.86488408281218, 59.09956437912396, 24.287602955201418, 48.242
320083419564, 82.09305990747445, 71.12849614966039, 48.686889728880736, 88.42204269462404, 34.742430125
58279, 22.723057803765176, 9.24940682703363, 13.200732738412492, 85.04279642762566, 64.93189439405533,
16.674598090606708, 39.12981507929184, 70.91213220267058, 0.0, 57.67865969864591, 36.56596636416271, 50
.19942138306877, 9.950722167214863, 100.0, 30.264695342722636, 75.65537704917396, 38.17557480497076, 21
.70198633870567, 17.841641132013255, 53.45989410979863, 0.0, 23.36238009396729, 75.28815856783605, 52.1
3120342949849, 19.65864719083937, 92.01272364762268, 41.99746146853444, 66.76808278398089, 54.274273438
30942, 34.03663290452373, 43.32963207044116, 30.796340596432753, 83.16174951148822, 26.32426558780732,
23.364740038951126, 86.30607891415814, 0.0, 17.10901915609576, 63.490508551358765, 0.0, 53.261051305576
046, 34.57597794001189, 50.681932978225284, 19.326038314294806, 58.86314379653581, 0.0, 26.227868574781
06, 46.051678899555945, 36.23180626714339, 49.77304460059955, 15.835609425100927, 30.362712386904104, 5
8.786204962982175, 26.339163730496765, 28.34445971276538, 47.54954161455276, 28.68214465613814, 58.7657
9451718475, 59.75455497715732, 43.21959851073228, 39.29344390212184, 32.092287997133774, 16.71738726274
157, 33.067161979397234, 37.017420941778546, 33.17112254186632, 65.29913695775186, 60.379943842588546,
59.88356028989764, 58.18310116757241, 54.19871895129742, 68.16887931120704, 41.86021353402728, 73.90404
787625657, 43.991306084553834, 55.32541203785718, 48.2638660085501, 14.41511180180541, 47.7335041512541
3, 30.88319738906593, 19.11908247931352, 12.455602617861608, 34.810300191764675, 58.45741738939068, 41.
54465754290024, 6.503188292815291, 54.865912104522444, 43.036138509439375, 39.46774971783186, 53.482184
052862216, 32.6911061867987, 38.322645656617794, 37.77905503308161, 32.00388183901419, 0.0, 50.34515351
914379, 36.61279247582929, 21.559331821408602, 28.488690059200326, 56.89644090513183, 28.06643005683787
4, 36.262972091823364, 80.91951238543281, 40.919102669318896, 66.66666666666667, 67.73334098706306, 44.
51844080191274, 41.297398509382944, 23.730473191707613, 17.67407199416895, 27.072926426289815, 34.62545
980977048, 0.0, 44.08184683556606, 4.18178178540948, 31.109227149521445, 14.637139498992639, 35.1047229
8213917, 6.764026110535696, 92.28990758735581, 17.014190362535476, 56.14305916636601, 14.69231936809126
2, 25.82552395808985, 18.169194401515856, 42.971213732300356, 0.0, 32.79948398540716, 29.05489901720581
, 36.64151235601759, 36.0259678931345, 38.31744063405919, 42.65617590691864, 36.284890705847076, 43.130
15554888191, 0.0, 72.95619482746955, 26.86914769391484, 8.08571830150431, 39.90985020895192, 51.2996078
69693354, 41.00182824400582, 5.619308308588226, 6.477504344438851, 53.28000980960899, 76.75702744244678
, 57.98117478502464, 27.251030814862116, 47.90481212252531, 59.446531444350974, 22.908863677142815, 66.
66666666666667, 57.26792548757887, 45.489309543973604, 59.470035873806346, 34.76758181164431, 51.759995
425750965, 31.5187300874521, 36.745432417811024, 19.303367846766537, 59.39691852203851, 32.960623122207
11, 48.24167015125059, 63.67809985987312, 24.24713112390965, 29.29806912450346, 49.421628160679006, 55.
940224630512866, 44.72592203354538, 53.73093099913177, 0.0, 38.79446877199004, 51.98235911334765, 73.85
570615520653, 21.56402928867144, 44.33674790090617, 58.84818093562669, 57.8506942025387, 21.13790810970
482, 56.932849803706496, 28.6781659017761, 36.676532215902746, 38.46057219419577, 61.440219116275735, 2
0.011919636244656, 83.70426464686214, 22.35808148093488, 30.69352321237695, 34.324475216026954, 25.1506
38195916013, 58.83669560254348, 31.693130786409952, 35.61506288875056, 32.85349503213043, 29.0857345748
5892, 10.196069081508284, 60.17306171249988, 25.462899911589208, 6.434368342603913, 40.06318161442926,
55.15640082680803, 20.082909886990024, 38.959622475871925, 75.37676333405518, 36.11123832756676, 80.038
19131884438, 24.733930682212062, 14.993811048759582, 51.34024252532084, 48.33715268128751, 72.859235942
58914, 25.55028530561388, 0.0, 39.710649534696664, 48.7006590260003, 36.623922146629575, 58.26211241924
555, 74.3218041148736, 62.453565009467845, 28.03817142621642, 49.965279316327894, 75.4356098340475, 13.
043312537714181, 44.02554618562209, 74.0315366042967, 0.0, 50.40913057873797, 70.20251198928317, 47.755
454278797494, 47.2883901657915, 44.82322949172931, 26.297861642645444, 19.99035263079878, 68.4646463135
3649, 51.09258043195979, 45.26184216331302, 85.5223191165778, 0.0, 23.820340511095512, 49.3689030793438
3, 38.50912336586015, 76.45756313000184, 60.87333665871906, 39.24332244562001, 64.63582811758619, 100.0
, 40.150103245746486, 42.316260572928826, 0.0, 32.779638397422666, 25.21611569143394, 24.46336863627119
5, 60.306104976078366, 67.58909335811148, 42.095623336689165, 49.552443958823424, 20.083308682797064, 1
1.323802344125118, 66.66514416086544, 0.0, 31.37861217366056, 29.775143703695008, 45.041173590409954, 4
4.53314846996981, 53.92820309307495, 74.78901908648099, 62.604236343093966, 26.393901117930593, 0.0, 29
.006479745427324, 58.12585441046326, 44.10643630798665, 21.22723310703631, 19.48831005318316, 39.192556
7779416, 24.21476923086256, 53.50499590576589, 14.020697549582374, 49.330163736966334, 0.0, 73.47588820
258619, 34.42818105568639, 0.0, 33.80588461004188, 28.42172942846908, 58.66753991031129, 74.56980402096
147, 85.09619053384785, 0.0, 42.69728979036745, 31.651361162343886, 38.537233136600925, 27.062924407030
383, 30.78824755291671, 20.765968748225443, 32.2962433629478, 15.878172777772269, 60.02321260195347, 66
.75071608100761, 52.09231310985809, 31.706536069046482, 69.06187888160939, 66.66666666666667, 37.920093
49965041, 28.109300351817723, 100.0, 59.608853435384596, 50.29155576883122, 39.71412628714069, 47.84187
0236985415, 16.137113251840304, 54.9405347428405, 50.60757056882934, 69.12460061505416, 0.0, 8.40432449
9687979, 23.123720089196706, 39.7433039145353, 30.479000395023434, 44.97238750496725, 39.35302716557836
, 43.99341061812558, 54.05208158823256, 71.54406274494653, 40.76509064319402, 15.364780090690894, 49.46
085737782327, 64.78953287075905, 53.09950071812858, 7.601223066342428, 86.08383033786004, 45.7516655849
8378, 44.732499730923166, 24.555945203846914, 15.703541041045256, 35.67714763313497, 1.8798270701685358
, 67.19309907879239, 26.24630652295293, 61.481294354702015, 31.767061449218385, 27.600301039208222, 35.
661191594076406, 62.701073311938046, 12.363878932960228, 27.278677773242467, 33.08969660932357, 57.1369
80320805115, 41.02677531288682, 20.490036882967136, 77.47734458468146, 38.18883334511, 68.3609671015719
9, 0.0, 30.538688576193874, 12.969862290753346, 34.76094308844093, 36.881282397438696, 26.4615059373066
58, 24.828220836377948, 47.60551586806395, 50.494432606322256, 40.10606204196975, 20.657908323388487, 4
5.42386964120056, 60.10769656630239, 35.08896650881042, 56.42922602274094, 63.881142481547386, 38.81319
9127972524, 27.365874668847514, 26.841825970869944, 60.98791731522193, 43.31260815904329, 66.6666666666
6667, 29.295281821630397, 46.58621950599089, 61.66651928762502, 46.44220756158223, 73.97697714746475, 5
6.169873133248984, 44.32439185109214, 31.011604581074156, 23.29432614916347, 17.066346002568224, 45.520
248190992845, 28.872234647373944, 12.997890504176945, 62.13873053163496, 38.78108337653615, 19.95889249
5463555, 32.940384990557625, 58.316322663709315, 33.706717661777574, 75.06694112438042, 0.0, 54.5231241
6572415, 40.39756015619278, 35.73638780390484, 45.50676704225944, 66.10178507105199, 27.67022763796055,
31.59442296914857, 71.16635932502196, 22.258252923343793, 55.57620391037011, 19.366047168930496, 35.69
823613798495, 57.51256538902595, 33.39029151115628, 17.666252443140984, 26.621144444775968, 25.38897497
828009, 32.391668545930806, 37.65693762640053, 26.397354803352766, 18.699419206804887, 23.8436185408476
04, 60.06480321224873, 57.254813181186336, 75.86326279272686, 44.80509180206662, 21.11234743351417, 19.
58150906324097, 18.022130930533788, 10.25916538372014, 43.00396638970174, 19.778119350197183, 14.571397
102228957, 28.895314441187583, 16.272553334632068, 29.59636843255895, 23.14217179751792, 44.56135898671
945, 45.48175237820877, 64.31064806713903, 20.950945312971218, 30.874033312828953, 18.49559958255273, 2
8.415982293994148, 21.023353247123833, 0.0, 46.440917800765476, 57.58542038337305, 78.05598295995817, 5
3.04903870472584, 81.38683911758606, 52.18473744312754, 31.298478240444293, 73.76750371760185, 27.02642
9499909298, 15.59538779942433, 56.03369927746407, 40.8239622966365, 20.504595930892403, 18.356386820576
67, 41.66173105886745, 55.692198294511144, 29.663076041319282, 17.196661269400916, 19.034276001036226,
54.85570661324152, 58.0481918064211, 11.724937105828852, 12.126808054871901, 22.840947903012918, 21.344
70475668166, 51.487505036623396, 45.7290497903036, 72.17627811670486, 31.57066512771151, 40.05472032994
672, 16.510557130494522, 40.92653557276875, 20.788505835886138, 36.56123017551707, 5.392214638376217, 4
3.86204732108259, 13.942535112227842, 18.953753905204653, 7.051629720143278, 56.32539656299272, 34.6068
9080280222, 34.83544047249006, 0.0, 14.890612611859856, 14.282841258696044, 24.033711684337575, 22.1875
4393159223, 24.763621733322328, 18.901781342825817, 35.65077456240737, 76.81332889593882, 95.0565947462
3754, 59.22912873226344, 35.54823418346221, 20.157710381326122, 28.03933390858018, 63.39065946225219, 3
9.240076044617986, 38.73911108280977, 51.50209703751961, 26.02391565328337, 11.717549982359637, 60.0195
5885455431, 17.48389060897973, 37.72574090950122, 72.5895615149236, 22.06682218782646, 47.7006474108700
4, 19.304060648203077, 21.623923206381505, 6.137897292475984, 19.746625608133964, 83.007957071188, 45.5
31069396103135, 36.01156659613892, 11.693937275829995, 61.01819499771705, 4.781259986983888, 14.1591736
44998987, 62.655003984435474, 6.254170434135697, 34.028611901221545, 26.428525919559252, 38.15297686065
969, 85.45135896704129, 63.42445176319316, 0.0, 65.83735021431686, 43.868336516738, 42.01000977070967,
31.495283285171336, 51.86639953720172, 33.61663487526398, 21.766789301319616, 8.6549210260971, 8.046229
286190538, 38.82780440613599, 16.778539801764207, 21.44813312548652, 85.99407482858443, 32.319780641174
96, 14.872228801676775, 65.49429001475514, 24.804398097719385, 49.06152859050644, 59.870703521109135, 1
6.63109211552653, 46.17684644900609, 43.44033298242072, 0.0, 16.032001535390208, 9.28069960018494, 48.1
9592254140613, 18.287266687854316, 31.571231211209206, 21.928780625135232, 59.62292304206687, 39.074744
794038125, 20.75775391891575, 59.87259707511469, 19.07176486104872, 31.609686607389367, 11.236771836069
465, 0.0, 21.782306613111775, 28.942009719455044, 14.282675686533352, 30.422134785482758, 16.7992379402
70185, 100.0, 47.85674548994143, 8.354740284011001, 88.6171704420863, 56.56908028696185, 10.16799979742
6594, 37.211054729190124, 12.919001663832788, 42.84208708250435, 22.262989539819085, 20.293324458544323
, 38.89555952018404, 12.152328360330023, 47.90756650714766, 57.70956221906505, 17.773541951093822, 23.1
5008283521644, 22.206967562538626, 38.667664096468194, 78.59953821446311, 29.728675983625802, 41.578853
508125384, 15.004419734741761, 26.276080636625267, 17.255740842771413, 56.096669317601396, 43.782376524
37111, 22.152969236846314, 66.66666666666667, 58.80774987150439, 44.04519413521115, 22.13584381948611,
66.66666666666667, 40.069552406531166, 22.620104054111664, 11.038969435603088, 20.101769219114882, 66.6
6666666666667, 17.336007740828762, 0.0, 25.534159085117494, 17.42363187639199, 13.885337839077506, 14.1
7698798624617, 13.148476780835486, 15.315310332842097, 15.906986681876923, 17.76491147824732, 19.758544
119977703, 34.48873901122165, 11.536689422752948, 18.983683131966245, 11.512435865295648, 32.0188162628
0628, 48.03102271424194, 49.56111996297671, 23.91541692601669, 27.80503109047859, 50.036521303004925, 4
2.183957132686636, 68.9629724066668, 21.727777581811253, 35.57830126731174, 44.35658420408895, 46.98633
0124926475, 39.212054807916495, 15.92262664433405, 61.29374049555398, 29.505560245955888, 45.8018590053
89254, 41.161907427979145, 50.05876712189083, 37.36656587798757, 0.0, 9.172904858731945, 72.28042306253
273, 28.245395918488146, 100.0, 72.93511743805549, 55.1522172839931, 78.10492521676586, 0.0, 48.2605218
2009814, 57.99343352635819, 55.005907483310445, 0.0, 0.0, 44.046348105329656, 10.051155761595405, 4.766
565222527826, 38.01981538415851, 78.34247227176306, 36.178687548642216, 11.08944054616938, 20.705302446
78919, 27.470225918734773, 30.69352321237695, 23.211345581301487, 20.247346421922288, 6.662966473477279
, 26.603571963909626, 39.41070438390794, 20.87480381344613, 15.080639616387137, 35.6751309766066, 39.44
680944325226, 50.943288237313396, 66.66666666666667, 80.8775313809012, 33.77203055715051, 62.4964375296
43536, 30.895372123790708, 14.708543826907729, 19.01275329662542, 76.43831971878974, 13.812698477876614
, 50.57651452312422, 33.30932743066159, 85.63793113816978, 4.18178178540948, 9.479886412120862, 42.3948
3706343128, 24.21269939246999, 39.95220351255622, 36.663120697214374, 100.0, 57.314718733942975, 21.644
090419882474, 41.31205254159311, 70.687735777511, 52.662749680436974, 79.76451312816322, 54.09346109182
311, 53.16927644812338, 55.32905154880692, 20.288409356478567, 53.31837270272944, 0.0, 55.2940758631246
7, 65.40197425793095, 36.11805714306813, 0.0, 50.626475372207715, 46.406069044552176, 0.0, 9.0649523428
81705, 38.00289645904331, 30.875061101581156, 33.596859454071925, 80.4984314404074, 66.227926376631, 58
.69844865962428, 32.66294834174867, 87.74950794378226, 54.042375721446916, 57.98536789235148, 29.132481
41901049, 75.7584863566662, 89.35886299599025, 39.85997502531199, 19.449375779885685, 0.0, 19.767934843
868264, 0.0, 38.15381535867127, 78.71267108442103, 44.80501733475247, 65.94509898395026, 56.39211434478
731, 44.85677419073486, 70.2385121325422, 27.410905403476335, 76.48397254868155, 53.58151773633909, 26.
285169646776126, 14.1048804468153, 0.0, 26.92948728842526, 5.749526606331277, 25.51464810808682, 57.299
62811502628, 14.839591778524442, 61.07501771383261, 50.45306953697653, 21.676939949748714, 12.548410588
243982, 41.09899865593189, 43.352118311254706, 7.123067879542039, 29.21519339739253, 18.002612443480213
, 66.66666666666667, 87.16025814080774, 9.978494617533556, 30.574534367457165, 32.99016568964311, 91.07
660981491192, 79.84764533757195, 32.83254365849988, 18.279211729857725, 26.433214475069445, 66.66666666
666667, 70.80247094284702, 81.3973213339941, 64.34913815904902, 13.188888696978132, 0.0, 34.82803038685
861, 57.00774777765847, 53.581170914037465, 44.14433581744475, 32.506258167472176, 77.37832439471092, 5
1.387656977500775, 30.248389933408315, 18.659121942866268, 45.25201749531826, 56.16290595431307, 28.160
7415442316, 62.52984778405062, 37.3196916975872, 56.21245377179633, 38.80560545917276, 17.3338986042728
3, 18.314924805823125, 47.10712902820856, 73.62154724611419, 29.64634806776897, 71.52913718876151, 100.
0, 57.12082669747139, 43.26255230075469, 0.0, 30.205163254081466, 56.19579802678536, 28.720723998217892
, 21.65801445652779, 36.72114087214337, 24.230924495960938, 82.42063830682174, 32.73076250337986, 49.62
478889973781, 28.454207091065808, 42.84535054532525, 83.81694094308922, 63.0784550292528, 43.9481946663
1631, 78.21359186511266, 81.40385635985727, 72.50940881931967, 8.829209409883532, 36.45201030326667, 11
.717549982359637, 80.84217910722657, 45.455767388109756, 72.03871173736256, 49.34400689393343, 81.11149
581963132, 5.263331950061153, 55.13356580449889, 45.464962300503025, 25.501030945923606, 23.44606851787
2067, 58.19716620194904, 29.65662122218986, 8.499137412935406, 37.36680081257088, 27.330983479323432, 5
.0731893248281255, 49.53721223617074, 15.607896145290686, 48.0582695762075, 56.8795239506735, 26.703648
520220554, 6.027834077785485, 8.016153016129072, 0.0, 76.89614868975563, 32.48197593517273, 64.92295548
664124, 20.286172866783925, 18.901781342825817, 76.02819145682399, 29.452550789003407, 53.4794332542459
54, 36.943425310424004, 36.432706479524526, 76.51248217788515, 100.0, 22.319667325926616, 7.36493095757
57895, 51.678778939906245, 16.454754504271104, 0.0, 76.98017216272027, 39.49712596767643, 42.9476065613
3906, 37.31541053709108, 91.54319080220728, 73.42436336644526, 33.50464186010092, 37.762488596806094, 4
4.75189005212594, 46.657123522078685, 18.341846483635425, 38.30097004913947, 23.112398174918933, 73.102
70845758765, 23.617951182043537, 35.40564972606079, 28.908792047872552, 0.0, 22.62447575981591, 14.1331
30363791437, 32.62726509923021, 34.86071207346709, 54.71238289078192, 33.91967206942842, 69.05310057243
364, 45.336933251746224, 46.38683046730623, 92.53078256703422, 71.88232718224471, 21.318504107059468, 7
8.78416919888528, 24.11760822135916, 64.66696573695349, 56.93105342144333, 51.9329983222815, 41.8962333
7012794, 24.018309667642026, 48.2002678904525, 38.042245113268855, 47.654430413649294, 55.4927422467407
9, 45.215931935465676, 25.62849435001902, 47.48312696830225, 50.432549498477194, 29.998899964895788, 75
.38352406446494, 72.36683886100356, 25.141401592602744, 19.526214587563505, 71.39829564192956, 23.73827
57734524, 60.39082245735033, 33.729358264695406, 13.573320338631751, 37.942617045894345, 15.00516851140
197, 89.43255620492191, 69.9590060589047, 42.56422544411551, 18.8192827609126, 26.656736623862972, 27.9
60100754012796, 45.010342528914606, 78.68789191632055, 27.221307507956908, 61.21530192207499, 49.285461
53831377, 9.473812715457106, 14.212535231675206, 56.71919037403143, 69.79879178538049, 55.6337741247697
5, 36.13368063702275, 66.66666666666667, 85.30310616574509, 80.195514211329, 0.0, 14.63432883502484, 8.
907410902172899, 14.864132079730856, 37.575902361689344, 72.98966171511368, 76.50784931662861, 74.09963
415307529, 7.293207019773348, 82.38051428403398, 48.68868051512349, 62.608738068205895, 14.259507282140
424, 11.4344434816422, 42.24848848798681, 46.50469655948731, 32.97092054089424, 94.8615877213643, 31.99
165442186033, 18.550303397712423, 25.00714270528457, 27.304073111991784, 6.4538893733267555, 25.8874231
8844464, 24.546017155590164, 37.70580557566965, 48.22466525544664, 43.242761149289656, 0.0, 66.66666666
666667, 26.337944532366777, 18.785562877815494, 35.527747671805635, 27.97446804654575, 8.00332561935129
6, 20.293324458544323, 23.277133096099448, 35.275863690820806, 22.273360158054345, 68.97294506902763, 2
0.99685680284009, 87.24484474621217, 63.86604471766879, 38.17936458025758, 54.584598919173764, 37.51701
1242882624, 10.625925880673307, 35.81769869444084, 89.20759095618119, 12.590689163839329, 7.73469426515
922, 64.18024317526732, 65.17549303225582, 27.026429499909298, 11.71681104239309, 17.947214519651425, 5
6.628195655563424, 4.806781837194175, 48.6294267647809, 38.19914509709509, 63.198779969495526, 45.02010
890308684, 30.03211956668807, 80.89591838635245, 8.016153016129072, 42.85507182188804, 78.5926076250335
, 28.19748781105635, 68.98370023092247, 21.912380366228984, 100.0, 22.128200720186545, 29.7309936259833
02, 35.17861985960845, 11.585786505148036, 62.714727890875004, 10.009119534046723, 45.518277062914045,
75.38333897123118, 49.284424588090154, 82.43105333373457, 87.61117096298993, 42.900120433097555, 0.0, 6
.212132910367667, 88.85974206558281, 89.08718755899584, 35.641654499619385, 21.748266671291276, 10.7044
86108444428, 22.067255721050625, 66.23154162703167, 31.317075929922108, 18.309328275508285, 34.21299876
931571, 26.859514130085802, 12.25280243285244, 13.061743620507926, 52.3234134747269, 59.192041993110486
, 11.942336863033193, 39.453708574467036, 64.75623672779382, 57.04618272811848, 22.119885445537836, 49.
30331473932353, 65.98405351671111, 33.05501593010367, 46.02536073053605, 0.0, 64.43936197219367, 19.352
9386599063, 43.7327770272947, 53.16327202431285, 25.988139621395884, 13.314060916465598, 23.15886640869
7634, 48.721199777718255, 59.57555134191574, 12.62649529135416, 29.306413915424766, 54.74647165742408,
17.765432657100984, 47.64890459309162, 75.19635771365286, 53.0069390989222, 70.27638981516776, 26.86123
0915595666, 20.797232268373772, 49.02489287016672, 37.614741101532296, 48.053166898703346, 23.653817012
851736, 17.4201997894388, 0.0, 23.331167175676388, 25.856784473577562, 11.895513744318615, 22.586781803
40119, 27.145512805932544, 50.90844239598738, 63.936009217878336, 55.1975088054665, 66.66666666666667,
22.267586321227345, 0.0, 14.780039053166377, 67.6187215215345, 66.66666666666667, 32.137726715261515, 5
9.43797429964602, 81.83265453146095, 62.914107824515575, 38.035352150290684, 29.237526260208337, 43.822
419582141485, 41.706666754764285, 50.928394470210094, 27.86514116764712, 17.416565604181343, 41.1605023
53225716, 100.0, 25.225738690601485, 43.076048579076065, 52.61234588715171, 22.658804185376482, 50.7871
2323410909, 0.0, 40.51064800652885, 56.74783199349443, 57.91991977506936, 63.553700159121824, 28.909759
376964047, 16.85376036720604, 31.817197239168994, 10.592194951608272, 11.5611979117916, 31.794636933560
77, 40.436783623144166, 20.166005670095956, 29.17601878132268, 11.913003353550996, 17.793988825311015,
22.86708287210493, 25.091759001602522, 57.08430065075961, 47.269462932998636, 33.275210473860085, 23.14
217179751792, 37.39532383613966, 61.4379310863816, 43.14004925384909, 15.129834586868215, 25.7270937749
8366, 14.812591846305205, 10.411093899336239, 34.57267374622315, 25.494654501049922, 31.664207965306957
, 13.03844236680061, 42.35359020422804, 13.03844236680061, 44.21904028678091, 0.0, 75.23813585386138, 1
8.835212136762497, 16.046032649525078, 19.61993349410076, 40.765756451313315, 88.55062193963464, 41.525
14760592381, 19.256664018630385, 59.83385454924301, 87.38506287469376, 46.14961837448087, 83.2035983824
6789, 17.95037272330592, 54.306161175084064, 22.382033220449156, 15.688966408005493, 6.544731821434517,
50.03506371493457, 51.981509641961736, 16.597688012166884, 40.36765914321378, 48.62753265477153, 60.61
898146556948, 8.499137412935406, 54.86871913472237, 40.25435354192816, 11.708299882072533, 42.376736847
9928, 83.2275079115662, 0.0, 18.30342862817614, 17.4201997894388, 62.2382238138955, 48.999147454350755,
59.54065181415597, 43.86544775478101, 35.17892401108887, 33.98817879543383, 76.22908954173263, 90.3560
7916100616, 24.759118376449077, 7.051629720143278, 32.91258250269417, 19.495092786255864, 47.1097527655
5064, 9.905149383092857, 24.287602955201418, 53.10465530154895, 50.73260745031458, 36.40929221978146, 7
1.89671931009268, 52.735870372712846, 29.867561757091, 38.82570478410775, 52.23068758931277, 41.6032468
6025552, 55.629262750212625, 46.69909352308442, 82.27386187052532, 0.0, 66.12981426982425, 79.659435991
47363, 66.66666666666667, 43.66166525832923, 68.32995955388681, 48.128973234379025, 18.596103422559125,
47.289935127603066, 31.467782232245256, 24.121658087335586, 23.024539392217264, 36.66706857940543, 44.
60140100269302, 63.12531462247904, 15.756128695913233, 13.547063653779865, 27.072926426289815, 0.0, 14.
1048804468153, 59.909679569755035, 31.097552869816397, 18.835212136762497, 18.305304146378376, 27.67881
178052257, 58.260283355287584, 55.386557495717504, 25.507627557624563, 22.797532901583583, 82.528322977
48606, 37.23924270697351, 26.251026622841824, 44.9500700297285, 39.162017049808135, 45.32366888492381,
64.17424625348978, 60.96917847667631, 65.71351301800723, 50.5330855011649, 24.550184031710916, 41.09113
9202363486, 8.959796316388614, 45.07420779578978, 41.50017183819377, 25.959413554801912, 28.36594391076
6408, 34.99643465201296, 16.352611666774106, 44.36283574847403, 18.44935056931412, 47.101339997363986,
32.20422788176115, 40.95516071370323, 28.44852308016479, 18.69512754313304, 36.114615749132774, 56.0590
5645744196, 22.946782681184583, 26.792208985485217, 37.77117192047056, 41.71535527784183, 87.8810910153
1033, 0.0, 22.063510474005152, 21.039607067517945, 14.830617273585224, 63.93718652561473, 41.3142553714
13346, 18.22915079880609, 6.96795128316442, 59.28204210718635, 3.1811034536498064, 39.06401064769555, 0
.0, 16.747063422440345, 29.88780795569292, 28.41142275681405, 66.66666666666667, 32.74345063787114, 45.
438777698815336, 87.81322394536959, 14.71803212875536, 73.52925039045647, 35.13548424434442, 34.5591589
09793815, 100.0, 67.79708878091981, 7.3649309575757895, 23.217802271573408, 71.90403483294725, 48.67165
825652836, 80.90372651444214, 67.12495573287788, 30.501340621918363, 49.1364860117406, 45.3131194557602
5, 18.691231552647697, 40.49950521133402, 68.65340305669272, 10.757301205288067, 54.398798378023265, 28
.18448728551746, 33.70266529220455, 30.91492494917284, 66.60035402869434, 64.87095700504716, 73.3672026
6984848, 18.745207379583963, 33.878921836675296, 36.892202495894246, 34.785057551548796, 0.0, 42.804422
14122135, 6.304171567278753, 0.0, 100.0, 24.51077596612623, 60.45162685630049, 43.15161576436149, 0.0,
42.03912763502127, 35.394754404269456, 35.92806378490184, 48.10070578350695, 44.85792871294567, 33.3410
79606478296, 66.66666666666667, 38.67228330581132, 45.88471965612789, 13.081499462390667, 9.79402465521
283, 21.590479037200552, 63.034777976631936, 27.477921162487785, 55.55627278630467, 22.061003123966525,
13.894294714511181, 65.44528528726008, 20.293324458544323, 42.2240758209661, 62.915543422836286, 44.51
80179626739, 22.25597999742385, 69.06241837414971, 6.86203282687362, 45.6495285976519, 13.0384423668006
1, 40.39343097041727, 52.39431513249533, 92.5765116371415, 22.78722276295349, 26.210161863091184, 77.97
890454657461, 33.5194298347058, 61.7484571335298, 29.396679243554445, 4.456600172035526, 41.81867491402
085, 35.51088042674098, 45.87456798269614, 21.17943011854199, 29.785141195563302, 48.4112903584839, 34.
27330090360887, 17.325138492427307, 44.364482628188966, 44.638268049819295, 26.35973862701729, 44.29401
013056425, 49.66341246218597, 32.89143371286113, 45.87311996118358, 80.83928504100997, 75.0828271109498
6, 8.198263760492152, 48.568140716841846, 37.05518931710741, 8.256261445270306, 66.66666666666667, 48.4
2650312148253, 8.626003008364762, 19.84895098556828, 50.41054944491379, 24.174720415705092, 14.94796210
7238183, 36.318362066073206, 18.64796322464427, 74.30600083785465, 37.35953396487447, 0.0, 66.589058438
8199, 8.208912497156007, 40.93114393728407, 45.591943204556905, 35.54171760422738, 27.399659946438536,
67.49096915565507, 38.70413822788323, 52.95674895162839, 74.91739031103407, 43.660558200712046, 23.3249
9424279075, 64.49707035301203, 12.171900394735887, 69.8863664959776, 31.02753710862439, 45.519452765344
255, 8.492357788041796, 85.88665564559138, 38.23817256439289, 37.95776034236564, 51.781634836749724, 41
.106273214390804, 57.13823706511939, 35.662353896217326, 47.20915311159692, 27.05632415287894, 27.12511
7748697992, 45.11834676209971, 53.95367887099658, 48.57336031793853, 43.14403211667551, 17.111350518626
367, 31.564280636971358, 51.17878366020617, 69.63908663996366, 7.3078966910364995, 1.6268547568817127,
16.098740979095343, 48.252740466866555, 11.631426658079503, 43.84380540825588, 41.82955580253, 100.0, 8
.755281887451053, 18.057209789641263, 24.470147211404054, 18.101214486678806, 60.4672209955822, 30.1421
73856264616, 57.22862163823074, 20.672165661782877, 44.822139086562146, 0.0, 11.4446882475633, 47.93676
4225952004, 53.83750574157841, 68.6958764329078, 49.69126743994753, 13.993573296990133, 51.524434140398
284, 45.72986441424646, 58.501926263633386, 47.639685888970554, 82.09050113627431, 21.086231993224768,
52.3200189452729, 71.7822382634989, 33.57567041285073, 23.339086123972688, 18.22915079880609, 25.903393
091742796, 36.061022997276844, 9.736721841813534, 39.09186738006747, 6.4538893733267555, 0.0, 37.831488
600515975, 12.020276617371408, 44.6649709356843, 94.09631699967578, 57.872164891116455, 51.295179949860
71, 36.9729808626414, 49.72387932025138, 49.78076737594972, 51.81215995819986, 47.242320350829374, 86.5
5472642466704, 77.6633045804641, 0.0, 51.17432938990935, 86.73006732873067, 100.0, 41.62366695905041, 7
0.31136127734291, 71.1057188355525, 51.01754084386213, 23.586329040360948, 32.77302272448689, 45.947952
58197258, 40.58055519583012, 81.52108112019748, 29.352715535642627, 71.59460861257382, 25.7433554630075
14, 52.26709178414248, 87.84097660551681, 69.07331262316406, 58.007144285525186, 7.52867223525477, 68.5
0570547183757, 26.0424846910883, 38.074980390040814, 25.81156252242536, 87.34528566209475, 52.426070196
394164, 18.21551698632435, 24.629003253754448, 76.53297294804223, 51.68159001347603, 51.650289974058694
, 14.618741457570573, 48.39518018747015, 42.25999419387821, 35.523953373423566, 35.23551289134429, 94.2
3085965225039, 63.35181391246732, 45.099728757643675, 70.08091841267867, 51.01733163979449, 31.69137498
0046476, 0.0, 12.228943605098836, 19.661973737571074, 35.9444539692708, 65.96590314952545, 13.618443071
839039, 64.97017815171567, 40.310175592974986, 38.43997795784943, 92.35523742441605, 46.25516369566814,
32.96551453661747, 30.35041080696892, 72.52263761481088, 13.763940228222785, 72.8588654512431, 16.9510
0578352255, 2.3012615001388403, 76.85420691504665, 0.0, 53.86454589360494, 0.0, 33.67871993865609, 68.6
6547609937378, 74.1224099318995, 56.506565637408045, 84.0674640022147, 0.0, 32.8474283001641, 25.056812
879400344, 34.307357568049554, 21.74453589893095, 38.67006659825595, 27.25777421936313, 72.541634249515
51, 51.58746309889685, 42.884402362680646, 50.94478139576943, 45.16022153455225, 76.81332889593882, 24.
02059826151653, 64.59963549568444, 56.06767811596097, 26.98540506061923, 40.469002006470625, 48.5592567
89652835, 13.621294212270167, 7.057415901368728, 45.95587993977674, 11.873437986096471, 75.061548087827
09, 26.099746754305304, 30.96721870975465, 56.73371162198672, 30.69352321237695, 17.931358082132572, 10
0.0, 28.73952291391497, 36.53190146691456, 22.72664602433353, 31.055265067519343, 52.98544389479979, 32
.82286929684272, 80.17988141662131, 24.527538900233015, 28.002858821928495, 20.160840017096305, 76.8870
9866089133, 65.89452319453426, 41.59067004834467, 33.246211523121644, 48.49652661957638, 66.66666666666
667, 90.18007227317797, 50.186411549197, 42.772237837254615, 73.92824847183478, 44.98090162756864]
mean discrepancy: 39.82948695047773
bleus: 20.70490720882358
(base) ➜ exp2018 OA
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(77)<module>()
-> b = bleu(tokens, ref_tokens)
(Pdb++) tokens
['the', 'dose', 'rate', 'of', 'the', '``', 'Fugen', "''", 'can', 'be', 'calculated', 'in', 'detail', 'u
sing', 'the', 'DERS', 'software', '.']
(Pdb++) ref_tokens
['details', 'of', 'dose', 'rate', 'of', '``', 'Fugen', 'Power', 'Plant', "''", 'can', 'be', 'calculated
', 'by', 'using', 'DERs', 'software', '.']
(Pdb++) n
[0] > /home/acb11190vf/research/exp2018/stnmt/random_diversity_bleu.py(78)<module>()
-> bleus.append(b)
(Pdb++) b
24.202875575621302
(Pdb++)
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 78, in <module>
print("mean discrepancy:", np.mean(diversities))
File "stnmt/random_diversity_bleu.py", line 78, in <module>
print("mean discrepancy:", np.mean(diversities))
File "/apps/python/3.6.5/lib/python3.6/bdb.py", line 51, in trace_dispatch
return self.dispatch_line(frame)
File "/apps/python/3.6.5/lib/python3.6/bdb.py", line 70, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 39.82948695047773
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 39.82948695047773
26.67
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
mean discrepancy: 49.075146062140355
24.4
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(49.0864346504914, 24.78),
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(48.83865152348552, 24.35),
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(49.31695409010472, 24.51),
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 69, in <module>
b = bleu(sents[i], sents[j])
TypeError: 'float' object is not callable
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(48.56395505580279, 24.53),
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 69, in <module>
b = bleu(sents[i], sents[j])
TypeError: 'float' object is not callable
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(49.22429777472472, 24.74),
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 76, in <module>
tmp_hyp.write("{}\n".format(" ".join(tokens)))
ValueError: I/O operation on closed file.
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(49.155775706686654, 24.44),
(48.967119832296184, 24.73),
(50.08041891456354, 24.56),
(49.07116175604126, 24.59),
(49.48010269238074, 24.57),
(49.67245988638564, 24.48),
(49.62673321407424, 24.55),
(49.13196426157736, 24.47),
(48.72484227561455, 24.53),
(49.446639601775075, 24.47),
(48.93647990450581, 24.79),
(49.334280425685556, 24.8),
(48.925940513760885, 24.57),
(49.17684987942691, 24.47),
(49.46412869198591, 24.61),
(48.7910448721107, 24.51),
(49.57375698159605, 24.57),
(49.21899274959137, 24.58),
(49.598129252740094, 24.62),
(49.32419022232728, 24.41),
(49.332493872529625, 24.65),
(49.6525261199818, 24.44),
(49.4061529634203, 24.4),
(50.10873377429954, 24.58),
(49.03901007589842, 24.51),
(48.590313989934, 24.65),
(49.334127701141625, 24.59),
(49.38200634317079, 24.49),
(49.60164575664628, 24.48),
(49.23058167095367, 24.41),
(49.53966911670997, 24.42),
(50.49538197517862, 24.48),
(49.83758826236762, 24.4),
(48.6335138018526, 24.53),
(49.27930921136892, 24.36),
(49.32141285715291, 24.75),
(49.20844090665602, 24.43),
(49.431407133363535, 24.58),
(49.91478695868962, 24.54),
(49.68880214788015, 24.33),
(49.13849106840822, 24.58),
(48.81504422713444, 24.6),
(49.529630389362445, 24.54),
(49.29429623115902, 24.44),
(49.403607592696844, 24.55),
(49.35567348705227, 24.48),
(49.566637179285486, 24.42),
(49.06282990527093, 24.49),
(48.8485874294606, 24.5),
(49.40702230121276, 24.49),
(49.85168759874317, 24.3),
(49.341405446465, 24.4),
(49.139504730108264, 24.5),
(49.44850855182465, 24.5),
(49.88992212967368, 24.27),
(48.401213354343334, 24.49),
(48.86210787522772, 24.49),
(49.46165713177809, 24.31),
(49.30938645183942, 24.45),
(48.554117586543114, 24.51),
(49.632720151809735, 24.47),
(49.00028869924906, 24.53),
(49.71170020815506, 24.56),
(49.73791957481751, 24.36),
(49.70685713224026, 24.48),
(49.883334899937324, 24.47),
(49.3628622306444, 24.45),
(49.76074696676044, 24.43),
(49.65988468022382, 24.4),
(49.993850992023226, 24.64),
(48.65174371991224, 24.54),
(50.080444331745, 24.59),
(49.12105568748344, 24.65),
(49.035184619308744, 24.66),
(50.26278514401089, 24.45),
(49.248414063791536, 24.5),
(49.2282522186295, 24.57),
(49.707104222900014, 24.58),
(48.82991145501041, 24.46),
(49.363014049548624, 24.44),
(49.752969207108755, 24.45),
(49.03348977920197, 24.5),
(49.329651529504275, 24.55),
(48.77975572828501, 24.49),
(49.241289685287974, 24.34),
(48.97777955243215, 24.75),
(49.38122648534951, 24.39),
(49.34437902930087, 24.43),
(48.99549466166097, 24.58),
(49.09267001215751, 24.58),
(49.254240769006245, 24.59),
(49.60318734256479, 24.55),
(49.34777839900686, 24.34),
(49.13728374424563, 24.36),
(49.07587717460841, 24.65),
(49.31239550197933, 24.52),
(49.68533762882751, 24.49),
(49.34648647284444, 24.63),
(49.557479921910186, 24.37),
(49.55298237294098, 24.38),
(base) ➜ exp2018 OA
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 61, in <module>
sents = [sents[0], sents[a], sents[b]]
TypeError: list indices must be integers or slices, not numpy.float64
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 61, in <module>
sents = [sents[0], sents[a], sents[b]]
TypeError: list indices must be integers or slices, not numpy.float64
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(39.82948695047773, 26.67),
(39.921732443549814, 26.55),
(41.21212463838402, 26.44),
(41.37620997269633, 26.44),
(42.34397391596121, 26.21),
(43.395860447105115, 26.22),
(43.44964239922709, 26.15),
(44.10371382290033, 26.29),
(43.924044011114006, 26.15),
(44.64349800827693, 26.06),
(44.26331433475569, 26.13),
(44.82253194575076, 26.1),
(44.24759993562442, 26.12),
(44.51156355825938, 26.05),
(44.484043399267875, 26.11),
(44.98534205587561, 26.19),
(44.79791827638301, 26.1),
(45.06015791536229, 25.93),
(44.727791606512675, 26.25),
(45.10796871648121, 26.09),
(45.089071891894996, 26.03),
(45.1505385398148, 26.05),
(45.665255120837855, 25.99),
Traceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 61, in <module>
sents = [sents[0], sents[a], sents[b]]
IndexError: list index out of range
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
^CTraceback (most recent call last):
File "stnmt/random_diversity_bleu.py", line 85, in <module>
bleus = moses.evaluate("/tmp/tmp_hyp.txt")
File "/home/acb11190vf/research/nmtlab/nmtlab/evaluation/moses_bleu.py", line 23, in evaluate
pipe = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE)
File "/apps/python/3.6.5/lib/python3.6/subprocess.py", line 405, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/apps/python/3.6.5/lib/python3.6/subprocess.py", line 830, in communicate
stdout = self.stdout.read()
KeyboardInterrupt
(base) ➜ exp2018
(base) ➜ exp2018
(base) ➜ exp2018 python stnmt/random_diversity_bleu.py
(39.82948695047773, 26.67),
(39.921732443549814, 26.55),
(41.21212463838402, 26.44),
(41.37620997269633, 26.44),
(42.34397391596121, 26.21),
(43.395860447105115, 26.22),
(43.44964239922709, 26.15),
(44.10371382290033, 26.29),
(43.924044011114006, 26.15),
(44.64349800827693, 26.06),
(44.26331433475569, 26.13),
(44.82253194575076, 26.1),
(44.24759993562442, 26.12),
(44.51156355825938, 26.05),
(44.484043399267875, 26.11),
(44.98534205587561, 26.19),
(44.79791827638301, 26.1),
(45.06015791536229, 25.93),
(44.727791606512675, 26.25),
(45.10796871648121, 26.09),
(45.089071891894996, 26.03),
(45.1505385398148, 26.05),
(45.665255120837855, 25.99),
(45.61345505485704, 26.17),
(45.64791292280663, 25.91),
(46.01376494842505, 25.94),
(45.89611910478895, 25.89),
(47.26933932232297, 25.5),
(40.5510106236001, 26.45),
(41.72051940894774, 26.33),
(42.07470373569235, 26.34),
(42.97979118500063, 26.1),
(43.89045877870353, 26.11),
(44.21878247613402, 26.05),
(44.785556201141354, 26.18),
(44.53081324641689, 26.04),
(44.97477936833917, 25.95),
(44.86333840933811, 26.02),
(45.18232327618556, 25.99),
(44.506922866981895, 26.01),
(44.963226025863776, 25.94),
(44.714638843226304, 26.0),
(45.04484277839373, 26.08),
(45.36514201657611, 25.99),
(45.39631885356783, 25.82),
(45.17440567507878, 26.14),
(45.844458908322, 25.98),
(45.49144333098743, 25.92),
(45.23514886681576, 25.94),
(45.84001295970317, 25.88),
(45.949644226021725, 26.06),
(46.033178660260184, 25.81),
(46.53956471744138, 25.83),
(46.024164262170714, 25.79),
(47.28049854994468, 25.17),
(41.713892797617056, 26.21),
(42.03008294886439, 26.22),
(42.70792874387425, 25.99),
(43.71807689647193, 25.99),
(44.0937339245558, 25.93),
(44.28192597766818, 26.06),
(44.322108694696915, 25.92),
(44.693603515685474, 25.84),
(44.72906278051798, 25.9),
(44.978678580514284, 25.87),
(44.267542079660075, 25.9),
(44.45473971253462, 25.82),
(44.64179382379081, 25.88),
(45.046866411539284, 25.96),
(45.47587342143655, 25.88),
(45.063637576727245, 25.7),
(45.04256622899529, 26.03),
(45.554257221039876, 25.86),
(45.18232785556369, 25.8),
(45.48205318665328, 25.83),
(45.74479090245982, 25.76),
(45.894408110371344, 25.94),
(45.83633084129893, 25.68),
(46.36607357101619, 25.7),
(46.12980209972818, 25.64),
(47.018795504564835, 25.24),
(43.35879170913937, 26.1),
(44.02386411152032, 25.87),
(44.91579484405735, 25.87),
(45.27271254433308, 25.81),
(45.60065228477903, 25.94),
(45.380494239867524, 25.8),
(45.86583597904067, 25.72),
(45.759410963541136, 25.78),
(45.97912530077361, 25.75),
(45.48699206435369, 25.78),
(45.788981504423596, 25.7),
(45.42080811192582, 25.76),
(46.238325741134446, 25.84),
(46.25964932306098, 25.76),
(46.3239762304704, 25.58),
(46.0079550454279, 25.91),
(46.470376886303455, 25.74),
(46.140590028405924, 25.68),
(46.29420201911013, 25.71),
(46.62985527270531, 25.64),
(46.80394952833814, 25.82),
(47.07023354876688, 25.57),
(47.31106130756358, 25.6),
(46.9954265272301, 25.55),
(47.95964502972581, 25.14),
(44.38529119509883, 25.87),
(45.248843426521944, 25.88),
(45.218757245427476, 25.82),
(45.69824454211143, 25.95),
(45.841390524545304, 25.81),
(46.08984309458897, 25.72),
(46.04384342942819, 25.79),
(46.21412155017961, 25.76),
(45.65717546784199, 25.79),
(45.89087247837839, 25.71),
(45.81333158662034, 25.77),
(46.305418481938474, 25.85),
(46.431422779088685, 25.76),
(46.226939954414426, 25.59),
(46.30958125995159, 25.91),
(46.63272744835842, 25.75),
(46.45055496357906, 25.69),
(46.379632419732964, 25.71),
(47.016484610055464, 25.65),
(46.83891657144586, 25.8),
(47.41595162199397, 25.58),
(47.28319754176408, 25.6),
(47.66153255333818, 25.55),
(47.702372859434384, 25.28),
(46.04648053787576, 25.65),
(46.01366570775044, 25.58),
(46.17127663566873, 25.72),
(46.407355141858694, 25.58),
(46.69317524316849, 25.49),
(46.746730211775564, 25.55),
(46.49748836893874, 25.52),
(46.12653158905748, 25.55),
(46.33982315914305, 25.47),
(46.38006716060809, 25.53),
(46.90931067270337, 25.62),
(46.88733569053203, 25.53),
(46.92149446261251, 25.36),
(46.88042957715454, 25.68),
(47.12164429048729, 25.5),
(46.670194318857114, 25.46),
(46.84068144759319, 25.48),
(47.65116966660585, 25.41),
(47.80045752014465, 25.53),
(47.44658401059468, 25.34),
(47.929816651861394, 25.37),
(47.84606773141164, 25.32),
(47.91548349249342, 25.1),
(46.946303161735955, 25.59),
(47.01290739975307, 25.72),
(47.16177514591198, 25.58),
(47.44864892426391, 25.49),
(47.34023972131358, 25.56),
(47.44659910273985, 25.53),
(46.93015559013736, 25.56),
(47.05012503274191, 25.48),
(47.24887524281893, 25.54),
(47.57199677573059, 25.62),
(47.4924383703557, 25.53),
(47.60655981037535, 25.36),
(47.33422179993507, 25.67),
(48.019364637156954, 25.48),
(47.41166852185693, 25.46),
(47.43598452666619, 25.48),
(47.91299161862627, 25.4),
(48.11123798164197, 25.51),
(48.33859260293006, 25.35),
(48.48977029195067, 25.36),
(48.37484301518974, 25.34),
(48.882620099695664, 25.13),
(47.41393554487326, 25.66),
(47.419293997889156, 25.52),
(47.5673029838246, 25.43),
(47.474692235614775, 25.49),
(47.543263707961614, 25.46),
(47.060880455639484, 25.49),
(47.384578272528344, 25.42),
(47.24526994127559, 25.47),
(47.69724250507507, 25.56),
(47.889812467830374, 25.47),
(47.88672124319456, 25.3),
(47.499696180024515, 25.62),
(48.258500194998064, 25.45),
(47.722048354380696, 25.4),
(47.989847891802086, 25.42),
(48.48002272256597, 25.35),
(48.365085873167835, 25.49),
(48.54997693821752, 25.28),
(48.52239400840497, 25.31),
(48.762599406341614, 25.26),
(48.739766094557304, 25.0),
(47.77881363202272, 25.65),
(47.920722255026405, 25.57),
(48.08981690474227, 25.63),
(48.16599596974153, 25.6),
(47.41346674063219, 25.63),
(47.86004336734354, 25.55),
(47.52282155100325, 25.61),
(47.947800236172554, 25.68),
(48.14433906975392, 25.61),
(48.35474948992431, 25.43),
(47.69369201998648, 25.73),
(48.2248176233622, 25.55),
(48.144514998271255, 25.53),
(48.289675565751516, 25.55),
(48.488434528346964, 25.47),
(48.72921235935046, 25.57),
(48.993731438660284, 25.4),
(48.904020885899165, 25.44),
(49.151836371973, 25.4),
(48.197149985230894, 24.98),
(47.87199680829972, 25.43),
(47.63318460932617, 25.49),
(48.035946582014795, 25.46),
(47.6446825950261, 25.49),
(47.59924305005353, 25.41),
(47.32094645906453, 25.47),
(48.135068375259216, 25.55),
(48.20844987704578, 25.47),
(48.06069053906139, 25.29),
(47.88636411274257, 25.62),
(48.42596371702188, 25.44),
(48.103136178113644, 25.39),
(48.45537333090263, 25.41),
(48.21844269022704, 25.35),
(48.631906796517086, 25.47),
(48.90496760557858, 25.27),
(48.93829370570112, 25.29),
(48.59947365598767, 25.25),
(48.70795448288847, 25.08),
(48.25513060036799, 25.4),
(48.30608136561367, 25.37),
(47.714924430183345, 25.4),
(48.00686375852038, 25.32),
(48.00073325322408, 25.38),
(48.373169334678686, 25.47),
(48.56193011262106, 25.38),
(48.54582283020954, 25.21),
(48.24610821207154, 25.53),
(48.59486163948076, 25.36),
(48.53029130196271, 25.31),
(48.651093248659514, 25.33),
(48.753305850519496, 25.26),
(48.77335361899825, 25.41),
(49.19243119876065, 25.19),
(49.00898858281365, 25.22),
(48.97389837307253, 25.17),
(48.738663998531685, 25.04),
(48.346365868874166, 25.43),
(47.67917883326784, 25.46),
(47.74861750136596, 25.39),
(47.834760272760505, 25.44),
(48.415541628178524, 25.52),
(48.35915619382758, 25.44),
(48.423142363533174, 25.27),
(48.235729102572954, 25.58),
(48.57942126782429, 25.39),
(48.225732277731474, 25.37),
(48.6106827643482, 25.39),
(48.60115779370445, 25.31),
(48.74056026516165, 25.42),
(48.873756714729325, 25.25),
(49.16629526792378, 25.27),
(49.11263223631315, 25.25),
(48.48110118692675, 25.07),
(48.12917329676173, 25.43),
(48.1588547081682, 25.36),
(47.89102619033708, 25.41),
(48.6978262138075, 25.49),
(48.351858183463705, 25.41),
(48.23335738467108, 25.24),
(48.31809902044862, 25.54),
(48.733190186028885, 25.35),
(48.38432601111908, 25.34),
(48.819990354602304, 25.36),
(48.81925196561958, 25.28),
(48.76490565280748, 25.38),
(49.00425087705617, 25.21),
(49.57492256244915, 25.24),
(49.06855323344808, 25.2),
(49.75767828557574, 25.0),
(47.48756686048487, 25.38),
(47.50902753939235, 25.44),
(47.98613651551253, 25.53),
(47.83264871119998, 25.44),
(47.916978836210326, 25.27),
(47.792199873873415, 25.59),
(48.10240154384617, 25.42),
(47.90458284482842, 25.37),
(48.039358870170275, 25.39),
(48.10950986141739, 25.32),
(48.35442393874981, 25.45),
(48.36414588655999, 25.27),
(48.86838570862126, 25.28),
(48.65368061650863, 25.24),
(49.092890761741764, 24.95),
(47.54735134392904, 25.37),
(48.25947566479374, 25.43),
(48.35129818493405, 25.36),
(48.3477150687926, 25.18),
(47.95172350709921, 25.48),
(48.325966223365256, 25.29),
(48.2135055076112, 25.29),
(48.30554399068186, 25.3),
(48.549996330732924, 25.22),
(48.77785909475007, 25.32),
(48.72532396521632, 25.15),
(49.149634143716504, 25.18),
(48.75904321348554, 25.17),
(48.78563055189263, 24.75),
(48.142731667065696, 25.51),
(48.16623610967642, 25.42),
(48.16546207838846, 25.25),
(48.054131307500455, 25.57),
(48.21793331333163, 25.39),
(47.93772753805273, 25.35),
(48.169978354310445, 25.37),
(48.452462947799894, 25.3),
(48.62757078419234, 25.42),
(48.74461872679772, 25.23),
(48.98668814510855, 25.25),
(48.999279414329244, 25.2),
(48.89764376704078, 25.1),
(48.57329097813819, 25.49),
(48.488231425758855, 25.31),
(48.37729018619061, 25.62),
(48.87229534257601, 25.43),
(48.59292980500525, 25.43),
(48.753145461685584, 25.43),
(48.76332189868683, 25.35),
(48.82982690856337, 25.45),
(49.30997177971494, 25.28),
(49.3763799823273, 25.3),
(49.298689305151946, 25.3),
(48.75821405811614, 25.03),
(48.55387401474798, 25.24),
(48.55317520044192, 25.55),
(49.09507042967008, 25.36),
(48.56939620901658, 25.35),
(48.42544022059589, 25.36),
(48.9565368691966, 25.28),
(48.771017907291075, 25.38),
(49.114520282974524, 25.22),
(49.461754145611565, 25.22),
(49.12400186826361, 25.2),
(48.71341467554925, 24.93),
(48.27775939706738, 25.36),
(48.88131425429222, 25.17),
(48.59789074774376, 25.17),
(48.79913402176898, 25.18),
(48.8774491395454, 25.1),
(49.20794334412338, 25.2),
(49.187977792507034, 25.04),
(49.37100249481284, 25.05),
(49.332951643844005, 25.04),
(49.83466853888148, 24.72),
(48.73142768351878, 25.48),
(48.445917356753355, 25.49),
(48.34080230696471, 25.48),
(48.608913037770755, 25.4),
(48.911313711233376, 25.51),
(48.95904001306515, 25.34),
(49.09228090786886, 25.36),
(49.06651208037253, 25.36),
(48.79220870606323, 24.98),
(48.86429825850008, 25.3),
(48.66215287587151, 25.29),
(49.185188941897984, 25.21),
(49.15148055028493, 25.32),
(49.454369320173065, 25.15),
(49.705013438415584, 25.18),
(49.47859815202709, 25.17),
(48.74752689193748, 24.96),
(48.47032745358896, 25.3),
(48.96883340641039, 25.23),
(48.81926323894478, 25.33),
(49.19098863240666, 25.16),
(49.381550653689835, 25.19),
(49.13697956661195, 25.14),
(48.61054846855778, 24.97),
(49.12432894604305, 25.22),
(49.12162394733839, 25.33),
(49.18276227964962, 25.16),
(49.318892812449995, 25.17),
(49.143315003865375, 25.16),
(48.90434785637446, 24.78),
(49.15007119710492, 25.24),
(49.37961043611076, 25.09),
(49.44367725610728, 25.11),
(49.359127213998036, 25.1),
(49.63289839054514, 25.0),
(49.57243527550616, 25.19),
(49.47621903418782, 25.2),
(49.634028253018165, 25.19),
(49.11855162785054, 24.99),
(49.830113087473954, 25.05),
(49.887574042826586, 25.03),
(49.25291201541812, 24.87),
(50.24071417274744, 25.05),
(49.119136227384686, 24.89),
(49.67307390595991, 24.81),
(base) ➜ exp2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment