Error Analysis in Online/Offline NMT
IWSLT De->EN
Offline systems
Samples from the test set formatted as:id)
src: Source sequence.
ref: Reference target sequence.
Difficulty: AL score + the alignement between tokenized src-ref
PA (beam): Hypothesis with Pervasive Attention using beam search.
TF (beam): Hypothesis with Transformer using beam search.
PA (greedy): Hypothesis with Pervasive Attention greeedily decoded.
TF (greedy): Hypothesis with Transformer greedily decoded.
Removing BPE || Keeping the BPE tokenization
Online systems
Samples from the test set formatted as:id)
src: Source sequence.
ref: Reference target sequence.
Difficulty: AL score.
PA (offline): Hypothesis with Pervasive Attention greeedily decoded.
TF (offline): Hypothesis with Transformer greedily decoded.
PA (online): Hypothesis with Pervasive Attention trained above k=1. Decoding along wait-3.
TF (online): Hypothesis with Transformer trained on k=7. Decoding along wait-3.
Removing BPE || Keeping the BPE tokenization