9107c93764e8af783f08b4c8cb2252cd48dc8f32,onmt/IO.py,ONMTDataset,__init__,#ONMTDataset#Any#Any#Any#Any#Any#,97

Before Change


                self.src_vocabs.append(src_vocab)
                // mapping source tokens to indices in the dynamic dict
                src_map = torch.zeros(len(src)).long()
                for j, w in enumerate(src):
                    src_map[j] = src_vocab.stoi[w]

                self.src_vocabs.append(src_vocab)
                example["src_map"] = src_map

                if "tgt" in example:

After Change


                src_vocab = torchtext.vocab.Vocab(Counter(src))
                self.src_vocabs.append(src_vocab)
                // mapping source tokens to indices in the dynamic dict
                src_map = torch.LongTensor([src_vocab.stoi[w] for w in src])

                self.src_vocabs.append(src_vocab)
                example["src_map"] = src_map
Italian Trulli
In pattern: SUPERPATTERN

Frequency: 3

Non-data size: 3

Instances


Project Name: OpenNMT/OpenNMT-py
Commit Name: 9107c93764e8af783f08b4c8cb2252cd48dc8f32
Time: 2017-09-02
Author: bpeters@coli.uni-saarland.de
File Name: onmt/IO.py
Class Name: ONMTDataset
Method Name: __init__


Project Name: pytorch/fairseq
Commit Name: e73fddf45377e8a3c0ea2e8281fae18f7b498dd6
Time: 2018-03-05
Author: myleott@fb.com
File Name: tests/utils.py
Class Name: TestIncrementalDecoder
Method Name: forward