ca10c185d94470054d693f19e7691523dbe3ec55,skll/data/readers.py,NDJReader,_sub_read,#NDJReader#Any#,434

Before Change


            If IDs cannot be converted to floats, and ``ids_to_floats``
            is ``True``.
        
        for example_num, line in enumerate(f):
            // Remove extraneous whitespace
            line = line.strip()

            // If this is a comment line or a blank line, move on
            if line.startswith("//") or not line:
                continue

            // Process good lines
            example = json.loads(line)
            // Convert all IDs to strings initially,
            // for consistency with csv and megam formats.
            curr_id = str(example.get("id",
                                      "EXAMPLE_{}".format(example_num)))
            class_name = (safe_float(example["y"],
                                     replace_dict=self.class_map)
                          if "y" in example else None)
            example = example["x"]

            if self.ids_to_floats:
                try:
                    curr_id = float(curr_id)
                except ValueError:
                    raise ValueError(("You set ids_to_floats to true, but" +
                                      " ID {} could not be converted to " +
                                      "float").format(curr_id))

            yield curr_id, class_name, example


class MegaMReader(Reader):

    
    Reader to create a ``FeatureSet`` instance from  a MegaM -fvals file.

After Change


        features : list of dicts
            The features for the features set.
        
        with open(f, "r" if PY3 else "rb") as buff:
            lines = [json.loads(line.strip()) for line in buff
                     if line.strip() and not line.startswith("//")]
Italian Trulli
In pattern: SUPERPATTERN

Frequency: 3

Non-data size: 7

Instances


Project Name: EducationalTestingService/skll
Commit Name: ca10c185d94470054d693f19e7691523dbe3ec55
Time: 2019-03-05
Author: jbiggs@ets.org
File Name: skll/data/readers.py
Class Name: NDJReader
Method Name: _sub_read


Project Name: commonsense/conceptnet5
Commit Name: db4b9f4d50af9e83733fc301c32e85cbe6d9c812
Time: 2014-02-25
Author: rob@luminoso.com
File Name: conceptnet5/builders/json_to_csv.py
Class Name:
Method Name: convert_to_tab_separated


Project Name: commonsense/conceptnet5
Commit Name: db4b9f4d50af9e83733fc301c32e85cbe6d9c812
Time: 2014-02-25
Author: rob@luminoso.com
File Name: conceptnet5/builders/json_to_solr.py
Class Name:
Method Name: convert_to_solr