ca10c185d94470054d693f19e7691523dbe3ec55,skll/data/readers.py,NDJReader,_sub_read,#NDJReader#Any#,434
Before Change
If IDs cannot be converted to floats, and ``ids_to_floats``
is ``True``.
for example_num, line in enumerate(f):
// Remove extraneous whitespace
line = line.strip()
// If this is a comment line or a blank line, move on
if line.startswith("//") or not line:
continue
// Process good lines
example = json.loads(line)
// Convert all IDs to strings initially,
// for consistency with csv and megam formats.
curr_id = str(example.get("id",
"EXAMPLE_{}".format(example_num)))
class_name = (safe_float(example["y"],
replace_dict=self.class_map)
if "y" in example else None)
example = example["x"]
if self.ids_to_floats:
try:
curr_id = float(curr_id)
except ValueError:
raise ValueError(("You set ids_to_floats to true, but" +
" ID {} could not be converted to " +
"float").format(curr_id))
yield curr_id, class_name, example
class MegaMReader(Reader):
Reader to create a ``FeatureSet`` instance from a MegaM -fvals file.
After Change
features : list of dicts
The features for the features set.
with open(f, "r" if PY3 else "rb") as buff:
lines = [json.loads(line.strip()) for line in buff
if line.strip() and not line.startswith("//")]
In pattern: SUPERPATTERN
Frequency: 3
Non-data size: 7
Instances Project Name: EducationalTestingService/skll
Commit Name: ca10c185d94470054d693f19e7691523dbe3ec55
Time: 2019-03-05
Author: jbiggs@ets.org
File Name: skll/data/readers.py
Class Name: NDJReader
Method Name: _sub_read
Project Name: commonsense/conceptnet5
Commit Name: db4b9f4d50af9e83733fc301c32e85cbe6d9c812
Time: 2014-02-25
Author: rob@luminoso.com
File Name: conceptnet5/builders/json_to_csv.py
Class Name:
Method Name: convert_to_tab_separated
Project Name: commonsense/conceptnet5
Commit Name: db4b9f4d50af9e83733fc301c32e85cbe6d9c812
Time: 2014-02-25
Author: rob@luminoso.com
File Name: conceptnet5/builders/json_to_solr.py
Class Name:
Method Name: convert_to_solr