22bce0332c94639df80820a439e4c1893a5552f8,mindsdb/libs/data_sources/file_ds.py,FileDS,_getDataIo,#FileDS#Any#,58
Before Change
try :
data.seek(0 )
full = len(data.read())
data.seek(0 )
bytes_to_read = int (full*0.3 )
dialect = csv.Sniffer().sniff(data.read(bytes_to_read))
data.seek(0 )
After Change
try :
data.seek(0 )
first_few_lines = []
i = 0
for line in data:
i += 1
first_few_lines.append(line)
if i > 500 :
break
dialect = csv.Sniffer().sniff("" .join(first_few_lines))
data.seek(0 )
if dialect:
In pattern: SUPERPATTERN
Frequency: 4
Non-data size: 8
Instances Project Name: mindsdb/mindsdb
Commit Name: 22bce0332c94639df80820a439e4c1893a5552f8
Time: 2019-02-01
Author: george@cerebralab.com
File Name: mindsdb/libs/data_sources/file_ds.py
Class Name: FileDS
Method Name: _getDataIo
Project Name: mindsdb/mindsdb
Commit Name: bafbfaa5718e1de72b805237d2c350aec11de9fe
Time: 2019-02-01
Author: george@cerebralab.com
File Name: mindsdb/libs/data_sources/file_ds.py
Class Name: FileDS
Method Name: _getDataIo
Project Name: HazyResearch/fonduer
Commit Name: 66c21553343edf0e76c3c728ac3fa10b1fb6720b
Time: 2018-12-20
Author: SenWu@users.noreply.github.com
File Name: src/fonduer/parser/preprocessors/csv_doc_preprocessor.py
Class Name: CSVDocPreprocessor
Method Name: __len__
Project Name: HazyResearch/fonduer
Commit Name: 66c21553343edf0e76c3c728ac3fa10b1fb6720b
Time: 2018-12-20
Author: SenWu@users.noreply.github.com
File Name: src/fonduer/parser/preprocessors/tsv_doc_preprocessor.py
Class Name: TSVDocPreprocessor
Method Name: __len__