22bce0332c94639df80820a439e4c1893a5552f8,mindsdb/libs/data_sources/file_ds.py,FileDS,_getDataIo,#FileDS#Any#,58

Before Change


        // lets try to figure out if its a csv
        try:
            data.seek(0)
            full = len(data.read())
            data.seek(0)
            bytes_to_read = int(full*0.3)
            dialect = csv.Sniffer().sniff(data.read(bytes_to_read))
            data.seek(0)

After Change


        try:
            data.seek(0)
            first_few_lines = []
            i = 0
            for line in data:
                i += 1
                first_few_lines.append(line)
                if i > 500:
                    break
            dialect = csv.Sniffer().sniff("".join(first_few_lines))
            data.seek(0)
            // if csv dialect identified then return csv
            if dialect:
Italian Trulli
In pattern: SUPERPATTERN

Frequency: 4

Non-data size: 8

Instances


Project Name: mindsdb/mindsdb
Commit Name: 22bce0332c94639df80820a439e4c1893a5552f8
Time: 2019-02-01
Author: george@cerebralab.com
File Name: mindsdb/libs/data_sources/file_ds.py
Class Name: FileDS
Method Name: _getDataIo


Project Name: mindsdb/mindsdb
Commit Name: bafbfaa5718e1de72b805237d2c350aec11de9fe
Time: 2019-02-01
Author: george@cerebralab.com
File Name: mindsdb/libs/data_sources/file_ds.py
Class Name: FileDS
Method Name: _getDataIo


Project Name: HazyResearch/fonduer
Commit Name: 66c21553343edf0e76c3c728ac3fa10b1fb6720b
Time: 2018-12-20
Author: SenWu@users.noreply.github.com
File Name: src/fonduer/parser/preprocessors/csv_doc_preprocessor.py
Class Name: CSVDocPreprocessor
Method Name: __len__


Project Name: HazyResearch/fonduer
Commit Name: 66c21553343edf0e76c3c728ac3fa10b1fb6720b
Time: 2018-12-20
Author: SenWu@users.noreply.github.com
File Name: src/fonduer/parser/preprocessors/tsv_doc_preprocessor.py
Class Name: TSVDocPreprocessor
Method Name: __len__