Problem importing tab-separated file

Wappler General
1

I’ve just some time tracking down the issue with an import failing, with the error:
array_combine(): Both parameters should have an equal number of elements

The file being imported is a tab-separated text file, itself generated from a database - so the number of fields/separators in each record is correct (I double-checked anyway). The file contains about 7000 records containgin about 25 fields, including some long text fields.

I tracked the problem down to a double quotation mark in one of records. The field concerned started:
"Esta edición presenta la
If I removed the leading double quote the import worked.
Also, if I moved the double quote so it wasn’t at the beginning of the field, eg:
Esta "edición presenta la
it worked.

There are a number of other records with double quotes like this but only this record caused a problem. I’ve had problems before importing text files (nothing to do with Wappler) where double quotation marks within a field were not paired, but I wouldn’t expect this to be a problem with tab-separated data. In any case, some of the other records contained unpaired double quotations marks.

I appreciate the feature in Wappler is called CSV import, but it works fine with tab-separated files if the correct delimiter is specifed. Could Wappler nevertheless be checking the integrity of a file, taking double quotation marks into account (unnecessarily)?

A couple of my customers have used this process without any problems during the last few months. It seems there is a very specific problem and I can’t identify it. However I wouldn’t have thought it should be an issue so perhaps there is a bug somewhere when it comes to tab-separated data.

2

This is still an issue. Importing the same tab-separated files using Navicat works without any problem - providing no text qualifier is specified - ie if the default quote mark is removed:

image

A text qualifier is not required for tab-separated text (which is one of its advantages). Would it be possible to make the import process only consider record delimiters to determine the file structure, and ignore " marks (for tab-separated text)?

3

We follow the standards for csv: https://tools.ietf.org/html/rfc4180. Will see what is possible.

1 Like
4

Thanks Patrick. If I’m importing/exporting csv files, I would follow these specifications - eg I would escape double-quotes.

However, it can be useful to a be able to specify a record delimiter and given Wappler allows you to specify a tab character, I suppose you might reasonably assume the import will be treated as standard tab-separated text (where double-quotes don’t need escaping). As it is, in some circumstances the import may fail. Double-quotes don’t necessarily cause it to fail, but in certain circumstances they do. I’m not quite sure what triggers a failure, but the import always succeeds if the quotes are removed.