Acute accented letters in CSV file importing as question marks

Wappler General
1

Hi,

I have a strange problem where we have some French names with acute accents which are inserting into the DB as ?.

Example names

Renée
Andrée
Désirée
Aimée

The DB gets populated with the values retrieved from the import CSV step and they show like this;

Ren?e
Andr?e
Désir?e
Aim?e

I though it was a DB issue, and possible still is, however, I can edit the name via our Wappler built backend and its updates the name correctly into the DB.

So it only seems to be when we are importing the names which have been read from the import CSV step. I am using ASP.NET and I am running Version: 7.3.8

If anyone has any ideas or has come across the same issue, I would be grateful for any help.

Thanks
Ray

2

Are you stipulating UTF-8 coding when reading the CSV?

Not sure, but I believe that ASP.NET has something like this

var reader = new StreamReader(filePath, Encoding.UTF8);
3

Hi Ben,

Thanks for your reply.

I am using the Wappler Import CSV.

image
image866×577 10.7 KB

I am then using a repeat step to insert the records into the MS SQL DB.

Where should be setting the encoding to UTF8?

Regards,
Ray

4

Great question.

Normally, using NodeJS, I would use Wappler's file system to read the file and write (save) the file. Wappler uses NodeJS under the hood and writes files as UTF-8 by default - without BOM (Byte Order Mark)

Unfortunately, this will not work for your ASP.NET project.

I am now going to ask Copilot to help me out - I am no ASP.NET expert - to save the file with UTF-8

:puzzle_piece: Final Setup: Wappler ASP.NET Module to Resave CSV as UTF-8 Without BOM

:file_folder: 1. Create the Module File

Create a new file in dmxConnect/api/modules/ called resave_csv_utf8.cshtml.

:receipt: 2. Paste This Code

@{
    Response.ContentType = "application/json";

    try {
        // Get query parameters
        var inputParam = Request.QueryString["input"];
        var outputParam = Request.QueryString["output"];

        if (string.IsNullOrEmpty(inputParam) || string.IsNullOrEmpty(outputParam)) {
            Response.Write("{\"status\":\"error\",\"message\":\"Missing input or output path.\"}");
            Response.End();
        }

        // Resolve full paths
        var inputPath = Server.MapPath("~/" + inputParam);
        var outputPath = Server.MapPath("~/" + outputParam);

        // Read original file
        string content = System.IO.File.ReadAllText(inputPath, System.Text.Encoding.Default);

        // Save as UTF-8 without BOM
        var utf8WithoutBom = new System.Text.UTF8Encoding(false);
        System.IO.File.WriteAllText(outputPath, content, utf8WithoutBom);

        // Success response
        Response.Write("{\"status\":\"success\",\"message\":\"File resaved as UTF-8 without BOM.\"}");
    } catch (Exception ex) {
        // Error response
        Response.Write("{\"status\":\"error\",\"message\":\"" + ex.Message.Replace("\"", "'") + "\"}");
    }

    Response.End();
}

:link: 3. Call It from Server Connect

  • Create a Server Action (e.g., resave_csv_action).
  • Add an API Action step.
  • Set the URL to:
    /api/modules/resave_csv_utf8.cshtml?input=uploads/original.csv&output=uploads/resaved_utf8.csv

You can make input and output dynamic by binding them to form inputs or query parameters.

:test_tube: 4. Test and Confirm

  • Run the Server Action.
  • Check the resaved_utf8.csv file in your uploads folder.
  • Open it in Wappler to confirm it’s UTF-8 without BOM.

:light_bulb:Alternative

Convert the site to NodeJS :ogre:

5

Please check how are your CSV files saved - ANSI / Windows-1252 encoding or UTF-8?

6

@Teodor Good point. I will validate that first.

If it’s as simple selecting the encoding when saving the CSV, I just need to ensure the client knows this beforehand.

I’ll reply soon with an update.

@ben Great suggestion, will give that a go also.

Thanks,
Ray

1 Like
7

Hi @Teodor

Thanks, saving the CSV file to UTF-8 worked.

@ben I think I will still look at your option as my client has very little computer knowledge and it would be better to have a module built to handle this.

Thanks for your help.

1 Like