Wappler escaping form inputs on save?

Wappler General Bugs
1

We are not sure if this is a 'bug' or a 'feature' but to us this is a bug...

Wappler is escaping the following text input, this is the form input on the page, standard form input:

image
image816×64 6.68 KB

[{"id":"introductionWidget","x":0,"y":0,"w":12,"h":2},{"id":"basic-blog-post-widget","x":0,"y":2,"w":6,"h":6},{"id":"cookie-policy-widget","x":6,"y":2,"w":6,"h":3},{"id":"blankWidgetExample","x":6,"y":5,"w":6,"h":3},{"id":"pageIntroductionWidget","x":0,"y":8,"w":12,"h":2}]

When saved to the database with an Insert Action it appears as (within the database and we have tried storing as json and longtext and other datatypes, varchar etc, still the same result?):

image

"[{\"id\":\"introductionWidget\",\"x\":0,\"y\":0,\"w\":12,\"h\":2},{\"id\":\"basic-blog-post-widget\",\"x\":0,\"y\":2,\"w\":6,\"h\":6},{\"id\":\"cookie-policy-widget\",\"x\":6,\"y\":2,\"w\":6,\"h\":3},{\"id\":\"blankWidgetExample\",\"x\":6,\"y\":5,\"w\":6,\"h\":3},{\"id\":\"pageIntroductionWidget\",\"x\":0,\"y\":8,\"w\":12,\"h\":2}]"

I'm confused as to why this is happening? I've worked around it but the input is escaped during save to the database there is nothing special happening here, just a straight save of an input, as a post input, Server Action pointed at the form for its inputs, imported to Server Connect all as normal, as soon as it is saved the above occurs?

2

Wappler 7.0.0 Beta 14

Ubuntu 24.04.1 LTS

Node JS

3

Now you say that, yesterday found that a custom query was returning that..
But doing a select directly on mysql throwed the right thing..

4

This is saving the data like this when there are no \ in any of our inputs, so this is as it is stored so thus would return the query as Wappler stored/saved it to the database, ie with the \ escaping...

Which is not ideal (nor what should be occurring) as say we want to store code snippets, or even anything in quotes for example, they will be escaped upon save. I'm sure we had no issues with this previously but I need to go back to where we built a source code repository which is on a drive I don't currently have access to..

5

Maybe check your page charset to be utf-8 as this is how the form data is submitted and the server side expects that for the right enconding.

1 Like
6

I'll check that now @George. I did not think of that. Thank you.

7

image
image930×118 11.2 KB

All looks fine and in the database too. Was worth a shot!

8

Appears Wappler is having issues with interpreting JSON data for saving and retrieval to and from the database..?

9

Have changed this to a Bug Report as requires investigation.

10

Ah now I see you mean the json encoding! For that you have to setup the right database type.

See this guide:

1 Like
11

Why such a complicated set of steps to do such a simple thing? Really needs improvement as storing and retrieving JSON shouldn't require such interventions surely? However I do appreciate the guidance but do feel this needs some real thought and attention and above all significant improvement as obviously you guys knew there was an issue hence the guide...

Thank you regardless though @George being Boxing Day and all. We should all be AFK! Well according to my better half hahaha...

12

@George in my case, I'm making a custom query (actually provided by ChatGPT), where it generates a JSON with

SELECT 
    u.nombre_users AS jugador, 
    JSON_ARRAYAGG(
        JSON_OBJECT(
            'partido', ap.text_apipartidos,
            'equipo_local', f.local_formulario,
            'equipo_visitante', f.visitante_formulario,
            'local_apiresultados', ar.local_apiresultados,
            'visitante_apiresultados', ar.visitante_apiresultados,
            'puntos', p.puntos_puntos

Where I retrieve something like this directly on a mysql query (heidisql for example):

[{"partido": "partido_1", "equipo_local": 1, "equipo_visitante": 2, "local_apiresultados": "1", "visitante_apiresultados": "2", "puntos": 9},{"partido": "partido_2", "equipo_local": 1, "equipo_visitante": 1, "local_apiresultados": "0", "visitante_apiresultados": "1", "puntos": 1}

But on custom query output I receive:

[{\"partido\": \"partido_1\", \"equipo_local\": 1, \"equipo_visitante\": 2, \"local_apiresultados\": \"1\", \"visitante_apiresultados\": \"2\", \"puntos\": 9},{\"partido\": \"partido_2\", \"equipo_local\": 1, \"equipo_visitante\": 1, \"local_apiresultados\": \"0\", \"visitante_apiresultados\": \"1\", \"puntos\": 1}

Could be related somehow?

13

Did not work the data was still saved in the same escaped way outlined above. Remember this is a text input not a checkbox group. We just want the text input to save the way it is represented and not to be escaped. There is an underlying issue here.

14

Actually the data is fully stored as json if the field is defined like this and all encoding and decoding is done automatically. So it is not that complex at all

15

Have just carried out everything per the guide and no change in the way the data was saved to the database. To be sure I'm not going mad I just run a manual test and inserted the value through the console to my database and stored correctly with no malformed value.

Please try it for yourself George.

16

Here is an example input for you:

<input id="usersLayout" name="usersLayout" class="form-control mt-3 mb-3 form-control-sm" required="" readonly="true" is="dmx-input" value="[{"id":"introductionWidget","x":0,"y":0,"w":12,"h":2},{"id":"basic-blog-post-widget","x":0,"y":2,"w":6,"h":6},{"id":"cookie-policy-widget","x":6,"y":2,"w":6,"h":3},{"id":"blankWidgetExample","x":6,"y":5,"w":6,"h":3},{"id":"pageIntroductionWidget","x":0,"y":8,"w":12,"h":2}]">

Column type in the database (MySQL) is JSON.

17

By using just the value attribute you are giving it a static string. If you want to pass a an expression then use dmx-bind:value

18

But also server side the value should be json decoded and this is only done if the db field is specified tobbe of type json. You can also specify it in the insert database action

19

The above is an example @George we do use an expression but we have to format it to remove the 'junk', the value in turn is passed along to replace a value in a keyed array in local storage and has to be in valid JSON. Hence the formatting required after the Wappler mutation of the value.

Example:
We save the input to the database initially (as the example input above). Wappler mangles that value. We then retrieve the value in to a hidden text input. This value is then used. But before we can use it and inject it as the value for the local storage array we have to format it as what Wappler sends back is next to useless (and I hate to use that term) without us then formatting it first...

<input id="value" name="value" class="form-control" readonly="true" dmx-bind:value="get_layout_values.data.getLayout.layout_data.replace('&quot;[', '[', 'false').replace(']&quot;', ']', 'false').replace('\\', '', false)" placeholder="Layout Array Local Storage Value">

Only then can we insert it in to the local storage as a value! Otherwise poo hits the Dev Console... And it all goes Pete Tong...

The value does not have to be JSON. We just want our input stored as it is intended, not interpreted. Even if we store it as LONGTEXT or VARCHAR Wappler still escapes it, regardless of its Datatype in the database. We don't care if it is in LONGTEXT, VARCHAR or JSON. We just want the value inserted and retrieved as it was saved not in any other way that Wappler feels it should be interpreted as. This is a flaw and a bug. The input is a text input what is in the text input should be inserted as it is definied in the text input especially if the Datatype for the columns is VARCHAR or LONGTEXT...

20

When I refer to it as a complex work-around I am saying that we have Client's with databases full of JSON Datatype columns, thousands of them! So we would have to do this modification, outlined in the guide, for each and every one of them? :joy:

Surely not?