How to import CSV from remote API?

[Apple]

Edit: Currently using the array fields directly, so low priority issue

Hi,

I’m using API Action to download a CSV file from a remote API. It’s downloaded as a text string, how do I use the Import CSV step with a CSV that isn’t present on the file-system? Or how do I save the file to filesystem so I can then use Import CSV? I think Wappler doesn’t support this yet?

Thanks for any advice! This is NodeJS.

Edit: Previously I’ve attempted to split the string at \n to get an array of lines, and then at “,” to get an array of the individual fields. Is there a way I can create an object with the fields inside? I’ve attempted to use “Set Value”, but I couldn’t get to the point of building the object (typing a JSON object in the data expression threw a syntax error or something), see screenshot below. I’ve ended up deleting those steps mistakenly

The goal of creating the object is just so I have some sort of type-safety, instead of going around with array[3] around the script

(forgot to .split(",") to get the individual fields, don’t mind that)

[jellederijke]

Hi @Apple,

I have a project where I download a csv from some url. This csv content needs to be matched with the database content and modify changed data / add new data. I’m not sure if this is similar to your situation.

If it is I can suggest a workaround, because I couldn’t figure out how to do this with the Wappler UI. What I have done is I created a Python script using Requests to get the csv file (or connect to your API) and I use Pandas to extract only the information I need from the CSV.

Then using Requests again I send the data over to my Wappler api endpoint that does the queries on the database.

Does this make sense to you? I’d be happy to share some code if this direction feels like something you want to go for.

[Apple]

@jellederijke

have a project where I download a csv from some url. This csv content needs to be matched with the database content and modify changed data / add new data. I’m not sure if this is similar to your situation.

My use case is very similar to yours!

@jellederijke

If it is I can suggest a workaround, because I couldn’t figure out how to do this with the Wappler UI. What I have done is I created a Python script using Requests to get the csv file (or connect to your API) and I use Pandas to extract only the information I need from the CSV.

I was hoping to avoid creating a third-party script, but that would ultimately solve the problem. Good workaround!

For now, this is how I’ve managed to get this working within Wappler:

I can then access each CSV field by fields[0], fields[1], etc…

I’m sure your Python script offers better assurances in regards to type safety, as my current solution is more prone to mistakes handling data. For now, I think I’ll let it be this way, but thank you very much for the offer of code sharing!

[jellederijke]

Glad to hear that you have found your solution! If you ever feel the need to try out some scripting please do pet me know.

[iwian]

How do you access the fields after the last Set value, please? When you want to use each csv column for a specific column in Db insert. I can’t figure it out…I need to parse csv and work with all the rows and columns from it - to filter, db insert, etc.

[Apple]

The fields won’t appear in the Data Binding Picker, you can access them by their index number:

name,age,country

name = fields[0]
age = fields[1]
country = fields[2]

Use the “Code” section in the Data Binding Picker to manually access the fields.

You can also do Set Value name = fields[0] so you have that in the data binding picker

Edit: Also, you’re not splitting correctly on Set Value lines: you have \\n instead of \n