Custom Modules - IMAP eMail Mangement (node) (Build you own webmail)

Wappler IMAP extensions

Overview

IMAP is considered an email standard but in reality there can be a lot of variaion between how IMAP servers deal with folders.

This is currently Node only however it is hoped (can't promise) to bring a PHP version online soon.
Consider these docs beta also, i will tidy up over next few days

I hope to release some support videos soon.
Please note, schemas may change during this initial phase where additional infomation needed is added.
These will be appended to the existing returned object to minimise possibility of breaking code.

To understand IMAP you must understand a few basic concepts:

Email UID's

Each email has a unique UID which is separate from it's ID which is simply an index number representing the position in the mailbox.
UIDs are unique ONLY within the mailbox in which they reside, the UID may be duplicated in other mailboxes. For this reason it is necessary to address messages, not only by their UID but also the mailbox reference.

This is further complicated by the fact that mailboxes can be both deleted and created. Hence if a mailbox called "INBOX/Mymail" exists it may have a message with UID = 1. However of this mailbox is deleted and then recreated, a new email into that box may be re-allocated the UID of 1.
In consequence, INBOX/Mymail contains a message UID = 1 but this is NOT the same as the previous email with UID = 1 prior to the mailbox deletion.

Thankfully each mailbox also has a UID referred to as the UIDValidity, a unique number representing the mailbox which would change if the mailbox was deleted and recreated.

Ironically there is no easy way to find the UIDValidity ID from the mailbox name (all mailiboxes would need to be parsed and checked which is slow and inefficient)

For this reason,calls to retrieve messages are passsed three parameters to ensure the message integrity, UID, UIDValidity and mailbox name.

Mailbox names

Maillbox names do not follow an enforced standard. While mailboxes such as INBOX and Drafts as normally consistant, mailboxes names such as Sent and Trash may vary from server to server.

As a result, the names of the "Send" and Trash (AKA Deleted Items) can be defined via environment variables. I can add flexibility for the others if needed.

Lastly, some servers use "/" as a mailbox separator like 'INBOX/folder', others use a dot "." like this 'INBOX.folder'. Again this is configurable via ENV variables.

Installing

The npm can be found here:

Install via project manager.

Setup

Global settings can be set via your environment variables as below:

Components Overview

Individual Components

Each component will take it's settings from the $_ENV variables set within Server Connect HOWEVER these settings can be overridden at component level by selecting "Enable ENV Overrides" from the IMAP Settings selector dropdown


then entering the settings into the SC component.

Error management

In all cases the following convensions are followed:

Success returns status 200
Authorisation errors return status 401
All other errors return status 400 (other than enforced 500 errors)

Email management

IMAP Get Headers

This module collects headers from your IMAP server from the specified folder
There is a limit and offset option however it is for you to manage the paging, it is not (currently) incorporated in the module.

Performance is generally very good(sub second) and paging is probably only necessary with large header sets (1000s+) or where preferable in the app UI otherwise you can download segments by setting the offset to 0 and using the Count Emails module to return the number of emails in the mailbox

Output Schema

IMAP Get Email Content

This module returns the content (to,from,cc,bcc,Subject, body and attachments) of an email identifed by the specified by the UID parameters sent (typically taken from the output of the headers output).

Output Schema

IMAP Count emails

This module returns the number of emails in a specified mailbox

Schema

image

Example

    "mail1": {
        "mailbox": "INBOX",
        "count": 2,
        "status": 200
    }

IMAP Delete message by UID

This module deletes a message based on the UID parameters sent

Schema:

image

Example Output

    "mail": {
        "uid": "2",
        "mailbox": "INBOX",
        "destination": "Trash",
        "status": 200
    }

IMAP Get Attachments by UID

This module detaches and attachments from a specified message and places them in a specified folder on the server for download etc

The action returns an array of attachment names, a field names meesage which contains a comma separated list of all the attachments, the URL path to the attachments, the path of the attachments and a status (200 for success)

Schema:

image

Example:

    "imapga": {
        "savedFiles": [
            "featured-img-domains-1050x600.jpg",
            "haggis.jpg"
        ],
        "message": "featured-img-domains-1050x600.jpg, haggis.jpg",
        "url": "assets/attachments/testytest/",
        "path": "/public/assets/attachments/testytest/",
        "status": 200
    }

IMAP Send Email

This module performs an email send cased on input parameters to,from,cc,bcc,Subject, body and attachments. Attachments format as compatible with the output of the Wappler File Upload component.

usage:

Schema:

image

Example Output:

    "mail1": {
        "status": 200,
        "messageId": "<7ef3648d-0681-a227-b1b1-64d801f56106@hyperbytes.co.uk>"
    }

IMAP Save as Draft

Availability:
Node:Still under development

This module takes the input parameters (to,from,cc,bcc,Subject, body and attachments) and saves that as a Draft. If not subscribed to the Drafts folder then the user is auto subscribed.
Optionally a UID can be passed whereupon the message with that UID will be deleted before a new one created. This is designed to avoid duplicates in the event of re-saving a draft

Usage:

Schema:
image

Example Output:

    "imap": {
        "status": 200,
        "message": "Draft updated successfully.",
        "uid": 48,
        "folder": "Drafts"

Mailbox Management

IMAP List Mailboxes

This module lists ALL mailboxes on the server, subscribed or Unsubscribed
Usage:
This has no extra inputs

Schema:
image

Sample Output:

        "mailboxes": [
            "Trash",
            "Junk",
            "Drafts",
            "Sent",
            "INBOX",
            "INBOX/myMail"
        ]
    }

IMAP List Subscribed Mailboxes

This module lists all mailboxes on the server to which the user is subscribed
The use and output is identical in format to List Mailboxes module.

IMAP List UnSubscribed Mailboxes

This module lists all mailboxes on the server to which the user is NOT subscribed
The use and output is identical in format to List Mailboxes module.

IMAP Subscribe to Folder

This module performs subscription to a specifed folder
Usage:

Schema:
image

Sample Output:

imap: {
status: 200,
message: "Successfully subscribed to folder 'INBOX'"
}

IMAP Create Folder

This module create an IMAP sub folder under the current folder i.e. INBOX/New_folder
Simply send the folder name to be created under "INBOX" i.e "newsubfolder".
I hope to add nested folders in a fuTure release

Usage:

Schema

Sample output:

    "mail1": {
        "status": 200,
        "message": "Folder \"INBOX/another\" created successfully."
    }

IMAP Delete Folder

This module deleted a folder. This should only be applied to empty folders, check with IMAP Count Emails first!
Also i advise you check it exists (with reference to lsy folders) or an error will be generated.

This module operates in the same way as create folder although, of course, the folder is deleted, not created.

IMAP Move Folder

This module moves a message form its current folder to a specified folder
ThIs module moves a message to a new folder

Usage:

Schema:
image

Sample Output:

 "mail1": {
        "uid": "1",
        "mailbox": "INBOX",
        "destination": "INBOX/myMail",
       "uidvalididity":"2345234564",
        "status": 200
    }

The uid returned is the uid of the destination folder

Get uidValidity of folder

Usage:

Schema
image

Sample output:

  • imap:{
    • mailbox: "INBOX",

    • uidvalidity: 1747210497,

    • status: 200}

IMAP Empty Trash

This module empties the trash i.e deleted all massages marked /deleted

Flags

IMAP Flag by UID

IMAP uses "flags" internally to mark messges with attributes such as "Seen', "Recent" etc
This module adds a specified flag (i.e. /Deleted, /Flagged /Seen etc) to the message specified by the UID parameters sent.

Usage:

Schema:
image

Sample Output:
"mail1": {
"uid": "1",
"mailbox": "INBOX",
"flag": "\Seen",
"status": 200
}

IMAP Remove Flag by UID

This module removes a specified flag (i.e. /deleted/ /flagged /seen etc) to the message specified by the UID parameters sent.
If is functionally identical to Flag ny UID except the flag is removed, not added

Diagnostics

IMAP Trash Folder Permissions

IMAP Get raw headers

These are temporarily available as a diagnostic tools for the beta phase

8 Likes

This is so cool!

The only point I'm concerned and at the same time amazed is the amount of .js and .hjson files exist for a single extension.

I'm wondering if Wappler could support "depth-1" loading of a subfolder so a folder e.g. "imap" could contain all of those .js and .hjson files instead of having them all in "modules" root folder

Just wondering about the benefit in doing that?

All the hjson and .js files could be contained within a single file but it makes debugging much more difficult so i tend not to use that approach. Each component has it's own .js and .hjson file
There are a load of .PHP files which are nothing more than placeholders presently which can be deleted.

There is only around 150K of files so not that much.in reality.

2 Likes

Better organization of code, clear distinction of files between different extensions, no filename collisions between different extensions, ability to collapse the folder in the tree view in Visual Studio Code.

This is the first time organization of files is being debated, because previous extensions only consisted in one .js and .hjson. Hyperbytes came with a clever way to simplify extension development by having each method in a separate .js file, so now Wappler should revise extension loading to support subfolder loading

If someone wants to create a feature request feel free, or I'll create when I get more time

Ok, it's for proper housekeeping; I thought there might have been a reason that I was overlooking.

In node, doesnt happen, when installed from npm, each extension has it's own distinct folder within the node_modules structure

2 Likes

V1.0.2 - New component, "Get Mailbox uidValidity" added, npm and docs updated

1 Like

Actually @Apple , looking at how Wappler deals with extensions in the PHP environment, I agree this is a valid concern. No where near as clean and compartmentalised as node.