Infinite scroll component

[mebeingken]

I could use some feedback (especially from the javascript guru’s who can point out improvements!) on an infinite scroll component I’ve been working on that (hopefully) makes the process of building an infinite scroll easier than manually building.

The infinite scroll component attaches to a server connect, and triggers data loading based on the scrolling of the page, or a scrollable container. Displaying the content is done in the same manner as before (ie. setup a repeating row and use a server connect recordset for its expression.)

Screen Recording 2023-03-15 at 12.48.53 PM

npm at wappler_infinite_scroll

Step by step

Step by step:
Here is how you install (standard Wappler instructions):

You should see in the output:

Create a vanilla server API that has a paged query:

To your front end page, add a vanilla server connect using that API:

Add the Infinite Scroll component:

Set the Scrolling dataset on the properties by selecting the recordset from your paged query:

Create a repeat children row as you normally would, using your server connect for the expression (do not use the infinite scroll component here, use the server connect.)

Controls

• Scrolling dataset must be a server connect paged query

• Scrolling container is optional. The default is to use the entire page as the scrollable area. You may provide an element id of a scrollable container. For example, this container <div class="container" id="scrolling_container" style="height: 300px; overflow: scroll;"> will have a scrollable area within the page.

• Debounce inserts a delay in how often the data is refreshed

• Pixels before end allows you to set the scroll point at which time a refresh is triggered. A greater number indicates it should refresh earlier in the scroll.

• The Infinite Scroll component has a data value of “executing” which can be used to display whatever busy indicator you wish to use.

[JonL]

Nice one Ken!
You are missing the link to npm or repo.

Edit: sorry I thought wappler_extensions was a placeholder or something. It’s the actual node package name.

[mebeingken]

Sorry, that was in the screenshot. Will add now.

npm:

wappler_infinite_scroll

[JonL]

Are you planning to include all your extensions in the same package? If not you may want to figure out a more unique name. Named scopes might interest you also, although there is a bug with that at the moment.

[mebeingken]

That was my plan, but I definitely was going back and forth on this. I’m up for following some community guidelines or advice though!

[JonL]

I would keep them separated because although each component has their files they all share the same hjson. One error there and published by mistake and all extensions would fail to work in the UI.

[Teodor]

Nice I was just playing with a component with the same functionality this weekend but using IntersectionObserver instead of pageYOffset: https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API

[mebeingken]

I recall seeing IntersectionObserver in trying to pull this together. Maybe that’s a better solution. I’ll have a look.

[mebeingken]

@JonL

I would keep them separated

You are probably right…I’ll make that change.

[tbvgl]

Nice one. Something like that @mebeingken

1. Remove the scroll event listeners for both window and container.
2. Add a new function handleIntersection to handle the intersection observer’s callback.
3. Modify the setupComponent function to create an Intersection Observer using the specified scrolling_container if provided, otherwise using the document’s root.
4. Set up the observer to watch the last element in the scrolling container and trigger the loading of more data when it comes into view.

dmx.Component('mebeingken-infinite-scroll', {

    // ...

    render: function (node) {
        setupComponent(this, node);
    },

    // ...

});

// Add a function to handle the intersection observer
function handleIntersection(entries, observer, comp, nodeId, contentPath) {
    entries.forEach(entry => {
        if (entry.isIntersecting && !comp.data.executing) {
            comp.set('executing', true);
            processScroll(entry, nodeId, contentPath);
        }
    });
}

function setupComponent(comp, node) {
    let nodeId = node.id;

    // is this element on a content page
    let contentPath = '';
    if (comp.$node.parentNode.isComponent) {
        contentPath = 'content.';
    }

    // Create the intersection observer
    let options = {
        root: comp.props.scrolling_container ? document.getElementById(comp.props.scrolling_container.replaceAll("'", "")) : null,
        rootMargin: `0px 0px ${comp.props.px_before_end}px 0px`,
        threshold: 0.1
    };

    let observer = new IntersectionObserver((entries) => handleIntersection(entries, observer, comp, nodeId, contentPath), options);

    // Add the observer to the last element of the scrolling container
    let scrollingContainer = options.root || document.documentElement;
    let lastElement = scrollingContainer.lastElementChild;
    if (lastElement) {
        observer.observe(lastElement);
    }
}

// Remove the window.addEventListener and container.addEventListener from the previous code

[mebeingken]

So cool. I love learning from those that know more!

I’ll have a look on this tomorrow. I also have to clean up some hacky variable stuff.

Thanks Tobias!

[TMR]

@mebeingken I just started using this today and had to let you know that’s an excellent bit of work you did there. Thanks for taking the time to do it - I’m sure there will be a lot of people very happy with this.

I had it up and running literally in a couple of minutes - what a great time saver when compared to creating it myself. Only thing I couldn’t figure out was how to get a spinner (tested in a button) to show during execution, I couldn’t see the ‘state’ selector in the data picker.

Only improvement I could see would be the provision to have a ‘load more’ button/link so a user could manually load more records if required rather than auto-load it.

[mebeingken]

Hey Tom, glad it is helping you.

Looks like I forgot to add some output on the scroll, but for now you can manually type it in:

<i class="fas fa-circle-notch fa-spin fa-3x"  dmx-show="infinite_scroll_1.executing"></i>

@TMR

Only improvement I could see would be the provision to have a ‘load more’ button/link so a user could manually load more records if required rather than auto-load it.

Ya, I should probably add that.

[TMR]

Thanks for that - It works perfectly, makes it look even more polished.

On a side note, I tested it out on a DB table with 20,000+ rows and it worked flawlessly, very small footprint too, fast loading and no memory concerns at all.

It’s very slick.

[TMR]

@mebeingken I just implemented this in a coupe of other places, both of them only had three or four records (but were due to increase over time). I found that once the page had scrolled down to the bottom, the dataset started repeating itself over and over again showing the same records over and over again.

I have the relevant page offsets, limit and sort dir set in the query manager, I’ve also tried removing them to see if this makes any difference (it didnt) - have you experienced anything like this? I cant seem to see anything that would cause this.

[mebeingken]

Thanks for letting me know. I’ll have a look.

For reference, are you on the stable channel? Current version?

[TMR]

Stable channel current version…

[umeshhodwala]

I am also getting same problem where same data is repeated

[mebeingken]

@TMR

the dataset started repeating itself over and over again showing the same records over and over again.

Just published 1.0.6 to fix this issue

[TMR]

I just reinstalled infinite scroll then restarted Wappler and I’m still seeing the same issue, I installed it using the instructions above, is there anything else I need to do?