Improving Iscroll Performance On A Large Table

October 02, 2024 Post a Comment

I'm updating a table header and its first column positions programatically based on how the user scrolls around to keep them aligned. The issue I'm experiencing is that as soon as

Solution 1:

Just use ClusterizeJS! It can handle hundreds of thousands of rows and was built exactly for this purpose.

How does it work, you ask?

The main idea is not to pollute DOM with all used tags. Instead of that - it splits the list to clusters, then shows elements for current scroll position and adds extra rows to top and bottom of the list to emulate full height of table so that browser shows scrollbar as for full list

Solution 2:

To be able to handle big amounts of data you need data virtualization. It has some restrictions, though.

First you need to decide the size of a view port. Let's say you want to render 10 items in a row and 20 items in column. It would be 10x20 items then. In you fiddle it's div with id wrapper.

Then you need to know total amount of data you have. From your fiddle it would be 100x100 items. And, also you need to know height and width of a item (cell). Let's take 40x120 (in px).

So div#wrapper is a view port, it should have fixed sized like 10x20 items. Then you need to set up correct width and height for table. The height of table would be equal to total amount of data in column including head by item height. Width for table would be total amount of items in single row by item width.

Once you set up these, div#wrapper will receive horizontal and vertical scrolls. Now you able to scroll left and bottom, but it will be just empty space. However this empty space is able to hold exact amount of data you have.

Then you need to take scroll data left and top (position), which comes in pixels and normalize it to amount of items, so you could know not how many pixels you've scrolled, but how many items you've scrolled(or rows if we scroll from top to bottom).

It could be done by division of pixels scrolled on item height. For example, you scrolled to left by 80px, that's 2 items. It means these items should be invisible because you've scrolled past them. So you know that you scrolled past 2 items, and you know that you should see 10 items in a row. That means you take your data array which has data for row with 100 items, and slice it like this:

var visibleItems = rowData.slice(itemsScrolled, itemsScrolled + 10);

It will give you items which should be visible in viewport at current scroll position. Once you have these items you need to construct html and append it to table.

Also on each scroll event you need to set top and left position for tbody and thead so they would move with scroll, otherwise you will have your data, but it will be at (0; 0) inside a viewport.

Anyway, code speaks thousand of words, so here's the fiddle: https://jsfiddle.net/Ldfjrg81/9/

Note, that this approach requires heights and widths to be precise, otherwise it will work incorrectly. Also if you have items of different sizes, this also should be taken into consideration, so better if you have fixed and equal sizes of items. In jsfiddle, I commented out the code which forces first column to stay in place, but you can render it separately.

It's a good solution to stick to some library as suggested in comments, since it handles a lot of cases for you.

You can make rendering even faster if use react.js or vue.js

Solution 3:

This won't be the answer your are looking for but here's my 2 cents anyway.

Javascript animation (especially given the amount that the DOM has to render) will never be as smooth as you want it. Even if you could get it smooth on your machine, chances are that it will vary drastically on other peoples (Older PC's, Browsers etc).

I would see 2 options if I were to tackle this myself.

Go old school and add a horizontal and vertical scrollbar. I know it's not a pretty solution but it would work well.
Only render a certain amount of rows and discard those off screen. This could be a bit complicated but in essence you would render say 10 rows. Once the user scrolls to a point where the 11th should be there, render that one and remove the 1st. You would pop them in and out as needed.

In terms of the actual JS (you mentioned putting elements in to an array), that isn't going to help. The actual choppyness is due to the browser needing to render that many elements in the first place.

Solution 4:

You're experiencing choppy / non-smooth scrolling because the scroll event fires at a very high pace.

And every time it fires you're adjusting the position of many elements: this is expensive and furthermore until the browser has completed the repaint it's unresponsive (here the choppy scrolling).

I see two options:

Option number one: display only the visible subset of the whole data set (this has been already suggested in another answer so I won't go futher)

Option number two (easier)

First, let animations on left and top css changes occurr via transitions. This is more efficient, is non-blocking and often let the browser take advantage of the gpu

Then instead of repeteadly adjust left and top, do it once a while; for example 0.5 seconds. This is done by the function ScrollWorker() (see code below) that recalls itself via a setTimeout().

Finally use the callback invoked by the scroll event to keep the #scroller position (stored in a variable) updated.

// Position of the `#scroller` element// (I used two globals that may pollute the global namespace// that piece of code is just for explanation purpose)    var oldPosition, 
    newPosition;


// Use transition to perform animations// You may set this in the stylesheet

$('th').css( { 'transition': 'left 0.5s, top 0.5s' } );
$('td').css( { 'transition': 'left 0.5s, top 0.5s' } );


// Save the initial position

newPosition = $('#scroller').position();
oldPosition = $('#scroller').position();


// Upon scroll just set the position value

iScroll.on('scroll', function() {
    newPosition = $('#scroller').position();
} );


// Start the scroll workerScrollWorker();


functionScrollWorker() { 

    // Adjust the layout if position changed (your original code)if( newPosition.left != oldPosition.left || newPosition.top != oldPosition.top ) {
        $('#scroller th:nth-child(1)').css({top: (-newPosition.top), left: (-newPosition.left), position:'relative'});
        $('#scroller th:nth-child(n+1)').css({top: (-newPosition.top), position:'relative'});
        $('#scroller td:nth-child(1)').css({left: (-newPosition.left), position:'relative'});

        // Update the stored position

        oldPosition.left = newPosition.left;
        oldPosition.top  = newPosition.top;

        // Let animation complete then check again// You may adjust the timer value// The timer value must be higher or equal the transition timesetTimeout( ScrollWorker, 500 );

    } else {

        // No changes// Check again after just 0.1secssetTimeout( ScrollWorker, 100 );
    }
}

Here is the Fiddle

I set the Worker pace and the transition time to 0.5 secs. You may adjust the value with higher or lower timing, eventually in a dinamic way based on the number of elements in the table.

Solution 5:

Yes! Here are some improvements to the code from your JS Fiddle. You can view my edits at: https://jsfiddle.net/briankueck/u63maywa/

Some suggested improvements are:

Switching position:relative values in the JS layer to position:fixed in the CSS layer.
Shortening the jQuery DOM chains, so that the code doesn't start at the root element & walk all the way through the dom with each $ lookup. The scroller is now the root element. Everything uses .find() off of that element, which creates shorter trees & jQuery can traverse those branches faster.
Moving the logging code out of the DOM & into the console.log. I've added a debugging switch to disable it, as you're looking for the fastest scrolling on the table. If it runs fast enough for you, then you can always re-enable it to see it in the JSFiddle. If you really need to see that on the iPhone, then it can be added into the DOM. Although, it's probably not necessary to see the left & top position values in the iPhone.
Remove all extraneous $ values, which aren't mapped to the jQuery object. Something like $scroller gets confusing with $, as the latter is the jQuery library, but the former isn't.
Switching to ES6 syntax, by using let instead of var will make your code look more modern.
There is a new left calculation in the <th> tag, which you'll want to look at.
The iScroll event listener has been cleaned up. With position:fixed, the top <th> tags only need to have the top property applied to them. The left <td> tags only need to have the left property applied to them. The corner <th> needs to have both the top & left property applied to it.
Remove everything that's unnecessary, like the extraneous HTML tags which were used for logging purposes.
If you really want to go more vanilla, change out the .css() methods for the actual .style.left= -pos.left + 'px'; and .style.top= -pos.top + 'px'; properties in the JS code.

Try using a diff tool like WinMerge or Beyond Compare to compare the code from your version to what's in my edits, so that you can easily see the differences.

Hopefully, this will make the scrolling smoother, as the scroll event doesn't have to process anything that it doesn't need to do... like 5 full DOM traversing look-ups, rather than 3 short-tree searches.

Enjoy! :)

HTML:

<body><divid="wrapper"><tableid="scroller"><thead></thead><tbody></tbody></table></div></body>

CSS:

/* ... only the relevant bits ... */theadth {
  background-color: #99a;
  min-width: 120px;
  height: 32px;
  border: 1px solid #222;
  position: fixed; /* New */z-index: 9;
}

theadth:nth-child(1) {/*first cell in the header*/border-left: 1px solid #222; /* New: Border fix */border-right: 2px solid #222; /* New: Border fix */position: fixed; /* New */display: block; /*seperates the first cell in the header from the header*/background-color: #88b;
  z-index: 10;
}

JS:

// main codelet debug = false;

$(function(){ 
  let scroller = $('#scroller');
  let top = $('<tr/>');
  for (var i = 0; i < 100; i++) {
    let left = (i === 0) ? 0 : 1;
    top.append('<th style="left:' + ((123*i)+left) + 'px;">'+ Math.random().toString(36).substring(7) +'</th>');
  }
  scroller.find('thead').append(top);
  for (let i = 0; i < 100; i++) {
    let row = $('<tr/>');
    for (let j = 0; j < 100; j++) {
      row.append('<td>'+ Math.random().toString(36).substring(7) +'</td>');
    }
    scroller.find('tbody').append(row);
  }

  if (debug) console.log('initialize iscroll');
  let iScroll = null;
  try {
    iScroll = newIScroll('#wrapper', { 
      interactiveScrollbars: true, 
      scrollbars: true, 
      scrollX: true, 
      probeType: 3, 
      useTransition:false, 
      bounce:false
    });
  } catch(e) {
    if (debug) console.error(e.name + ":" + e.message + "\n" + e.stack);
  }

  if (debug) console.log('initialized');

  iScroll.on('scroll', function(){
    let pos = scroller.position();
    if (debug) console.log('pos.left=' + pos.left + ' pos.top=' + pos.top);

    // code to hold first row and first column
    scroller.find('th').css({top:-pos.top}); // Top Row
    scroller.find('th:nth-child(1)').css({left:-pos.left}); // Corner
    scroller.find('td:nth-child(1)').css({left:-pos.left}); // 1st Left Column
  });
});

JavaScript Creator