CODEWITHSUNDEEP
javascriptnode.jsexpressasynchronousdatatable
sakib11
sakib11

Reputation: 536

Handling large json array on the browser to create tables

I am currently working on an inventory management web app where I load a large amount of data on an HTML table by using fetch with async.

async function getNewSummary() {
    try {
        let groups = await fetch(
            "http://localhost:3000/dashboard/summary/groups", {
                method: "GET"
            }
        );
        let groupData = await groups.json();
        console.log(json);
        var table = document.getElementById("grouptable");
        for (data in groupData)
            table.innerHTML = `<tr>
    <td>${groupData[data].groups}</td>
    <td>${groupData[data].permission}</td>
    </tr>`;
    } catch (e) {
        console.log(e);
    }
}

My Express API returns a large JSON array which I loop through and create table rows from. I expected the browser to stop working when it does that. Is there any other way I can get it done? I need to create the table and then use it as a DataTable.

Upvotes: 0

Views: 1595

Answers (2)

djs
djs

Reputation: 4065

Just curious, but I tried loading a pretty large dataset (28795 JSON objects) using your code as an example and it loaded pretty quickly. I adjusted the DOM manipulation to use something besides innerHTML, but other than that it is pretty much the same. Here is the request I made:

main.js

async function getNewSummary() {
    try {
        const groups = await fetch("https://raw.githubusercontent.com/prust/wikipedia-movie-data/master/movies.json");
        const groupData = await groups.json();
        console.log('Num documents: ', groupData.length);
        console.log('Example document: ', groupData[0]);
        const table = document.getElementById("grouptable");
        for (data in groupData) {
            let tr = document.createElement('tr');
            let td = document.createElement('td');
            td.textContent = groupData[data].title;
            tr.appendChild(td);
            table.appendChild(tr);
        }
    } catch (e) {
        console.log(e);
    }
}

getNewSummary();

And here is a gif of it working, it only takes about 2 seconds to create all 28,795 table rows. Could the problem be the way you are adding the data to the DOM and not the size of the dataset? Or is you dataset much larger? I've never worked with large amounts of data, so I could be wayyyy off base here, but wanted to try and help if I could.

enter image description here

Here is another version where I click a button that changes the background to red and adds some p tags while it's loading. It definitely hangs, but it's only for a second:

enter image description here

Upvotes: 2

sakib11
sakib11

Reputation: 536

Here's my current iteration. inside the browser

<script>
function inventoryWorker() {
w = new Worker("js/tableHelper.js");
w.onmessage = function (event) {
DatatableMaker(event.data);
}
}


function DatatableMaker(data) {
$(document).ready(function () {
$('#main').DataTable().destroy();
var events = $("#events");
// Setup text input
$("#main tfoot th").each(function () {
    var title = $(this).text();
    $(this).html(
        '<input class="form-control" type="text" placeholder="Search ' + title +
        '" />'
    );
});
// DataTable
var table = $("#main").DataTable({
    data: data,
    paging: true,
    lengthChange: true,
    searching: true,
    ordering: true,
    info: true,
    scrollX: true,
    scrollCollapse: true,
    autoWidth: true
});
}
</script>

In a seperate file tableHelper.js

async function getInventory() {
    var tableRow = [];
    try {
        let inventory = await fetch("http://localhost:3000/dashboard/inventory", {
            method: "GET"
        });
        let inventoryData = await inventory.json();
        console.log(inventoryData);
        for (data in inventoryData) {
            var temp = [];
            temp.push(
                inventoryData[data].ReceivingDate,
// etc
            );
            tableRow.push(temp);
        }
        postMessage(tableRow);
    } catch (e) {
        console.log(e);
    }
}

It works better but there is a slight pause when the inventoryWorker function is called

Upvotes: 0

Related Questions