Skip to content

Infinite loading when file is too large #3

@andrsdt

Description

@andrsdt

I'm trying to process this one-year conversation txt file:

9.5 MiB
156279 lines
9463900 characters

The website is stuck with a loading message and the following message appears in the console:

Uncaught (in promise) DOMException: Failed to execute 'setItem' on 'Storage': Setting the value of 'WAWData' exceeded the quota.
    at https://whatsappwrapped.tech/js/script.js:86:20

I have been reading about sessionStorage and localStorage and their capacity seems to be limited to 5MB each on most browsers (I have tried Chrome, Firefox, Opera and Brave without success).

An ugly workaround that would work would be to split the JSON string in two parts and store a half in localStorage and another half in sessionStorage, like this:

script.js

// (...)
const half = toStore.length/2;
const toStore1 = toStore.substring(0, half);
const toStore2 = toStore.substring(half, toStore.length);
sessionStorage.setItem('WAWData1', toStore1)
localStorage.setItem('WAWData2', toStore2)
// (...)

wrap.js

google.charts.load('current', {'packages':['corechart', 'bar']});
google.charts.setOnLoadCallback(drawChart);
data1 = sessionStorage.getItem('WAWData1')
data2 = localStorage.getItem('WAWData2')
fullData = data1.concat(data2)
JSONfullData = JSON.parse(fullData)
const {
  user1,
  user2,
  firstName1,
  firstName2,
  numMessages,
  messagesPerPerson,
  wordCountTotal,
  topThreeEmojisPerPerson,
  currentStreak,
  mostActiveHour,
} = JSONfullData || '{}'
let lastWidth;
let lastCloudWidth;
// (...)

This way, the max size would be ~10MB. There must be a cleaner way to achieve this anyway.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions