One of our teams recently faced the following problem: website users needed to generate large reports that froze the browser window and eventually timed out.
How do we solve this?
Let's think out of the box for a second: we can delegate the generation of large reports to a back-office batch running at short intervals (1 to 5 minutes). Here's a typical implementation of it:
- UI: On the report page, the user chooses the desired filters and clicks "Request Report".
- The system stores a report request in the DB for retrieval from the batch job.
- UI: The user is shown the message: "Your report will be generated within a few minutes. You will be notified when the report is available for download."
...our user is now free to go on about her business :)
- A batch running every minute picks the oldest pending report and generates the file. There's no need to multi-thread this, unless your system has several simultaneous report requests.
- (optional) The batch will email the user when the report is available.
- UI: the page template's script will query batch status on page load and then again every 5 minutes. If a report is found, the script will notify the user. Don't use popups, as they could interrupt the user's activities. Instead, place an icon on the header of your page to alert the user that the report is ready.
- (optional) UI: add a "reports" page to show a user all of his reports, together with their current status (Pending/Generating/Ready).
- UI: The user can now download the finished report.
That's how you usually build a reporting feature capable of handling large amounts of data. It is by no means the ONLY way, but it serves it's purpose with a relatively minimal effort.
Happy coding! ;)