Brought to my consideration recently by a change in the way I use a program for school, I’ve been thinking about good and bad ways to store data in networked applications.  In normal programs, resources are stored locally on the machine that runs the application and retrieved directly by the system when they are needed.  Applications that run over the network, however, are subject to a different kind of data storage model, one where the "master" or "server" holds the authoritative copy of everything, and all data on the client machine, save in some cases for the program executable itself, is considered to be a temporary ("working") copy.

There are some humps to overcome in the latter (network) model.  First, how does the program handle resources that are unlikely to change?  Let’s take interface sounds as an example.  It is unlikely that the sounds played when the user clicks a button or correctly answers a prompt (for e.g. educational software) will change frequently.  Thus, it does not make sense to load such resources again from the master copy every time the program is started.  The first time the application is run, it should retrieve and cache all common, static resources on the local disk, and merely query the server for updates on subsequent runs.  This saves time and network (LAN or Internet, depending on context) bandwidth.

What of resources that might change from one day to the next, or even one hour to the next?  The same principles should be applied, because it is likely that users will use the program more frequently than the data is updated.  For example, a course content delivery system I am using for a class I’m taking this quarter violates the best practice I’ve set out above.  (I came up with this post myself, with no input from any research at all.)  On each program load, it proceeds to download a fresh copy of all the resources it uses, including question databases for practice, sound files, and text.  When run locally, the application takes a few seconds to start; running over the Internet, startup times jump to over five minutes, perhaps ten or fifteen.

It is not the fact that the first load takes ten or fifteen minutes that is bad for usability.  The fact that the delay occurs at every single startup indicates that no caching at all is taking place, and that discounts the fact that a full copy of the materials on the server is present on the local machine, included in the normal installation.  All sound files, text, and course material is already loaded on the local disk, and is downloaded across the Internet on each load for no reason.  So, not only does the program not cache what it downloads, but it doesn’t even necessarily need to download the data in the first place because of the presence of a local copy.

In summary, an application that can be configured to use a central database or storage system, whether for centralized record-keeping or otherwise, should follow these points:

• Download only what is necessary. This means don’t download files that won’t be used.  Load material dynamically each time a lesson is requested, for instance; this reduces initial load time and makes the user wait less.
• Cache what is downloaded.  This further reduces load times and cuts bandwidth usage.
• Intelligently determine what is available locally.  There is no sense in downloading a copy of a file already present on the local machine.  Resources identical to those already available locally should never be downloaded.

If the program I am thinking of follows these principles, and perhaps (could it be possible?) improves on them, it would reduce the wait time of everyone in my class, including the teacher (who works in no closer proximity to the server than us students).  Currently, one has to plan on reading a chapter of a novel or jogging half a mile (or writing a blog post on how to improve network applications ) while the software loads.  Urgh.

For a more detailed explanation of the problems with the specific program that inspired this post, see my blog, Technobabbles.  For a fictional perspective, see The Queiba Wars.