Store and update JSON Data on a Server

Question

My web-application should be able to store and update (also load) JSON data on a Server. However, the data may contain some big arrays where every time they are saved only a new entry was appended.

My solution:

send updates to the server with a key-path within the json data.

Currently I'm sending the data with an xmlhttprequest by jquery, like this

/**
 * Asynchronously writes a file on the server (via PHP-script).
 * @param {String} file complete filename (path/to/file.ext)
 * @param content content that should be written. may be a js object.
 * @param {Array} updatePath (optional), json only. not the entire file is written,
 * but the given path within the object is updated. by default the path is supposed to contain an array and the
 * content is appended to it.
 * @param {String} key (optional) in combination with updatePath. if a key is provided, then the content is written
 * to a field named as this parameters content at the data located at the updatePath from the old content.
 *
 * @returns {Promise}
 */
io.write = function (file, content, updatePath, key) {
    if (utils.isObject(content)) content = JSON.stringify(content, null, "\t");
    file = io.parsePath(file);
    var data = {f: file, t: content};
    if (typeof updatePath !== "undefined") {
        if (Array.isArray(updatePath)) updatePath = updatePath.join('.');
        data.a = updatePath;
        if (typeof key !== "undefined") data.k = key;
    }
    return new Promise(function (resolve, reject) {
        $.ajax({
            type: 'POST',
            url: io.url.write,
            data: data,
            success: function (data) {
                data = data.split("\n");
                if (data[0] == "ok") resolve(data[1]);
                else reject(new Error((data[0] == "error" ? "PHP error:\n" : "") + data.slice(1).join("\n")));
            },
            cache: false,
            error: function (j, t, e) {
                reject(e);
                //throw new Error("Error writing file '" + file + "'\n" + JSON.stringify(j) + " " + e);
            }
        });
    });
};

On the Server, a php script manages the rest like this:

1. recieves the data and checks if its valid
2. check if the given file path is writable
3. if the file exists and is .json
   
   • read it and decode the json
   • return an error on invalid json
4. if there is no update path given
   
   • just write the data
5. if there is an update path given
   
   • return an error if the update path in the JSON data can't be traversed (or file didn't exist)
6. update the data at update-path
7. write the pretty-printed json to file

However I'm not perfectly happy and problems kept coming for the last weeks.

My Questions

1. Generally: How would you approach this problem? alternative suggestions, databases? any libraries that could help?
   Note: I would prefer solutions, that just use php or some standart apache stuff.
2. One problem was, that sometimes, multiple writes on the same file were triggered. To avoid this I used the Promises (wrapped it because I read jquerys deferred stuff isnt Promise/A compliant) client side, but I dont feel 100% sure it is working. Is there a (file) lock in php that works across multiple requests?
3. Every now and then the JSON files break and its not clear to me how to reproduce the problem. At the time it breaks, I don't have a history of what happened. Any general debugging strategies with a client/server saving/loading process like this?

Answer 1

1. I wrote a comet enable web server that does diffs on updates of json data structures. For the exactly same reason. The server keeps a few version of a json document and serves client with different version of the json document with the update they need to get to the most reason version of the json data.

Maybe you could reuse some of my code, written in C++ and CoffeeScript: https://github.com/TorstenRobitzki/Sioux

3. If you have concurrent write accesses to your data structure, are your sure, that who ever writes to the file has the right version of the file in mind when reading the file?