CODEWITHSUNDEEP
google-apps-scriptgoogle-fusion-tables
Doug Chestnut
Doug Chestnut

Reputation: 31

Fusion tables importRows via Apps Script Failing more than not

I am trying to pull csv attachments from a GMail account and push them to a Fusion table. It works sometimes but most of the time it fails with the error "Execution failed: Internal error when processing import. Please try again." from the importRows call for the Fusion table. The size of the import is ~4k so well below the 1m limit.

I will try using the Fusion table SQL INSERT option instead of importRows but just wanted to see if I was perhaps doing something wrong with the importRows call. I even placed a delay of 5 seconds between calls to see if perhaps the Fusion table needed to be fed more slowly.

Here is the App Script:

function getDataFromMailbox() {
  var tableID = "XXXXXXXX"; 
  var threads = GmailApp.getInboxThreads();
  if (threads.length) {
    var thread = threads.pop();
    var messages = thread.getMessages();
    for (var j=0; j < messages.length; j++) {
      var message = messages[j];
      if ( message.isUnread() ) {
        var from = message.getFrom();
        if (from == "[email protected]") {
          var attachments = message.getAttachments();
          for (var k=0; k< attachments.length; k++) {
            var attachment = attachments[k];
            Logger.log( attachment.getSize() );
            FusionTables.Table.importRows(tableID, attachment, {startLine:2});
          }
        } else {
          thread.moveToTrash(); // Thread isn't from our source
        }
        message.markRead();
      }
    }
    Logger.log("Processed %s messages", messages.length);
    if ( !thread.isUnread() ) { thread.moveToArchive(); }
  }
}

Upvotes: 0

Views: 173

Answers (1)

Doug Chestnut
Doug Chestnut

Reputation: 31

I finally had the chance to look into this again. It looks like the sleep call was the key. If I have a 5 second sleep after a fusion table import I get the import error. If I sleep my script for 10 seconds I get a timeout error. 7 seconds seems to be just right (at least today). Here is my updated script in case anyone wants to log data to a gmail account and then pull it to a Fusion table with an App Script.

function getDataFromMailbox() {
  var tableID = "XXXXXXXXXXXXXXXXXX";

  var threads = GmailApp.getInboxThreads();
  while (threads.length) {
    var thread = threads.pop();
    var messages = thread.getMessages();
    for (var j=0; j < messages.length; j++) {
      var message = messages[j];
      if ( message.isUnread() ) {
        var from = message.getFrom();
        if (from == "[email protected]") {
          var attachments = message.getAttachments();
          for (var k=0; k< attachments.length; k++) {
            var attachment = attachments[k];

            var attString = attachment.getDataAsString();
            var lines = attString.match(/^.*((\r\n|\n|\r)|$)/gm);
            lines.shift(); lines.shift();

            Logger.log( lines.length );
            FusionTables.Table.importRows(tableID, attachment, {startLine:2});
          }
        } else {
          thread.moveToTrash(); // Thread isn't from our source
        }
        Logger.log( "Processed message: " + message.getId() );
        message.markRead();
        Utilities.sleep(7000);  // Wait for fusion tables to catch up
      }
    }
    Logger.log("Processed %s messages", messages.length);
    if ( !thread.isUnread() ) { thread.moveToArchive(); }
  }
}

Upvotes: 2

Related Questions