Python threading in for loop

Question

I am creating a python script that grabs information from an API and creates a context menu that gives you access to them. I want to use threading as it runs a little slow on the one call to the API, but I am not sure how to implement threading with my code. I am using this site for threading reference: http://www.ibm.com/developerworks/aix/library/au-threadingpython/ I understand the logic in the code I just don't want to write a threading class for every method that I want threaded.

Here is the class that creates the context menu and then parses the json returned, I think I should add it to the for loop in the run command. Any help is greatly appreciated.

class SyncsnippetsCommand(sublime_plugin.TextCommand):

  def __init__(self, queue):
      threading.Thread.__init__(self)
      self.queue = queue

  def buildLexerDict(self,snippets):
      lexers = snippets[0]['user']['lexers']
      lexer_dict = {}
      for lexer in lexers:
          lexer_dict[lexer] = []
      return lexer_dict

  def buildsnippetsContextDict(self,snippets,lexer_dict):
      snippets_dict = lexer_dict
      for snippet in snippets:
          snippets_dict[snippet['lexer']].append({"id":str(snippet['id']),
                                         "title":snippet['title']})
      return snippets_dict

  def run(self, edit):
      snippet_url = buildsnippetURL()
      snippets_count = 1;
      snippets = getsnippets(snippet_url)
      context_menu = '['
      context_menu += '\n\t{ "caption": "snippets", "id": "file", "children":'
      context_menu += '\n\t\t['
      if snippets == None:
          {"caption":"No snippets available"}
      else:
          snippets = snippets['objects']
          lexers = self.buildLexerDict(snippets)
          snippets_dict = self.buildsnippetsContextDict(snippets, lexers)
          for j,key in reversed(list(enumerate(reversed(snippets_dict.keys())))):
              ... loop through JSON and create menu ...
              if j == 0:
                  context_menu += ''
              else:
                  context_menu += ',' 
      context_menu += '\n\t\t]'
      context_menu += '\n\t}'
      context_menu += '\n]'
      f = open(sublime.packages_path() + '\snippetSync\\Context.sublime-menu', 'w')
      f.write(context_menu)
      f.close
      self.view.set_status('snippet', 'snippet Sync: Added ' + str(snippets_count) + ' snippets from your account.')
      sublime.set_timeout(lambda: self.view.erase_status('snippet'), 3000)
      return

Answer 1

Here is a simple Sublime Text 2 plugin with threading. What it does is insert Hello World! after 3 seconds. What you'll notice is that you can still move the cursor during those three seconds.

In your case, it looks like you just need to grab a bunch of snippets from an API and create a context menu from the returned data. Then there will be a notification at the bottom telling you how many snippets were added. I could be wrong, but you should be able to modify this code to make your plugin work.

import threading
import time
import sublime
import sublime_plugin

"""
The command just creates and runs a thread.
The thread will do all the work in the background.

Note that in your Thread constructor, you will need to pass in an 
instance of your Command class to work with in your thread.
"""
class ExampleCommand(sublime_plugin.TextCommand):

    def run(self, edit):

        exampleThread = ExampleThread(self, edit)
        exampleThread.start()

"""
Extend the Thread class and add your functionality in 
the run method below.

One thing to remember when moving your code over is 
you need to use self.cmd instead of self.
"""
class ExampleThread(threading.Thread):

    """
    Remember to pass in the parameters you need
    in this thread constructor.
    """
    def __init__(self, cmd, edit):
        threading.Thread.__init__(self)
        self.cmd = cmd
        self.edit = edit

    """
    Add your functionality here.

    If you need to access the main thread, you need to
    use sublime.set_timeout(self.callback, 1).

    In my example here, you can't call insert text into the editor
    unless you are in the main thread.

    Luckily that is fast operation.

    Basically, time.sleep(3) is a slow operation and will block, hence it
    is run in this separate thread.
    """
    def run(self):
        time.sleep(3)
        sublime.set_timeout(self.callback, 1)

    """
    This is the callback function that will be called to 
    insert HelloWorld. 

    You will probably need to use this to set your status message at 
    the end. I'm pretty sure that requires that you be on main thread 
    to work.
    """
    def callback(self):
        self.cmd.view.insert(self.edit, 0, "Hello, World!")

Update

I found some time integrate your code snippet above using the approach I outlined above. You'll still need to fill in some blanks, but hopefully this gives you an idea of where to put your code. I tested that the basic skeleton still works, which is why the section where you are building the context menu is commented out in this example.

import threading
import time
import sublime
import sublime_plugin


def buildsnippetURL():
    return ""

def getsnippets(snippet_url):
    time.sleep(3)
    return ""

class SyncsnippetsCommand(sublime_plugin.TextCommand):
    def run(self, edit):
        syncsnippetsThread = SyncsnippetsThread(self, edit)
        syncsnippetsThread.start()


class SyncsnippetsThread(threading.Thread):
    def __init__(self, cmd, edit):
        threading.Thread.__init__(self)
        self.cmd = cmd
        self.edit = edit

    def buildLexerDict(self,snippets):
        lexers = snippets[0]['user']['lexers']
        lexer_dict = {}
        for lexer in lexers:
            lexer_dict[lexer] = []
        return lexer_dict

    def buildsnippetsContextDict(self,snippets,lexer_dict):
        snippets_dict = lexer_dict
        for snippet in snippets:
            snippets_dict[snippet['lexer']].append({"id":str(snippet['id']),
                                         "title":snippet['title']})
        return snippets_dict

    def run(self): 
        snippet_url = buildsnippetURL()
        snippets_count = 1;
        snippets = getsnippets(snippet_url)

        """
        context_menu = '['
        context_menu += '\n\t{ "caption": "snippets", "id": "file", "children":'
        context_menu += '\n\t\t['
        if snippets == None:
            {"caption":"No snippets available"}
        else:
            snippets = snippets['objects']
            lexers = self.buildLexerDict(snippets)
            snippets_dict = self.buildsnippetsContextDict(snippets, lexers)
            for j,key in reversed(list(enumerate(reversed(snippets_dict.keys())))):
                ... loop through JSON and create menu ...
                if j == 0:
                    context_menu += ''
                else:
                    context_menu += ',' 
        context_menu += '\n\t\t]'
        context_menu += '\n\t}'
        context_menu += '\n]'
        f = open(sublime.packages_path() + '\snippetSync\\Context.sublime-menu', 'w')
        f.write(context_menu)
        f.close
        """

        sublime.set_timeout(lambda: self.callback(snippets_count), 1)

    def callback(self, snippets_count):
        self.cmd.view.set_status('snippet', 'snippet Sync: Added ' + str(snippets_count)  + ' snippets from your account.')
        sublime.set_timeout(lambda: self.cmd.view.erase_status('snippet'), 3000)