RuntimeError on windows trying python multiprocessing

I am trying my very first formal python program using Threading and Multiprocessing on a windows machine. I am unable to launch the processes though, with python giving the following message. The thing is, I am not launching my threads in the main module. The threads are handled in a separate module inside a class.

EDIT : By the way this code runs fine on ubuntu. Not quite on windows

    RuntimeError: 
                Attempt to start a new process before the current process
                has finished its bootstrapping phase.
                This probably means that you are on Windows and you have
                forgotten to use the proper idiom in the main module:
                    if __name__ == '__main__':
                        freeze_support()
                        ...
                The "freeze_support()" line can be omitted if the program
                is not going to be frozen to produce a Windows executable.

My original code is pretty long, but I was able to reproduce the error in an abridged version of the code. It is split in two files, the first is the main module and does very little other than import the module which handles processes/threads and calls a method. The second module is where the meat of the code is.

testMain.py:

    import parallelTestModule

    extractor = parallelTestModule.ParallelExtractor()
    extractor.runInParallel(numProcesses=2, numThreads=4)

parallelTestModule.py:

    import multiprocessing
    from multiprocessing import Process
    import threading

    class ThreadRunner(threading.Thread):
        """ This class represents a single instance of a running thread"""
        def __init__(self, name):
            threading.Thread.__init__(self)
            self.name = name
        def run(self):
            print self.name,'\n'

    class ProcessRunner:
        """ This class represents a single instance of a running process """
        def runp(self, pid, numThreads):
            mythreads = []
            for tid in range(numThreads):
                name = "Proc-"+str(pid)+"-Thread-"+str(tid)
                th = ThreadRunner(name)
                mythreads.append(th) 
            for i in mythreads:
                i.start()
            for i in mythreads:
                i.join()

    class ParallelExtractor:    
        def runInParallel(self, numProcesses, numThreads):
            myprocs = []
            prunner = ProcessRunner()
            for pid in range(numProcesses):
                pr = Process(target=prunner.runp, args=(pid, numThreads)) 
                myprocs.append(pr) 
    #        if __name__ == 'parallelTestModule':    #This didnt work
    #        if __name__ == '__main__':              #This obviously doesnt work
    #        multiprocessing.freeze_support()        #added after seeing error to no avail
            for i in myprocs:
                i.start()

            for i in myprocs:
                i.join()

On Windows the subprocesses will import (i.e. execute) the main module at start. You need to insert an if __name__ == '__main__': guard in the main module to avoid creating subprocesses recursively.

Modified testMain.py:

    import parallelTestModule

    if __name__ == '__main__':    
        extractor = parallelTestModule.ParallelExtractor()
        extractor.runInParallel(numProcesses=2, numThreads=4)

From: stackoverflow.com/q/18204782

Back to homepage or read more recommendations: