Multi Threading in Python

Multithreading

Running several threads is similar to running several different programs concurrently, but with the following benefits:

  • Multiple threads within a process share the same data space with the main thread and can therefore share information or communicate with each other more easily than if they were separate processes.
  • Threads sometimes called light-weight processes and they do not require much memory overhead; they care cheaper than processes.

A thread has a beginning, an execution sequence, and a conclusion. It has an instruction pointer that keeps track of where within its context it is currently running.

  • It can be pre-empted (interrupted)
  • It can temporarily be put on hold (also known as sleeping) while other threads are running - this is called yielding.

         Starting a New Thread

To spawn another thread, you need to call following method available in thread module:

thread.start_new_thread ( function, args[, kwargs] )

This method call enables a fast and efficient way to create new threads in both Linux and Windows.

The method call returns immediately and the child thread starts and calls function with the passed list of agrs. When function returns, the thread terminates.

Here, args is a tuple of arguments; use an empty tuple to call function without passing any arguments. kwargs is an optional dictionary of keyword arguments.

Example

#!/usr/bin/python

import thread

import time

  • Define a function for the thread def print_time( threadName, delay):

count = 0

while count < 5: time.sleep(delay) count += 1

print "%s: %s" % ( threadName, time.ctime(time.time()) )

  • Create two threads as follows

try:

thread.start_new_thread( print_time, ("Thread-1", 2, ) ) thread.start_new_thread( print_time, ("Thread-2", 4, ) )

except:

print "Error: unable to start thread"

while 1: pass

When the above code is executed, it produces the following result:

Thread-1: Thu Jan 22 15:42:17 2009

Thread-1: Thu Jan 22 15:42:19 2009

Thread-2: Thu Jan 22 15:42:19 2009

Thread-1: Thu Jan 22 15:42:21 2009

Thread-2: Thu Jan 22 15:42:23 2009

Thread-1: Thu Jan 22 15:42:23 2009

Thread-1: Thu Jan 22 15:42:25 2009

Thread-2: Thu Jan 22 15:42:27 2009

Thread-2: Thu Jan 22 15:42:31 2009

Thread-2: Thu Jan 22 15:42:35 2009

Although it is very effective for low-level threading, but the thread module is very limited compared to the newer threading module.

Threading Module

The newer threading module included with Python 2.4 provides much more powerful, high-level support for threads than the thread module discussed in the previous section.

The threading module exposes all the methods of the thread module and provides some additional methods:

  • activeCount(): Returns the number of thread objects that are active.
  • currentThread(): Returns the number of thread objects in the caller's thread control.
  • enumerate(): Returns a list of all thread objects that are currently active.

In addition to the methods, the threading module has the Thread class that implements threading. The methods provided by the Thread class are as follows:

  • run(): The run() method is the entry point for a thread.
  • start(): The start() method starts a thread by calling the run method.
  • join([time]): The join() waits for threads to terminate.
  • isAlive(): The isAlive() method checks whether a thread is still executing.
  • getName(): The getName() method returns the name of a thread.
  • setName(): The setName() method sets the name of a thread.

Creating Thread 

To implement a new thread using the threading module, you have to do the following:

Then, override the run(self [,args]) method to implement what the thread should do when started.

  • Override the __init_(self [,args]) method to add additional arguments.Define a new subclass of the Thread class.
  • Once you have created the new Thread subclass, you can create an instance of it and then start a new thread by invoking the start(), which in turn calls run() method.Example#!/usr/bin/python
  • import threading import time
  • exitFlag = 0class myThread (threading.Thread):def __init__(self, threadID, name, counter): threading.Thread.__init__(self) self.threadID = threadIDself.name = name self.counter = counterdef run(self):print "Starting " + self.name print_time(self.name, self.counter, 5) print "Exiting " + self.namedef print_time(threadName, delay, counter): while counter:if exitFlthread.exit()time.sleep(delay)

    print "%s: %s" % (threadName, time.ctime(time.time())) counter -= 1

    # Create new threads

    thread1 = myThread(1, "Thread-1", 1) thread2 = myThread(2, "Thread-2", 2)

    # Start new Threads thread1.start() thread2.start()

    print "Exiting Main Thread"

    When the above code is executed, it produces the following result:

    Starting Thread-1

    Starting Thread-2

    Exiting Main Thread

    Thread-1: Thu Mar 21 09:10:03 2013

    Thread-1: Thu Mar 21 09:10:04 2013

    Thread-2: Thu Mar 21 09:10:04 2013

    Thread-1: Thu Mar 21 09:10:05 2013

    Thread-1: Thu Mar 21 09:10:06 2013

    Thread-2: Thu Mar 21 09:10:06 2013

    Thread-1: Thu Mar 21 09:10:07 2013

    Exiting Thread-1

  • Add threads to thread listThread-2: Thu Mar 21 09:10:08 2013Thread-2: Thu Mar 21 09:10:10 2013Thread-2: Thu Mar 21 09:10:12 2013Exiting Thread-2

  • Synchronizing Threads

  • The threading module provided with Python includes a simple-to-implement locking mechanism that allows you to synchronize threads. A new lock is created by calling theLock() method, which returns the new lock.The acquire(blocking) method of the new lock object is used to force threads to run synchronously. The optional blocking parameter enables you to control whether the thread waits to acquire the lock.If blocking is set to 0, the thread returns immediately with a 0 value if the lock cannot be acquired and with a 1 if the lock was acquired. If blocking is set to 1, the thread blocks and wait for the lock to be released.The release() method of the new lock object is used to release the lock when it is no longer required.Example

    #!/usr/bin/python

    import threading import time

    class myThread (threading.Thread):

    def __init__(self, threadID, name, counter): threading.Thread.__init__(self) self.threadID = threadID

    self.name = name self.counter = counter

    Free lock to release next thread threadLock.release()def run(self):print "Starting " + self.namedef print_time(threadName, delay, counter): while counter:

    time.sleep(delay)

    print "%s: %s" % (threadName, time.ctime(time.time())) counter -= 1

    threadLock = threading.Lock() threads = []

    # Create new threads

    thread1 = myThread(1, "Thread-1", 1) thread2 = myThread(2, "Thread-2", 2)

    • Start new Threads thread1.start() thread2.start()

    Get lock to synchronize threads threadLock.acquire() print_time(self.name, self.counter, 3)

    threads.append(thread1)threads.append(thread2)# Wait for all threads to complete for t in threads:

    t.join()

    print "Exiting Main Thread"

    When the above code is executed, it produces the following result:

    Starting Thread-1

    Starting Thread-2

    Thread-1: Thu Mar 21 09:11:28 2013

    Thread-1: Thu Mar 21 09:11:29 2013

    Thread-1: Thu Mar 21 09:11:30 2013

    Thread-2: Thu Mar 21 09:11:32 2013

    Thread-2: Thu Mar 21 09:11:34 2013

    Thread-2: Thu Mar 21 09:11:36 2013

    Exiting Main Thread

  • Multithreaded Priority Queue

    The Queue module allows you to create a new queue object that can hold a specific number of items. There are following methods to control the Queue:

    • get(): The get() removes and returns an item from the queue.

     

    • put(): The put adds item to a queue.

     

    • qsize() : The qsize() returns the number of items that are currently in the

     

    • empty(): The empty( ) returns True if queue is empty; otherwise, False.

     

    • full(): the full() returns True if queue is full; otherwise, False.

    Example

    #!/usr/bin/python

    import Queue import threading import time

    exitFlag = 0

    class myThread (threading.Thread):

    def __init__(self, threadID, name, q): threading.Thread.__init__(self) self.threadID = threadID

    self.name = name self.q = q

    def run(self):

    print "Starting " + self.name process_data(self.name, self.q) print "Exiting " + self.name

    def process_data(threadName, q): while not exitFlag:

    queueLock.acquire()

    if not workQueue.empty(): data = q.get() queueLock.release()

    print "%s processing %s" % (threadName, data) else:

    queueLock.release()

    time.sleep(1)

    threadList = ["Thread-1", "Thread-2", "Thread-3"] nameList = ["One", "Two", "Three", "Four", "Five"] queueLock = threading.Lock()

    workQueue = Queue.Queue(10) threads = []

    threadID = 1

    # Create new threads

    for tName in threadList:

    thread = myThread(threadID, tName, workQueue) thread.start()

    threads.append(thread) threadID += 1

    • Fill the queue queueLock.acquire() for word in nameList:

    workQueue.put(word)

    queueLock.release()

    • Wait for queue to empty

    while not workQueue.empty(): pass

    • Notify threads it's time to exit exitFlag = 1

     

    • Wait for all threads to complete for t in threads:

    t.join()

    print "Exiting Main Thread"

    When the above code is executed, it produces the following result:

    Starting Thread-1

    Starting Thread-2

    Starting Thread-3

    Thread-1 processing One

    Thread-2 processing Two

    Thread-3 processing Three

    Thread-1 processing Four

    Thread-2 processing Five

    Exiting Thread-3

    Exiting Thread-1

    Exiting Thread-2

    Exiting Main Thread

    ####################Thread with Process Synchronization############################

    import threading

    class trd1(threading.Thread):

    def __init__(self,id):
    threading.Thread.__init__(self)
    self.id=id
    print("thread is created")

    def run(self): #function Overriding
    lck.acquire()
    self.print_loop()
    lck.release()

    def print_loop(self):
    i=0
    while(i<10):
    print("Thread id:",self.id)
    i=i+1

    lck=threading.Lock() # Process Synchronization

    t1=trd1(1)
    t1.start()

    t2=trd1(2)

     

    t2.start()

  • ###########     OUTPUT############################################

    thread is created
    Thread id: 1
    thread is created
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 1
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2
    Thread id: 2

########################################################

import threading
import time;
import queue

class thread1(threading.Thread) :

def __init__(self,trdid,start1,end1,q):

threading.Thread.__init__(self);
data = threading.local()
self.start1=start1
self.trdid=trdid
self.end1=end1
self.q = q

def run(self):
self.add(self.start1,self.end1)
def add (self,start,end):
data = threading.local()
i=start;
sum=0;
while(i<=end):
sum=sum+i;
i=i+1

lck.acquire()
q.put(sum)
lck.release()
print("trd %d sum=%d"%(self.trdid,sum))
return data

q = queue.Queue(10)
lck=threading.Lock()
max_n=int(input("Enter range"))

trds=int(input("Enter No of threads"))

step1=int(max_n/trds);

start1=1;

end1=step1;

print("step",step1)

i=1;

l=[];
# l=[t1,t2,t3...]
ticks1 = time.time()
while(i<=trds):
print("Thread %d range %d - %d "%(i,start1,end1))
t1=thread1(i,start1,end1,q);
t1.start()
l.append(t1)

start1=start1+step1
end1=end1+step1
i=i+1
n=len(l)
i=0;

while (i<n) :
print("I amhere")
t=l[i]
t.join(10)
i=i+1

lck=threading.Lock()

sum=0
while not q.empty():
data = q.get()
sum=sum+data
print("sum",sum)

ticks2 = time.time()

print("Run time",ticks2-ticks1)

###################################

import threading
import time;

class thread1(threading.Thread) :

def __init__(self,trdid,start,end):
threading.Thread.__init__(self)
self.tread_id=trdid;
self.start=start
self.end=end
def run(self):
add(self.start,self.end)

def add (start,end):

i=self.start;
sum=0;
while(i<=self.end):
sum=sum+i;
i=i+1
# lck.release()
# threading.Thread.join(10)
print("sum=",sum)
# return sum

t1=thread1(1,1,50);
#t2=thread1(2,51,100);
#lck=threading.Lock()

t1.start()
#t2.start()

t1.join(10)

#t2.join(10)

#print("sum by trd 1", s1);
#print("sum by trd 2", s2);

#print("sum=",s1+s2)

#####################################################################

import threading
import time;
import queue

class thread1(threading.Thread) :

def __init__(self,trdid,start1,end1,q):

threading.Thread.__init__(self);
data = threading.local()
self.start1=start1
self.trdid=trdid
self.end1=end1
self.q = q

def run(self):
self.add(self.start1,self.end1)
def add (self,start,end):
data = threading.local()
i=start;
sum=0;
while(i<=end):
sum=sum+i;
i=i+1

lck.acquire()
q.put(sum)
lck.release()
# threading.Thread.join(10)

# threading.Thread.join(self)
print("trd %d sum=%d"%(self.trdid,sum))
return data

q = queue.Queue(10)

t1=thread1(1,1,5,q);
t2=thread1(2,6,10,q);
lck=threading.Lock()

ticks1 = time.time()
t1.start()

t2.start()

t1.join(10)

t2.join(10)

sum=0
while not q.empty():
data = q.get()
sum=sum+data
print("sum",sum)

ticks2 = time.time()

print("Run time",ticks2-ticks1)

############################################

 

Multi processing

##########################################

from multiprocessing import Process
import os

def info(title):
print(title)
print('module name:', __name__)
print('parent process:', os.getppid())
print('process id:', os.getpid())

def f(name):
info('function f')
os.system('python plt1.py')
print('hello', name)

if __name__ == '__main__':
info('main line')
p = Process(target=f, args=('bob',))
p.start()
print("I am here")

p.join()