Tech Monger

In this tutorial we will learn how to write python code that provides solution to the classic producer consumer problem with the help of an example. We will make use of inbuilt python features such as Thread and Queue.

---

---

Queue

Python provide Queue class which implements queue data structure. We can put an item inside the queue and we can get an item from the queue. By default this works in FIFO manner.

In our example, function producer will put an item inside queue and function consumer will get an item from the queue. We will use following method of queue class by instantiating queue object q = Queue().

Method	Description
q.put()	To put an item inside queue.
q.get()	To get an item which is present inside queue.
q.join()	This method stops python program from exiting till it gets signal from the below method task_done. Hence this method should always be used in conjunction with method task_done
q.task_done()	This method should be called when item got outside from the queue using q.get() has been completely processed by consumer. When all items make call to their respective task_done it sends signals to q.join() that all items have been processed and program can exit.

---

Threads

Python allows writing multi-threaded program using Thread class. We will instantiate object of thread class and make use of following methods to process (consume) multiple items concurrently.

Constructor / Method	Description
t = Thread(target=consumer)	Instantiate thread object which would make call to function consumer.
t.start()	Starts execution of thread by making call to function consumer.

---

Example

from threading import Thread
from Queue import Queue

q = Queue()
final_results = []

def producer():
    for i in range(100):
        q.put(i)
        

def consumer():
    while True:
        number = q.get()
        result = (number, number**2)
        final_results.append(result)
        q.task_done()
   
   
for i in range(5):
    t = Thread(target=consumer)
    t.daemon = True
    t.start()
    
producer()

q.join()

print final_results

Conclusion

Tagged Under : CS Python