Python

Python in a Nutshell (Thread Tips)

A thread is a flow of control that shares global state with other threads; all threads appear to execute simultaneously.

A process is an instance of a running program.

创建线程的两种方法

  • 继承Thread类, 重写run()方法, 而不是start()
  • 创建threading.Thread对象,初始化函数__init__()中可将调用对象作为参数传入

The thread Module

1
2
3

L.acquire(wait=True) # When wait is True, acquire locks L.
L.locked( )          # Returns True if L is locked; otherwise, False.
L.release( )         # Unlocks L, which must be locked.

The Queue Module

The Queue module supplies first-in, first-out (FIFO) queues that support multithread access, with one main class and two exception classes.

1
2
3
4
5
6
7
8

class Queue(maxsize=0)
q.empty()
q.get(False)
q.full()
q.put(x,False)
q.get(block=True,timeout=None)
q.put(item,block=True,timeout=None)
q.qsize( )

“It’s easier to ask forgiveness than permission” (*EAFP) idiom vs
“look before you leap” (LBYL) idiom 

1
2
3
4

try: 
    x=q.get_nowait()
except Queue.Empty: 
    print "no work to perform"

Customizing Class Queue by Subclassing

You may customize Queue.Queue by subclassing, and you need to override some or all of the hook methods that Queue.Queue provides for the purpose: _qsize,_empty, _full, _put, and _get.

1
2
3

import Queue
class LIFOQueue(Queue.Queue):
    def _get(self): return self.queue.pop()

The threading Module 推荐

threading provides numerous classes for dealing with threads, including Thread, Condition, Event, RLock, and Semaphore.

1
2
3
4
5
6
7
8
9
10

currentThread( )
class Thread(name=None,target=None,args=(),kwargs={ })
t.getName() 
t.setName(name)
t.isAlive( )
t.isDaemon() 
t.setDaemon(daemonic)
t.join(timeout=None)
t.run( )
t.start( )

Ex1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

import threading, time, sys
class test(threading.Thread):
	def __init__(self, name, delay):
		threading.Thread.__init__(self)
		self.name = name
		self.delay = delay

	def run(self):
		print "%s delay for %s" % (self.name, self.delay)
		time.sleep(self.delay)
		c = 0
		while Ture:
			print "This is thread %s on line %s" % (self.name, c)
			c += 1
			if c == 3
				print "End of thread %s" % self.name
				break

def main():
	t1 = test('Thread 1', 1)
	t2 = test('Thread 2', 2)
	
	t1.start()
	print "Wait t1 to end"
	t1.join()
	print "End of t1"

	t2.start()
	print "Wait t2 to end"
	t2.join()
	print "End of t2"
	print "End of main"

Thread Synchronization Objects

Timeout parameters

Lock and RLock objects

Condition objects

1
2
3
4
5
6

class Condition(lock=None)
c.acquire(wait=1) 
c.release( )
c.notify( ) 
c.notifyAll( )
c.wait(timeout=None)

In typical use, a Condition object c regulates access to some global state s that is shared between threads. When a thread needs to wait for s to change, the thread loops as follows:

1
2
3
4
5

c.acquire( )
while not is_ok_state(s):
    c.wait( ) 
do_some_work_using_state(s) 
c.release( )

Meanwhile, each thread that modifies s calls notify (or notifyAll if it needs to wake up all waiting threads, not just one) each time s changes:

1
2
3
4

c.acquire( ) 
do_something_that_modifies_state(s) 
c.notify() # or, c.notifyAll() 
c.release( )

Event objects

Event objects let any number of threads suspend and wait. All threads waiting on Event object e become ready when any other thread calls e.set( ).

1
2
3
4
5
6

class Event()
e.clear( ) # Sets e’s flag to False.
e.isSet( ) # Returns the value of e’s flag, True or False.
e.set( )   # Sets e’s flag to True. 
e.wait(timeout=None) # If e’s flag is True, wait returns immediately. 
                     # Otherwise, wait suspends the calling thread until some other thread calls set.

Semaphore objects

Semaphores (also known as counting semaphores) are a generalization of locks. The state of a Lock can be seen as True or False; the state of a Semaphore s is a number between 0 and somen set when s is created.

1
2
3

class Semaphore(n=1)
s.acquire(wait=True)
s.release( )

Thread Local Storage

Module threading supplies a class local, which threads can use to obtain thread-local storage (TLS), also known as per-thread data. An instance L of local has arbitrary named attributes that you can set and get, and stores them in a dictionary L.__dict__ that you can also access. L is fully thread-safe, meaning there is no problem if multiple threads simultaneously set and get attributes on L.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import threading
L = threading.local()
print 'in main thread, setting zop to 42' 
L.zop = 42

def targ():
    print 'in subthread, setting zop to 23'
    L.zop = 23
    print 'in subthread, zop is now', L.zop

t = threading.Thread(target=targ) t.start( )
t.join( )

print 'in main thread, zop is now', L.zop

Ex2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

import os, Queue, threading, urllib2

class DownLoadThread(threading.Thread):
	"""docstring for DownLoadThread"""
	def __init__(self, queue):
		#super(DownLoadThread, self).__init__()
		threading.Thread.__init__(self)
		self.queue = queue

	def run(self):
		while True:
			url = self.queue.get() # 从队列中取出一个url元素
			print self.name + " begin downloading " + url + ' ...'
			self.download_file(url)
			self.queue.task_done()
			print self.name + " finished downloading " + url + ' !!!'

	def download_file(self, url):
		urlhander = urllib2.urlopen(url)
		fname = os.path.basename(url) + ".html"
		with open(fname, "wb") as f:
			chunk = urlhander.read(1024)
			if not chunk: 
				break
			f.write(chunk)


if __name__ == '__main__':
	urls = ["http://weibo.com/buttonwood",
			"http://www.douban.com/people/buttonwood/",
			"http://buttonwood.github.io"
	]

	queue = Queue.Queue()
    # queue 队列本身能够保证线程安全，不需要额外的同步机制
	# creat a thread pool and give them a queue
	for x in xrange(1,5):
		t = DownLoadThread(queue) # 5 thread for doanloading
		t.setDaemon(True)
		t.start()

	# give the queue some data	
	for y in urls:
		queue.put(url)

	# wait for the queue to finish	
	queue.join()