当前位置： 天天编程网 > 技术新闻 > Pyhton常见问题 > 正文

python如何将任务分配给多个工作线程？

  • 来源:网络转载

python如何将任务分配给多个工作线程？

最简单的方法是使用新的 concurrent.futures 模块，尤其是其中的 ThreadPoolExecutor 类。

或者，如果你想更好地控制分发算法，你也可以自己写逻辑实现。使用 queue 模块来创建任务列表队列。Queue 类维护一个了一个存有对象的列表，提供了 .put(obj) 方法添加元素，并且可以用 .get() 方法获取元素。这个类会使用必要的加锁操作，以此确保每个任务只会执行一次。

这是一个简单的例子：

import threading, queue, time

# The worker thread gets jobs off the queue.  When the queue is empty, it
# assumes there will be no more work and exits.
# (Realistically workers will run until terminated.)
def worker():
    print("Running worker")
    time.sleep(0.1)
    while True:
        try:
            arg = q.get(block=False)
        except queue.Empty:
            print("Worker", threading.currentThread(), end=" ")
            print("queue empty")
            break
        else:
            print("Worker", threading.currentThread(), end=" ")
            print("running with argument", arg)
            time.sleep(0.5)

# Create queue
q = queue.Queue()

# Start a pool of 5 workers
for i in range(5):
    t = threading.Thread(target=worker, name="worker %i" % (i+1))
    t.start()

# Begin adding work to the queue
for i in range(50):
    q.put(i)

# Give threads time to run
print("Main thread sleeping")
time.sleep(5)

运行时会产生如下输出：

Running worker
Running worker
Running worker
Running worker
Running worker
Main thread sleeping
Worker <Thread(worker 1, started 130283832797456)> running with argument 0
Worker <Thread(worker 2, started 130283824404752)> running with argument 1
Worker <Thread(worker 3, started 130283816012048)> running with argument 2
Worker <Thread(worker 4, started 130283807619344)> running with argument 3
Worker <Thread(worker 5, started 130283799226640)> running with argument 4
Worker <Thread(worker 1, started 130283832797456)> running with argument 5
...

查看模块的文档以获取更多信息；Queue 类提供了多种接口。

来源：PY学习网：原文地址：https://www.py.cn/article.html