我在学校的一个实验室集群上做东西,每台服务器都是32个cpu32G内存的机子。
我在一台服务器上运行了一个TCPServer
端,用twisted写的。
并在另一台服务器上运行如下python脚本创建1万个TCPClient
客户端连接到服务器:
# -*- coding: UTF-8 -*-.
import socket, optparse, time, os, threading
from sensor import Sensor
def strWrapper(sock, data):
host, port = sock.getsockname() # local IP_addr
""" 封装数据(IP地址 传感器类型ID 采集值 采集时间)"""
str = '%s:%s,%d,%s,%s\r\n' % (host, port, data[0], data[1], data[2])
return str
def main(threadID, args):
options, address = args
host, port = address # remote addr
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(address)
print "%d.Connected with %s:%s" % (threadID, host, port)
sensor = Sensor(options.typeid)
for j in xrange(0, 5000):
#sensor.get_data()模拟的随机数,不耗时
req = strWrapper(s, sensor.get_data())
s.send(req)
s.close()
if __name__ == '__main__':
start = time.time()
args = parse_args()
threads = []
for i in xrange(0, 10000):
thread = threading.Thread(target=main, args=(i, args))
threads.append(thread)
for thr in threads:
thr.start()
# thr.join()
# end = time.time()
# print 'Task runs %0.2f seconds.' % ((end - start))
但是在600多个连接线程的时候就出问题了:
左边是TCP服务器,右边是TCP客户端
内存什么的资源都很充足 ulimit -a
结果如下
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256516
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
我参考了一下这位同学可以并发达10w
现在想解决的问题如下:
1.我的代码哪里出现了问题?
2.或者说如何写测试程序
诸位前辈的任何回答将不甚感激,谢谢了
684*10240/1024/1024 = 6G
stack size 有10M,太大了吧,改小试试?