Docker是如何做进程/线程限制的?


看了 (http://dockerone.com/article/216),里面提到Docker关于CPU、内存、IO的限制,但是没提到进程/线程资源的限制,那Docker具体是如何限制容器可创建的进程/线程数呢?如何配置呢?

我在一台服务器上跑了两组服务容器,一组大概包含nginx mysql redis memcached zk和20+tomcat,当我启动第二组的时候会报java.lang.OutOfMemoryError: unable to create new native thread。
后来我修改了容器的nproc.conf解决,但是这一块的内容不是很了解,希望各位能解答一下。

=====================华丽的分割线 2015-03-06=======================
补充说明:
1 两组总共50个容器左右,都是标准用法,一个应用一个容器
2 Java web应用中大部分使用线程池
3 容器内使用centos7,nproc.conf默认是4096,宿主机默认是65536

可以看到宿主机的限制已经足够大了;而且在起第二组服务的时候,第一组和第二组的tomcat都报java.lang.OutOfMemoryError: unable to create new native thread。是否可以推测出容器内的nproc.conf也是全局共享的?或者是不共享但是由于映射到宿主机都是统一个用户,统计做限制时是统计所有容器的线程数?
已邀请:

叶可强 - DockOne.io翻译编辑

赞同来自: 寻觅神迹 郑伟-风刃


1、你一台服务器上启动了多少个容器?
2、你是每个容器放一个应用么?还是说在一个容器里面放了多个,如一个容器里面放了 nginx、 mysql、redis?
3、你这个修改的文件是服务器上的 nproc.conf 还是你第二组容器里面的?第一组没有修改?服务器的也没有修改?

另外这里有个解答,希望对你有帮助,这个在命令行是没法修改的。下面看的有疑问可以回复我。
<pre>
Spent some time looking into this again today - and I wonder if I was not setting ulimit correctly due to the centos container defaults.

The Docker Host is running on Ubuntu 14.04 with default limits. The Docker service is configured to start with overrides:
ubuntu@:~$ cat /etc/init/docker.conf
description "Docker daemon"

start on (local-filesystems and net-device-up IFACE!=lo)
stop on runlevel
limit nofile 524288 1048576
limit nproc 524288 1048576
...

If you run the centos container it will inherit the limits that Docker service on the host is running with:
ubuntu@:~$ docker run -t -i centos:centos6 /bin/bash
bash-4.1# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 122310
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 524288
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 524288
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

However - if you run as a user (not root) in the centos container it will have a restricted ulimit:
bash-4.1# /usr/sbin/adduser testuser
bash-4.1# su -c "ulimit -a" testuser
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 122310
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 524288
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

This is due to the centos container defaults:
bash-4.1# cat /etc/security/limits.d/90-nproc.conf

Default limit for number of user's processes to prevent

accidental fork bombs.

See rhbz #432903 for reasoning.

  • soft nproc 1024
    root soft nproc unlimited


Overwriting that file via a Docker file:
ubuntu@:~/test$ cat limits-90-nproc.conf

Increase default limit

* soft nproc 524288
root soft nproc unlimited

ubuntu@:~/test$ cat Dockerfile
FROM centos:centos6
ADD ./limits-90-nproc.conf /etc/security/limits.d/90-nproc.conf

Then we now get correct ulimit:
ubuntu@:~/test$ docker build -t test .
...
ubuntu@:~/test$ docker run -t -i test /bin/bash
bash-4.1# /usr/sbin/adduser testuser
bash-4.1# su -c "ulimit -a" testuser
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 122310
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 524288
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 524288
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

</pre>

要回复问题请先登录注册