Linux-VServer - User contributions [en]

Memory Limits

2010-04-25T08:27:33Z

Gl2k10: /* Overview */

== Overview ==
A vserver kernel keeps track many resources used by each guest (context). Some of these relate to memory usage by the guest. You can place limits on these resources to prevent guests from using all the host memory and making the host unusable.

Two resources are particularly important in this regard:

* The '''Resident Set Size''' (<code>rss</code>) is the amount of pages currently present in RAM.
* The '''Address Space''' (<code>as</code>) is the total amount of memory (pages) mapped in all the processes in the context.

Both are measured in '''pages''', which are 4 kB each on Intel machines (i386). So a value of 200000 means a limit of 800,000 kB, a little less than 800 MB.

''To easily find out the page size on your host try this line and ignore the warnings''
<pre>echo 'int main () { printf ("%dKiB\n", getpagesize ()/1024); return 0; }' | gcc -xc - -o getpagesize && ./getpagesize</pre>
Each resource has a '''soft''' and a '''hard limit'''.

* If a guest exceeds the <code>rss</code> hard limit, the kernel will invoke the Out-of-Memory (OOM) killer to kill some process in the guest.
* The <code>rss</code> soft limit is shown inside the guest as the maximum available memory. If a guest exceeds the <code>rss</code> soft limit, it will get an extra "bonus" for the OOM killer (proportional to the oversize).
* If a guest exceeds the <code>as</code> hard limit, memory allocation attempts will return an error, but no process is killed.
* The <code>as</code> soft limit is not in utilized until now. In the future it may be used to penalizing guests over that limit or it could be used to force swapping on them and such ...

Bertl explained the difference between '''rss''' and '''as''' with the following example. If two processes share 100 MB of memory, then only 100 MB worth of virtual memory pages can be used at most, so the RSS use of the guest increases by 100 MB. However, two processes are using it, so the AS use increases by 200 MB.

This makes me think that limiting AS is less useful than limiting RSS, since it doesn't directly reflect real, limited resources (RAM and swap) on the host, that deprive other virtual machines of those resources. Bertl says that AS limits can be used to give guests a "gentle" warning that they are running out of memory, but I don't know how much more gentle it is, or how to set it accurately.

For example, 100 processes each mapping a 100 MB file would consume a total of 10 GB of address space (AS), but no more than 100 MB of resources on the host. But if you set the AS limit to 10 GB, then it will not stop one process from allocating 4 GB of RAM, which could kill the host or result in that process being killed by the OOM killer.

this is posted by siddharth

== Setting memory limits ==
You can set the hard limit on a particular context, effective immediately, with this command:

<pre>
/usr/sbin/vlimit -c <xid> --<resource> <value>
</pre>

<code><xid></code> is the context ID of the guest, which you can determine with the <code>/usr/sbin/vserver-stat</code> command.

For example, if you want to change the '''rss''' hard limit for the vserver with <code><xid></code> 49000, and limit it to 10,000 pages (40 MB), you could use this command:

<pre>
/usr/sbin/vlimit -c 49000 --rss 10000
</pre>

You can change the soft limit instead by adding the <code>-S</code> parameter.

Changes made with the vlimit command are effective only until the vserver is stopped. To make permanent changes, write the value to this file:

<pre>
/etc/vservers/<name>/rlimits/<resource>.hard
</pre>

To set a soft limit, use the same file name with the <code>.soft</code> extension. The <code>rlimits</code> directory is not created by default, so you may need to create it yourself.

If you omit the suffix after the <code>/etc/vservers/<name>/rlimits/rss</code> file, the value will be set for both, the hard and soft limit.

Changes to these files take effect only when the vserver is started. To make immediate and permanent changes to a running vserver, you need to run <code>vlimit</code> '''and''' update the rlimits file.

The safest setting, to prevent any guest from interfering with any other, is to set the total of all RSS hard limits (across all running guests) to be less than the total virtual memory (RAM and swap) on the host. It should be sufficiently less to leave room for processes running on the host, and some disk cache, perhaps 100 MB.

However, this is very conservative, since it assumes the worst case where all guests are using the maximum amount of memory at one time. In practice, you can usually get away with contended resources, i.e. allowing guests to use more than this value.

== Displaying current memory limits ==
To display the currently active RSS limits for a vserver execute the following command:
<pre>
vlimit -c <xid> -a -d | grep RSS
</pre>

The above command will display a similar result, whereas the third value (5000) is the soft limit and the last reflects the current hard limit (10000).
<pre>
RSS N/A 5000 10000
</pre>

== Display memory limits within a vserver ==
Normally the <code>top</code> and <code>free</code> command will display the RAM and Swap usage of the host while invoked within a vserver. To change this behavior, add the <code>VIRT_MEM</code> [[Capabilities_and_Flags#Context_flags_.28cflags.29|context flag]] to your vserver configuration:
<pre>
echo "VIRT_MEM" >> /etc/vservers/<name>/flags
</pre>

After a successful restart of the related vserver, the total available RAM will equal to the value of <code>rss.soft</code> while the difference of <code>rss.hard</code> - <code>rss.soft</code> will be displayed as swap space.

As an example, if you set the <code>rss.hard</code> limit to 10'000 pages and the <code>rss.soft</code> limit to 7'000 pages:
<pre>
vlimit -c <xid> --rss 10000
vlimit -c <xid> -S --rss 7000
vlimit -c <xid> -a -d | grep RSS
</pre>
<pre>
RSS N/A 7000 10000
</pre>

<code>free</code> will report 28'000 KB (7'000 pages) of total memory and 12'000 KB (10'000 - 7'000 pages) of Swap space (assuming that one page is 4 KB):
<pre>
free -k
</pre>
<pre>
total used free shared buffers cached
Mem: 28000 4396 23604 0 0 0
-/+ buffers/cache: 4396 23604
Swap: 12000 0 12000
</pre>

'''Note:''' According to Herbert, the kernel won't use any ''real'' swap as soon as the <code>rss.soft</code> limit has been reached. Swapping will be done on the host level, not per vserver (see [http://list.linux-vserver.org/archive?mss:653:200801:iiagcodpghkhehndkgjg "free" command inside vserver]).

User:Gl2k10

2010-04-22T07:54:23Z

Gl2k10: Created page with 'i just started!'

i just started!

Installation on Debian

2010-04-21T11:25:53Z

Gl2k10: /* Install util-vserver by source */

This guide is written against Debian Etch (4.0) and works on Lenny (5.0) as well. Both releases include kernel '''linux-image-vserver-686''', so no manual patching is needed. Hence, Installation on Debian Etch/Lenny is pretty easy and straightforward.

If you need to compile your own kernel, you need to apply the vserver-version.patch. [http://www.kwu.hu/vserver.txt Details at 2007/May/04]

In lenny and etch the tools are for the 2.2 version of vservers, you can find on beng repository packages for the 2.3 version of util-vserver until it is integrated in debian. See
* [[util-vserver:Devdebianpackage]] - Info about debian v2.3 package from the community
* explanation on how to use this repository from: http://kernels.bristolwireless.net/ How to use the Debian Repository

== Packages installation ==
The packages required by Linux-VServer are:
* '''linux-image-vserver-686''' - This is the current kernel, use '''linux-image-vserver-amd64''' on 64-bit systems, you can still create 32-bit guests
* '''util-vserver''' - These are the utilities used to administer the guests
* '''ssh''' - This is probably already installed, but just in case it isn't

All the packages you need can be obtained via
<pre>aptitude install linux-image-vserver-686 util-vserver ssh</pre>
so run this as ''root'' and reboot.
To check out wherever everything went fine you may run
<pre>uname -r</pre>
and check that kernel version contains '''vserver''', e.g. '''2.6.18-4-vserver-686'''. That's it.

Now that the host system is ready, you can proceed with [[Building Guest Systems|building guests]].

=== Install util-vserver by source ===
Occasionally, Debian's util-vserver package can be too old. So, we'll need to compile from [http://people.linux-vserver.org/~dhozac/t/uv-testing/ source].

First, install the required packages for util-vserver to compile.
<pre>apt-get install vlan dietlibc-dev pkg-config libnss3-dev</pre>

Then, we configure util-vserver
<pre>./configure --prefix=/usr --enable-release --mandir=/usr/share/man \
--infodir=/usr/share/info --sysconfdir=/etc --enable-dietlibc \
--localstatedir=/var --with-vrootdir=/home</pre>
Note: You should change ''--with-vrootdir'' accordingly

Finally, we run make to finalise the installation
<pre>make && make install install-distribution</pre>

Running ''vserver-info'' will show you that the proper util-vserver is installed. :)

Debian likes to be funny, so we need to enable the following,
* echo /usr/lib/util-vserver/vshelper >| /proc/sys/kernel/vshelper
* echo kernel.vshelper = /usr/lib/util-vserver/vshelper >> /etc/sysctl.conf
* update-rc.d vprocunhide defaults
* update-rc.d vservers-default defaults

== Versions ==

Debian already contains vservers kernels, so no manual patching and compiling is needed.

{|class="wikitablenowrap"
!Debian release
!Kernel version
!VServer version
|-
| Etch
| 2.6.18+6
| 2.0.2.2-rc9
|-
| Lenny
| 2.6.26+17
| 2.3.0.35
|-
|}

Unofficial debian vserver packages : http://zbla.net/debian/
'''WARNING : i386 packets are compiled for 64bits !'''

<pre>
apt source line: deb http://zbla.net/debian/ ./
</pre>

== Issues with the current 2.6.26 Kernel ==

=== Hard CPU scheduling ===

This will not work in the Debian 'Lenny' Kernel, the patch used simply does not contain any of this functionality.

=== Problems due to Xattrs ===

There are two sets of issues within the Lenny kernel caused by the change in value of the Xattrs (extended attributes) applied to file in Vserver setups. The patch used in Debian Lenny uses Xattr flags which are set in positions which differ from the flags set by Debian kernels as well as most of the mainline Vserver patches. This result is that Xattrs of files in a non lenny system appear to have completely different flags in Lenny and vice versa. Since these flags are crucial to vserver hashification and chroot security, they can have devastating effects on Vserver guests and on host system security. If you have recently moved to or away from the stock Lenny Vserver kernel, have look at the symptoms below to see if any match your experiences, and apply the fixes/use another kernel as you see fit.

As of writing these issue has not been corrected within the Debian archive. These fixes must be applied whenever moving vserver guest '''from''' or '''to''' the Debian 'Lenny's vserver kernel. For more details and a more concise explanation see [http://irc.13thfloor.at/LOG/2009-05/LOG_2009-05-12.txt Bertls IRC explanation ].

==== Chroot Security Problems ====

Linux-Vserver uses file Xattrs to protect guest superusers from being able to view files above their root, preventing access to host file. This creates issues for anyone who:

* has created a guest with a Debian 2.6.26-*-vserver kernel and wishes to use it with another kernel.
* has created a guest with a different kernel and wishes to use it on a Debian 2.6.26-*-vserver kernel based host.

In effect, the barrier normally in place for guest servers is not recognised by the kernel (the chroot problem) in the situation above and/or immutable links will not function correctly (the unification problem)failing to break when overwritten) in a unified guest setup. Symptoms suffered may include:

* the possibility of vserver guest processes escaping their chroots and accessing other parts of the filesystem
* guest not starting

To fix the barrier flags for a current kernel, see [[Secure_chroot_Barrier#Solution:_Secure_Barrier | these instructions]]. Note that on some setups a barrier flags will appear on all directories under the guest hierarchy, and need to be unset in order to allow the servers to run. Use showattr to reveal the state of play for your guests and fix appropriately.

==== Unification Problems ====

There is a discrepancy between the immutable-unlink flag used for file unification, the process used in vhashify. This creates considerable issues for anyone who:

* has unified guests with a Debian 2.6.26-*-vserver kernel and wishes to use them with another kernel.
* has unified guests with a different kernel and wishes to then it on a Debian 2.6.26-*-vserver kernel based host.

Symptoms suffered may include:

* file that cannot be deleted
* any process involving the writing of files in guests not working
* files not being unlinked on write

To fix the problem each file must be unlinked then the unification re-applied, or one could try this script submitted to [http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=508523 bugs.debian.org].

=== /proc/mounts issue ===

The vserver's /proc/mounts let appear the vserver path on the host. lsof (for example) is able to print it.

=== "Ghosts" guests ===

==== Issue ====
Sometimes a guests loose it's name in vserver-stats and is acting like a zombie. It's impossible to restart or kill it. Stopping all the guests with the util-vserver init.d script doesn't solve the issue. vkill --xid $CTX doesn't either.

==== Fix ====
echo 50 > /var/run/vservers/$vserver

Seems to fix the issue

== References ==
* Linux-VServer HOWTO by Daniel15: http://howtoforge.com/linux_vserver_debian_etch

Disk Limits and Quota

2010-04-21T07:45:36Z

Gl2k10: /* Activating disk limits */

== Introduction ==
We call ''Disk Limit'' a disk space limit in a filesystem for one context (i.e. VServer). If you are interested in using user/group quotas inside a VServer, this is not the right place, look at the [[Quotas]] page.

Of course, you can also use a dedicated partition (or logical volume) for each guest, which is another way to limit disk space usage per guest.

== Activating disk limits ==

For VServer version 2.0, please read http://oldwiki.linux-vserver.org/Disk+Limits

For VServer 2.2+, the procedure is nearly the same, only the mount option has changed:

# Activate the XID tagging on the filesystem hosting the VServer, using the ''tag'' mount option. You cannot use the ''remount'' option to add the tag on an already mounted filesystem. '''Be warn''', it is not advised to use it on the root filesystem! In the next example, we use the <tt>/dev/sdb1</tt> partition to host all the guests, under the <tt>/vservers</tt> directory <pre>mount -o tag /dev/sdb1 /vservers</pre>
# Consider modifying your <tt>/etc/fstab</tt> file if necessary, to make the mount option persist across reboots
# Tag the files belonging to the VServer with its XID <pre>chxid -URx -c vs_name /vservers/vs_name </pre>
# Write the limits under the <tt>/etc/vservers/vs_name/dlimits</tt> directory :
## Create this directory if necesary
## Create one sub-directory for each filesystem (in most cases, you should have only one), using any identifier (for example <tt>root</tt>)
## Under <tt>/etc/vservers/vs_name/dlimits/root/</tt> create the following files :
### <tt>directory</tt> : contains the directory to which the limit should be applied (for example <tt>/vservers/vs_name</tt>)
### <tt>inodes_total</tt> : the amount of inodes this vserver should be limited to
### <tt>reserved</tt> : how much space (percentage-wise) should be reserved for the root user
### <tt>space_total</tt> : the amount of space this vserver should be limited to (measured in blocks of 1024 bytes)

== Controling disk limits ==

After starting your VServer, you can control the actual limit using the ''vdlimit'' command (in this example, 300 is the context number associated with ''vs_name'') :
# vdlimit --xid vs_name /vservers
300 /vservers
space_used=154604
space_total=204800
inodes_used=7834
inodes_total=100000
reserved=5

You can also use the ''vdu'' command to count space or inodes used by a specific guest
# vdu --xid vs_name --space /vservers
/vservers 154604
# vdu --xid vs_name --inodes /vservers
/vservers 7834

If you just want to update disk usage of a guest activating disk limits, then you can use this script.
It will count the usage of all files and update the limits:
<pre>
#/bin/sh
_dir=/var/lib/vservers
if [ ! -d $_dir/$1 ]; then
echo no such vserver
exit 1;
fi;
chxid -URx -c $1 $_dir/$1
vdlimit --xid $1 -s space_used=$(vdu --xid $1 --space $_dir/$1 | cut -d" " -f2) $_dir/$1
vdlimit --xid $1 -s inodes_used=$(vdu --xid $1 --inode $_dir/$1 | cut -d" " -f2) $_dir/$1
</pre>

== Removing disk limits ==
To remove disk limits, remove the <tt>/etc/vservers/vs_name/dlimits/root</tt> directory '''and''' remove the limits with the following command
# vdlimit --xid vs_name --remove /vservers

== More information ==
* See also ''dlimits'' on the util-vserver homepage http://www.nongnu.org/util-vserver/doc/conf/configuration.html
* For information about limiting diskspace with XFS, please see the [[XFS]] page.

Disk Limits and Quota

2010-04-21T07:45:00Z

Gl2k10: /* Activating disk limits */

== Introduction ==
We call ''Disk Limit'' a disk space limit in a filesystem for one context (i.e. VServer). If you are interested in using user/group quotas inside a VServer, this is not the right place, look at the [[Quotas]] page.

Of course, you can also use a dedicated partition (or logical volume) for each guest, which is another way to limit disk space usage per guest.

== Activating disk limits ==

For VServer version 2.0, please read http://oldwiki.linux-vserver.org/Disk+Limits

For VServer 2.2+, the procedure is nearly the same, only the mount option has changed:

# Activate the XID tagging on the filesystem hosting the VServer, using the ''tag'' mount option. You cannot use the ''remount'' option to add the tag on an already mounted filesystem. '''Be warn''', it is not advised to use it on the root filesystem! In the next example, we use the <tt>/dev/sdb1</tt> partition to host all the guests, under the <tt>/vservers</tt> directory <pre>mount -o tag /dev/sdb1 /vservers</pre>
# Consider modifying your <tt>/etc/fstab</tt> file if necessary, to make the mount option persist across reboots
# Tag the files belonging to the VServer with its XID <pre>chxid -URx -c vs_name /vservers/vs_name </pre>
# Write the limits under the <tt>/etc/vservers/vs_name/dlimits</tt> directory :
## Create this directory if necesary
## Create one sub-directory for each filesystem (in most cases, you should have only one), using any identifier (for example <tt>root</tt>)
## Under <tt>/etc/vservers/vs_name/dlimits/root/</tt> create the following files :
### <tt>directory</tt> : contains the directory to which the limit should be applied (for example /vservers/vs_name)
### <tt>inodes_total</tt> : the amount of inodes this vserver should be limited to
### <tt>reserved</tt> : how much space (percentage-wise) should be reserved for the root user
### <tt>space_total</tt> : the amount of space this vserver should be limited to (measured in blocks of 1024 bytes)

== Controling disk limits ==

After starting your VServer, you can control the actual limit using the ''vdlimit'' command (in this example, 300 is the context number associated with ''vs_name'') :
# vdlimit --xid vs_name /vservers
300 /vservers
space_used=154604
space_total=204800
inodes_used=7834
inodes_total=100000
reserved=5

You can also use the ''vdu'' command to count space or inodes used by a specific guest
# vdu --xid vs_name --space /vservers
/vservers 154604
# vdu --xid vs_name --inodes /vservers
/vservers 7834

If you just want to update disk usage of a guest activating disk limits, then you can use this script.
It will count the usage of all files and update the limits:
<pre>
#/bin/sh
_dir=/var/lib/vservers
if [ ! -d $_dir/$1 ]; then
echo no such vserver
exit 1;
fi;
chxid -URx -c $1 $_dir/$1
vdlimit --xid $1 -s space_used=$(vdu --xid $1 --space $_dir/$1 | cut -d" " -f2) $_dir/$1
vdlimit --xid $1 -s inodes_used=$(vdu --xid $1 --inode $_dir/$1 | cut -d" " -f2) $_dir/$1
</pre>

== Removing disk limits ==
To remove disk limits, remove the <tt>/etc/vservers/vs_name/dlimits/root</tt> directory '''and''' remove the limits with the following command
# vdlimit --xid vs_name --remove /vservers

== More information ==
* See also ''dlimits'' on the util-vserver homepage http://www.nongnu.org/util-vserver/doc/conf/configuration.html
* For information about limiting diskspace with XFS, please see the [[XFS]] page.