Commit c6583a39 authored by Jon Bernard's avatar Jon Bernard

Imported Upstream version 0.34

This diff is collapsed.
Installation Instructions
Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,
2006, 2007 Free Software Foundation, Inc.
This file is free documentation; the Free Software Foundation gives
unlimited permission to copy, distribute and modify it.
Basic Installation
Briefly, the shell commands `./configure; make; make install' should
configure, build, and install this package. The following
more-detailed instructions are generic; see the `README' file for
instructions specific to this package.
The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation. It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions. Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, and a
file `config.log' containing compiler output (useful mainly for
debugging `configure').
It can also use an optional file (typically called `config.cache'
and enabled with `--cache-file=config.cache' or simply `-C') that saves
the results of its tests to speed up reconfiguring. Caching is
disabled by default to prevent problems with accidental use of stale
cache files.
If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release. If you are using the cache, and at
some point `config.cache' contains results you don't want to keep, you
may remove or edit it.
The file `' (or `') is used to create
`configure' by a program called `autoconf'. You need `' if
you want to change it or regenerate `configure' using a newer version
of `autoconf'.
The simplest way to compile this package is:
1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system.
Running `configure' might take a while. While running, it prints
some messages telling which features it is checking for.
2. Type `make' to compile the package.
3. Optionally, type `make check' to run any self-tests that come with
the package.
4. Type `make install' to install the programs and any data files and
5. You can remove the program binaries and object files from the
source code directory by typing `make clean'. To also remove the
files that `configure' created (so you can compile the package for
a different kind of computer), type `make distclean'. There is
also a `make maintainer-clean' target, but that is intended mainly
for the package's developers. If you use it, you may have to get
all sorts of other programs in order to regenerate files that came
with the distribution.
6. Often, you can also type `make uninstall' to remove the installed
files again.
Compilers and Options
Some systems require unusual options for compilation or linking that the
`configure' script does not know about. Run `./configure --help' for
details on some of the pertinent environment variables.
You can give `configure' initial values for configuration parameters
by setting variables in the command line or in the environment. Here
is an example:
./configure CC=c99 CFLAGS=-g LIBS=-lposix
*Note Defining Variables::, for more details.
Compiling For Multiple Architectures
You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory. To do this, you can use GNU `make'. `cd' to the
directory where you want the object files and executables to go and run
the `configure' script. `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'.
With a non-GNU `make', it is safer to compile the package for one
architecture at a time in the source code directory. After you have
installed the package for one architecture, use `make distclean' before
reconfiguring for another architecture.
Installation Names
By default, `make install' installs the package's commands under
`/usr/local/bin', include files under `/usr/local/include', etc. You
can specify an installation prefix other than `/usr/local' by giving
`configure' the option `--prefix=PREFIX'.
You can specify separate installation prefixes for
architecture-specific files and architecture-independent files. If you
pass the option `--exec-prefix=PREFIX' to `configure', the package uses
PREFIX as the prefix for installing programs and libraries.
Documentation and other data files still use the regular prefix.
In addition, if you use an unusual directory layout you can give
options like `--bindir=DIR' to specify different values for particular
kinds of files. Run `configure --help' for a list of the directories
you can set and what kinds of files go in them.
If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
Optional Features
Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System). The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.
For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.
Specifying the System Type
There may be some features `configure' cannot figure out automatically,
but needs to determine by the type of machine the package will run on.
Usually, assuming the package is built to be run on the _same_
architectures, `configure' can figure that out, but if it prints a
message saying it cannot guess the machine type, give it the
`--build=TYPE' option. TYPE can either be a short name for the system
type, such as `sun4', or a canonical name which has the form:
where SYSTEM can have one of these forms:
See the file `config.sub' for the possible values of each field. If
`config.sub' isn't included in this package, then this package doesn't
need to know the machine type.
If you are _building_ compiler tools for cross-compiling, you should
use the option `--target=TYPE' to select the type of system they will
produce code for.
If you want to _use_ a cross compiler, that generates code for a
platform different from the build platform, you should specify the
"host" platform (i.e., that on which the generated programs will
eventually be run) with `--host=TYPE'.
Sharing Defaults
If you want to set default values for `configure' scripts to share, you
can create a site shell script called `' that gives default
values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/' if it exists, then
`PREFIX/etc/' if it exists. Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.
Defining Variables
Variables not defined in a site shell script can be set in the
environment passed to `configure'. However, some packages may run
configure again during the build, and the customized values of these
variables may be lost. In order to avoid this problem, you should set
them in the `configure' command line, using `VAR=value'. For example:
./configure CC=/usr/local2/bin/gcc
causes the specified `gcc' to be used as the C compiler (unless it is
overridden in the site shell script).
Unfortunately, this technique does not work for `CONFIG_SHELL' due to
an Autoconf bug. Until the bug is fixed you can use this workaround:
CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash
`configure' Invocation
`configure' recognizes the following options to control how it operates.
Print a summary of the options to `configure', and exit.
Print the version of Autoconf used to generate the `configure'
script, and exit.
Enable the cache: use and save the results of the tests in FILE,
traditionally `config.cache'. FILE defaults to `/dev/null' to
disable caching.
Alias for `--cache-file=config.cache'.
Do not print messages saying which checks are being made. To
suppress all normal output, redirect it to `/dev/null' (any error
messages will still be shown).
Look for the package's source code in directory DIR. Usually
`configure' can determine that directory automatically.
`configure' also accepts some other, not widely useful, options. Run
`configure --help' for more details.
SUBDIRS = dist doc include samples scripts src tests
EXTRA_DIST = README_daemon libcgroup.doxyfile
This diff is collapsed.
Please follow the following steps (TODO: Automate these steps)
0. Update package version in AC_INIT (
1. Tag the current sources in trunk to the specified version
1. Run autoreconf -i to generate the configure file again
(on changing any of the automake files)
2. Run configure with --prefix=<desired prefix> default is /usr/local
3. This step should generate
a. Makefile
b. libcgroup.spec
4. To build the RPM, carry out the following steps (obsolete, will be revised)
a. Create a tag with the same version as specified in step 0
a. Run mktarball
b. copy tarball and spec file to correct place in the RPMBUILD
c. Build an rpm of the spec file
After cgroup system has taken shape, its time to have some basic tools
in user space which can enable a user to use the resource management
functionality effictively.
One of the needed functionality is rule based placement of a task. In general,
there can be either uid or gid or exec based rules. Admin/root will
control/enforce uid/gid based rules and exec based rules can be defined by
user in a config file residing in user's home dir and fully controlled by user.
uid/gid based rules will be defined in /etc/cgrules.conf config file and
this file will be managed by root.
Basic idea is that to begin with provide facility to implement rules
based on uid and gid. So a hierarchy might look like as follows.
| |
gid1 gid2
| |
uid1 uid2
| |
proj1 proj2
Admin will write rules to control the resources among users. Then users
can manage their own cgroup at their own (proj1 and proj2) and control
the resources as they want.
Following are the few methods using which tasks can be placed in right
- Use pam_cgroup PAM plugin which will make sure users are placed in right
cgroup at login time and any tasks launch after login, will continue to run
in user's cgroup.
- Use command line tool "cgexec" to launch the task in right cgroup.
- Modify the program and use libcgroup provided APIs for placing a task
in right cgroup before doing exec().
- Use "cgclassify" tool to classify a already running task.
- May be, a user space daemon can also be implemented which will listen to
kernel events and place the task in right group based on the rules.
This method involves few concerns.
- Reliability of netlink socket. Messages can be dropped.
- Change the netlink with a cgroup controller which exports the
- Delay incurred since the event took place and task was actually placed
in right cgroup.
- daemon will interfere with container's tasks which is not desired.
Section 1:
To use "cgexec" to place the task in right cgroup.
- make cgexec
- Run a task using cgexec. Following is the cgexec syntax.
cgexec [-g <list of controllers>:<relative path to cgroup>] command [arguments]
Note: Cgroup controllers and path are optional. If user does not know the
right cgroup, cgexec will automatically place the task in right
cgroup based on /etc/cgrules.conf
cgexec -g *:test1 ls
cgexec -g cpu,memory:test1 ls -l
cgexec -g cpu,memory:test1 -g swap:test2 ls -l
Section 2
To use "cgclassify" to place task in right cgroup.
- make cgclassify
- Pick a task's pid to be classified, and run
cgclassify <list of pids>
cgclassify 2140 4325
Note: After classification check out whether tasks 2140 and 4325
are in the right cgroup or not (Based on rules in /etc/cgrules.conf)
Section 3:
To use a pam plugin which will automatically place the task in right
cgroup upon login.
- Build
- Copy to /lib/security/
- Edit /etc/pam.d/su to make use of session module upon
execution of su.
Add following line at the end of /etc/pam.d/su file
session optional
- Now launch a shell for a user "xyz" using su and the resulting shell
should be running in the cgroup designated for the user as specified
by cgrules.conf
ex. "su test1"
Try similar things with other services like sshd.
Note: moves the service providing process in the right cgroup
and not the process which will be launched later. Due to parent child
relationship, yet to be forked/execed process will launch in right
Ex. Lets say user root does "su test1". In this case process "su" is the
one providing service (launching a shell) for user "test1".
will move process "su" to the user "test1"'s cgroup (Decided by the uid
and gid of "test1"). Now once su forks/execs a shell for user test1,
final shell is effectively running in the cgroup it should have been
running based on /etc/cgrules.conf for user test1.
Section 4:
To use cgrulesengd which will move a task to right cgroup based on
rules in /etc/cgrules.conf do following.
- build and install latest
- build cgrulesengd
make cgrulesengd
- specify some uid/gid based rules in /etc/cgrules.conf
- Mount some controllers and create an hierarchy of cgroups (matching
your rules).
- Run cgrulesengd.
- ./cgrulesengd
- Launch some task or login as a user to the sytem. Daemon should automatically
place the task in right cgroup.
Q. Why admin should not control "exec" based rules.
A. Unix file system provides access control only based on uid/gid. So
even if admin starts putting tasks based on uid/gid, it can't enforce
it. For example, consider following scenario.
Lets say an admin creates following cgroup hierarchy.
| |
database browser
| | | |
user1 user2 user1 user2
Now admin wants to run all the database jobs under /container/database/
and all the firefox jobs under /container/browser/. Based on which user
launched it, jobs should go in either user1 or user2 dir.
Now assume that database subdir has to more cpu resources as compared to
browser subdir. Then a user, say user2, can always move his browser job
also to /container/database/user2 dir to get more cpu resources and
admin will not be able to control that.
[Note: user2 will control what tasks can be added in /container/database/user2
and will contol what further subdirs can be created under user2 dir. Root
should not restrict the control to root only for practical purposes. Its
something like that till /container/databse, admin controls the resources
and below that how resources are further subdivided among various projects
should be controlled by respective user].
In the light of above, it seems to make more sense that admin should enforce
rules only based on uid and gid. Probably later we can have a per user exec
based rules config file (~/.cgrules.conf), which can be parsed by cgrulesd
and then jobs launched by user will be placed in right cgroup based on
combination of rules in /etc/cgrules.conf and ~/cgrules.conf.
The CGroup Rules Engine Daemon is a tool that will automatically place tasks
into the correct cgroup based on UID/GID events from the kernel. It will not
automatically classify tasks that are already running, but it will classify
any new tasks, and any tasks which change their UID/GID. Note that we use
the euid and egid, not the ruid and rgid.
Unlike other tools, cgrulesengd caches the rules configuration in a data
structure (it's actually just a FIFO linked list) so that it doesn't need to
parse the configuration file more than once. This should be much faster than
parsing the rules for each UID/GID event. Eventually, this caching logic
should be part of libcgroup, so that any other program can take advantage of it
(and so that all programs are using the same table). The configuration can
be reloaded without stopping the daemon (more information below).
A daemon is easy to use, and allows an administrator to ensure that all tasks
are classified into the correct cgroups without constantly monitoring the
system. The daemon is transparent to the users, and does not require any
modifications to existing userspace programs. Finally, the daemon can be
started and stopped at any time, including at boot time with other services.
Thus, sytem administrators can decide not to use the daemon if they choose.
Most importantly, some programs create new users and/or run scripts,
threads, etc. as those users using suexec(). This call does not go through
PAM, so these scripts would continue running in the same cgroup as the parent
program. This behavior is likely not ideal, and the daemon would solve this
Apache does this. Apache creates a user called 'apache' and uses setuid() to
launch tasks as that user. This does not go through PAM, so without a daemon,
these tasks would continue to run in the 'root' cgroup rather than in the
'apache' or 'webserver' cgroup. The daemon fixes this problem by catching the
setuid() call and moving the tasks into the correct cgroup.
We would ask Apache to modify their software to interface with libcgroup, but
this solution is less than optimal because a lot of userspace software would
have to be changed, and some authors might intentionally not interact with
libcgroup, which could create an exploit. The daemon is a simple, transparent
The daemon can be used as a service with the cgred script, which is shipped
as scripts/init.d/cgred. This script should be installed as /etc/init.d/cgred
and used like any other service. To start the daemon,
/etc/init.d/cgred start
To stop it,
/etc/init.d/cgred stop
The restart (stop,start), condrestart (same as restart, but only if the daemon
was already started), and status (print whether the daemon is started or
stopped) commands are also supported. An additional command, "reload", allows
you to reload the configuration file without stopping the daemon.
/etc/init.d/cgred reload
The cgred script automatically loads configuration from /etc/sysconfig/cgred.conf,
which is shipped as samples/cgred.conf. See that file for more information.
If you choose not to run the daemon as a service, the following options are
currently supported:
--nodaemon Do not run as a daemon
--nolog Write log output to stdout instead of a log file
--config [FILE] Read rules configuration from FILE instead of
You can ask the daemon to reload the configuration by sending it SIGUSR2. The
easiest way to do this is with the 'kill' command:
kill -s SIGUSR2 [PID]
The program setuid (found in tests/setuid.c) can help you test the daemon. By
default, this program attempts to change its UID to root and then idles until
you kill it. You can change the default behavior to use a different UID, or
you can uncomment the second block of code to instead attempt to change the
In order to make sure that everything works, I used the following rules:
sjo cpu default
cgtest cpu cgtest
% memory default
@cgroup cpu,memory cgtest
peter cpu test1
% memory test2
@root * default
* * default
The users 'sjo' and 'cgtest' were normal users. 'peter' is not a user on the
system. The group 'cgroup' is a group containing sjo,root,cgtest as members,
and the group 'root' contains only root. The cgroups 'default' and 'cgtest'
exist, while 'test1' and 'test2' do not. Currently, the daemon does not check
for the existance of 'test1', though this would be easier to do once the
parsing and caching logic is moved into libcgroup.
I ran the following tests, all of which were successful:
- set UID to sjo (should move cpu controller into default)
- set UID to root (should move cpu,memory controllers into cgtest)
- set UID to cgtest (should move cpu controller into cgtest, memory
controller into default)
- set GID to root (should move all controllers into default)
- set GID to cgroup (should move cpu, memory into cgtest)
- set GID to users (should move all controllers into default)
The parsing logic will skip the 'peter' rule as well as its multi-line
components (in this case "% memory test2"), because the user does not exist.
This should work for group rules, too. Attempting to setuid() or setgid() to a
user/group that doesn't exist will just return an error and not generate a
kernel event of the PROC_EVENT_UID or PROC_EVENT_GID type, so the daemon won't
do anything for it.
- Netlink can be unreliable, and the daemon might miss an event if the buffer
is full. One possible solution is to have one or two files that the kernel
can queue UID/GID changes in, and have the daemon read those files whenever
they are updated. From testing, this does not actually appear to be a real
problem, but it could become one with faster machines.
- The daemon does not care for namespaces at all, which can cause conflicts
with containers. If a user places his tasks into exec-based cgroups (such
as 'network' and 'development'), the daemon will not realize this and will
simply place them into the user's cgroup (so, sjo/ instead of sjo/network/).
- Updated documentation, because it was very old and incorrect.
- Reverted the changes to cgexec and cgclassify.
- New API function: cgroup_change_cgroup_uid_gid_flags().
- Deprecated cgroup_change_cgroup_uid_gid().
- Rewrote some of the rule matching and execution logic in api.c to be
faster, and easier to read.
- Changes all negative return values to positive values. As a side effect,
cgroup_parse_rules() now returns -1 when we get a match and we are using
non-cached rules.
- Flags are now enumerated (cgflags), instead of #defines.
- Moved the event-handling logic back into the daemon, where it should be.
- Changed cgroup_parse_rules() to work with cached rules or non-cached rules.
The other parsing function is no longer needed, and should be deprecated.
- Non-cached rules now work with the same structs as cached rules.
- Modified cgroup_change_cgroup_uid_gid() with a new 'flags' parameter.
Currently, the only flag is "CGROUP_FUSECACHE" to use the cached rules logic
(or not).
- Added cgroup_rules_loaded() boolean, to check whether the cached rules have
been loaded yet, and cgroup_init_rules_cache() to load them.
- Modified cgexec and cgclassify to work with the new
- Moved parsing and caching logic into libcgroup.
- Added locking mechanisms around the list of rules.
- Cleaned up #includes in cgrulesegnd.[h,c].
- Added notification if netlink receive queue overflows.
- Added logic to catch SIGINT in addition to SIGTERM.
- New API functions:
- cgroup_free_rule(struct cgroup_rule*)
- cgroup_free_rule_list(struct cgroup_rule_list*)
- cgroup_parse_rules(void)
- cgroup_print_rules_config(FILE*)
- cgroup_reload_cached_rules(void)
- cgroup_change_cgroup_event(struct proc_event*, int, FILE*)
- Wrote new parsing logic, which is cleaner and simpler.
- Added cgred script to enable using the daemon as a service.
- Wrote caching logic to cache rules table.
- Added the ability to force a reload of the rules table with SIGUSR2 signal.
- Added two structures to libcgroup: cgre_rule and cgre_rules_list
- New API function: cgroup_reload_cached_rules, which reloads the rules table.
- Added logging capabilities (default log is /root/cgrulesengd.conf)
- Find a way to replace Netlink, or at least clean up that code.
- Find a solution to the namespace problem.
This diff is collapsed.
This diff is collapsed.
/* Generated from by autoheader. */
/* Define to enable extra debugging output. */
/* Cgrulesengd socket path */
/* Define to 1 if your system has a working `chown' function. */
/* Define to 1 if you have the <dirent.h> header file, and it defines `DIR'.
/* Define to 1 if you have the <dlfcn.h> header file. */
/* Define to 1 if you have the `getmntent' function. */
/* Define to 1 if you have the `hasmntopt' function. */
/* Define to 1 if you have the <inttypes.h> header file. */
/* Define to 1 if you have the `pam' library (-lpam). */
/* Define to 1 if you have the <limits.h> header file. */
/* Define to 1 if your system has a GNU libc compatible `malloc' function, and
to 0 otherwise. */
/* Define to 1 if you have the <memory.h> header file. */
/* Define to 1 if you have the `memset' function. */
/* Define to 1 if you have the `mkdir' function. */
/* Define to 1 if you have the <mntent.h> header file. */
/* Define to 1 if you have the <ndir.h> header file, and it defines `DIR'. */
#undef HAVE_NDIR_H
/* Define to 1 if your system has a GNU libc compatible `realloc' function,
and to 0 otherwise. */
/* Define to 1 if you have the `rmdir' function. */
/* Define to 1 if you have the <security/pam_ext.h> header file. */
/* Define to 1 if you have the <security/pam_modules.h> header file. */
/* Define to 1 if you have the <security/pam_modutil.h> header file. */
/* Define to 1 if `stat' has the bug that it succeeds when given the
zero-length file name argument. */
/* Define to 1 if stdbool.h conforms to C99. */
/* Define to 1 if you have the <stdint.h> header file. */
/* Define to 1 if you have the <stdlib.h> header file. */
/* Define to 1 if you have the `strdup' function. */
/* Define to 1 if you have the <strings.h> header file. */
/* Define to 1 if you have the <string.h> header file. */
/* Define to 1 if you have the <sys/dir.h> header file, and it defines `DIR'.
/* Define to 1 if you have the <sys/mount.h> header file. */
/* Define to 1 if you have the <sys/ndir.h> header file, and it defines `DIR'.
/* Define to 1 if you have the <sys/stat.h> header file. */
/* Define to 1 if you have the <sys/types.h> header file. */
/* Define to 1 if you have the <unistd.h> header file. */
/* Define to 1 if the system has the type `_Bool'. */
#undef HAVE__BOOL
/* Define to 1 if `lstat' dereferences a symlink specified with a trailing
slash. */
/* Name of package */
#undef PACKAGE
/* Define to the address where bug reports for this package should be sent. */
/* Define to the full name of this package. */