pytalloc_guide.txt 10.7 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Using talloc in Samba4
======================

.. contents::

Jelmer Vernooij
August 2013

The most current version of this document is available at
   http://samba.org/ftp/unpacked/talloc/pytalloc_guide.txt

pytalloc is a small library that provides glue for wrapping
talloc-allocated objects from C in Python objects.

What is pytalloc, and what is it not?
-------------------------------------

pytalloc is merely a helper library - it provides a convenient base type object
for objects that wrap talloc-maintained memory in C. It won't write your
bindings for you but it will make it easier to write C bindings that involve
talloc, and take away some of the boiler plate.

23 24 25 26 27 28 29 30
Python 3
--------

pytalloc can be used with Python 3. Usage from Python extension remains
the same, but for the C utilities, the library to link to is tagged with
Python's PEP3149 ABI tag, for example "pytalloc.cpython34m".
To make a build for Python 3, configure with PYTHON=/usr/bin/python3.
.
31
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
32
pytalloc_Object / pytalloc_BaseObject
33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

This is the new base class that all Python objects that wrap talloc pointers
derive from. It is itself a subclass of the "Object" type that all objects
in Python derive from.

Note that you will almost never create objects of the pytalloc_Object type
itself, as they are just opaque pointers that can not be accessed from
Python. A common pattern is other objects that subclass pytalloc_Object and
rely on it for their memory management.

Each `pytalloc_Object` wraps two core of information - a talloc context
and a pointer. The pointer is the actual data that is wrapped. The talloc
context is used for memory management purposes only; when the wrapping Python object
goes away, it unlinks the talloc context. The talloc context pointer and the ptr
can (and often do) have the same value.

Each pytalloc_Object has a custom __repr__ implementation that
describes that it is a talloc object and the location of the
pointer it is wrapping. it also has a custom __cmp__/__eq__/__neq__ method that
compares the pointers the object is wrapping rather than the objects
themselves (since there can be multiple objects that wrap the same talloc
pointer).

56 57 58 59
It is preferred to use pytalloc_BaseObject as this implementation
exposes less in the C ABI and correctly supports pointers in C arrays
in the way needed by PIDL.

60 61 62
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyTypeObject *pytalloc_GetObjectType(void)

63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
Obtain a pointer to the PyTypeObject for `pytalloc_Object`. The
reference counter for the object will be NOT incremented, so the
caller MUST NOT decrement it when it no longer needs it (eg by using
`Py_DECREF`).

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyTypeObject *pytalloc_GetBaseObjectType(void)

Obtain a pointer to the PyTypeObject for `pytalloc_BaseObject`. The
reference counter for the object will be NOT incremented, so the
caller MUST NOT decrement it when it no longer needs it (eg by using
`Py_DECREF`).

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
int pytalloc_BaseObject_PyType_Ready(PyTypeObject *type);

Wrapper for PyType_Ready() that will set the correct values into
the PyTypeObject to create a BaseObject
81 82 83 84 85 86 87

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-
int pytalloc_Check(PyObject *)

Check whether a specific object is a talloc Object. Returns non-zero if it is
a pytalloc_Object and zero otherwise.

88 89 90 91 92 93
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-
int pytalloc_BaseObject_Check(PyObject *)

Check whether a specific object is a talloc BaseObject. Returns non-zero if it is
a pytalloc_BaseObject and zero otherwise.

94 95 96 97 98 99 100 101 102
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
int pytalloc_check_type(PyObject *py_obj, type)

Check if the object based on `pytalloc_*Object` py_obj. type should be a
C type, similar to a type passed to `talloc_get_type`.
This can be used as a check before using pytalloc_get_type()
or an alternative codepath. Returns non-zero if it is
an object of the expected type and zero otherwise.

103 104 105 106 107 108 109 110 111
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
type *pytalloc_get_type(PyObject *py_obj, type)

Retrieve the pointer from a `pytalloc_Object` py_obj. type should be a
C type, similar to a type passed to `talloc_get_type`.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
pytalloc_get_ptr(PyObject *py_obj)

112 113 114
Retrieve the pointer from a `pytalloc_Object` or `pytalloc_BaseObject`
py_obj. There is no type checking - use `pytalloc_get_type` if
possible.
115 116 117 118

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TALLOC_CTX *pytalloc_get_mem_ctx(PyObject *py_obj)

119
Retrieve the talloc context associated with a pytalloc_Object or pytalloc_BaseObject.
120 121 122 123 124

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_steal_ex(PyTypeObject *py_type, TALLOC_CTX *mem_ctx, void *ptr)

Create a new Python wrapping object for a talloc pointer and context, with
125 126 127
py_type as associated Python sub type object. This typically used
when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element.
`pytalloc_get_ptr()` can be used to get the pointer out of the object again.
128 129 130 131 132 133 134 135 136

This will *not* increment the reference counter for the talloc context,
so the caller should make sure such an increment has happened. When the Python
object goes away, it will unreference the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_steal(PyTypeObject *py_type, void *ptr)

Create a new Python wrapping object for a talloc pointer and context, with
137 138 139
py_type as associated Python sub type object. The pointer will also be used
as the talloc context. `pytalloc_get_type()` can be used to get
the pointer out of the object again.
140 141 142 143 144 145 146 147 148

This will *not* increment the reference counter for the talloc context,
so the caller should make sure such an increment has happened. When the Python
object goes away, it will unreference the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_reference_ex(PyTypeObject *py_type, TALLOC_CTX *mem_ctx, void *ptr)

Create a new Python wrapping object for a talloc pointer and context, with
149 150 151
py_type as associated Python sub type object. This typically used
when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element.
`pytalloc_get_ptr()` can be used to get the pointer out of the object again.
152 153 154 155 156 157 158 159

This will increment the reference counter for the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_reference(PyTypeObject *py_type, void *talloc_ptr)

Create a new Python wrapping object for a talloc pointer, with
py_type as associated Python sub type object. The pointer will also be used
160 161
as the talloc context. `pytalloc_get_type()` can be used to get
the pointer out of the object again.
162 163 164 165 166 167 168 169 170 171

This will increment the reference counter for the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_new(type, PyTypeObject *typeobj)

Create a new, empty pytalloc_Object with the specified Python type object. type
should be a C type, similar to talloc_new().

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216
PyObject *pytalloc_GenericObject_steal_ex(void *ptr)

Create a new Python wrapping object for a generic talloc pointer,
as sub type of `pytalloc_BaseObject`. This typically used
when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element.
`pytalloc_get_ptr()` can be used to get the pointer out of the object again.

This will *not* increment the reference counter for the talloc context,
so the caller should make sure such an increment has happened. When the Python
object goes away, it will unreference the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_GenericObject_steal(void *ptr)

Create a new Python wrapping object for a generic talloc pointer,
as sub type of `pytalloc_BaseObject`. The pointer will also be used
as the talloc context. `pytalloc_get_type()` can be used to get
the pointer out of the object again.

This will *not* increment the reference counter for the talloc context,
so the caller should make sure such an increment has happened. When the Python
object goes away, it will unreference the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_GenericObject_reference_ex(void *ptr)

Create a new Python wrapping object for a generic talloc pointer,
as sub type of `pytalloc_BaseObject`. This typically used
when `mem_ctx` and `ptr` differ, e.g. a pointer to an array element.
`pytalloc_get_ptr()` can be used to get the pointer out of the object again.

This will increment the reference counter for the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PyObject *pytalloc_GenericObject_reference(void *ptr)

Create a new Python wrapping object for a generic talloc pointer,
as sub type of `pytalloc_BaseObject`. The pointer will also be used
as the talloc context. `pytalloc_get_type()` can be used to get
the pointer out of the object again.

This will increment the reference counter for the talloc context.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
DEPRECATED! PyObject *pytalloc_CObject_FromTallocPtr(void *);
217 218 219 220 221 222

Create a new pytalloc_Object for an abitrary talloc-maintained C pointer. This will
use a generic VoidPtr Python type, which just provides an opaque object in
Python. The caller is responsible for incrementing the talloc reference count before calling
this function - it will dereference the talloc pointer when it is garbage collected.

223 224
This function is deprecated and only available on Python 2.
Use pytalloc_GenericObject_{reference,steal}[_ex]() instead.
225

226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245
Debug function for talloc in Python
-----------------------------------

The "talloc" module in Python provides a couple of functions that can be used
to debug issues with objects wrapped by pytalloc.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
report_full(obj?)

Print a full report on a specific object or on all allocated objects by Python.
Same behaviour as the `talloc_report_full()` function that is provided by
C talloc.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
enable_null_tracking()

This enables tracking of the NULL memory context without enabling leak
reporting on exit. Useful for when you want to do your own leak
reporting call via talloc_report_null_full().

246 247
This must be done in the top level script, not an imported module.

248 249 250 251 252
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
pytalloc_total_blocks(obj?)

Return the talloc block count for all allocated objects or a specific object if
specified.