This builds on the scitbx-tour, adding some trivial functions and the wrappers needed for joint Python/C++ array access.
This tour includes
The version array-family-tour-0_1 of voea contains the complete corresponding code.
Any module providing functionality for both C++ and Python consists of three parts: the main C++ files, the wrapper files, and a Python initializer for the wrapper.
Customization of the conversions are thus handled by, and separate from, the primary C++ code.
To accomodate a broad range of needs within C++, there are several ways of handling the same underlying array type. Briefly, the types involved are tiny, small, shared, and versa.
All types have an array interface following std::vector as much as possible, e.g.
.begin(), .end(), .size(), []
Because the interface follows std::vector, refer to documentation of the latter when in doubt.
No use of new for allocation of any of these -- they handle their own memory. All exceptions (size violations, memory problems, etc.) propagate to Python. [] is available for all types.
Example C++: tiny<double, 3> a; a[1]; a[3]; // Error, but no bounds check, no exceptions. Fast. tiny<double, 3> a(1,2,3); // Up to ~10 args.
Example: // C indexing, homogeneous data. // Initial 11 by 22 array versa<float, c_grid<2> > va(c_grid<2>(11,22)); // Same memory, now 2 by 5 array. va.resize( c_grid<2>(2,array) );
Using
namespace array = scitbx::af;
array::shared< type >
array::shared<double> range_inefficient(int m, int n) { array::shared<double> result; for(unsigned i = m; i < n; i++) { result.push_back( static_cast<double>(i) ); } return result; }
Here, the size of the array is known; by pre-allocating the space for the array, some work is saved:
array::shared<double> range_better(int m, int n) { array::shared<double> result; result.reserve(n - m); /* Avoid reallocation for known size. */ for (unsigned i = m; i < n; i++) { result.push_back( static_cast<double>(i) ); } return result; }
Internally, there is still much checking going on. For most arrays, random access is desirable, so the indexing operator [] can be used. Also, initializing memory can take time and is not needed here, so the fastest array::shared version is
array::shared<double> range_fast(int m, int n) { /* Use correct initial size, do not initialize memory. */ array::shared<double> result(n - m, array::init_functor_null<double>()); for (unsigned i = m; i < n; i++) { /* Direct memory access */ result[i] = static_cast<double>(i); } return result; }
The fastest access for heap-based arrays is offered via the ref and const_ref types.
The automatic types (shared, versa) still dereference memory on every access -- this is needed for cooperation between multiple "owners" of the array. For the most direct, C-like access, the range example can be done as follows
array::shared<double> range_fastest(int m, int n) { array::shared<double> result(n - m, array::init_functor_null<double>()); /* Local raw ref. */ array::ref<double> R = result.ref(); for (unsigned i = m; i < n; i++) { R[i] = static_cast<double>(i); } return result; }
The two C++/Python array conversion mechanisms are
C++ produced return values correspond 1-1 to Python types; values passed from Python to C++ are many-to-1.
In pass-by-value, conversion to Python is always as tuple, while conversion to C++ is done for tuples, lists, ranges, and iterators -- most sequence types.
The tiny and small types use pass-by-value. C++ instances of these types do not exist as such in Python. They are all converted to a tuple, losing their type. E.g., the C++
tiny<double, 3> a(1,2,3); // Up to ~10 args.
(1,2,3)
tiny<int, 3> a;
(int, int, int)
In the other direction, the C++ function signature determines the tuple/list conversion's target type. E.g., given
foo(tiny<int, 3> x)
tiny<int, 3>(1,2,3)
foo(small<float, 2> x)
small<float, 2>(2,3)
The trivial function
int tiny_to_cpp(array::tiny<int, 2> t) { return (t[0] + t[1]); }
def("tiny_to_cpp", voea::tiny_to_cpp );
BOOST_PYTHON_MODULE(voea_ext) { using namespace boost::python; ... def("tiny_to_cpp", voea::tiny_to_cpp ); ... }
On the Python side,
print voea.tiny_to_cpp( [1,2] )
The previous examples involve new functions using types already wrapped in the array toolbox.
When introducing new types, say
array::tiny<float, 2>
While the global boost::python registry can be queried to avoid overlaps/contradictions as needed, it is simpler and cleaner to centralize the wrapping of native types -- all modules will have consistent access to them.
Here are the steps needed to add a new type to the central registry.
As these wrappers are new, they can reside anywhere; for now, they are defined locally. Later, the main package owner (Ralf) can/may include them as needed for conflict resolution.
Following
$SCITBX_DIST/array_family/boost_python/flex_ext.cpp
#include <scitbx/boostpython/containerconversions.h>and
scitbx::boost_python::container_conversions::tuple_mapping_fixed_size< scitbx::af::tiny<float, 2> >();
This makes the type array::tiny<float, 2> available for use via Python's tuple-to-tiny conversion.
Overloaded operators and functions are very common. To continue the previous sample, we want the function tinytocpp to also handle floats. Thus, in addition to the previous
int tiny_to_cpp(array::tiny<int, 2> t)
float tiny_to_cpp(array::tiny<float, 2> t) { return (t[0] + t[1]); }
def("tiny_to_cpp", voea::tiny_to_cpp );
def("tiny_to_cpp", (float (&)(array::tiny<float, 2>) ) voea::tiny_to_cpp ); def("tiny_to_cpp", (int (&)(array::tiny<int, 2>) ) voea::tiny_to_cpp );
With these, the Python session becomes
>>> print voea.tiny_to_cpp( [1,2] ) 3 >>> print voea.tiny_to_cpp( [1.1, 2] ) 3.10000002384
In pass-by-reference, exchanges in both directions use a wrapped C++ object.
The pass-by-reference types are shared and versa. However, these do not show up in Python directly. Instead, a C++ wrapper module, flex, is used to interface to these. E.g., an instance of the C++ type
array::shared< scitbx::vec3<double> >
<scitbx_array_family_flex_ext.vec3_double object>
The Python interface strongly resembles that of the Python list type; see list examples and documentation for details.
On the C++ side, flex is a module only; there is no C++ type flex. Only the file
scitbx_array_family_flex_ext.cpp
On the Python side, there is only
scitbx.array_family.flex
To provide the type
flex.double
versa<double, flex_grid<> >
A note on implementation
The scitbx.array_family.flex Python module handles specific versa template instantiations.
Internal python array type is
scitbx::af::versa< ELEMENTTYPE, flex_grid< af::small<INDEXTYPE,10> > >
INDEXTYPE is usually long
Python instances of
scitbx::af::versa<double>
scitbx::af::shared<double>
In C++, this causes interference between boost, inheritance, and the (boost) flex extension module.
Python's accessors are ALL in the flex module. This means using
typedef scitbx::af::versa< scitbx::af::tiny<double, 3 > > euler_angles;
class_<voea::euler_angles > ("euler_angles");
The previously written
array::shared<double> range_inefficient(int m, int n) { array::shared<double> result; for(unsigned i = m; i < n; i++) { result.push_back( static_cast<double>(i) ); } return result; }
def("range_inefficient", voea::range_inefficient);
Basic access as usual:
>>> arr = voea.range_inefficient(2,10) >>> arr <scitbx_array_family_flex_ext.double object at 0x4016e86c>
By itself, this may not justify the connection effort. But some extra functionality, obtained without repeated programmer effort, does. First, there is direct conversion to Python types which makes vast functionality available immediately. Consider the printing problem. Instead of starting with
>>> for i in arr: ... print i 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
>>> list(arr) [2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]
>>> new_arr = eval('[2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]') >>> new_arr [2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]
Of course, data serialization is equally short:
ps = pickle.dumps(arr) unpickled_arr = pickle.loads(ps)
Then, there is Matlab style slicing:
print list(unpickled_arr[1:5]) print list(unpickled_arr[1:5:2])
For complex structures, re-using existing facilities is mandatory; as a preview, the call
angles = voea.py(15.0, [0.0, 90.0], [0.0, 359.0])
>>> pprint(list(angles))
>>> from pprint import pprint >>> pprint(list(arr)) >>> angles <scitbx_array_family_flex_ext.vec3_double object at 0x407e35cc> >>> pprint(list(angles)) [(0.0, 0.0, 0.0), (0.0, 15.0, 0.0), (0.0, 15.0, 71.799999999999997), ... (0.0, 90.0, 163.18181818181816), (0.0, 90.0, 179.49999999999997)]
This document was generated using AFT v5.095