Scitbx Tour: minimal framework add-on

This documentation gives a quick tour on the scitbx.

Introduction

The scitbx contains libraries for general scientific computing (i.e. libraries that are not specific to crystallographic applications): a family of high-level C++ array types, a fast Fourier transform library, and a C++ port of the popular L-BFGS quasi-Newton minimizer, all including Python bindings.

Extending with C++

The simplest thing is to add your own code into the framework. A complete component requires

  1. The C++ code
  2. The boost.python wrapper (potentially in the same file)
  3. scons instructions for building

The following sections contain cut-and-paste examples for a simple addition using existing facilities.

Setup

Prepare the module tree:

mkdir voea
cd voea

Prepare a minimal top-level SConscript in voea:

SConscript

Make a subdirectory (so you'll have the tree voea/voea/):

mkdir voea
cd voea

Standalone sample, statically linked

In voea/voea/ prepare the following files.

The C++ library file voea.cpp

#include "voea.h"
namespace voea {
        double
        tiny_to_cpp(array::tiny<double, 2> t)
        {
                return (t[0] + t[1]);
        }
}

The header file voea.h

#ifndef VOEA_VOEA_H
#define VOEA_VOEA_H

#include <scitbx/array_family/tiny_types.h>

namespace voea {
namespace array = scitbx::af;

double
tiny_to_cpp(array::tiny<double, 2> t);
}

#endif

The (trivial) C++ main() function test.cpp:

#include "voea.h"
#include <iostream>

using namespace std;
namespace array = scitbx::af;

int
main(int argc, char **argv) {
        cout << voea::tiny_to_cpp( array::tiny<double, 2>(1.1, 2.2) ) << endl;
        return 0;
}

Put the module's dependencies, enumerated, in voea/libtbx_config

{
  "modules_required_for_build": ["scitbx", "boost"],
  "modules_required_for_use": [],
}

Put the module's SConscript in voea/voea/SConscript


import libtbx.load_env
# env_etc is NOT a scons Environment, but a data dictionary.
Import("env_base", "env_etc")

env = env_base.Copy(
  CXXFLAGS=env_etc.cxxflags_base,
  LIBS=env_etc.libm,
  LIBPATH=["#lib"]
)

# For later or external references via $VOEA_DIST.
env_etc.voea_dist = libtbx.env.dist_path("voea")

env_etc.voea_common_includes = [
  env_etc.libtbx_include,
  env_etc.scitbx_include,
  env_etc.boost_include,
]

env_etc.include_registry.append(
  env = env,
  paths = env_etc.voea_common_includes)

# Standalone test, no library.
env.Program(source=["test.cpp", "voea.cpp"],
                        target="test-static")

Include the new module in the system via

pushd ../cctbx_build
python ../cctbx_sources/libtbx/configure.py voea

Now, the simple test can be built and run by

libtbx.scons voea/voea/test-static
./voea/voea/test-static

A shared library version

Useful for larger projects, and needed later for the Python connection. The only addition is in the voea/voea/SConscript:

# Shared Library
env.SharedLibrary(source=["voea.cpp"],
                                  target="#lib/voea")
# Shared library based test executable.
env_test = env.Copy()
env_test.Prepend(LIBS = ["voea"])
env_test.Program(source=["test.cpp"],
                                 target="test")

Compile and run via

libtbx.scons voea/voea/test
./voea/voea/test

This executable now uses the library; on linux,

$ ldd ./voea/voea/test

shows

...
libvoea.so => ...cctbx_build/libtbx/libvoea.so (0x40017000)
...

Python connection

The additional C++ part voea/voea/voea_ext.cpp:

/*
 * First include this !!!
 */
#include <scitbx/array_family/boost_python/flex_fwd.h>
#include <scitbx/boost_python/container_conversions.h>
#include <scitbx/array_family/tiny_types.h>
#include <boost/python.hpp>
#include "voea.h"
BOOST_PYTHON_MODULE(voea_ext)
{
        using namespace boost::python;
        namespace array = scitbx::af;
        def("tiny_to_cpp",
                (double (*)(array::tiny<double, 2>) ) voea::tiny_to_cpp );
}

The Python module additions voea/voea/__init__.py:

#* C++ import redirections.
import scitbx.array_family.flex                 # register tuple mappings
import boost.python
ext = boost.python.import_ext("voea_ext")
from voea_ext import *

The SConscript additions /voea/voea/SConscript:

Import("env_scitbx_boost_python_ext")
# Borrow configuration.
env_bpl = env_scitbx_boost_python_ext.Copy()
env_bpl.Prepend(LIBS = ["voea"])
#
env_etc.include_registry.append(
  env=env_bpl,
  paths=env_etc.voea_common_includes)
#
env_bpl.SharedLibrary(
  target="#lib/voea_ext",
  source=["voea_ext.cpp"])

where #lib is a reference for the top-level build directory.

Compile everything

libtbx.scons .

and test in Python

$ python
import voea
print voea.tiny_to_cpp( [1,2] )
print voea.tiny_to_cpp( [1.1, 2] )

Some SConscript details

  • env_base: instance of scons' Environment holding basic settings
  • env_etc: instance of empty class shared across SConscript hierarchy (levels irrelevant, topological ordering matters) physical tree structure, dag logical structure. For utility only.
  • module dependencies: Always in MODULE/libtbx_config. e.g.
{
  "modules_required_for_build": ["boost"],
  "modules_required_for_use": ["boost_adaptbx"],
}

Here,

  • modules_required_for_build: include C++ building, linking, libraries.
  • modules_required_for_use: are the Python dependencies -- modules.

After changing libtbx_config, re-run

python ../cctbx_sources/libtbx/configure.py voea

in the build tree (the cwd becomes a build tree)

source setpaths.csh
libtbx.scons -j4

After rearrangements,

source unsetpaths.csh