Log in

Fewer DSOs, not more! - Ulrich Drepper [entries|archive|friends|userinfo]
Ulrich Drepper

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Fewer DSOs, not more! [Oct. 9th, 2005|09:27 pm]
Ulrich Drepper

If my write-up on writing DSOs causes people to write more of them I failed in one aspect: if a DSO is not really needed, none should be created! This article shows what I mean.

The underlying idea might be right, functionality should be shared. But not the way it is proposed in this blog. Every DSO used increases resource usage. As the DSO How-To points out, loading a DSO isn't free. There is usually measurably more memory used. The calls into the DSO are more expensive than normal function calls. And a lot more.

So what is the right idea? Well, it should be obvious to everybody who actually read and understood the DSO How-To. DSOs are nothing special, they are just relocatable executables. The property of being relocatable allows them to be loaded in addition to other objects. That's all there is!

Now, to solve the problem at hand (writing an application and making the functionality available to other programs without popen and system). It's terribly simple, even more so with a modern toolchain: create the program as a PIE (Position Independent Executable; I should have registered the acronym as a trademark). This PIE can be executed as well as loaded as an DT_NEEDED dependency or via dlopen. And there is more: some languages allow extensions to be implemented via DSO. One example: Python. So, how does it look like?

#include <error.h>
#include <stdio.h>
#include <stdlib.h>
#include <Python.h>
#include <modsupport.h>

char *
fct (char *str, int idx)
  if (idx > strlen (str))
    return (char *) "*** FAIL";
  return str + idx;

main (int argc, char *argv[])
  if (argc != 3)
    error (1, 0, "Usage: %s STRING NUM", argv[0]);
  char *s = fct (argv[1], atoi (argv[2]));
  printf ("result = %s\n", s);
  return 0;

#pragma weak PyArg_ParseTuple
#pragma weak Py_BuildValue
#pragma weak Py_InitModule4

static PyObject *
piepython_test (PyObject *self, PyObject *args)
  char *str;
  int idx;
  if (! PyArg_ParseTuple (args, "si", &str, &idx))
    return NULL;
  char *s = fct (str, idx);
  return Py_BuildValue ("s", s);

static PyMethodDef PIEPythonMethods[] =
  { "test",  piepython_test, METH_VARARGS, "Test function."},
    {NULL, NULL, 0, NULL}        /* Sentinel */

__attribute__ ((constructor))
initpiepython (void)
  if (&Py_InitModule4 != NULL)
    (void) Py_InitModule ("piepython", PIEPythonMethods);

This is the module itself. Compile it using

gcc -o module -pie -fpie module.c -I /usr/include/python2.4 -g -rdynamic -Wl,--version-script,module.map,-soname,module

I'm using -pie and -fpie here. Depending on the situation -shared and -fpic are needed. The symbol map file is trivial:

    initpiepython; fct;

The function fct is silly but that's unimportant here. If Python support isn't needed, everything after the main function can be left out. The Python support is just some glue around fct, consisting of piepython_test. This function is exported under the name test in the module. All that is needed is to make the executable available under a name with the suffix .so in a place Python looks for modules. For testing use

ln -fs module piepython.so

Now use a little Python script to test it:

import piepython

print "result =", piepython.test ("abcdef", 2)

That's it. Now we have a binary which can be used directly as an executable, as a dependency, explicitly loaded (note: none of the previous two requires the binary to have any special suffix), and as a Python module. If one wants more, I guess it would be possible to extend the example to cover other languages like Perl, LISP, etc.

So, do NOT write extra DSOs. Use PIEs.