Let’s talk about Autotools

Coming from Java background. The most difficult part for writting C/C++ programs(or shared libraries) is how to make the code to run on other machines(live server, for example).

Java’s virtual machine’s architecture really saves lots of people and time, you can just compile the code on your DEV machine, and deploy(mostly copy) the jar(or war, or even ear) to the distination machine, and you’re done.

So, you can test the same code that’ll be run, and copy everything it depends along with it(war for example).

When using C/C++, you don’t have these things, you should make quite good sense to everything you code depends on.

The Problem

Why I need to write an C/C++ shared library? The story begins with a little request for writting a SMILES tokenizer for MySQL. The reason for why I need to write that tokenizer, is another story 😉

For compile that plugin’s code you’ll need:

1. OpenBabel [headers and shared library are needed]: The foundation part of the conversion, I’ll need that to convert smiles to molecue structures so that I can tokenize it using a better context
2. MySQL [headers are needed]: Yes, there must be a MySQL isntallation on the server, and since MySQL has a very nice plugin architecture, I didn’t need to link to any libraries of MySQL, oh yeah!

Not quite hard, for it seems.

But, your are wrong:

Headers are not so easy to find.

Different System, different version and different distribution(even the different installation method), will cause the headers you need locate at different folders.

Take OpenBabel for example:

1. LibTool’s default location(code install methdo) will put the headers to /usr/local/include/openbabel-2.0 (yes, we’re using openbabel 2.0’s api)
2. If you’re using systems like Fedora(CentOS for example), and install openbabel-devel using yum, and you’ll find the headers will be locate at /usr/include/openbabel-2.0
3. If you like me, are using OS X to do the development, and install the openbabel using MacPorts, you’ll find the headers are here /opt/local/include/openbabel-2.0

Yes, for a very limit of systems(only CentOS and OS X), you’ll get at least 3 kind of locations for the headers you need, and user may change the default path too.

And, yes, for the worst, the server may not have any OpenBabel installation, you’ll inform the user that you need that.

The Libraries That You Want To Link Is Not Easy To Find Too

Like headers, libraries are quite hard to find too, because:

1. For libtool’s default location, the static library will be locate at /usr/local/lib name like libxxx.a, and hte dynamic library will be locate at the same location with name like libxxx.so or libxxx.dylib(for BSD users, on OS X)
2. If you install the library using yum, it’ll be here /usr/lib
3. If you install the library using mac ports, it’ll be here /opt/local/lib

That’s not all, for Fedora, if you are using 64bit OS, the 64bit library will locate to /usr/lib64.

And yes, for the worst, the server may not have any libraries you need installed.

The Deployment Location Is Uncertain

Since I’m writting a MySQL plugin, what I want to do for target make install is to install the code to MySQL’s plugin folder.

And different installation of MySQL, different system, even the default plugin folder will be quite different.

And, even worse, there maybe no MySQL plugin folder at all.

The Function You Need May Not Exists

Yes, that’s not all of the problem. For my another application, I came to a problem that some api I used in OpenBabel 2.3.2(from MacPorts) is not exist in 2.2.3(from CentOS6’s epel yum repository). So I must disable some function when compiling my code on the system that didn’t support the api of OpenBabel 2.3.2, and let other functions to work as well.

I should have a better way to do this.

My Solution

So I came to GNU’s Autotools. The reason I choose that is that MacPorts use it by default, and PHP use it to build plugins, 😀

Then I found out, Autotools is so hard to use, especially for newbie users…..

This blog will needs you have a little background knowledge of GNU Make and the knowledge about how to write Makefiles.

Problems When I Use Autotools For The First Time

1. What’s the working flow using Autotools?
2. How should i start?
3. What are the commands I should use, and how to use?
4. What file that I need to code?
5. If I want to write a shared library(like MySQL plugin, what should I do)?
6. What are AC Macros? Where is the Fking documentation for the Fking AC Macros?

These problem is the motivition for this blog.

Since its TOO HARD to beginers!!!! There is very less documentation for Autotools for beginers, and the offical documentation is a piece of SHIT!

This will scare most of the beginners away from it! I’ll try to make it a little simpler to beginners so that they can begin to play with Autotools.

What’s the working flow to use Autotools? How should I start? What commands that I should use?

Autotools is a set of tools to help you write the code that adapt to migration between different systems and installations. It can be break down to these command:

The Commands

1. autoscan: This program will scan all of your code, and generate a boilerplate for your configuration(configure.ac) for Autotools
2. aclocal: Generating autoconf’s local macros, if you do not use this command to generate the macros, you’ autoconf execution will probably get a macro is not defined error
3. autoheader: This will use the configuration in configure.ac to generate your config.h.in (The input file for automake to generate Makefile.in)
4. autoconf: This is the core part of header and library resoving macro support. This command will using the configuration in configure.ac to generate the configure script
5. automake: This will take the Makefile configuration file Makefile.am to generate the Makefile template Makefile.in

And that’s not all, if you want to write share library, you’ll need this:

1. libtool: The command line tool to create and install libraries, Autotools will support this by default(sure, they are from the same orgnization, aren’t they?)

So, there is at least 6 commands you should know, and I’ll list the files that you should write or get(for beginners, this is quite difficult):

The Files

• configure.scan: This is the output file of autoscan, you can rename it to configure.ac(it’ll create some boilerplate for you)
• configure.ac: This file is very important, this is the core configuration file for your Autotool build system, nearly every magic part of Autotools is configured here(using M4 macros)
• aclocal.m4: This file is generated by aclocal, this file will read the configuration of configure.ac and initialize the macros you’ll need(for example, the automake macros and libtool macros), this is quite quite important for the Autotool’s command execution
• config.h.in: This file is generated by command autoheader, will be the input file for automake to generate the file Makefile.in
• Makefile.am: This file is the Makefile template that you need to write, in this Makefile you’ll need to define the targets and the variables(but strongly suggest you define these variables in configure.ac and let Autotools write these variables automaticly for you to your Makefile, I’ll discuss about this later)
• Makefile.in: This file can be generate using automake, this file will is the template for the final Makefile(without pathes, since the path resoving is done by configure)
• configure: This is the final product for Autotools, since other product is generate by this script or the product of this script. This script can be created by autoconf command
• config.status: This file is generated by configure, and this script will generate the final config.h and Makefile
• config.h: This is the core part for migration, you can generate all the detection as the macros in this header file, so you can add macros in your code to do the tricks(for example, if some function is missing, will remove some functions, or if is in Windows, using some F**king api instead of using POSIX API)
• Makefile: Ah~~~ At last, we come to a file that means something…..

See? That’s why I said Autotools is quite hard for beginners. It has 6 commands(7, for including libtoolize), and 9 kind of files (input or output or input and output).

The Workflow

I’ll just describe the workflow of the share library development(since it is more complex).

1. Run autoscan to generate the configure.scan
2. Rename configure.scan to configure.ac
3. Run libtoolize –force to add the libtool support (you’ll need AUTHORS, COPYING, ChangeLog, INSTALL, NEWS and README files in the folder, or add –install option to let libtool copy these files for you.)

4. You’ll need to enable libtool and automake in your configuration, so add these code into your configure.ac

   AM_INIT_AUTOMAKE # This Macro will initialize the automake
   AC_ENABLE_SHARED # This Macro will configure the libtool to use shared library other than static
   LT_INIT # This Macro will initialize the libtool
   AC_CONFIG_MACRO_DIR([m4]) # This will provide libtool’s macros to your autoconf configuration file
   AC_OUTPUT(Makefile src/Makefile) # This will let configure generate the Makefiles for you

5. Run autoheader to generate config.h.in

6. Create your own Makefile.am(you can see here for the exmaple of writting the program’s Makefile.am), for shared library, you should use this code(if you want to install the library to your destination other than /usr/local/lib):

   pkgplugin_LTLIBRARIES= xxx.la
   xxx_la_SOURCES = xxx.h xxx.c

7. Run command autoconf to generate the configure script
8. Run command automake to generate the Makefile.in (maybe you should add option –add-missing to add the missing files)
9. Run command automake to generate the Makefile.in
10. You’re almost done, you can run ./configure to generate the config.h and Makefile then

Yes, 10 steps. 2 kind of files to write (configure.ac and Makefile.am).

The workflow will be like the image below:

Tricks

1. Use AC_MSG_CHECKING Macro to send the checking information to your user like this AC_MSG_CHECKING(F**king Windows API)
2. Use AC_MSG_RESULT Macro to send the checking result to your user like this AC_MSG_RESULT(Yes, you are using F**king Windows XP)
3. Use AC_MSG_WARN Macro to warn the user that some of function is not working, but don’t stop the checking AC_MSG_WARN(I’m afraid some is not going to work….)
4. Use AC_MSG_ERROR Macro to stop the flow, let user to install the dependencies like this AC_MSG_ERROR(You should at least to have brain to go on)
5. If you are using F**king C++, you can’t use AC_CHECK_LIB since C++ have a bad naming convention…. You should use AC_LINK_IFELSE to do this, the details is here
6. Use AC_SUBST Macro to add varibles to your Makefile like this AC_SUBST([stair_to_heaven], [not exists])

The complete documentation for Autoconf Macros is here, help yourself.

References

• GNU build system: The Wikipedia page for Autotools, very good beginning
• Autotools Mythbuster: Many good booke to read, yes, indeed
• Detecting C++ Libraries with Autotools: Awesome blog, indeed
• A tutorial for porting to autoconf & automake: Very nice short and simple tutorial for simple use of Autotools

Thanks for watching….