README-design.md
1Design document for the unified scheme data
2===========================================
3
4How are things connected?
5-------------------------
6
7The unified scheme takes all its data from the `build.info` files seen
8throughout the source tree. These files hold the minimum information
9needed to build end product files from diverse sources. See the
10section on `build.info` files below.
11
12From the information in `build.info` files, `Configure` builds up an
13information database as a hash table called `%unified_info`, which is
14stored in configdata.pm, found at the top of the build tree (which may
15or may not be the same as the source tree).
16
17[`Configurations/common.tmpl`](common.tmpl) uses the data from `%unified_info` to
18generate the rules for building end product files as well as
19intermediary files with the help of a few functions found in the
20build-file templates. See the section on build-file templates further
21down for more information.
22
23build.info files
24----------------
25
26As mentioned earlier, `build.info` files are meant to hold the minimum
27information needed to build output files, and therefore only (with a
28few possible exceptions [1]) have information about end products (such
29as scripts, library files and programs) and source files (such as C
30files, C header files, assembler files, etc). Intermediate files such
31as object files are rarely directly referred to in `build.info` files (and
32when they are, it's always with the file name extension `.o`), they are
33inferred by `Configure`. By the same rule of minimalism, end product
34file name extensions (such as `.so`, `.a`, `.exe`, etc) are never mentioned
35in `build.info`. Their file name extensions will be inferred by the
36build-file templates, adapted for the platform they are meant for (see
37sections on `%unified_info` and build-file templates further down).
38
39The variables `PROGRAMS`, `LIBS`, `MODULES` and `SCRIPTS` are used to declare
40end products. There are variants for them with `_NO_INST` as suffix
41(`PROGRAM_NO_INST` etc) to specify end products that shouldn't get installed.
42
43The variables `SOURCE`, `DEPEND`, `INCLUDE` and `DEFINE` are indexed by a
44produced file, and their values are the source used to produce that
45particular produced file, extra dependencies, include directories
46needed, or C macros to be defined.
47
48All their values in all the `build.info` throughout the source tree are
49collected together and form a set of programs, libraries, modules and
50scripts to be produced, source files, dependencies, etc etc etc.
51
52Let's have a pretend example, a very limited contraption of OpenSSL,
53composed of the program `apps/openssl`, the libraries `libssl` and
54`libcrypto`, an module `engines/ossltest` and their sources and
55dependencies.
56
57 # build.info
58 LIBS=libcrypto libssl
59 INCLUDE[libcrypto]=include
60 INCLUDE[libssl]=include
61 DEPEND[libssl]=libcrypto
62
63This is the top directory `build.info` file, and it tells us that two
64libraries are to be built, the include directory `include/` shall be
65used throughout when building anything that will end up in each
66library, and that the library `libssl` depend on the library
67`libcrypto` to function properly.
68
69 # apps/build.info
70 PROGRAMS=openssl
71 SOURCE[openssl]=openssl.c
72 INCLUDE[openssl]=.. ../include
73 DEPEND[openssl]=../libssl
74
75This is the `build.info` file in `apps/`, one may notice that all file
76paths mentioned are relative to the directory the `build.info` file is
77located in. This one tells us that there's a program to be built
78called `apps/openss` (the file name extension will depend on the
79platform and is therefore not mentioned in the `build.info` file). It's
80built from one source file, `apps/openssl.c`, and building it requires
81the use of `.` and `include/` include directories (both are declared
82from the point of view of the `apps/` directory), and that the program
83depends on the library `libssl` to function properly.
84
85 # crypto/build.info
86 LIBS=../libcrypto
87 SOURCE[../libcrypto]=aes.c evp.c cversion.c
88 DEPEND[cversion.o]=buildinf.h
89
90 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
91 DEPEND[buildinf.h]=../Makefile
92 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm
93
94This is the `build.info` file in `crypto/`, and it tells us a little more
95about what's needed to produce `libcrypto`. LIBS is used again to
96declare that `libcrypto` is to be produced. This declaration is
97really unnecessary as it's already mentioned in the top `build.info`
98file, but can make the info file easier to understand. This is to
99show that duplicate information isn't an issue.
100
101This `build.info` file informs us that `libcrypto` is built from a few
102source files, `crypto/aes.c`, `crypto/evp.c` and `crypto/cversion.c`.
103It also shows us that building the object file inferred from
104`crypto/cversion.c` depends on `crypto/buildinf.h`. Finally, it
105also shows the possibility to declare how some files are generated
106using some script, in this case a perl script, and how such scripts
107can be declared to depend on other files, in this case a perl module.
108
109Two things are worth an extra note:
110
111`DEPEND[cversion.o]` mentions an object file. DEPEND indexes is the
112only location where it's valid to mention them
113
114 # ssl/build.info
115 LIBS=../libssl
116 SOURCE[../libssl]=tls.c
117
118This is the build.info file in `ssl/`, and it tells us that the
119library `libssl` is built from the source file `ssl/tls.c`.
120
121 # engines/build.info
122 MODULES=dasync
123 SOURCE[dasync]=e_dasync.c
124 DEPEND[dasync]=../libcrypto
125 INCLUDE[dasync]=../include
126
127 MODULES_NO_INST=ossltest
128 SOURCE[ossltest]=e_ossltest.c
129 DEPEND[ossltest]=../libcrypto.a
130 INCLUDE[ossltest]=../include
131
132This is the `build.info` file in `engines/`, telling us that two modules
133called `engines/dasync` and `engines/ossltest` shall be built, that
134`dasync`'s source is `engines/e_dasync.c` and `ossltest`'s source is
135`engines/e_ossltest.c` and that the include directory `include/` may
136be used when building anything that will be part of these modules.
137Also, both modules depend on the library `libcrypto` to function
138properly. `ossltest` is explicitly linked with the static variant of
139the library `libcrypto`. Finally, only `dasync` is being installed, as
140`ossltest` is only for internal testing.
141
142When `Configure` digests these `build.info` files, the accumulated
143information comes down to this:
144
145 LIBS=libcrypto libssl
146 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
147 DEPEND[crypto/cversion.o]=crypto/buildinf.h
148 INCLUDE[libcrypto]=include
149 SOURCE[libssl]=ssl/tls.c
150 INCLUDE[libssl]=include
151 DEPEND[libssl]=libcrypto
152
153 PROGRAMS=apps/openssl
154 SOURCE[apps/openssl]=apps/openssl.c
155 INCLUDE[apps/openssl]=. include
156 DEPEND[apps/openssl]=libssl
157
158 MODULES=engines/dasync
159 SOURCE[engines/dasync]=engines/e_dasync.c
160 DEPEND[engines/dasync]=libcrypto
161 INCLUDE[engines/dasync]=include
162
163 MODULES_NO_INST=engines/ossltest
164 SOURCE[engines/ossltest]=engines/e_ossltest.c
165 DEPEND[engines/ossltest]=libcrypto.a
166 INCLUDE[engines/ossltest]=include
167
168 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
169 DEPEND[crypto/buildinf.h]=Makefile
170 DEPEND[util/mkbuildinf.pl]=util/Foo.pm
171
172A few notes worth mentioning:
173
174`LIBS` may be used to declare routine libraries only.
175
176`PROGRAMS` may be used to declare programs only.
177
178`MODULES` may be used to declare modules only.
179
180The indexes for `SOURCE` must only be end product files, such as
181libraries, programs or modules. The values of `SOURCE` variables must
182only be source files (possibly generated).
183
184`INCLUDE` and `DEPEND` shows a relationship between different files
185(usually produced files) or between files and directories, such as a
186program depending on a library, or between an object file and some
187extra source file.
188
189When `Configure` processes the `build.info` files, it will take it as
190truth without question, and will therefore perform very few checks.
191If the build tree is separate from the source tree, it will assume
192that all built files and up in the build directory and that all source
193files are to be found in the source tree, if they can be found there.
194`Configure` will assume that source files that can't be found in the
195source tree (such as `crypto/bildinf.h` in the example above) are
196generated and will be found in the build tree.
197
198The `%unified_info` database
199----------------------------
200
201The information in all the `build.info` get digested by `Configure` and
202collected into the `%unified_info` database, divided into the following
203indexes:
204
205 depends => a hash table containing 'file' => [ 'dependency' ... ]
206 pairs. These are directly inferred from the DEPEND
207 variables in build.info files.
208
209 modules => a list of modules. These are directly inferred from
210 the MODULES variable in build.info files.
211
212 generate => a hash table containing 'file' => [ 'generator' ... ]
213 pairs. These are directly inferred from the GENERATE
214 variables in build.info files.
215
216 includes => a hash table containing 'file' => [ 'include' ... ]
217 pairs. These are directly inferred from the INCLUDE
218 variables in build.info files.
219
220 install => a hash table containing 'type' => [ 'file' ... ] pairs.
221 The types are 'programs', 'libraries', 'modules' and
222 'scripts', and the array of files list the files of
223 that type that should be installed.
224
225 libraries => a list of libraries. These are directly inferred from
226 the LIBS variable in build.info files.
227
228 programs => a list of programs. These are directly inferred from
229 the PROGRAMS variable in build.info files.
230
231 scripts => a list of scripts. There are directly inferred from
232 the SCRIPTS variable in build.info files.
233
234 sources => a hash table containing 'file' => [ 'sourcefile' ... ]
235 pairs. These are indirectly inferred from the SOURCE
236 variables in build.info files. Object files are
237 mentioned in this hash table, with source files from
238 SOURCE variables, and AS source files for programs and
239 libraries.
240
241 shared_sources =>
242 a hash table just like 'sources', but only as source
243 files (object files) for building shared libraries.
244
245As an example, here is how the `build.info` files example from the
246section above would be digested into a `%unified_info` table:
247
248 our %unified_info = (
249 "depends" =>
250 {
251 "apps/openssl" =>
252 [
253 "libssl",
254 ],
255 "crypto/buildinf.h" =>
256 [
257 "Makefile",
258 ],
259 "crypto/cversion.o" =>
260 [
261 "crypto/buildinf.h",
262 ],
263 "engines/dasync" =>
264 [
265 "libcrypto",
266 ],
267 "engines/ossltest" =>
268 [
269 "libcrypto.a",
270 ],
271 "libssl" =>
272 [
273 "libcrypto",
274 ],
275 "util/mkbuildinf.pl" =>
276 [
277 "util/Foo.pm",
278 ],
279 },
280 "modules" =>
281 [
282 "engines/dasync",
283 "engines/ossltest",
284 ],
285 "generate" =>
286 {
287 "crypto/buildinf.h" =>
288 [
289 "util/mkbuildinf.pl",
290 "\"\$(CC)",
291 "\$(CFLAGS)\"",
292 "\"$(PLATFORM)\"",
293 ],
294 },
295 "includes" =>
296 {
297 "apps/openssl" =>
298 [
299 ".",
300 "include",
301 ],
302 "engines/ossltest" =>
303 [
304 "include"
305 ],
306 "libcrypto" =>
307 [
308 "include",
309 ],
310 "libssl" =>
311 [
312 "include",
313 ],
314 "util/mkbuildinf.pl" =>
315 [
316 "util",
317 ],
318 }
319 "install" =>
320 {
321 "modules" =>
322 [
323 "engines/dasync",
324 ],
325 "libraries" =>
326 [
327 "libcrypto",
328 "libssl",
329 ],
330 "programs" =>
331 [
332 "apps/openssl",
333 ],
334 },
335 "libraries" =>
336 [
337 "libcrypto",
338 "libssl",
339 ],
340 "programs" =>
341 [
342 "apps/openssl",
343 ],
344 "sources" =>
345 {
346 "apps/openssl" =>
347 [
348 "apps/openssl.o",
349 ],
350 "apps/openssl.o" =>
351 [
352 "apps/openssl.c",
353 ],
354 "crypto/aes.o" =>
355 [
356 "crypto/aes.c",
357 ],
358 "crypto/cversion.o" =>
359 [
360 "crypto/cversion.c",
361 ],
362 "crypto/evp.o" =>
363 [
364 "crypto/evp.c",
365 ],
366 "engines/e_dasync.o" =>
367 [
368 "engines/e_dasync.c",
369 ],
370 "engines/dasync" =>
371 [
372 "engines/e_dasync.o",
373 ],
374 "engines/e_ossltest.o" =>
375 [
376 "engines/e_ossltest.c",
377 ],
378 "engines/ossltest" =>
379 [
380 "engines/e_ossltest.o",
381 ],
382 "libcrypto" =>
383 [
384 "crypto/aes.c",
385 "crypto/cversion.c",
386 "crypto/evp.c",
387 ],
388 "libssl" =>
389 [
390 "ssl/tls.c",
391 ],
392 "ssl/tls.o" =>
393 [
394 "ssl/tls.c",
395 ],
396 },
397 );
398
399As can be seen, everything in `%unified_info` is fairly simple suggest
400of information. Still, it tells us that to build all programs, we
401must build `apps/openssl`, and to build the latter, we will need to
402build all its sources (`apps/openssl.o` in this case) and all the
403other things it depends on (such as `libssl`). All those dependencies
404need to be built as well, using the same logic, so to build `libssl`,
405we need to build `ssl/tls.o` as well as `libcrypto`, and to build the
406latter...
407
408Build-file templates
409--------------------
410
411Build-file templates are essentially build-files (such as `Makefile` on
412Unix) with perl code fragments mixed in. Those perl code fragment
413will generate all the configuration dependent data, including all the
414rules needed to build end product files and intermediary files alike.
415At a minimum, there must be a perl code fragment that defines a set of
416functions that are used to generates specific build-file rules, to
417build static libraries from object files, to build shared libraries
418from static libraries, to programs from object files and libraries,
419etc.
420
421 generatesrc - function that produces build file lines to generate
422 a source file from some input.
423
424 It's called like this:
425
426 generatesrc(src => "PATH/TO/tobegenerated",
427 generator => [ "generatingfile", ... ]
428 generator_incs => [ "INCL/PATH", ... ]
429 generator_deps => [ "dep1", ... ]
430 incs => [ "INCL/PATH", ... ],
431 deps => [ "dep1", ... ],
432 intent => one of "libs", "dso", "bin" );
433
434 'src' has the name of the file to be generated.
435 'generator' is the command or part of command to
436 generate the file, of which the first item is
437 expected to be the file to generate from.
438 generatesrc() is expected to analyse and figure out
439 exactly how to apply that file and how to capture
440 the result. 'generator_incs' and 'generator_deps'
441 are include directories and files that the generator
442 file itself depends on. 'incs' and 'deps' are
443 include directories and files that are used if $(CC)
444 is used as an intermediary step when generating the
445 end product (the file indicated by 'src'). 'intent'
446 indicates what the generated file is going to be
447 used for.
448
449 src2obj - function that produces build file lines to build an
450 object file from source files and associated data.
451
452 It's called like this:
453
454 src2obj(obj => "PATH/TO/objectfile",
455 srcs => [ "PATH/TO/sourcefile", ... ],
456 deps => [ "dep1", ... ],
457 incs => [ "INCL/PATH", ... ]
458 intent => one of "lib", "dso", "bin" );
459
460 'obj' has the intended object file with `.o`
461 extension, src2obj() is expected to change it to
462 something more suitable for the platform.
463 'srcs' has the list of source files to build the
464 object file, with the first item being the source
465 file that directly corresponds to the object file.
466 'deps' is a list of explicit dependencies. 'incs'
467 is a list of include file directories. Finally,
468 'intent' indicates what this object file is going
469 to be used for.
470
471 obj2lib - function that produces build file lines to build a
472 static library file ("libfoo.a" in Unix terms) from
473 object files.
474
475 called like this:
476
477 obj2lib(lib => "PATH/TO/libfile",
478 objs => [ "PATH/TO/objectfile", ... ]);
479
480 'lib' has the intended library file name *without*
481 extension, obj2lib is expected to add that. 'objs'
482 has the list of object files to build this library.
483
484 libobj2shlib - backward compatibility function that's used the
485 same way as obj2shlib (described next), and was
486 expected to build the shared library from the
487 corresponding static library when that was suitable.
488 NOTE: building a shared library from a static
489 library is now DEPRECATED, as they no longer share
490 object files. Attempting to do this will fail.
491
492 obj2shlib - function that produces build file lines to build a
493 shareable object library file ("libfoo.so" in Unix
494 terms) from the corresponding object files.
495
496 called like this:
497
498 obj2shlib(shlib => "PATH/TO/shlibfile",
499 lib => "PATH/TO/libfile",
500 objs => [ "PATH/TO/objectfile", ... ],
501 deps => [ "PATH/TO/otherlibfile", ... ]);
502
503 'lib' has the base (static) library file name
504 *without* extension. This is useful in case
505 supporting files are needed (such as import
506 libraries on Windows).
507 'shlib' has the corresponding shared library name
508 *without* extension. 'deps' has the list of other
509 libraries (also *without* extension) this library
510 needs to be linked with. 'objs' has the list of
511 object files to build this library.
512
513 obj2dso - function that produces build file lines to build a
514 dynamic shared object file from object files.
515
516 called like this:
517
518 obj2dso(lib => "PATH/TO/libfile",
519 objs => [ "PATH/TO/objectfile", ... ],
520 deps => [ "PATH/TO/otherlibfile",
521 ... ]);
522
523 This is almost the same as obj2shlib, but the
524 intent is to build a shareable library that can be
525 loaded in runtime (a "plugin"...).
526
527 obj2bin - function that produces build file lines to build an
528 executable file from object files.
529
530 called like this:
531
532 obj2bin(bin => "PATH/TO/binfile",
533 objs => [ "PATH/TO/objectfile", ... ],
534 deps => [ "PATH/TO/libfile", ... ]);
535
536 'bin' has the intended executable file name
537 *without* extension, obj2bin is expected to add
538 that. 'objs' has the list of object files to build
539 this library. 'deps' has the list of library files
540 (also *without* extension) that the programs needs
541 to be linked with.
542
543 in2script - function that produces build file lines to build a
544 script file from some input.
545
546 called like this:
547
548 in2script(script => "PATH/TO/scriptfile",
549 sources => [ "PATH/TO/infile", ... ]);
550
551 'script' has the intended script file name.
552 'sources' has the list of source files to build the
553 resulting script from.
554
555Along with the build-file templates is the driving template
556[`Configurations/common.tmpl`](common.tmpl), which looks through all the
557information in `%unified_info` and generates all the rulesets to build libraries,
558programs and all intermediate files, using the rule generating
559functions defined in the build-file template.
560
561As an example with the smaller `build.info` set we've seen as an
562example, producing the rules to build `libcrypto` would result in the
563following calls:
564
565 # Note: obj2shlib will only be called if shared libraries are
566 # to be produced.
567 # Note 2: obj2shlib must convert the '.o' extension to whatever
568 # is suitable on the local platform.
569 obj2shlib(shlib => "libcrypto",
570 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ],
571 deps => [ ]);
572
573 obj2lib(lib => "libcrypto"
574 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ]);
575
576 src2obj(obj => "crypto/aes.o"
577 srcs => [ "crypto/aes.c" ],
578 deps => [ ],
579 incs => [ "include" ],
580 intent => "lib");
581
582 src2obj(obj => "crypto/evp.o"
583 srcs => [ "crypto/evp.c" ],
584 deps => [ ],
585 incs => [ "include" ],
586 intent => "lib");
587
588 src2obj(obj => "crypto/cversion.o"
589 srcs => [ "crypto/cversion.c" ],
590 deps => [ "crypto/buildinf.h" ],
591 incs => [ "include" ],
592 intent => "lib");
593
594 generatesrc(src => "crypto/buildinf.h",
595 generator => [ "util/mkbuildinf.pl", "\"$(CC)",
596 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ],
597 generator_incs => [ "util" ],
598 generator_deps => [ "util/Foo.pm" ],
599 incs => [ ],
600 deps => [ ],
601 intent => "lib");
602
603The returned strings from all those calls are then concatenated
604together and written to the resulting build-file.
605
README.md
1Intro
2=====
3
4This directory contains a few sets of files that are used for
5configuration in diverse ways:
6
7 *.conf Target platform configurations, please read
8 'Configurations of OpenSSL target platforms' for more
9 information.
10 *.tmpl Build file templates, please read 'Build-file
11 programming with the "unified" build system' as well
12 as 'Build info files' for more information.
13 *.pm Helper scripts / modules for the main `Configure`
14 script. See 'Configure helper scripts for more
15 information.
16
17Configurations of OpenSSL target platforms
18==========================================
19
20Configuration targets are a collection of facts that we know about
21different platforms and their capabilities. We organise them in a
22hash table, where each entry represent a specific target.
23
24Note that configuration target names must be unique across all config
25files. The Configure script does check that a config file doesn't
26have config targets that shadow config targets from other files.
27
28In each table entry, the following keys are significant:
29
30 inherit_from => Other targets to inherit values from.
31 Explained further below. [1]
32 template => Set to 1 if this isn't really a platform
33 target. Instead, this target is a template
34 upon which other targets can be built.
35 Explained further below. [1]
36
37 sys_id => System identity for systems where that
38 is difficult to determine automatically.
39
40 enable => Enable specific configuration features.
41 This MUST be an array of words.
42 disable => Disable specific configuration features.
43 This MUST be an array of words.
44 Note: if the same feature is both enabled
45 and disabled, disable wins.
46
47 as => The assembler command. This is not always
48 used (for example on Unix, where the C
49 compiler is used instead).
50 asflags => Default assembler command flags [4].
51 cpp => The C preprocessor command, normally not
52 given, as the build file defaults are
53 usually good enough.
54 cppflags => Default C preprocessor flags [4].
55 defines => As an alternative, macro definitions may be
56 given here instead of in 'cppflags' [4].
57 If given here, they MUST be as an array of
58 the string such as "MACRO=value", or just
59 "MACRO" for definitions without value.
60 includes => As an alternative, inclusion directories
61 may be given here instead of in 'cppflags'
62 [4]. If given here, the MUST be an array
63 of strings, one directory specification
64 each.
65 cc => The C compiler command, usually one of "cc",
66 "gcc" or "clang". This command is normally
67 also used to link object files and
68 libraries into the final program.
69 cxx => The C++ compiler command, usually one of
70 "c++", "g++" or "clang++". This command is
71 also used when linking a program where at
72 least one of the object file is made from
73 C++ source.
74 cflags => Defaults C compiler flags [4].
75 cxxflags => Default C++ compiler flags [4]. If unset,
76 it gets the same value as cflags.
77
78 (linking is a complex thing, see [3] below)
79 ld => Linker command, usually not defined
80 (meaning the compiler command is used
81 instead).
82 (NOTE: this is here for future use, it's
83 not implemented yet)
84 lflags => Default flags used when linking apps,
85 shared libraries or DSOs [4].
86 ex_libs => Extra libraries that are needed when
87 linking shared libraries, DSOs or programs.
88 The value is also assigned to Libs.private
89 in $(libdir)/pkgconfig/libcrypto.pc.
90
91 shared_cppflags => Extra C preprocessor flags used when
92 processing C files for shared libraries.
93 shared_cflag => Extra C compiler flags used when compiling
94 for shared libraries, typically something
95 like "-fPIC".
96 shared_ldflag => Extra linking flags used when linking
97 shared libraries.
98 module_cppflags
99 module_cflags
100 module_ldflags => Has the same function as the corresponding
101 'shared_' attributes, but for building DSOs.
102 When unset, they get the same values as the
103 corresponding 'shared_' attributes.
104
105 ar => The library archive command, the default is
106 "ar".
107 (NOTE: this is here for future use, it's
108 not implemented yet)
109 arflags => Flags to be used with the library archive
110 command. On Unix, this includes the
111 command letter, 'r' by default.
112
113 ranlib => The library archive indexing command, the
114 default is 'ranlib' it it exists.
115
116 unistd => An alternative header to the typical
117 '<unistd.h>'. This is very rarely needed.
118
119 shared_extension => File name extension used for shared
120 libraries.
121 obj_extension => File name extension used for object files.
122 On unix, this defaults to ".o" (NOTE: this
123 is here for future use, it's not
124 implemented yet)
125 exe_extension => File name extension used for executable
126 files. On unix, this defaults to "" (NOTE:
127 this is here for future use, it's not
128 implemented yet)
129 shlib_variant => A "variant" identifier inserted between the base
130 shared library name and the extension. On "unixy"
131 platforms (BSD, Linux, Solaris, MacOS/X, ...) this
132 supports installation of custom OpenSSL libraries
133 that don't conflict with other builds of OpenSSL
134 installed on the system. The variant identifier
135 becomes part of the SONAME of the library and also
136 any symbol versions (symbol versions are not used or
137 needed with MacOS/X). For example, on a system
138 where a default build would normally create the SSL
139 shared library as 'libssl.so -> libssl.so.1.1' with
140 the value of the symlink as the SONAME, a target
141 definition that sets 'shlib_variant => "-abc"' will
142 create 'libssl.so -> libssl-abc.so.1.1', again with
143 an SONAME equal to the value of the symlink. The
144 symbol versions associated with the variant library
145 would then be 'OPENSSL_ABC_<version>' rather than
146 the default 'OPENSSL_<version>'. The string inserted
147 into symbol versions is obtained by mapping all
148 letters in the "variant" identifier to uppercase
149 and all non-alphanumeric characters to '_'.
150
151 thread_scheme => The type of threads is used on the
152 configured platform. Currently known
153 values are "(unknown)", "pthreads",
154 "uithreads" (a.k.a solaris threads) and
155 "winthreads". Except for "(unknown)", the
156 actual value is currently ignored but may
157 be used in the future. See further notes
158 below [2].
159 dso_scheme => The type of dynamic shared objects to build
160 for. This mostly comes into play with
161 modules, but can be used for other purposes
162 as well. Valid values are "DLFCN"
163 (dlopen() et al), "DLFCN_NO_H" (for systems
164 that use dlopen() et al but do not have
165 fcntl.h), "DL" (shl_load() et al), "WIN32"
166 and "VMS".
167 asm_arch => The architecture to be used for compiling assembly
168 source. This acts as a selector in build.info files.
169 uplink_arch => The architecture to be used for compiling uplink
170 source. This acts as a selector in build.info files.
171 This is separate from asm_arch because it's compiled
172 even when 'no-asm' is given, even though it contains
173 assembler source.
174 perlasm_scheme => The perlasm method used to create the
175 assembler files used when compiling with
176 assembler implementations.
177 shared_target => The shared library building method used.
178 This serves multiple purposes:
179 - as index for targets found in shared_info.pl.
180 - as linker script generation selector.
181 To serve both purposes, the index for shared_info.pl
182 should end with '-shared', and this suffix will be
183 removed for use as a linker script generation
184 selector. Note that the latter is only used if
185 'shared_defflag' is defined.
186 build_scheme => The scheme used to build up a Makefile.
187 In its simplest form, the value is a string
188 with the name of the build scheme.
189 The value may also take the form of a list
190 of strings, if the build_scheme is to have
191 some options. In this case, the first
192 string in the list is the name of the build
193 scheme.
194 Currently recognised build scheme is "unified".
195 For the "unified" build scheme, this item
196 *must* be an array with the first being the
197 word "unified" and the second being a word
198 to identify the platform family.
199
200 multilib => On systems that support having multiple
201 implementations of a library (typically a
202 32-bit and a 64-bit variant), this is used
203 to have the different variants in different
204 directories.
205
206 multibin => On systems that support having multiple
207 implementations of a library and binaries
208 (typically a 32-bit and a 64-bit variant),
209 this is used to have the different variants
210 in different binary directories. This setting
211 works in conjunction with multilib.
212
213 bn_ops => Building options (was just bignum options in
214 the earlier history of this option, hence the
215 name). This is a string of words that describe
216 algorithms' implementation parameters that
217 are optimal for the designated target platform,
218 such as the type of integers used to build up
219 the bignum, different ways to implement certain
220 ciphers and so on. To fully comprehend the
221 meaning, the best is to read the affected
222 source.
223 The valid words are:
224
225 THIRTY_TWO_BIT bignum limbs are 32 bits,
226 this is default if no
227 option is specified, it
228 works on any supported
229 system [unless "wider"
230 limb size is implied in
231 assembly code];
232 BN_LLONG bignum limbs are 32 bits,
233 but 64-bit 'unsigned long
234 long' is used internally
235 in calculations;
236 SIXTY_FOUR_BIT_LONG bignum limbs are 64 bits
237 and sizeof(long) is 8;
238 SIXTY_FOUR_BIT bignums limbs are 64 bits,
239 but execution environment
240 is ILP32;
241 RC4_CHAR RC4 key schedule is made
242 up of 'unsigned char's;
243 Note: should not be used
244 for new configuration
245 targets
246 RC4_INT RC4 key schedule is made
247 up of 'unsigned int's;
248 Note: should not be used
249 for new configuration
250 targets
251
252[1] as part of the target configuration, one can have a key called
253 `inherit_from` that indicates what other configurations to inherit
254 data from. These are resolved recursively.
255
256 Inheritance works as a set of default values that can be overridden
257 by corresponding key values in the inheriting configuration.
258
259 Note 1: any configuration table can be used as a template.
260 Note 2: pure templates have the attribute `template => 1` and
261 cannot be used as build targets.
262
263 If several configurations are given in the `inherit_from` array,
264 the values of same attribute are concatenated with space
265 separation. With this, it's possible to have several smaller
266 templates for different configuration aspects that can be combined
267 into a complete configuration.
268
269 Instead of a scalar value or an array, a value can be a code block
270 of the form `sub { /* your code here */ }`. This code block will
271 be called with the list of inherited values for that key as
272 arguments. In fact, the concatenation of strings is really done
273 by using `sub { join(" ",@_) }` on the list of inherited values.
274
275 An example:
276
277 "foo" => {
278 template => 1,
279 haha => "ha ha",
280 hoho => "ho",
281 ignored => "This should not appear in the end result",
282 },
283 "bar" => {
284 template => 1,
285 haha => "ah",
286 hoho => "haho",
287 hehe => "hehe"
288 },
289 "laughter" => {
290 inherit_from => [ "foo", "bar" ],
291 hehe => sub { join(" ",(@_,"!!!")) },
292 ignored => "",
293 }
294
295 The entry for "laughter" will become as follows after processing:
296
297 "laughter" => {
298 haha => "ha ha ah",
299 hoho => "ho haho",
300 hehe => "hehe !!!",
301 ignored => ""
302 }
303
304[2] OpenSSL is built with threading capabilities unless the user
305 specifies `no-threads`. The value of the key `thread_scheme` may
306 be `(unknown)`, in which case the user MUST give some compilation
307 flags to `Configure`.
308
309[3] OpenSSL has three types of things to link from object files or
310 static libraries:
311
312 - shared libraries; that would be libcrypto and libssl.
313 - shared objects (sometimes called dynamic libraries); that would
314 be the modules.
315 - applications; those are apps/openssl and all the test apps.
316
317 Very roughly speaking, linking is done like this (words in braces
318 represent the configuration settings documented at the beginning
319 of this file):
320
321 shared libraries:
322 {ld} $(CFLAGS) {lflags} {shared_ldflag} -o libfoo.so \
323 foo/something.o foo/somethingelse.o {ex_libs}
324
325 shared objects:
326 {ld} $(CFLAGS) {lflags} {module_ldflags} -o libeng.so \
327 blah1.o blah2.o -lcrypto {ex_libs}
328
329 applications:
330 {ld} $(CFLAGS) {lflags} -o app \
331 app1.o utils.o -lssl -lcrypto {ex_libs}
332
333[4] There are variants of these attribute, prefixed with `lib_`,
334 `dso_` or `bin_`. Those variants replace the unprefixed attribute
335 when building library, DSO or program modules specifically.
336
337Historically, the target configurations came in form of a string with
338values separated by colons. This use is deprecated. The string form
339looked like this:
340
341 "target" => "{cc}:{cflags}:{unistd}:{thread_cflag}:{sys_id}:{lflags}:
342 {bn_ops}:{cpuid_obj}:{bn_obj}:{ec_obj}:{des_obj}:{aes_obj}:
343 {bf_obj}:{md5_obj}:{sha1_obj}:{cast_obj}:{rc4_obj}:
344 {rmd160_obj}:{rc5_obj}:{wp_obj}:{cmll_obj}:{modes_obj}:
345 {padlock_obj}:{perlasm_scheme}:{dso_scheme}:{shared_target}:
346 {shared_cflag}:{shared_ldflag}:{shared_extension}:{ranlib}:
347 {arflags}:{multilib}"
348
349Build info files
350================
351
352The `build.info` files that are spread over the source tree contain the
353minimum information needed to build and distribute OpenSSL. It uses a
354simple and yet fairly powerful language to determine what needs to be
355built, from what sources, and other relationships between files.
356
357For every `build.info` file, all file references are relative to the
358directory of the `build.info` file for source files, and the
359corresponding build directory for built files if the build tree
360differs from the source tree.
361
362When processed, every line is processed with the perl module
363Text::Template, using the delimiters `{-` and `-}`. The hashes
364`%config` and `%target` are passed to the perl fragments, along with
365$sourcedir and $builddir, which are the locations of the source
366directory for the current `build.info` file and the corresponding build
367directory, all relative to the top of the build tree.
368
369`Configure` only knows inherently about the top `build.info` file. For
370any other directory that has one, further directories to look into
371must be indicated like this:
372
373 SUBDIRS=something someelse
374
375On to things to be built; they are declared by setting specific
376variables:
377
378 PROGRAMS=foo bar
379 LIBS=libsomething
380 MODULES=libeng
381 SCRIPTS=myhack
382
383Note that the files mentioned for PROGRAMS, LIBS and MODULES *must* be
384without extensions. The build file templates will figure them out.
385
386For each thing to be built, it is then possible to say what sources
387they are built from:
388
389 PROGRAMS=foo bar
390 SOURCE[foo]=foo.c common.c
391 SOURCE[bar]=bar.c extra.c common.c
392
393It's also possible to tell some other dependencies:
394
395 DEPEND[foo]=libsomething
396 DEPEND[libbar]=libsomethingelse
397
398(it could be argued that 'libsomething' and 'libsomethingelse' are
399source as well. However, the files given through SOURCE are expected
400to be located in the source tree while files given through DEPEND are
401expected to be located in the build tree)
402
403It's also possible to depend on static libraries explicitly:
404
405 DEPEND[foo]=libsomething.a
406 DEPEND[libbar]=libsomethingelse.a
407
408This should be rarely used, and care should be taken to make sure it's
409only used when supported. For example, native Windows build doesn't
410support building static libraries and DLLs at the same time, so using
411static libraries on Windows can only be done when configured
412`no-shared`.
413
414In some cases, it's desirable to include some source files in the
415shared form of a library only:
416
417 SHARED_SOURCE[libfoo]=dllmain.c
418
419For any file to be built, it's also possible to tell what extra
420include paths the build of their source files should use:
421
422 INCLUDE[foo]=include
423
424It's also possible to specify C macros that should be defined:
425
426 DEFINE[foo]=FOO BAR=1
427
428In some cases, one might want to generate some source files from
429others, that's done as follows:
430
431 GENERATE[foo.s]=asm/something.pl $(CFLAGS)
432 GENERATE[bar.s]=asm/bar.S
433
434The value of each GENERATE line is a command line or part of it.
435Configure places no rules on the command line, except that the first
436item must be the generator file. It is, however, entirely up to the
437build file template to define exactly how those command lines should
438be handled, how the output is captured and so on.
439
440Sometimes, the generator file itself depends on other files, for
441example if it is a perl script that depends on other perl modules.
442This can be expressed using DEPEND like this:
443
444 DEPEND[asm/something.pl]=../perlasm/Foo.pm
445
446There may also be cases where the exact file isn't easily specified,
447but an inclusion directory still needs to be specified. INCLUDE can
448be used in that case:
449
450 INCLUDE[asm/something.pl]=../perlasm
451
452NOTE: GENERATE lines are limited to one command only per GENERATE.
453
454Finally, you can have some simple conditional use of the `build.info`
455information, looking like this:
456
457 IF[1]
458 something
459 ELSIF[2]
460 something other
461 ELSE
462 something else
463 ENDIF
464
465The expression in square brackets is interpreted as a string in perl,
466and will be seen as true if perl thinks it is, otherwise false. For
467example, the above would have "something" used, since 1 is true.
468
469Together with the use of Text::Template, this can be used as
470conditions based on something in the passed variables, for example:
471
472 IF[{- $disabled{shared} -}]
473 LIBS=libcrypto
474 SOURCE[libcrypto]=...
475 ELSE
476 LIBS=libfoo
477 SOURCE[libfoo]=...
478 ENDIF
479
480Build-file programming with the "unified" build system
481======================================================
482
483"Build files" are called `Makefile` on Unix-like operating systems,
484`descrip.mms` for MMS on VMS, `makefile` for `nmake` on Windows, etc.
485
486To use the "unified" build system, the target configuration needs to
487set the three items `build_scheme`, `build_file` and `build_command`.
488In the rest of this section, we will assume that `build_scheme` is set
489to "unified" (see the configurations documentation above for the
490details).
491
492For any name given by `build_file`, the "unified" system expects a
493template file in `Configurations/` named like the build file, with
494`.tmpl` appended, or in case of possible ambiguity, a combination of
495the second `build_scheme` list item and the `build_file` name. For
496example, if `build_file` is set to `Makefile`, the template could be
497`Configurations/Makefile.tmpl` or `Configurations/unix-Makefile.tmpl`.
498In case both `Configurations/unix-Makefile.tmpl` and
499`Configurations/Makefile.tmpl` are present, the former takes precedence.
500
501The build-file template is processed with the perl module
502Text::Template, using `{-` and `-}` as delimiters that enclose the
503perl code fragments that generate configuration-dependent content.
504Those perl fragments have access to all the hash variables from
505configdata.pem.
506
507The build-file template is expected to define at least the following
508perl functions in a perl code fragment enclosed with `{-` and `-}`.
509They are all expected to return a string with the lines they produce.
510
511 generatesrc - function that produces build file lines to generate
512 a source file from some input.
513
514 It's called like this:
515
516 generatesrc(src => "PATH/TO/tobegenerated",
517 generator => [ "generatingfile", ... ]
518 generator_incs => [ "INCL/PATH", ... ]
519 generator_deps => [ "dep1", ... ]
520 generator => [ "generatingfile", ... ]
521 incs => [ "INCL/PATH", ... ],
522 deps => [ "dep1", ... ],
523 intent => one of "libs", "dso", "bin" );
524
525 'src' has the name of the file to be generated.
526 'generator' is the command or part of command to
527 generate the file, of which the first item is
528 expected to be the file to generate from.
529 generatesrc() is expected to analyse and figure out
530 exactly how to apply that file and how to capture
531 the result. 'generator_incs' and 'generator_deps'
532 are include directories and files that the generator
533 file itself depends on. 'incs' and 'deps' are
534 include directories and files that are used if $(CC)
535 is used as an intermediary step when generating the
536 end product (the file indicated by 'src'). 'intent'
537 indicates what the generated file is going to be
538 used for.
539
540 src2obj - function that produces build file lines to build an
541 object file from source files and associated data.
542
543 It's called like this:
544
545 src2obj(obj => "PATH/TO/objectfile",
546 srcs => [ "PATH/TO/sourcefile", ... ],
547 deps => [ "dep1", ... ],
548 incs => [ "INCL/PATH", ... ]
549 intent => one of "lib", "dso", "bin" );
550
551 'obj' has the intended object file with '.o'
552 extension, src2obj() is expected to change it to
553 something more suitable for the platform.
554 'srcs' has the list of source files to build the
555 object file, with the first item being the source
556 file that directly corresponds to the object file.
557 'deps' is a list of explicit dependencies. 'incs'
558 is a list of include file directories. Finally,
559 'intent' indicates what this object file is going
560 to be used for.
561
562 obj2lib - function that produces build file lines to build a
563 static library file ("libfoo.a" in Unix terms) from
564 object files.
565
566 called like this:
567
568 obj2lib(lib => "PATH/TO/libfile",
569 objs => [ "PATH/TO/objectfile", ... ]);
570
571 'lib' has the intended library filename *without*
572 extension, obj2lib is expected to add that. 'objs'
573 has the list of object files to build this library.
574
575 libobj2shlib - backward compatibility function that's used the
576 same way as obj2shlib (described next), and was
577 expected to build the shared library from the
578 corresponding static library when that was suitable.
579 NOTE: building a shared library from a static
580 library is now DEPRECATED, as they no longer share
581 object files. Attempting to do this will fail.
582
583 obj2shlib - function that produces build file lines to build a
584 shareable object library file ("libfoo.so" in Unix
585 terms) from the corresponding object files.
586
587 called like this:
588
589 obj2shlib(shlib => "PATH/TO/shlibfile",
590 lib => "PATH/TO/libfile",
591 objs => [ "PATH/TO/objectfile", ... ],
592 deps => [ "PATH/TO/otherlibfile", ... ]);
593
594 'lib' has the base (static) library filename
595 *without* extension. This is useful in case
596 supporting files are needed (such as import
597 libraries on Windows).
598 'shlib' has the corresponding shared library name
599 *without* extension. 'deps' has the list of other
600 libraries (also *without* extension) this library
601 needs to be linked with. 'objs' has the list of
602 object files to build this library.
603
604 obj2dso - function that produces build file lines to build a
605 dynamic shared object file from object files.
606
607 called like this:
608
609 obj2dso(lib => "PATH/TO/libfile",
610 objs => [ "PATH/TO/objectfile", ... ],
611 deps => [ "PATH/TO/otherlibfile",
612 ... ]);
613
614 This is almost the same as obj2shlib, but the
615 intent is to build a shareable library that can be
616 loaded in runtime (a "plugin"...).
617
618 obj2bin - function that produces build file lines to build an
619 executable file from object files.
620
621 called like this:
622
623 obj2bin(bin => "PATH/TO/binfile",
624 objs => [ "PATH/TO/objectfile", ... ],
625 deps => [ "PATH/TO/libfile", ... ]);
626
627 'bin' has the intended executable filename
628 *without* extension, obj2bin is expected to add
629 that. 'objs' has the list of object files to build
630 this library. 'deps' has the list of library files
631 (also *without* extension) that the programs needs
632 to be linked with.
633
634 in2script - function that produces build file lines to build a
635 script file from some input.
636
637 called like this:
638
639 in2script(script => "PATH/TO/scriptfile",
640 sources => [ "PATH/TO/infile", ... ]);
641
642 'script' has the intended script filename.
643 'sources' has the list of source files to build the
644 resulting script from.
645
646In all cases, file file paths are relative to the build tree top, and
647the build file actions run with the build tree top as current working
648directory.
649
650Make sure to end the section with these functions with a string that
651you thing is appropriate for the resulting build file. If nothing
652else, end it like this:
653
654 ""; # Make sure no lingering values end up in the Makefile
655 -}
656
657Configure helper scripts
658========================
659
660Configure uses helper scripts in this directory:
661
662Checker scripts
663---------------
664
665These scripts are per platform family, to check the integrity of the
666tools used for configuration and building. The checker script used is
667either `{build_platform}-{build_file}-checker.pm` or
668`{build_platform}-checker.pm`, where `{build_platform}` is the second
669`build_scheme` list element from the configuration target data, and
670`{build_file}` is `build_file` from the same target data.
671
672If the check succeeds, the script is expected to end with a non-zero
673expression. If the check fails, the script can end with a zero, or
674with a `die`.
675
