Intel®XED-BUILD
Intel® XED Build Guide

November 2023

Introduction

Intel® XED can be built with many compilers:

  • GNU GCC
  • Microsoft Visual Studio
  • Intel ICL/ICC
  • LLVM/Clang

Intel® XED also works with the following operating systems:

  • Linux
  • Microsoft Windows (with or without cygwin)
  • Apple Mac OS/X
  • FreeBSD

The default compiler on Linux and FreeBSD is GNU gcc. The default on Windows is to use MSVS. The default compiler on OS X is clang.

The build system uses the compilers from your PATH by default. You can override this with various command line options.

Table of Contents

  • REPOS Repository access
  • MBUILD Using mbuild to build Intel® XED
  • INSTALL Making Intel® XED kits
  • WINDOWS Windows notes
  • SHARED Shared libraries and DLLs
  • HELP The mbuild options

Repository Access

Here's how to check out Intel® XED and mbuild, as sibling directories:

git clone https://github.com/intelxed/xed.git xed
git clone https://github.com/intelxed/mbuild.git mbuild

If you are working in a directory called foo, then after the clones, your trees should look like this:

|-mbuild-|-mbuild----
foo-| |-tests-----
|
|-xed----|-datafiles-|-avx--------
| |-avxivb-----
| |-avxhsw-----
|-docsrc----
|-enumer.d--
|-examples--
|
|-include---|-private----
| |-public-----
|-misc------
|-scripts---
|
|-tests-----|-test-00000-
| |-test-00001-
...
| |-test-00147-
|-use-------

Introduction to using mbuild

Mbuild is a python-based build system. For more information about mbuild see: https://intelxed.github.io

The Intel® XED build using mbuild is dependency driven. It uses file and command signatures to detect the need to rebuild files.

The mbuild script requires python 3.8 or later.

Assuming you checked out the tree as described above, you can build Intel® XED using mbuild as follows on Linux:

mkdir build
cd build
../xed/mfile.py
../xed/mfile.py examples (optional)
../xed/mfile.py doc (optional, requires doxygen)
../xed/mfile.py doc-build (optional, requires doxygen)
../xed/mfile.py install (optional)
../xed/mfile.py install zip (optional, makes a zip file)
../xed/mfile.py examples install zip (optional, makes a zip file that includes the examples)

Actual Windows example:

git clone https://github.com/intelxed/xed.git xed
git clone https://github.com/intelxed/mbuild.git mbuild
mkdir test
cd test
C:/Python3/python ../xed/mfile.py examples install zip

For more information on building for windows see section Windows notes

Actual Linux example (password prompts omitted):

git clone https://github.com/intelxed/xed.git xed
git clone https://github.com/intelxed/mbuild.git mbuild
mkdir test
cd test
python ../xed/mfile.py examples install zip

To build 32b on a 64b platform, add to the mfile.py execution "host_cpu=ia32":

C:/Python3/python ../xed/mfile.py examples install zip host_cpu=ia32

Making IntelĀ® XED kits

Some headers are private (for building Intel® XED), some are public and some are generated. It is recommended that all users work from Intel® XED kits to simplify use, rather from the Intel® XED sources. This section describes how to build Intel® XED kits using the "install" target.

The install target collects all the pieces and puts them in one subdirectory. The static public headers and dynamically generated public header files are put in an "include" subdirectory.

Static libraries are put in the "lib" directory. If you are doing a shared object or DLL build, the shared object or DLL is put in the bin directory. This simplifies using the examples.

The manual and examples, if generated, are also included in the doc directory.

The structure of a Intel® XED kit is as follows:

|-bin------
|-doc------|-html-
|-examples-
|-xed-kit-name--|-include--
|-lib------
|-misc-----

You can also combine the various targets in one invocation. It will be quicker than the sequence of commands.

../xed/mfile.py doc doc-build examples install

Building the fast encoder

A set of additional encoder libraries can be created and built by adding the –enc2 option to the command line. See the Intel® XED reference manual for more information about the Intel® XED ENC2. A test for every encode function can be built by adding the –enc2-test option. One library is built for 64b mode with 64b addressing (m64,a64) and another library is built for 32b mode with 32b addressing (m32,a32). Due to the large amount of code generated, compilation time is significantly longer when using the –enc2 options. The Intel® XED ENC2 libraries and headers are placed in the build directory's wkit/{include,lib} directories as well as the installed kit if the "install" target is used.

Windows notes

For building on Windows, the easiest build method assumes that you've configured your environment by running the appropriate BAT file, such as vcvars32.bat or vcvarsamd64.bat from the bin directory of your MSVS installation, by opening a Visual Studio command prompt or by having installed the compiler environment in to the default environment during MSVS installation.

On windows you can build from a cmd.exe or a cygwin window, with MSVS or GNU gcc.

/* Run the Visual Studio setup script (vcvars32.bat or vcvarsamd64.bat) from your compiler installation */
C:/Python3/python ../xed/mfile.py examples ( from a cmd.exe window using win32 python)
/* or from cygwin, using MSVS, you must ensure your MSVS compiler environment is set up properly */
../xed/mfile.py examples ( from a cygwin window, using cygwin's python)
/* if you installed the compiler in a standard place, you can have MBUILD set it up*/
C:/Python3/python ../xed/mfile.py --setup-msvs --msvs-version 10 examples
/* or from cygwin, using GCC */
../xed/mfile.py --compiler=gnu examples ( from a cygwin window, using cygwin's gcc)

The cygwin builds will use cygwin's python to launch the mfile, but win32 c:/Python3/python to run the generators. I try to avoid building using cygwin's python because those builds are single threaded because of long standing bugs in cygwin.

If you want to have symbols in the Intel® XED command line example, you must build with MSVS 8 (2005) or later and supply the "--dbghelp" option to the build line. During execution, dbghelp.dll version 6.9.3.113 or later, is required. Dbghelp.dll must be placed in the same directory as xed.exe. Dbghelp.dll is available from Microsoft.

Shared libraries and DLLs

If you build Intel® XED using the --shared object, you'll get a libxed.so on Linux or a libxed.dll on Windows. When you link against these shared objects/DLL you are required to define XED_DLL (/DXED_DLL on Windows or -DXED_DLL on Linux).

The mbuild options

If you supply the --help message to the ../xed/mfile.py you'll see the various options controlling compilation. Some options are built into mbuild and are not relevant for Intel® XED build (like the assembler overrides). I've omitted the irrelevant ones in the following options listing:

% ./mfile.py --help
Usage: mfile.py [options]
Options:
-h, --help show this help message and exit
-j JOBS, --jobs=JOBS Number of concurrent worker threads to use.
--mbuild-version Emit the version information
--build-dir=BUILD_DIR
Build directory, default is 'obj'
--src-dir=SRC_DIR The directory where the sources are located.
--gen-dir=GEN_DIR The directory where generated sources are assumed to
be located.
-v VERBOSE, --verbose=VERBOSE
Verbosity level. Defaults to value passed to env_t()
--compiler=COMPILER Compiler (ms,gnu,clang,icc,icl,iclang, icx). Default
is gnu on linux and ms on windows. Default is: ms
--debug Debug build
--shared Shared DLL build
--static Statically link executables
--opt=OPT Optimization level noopt, 0, 1, 2, 3
-s, --silent Silence all but the most important messages
--extra-defines=EXTRA_DEFINES
Extra preprocessor defines
--extra-flags=EXTRA_FLAGS
Extra values for CXXFLAGS and CCFLAGS
--extra-cxxflags=EXTRA_CXXFLAGS
Extra values for CXXFLAGS
--extra-ccflags=EXTRA_CCFLAGS
Extra values for CCFLAGS
--extra-linkflags=EXTRA_LINKFLAGS
Extra values for LINKFLAGS
--extra-libs=EXTRA_LIBS
Extra values for LIBS
--toolchain=TOOLCHAIN
Compiler toolchain
--vc-dir=VC_DIR MSVS Compiler VC directory. For finding libraries and
setting the toolchain
--msvs-version=MSVS_VERSION, --msvc-version=MSVS_VERSION, --msvsversion=MSVS_VERSION, --msvcversion=MSVS_VERSION
MSVS version 6=VC98, 7=VS .Net 2003, 8=VS 2005,
9=VS2008, 10=VS 2010/DEV10, 11=VS2012/DEV11,
12=VS2013, 14=VS2015, 15=VS2017, 16=VS2019, 17=VS2022.
This sets certain flags and idioms for quirks in some
compilers.
--setup-msvc, --setup-msvs, --msvs-setup, --msvc-setup
Use the value of the --msvc-version to initialize the
MSVC configuration.
--icc-version=ICC_VERSION, --iccver=ICC_VERSION, --icc-ver=ICC_VERSION
ICC/ICL version 7, 8, 9, 10, 11
--gcc-version=GCC_VERSION, --gccversion=GCC_VERSION, --gcc-ver=GCC_VERSION
GCC version, with dots as in 2.96, 3.4.3, 4.2.0, etc.
--cc=CC full path to C compiler
--cxx=CXX full path to C++ compiler
--linker=LINKER full path to linker
--ar=AR full path to archiver (lib/ar)
--as=AS full path to assembler (gas/as/ml/ml64)
--yasm Use yasm
--no-cygwin-limit Do not limit cygwin to one job at a time. Default is
to limit cygwin to one job.
--host-cpu=ARG_HOST_CPU
Host CPU, typically ia32, intel64 or x86-64
--host-os=ARG_HOST_OS
Host OS (where the binary runs)
--android Android build (avoid rpath for examples)
--copy-runtime-libs Copy the libc to the kit. Rarely necessary if building
on old linux dev systems. Default: false
--example-linkflags=EXAMPLE_LINKFLAGS
Extra link flags for the examples
--example-flags=EXAMPLE_FLAGS
Extra compilation flags for the examples
--example-rpath=EXAMPLE_RPATHS
Extra rpath dirs for examples
--doxygen-install=DOXYGEN_INSTALL
Doxygen installation directory
--doxygen=DOXYGEN Doxygen command name
--doxygen-internal Create internal version of build documentation (just
changes paths for git repos)
-c, --clean Clean targets
-k, --keep-going Keep going after errors occur when building
--messages Enable use xed's debug messages
--no-pedantic Disable -pedantic (gnu/clang compilers).
--asserts Enable use xed's asserts
--clr Compile for Microsoft CLR
--no-werror Disable use of -Werror on GNU compiles
--show-dag Show the dependence DAG
--ext=EXT Add extension files of the form pattern-name:file-
name.txt
--extf=EXTF Add extension configuration files that contain lines
of form pattern-name:file-name.txt. All files
references will be made relative to the directory in
which the config file is located.
--xedext-dir=XEDEXT_DIR
XED extension dir
--tests-extension=TESTS_EXT
Tests directories extension
--default-isa-extf=DEFAULT_ISA
Override the default ISA files.cfg file
--no-avx Do not include AVX (nor down-stream unrelated
technologies).
--no-avx512 Do not include AVX512 (nor down-stream unrelated
technologies).
--no-ivb Do not include IVB.
--no-hsw Do not include HSW.
--no-mpx Do not include MPX.
--no-cet Do not include CET.
--no-knl Do no include KNL AVX512{PF,ER}.
--no-knm Do not include KNM.
--no-skl Do not include SKL (Skylake Client).
--no-skx Do not include SKX (Skylake Server).
--no-clx Do not include CLX (Cascade Lake Server).
--no-cpx Do not include CPX (Cooper Lake Server).
--no-cnl Do not include CNL.
--no-icl Do not include ICL.
--no-tgl Do not include TGL.
--no-adl Do not include ADL.
--no-spr Do not include SPR.
--no-future Do not include future NI.
--no-amd Disable AMD public instructions
--no-via Disable VIA public instructions
--no-lakefield Disable lakefield public instructions
--no-gnr Disable Granite Rapids public instructions
--no-srf Disable Sierra Forest public instructions
--no-cwf Disable Clearwater Forest public instructions
--no-ptl Disable Panther Lake public instructions
--no-emr Disable Emerald Rapids public instructions
--no-arl Disable Arrow Lake public instructions
--no-lnl Disable Lunar Lake public instructions
--dbghelp Use dbghelp.dll on windows.
--prefix=PREFIX_DIR XED System install directory.
--prefix-lib-dir=PREFIX_LIB_DIR
library subdirectory name. Default: lib
--install-dir=INSTALL_DIR
XED Install directory. Default: kits/xed-install-date-
os-cpu
--kit-kind=KIT_KIND Kit version string. The default is 'base'
--limit-strings Remove some strings to save space.
--no-encoder Disable the encoder
--no-decoder Disable the decoder
--generator-options=GENERATOR_OPTIONS
Options to pass through for the decode generator
--legal-header=LEGAL_HEADER
Use this special legal header on public header files
and examples.
--python=PYTHONARG Use a specific version of python for subprocesses.
--ld-library-path=LD_LIBRARY_PATH
Specify additions to LD_LIBRARY_PATH for use when
running ldd and making kits
--ld-library-path-for-tests=LD_LIBRARY_PATH_FOR_TESTS
Specify additions to LD_LIBRARY_PATH for use when
running the tests
--elf-dwarf, --dwarf Use libelf/libdwarf. (Linux only)
--elf-dwarf-precompiled
Use precompiled libelf/libdwarf from the XED source
distribution. This is the currently required if you
are installing a kit. Implies the --elf-dwarf knob.
(Linux only)
--strip=STRIP Path to strip binary. (Linux only)
--pti-test INTERNAL TESTING OPTION.
--compress-operands use bit-fields to compress the operand storage.
--add-orphan-inst-to-future-chip
Add orphan isa-sets to future chip definition.
--test-perf Do performance test (on linux). Requires specific
external test binary.
--pin-crt=PIN_CRT Compile for the Pin C-runtime. Specify path to pin kit
--static-stripped Make a static libxed.a renaming internal symbols
--set-copyright Set the Intel copyright on Windows XED executable
--asan Use Address Sanitizer (on linux)
--enc2 Build the enc2 fast encoder. Longer build.
--enc2-test Build the enc2 fast encoder *tests*. Longer build.
--enc2-test-checked Build the enc2 fast encoder *tests*. Test the checked
interface. Longer build.
--encoder-chip=ENCODER_CHIP
Specific encoder chip. Default is ALL
--setup-hooks Copies git hook scripts locally and exits. Does NOT
build XED