CMake Zlib Build On Windows


Answer :

According to https://wiki.apache.org/httpd/Win64Compilation a very similar error means:

This means you have a typo in either -DASMV -DASMINF or your OBJ="inffasx64.obj gvmat64.obj inffas8664.obj" since inflate_fast is defined in inffas8664.c.


I was able to successfully build with a simple:

mkdir C:\Builds\zlib; cd C:\Builds\zlib cmake -G "Visual Studio 12 2013" -A x64 D:\Downloads\zlib-1.2.8\ cmake --build . 

I looked at my cmake cache and I see that AMD64 is set to false, unlike what your cmake-gui window shows. Setting it to true it results all kinds of build errors for me, though not the ones you show.

CMakeLists.txt says this option is to enable an AMD64 assembly implementation. Just doing without this seems to be the easiest solution.


You need contrib\masmx64\inffas8664.c included in visual studio project file. This file contains inflate_fast function which calls corresponding asm functions.


Date: 20180804 (Aug 4th 2018)

While playing with assembler speedups, I discovered that the issue is reproducible on the (currently) latest version: v1.2.11 ([GitHub]: madler/zlib - A massively spiffy yet delicately unobtrusive compression library (http://zlib.net)).

This error happens only (obviously, OS: Win, build toolchain: VStudio and assembly speedups enabled) for:

  • CMake build (works OK for "${ZLIB_SRC_DIR}/win32/Makefile.msc")
  • x64 (AMD64) architecture (works OK for x86)

Below is a "callstack" (top -> down is equivalent to outer -> inner) during decompression.

  • Normal case:

    1. inflate (inflate.c)
    2. inflate_fast (inffast.c)
    3. ...
  • Assembler case:

    1. inflate (inflate.c)
    2. inflate_fast (contrib/masmx64/inffast8664.c)
    3. inffas8664fnc (contrib/masmx64/inffasx64.asm)
    4. ...

Problem:

#2. is missing ("${ZLIB_SRC_DIR}/CMakeLists.txt" doesn't know anything about inffast8664.c), so the chain is broken, leading to an invalid library.

Solution:

Make CMakeLists.txt aware of that file, and that is by adding:

set(ZLIB_SRCS     ${ZLIB_SRCS}     contrib/masmx64/inffas8664.c ) 

at line ~#158 (enclosed by if(MSVC) and elseif (AMD64) conditionals).


Posting full changes as well.

zlib-1.2.11-msvc_x64_asm_speedups.diff:

--- CMakeLists.txt.orig 2017-01-15 08:29:40.000000000 +0200 +++ CMakeLists.txt  2018-09-03 13:41:00.314805100 +0300 @@ -79,10 +79,10 @@  endif()   set(ZLIB_PC ${CMAKE_CURRENT_BINARY_DIR}/zlib.pc) -configure_file( ${CMAKE_CURRENT_SOURCE_DIR}/zlib.pc.cmakein -       ${ZLIB_PC} @ONLY) -configure_file(    ${CMAKE_CURRENT_SOURCE_DIR}/zconf.h.cmakein -       ${CMAKE_CURRENT_BINARY_DIR}/zconf.h @ONLY) +configure_file(${CMAKE_CURRENT_SOURCE_DIR}/zlib.pc.cmakein +        ${ZLIB_PC} @ONLY) +configure_file(${CMAKE_CURRENT_SOURCE_DIR}/zconf.h.cmakein +        ${CMAKE_CURRENT_BINARY_DIR}/zconf.h @ONLY)  include_directories(${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_SOURCE_DIR})   @@ -136,30 +136,34 @@          set(ZLIB_ASMS contrib/amd64/amd64-match.S)      endif ()  -   if(ZLIB_ASMS) -       add_definitions(-DASMV) -       set_source_files_properties(${ZLIB_ASMS} PROPERTIES LANGUAGE C COMPILE_FLAGS -DNO_UNDERLINE) -   endif() +    if(ZLIB_ASMS) +        add_definitions(-DASMV) +        set_source_files_properties(${ZLIB_ASMS} PROPERTIES LANGUAGE C COMPILE_FLAGS -DNO_UNDERLINE) +    endif()  endif()   if(MSVC)      if(ASM686) -       ENABLE_LANGUAGE(ASM_MASM) +        ENABLE_LANGUAGE(ASM_MASM)          set(ZLIB_ASMS -           contrib/masmx86/inffas32.asm -           contrib/masmx86/match686.asm -       ) +            contrib/masmx86/inffas32.asm +            contrib/masmx86/match686.asm +        )      elseif (AMD64) -       ENABLE_LANGUAGE(ASM_MASM) +        ENABLE_LANGUAGE(ASM_MASM)          set(ZLIB_ASMS -           contrib/masmx64/gvmat64.asm -           contrib/masmx64/inffasx64.asm -       ) +            contrib/masmx64/gvmat64.asm +            contrib/masmx64/inffasx64.asm +        ) +        set(ZLIB_SRCS +            ${ZLIB_SRCS} +            contrib/masmx64/inffas8664.c +        )      endif()  -   if(ZLIB_ASMS) -       add_definitions(-DASMV -DASMINF) -   endif() +    if(ZLIB_ASMS) +        add_definitions(-DASMV -DASMINF) +    endif()  endif()   # parse the full version number from zlib.h and include in ZLIB_FULL_VERSION 

The above is a diff. See [SO]: Run/Debug a Django application's UnitTests from the mouse right click context menu in PyCharm Community Edition? (@CristiFati's answer) (Patching utrunner section) for how to apply patches on Win (basically, every line that starts with one "+" sign goes in, and every line that starts with one "-" sign goes out). I am using Cygwin, btw.
I also submitted this patch to [GitHub]: madler/zlib - Ms VisualStudio - Assembler speedups on x64, but I'm not sure what its fate is gonna be, since there are more than 100 pull requests that are pending.

Output:

e:\Work\Dev\StackOverflow\q029505121\build\x64>"c:\Install\Google\Android_SDK\cmake\3.6.4111459\bin\cmake.exe" -G "NMake Makefiles" -DAMD64=ON "e:\Work\Dev\StackOverflow\q029505121\src\zlib-1.2.11" -- The C compiler identification is MSVC 19.0.24215.1 -- Check for working C compiler: C:/Install/x86/Microsoft/Visual Studio Community/2015/VC/bin/amd64/cl.exe -- Check for working C compiler: C:/Install/x86/Microsoft/Visual Studio Community/2015/VC/bin/amd64/cl.exe -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Looking for sys/types.h -- Looking for sys/types.h - found -- Looking for stdint.h -- Looking for stdint.h - found -- Looking for stddef.h -- Looking for stddef.h - found -- Check size of off64_t -- Check size of off64_t - failed -- Looking for fseeko -- Looking for fseeko - not found -- Looking for unistd.h -- Looking for unistd.h - not found -- Renaming --     E:/Work/Dev/StackOverflow/q029505121/src/zlib-1.2.11/zconf.h -- to 'zconf.h.included' because this file is included with zlib -- but CMake generates it automatically in the build directory. -- The ASM_MASM compiler identification is MSVC -- Found assembler: C:/Install/x86/Microsoft/Visual Studio Community/2015/VC/bin/amd64/ml64.exe -- Configuring done -- Generating done -- Build files have been written to: E:/Work/Dev/StackOverflow/q029505121/build/x64  e:\Work\Dev\StackOverflow\q029505121\build\x64>"c:\Install\Google\Android_SDK\cmake\3.6.4111459\bin\cmake.exe" --build . --target zlibstatic Scanning dependencies of target zlibstatic [  5%] Building C object CMakeFiles/zlibstatic.dir/adler32.obj adler32.c [ 10%] Building C object CMakeFiles/zlibstatic.dir/compress.obj compress.c [ 15%] Building C object CMakeFiles/zlibstatic.dir/crc32.obj crc32.c [ 21%] Building C object CMakeFiles/zlibstatic.dir/deflate.obj deflate.c Assembler code may have bugs -- use at your own risk [ 26%] Building C object CMakeFiles/zlibstatic.dir/gzclose.obj gzclose.c [ 31%] Building C object CMakeFiles/zlibstatic.dir/gzlib.obj gzlib.c [ 36%] Building C object CMakeFiles/zlibstatic.dir/gzread.obj gzread.c [ 42%] Building C object CMakeFiles/zlibstatic.dir/gzwrite.obj gzwrite.c [ 47%] Building C object CMakeFiles/zlibstatic.dir/inflate.obj inflate.c [ 52%] Building C object CMakeFiles/zlibstatic.dir/infback.obj infback.c [ 57%] Building C object CMakeFiles/zlibstatic.dir/inftrees.obj inftrees.c [ 63%] Building C object CMakeFiles/zlibstatic.dir/inffast.obj inffast.c Assembler code may have bugs -- use at your own risk [ 68%] Building C object CMakeFiles/zlibstatic.dir/trees.obj trees.c [ 73%] Building C object CMakeFiles/zlibstatic.dir/uncompr.obj uncompr.c [ 78%] Building C object CMakeFiles/zlibstatic.dir/zutil.obj zutil.c [ 84%] Building C object CMakeFiles/zlibstatic.dir/contrib/masmx64/inffas8664.obj inffas8664.c [ 89%] Building ASM_MASM object CMakeFiles/zlibstatic.dir/contrib/masmx64/gvmat64.obj Microsoft (R) Macro Assembler (x64) Version 14.00.24210.0 Copyright (C) Microsoft Corporation.  All rights reserved.   Assembling: E:\Work\Dev\StackOverflow\q029505121\src\zlib-1.2.11\contrib\masmx64\gvmat64.asm [ 94%] Building ASM_MASM object CMakeFiles/zlibstatic.dir/contrib/masmx64/inffasx64.obj Microsoft (R) Macro Assembler (x64) Version 14.00.24210.0 Copyright (C) Microsoft Corporation.  All rights reserved.   Assembling: E:\Work\Dev\StackOverflow\q029505121\src\zlib-1.2.11\contrib\masmx64\inffasx64.asm [100%] Linking C static library zlibstatic.lib [100%] Built target zlibstatic 

Notes:

  • I am using VStudio 2015
  • Regarding the above output:
    • To keep the output as small as possible, I am only building the static version
      • For the same reason (and also to keep it as just text), I'm building for "NMake Makefiles" (cmdline build)
    • inffas8664.c is being built (somewhere near the end)
  • You could also disable assembler speedups (by unchecking AMD64 in cmake-gui), but that would be just a workaround
  • I did some rough tests (by far, I'm not claiming these results to be general), and the performance improvement of the assembler implementation over the standard one (Debug versions) was (a percent below is the ratio between times took to perform the same operation (with / without) speedups):
    • Compress: ~86%
    • Decompress: ~62%



Update #0

([GitHub]: madler/zlib - ASM zlib build on Windows gives erroneous results (@madler's comment)) states:

What assembly code is being used? There are a few in zlib's contrib directory. By the way, the stuff in the contrib directory is not part of zlib. It is just there as a convenience and is supported (or not) by those third-party contributors. What I will do is simply remove the offending code from the next distribution.

So does the compile warning (that everyone must have seen (and most likely ignored)):

Assembler code may have bugs -- use at your own risk 

Apparently, assembler speedups and VStudio don't get along very well. More, on x86 there are several issues:

  • One fix is proposed by [SO]: module unsafe for SAFESEH image C++ (@NayanaAdassuriya's answer) (although it's not directly related to the question). In short, inffas32.asm and linker's option [MS.Docs]: /SAFESEH (Image has Safe Exception Handlers) don't match. To get rid of it, either:

    • Disable the option (enabled by default on x86 Release)
    • Pass the /safeseh option to the assembler (ml.exe)
    • Declare one in asm

    Since I use cmake to build for cmdline, I found a workaround for this. After CMakeFiles generation (but before the build), I specify it:

    • In the flags.make file (ASM_MASM_FLAGS)
    • Since I build a static lib (and for Debug builds I need the symbols as well), I also change [MS.Docs]: /Z7, /Zi, /ZI (Debug Information Format), so I modify the same file (C_FLAGS)

    I'm sure that cmake offers a way of doing the above in a proper manner, but I didn't find it (nor did I investigate thoroughly).

  • A nasty one is a segfault (Access Violation) during decompression. For that, [GitHub]: madler/zlib - inffas32.asm struct/enum binding to zlib 1.2.9 is needed.

After fixing these, everything works fine, and the performance improvements are similar to x64.


Comments

Popular posts from this blog

Are Regular VACUUM ANALYZE Still Recommended Under 9.1?

Can Feynman Diagrams Be Used To Represent Any Perturbation Theory?