New BIND releases ... 9.16.4: build, 'fatal error: lmdb.h: No such file or directory'

[PGNet Dev]

9.16.4 - https://downloads.isc.org/isc/bind9/9.16.4/RELEASE-NOTES-bind-9.16.4.html

on

lsb_release -rd
Description: openSUSE Leap 15.1
Release: 15.1

with

gcc --version
gcc (SUSE Linux) 10.1.1 20200515 [revision b0461f44076c26ced5526e48d4620b8ea026cfd6]

upgrading from 9.16.3

named -v
BIND 9.16.3 (Stable Release) <id:5ea41c1>

to 9.16.4, with my usual build config, including

--with-lmdb=/usr/local/lmdb \

where,

tree /usr/local/lmdb
/usr/local/lmdb
├── bin
│   ├── mdb_copy
│   ├── mdb_drop
│   ├── mdb_dump
│   ├── mdb_load
│   └── mdb_stat
├── include
!! │   └── lmdb.h
├── lib
├── lib64
│   ├── liblmdb.a
│   └── liblmdb.so
├── man
└── share
└── man
└── man1
├── mdb_copy.1
├── mdb_drop.1
├── mdb_dump.1
├── mdb_load.1
└── mdb_stat.1


build now fails @ "fatal error: lmdb.h: No such file or directory"

...
libtool: compile: /usr/bin/gcc-10 -include /usr/local/src/bind-9.16.4/config.h -I/usr/local/src/bind-9.16.4 -I../.. -I./include -I./unix/include -I. -I/usr/local/src/bind-9.16.4/lib/ns/include -I../../lib/ns/include -I/usr/local/src/bind-9.16.4/lib/dns/include -I../../lib/dns/include -I/usr/local/openssl11/include -I/usr/local/src/bind-9.16.4/lib/bind9/include -I../../lib/bind9/include -I/usr/local/src/bind-9.16.4/lib/isccfg/include -I../../lib/isccfg/include -I/usr/local/src/bind-9.16.4/lib/isccc/include -I../../lib/isccc/include -I/usr/local/src/bind-9.16.4/lib/isc/include -I../../lib/isc -I../../lib/isc/include -I../../lib/isc/unix/include -I../../lib/isc/pthreads/include -I/usr/local/openssl11/include -I/usr/include/json-c -I/usr/include/libxml2 -I/usr/local/include -O3 -Wall -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fmessage-length=0 -grecord-gcc-switches -march=native -mtune=native -fPIC -DPIC -D_GNU_SOURCE -fno-strict-aliasing -Wall -pthread -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -Wno-missing-field-initializers -fno-strict-aliasing -DPRODUCT=\"BIND\" -DVERSION=\"9.16.4\" -c ./server.c -fPIC -DPIC -o .libs/server.o
In file included from ./server.c:131:
/usr/local/src/bind-9.16.4/lib/dns/include/dns/lmdb.h:12:10: fatal error: lmdb.h: No such file or directory
12 | #include <lmdb.h>
| ^~~~~~~~
compilation terminated.
make[2]: *** [Makefile:569: server.lo] Error 1
make[2]: Leaving directory '/usr/local/src/bind-9.16.4/bin/named'
make[1]: *** [Makefile:81: subdirs] Error 1
make[1]: Leaving directory '/usr/local/src/bind-9.16.4/bin'
make: *** [Makefile:86: subdirs] Error 1


but checking,

ls -al /usr/local/lmdb/include/lmdb.h
-rw-r--r-- 1 root root 75K Jun 15 18:18 /usr/local/lmdb/include/lmdb.h

and

echo $LDFLAGS
-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lssl -lcrypto -L/usr/local/lmdb/lib64 -Wl,-rpath,/usr/local/lmdb/lib64 -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib64

echo $CPPFLAGS
-I/usr/local/include -I/usr/local/openssl11/include -I/usr/local/lmdb/include -I/usr/include


fwiw, drop back to 9.16.3 is still an OK build

[PGNet Dev]

Post by PGNet Dev
to 9.16.4, with my usual build config, including
--with-lmdb=/usr/local/lmdb \
where,
tree /usr/local/lmdb
/usr/local/lmdb
├── bin
│   ├── mdb_copy
│   ├── mdb_drop
│   ├── mdb_dump
│   ├── mdb_load
│   └── mdb_stat
├── include
!! │   └── lmdb.h
├── lib
├── lib64
│   ├── liblmdb.a
│   └── liblmdb.so
├── man
└── share
└── man
└── man1
├── mdb_copy.1
├── mdb_drop.1
├── mdb_dump.1
├── mdb_load.1
└── mdb_stat.1

I notice, now, after configure,


egrep "^LMDB" Makefile
LMDB_CFLAGS = -I/usr/local/lmdb/include
LMDB_LIBS = -L/usr/local/lmdb/lib none required


which certainly isn't correct.

checking further ...

[Ondřej Surý]

Virtually everybody except upstream has modified LMDB to follow a normal directory layout and BIND expect that.

You should just move the header files and library to /usr/local/include and /usr/local/lib respectively.

Ondrej
--
Ondřej Surý — ISC

Post by PGNet Dev
which certainly isn't correct.

[PGNet Dev]

Post by Ondřej Surý
Virtually everybody except upstream has modified LMDB to follow a normal directory layout and BIND expect that.
You should just move the header files and library to /usr/local/include and /usr/local/lib respectively.

That's just a silly statement.

1st, /usr/local/lib is NOT a 'normal directory' layout for a number of distros -- /usr/local/lib64 is. That assumption in bind's code is just wrong.

Software should link against the libs, and include the headers, you TELL it to -- not what it arbitrarily 'expects'.

Finally, if that's BIND's 'expectation', why does bind even have a =path config option for it?

What's the basis for arbitrary choice of bind using pkg-config for some pkgs, providing well-functioning _CFLAGS/_LDFLAGS/_LIBS overrides for others -- but in the case of lmdb, neither -- just an 'expectation'?

[Ondřej Surý]

Did you came to get help or to argue?

LMDB support in BIND 9 comes with pkg-config support too, you can use and adjust the .pc file from any Linux distribution package.

It’s really not our fault the LMDB upstream decided to make it hard to use the library.

Ondrej
--
Ondřej Surý — ISC

Post by PGNet Dev

Post by Ondřej Surý
Virtually everybody except upstream has modified LMDB to follow a normal directory layout and BIND expect that.
You should just move the header files and library to /usr/local/include and /usr/local/lib respectively.

That's just a silly statement.
1st, /usr/local/lib is NOT a 'normal directory' layout for a number of distros -- /usr/local/lib64 is. That assumption in bind's code is just wrong.
Software should link against the libs, and include the headers, you TELL it to -- not what it arbitrarily 'expects'.
Finally, if that's BIND's 'expectation', why does bind even have a =path config option for it?
What's the basis for arbitrary choice of bind using pkg-config for some pkgs, providing well-functioning _CFLAGS/_LDFLAGS/_LIBS overrides for others -- but in the case of lmdb, neither -- just an 'expectation'?

[Matt Pallissard]

Response in-line.

Post by PGNet Dev
...
libtool: compile: /usr/bin/gcc-10 -include /usr/local/src/bind-9.16.4/config.h -I/usr/local/src/bind-9.16.4 -I../.. -I./include -I./unix/include -I. -I/usr/local/src/bind-9.16.4/lib/ns/include -I../../lib/ns/include -I/usr/local/src/bind-9.16.4/lib/dns/include -I../../lib/dns/include -I/usr/local/openssl11/include -I/usr/local/src/bind-9.16.4/lib/bind9/include -I../../lib/bind9/include -I/usr/local/src/bind-9.16.4/lib/isccfg/include -I../../lib/isccfg/include -I/usr/local/src/bind-9.16.4/lib/isccc/include -I../../lib/isccc/include -I/usr/local/src/bind-9.16.4/lib/isc/include -I../../lib/isc -I../../lib/isc/include -I../../lib/isc/unix/include -I../../lib/isc/pthreads/include -I/usr/local/openssl11/include -I/usr/include/json-c -I/usr/include/libxml2 -I/usr/local/include -O3 -Wall -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fmessage-length=0 -grecord-gcc-switches -march=native -mtune=native -fPIC -DPIC -D_GNU_SOURCE -fno-strict-aliasing -Wall -pthread -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -Wno-missing-field-initializers -fno-strict-aliasing -DPRODUCT=\"BIND\" -DVERSION=\"9.16.4\" -c ./server.c -fPIC -DPIC -o .libs/server.o

There is no `-I/usr/local/lmdb/include` here

Post by PGNet Dev
echo $CPPFLAGS
-I/usr/local/include -I/usr/local/openssl11/include -I/usr/local/lmdb/include -I/usr/include

I'd imagine you want CPPFLAGS here. or you can export C_INCLUDE_PATH



Matt Pallissard

[Matt Pallissard]

Post by Matt Pallissard
Response in-line.

Post by PGNet Dev
...
libtool: compile: /usr/bin/gcc-10 -include /usr/local/src/bind-9.16.4/config.h -I/usr/local/src/bind-9.16.4 -I../.. -I./include -I./unix/include -I. -I/usr/local/src/bind-9.16.4/lib/ns/include -I../../lib/ns/include -I/usr/local/src/bind-9.16.4/lib/dns/include -I../../lib/dns/include -I/usr/local/openssl11/include -I/usr/local/src/bind-9.16.4/lib/bind9/include -I../../lib/bind9/include -I/usr/local/src/bind-9.16.4/lib/isccfg/include -I../../lib/isccfg/include -I/usr/local/src/bind-9.16.4/lib/isccc/include -I../../lib/isccc/include -I/usr/local/src/bind-9.16.4/lib/isc/include -I../../lib/isc -I../../lib/isc/include -I../../lib/isc/unix/include -I../../lib/isc/pthreads/include -I/usr/local/openssl11/include -I/usr/include/json-c -I/usr/include/libxml2 -I/usr/local/include -O3 -Wall -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fmessage-length=0 -grecord-gcc-switches -march=native -mtune=native -fPIC -DPIC -D_GNU_SOURCE -fno-strict-aliasing -Wall -pthread -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -Wno-missing-field-initializers -fno-strict-aliasing -DPRODUCT=\"BIND\" -DVERSION=\"9.16.4\" -c ./server.c -fPIC -DPIC -o .libs/server.o

There is no `-I/usr/local/lmdb/include` here

Post by PGNet Dev
echo $CPPFLAGS
-I/usr/local/include -I/usr/local/openssl11/include -I/usr/local/lmdb/include -I/usr/include

I'd imagine you want CPPFLAGS here. or you can export C_INCLUDE_PATH

Edit:

`s/CPPFLAGS/CFLAGS`

Matt Pallissrd

[PGNet Dev]

Post by Matt Pallissard
I'd imagine you want CPPFLAGS here. or you can export C_INCLUDE_PATH

...
`s/CPPFLAGS/CFLAGS`


specifically adding '-I/usr/local/lmdb/include' to $CFLAGS makes the build unhappy, here.

but, yep! ... there _is_ hoop-jumping that appears to workaround the WONTFIX silliness^^ ...

with lmdb installed as

tree /usr/local/lmdb
/usr/local/lmdb
├── bin
│   ├── mdb_copy
│   ├── mdb_drop
│   ├── mdb_dump
│   ├── mdb_load
│   └── mdb_stat
├── include
│   └── lmdb.h
├── lib
├── lib64
│   ├── liblmdb.a
│   └── liblmdb.so
├── man
└── share
└── man
└── man1
├── mdb_copy.1
├── mdb_drop.1
├── mdb_dump.1
├── mdb_load.1
└── mdb_stat.1

setting @ bind configure

--with-lmdb=/usr/local/lmdb

to avoid

checking for lmdb library... configure: error: include/lmdb.h not found.

and to ensure the feature is (at least) _reported_ as enabled

...
===============================================================================
Configuration summary:
-------------------------------------------------------------------------------
Optional features enabled:
...
LMDB database to store configuration for 'addzone' zones (--with-lmdb)
...
-------------------------------------------------------------------------------
Features disabled or unavailable on this platform:
...

then just this

CPPFLAGS= ... -I/usr/local/lmdb/include ...

LDFLAGS=... -L/usr/local/lmdb/lib64 -llmdb ...

is sufficient add'n to general flags

still to make add'l adjustments.
namely, to just completely circumvent the src code mess that results in,

LMDB_CFLAGS = -I/usr/local/lmdb/include
LMDB_LIBS = -L/usr/local/lmdb/lib none required

and build breakage, applying to clean source,

patch << 'EOF'
--- configure.ac.ORIG 2020-06-10 14:01:43.000000000 -0700
+++ configure.ac 2020-06-18 09:45:42.142523855 -0700
@@ -1274,6 +1274,8 @@
else
AC_MSG_RESULT(no)
fi
+LMDB_CFLAGS="-I/usr/local/lmdb/include"
+LMDB_LIBS="-L/usr/local/lmdb/lib64 -llmdb"
AC_SUBST([LMDB_CFLAGS])
AC_SUBST([LMDB_LIBS])
EOF

then, of course,

./autogen.sh

now, after config, correctly

s|^LMDB_CFLAGS =.*|LMDB_CFLAGS = -I/usr/local/lmdb/include|g
s|^LMDB_LIBS =.*|LMDB_LIBS = -L/usr/local/lmdb/lib64 -llmdb|g

on subsequent build, there's no longer any 'fatal error',

In file included from server.c:131:
/usr/local/src/bind-9.16.4/lib/dns/include/dns/lmdb.h:12:10: fatal error: lmdb.h: No such file or directory
12 | #include <lmdb.h>
| ^~~~~~~~
compilation terminated.

and, on install, named runs/functions

named -v
BIND 9.16.4 (Stable Release) <id:0849b42>

lmdb lib is linked,

ldd `which named` | grep lmdb
liblmdb.so => /usr/local/lmdb/lib64/liblmdb.so (0x00007fedf088b000)

but whether lmdb inclusion/usage now _functions_ correctly with that^ cleanup, i have yet to verify.
checking now ...

[PGNet Dev]

this

configure.ac
AC_MSG_CHECKING(for lmdb library)
AC_ARG_WITH(lmdb,
AS_HELP_STRING([--with-lmdb[=PATH]],
[build with LMDB library [yes|no|path]]),
use_lmdb="$withval", use_lmdb="auto")

have_lmdb=""
case "$use_lmdb" in
no)
LMDB_LIBS=""
;;
auto|yes)
for d in /usr /usr/local /opt/local
do
if test -f "${d}/include/lmdb.h"
then
if test ${d} != /usr
then
LMDB_CFLAGS="-I ${d}/include"
!! LMDB_LIBS="-L${d}/lib"
fi
have_lmdb="yes"
fi
done
;;
*)
if test -f "${use_lmdb}/include/lmdb.h"
then
LMDB_CFLAGS="-I${use_lmdb}/include"
!! LMDB_LIBS="-L${use_lmdb}/lib"
have_lmdb="yes"
else
AC_MSG_ERROR([$use_lmdb/include/lmdb.h not found.])
fi
;;
esac


is naive. it ignores commonplace multiarch support.



with =path enabled on config, generated Makefile is just wrong

egrep "^LMDB" Makefile
LMDB_CFLAGS = -I/usr/local/lmdb/include
LMDB_LIBS = -L/usr/local/lmdb/lib none required



bind advertises

./configure --help | grep -i LMDB
--with-lmdb=PATH build with LMDB library [yes|no|path]

it simply fails to work as advertised.

Post by Ondřej Surý
you can use and adjust the .pc file from any Linux distribution package

of course you can.

it's here

pkg-config --libs --cflags lmdb
-I/usr/local/lmdb/include -L/usr/local/lmdb/lib64 -llmdb

Post by Ondřej Surý
LMDB support in BIND 9 comes with pkg-config support too

there's NO such option advertised by bind configure, as

LMDB_CFLAGS
C compiler flags for LMDB, overriding pkg-config
LMDB_LIBS
linker flags for LMDB, overriding pkg-config

it's simply not used by this bind build

grep PKG_CHECK `grep -rln PKG_CHECK .` | grep -i lmdb
(empty)

& no mention at all of it in source

grep -i lmdb `grep -rlni lmdb .` | grep -i pkg
(empty)

this

--with-openssl=/usr/local/openssl

works.
this,

--with-lmdb=/usr/local/lmdb

does not.

That has _nothing_ to do with 'LMDB upstream'

Post by Ondřej Surý
Did you came to get help or to argue?

neither.

Post by Ondřej Surý
It’s really not our fault the LMDB upstream decided to make it hard to use the library.

::facepalm::


this

tree /usr/local/lmdb
/usr/local/lmdb
...
├── include
│   └── lmdb.h
...
├── lib64
│   ├── liblmdb.a
│   └── liblmdb.so
...

is about as simple as it gets.

fix it, remove it, or leave it as is -- your product, your choice.

good luck.

[Mark Andrews]

${LMDB_CFLAGS} is missing from make/includes.in or bin/named/Makefile.in

diff --git a/make/includes.in b/make/includes.in
index 9ff1bd8cf1..9eaf1d3dd5 100644
--- a/make/includes.in
+++ b/make/includes.in
@@ -28,7 +28,7 @@ ISCCFG_INCLUDES = @BIND9_ISCCFG_BUILDINCLUDE@ \
DNS_INCLUDES = @BIND9_DNS_BUILDINCLUDE@ \
-I${top_srcdir}/lib/dns/include \
${FSTRM_CFLAGS} ${PROTOBUF_C_CFLAGS} \
- ${OPENSSL_CFLAGS}
+ ${OPENSSL_CFLAGS} ${LMDB_CFLAGS}

NS_INCLUDES = @BIND9_NS_BUILDINCLUDE@ \
-I${top_srcdir}/lib/ns/include

As for ‘lib64’ vs ‘lib' the complier should adjust the -L paths to match the architecture as needed usually by appending “/../lib64” to account for given path being a symbolic link.

The pkg-config output is architecture dependent and it should be architecture independent.

Post by PGNet Dev
this
configure.ac
AC_MSG_CHECKING(for lmdb library)
AC_ARG_WITH(lmdb,
AS_HELP_STRING([--with-lmdb[=PATH]],
[build with LMDB library [yes|no|path]]),
use_lmdb="$withval", use_lmdb="auto")
have_lmdb=""
case "$use_lmdb" in
no)
LMDB_LIBS=""
;;
auto|yes)
for d in /usr /usr/local /opt/local
do
if test -f "${d}/include/lmdb.h"
then
if test ${d} != /usr
then
LMDB_CFLAGS="-I ${d}/include"
!! LMDB_LIBS="-L${d}/lib"
fi
have_lmdb="yes"
fi
done
;;
*)
if test -f "${use_lmdb}/include/lmdb.h"
then
LMDB_CFLAGS="-I${use_lmdb}/include"
!! LMDB_LIBS="-L${use_lmdb}/lib"
have_lmdb="yes"
else
AC_MSG_ERROR([$use_lmdb/include/lmdb.h not found.])
fi
;;
esac
is naive. it ignores commonplace multiarch support.
with =path enabled on config, generated Makefile is just wrong
egrep "^LMDB" Makefile
LMDB_CFLAGS = -I/usr/local/lmdb/include
LMDB_LIBS = -L/usr/local/lmdb/lib none required
bind advertises
./configure --help | grep -i LMDB
--with-lmdb=PATH build with LMDB library [yes|no|path]
it simply fails to work as advertised.

Post by Ondřej Surý
you can use and adjust the .pc file from any Linux distribution package

of course you can.
it's here
pkg-config --libs --cflags lmdb
-I/usr/local/lmdb/include -L/usr/local/lmdb/lib64 -llmdb

Post by Ondřej Surý
LMDB support in BIND 9 comes with pkg-config support too

there's NO such option advertised by bind configure, as
LMDB_CFLAGS
C compiler flags for LMDB, overriding pkg-config
LMDB_LIBS
linker flags for LMDB, overriding pkg-config
it's simply not used by this bind build
grep PKG_CHECK `grep -rln PKG_CHECK .` | grep -i lmdb
(empty)
& no mention at all of it in source
grep -i lmdb `grep -rlni lmdb .` | grep -i pkg
(empty)
this
--with-openssl=/usr/local/openssl
works.
this,
--with-lmdb=/usr/local/lmdb
does not.
That has _nothing_ to do with 'LMDB upstream'

Post by Ondřej Surý
Did you came to get help or to argue?

neither.

Post by Ondřej Surý
It’s really not our fault the LMDB upstream decided to make it hard to use the library.

this
tree /usr/local/lmdb
/usr/local/lmdb
...
├── include
│ └── lmdb.h
...
├── lib64
│ ├── liblmdb.a
│ └── liblmdb.so
...
is about as simple as it gets.
fix it, remove it, or leave it as is -- your product, your choice.
good luck.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users

[PGNet Dev]

Post by PGNet Dev
but whether lmdb inclusion/usage now _functions_ correctly with that^ cleanup, i have yet to verify.
checking now ...

iiuc @

https://bind9.readthedocs.io/en/latest/manpages.html
"The configuration is saved in a file called viewname.nzf (or, if named(8) is compiled with liblmdb, an LMDB database file called viewname.nzd)."


the 'nzd2nzf'

system/nzd2nzf/prereq.sh
if [ -z "$NZD" ]; then
echo_i "This test requires LMDB support (--with-lmdb)"
exit 255
fi

tests that lmdb function -- 1st to create, then convert, the .nzd. complete? dunno ...

in any case, with that^ workaround, lmdb test looks ok

sh run.sh nzd2nzf
S:nzd2nzf:2020-06-18T22:10:47-0700
T:nzd2nzf:1:A
A:nzd2nzf:System test nzd2nzf
I:nzd2nzf:PORTRANGE:5300 - 5399
I:nzd2nzf:starting servers
I:nzd2nzf:querying for non-existing zone data (1)
I:nzd2nzf:adding a new zone into default NZD using rndc addzone (2)
I:nzd2nzf:querying for existing zone data (3)
I:nzd2nzf:stopping ns1
I:nzd2nzf:dumping _default.nzd to _default.nzf (4)
I:nzd2nzf:checking that _default.nzf contains the expected content (5)
I:nzd2nzf:deleting _default.nzd database
I:nzd2nzf:starting ns1 which should migrate the .nzf to .nzd
I:nzd2nzf:querying for zone data from migrated zone config (6)
I:nzd2nzf:exit status: 0
I:nzd2nzf:stopping servers
R:nzd2nzf:PASS
E:nzd2nzf:2020-06-18T22:10:54-0700