Compile fails when exiv2 is not installed #433

D3vil0p3r · 2023-12-31T13:35:34Z

I'm trying to compile bulk_extractor on NixOS but I'm getting the following error during make:

g++ -std=c++17 -DHAVE_CONFIG_H -I. -I..  -I../src/be20_api -I../src/be20_api/utfcpp/source -DUTC_OFFSET=+0000  -g  -pthread -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -Weffc++ -fPIC -O3 -MT bulk_extractor.o -MD -MP -MF $depbase.Tpo -c -o bulk_extractor.o bulk_extractor.cpp &&\
mv -f $depbase.Tpo $depbase.Po
In file included from bulk_extractor.cpp:49:
bulk_extractor_restarter.h: In member function ‘void bulk_extractor_restarter::restart(Phase1::Config&, scanner_config&)’:
bulk_extractor_restarter.h:91:55: warning: declaration of ‘sc’ shadows a member of ‘bulk_extractor_restarter’ [-Wshadow]
   91 |     void restart(Phase1::Config &cfg, scanner_config &sc) {
      |                                       ~~~~~~~~~~~~~~~~^~
bulk_extractor_restarter.h:24:30: note: shadowed declaration is here
   24 |     scanner_config          &sc;
      |                              ^~
bulk_extractor_restarter.h:91:34: warning: declaration of ‘cfg’ shadows a member of ‘bulk_extractor_restarter’ [-Wshadow]
   91 |     void restart(Phase1::Config &cfg, scanner_config &sc) {
      |                  ~~~~~~~~~~~~~~~~^~~
bulk_extractor_restarter.h:25:30: note: shadowed declaration is here
   25 |     Phase1::Config          &cfg;
      |                              ^~~
bulk_extractor.cpp: In function ‘int bulk_extractor_main(std::ostream&, std::ostream&, int, char* const*)’:
bulk_extractor.cpp:485:22: error: no matching function for call to ‘bulk_extractor_restarter::restart()’
  485 |             r.restart();                    // load the restart file and rename report.xml
      |             ~~~~~~~~~^~
bulk_extractor_restarter.h:91:10: note: candidate: ‘void bulk_extractor_restarter::restart(Phase1::Config&, scanner_config&)’
   91 |     void restart(Phase1::Config &cfg, scanner_config &sc) {
      |          ^~~~~~~
bulk_extractor_restarter.h:91:10: note:   candidate expects 2 arguments, 0 provided
make[2]: *** [Makefile:1377: bulk_extractor.o] Error 1
make[2]: Leaving directory '/home/athena/pkgs/bulk-extractor/bulk_extractor/src'
make[1]: *** [Makefile:411: all-recursive] Error 1
make[1]: Leaving directory '/home/athena/pkgs/bulk-extractor/bulk_extractor'
make: *** [Makefile:352: all] Error 2

The text was updated successfully, but these errors were encountered:

D3vil0p3r · 2023-12-31T13:35:55Z

@simsong

D3vil0p3r · 2023-12-31T13:47:17Z

Solved by adding afflib and exiv2 as dependencies. It could be useful to have them in README or BUILD instructions.

simsong · 2023-12-31T13:53:07Z

Afflib is no longer supported and should not be used.
Where are the dependencies listed that you did not find them? Exiv should automatically detect and if it is not present the configure script shouldn't have included it.

Are you compiling from sources? It would be nice to fix this so that it doesn't cause other people problems.

D3vil0p3r · 2023-12-31T14:01:57Z

I took the list of dependencies from this BlackArch PKGBUILD. I'm compiling from source by downloading the .tar.gz from https://digitalcorpora.s3.amazonaws.com/downloads/bulk_extractor/bulk_extractor-2.0.3.tar.gz

I did a further test. I tried to remove afflib dependency and the compilation still works. If I remove exiv2, I get the error reported above. So I think that exiv2, at least in NixOS, should be necessary.

Furthermore, for BEViewer, is there a specific java version to install? For example jdk11, jdk17 or jdk19?

simsong · 2023-12-31T14:08:27Z

exiv2 is not required, but if it is not there the jpeg module won’t decide them. So the question is why did autoconf think exiv was present even when it wasn’t. Where should these requirements be documented?

…

_____________

On Sun, Dec 31, 2023 at 9:02 AM D3vil0p3r ***@***.***> wrote: I took the list of dependencies from this BlackArch PKGBUILD <https://github.com/BlackArch/blackarch/blob/master/packages/bulk-extractor/PKGBUILD>. I'm compiling from source by downloading the .tar.gz from https://digitalcorpora.s3.amazonaws.com/downloads/bulk_extractor/bulk_extractor-2.0.3.tar.gz I did a further test. I tried to remove afflib dependency and the compilation still works. If I remove exiv2, I get the error reported above. So I think that exiv2, at least in NixOS, should be necessary. — Reply to this email directly, view it on GitHub <#433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFHLAF6W4RD6ZLDO7YWRLYMFV6BAVCNFSM6AAAAABBIIN6AWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSHE2TKOJTGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

D3vil0p3r · 2023-12-31T14:11:25Z

My only references where that PKGBUILD I linked you before and this .nix file of an old version of bulk_extractor: NixOS/nixpkgs#82265 . On your repository, I referred to https://github.com/simsong/bulk_extractor/blob/main/etc/CONFIGURE_FEDORA36.bash to know what are the dependencies to use.

simsong · 2023-12-31T14:24:23Z

But those files aren’t part of the bulk_extractor release. I don’t have any control over those files. I don’t even know what NixOS is. I’ve never used it. You complained to me, and I am happy to fix this, but I don’t know how. Can you please provide me with a file that I can include in the release that would resolve your problem? If not, can you please file your bug report elsewhere?On Dec 31, 2023, at 9:11 AM, D3vil0p3r ***@***.***> wrote: My only references where that PKGBUILD I linked you before and this .nix file of an old version of bulk_extractor: NixOS/nixpkgs#82265 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

jonstewart · 2023-12-31T14:43:49Z

For [hopefully helpful] context, NixOS is a distro of Linux that uses the nix package manager for controlling the configuration of the system. The nix package manager constructs a Merkle tree of the software packages installed, where both content of source files and configuration options feed into the overall hash of the package. nix then uses an elaborate system of symlinks to set one's PATH and other environment variables so that the environment is somewhat normal. I've played with it a bit a couple years ago. It's neat but (then) was ultimately too frustrating for me to use. The benefit is that everything about the system is well-specified and easily replicated from a very small set of Haskell-based configuration scripts; builds are fully reproducible.

Given the degree to which nix buggers the normal Linux directory structure and environment, the inherent complexity therein, the number of dependencies of bulk_extractor, and the paleolithic technology that is autoconf, it's not surprising that there's a build problem with bulk_extractor on NixOS. Simson, you are correct that the focus should be on configure.ac and what its behavior is when exiv2 isn't found.

simsong · 2023-12-31T14:52:00Z

Yes, I'm concerned that it didn't work when exiv2 wasn't installed.

D3vil0p3r · 2023-12-31T15:07:19Z

Another small non-impacting note is that, at the end of building, also a test_be file is generated inside bin directory (accessible by PATH). Would be cleaner if test_be won't be automatically stored in bin directory?

simsong · 2023-12-31T15:08:51Z

Well, it's a binary. It just shouldn't be installed. Why is it getting installed on your system? It doesn't get installed on mine, the last time I checked.

D3vil0p3r · 2023-12-31T15:25:27Z

Well, it's a binary. It just shouldn't be installed. Why is it getting installed on your system? It doesn't get installed on mine, the last time I checked.

I'm investigating, I think it could be related how nix works. Is there a make config to prevent the creation of this test_be?

simsong · 2023-12-31T15:30:48Z

Looks like test_be is specified as a program and there is no install target. If you want to send a PR, I'm happy to take it. It may be that test_be doesn't need to be listed as a bin program if it is listed as a check program. https://github.com/simsong/bulk_extractor/blob/22da4f4f36040e854cdf8a4900146574f25e35f2/src/Makefile.am#L1

…

On Sun, Dec 31, 2023 at 10:25 AM D3vil0p3r ***@***.***> wrote: Well, it's a binary. It just shouldn't be installed. Why is it getting installed on your system? It doesn't get installed on mine, the last time I checked. I'm investigating, I think it could be related how nix works. Is there a make config to prevent the creation of this test_be? — Reply to this email directly, view it on GitHub <#433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFHLBNEPEE3KPIXXPUIS3YMF7XDAVCNFSM6AAAAABBIIN6AWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSHE3TKMBUGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

--

_____________

D3vil0p3r · 2023-12-31T15:32:46Z

What I see in bulk_extractor/src folder after make command in the root repo folder is that, inside generated Makefile, we have:

EXEEXT = 
...
bin_PROGRAMS = bulk_extractor$(EXEEXT) test_be$(EXEEXT)
...
install-binPROGRAMS: $(bin_PROGRAMS)
        @$(NORMAL_INSTALL)
        @list='$(bin_PROGRAMS)'; test -n "$(bindir)" || list=; \
        if test -n "$$list"; then \
          echo " $(MKDIR_P) '$(DESTDIR)$(bindir)'"; \
          $(MKDIR_P) "$(DESTDIR)$(bindir)" || exit 1; \
        fi; \
        for p in $$list; do echo "$$p $$p"; done | \
        sed 's/$(EXEEXT)$$//' | \
        while read p p1; do if test -f $$p \
          ; then echo "$$p"; echo "$$p"; else :; fi; \
        done | \
        sed -e 'p;s,.*/,,;n;h' \
            -e 's|.*|.|' \
            -e 'p;x;s,.*/,,;s/$(EXEEXT)$$//;$(transform);s/$$/$(EXEEXT)/' | \
        sed 'N;N;N;s,\n, ,g' | \
        $(AWK) 'BEGIN { files["."] = ""; dirs["."] = 1 } \
          { d=$$3; if (dirs[d] != 1) { print "d", d; dirs[d] = 1 } \
            if ($$2 == $$4) files[d] = files[d] " " $$1; \
            else { print "f", $$3 "/" $$4, $$1; } } \
          END { for (d in files) print "f", d, files[d] }' | \
        while read type dir files; do \
            if test "$$dir" = .; then dir=; else dir=/$$dir; fi; \
            test -z "$$files" || { \
              echo " $(INSTALL_PROGRAM_ENV) $(INSTALL_PROGRAM) $$files '$(DESTDIR)$(bindir)$$dir'"; \
              $(INSTALL_PROGRAM_ENV) $(INSTALL_PROGRAM) $$files "$(DESTDIR)$(bindir)$$dir" || exit $$?; \
            } \
        ; done
...

so I guess that when sudo make install is run, the function install-binPROGRAMS is run and I think it stores test_be in dest bin dir.

D3vil0p3r · 2023-12-31T15:37:38Z

Since in Makefile, test_be is already specified as check_PROGRAMS, the only change should be to remove test_be from bin_PROGRAMS, right? If so, I can open a PR for it.

simsong · 2023-12-31T15:56:17Z

I don’t know. If you want to give it a try and, if that works, send me a PR, that would be great.On Dec 31, 2023, at 10:37 AM, D3vil0p3r ***@***.***> wrote: Since in Makefile, test_be is already specified as check_PROGRAMS, the only change should be to remove test_be from bin_PROGRAMS, right? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

D3vil0p3r · 2023-12-31T16:12:40Z

I tested and the change does not erroneously generate test_be in bin anymore. Here the PR: #434

Just for info, for this small PR will be a small update to a stable version like 2.0.4?

simsong · 2024-01-14T18:45:08Z

bulk_extractor_restarter.h:24:30: note: shadowed declaration is here

This will be fixed in an upcoming PR

D3vil0p3r closed this as completed Dec 31, 2023

D3vil0p3r reopened this Dec 31, 2023

simsong changed the title ~~Compilation error~~ Compile fails when exiv2 is not installed Dec 31, 2023

simsong mentioned this issue Jan 14, 2024

fix shadowing issue in #433 #436

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compile fails when exiv2 is not installed #433

Compile fails when exiv2 is not installed #433

D3vil0p3r commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

simsong commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

simsong commented Dec 31, 2023 via email

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

simsong commented Dec 31, 2023 via email

jonstewart commented Dec 31, 2023

simsong commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

simsong commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

simsong commented Dec 31, 2023 via email

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

simsong commented Dec 31, 2023 via email

D3vil0p3r commented Dec 31, 2023

simsong commented Jan 14, 2024

Compile fails when exiv2 is not installed #433

Compile fails when exiv2 is not installed #433

Comments

D3vil0p3r commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

simsong commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023 • edited Loading

simsong commented Dec 31, 2023 via email

D3vil0p3r commented Dec 31, 2023 • edited Loading

simsong commented Dec 31, 2023 via email

jonstewart commented Dec 31, 2023

simsong commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

simsong commented Dec 31, 2023

D3vil0p3r commented Dec 31, 2023

simsong commented Dec 31, 2023 via email

D3vil0p3r commented Dec 31, 2023 • edited Loading

D3vil0p3r commented Dec 31, 2023 • edited Loading

simsong commented Dec 31, 2023 via email

D3vil0p3r commented Dec 31, 2023

simsong commented Jan 14, 2024

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

D3vil0p3r commented Dec 31, 2023 •

edited

Loading

D3vil0p3r commented Dec 31, 2023 •

edited

Loading