Make bulk_extractor compile under Debian #452

simsong · 2024-01-26T02:07:14Z

@zdavatz -This is for you.
@simsong will use this distribution: https://aws.amazon.com/marketplace/pp/prodview-fznsw3f7mq7to

zdavatz · 2024-01-26T08:25:51Z

Yes, also on Gentoo the compilation from source is not straight forward.

simsong · 2024-01-27T02:36:06Z

@zdavatz - Can you try out b1e500f?

zdavatz · 2024-01-27T19:18:26Z

sure! Can I just do a git pull for the latest update?

I done a git clone this time. Now I get:

~/.software/bulk_extractor> ./bootstrap.sh
submodule be20_api is not present.

simsong · 2024-01-27T19:40:32Z

When you do a git clone, you need to add the --recursive flag, because there are submodules.

Alternatively, you can do a git submodule init and git submodule update

zdavatz · 2024-01-28T12:11:50Z

I done that all, but still no luck:

~/.software/bulk_extractor> git submodule init
~/.software/bulk_extractor> git submodule update
Klone nach '/home/zeno/.software/bulk_extractor/dfxml_schema'...
Klone nach '/home/zeno/.software/bulk_extractor/src/be20_api'...
Submodul-Pfad 'dfxml_schema': 'f2a702e401dd15802ec8558b42c496a1184a0486' ausgecheckt
Submodul-Pfad 'src/be20_api': 'c031194542beed802c6303ac7c50904d10a5b67e' ausgecheckt
~/.software/bulk_extractor> ./bootstrap.sh
submodule be20_api/dfxml_cpp is not present.

zdavatz · 2024-01-28T12:14:34Z

Ok, this worked:
git clone --recursive https://github.com/simsong/bulk_extractor.git

this did not work:

git submodule init
git submodule update

simsong · 2024-01-28T12:16:11Z

Hm. that has a sub module too. And then that sub module has another sub module. Which is why I recommend a git clone recursive. I think that the init and update commands also have recurse flags but that they are different.

…

_____________

On Sun, Jan 28, 2024 at 7:12 AM Zeno R.R. Davatz ***@***.***> wrote: I done that all, but still no luck: ~/.software/bulk_extractor> git submodule init ~/.software/bulk_extractor> git submodule update Klone nach '/home/zeno/.software/bulk_extractor/dfxml_schema'... Klone nach '/home/zeno/.software/bulk_extractor/src/be20_api'... Submodul-Pfad 'dfxml_schema': 'f2a702e401dd15802ec8558b42c496a1184a0486' ausgecheckt Submodul-Pfad 'src/be20_api': 'c031194542beed802c6303ac7c50904d10a5b67e' ausgecheckt ~/.software/bulk_extractor> ./bootstrap.sh submodule be20_api/dfxml_cpp is not present. — Reply to this email directly, view it on GitHub <#452 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFHLFAAI3GAS34EYRUZ63YQY6BDAVCNFSM6AAAAABCLPEOR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTGU3TKNJXGI> . You are receiving this because you were assigned.Message ID: ***@***.***>

zdavatz · 2024-01-28T12:18:22Z

Ok, configure goes through, but make -j9 still fails.

 495 |         if ( std::filesystem::exists( sc.outdir/"report.xml" )){
      |                   ^~~~~~~~~~
bulk_extractor.cpp:495:42: Fehler: »struct scanner_config« hat kein Element namens »outdir«
  495 |         if ( std::filesystem::exists( sc.outdir/"report.xml" )){
      |                                          ^~~~~~
bulk_extractor.cpp:504:37: Fehler: »struct scanner_config« hat kein Element namens »input_fname«
  504 |         p = image_process::open( sc.input_fname, cfg.opt_recurse, cfg.opt_pagesize, cfg.opt_marginsize );
      |                                     ^~~~~~~~~~~
bulk_extractor.cpp:504:104: Fehler: »image_process::open« kann nicht als Funktion verwendet werden
  504 |         p = image_process::open( sc.input_fname, cfg.opt_recurse, cfg.opt_pagesize, cfg.opt_marginsize );
      |                                                                                                        ^
bulk_extractor.cpp:507:73: Fehler: »struct scanner_config« hat kein Element namens »input_fname«
  507 |         cerr << "error: file " << e.what() << " is in directory " << sc.input_fname << std::endl;
      |                                                                         ^~~~~~~~~~~
bulk_extractor.cpp:511:71: Fehler: »struct scanner_config« hat kein Element namens »input_fname«
  511 |         cerr << "       then place them in a sub directory of " << sc.input_fname << std::endl;
      |                                                                       ^~~~~~~~~~~
bulk_extractor.cpp:539:50: Fehler: »struct scanner_config« hat kein Element namens »outdir«
  539 |     dfxml_writer *xreport = new dfxml_writer( sc.outdir / Phase1::REPORT_FILENAME, false ); // do not make DTD
      |                                                  ^~~~~~
bulk_extractor.cpp:547:23: Fehler: »struct scanner_config« hat kein Element namens »input_fname«
  547 |     validate_path( sc.input_fname );
      |                       ^~~~~~~~~~~
bulk_extractor.cpp:547:5: Fehler: »validate_path« wurde in diesem Gültigkeitsbereich nicht definiert
  547 |     validate_path( sc.input_fname );
      |     ^~~~~~~~~~~~~
bulk_extractor.cpp:565:38: Fehler: »struct scanner_config« hat kein Element namens »input_fname«
  565 |         cout << "Input file: " << sc.input_fname << std::endl ;
      |                                      ^~~~~~~~~~~
bulk_extractor.cpp:566:44: Fehler: »struct scanner_config« hat kein Element namens »outdir«
  566 |         cout << "Output directory: " << sc.outdir << std::endl ;
      |                                            ^~~~~~
bulk_extractor.cpp:600:46: Fehler: »struct scanner_config« hat kein Element namens »input_fname«
  600 |     xreport->xmlout( "provided_filename", sc.input_fname ); // save this information
      |                                              ^~~~~~~~~~~
make[2]: *** [Makefile:1438: scan_aes.o] Fehler 1
make[2]: *** [Makefile:1438: bulk_extractor_scanners.o] Fehler 1
make[2]: *** [Makefile:1438: notify_thread.o] Fehler 1
make[2]: *** [Makefile:1438: bulk_extractor.o] Fehler 1
make[2]: Verzeichnis „/home/zeno/.software/bulk_extractor/src“ wird verlassen
make[1]: *** [Makefile:526: all-recursive] Fehler 1
make[1]: Verzeichnis „/home/zeno/.software/bulk_extractor“ wird verlassen
make: *** [Makefile:466: all] Fehler 2

zdavatz · 2024-01-28T12:19:16Z

@zdavatz - Can you try out b1e500f?

this commit is in the latest git pull right?

My repo is here: 88c2d72

zdavatz · 2024-01-28T12:22:25Z

I do not have libewf installed, but AFAICT that should not be a problem.

simsong · 2024-01-28T12:51:26Z

Correct, you should not need libewf.

It appears that git submodule update has a --recursive flag but git init does not. So you will need to use the git submodule foreach --recursive git init command (or something like that in addition to git init if you do not wish to do a new clone with git clone --recursive, which is what I recommend.

zdavatz · 2024-01-28T19:07:43Z

ok, git clone --recursive works but I still get a make error: #452 (comment)

simsong · 2024-01-28T19:47:04Z

Try switching to the branch rel-2.1.1 ?

simsong · 2024-01-28T19:50:24Z

Can you post your full config.log file?
I can try to spin up a Kali Linux somewhere. Is it available on AWS?
Is there a way to run Kali linux on Github actions?
Kali Linux is based on Debian. There is no github actions running that provides debian linux. Do you know if I can run it under docker?

zdavatz · 2024-01-29T07:57:28Z

Can you post your full config.log file? I can try to spin up a Kali Linux somewhere. Is it available on AWS? Is there a way to run
Kali linux on Github actions? Kali Linux is based on Debian. There is no github actions running that provides debian linux. Do
you know if I can run it under docker?

you can definitely do a github actions build from source. Config log is attached.

config.log

Configure runs fine for me.

simsong · 2024-01-29T10:06:23Z

I have GitHub actions that build from source, but the GitHub runners are Ubuntu, not Debian. Do you have an example of setting up Debian with docker to run an action?

…

_____________

On Mon, Jan 29, 2024 at 2:57 AM Zeno R.R. Davatz ***@***.***> wrote: Can you post your full config.log file? I can try to spin up a Kali Linux somewhere. Is it available on AWS? Is there a way to run Kali linux on Github actions? Kali Linux is based on Debian. There is no github actions running that provides debian linux. Do you know if I can run it under docker? you can definitely do a github actions build from source. Config log is attached. config.log <https://github.com/simsong/bulk_extractor/files/14081257/config.log> Configure runs fine for me. — Reply to this email directly, view it on GitHub <#452 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFHLF5IYYWZZACUHTH7OTYQ5I7HAVCNFSM6AAAAABCLPEOR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJUGE2DQNBQGU> . You are receiving this because you were assigned.Message ID: ***@***.***>

zdavatz · 2024-01-29T10:19:08Z

does this help:
https://github.com/sdolenc/debian-matrix/blob/master/.circleci/config.yml
https://github.com/sdolenc/debian-matrix

simsong · 2024-01-29T10:37:15Z

Thanks. It does.
I looked through your config file. You are compiling with C++17. Nothing obviously is wrong. I'll give this a whirl and see what happens.

zdavatz · 2024-01-29T12:41:41Z

Thank you!

simsong · 2024-01-30T00:36:04Z

I'm trying to compile under a debian container and I can't get that to work either. This is a deeper problem. Somebody who knows Debian or Kali Linux, which is based on Debian, needs to fix this.

zdavatz · 2024-01-30T06:49:27Z

On which Linux do you normally build?

simsong · 2024-01-30T11:32:51Z

We build on both Ubuntu with amazon actions. Fedora to build the windows executable. Also Amazon Linux. Debian has always been a mystery to me.

…

_____________

On Tue, Jan 30, 2024 at 1:49 AM Zeno R.R. Davatz ***@***.***> wrote: On which Linux do you normally build? — Reply to this email directly, view it on GitHub <#452 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFHLEU5SPRKC5BJBO7KALYRCJYDAVCNFSM6AAAAABCLPEOR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJWGE4DGMBUGM> . You are receiving this because you were assigned.Message ID: ***@***.***>

simsong added the bug label Jan 26, 2024

simsong self-assigned this Jan 26, 2024

This was referenced Jan 27, 2024

add extra help simsong/meeting_notifier#1

Merged

add extra help #455

Merged

simsong changed the title ~~Make bulk_extractor compile under Kali Linux~~ Make bulk_extractor compile under Debian Jan 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make bulk_extractor compile under Debian #452

Make bulk_extractor compile under Debian #452

simsong commented Jan 26, 2024 •

edited

Loading

zdavatz commented Jan 26, 2024

simsong commented Jan 27, 2024

zdavatz commented Jan 27, 2024 •

edited

Loading

simsong commented Jan 27, 2024 •

edited

Loading

zdavatz commented Jan 28, 2024

zdavatz commented Jan 28, 2024

simsong commented Jan 28, 2024 via email

zdavatz commented Jan 28, 2024

zdavatz commented Jan 28, 2024

zdavatz commented Jan 28, 2024

simsong commented Jan 28, 2024

zdavatz commented Jan 28, 2024

simsong commented Jan 28, 2024

simsong commented Jan 28, 2024

zdavatz commented Jan 29, 2024

simsong commented Jan 29, 2024 via email

zdavatz commented Jan 29, 2024 •

edited

Loading

simsong commented Jan 29, 2024

zdavatz commented Jan 29, 2024

simsong commented Jan 30, 2024

zdavatz commented Jan 30, 2024

simsong commented Jan 30, 2024 via email

Make bulk_extractor compile under Debian #452

Make bulk_extractor compile under Debian #452

Comments

simsong commented Jan 26, 2024 • edited Loading

zdavatz commented Jan 26, 2024

simsong commented Jan 27, 2024

zdavatz commented Jan 27, 2024 • edited Loading

simsong commented Jan 27, 2024 • edited Loading

zdavatz commented Jan 28, 2024

zdavatz commented Jan 28, 2024

simsong commented Jan 28, 2024 via email

zdavatz commented Jan 28, 2024

zdavatz commented Jan 28, 2024

zdavatz commented Jan 28, 2024

simsong commented Jan 28, 2024

zdavatz commented Jan 28, 2024

simsong commented Jan 28, 2024

simsong commented Jan 28, 2024

zdavatz commented Jan 29, 2024

simsong commented Jan 29, 2024 via email

zdavatz commented Jan 29, 2024 • edited Loading

simsong commented Jan 29, 2024

zdavatz commented Jan 29, 2024

simsong commented Jan 30, 2024

zdavatz commented Jan 30, 2024

simsong commented Jan 30, 2024 via email

simsong commented Jan 26, 2024 •

edited

Loading

zdavatz commented Jan 27, 2024 •

edited

Loading

simsong commented Jan 27, 2024 •

edited

Loading

zdavatz commented Jan 29, 2024 •

edited

Loading