Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS can't compile from source or benchmark vdf #611

Open
heavyarms2112 opened this issue Aug 27, 2024 · 15 comments
Open

macOS can't compile from source or benchmark vdf #611

heavyarms2112 opened this issue Aug 27, 2024 · 15 comments

Comments

@heavyarms2112
Copy link

heavyarms2112 commented Aug 27, 2024

sudo ./bin/benchmark-vdf

Failed to create dirty io scheduler thread 1005, error = 35 ./bin/benchmark-vdf: line 10: 1492 Abort trap: 6 $ARWEAVE foreground $ARWEAVE_OPTS -run ar benchmark_vdf

hardware: Mac mini m2
OS version: 13.2.1

Also on compilation from source hit the below error message.

===> ar_weave:test/0 calls undefined function eunit:test/1 ===> Release successfully assembled: _build/default/rel/arweave ===> Unable to copy from /Users/rjxm2/miners/arweave/arweave/apps/arweave/lib/RandomX/build/randomx-benchmark to /Users/jdoe/miners/arweave/arweave/_build/default/rel/arweave/bin/randomx-benchmark because of {copy_failed, "/Users/jdoe/miners/arweave/arweave/apps/arweave/lib/RandomX/build/randomx-benchmark", "/Users/jdoe/miners/arweave/arweave/_build/default/rel/arweave/bin/randomx-benchmark", enoent}

Copy link
Collaborator

Yeah you can ignore those. MacOS isn't really supported. We can only confirm that the arweave node runs as a VDF server - but beyond that it should be considered unsupported (e.g. the benchmark-xxx tools may not work)

@heavyarms2112
Copy link
Author

heavyarms2112 commented Aug 27, 2024

Yeah you can ignore those. MacOS isn't really supported. We can only confirm that the arweave node runs as a VDF server - but beyond that it should be considered unsupported (e.g. the benchmark-xxx tools may not work)

what's the link for VDF server then? Is there precompiled binary?
this is what I'm trying to run in first place.

@JamesPiechota
Copy link
Collaborator

Does bin/start not work? If not then can you upload the output from that?

There is no precompiled vdf binary. Only the instructions for compiling from source - but they have only been tested against bin/start. The benchmark binaries you've listed above haven't been tested on macos.

@heavyarms2112
Copy link
Author

heavyarms2112 commented Aug 28, 2024

./bin/start won't work either. It throws the same error message

+ cp /Users/rjxm2/miners/arweave/arweave/_build/default/rel/arweave/bin/arweave /Users/rjxm2/miners/arweave/arweave/bin/arweave-dev
+ ln -s /Users/rjxm2/miners/arweave/arweave/_build/default/rel/arweave/releases /Users/rjxm2/miners/arweave/arweave/releases
+ ln -s /Users/rjxm2/miners/arweave/arweave/_build/default/rel/arweave/lib /Users/rjxm2/miners/arweave/arweave/lib

Arweave launch script: /Users/rjxm2/miners/arweave/arweave/bin/arweave-dev
Arweave launch options: +Ktrue +A1024 +SDio1024     +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf     +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long     +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi
-h
Custom Erlang Options: 
Launching Erlang Virtual Machine...
Exec: /opt/homebrew/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/bin/erlexec -boot /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/start -mode embedded -boot_var SYSTEM_LIB_DIR /opt/homebrew/Cellar/erlang/26.2.5/lib/erlang/lib -config /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/sys.config -args_file /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/vm.args -- console +Ktrue +A1024 +SDio1024 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi -run ar main disable randomx_jit -h
Root: /Users/rjxm2/miners/arweave/arweave
/Users/rjxm2/miners/arweave/arweave
Failed to create dirty io scheduler thread 1005, error = 35
./bin/start: line 50:  2448 Abort trap: 6           $ARWEAVE $ARWEAVE_COMMAND $ARWEAVE_OPTS ${ERLANG_ARGS[@]} -run ar main $RANDOMX_JIT ${ARWEAVE_ARGS[@]}
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.```

@heavyarms2112
Copy link
Author

this is specific to erlang which I'm sure ./bin/start would also use.

@JamesPiechota
Copy link
Collaborator

this is specific to erlang which I'm sure ./bin/start would also use.

Oh cool - so you know what the problem is?

@heavyarms2112
Copy link
Author

this is specific to erlang which I'm sure ./bin/start would also use.

Oh cool - so you know what the problem is?

what I meant is this issue would pop up on running /bin/start as well as shown above.

@JamesPiechota
Copy link
Collaborator

If you go into your /Users/rjxm2/miners/arweave/arweave/ directory and run:

  1. git status
  2. git submodule status

What do they print?

@JamesPiechota
Copy link
Collaborator

Ah also, what does ulimit -n show? Arweave still needs the file descriptors to be increased on MacOS, but it's trickier to do.

We usually just do the ulimit -n 9000000 in the same session as the invocation e.g.:

ulimit -n 9000000; ./bin/start

@heavyarms2112
Copy link
Author

heavyarms2112 commented Aug 28, 2024

changing ulimit won't help I guess but I'll try anyways

I think it's related to kern.num_taskthreads which is set to 2048 per process. Are we requesting for more than that for erlang vm?

@heavyarms2112
Copy link
Author

Yup didn't help

Launching Erlang Virtual Machine...
Exec: /opt/homebrew/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/bin/erlexec -boot /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/start -mode embedded -boot_var SYSTEM_LIB_DIR /opt/homebrew/Cellar/erlang/26.2.5/lib/erlang/lib -config /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/sys.config -args_file /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/vm.args -- console +Ktrue +A1024 +SDio1024 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi -run ar main
Root: /Users/rjxm2/miners/arweave/arweave
/Users/rjxm2/miners/arweave/arweave
Failed to create dirty io scheduler thread 1005, error = 35
./bin/start: line 50:  9064 Abort trap: 6           $ARWEAVE $ARWEAVE_COMMAND $ARWEAVE_OPTS ${ERLANG_ARGS[@]} -run ar main $RANDOMX_JIT ${ARWEAVE_ARGS[@]}
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
^C
sh-3.2# ulimit -n
9000000
sh-3.2# git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
sh-3.2# git submodule status
 913873c13a2dffb7c4188c39b4eb188f912f523e apps/arweave/lib/RandomX (v1.0.3-210-g913873c)

@heavyarms2112
Copy link
Author

Looks like I could get some success.
I reduced the number of erlang threads in arweave's env variable for erlang. Looks like I hit a thread limit obviously as mentioned in my previous post.

unfortunately, the kern.num_taskthreads is a read only variable and can't be modified. Is 512 value below acceptable?
+Ktrue +A512 +SDio512

Arweave launch script: /Users/rjxm2/miners/arweave/arweave/bin/arweave-dev
Arweave launch options: +Ktrue +A512 +SDio512     +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf     +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long     +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi +S 16:16
Exec: /opt/homebrew/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/bin/erlexec -noinput +Bd -boot /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/start -mode embedded -boot_var SYSTEM_LIB_DIR /opt/homebrew/Cellar/erlang/26.2.5/lib/erlang/lib -config /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/sys.config -args_file /Users/rjxm2/miners/arweave/arweave/releases/2.7.4/vm.args -- foreground +Ktrue +A512 +SDio512 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi +S 16:16 -run ar benchmark_vdf
Root: /Users/rjxm2/miners/arweave/arweave
/Users/rjxm2/miners/arweave/arweave
=INFO REPORT==== 27-Aug-2024::21:58:41.653441 ===
    alarm_handler: {set,{system_memory_high_watermark,[]}}


VDF step computed in 0.86 seconds.

[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed

@JamesPiechota
Copy link
Collaborator

Yep for a VDF server +A512 +SDio512 or even lower is fine

@JamesPiechota
Copy link
Collaborator

If you want to open a PR against https://github.com/ArweaveTeam/arweave/blob/master/README.md with troubleshooting tips, I can merge it.

@JamesPiechota
Copy link
Collaborator

Also you can override the Erlang VM flags using the -- syntax (if you want to avoid modifying arweave.env):

./bin/start +A512 +SDio512 -- [normal arweave launch flags]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@JamesPiechota @heavyarms2112 and others