Facing Issue while setting up Arweave node in local machine #525

mayank-daga · 2024-02-29T04:27:38Z

Hi, I am getting following error while setting arweave node using docker:

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Processing triggers for dbus (1.12.20-2ubuntu4.1) ...
Arweave launch script: /arweave/bin/start
Building dependencies...
Removing build artifacts...

rm -f /arweave/bin/arweave-dev
rm -f /arweave/lib
rm -f /arweave/releases

/usr/bin/env: 'escript': No such file or directory
/usr/bin/env: 'escript': No such file or directory

Copying and linking build artifacts

cp /arweave/bin/arweave /arweave/bin/arweave-dev
cp: cannot stat '/arweave/bin/arweave': No such file or directory
ln -s /arweave/releases /arweave/releases
ln -s /arweave/lib /arweave/lib

Arweave launch script: /arweave/bin/arweave-dev
Arweave launch options: +Ktrue +A200 +SDio200 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.
Launching Erlang Virtual Machine...
/arweave/bin/start: line 14: /arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.

Dockerfile is :
FROM ubuntu:latest
COPY script.sh .
COPY ./arweave /arweave
RUN chmod +x script.sh
ENTRYPOINT ["sh","script.sh"]

script.sh contains:

#!/bin/bash
set -e
echo "NOTE. If you think that install is too slow, you probably should not mine arweave on this computer"

generic pack for almost all cryptocurrencies and comfortable work

export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y
htop screen tmux mc git nano curl wget g++ build-essential gcc make cmake autoconf automake psmisc net-tools mtr-tiny
libboost-all-dev libssl-dev libzmq3-dev libreadline-dev libsodium-dev pkg-config libunbound-dev libtool bsdmainutils libevent-dev libminiupnpc-dev autotools-dev python3
libudev-dev zlib1g-dev libseccomp-dev libcap-dev libncap-dev obfs4proxy libgmp-dev libtinfo5 software-properties-common apt-transport-https libsqlite3-dev

/arweave/arweave-server peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192

Also I have cloned the folllowing repo and moved the folder to container

git clone --recursive https://github.com/ArweaveTeam/arweave.git

mayank-daga · 2024-02-29T04:54:50Z

@ldmberman

vird · 2024-02-29T08:06:25Z

/usr/bin/env: 'escript': No such file or directory
Erlang is not installed

mayank-daga · 2024-03-01T06:41:06Z

yes thanks @vird was able to connect to arweave network. (i have attached screenshot for reference)

Also, I don't want to mine but query network for data at a blockheight, transaction hash details etc. How can I do that?

vird · 2024-03-01T11:30:04Z

By default you don't mine. For enable mining multiple arguments are needed

mine
mining_address
storage_module

mayank-daga · 2024-03-05T12:27:56Z

Hi @vird i am getting below error trying to dockerize the node setup:

Error
Protocol 'inet_tcp': register/listen error: econnrefused

DockerFile:

FROM ubuntu:latest
COPY script.sh .
#COPY ./arweave /arweave
RUN chmod +x script.sh
ENTRYPOINT ["sh","script.sh"]

Script,sh contains all the commands:

#!/bin/bash
set -e
echo "NOTE. If you think that install is too slow, you probably should not mine arweave on this computer"

generic pack for almost all cryptocurrencies and comfortable work

export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y
htop screen tmux mc git nano curl wget g++ build-essential gcc make cmake autoconf automake psmisc net-tools mtr-tiny
libboost-all-dev libssl-dev libzmq3-dev libreadline-dev libsodium-dev pkg-config libunbound-dev libtool bsdmainutils libevent-dev libminiupnpc-dev autotools-dev python3
libudev-dev zlib1g-dev libseccomp-dev libcap-dev libncap-dev obfs4proxy libgmp-dev libtinfo5 software-properties-common apt-transport-https libsqlite3-dev

arweave specific

wget -O- https://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | apt-key add -

echo "deb https://packages.erlang-solutions.com/ubuntu focal contrib" | tee /etc/apt/sources.list.d/erlang.list

echo "deb http://security.ubuntu.com/ubuntu impish-security main" | sudo tee /etc/apt/sources.list.d/impish-security.list

apt-get update

apt-get install -y erlang

git clone --recursive https://github.com/ArweaveTeam/arweave.git
cd arweave

/arweave/arweave-server peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192

vird · 2024-03-05T14:40:45Z

Sorry I can't read that. Pls make gist from that

mayank-daga · 2024-03-06T05:37:55Z

Hi @vird the error I am getting is:

Protocol 'inet_tcp': register/listen error: econnrefused

vird · 2024-03-06T07:23:28Z

Without details only this recipe helped me once
https://stackoverflow.com/questions/51616600/ssh-rabbitmq-protocol-inet-tcp-register-listen-error-econnrefused#:~:text=The%20error%20Protocol%20'inet_tcp'%3A,the%20Erlang%20node%20starts%20up.

vird · 2024-03-06T07:46:01Z

ENTRYPOINT ["sh","script.sh"]

That's not how docker works
That's not how script.sh works
script.sh is bash script #!/bin/bash, so it will not work with sh

You need to put each line of script.sh as separate RUN
Because it's how docker designed. Each RUN command will makes new snapshot of virtual machine with new "disk state" (storage)
COPY script.sh .
Each COPY command breaks sequence of snapshots. So next time you will build docker image all commands will be re-run from COPY command. Because it can't be snapshoted
Script.sh was for installing arweave node for running arweave node you need different script
You can put in launch.sh

#!/bin/bash
/arweave/arweave-server peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192

and make

FROM ubuntu:latest
SHELL ["/bin/bash", "-c"]
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get install -y htop screen tmux mc git nano curl wget g++ build-essential gcc make cmake autoconf automake psmisc net-tools mtr-tiny libboost-all-dev libssl-dev libzmq3-dev libreadline-dev libsodium-dev pkg-config libunbound-dev libtool bsdmainutils libevent-dev libminiupnpc-dev autotools-dev python3
// other commands from script.sh

// this should be used with `data_dir /data_dir` In arweave launch script otherwise you will lose your data after docker container will gone or updated
VOLUME /data_dir

COPY launch.sh .
RUN chmod +x launch.sh
CMD ./launch.sh

/bin/sh ignores .bashrc, sometimes ignore PATH and installed stuff (because installed stuff relies on default bash in ubuntu), so you can get /usr/bin/env: 'escript': No such file or directory, so I highly recommend to use bash

mayank-daga · 2024-03-13T14:09:02Z

Hi @vird I am trying install libsqlite3-dev, libgmp-dev on Ubuntu20.04 amd64 using following commands:

sudo apt get-install -y

but i am getting below errors:

Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package libgmp-dev

vird · 2024-03-13T14:13:02Z

Package should be available https://ubuntu.pkgs.org/20.04/ubuntu-main-arm64/libgmp-dev_6.2.0+dfsg-4_arm64.deb.html
Did you run apt-get update?

mayank-daga · 2024-03-13T14:14:13Z

yes @vird i ran sudo apt-get update

vird · 2024-03-13T14:16:51Z

apt-cache search libgmp ?

mayank-daga · 2024-03-13T14:17:54Z

it's giving :

libgmp10 - Multiprecision arithmetic library
libhogweed5 - low level cryptographic library (public-key cryptos)
libnettle7 - low level cryptographic library (symmetric and one-way cryptos)

vird · 2024-03-13T14:19:23Z

Then only dist-upgrade will help, because you don't have needed packages
It will update you to 22.04/22.10

vird · 2024-03-13T14:22:34Z

As alternative
https://github.com/virdpool/miner/blob/master/install_ubuntu_20.04.sh
This one was verified earlier on ubuntu 20.04 on hetzner
You can try hetzner cloud machine

mayank-daga · 2024-03-13T14:27:12Z

ok thank @vird i will try this and get back to you if any issues

mayank-daga · 2024-03-13T14:28:57Z

@vird can you just confirm one thing this setup https://github.com/virdpool/miner/blob/master/install_ubuntu_20.04.sh is official right? I am trying for the first just need to make sure

Also i see run.sh that is i guess run script which needs to be run after installation is complete right?

vird · 2024-03-13T14:30:34Z

This setup is not official
But at least it's most detailed setup that worked for arweave 2.5
So 99% of install should work for 2.7 without changes

vird · 2024-03-13T14:31:14Z

run.sh is script after you install
It's for launching arweave node but with virdpool 2.0 - 2.5, which is obsolete
So only this part is useful https://github.com/virdpool/miner/blob/master/run.sh#L31-L122

mayank-daga · 2024-03-14T10:25:59Z

Hi @vird getting this after installing all the packages and runnging this command:

./arweave-server peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192

Arweave launch script: ./bin/start
WARN: ************************************************************************
WARN: Your maximum number of open file descriptors is currently set to 10000.
WARN: We recommend setting that limit to 1000000 or higher.
WARN:
WARN: Otherwise, consider setting your max_connections setting to something
WARN: lower than your file descriptor limit.
WARN: ************************************************************************

Building dependencies...
Removing build artifacts...

rm -f /home/coiniaadmin/node-arweave/arweave/bin/arweave-dev
rm -f /home/coiniaadmin/node-arweave/arweave/lib
rm -f /home/coiniaadmin/node-arweave/arweave/releases

escript: exception error: undefined function rebar3:main/1
in function escript:run/2 (escript.erl, line 758)
in call from escript:start/1 (escript.erl, line 277)
in call from init:start_em/1
in call from init:do_boot/3

Copying and linking build artifacts

cp escript: exception error: undefined function rebar3:main/1 in function escript:run/2 '(escript.erl,' line '758)' in call from escript:start/1 '(escript.erl,' line '277)' in call from init:start_em/1 in call from init:do_boot/3 /arweave/bin/arweave /home/coiniaadmin/node-arweave/arweave/bin/arweave-dev
cp: target '/home/coiniaadmin/node-arweave/arweave/bin/arweave-dev' is not a directory
ln -s escript: exception error: undefined function rebar3:main/1 in function escript:run/2 '(escript.erl,' line '758)' in call from escript:start/1 '(escript.erl,' line '277)' in call from init:start_em/1 in call from init:do_boot/3 /arweave/releases /home/coiniaadmin/node-arweave/arweave/releases
ln: target '/home/coiniaadmin/node-arweave/arweave/releases' is not a directory
ln -s escript: exception error: undefined function rebar3:main/1 in function escript:run/2 '(escript.erl,' line '758)' in call from escript:start/1 '(escript.erl,' line '277)' in call from init:start_em/1 in call from init:do_boot/3 /arweave/lib /home/coiniaadmin/node-arweave/arweave/lib
ln: target '/home/coiniaadmin/node-arweave/arweave/lib' is not a directory

Arweave launch script: /home/coiniaadmin/node-arweave/arweave/bin/arweave-dev
Arweave launch options: +Ktrue +A200 +SDio200 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi
Launching Erlang Virtual Machine...
./bin/start: line 14: /home/coiniaadmin/node-arweave/arweave/bin/arweave-dev: No such file or directory
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.

mayank-daga · 2024-03-14T10:50:46Z

also @vird one doubt which machine we can use arm or amd64?

vird · 2024-03-14T12:33:34Z

escript: exception error: undefined function rebar3:main/

arweave requirement:
Erlang OTP v24, with OpenSSL support

what is erl ?
My is
Erlang/OTP 25 [erts-13.0.4] [source] [64-bit] [smp:24:24] [ds:24:24:10] [async-threads:1] [jit:ns]

Maybe there is problem with incorrect version of erlang (<24)

For ubuntu 20.04
apt-get install erlang will setup outdated erlang
This was recipe to fix it https://github.com/virdpool/miner/blob/master/install_ubuntu_20.04.sh#L22-L75

It should build on amd64 and arm, no difference

vird · 2024-03-15T08:09:06Z

So something like

apt-get install -y --allow-downgrades \
  erlang-appmon=1:24.3.3-1 \
  erlang-asn1=1:24.3.3-1 \
  erlang-base=1:24.3.3-1 \
  erlang-common-test=1:24.3.3-1 \
  erlang-crypto=1:24.3.3-1 \
  erlang-debugger=1:24.3.3-1 \
  erlang-dev=1:24.3.3-1 \
  erlang-dialyzer=1:24.3.3-1 \
  erlang-edoc=1:24.3.3-1 \
  erlang-erl-docgen=1:24.3.3-1 \
  erlang-et=1:24.3.3-1 \
  erlang-eunit=1:24.3.3-1 \
  erlang-gs=1:24.3.3-1 \
  erlang-ic=1:24.3.3-1 \
  erlang-inets=1:24.3.3-1 \
  erlang-inviso=1:24.3.3-1 \
  erlang-jinterface=1:24.3.3-1 \
  erlang-megaco=1:24.3.3-1 \
  erlang-mnesia=1:24.3.3-1 \
  erlang-mode=1:24.3.3-1 \
  erlang-observer=1:24.3.3-1 \
  erlang-odbc=1:24.3.3-1 \
  erlang-os-mon=1:24.3.3-1 \
  erlang-parsetools=1:24.3.3-1 \
  erlang-percept=1:24.3.3-1 \
  erlang-pman=1:24.3.3-1 \
  erlang-public-key=1:24.3.3-1 \
  erlang-reltool=1:24.3.3-1 \
  erlang-runtime-tools=1:24.3.3-1 \
  erlang-snmp=1:24.3.3-1 \
  erlang-ssh=1:24.3.3-1 \
  erlang-ssl=1:24.3.3-1 \
  erlang-syntax-tools=1:24.3.3-1 \
  erlang-test-server=1:24.3.3-1 \
  erlang-toolbar=1:24.3.3-1 \
  erlang-tools=1:24.3.3-1 \
  erlang-tv=1:24.3.3-1 \
  erlang-typer=1:24.3.3-1 \
  erlang-wx=1:24.3.3-1 \
  erlang-xmerl=1:24.3.3-1 \
  erlang=1:24.3.3-1 \
  erlang-diameter=1:24.3.3-1 \
  erlang-eldap=1:24.3.3-1 \
  erlang-ftp=1:24.3.3-1 \
  erlang-tftp=1:24.3.3-1 \
  erlang-ic-java=1:24.3.3-1 \
  erlang-src=1:24.3.3-1 \
  erlang-examples=1:24.3.3-1

should work
I picked latest 24 using apt-cache policy erlang

mayank-daga · 2024-03-15T09:37:36Z

Hi @vird we were able to install all the package, and erlang also but when i run development build command getting following error:

Command:
./arweave-server peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192

Error
[3:03 PM] Daga, Mayank
Error! Failed to eval: io:format("~s/erts-~s/include/", [code:root_dir(), erlang:system_info(version)]).

Error! Failed to eval: io:format("~s", [code:lib_dir(erl_interface, include)]).

g++ -O3 -finline-functions -fPIC -I -I -std=c++11 -Wall -c -o prometheus_process_collector_nif.o prometheus_process_collector_nif.cc

Error! Failed to eval: io:format("~s/erts-~s/include/", [code:root_dir(), erlang:system_info(version)]).

g++ -O3 -finline-functions -fPIC -I -I -std=c++11 -Wall -c -o prometheus_process_info_linux.o prometheus_process_info_linux.cc

Error! Failed to eval: io:format("~s", [code:lib_dir(erl_interface, lib)]).

cc prometheus_process_collector_nif.o prometheus_process_info_linux.o -shared -L -lei -lstdc++ -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so

make: Leaving directory '/NewDisk23Machine/node-arweave/arweave/_build/default/lib/prometheus_process_collector/c_src'

make: Entering directory '/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src'

Error! Failed to eval: io:format("~ts/erts-~ts/include/", [code:root_dir(), erlang:system_info(version)]).

Error! Failed to eval: io:format("~ts", [code:lib_dir(erl_interface, include)]).

cc -fPIC -O3 -std=c99 -Wall -Wmissing-prototypes -I -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:70:1: warning: no previous prototype for ‘dec_new’ [-Wmissing-prototypes]

70 | dec_new(ErlNifEnv* env)

  | ^~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:109:1: warning: no previous prototype for ‘dec_init’ [-Wmissing-prototypes]

109 | dec_init(Decoder* d, ErlNifEnv* env, ERL_NIF_TERM arg, ErlNifBinary* bin)

  | ^~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:129:1: warning: no previous prototype for ‘dec_error’ [-Wmissing-prototypes]

129 | dec_error(Decoder* d, const char* atom)

  | ^~~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:138:1: warning: no previous prototype for ‘dec_curr’ [-Wmissing-prototypes]

138 | dec_curr(Decoder* d)

  | ^~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:145:1: warning: no previous prototype for ‘dec_top’ [-Wmissing-prototypes]

145 | dec_top(Decoder* d)

  | ^~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:151:1: warning: no previous prototype for ‘dec_push’ [-Wmissing-prototypes]

151 | dec_push(Decoder* d, char val)

  | ^~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:170:1: warning: no previous prototype for ‘dec_pop’ [-Wmissing-prototypes]

170 | dec_pop(Decoder* d) {

  | ^~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:183:1: warning: no previous prototype for ‘dec_pop_assert’ [-Wmissing-prototypes]

183 | dec_pop_assert(Decoder* d, char val)

  | ^~~~~~~~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:191:1: warning: no previous prototype for ‘dec_string’ [-Wmissing-prototypes]

191 | dec_string(Decoder* d, ERL_NIF_TERM* value)

  | ^~~~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:375:1: warning: no previous prototype for ‘dec_number’ [-Wmissing-prototypes]

375 | dec_number(Decoder* d, ERL_NIF_TERM* value)

  | ^~~~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:622:1: warning: no previous prototype for ‘make_empty_object’ [-Wmissing-prototypes]

622 | make_empty_object(ErlNifEnv* env, int ret_map)

  | ^~~~~~~~~~~~~~~~~

/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/decoder.c:634:1: warning: no previous prototype for ‘make_array’ [-Wmissing-prototypes]

634 | make_array(ErlNifEnv* env, ERL_NIF_TERM list)

  | ^~~~~~~~~~

Error! Failed to eval: io:format("~ts/erts-~ts/include/", [code:root_dir(), erlang:system_info(version)]).

(no logger present) unexpected logger message: {log,error,"Error in process ~~p with exit value:np~~n",[<0.9.0>,{terminated,[{io,format,["~ts/erts-~ts/include/",["/usr/lib/erlang","14.2.2"]],[{file,"io.erl"},{line,98},{error_info,#{cause=>{io,terminated},module=>erl_stdlib_errors}}]},{erl_eval,do_apply,7,[{file,"erl_eval.erl"},{line,746}]},{init,start_it,1,[]},{init,start_em,1,[]},{init,do_boot,3,[]}]}],#{error_logger=>#{emulator=>true,tag=>error},pid=><0.9.0>,time=>1710495083855196,gl=><0.70.0>}}

Error! Failed to eval: io:format("~ts", [code:lib_dir(erl_interface, include)]).

cc -fPIC -O3 -std=c99 -Wall -Wmissing-prototypes -I -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/jiffy.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/jiffy.c

Error! Failed to eval: io:format("~ts/erts-~ts/include/", [code:root_dir(), erlang:system_info(version)]).

Error! Failed to eval: io:format("~ts", [code:lib_dir(erl_interface, include)]).

g++ -fPIC -O3 -Wall -I -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/doubles.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/doubles.cc

g++ -fPIC -O3 -Wall -I -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/bignum.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/bignum.cc

Error! Failed to eval: io:format("~ts/erts-~ts/include/", [code:root_dir(), erlang:system_info(version)]).

g++ -fPIC -O3 -Wall -I -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/bignum-dtoa.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/bignum-dtoa.cc

Error! Failed to eval: io:format("~ts/erts-~ts/include/", [code:root_dir(), erlang:system_info(version)]).

Error! Failed to eval: io:format("~ts", [code:lib_dir(erl_interface, include)]).

g++ -fPIC -O3 -Wall -I -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/double-conversion.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/double-conversion.cc

Error! Failed to eval: io:format("~ts", [code:lib_dir(erl_interface, include)]).

(no logger present) unexpected logger message: {log,error,"Error in process ~~p with exit value:np~~n",[<0.9.0>,{terminated,[{io,format,["~ts",["/usr/lib/erlang/lib/erl_interface-5.5.1/include"]],[{file,"io.erl"},{line,98},{error_info,#{cause=>{io,terminated},module=>erl_stdlib_errors}}]},{erl_eval,do_apply,7,[{file,"erl_eval.erl"},{line,746}]},{init,start_it,1,[]},{init,start_em,1,[]},{init,do_boot,3,[]}]}],#{error_logger=>#{emulator=>true,tag=>error},pid=><0.9.0>,time=>1710495102819260,gl=><0.70.0>}}

g++ -fPIC -O3 -Wall -I /usr/lib/erlang/erts-14.2.2/include/ -I -c -o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/strtod.o /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/strtod.cc

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/Scrt1.o: in function `_start':

(.text+0x24): undefined reference to `main'

/usr/bin/ld: /tmp/cclzzVKH.o: in function `double_conversion::CompareBufferWithDiyFp(double_conversion::Vector, int, double_conversion::DiyFp) [clone .isra.0]':

strtod.cc:(.text+0xca): undefined reference to `double_conversion::Bignum::Bignum()'

/usr/bin/ld: strtod.cc:(.text+0xd2): undefined reference to `double_conversion::Bignum::Bignum()'

/usr/bin/ld: strtod.cc:(.text+0xe5): undefined reference to `double_conversion::Bignum::AssignDecimalString(double_conversion::Vector)'

/usr/bin/ld: strtod.cc:(.text+0xf0): undefined reference to `double_conversion::Bignum::AssignUInt64(unsigned long)'

/usr/bin/ld: strtod.cc:(.text+0xfe): undefined reference to `double_conversion::Bignum::MultiplyByPowerOfTen(int)'

/usr/bin/ld: strtod.cc:(.text+0x10e): undefined reference to `double_conversion::Bignum::ShiftLeft(int)'

/usr/bin/ld: strtod.cc:(.text+0x119): undefined reference to `double_conversion::Bignum::Compare(double_conversion::Bignum const&, double_conversion::Bignum const&)'

/usr/bin/ld: strtod.cc:(.text+0x14e): undefined reference to `double_conversion::Bignum::MultiplyByPowerOfTen(int)'

/usr/bin/ld: strtod.cc:(.text+0x160): undefined reference to `double_conversion::Bignum::ShiftLeft(int)'

/usr/bin/ld: /tmp/cclzzVKH.o: in function `double_conversion::DiyFpStrtod(double_conversion::Vector, int, double*) [clone .isra.0]':

strtod.cc:(.text+0x318): undefined reference to `double_conversion::PowersOfTenCache::kMaxDecimalExponent'

/usr/bin/ld: strtod.cc:(.text+0x32b): undefined reference to `double_conversion::PowersOfTenCache::kMinDecimalExponent'

/usr/bin/ld: strtod.cc:(.text+0x393): undefined reference to `double_conversion::PowersOfTenCache::GetCachedPowerForDecimalExponent(int, double_conversion::DiyFp*, int*)'

/usr/bin/ld: strtod.cc:(.text+0x3ac): undefined reference to `double_conversion::PowersOfTenCache::kDecimalExponentDistance'

/usr/bin/ld: strtod.cc:(.text+0x405): undefined reference to `double_conversion::DiyFp::Multiply(double_conversion::DiyFp const&)'

/usr/bin/ld: strtod.cc:(.text+0x423): undefined reference to `double_conversion::DiyFp::Multiply(double_conversion::DiyFp const&)'

collect2: error: ld returned 1 exit status

make: *** [Makefile:76: /NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src/double-conversion/strtod.o] Error 1

make: Leaving directory '/NewDisk23Machine/node-arweave/arweave/_build/default/lib/jiffy/c_src'

===> Hook for compile failed!

Copying and linking build artifacts

cp /arweave/bin/arweave /NewDisk23Machine/node-arweave/arweave/bin/arweave-dev

cp: cannot stat '/arweave/bin/arweave': No such file or directory

ln -s /arweave/releases /NewDisk23Machine/node-arweave/arweave/releases
ln -s /arweave/lib /NewDisk23Machine/node-arweave/arweave/lib

Arweave launch script: /NewDisk23Machine/node-arweave/arweave/bin/arweave-dev

Arweave launch options: +Ktrue +A200 +SDio200 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi

Launching Erlang Virtual Machine...

./bin/start: line 14: /NewDisk23Machine/node-arweave/arweave/bin/arweave-dev: No such file or directory

Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.

Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server.

Also erl is giving following:
Erlang/OTP 26 [erts-14.2.2] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [jit:ns]
Eshell V14.2.2 (press Ctrl+G to abort, type help(). for help)

vird · 2024-03-15T11:14:30Z

OTP 26
There is problem that this OTP deprecated some stuff currently team uses in code. So I guess arweave node currently works on OTP 24 and 25 (maybe), but not 26

mayank-daga · 2024-03-21T10:41:23Z

Hi @vird getting below error after setting up everything :

===> Release successfully assembled: _build/default/rel/arweave

Copying and linking build artifacts

cp /SecondDisk24Machine/node-arweave/arweave/_build/default/rel/arweave/bin/arweave /SecondDisk24Machine/node-arweave/arweave/bin/arweave-dev
ln -s /SecondDisk24Machine/node-arweave/arweave/_build/default/rel/arweave/releases /SecondDisk24Machine/node-arweave/arweave/releases
ln -s /SecondDisk24Machine/node-arweave/arweave/_build/default/rel/arweave/lib /SecondDisk24Machine/node-arweave/arweave/lib

Arweave launch script: /SecondDisk24Machine/node-arweave/arweave/bin/arweave-dev
Arweave launch options: +Ktrue +A200 +SDio200 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi
Launching Erlang Virtual Machine...
Exec: /usr/lib/erlang/erts-12.3.1/bin/erlexec -boot /SecondDisk24Machine/node-arweave/arweave/releases/2.7.2/start -mode embedded -boot_var SYSTEM_LIB_DIR /usr/lib/erlang/lib -config /SecondDisk24Machine/node-arweave/arweave/releases/2.7.2/sys.config -args_file /SecondDisk24Machine/node-arweave/arweave/releases/2.7.2/vm.args -- console +Ktrue +A200 +SDio200 +MBsbct 103424 +MBsmbcs 10240 +MBlmbcs 410629 +MBmmmbc 1024 +MBas aobf +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi -run ar main peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192
Root: /SecondDisk24Machine/node-arweave/arweave
/SecondDisk24Machine/node-arweave/arweave
Erlang/OTP 24 [erts-12.3.1] [source] [64-bit] [smp:4:4] [ds:4:4:200] [async-threads:200] [jit]

Eshell V12.3.1 (abort with ^G)
([email protected])1>

VDF step computed in 1.74 seconds.

WARNING: your VDF computation speed is low - consider fetching VDF outputs from an external source (see vdf_server_trusted_peer and vdf_client_peer command line parameters).

Setting the mining address to 3dvd3Vg73HN1JfRKyx2tmete4jXgLRdoivD7FXyWgyc.
dets: file "./ar_tx_blacklist/ar_tx_blacklist" not properly closed, repairing ...
dets: file "./ar_tx_blacklist/ar_tx_blacklist_pending_headers" not properly closed, repairing ...
dets: file "./ar_tx_blacklist/ar_tx_blacklist_pending_data" not properly closed, repairing ...
dets: file "./ar_tx_blacklist/ar_tx_blacklist_offsets" not properly closed, repairing ...
dets: file "./ar_tx_blacklist/ar_tx_blacklist_pending_restore_headers" not properly closed, repairing ...

Initialising RandomX dataset for fast packing. Key: <<"ZGVmYXVsdCBhcndlYXZlIDIuNSBwYWNrIGtleQ">>. The process may take several minutes.
RandomX dataset initialisation complete.
Hashing benchmark
H0: 2.361 ms
H1/H2: 0.191 ms

The node is configured to pack around 40 chunks per second. To increase the packing rate, start with packing_rate [number]. Estimated maximum rate: 49.38 chunks/s.

Setting the packing chunk cache size limit to 1300 chunks.

Setting the data chunk cache size limit to 1100 chunks.

    Peer 138.197.232.192:1984 is not available.


    Peer 139.59.51.59:1984 is not available.


    Peer 163.47.11.64:1984 is not available.


    Peer 188.166.192.169:1984 is not available.


    Peer 188.166.200.45:1984 is not available.

The specified trusted peers are not valid.
[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
Arweave Heartbeat: Server terminated safely.

Note: I am using erlang 24

vird · 2024-03-21T10:57:00Z

As Peer 138.197.232.192:1984 is not available said

Check that curl 138.197.232.192:1984 works for you at host machine
Check that curl 138.197.232.192:1984 works for you inside docker container

mayank-daga · 2024-04-02T10:02:11Z

Hi @vird we checked the traffic on the server from firewall logs and also 1984 TCP is open. We wanted to know if we can alter this
./arweave-server peer 188.166.200.45 peer 188.166.192.169 peer 163.47.11.64 peer 139.59.51.59 peer 138.197.232.192 command to keep the node up even if it is not able to connect to peers

vird · 2024-04-02T12:52:41Z

Without public opened port 1984 node will receive updates later than usually
But still will work

mayank-daga · 2024-04-09T08:37:24Z

HI @vird i connected to the Arweave network but how to query network for latest block height and transaction details

vird · 2024-04-09T08:45:07Z

https://docs.arweave.org/developers/arweave-node-server/http-api

e.g. curl http://localhost:1984
similar to
https://arweave.net/ - latest block
https://arweave.net/block/height/1391731
https://arweave.net/block/hash/n_rB-pwjwgB0poWgOxWj9P-K0h7vNP9uvl3TKtgKw0jld3I43vQsHtQLx6625QXR
(same block but with indep_hash)
https://arweave.net/tx/uu49CtpFyAd6wDoB8kp8DtBHYQhDZ6C6HsNvgplyOQ8
tx from same block

Also for each block fetch you should check that prev block is actually known processed block, otherwise you should unroll until you find known block
That's known issue of blockchain called network reorg
If you want to make something like block explorer you should handle reorgs correctly, just fetching latest block repeatedly may be not enough for correctness (depends on application)

mayank-daga · 2024-04-09T17:13:53Z

yes working thank alot @vird for the help

mayank-daga · 2024-04-10T10:12:05Z

Hi @vird I had some doubts

I was trying to query initial blocks like 0,1 and I was getting block not found:

Curl query:
sudo curl --request GET --url 'http://localhost:1984/block/height/1'

My node was not syncing because of low CPU, I am upgrading to a higher one. But could that be the reason. I mean if node will sync will we able to query initial blocks also because when I check the info api it under blocks : 2349(shown below).
Does that mean only latest 2349 block can be queried?
{"network":"arweave.N.1","version":5,"release":69,"height":1401264,"current":"crh9201b99Vbn_btWi4dL9kEN1HqXSEe7LEe7vAKYQvML4V2GEuhSZwaDExCIL-5","blocks":2439,"peers":73,"queue_length":6,"node_state_latency":1}

Is the node is archival?
Also what is command/api to check that previous block is actually known processed block and is so how to unroll?

vird · 2024-04-10T10:30:05Z

There is no ethereum-like abstraction about "archive" mode
Did you noticed that http://localhost:1984/ shows amount of blocks you downloaded?
until node will download all blocks some blocks will be not available for fetch
same with tx. Block can be available, but some tx still not
Sync direction is mostly latest to oldest
So it's expected that height 1 is not available shortly after start

"previous block is actually known"
It's not previous block known for node. It's previous block known for your application. You should track what blocks do you know. Check fields indep_hash and previous_block

Until your node is not fully synced get historical blocks from https://arweave.net/

mayank-daga · 2024-04-17T11:20:11Z

Hi @vird small doubts mentioned below:

By "archive", I just want to reconfirm for the node that i am running that we will have data from latest to the 1st block?
Also, one question ideally how much time node will take to sync all the blocks from latest to 1st if we have machine configuration exactly as mentioned in arweave doc
What is the extension of each block file and where is data getting stored in arweave so that i can identify if the data is fetched

vird · 2024-04-17T13:52:13Z

Yes. Ideally with enough space arweave node will download all blocks and txs
2 months or more for all blocks and txs (and at least 400+ GB space)
blocks are not stored in folder except disk_cache/blocks
They are stored as .bin (can be de-serialized with ar_serialize:block_to_json_struct)
you can decode them using data-doctor dump (PR is pending)
(or get them using http API)

Facing Issue while setting up Arweave node in local machine #525

Facing Issue while setting up Arweave node in local machine #525

Comments

mayank-daga commented Feb 29, 2024 • edited Loading

generic pack for almost all cryptocurrencies and comfortable work

mayank-daga commented Feb 29, 2024

vird commented Feb 29, 2024

mayank-daga commented Mar 1, 2024

vird commented Mar 1, 2024

mayank-daga commented Mar 5, 2024

generic pack for almost all cryptocurrencies and comfortable work

arweave specific

wget -O- https://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | apt-key add -

echo "deb https://packages.erlang-solutions.com/ubuntu focal contrib" | tee /etc/apt/sources.list.d/erlang.list

echo "deb http://security.ubuntu.com/ubuntu impish-security main" | sudo tee /etc/apt/sources.list.d/impish-security.list

apt-get update

vird commented Mar 5, 2024

mayank-daga commented Mar 6, 2024

vird commented Mar 6, 2024

vird commented Mar 6, 2024

mayank-daga commented Mar 13, 2024

vird commented Mar 13, 2024

mayank-daga commented Mar 13, 2024

vird commented Mar 13, 2024

mayank-daga commented Mar 13, 2024

vird commented Mar 13, 2024

vird commented Mar 13, 2024

mayank-daga commented Mar 13, 2024

mayank-daga commented Mar 13, 2024 • edited Loading

vird commented Mar 13, 2024

vird commented Mar 13, 2024 • edited Loading

mayank-daga commented Mar 14, 2024 • edited Loading

mayank-daga commented Mar 14, 2024

vird commented Mar 14, 2024 • edited Loading

vird commented Mar 15, 2024

mayank-daga commented Mar 15, 2024 • edited Loading

vird commented Mar 15, 2024

mayank-daga commented Mar 21, 2024

vird commented Mar 21, 2024 • edited Loading

mayank-daga commented Apr 2, 2024

vird commented Apr 2, 2024

mayank-daga commented Apr 9, 2024 • edited Loading

vird commented Apr 9, 2024

mayank-daga commented Apr 9, 2024

mayank-daga commented Apr 10, 2024 • edited Loading

vird commented Apr 10, 2024 • edited Loading

mayank-daga commented Apr 17, 2024 • edited Loading

vird commented Apr 17, 2024 • edited Loading

mayank-daga commented Feb 29, 2024 •

edited

Loading

mayank-daga commented Mar 13, 2024 •

edited

Loading

vird commented Mar 13, 2024 •

edited

Loading

mayank-daga commented Mar 14, 2024 •

edited

Loading

vird commented Mar 14, 2024 •

edited

Loading

mayank-daga commented Mar 15, 2024 •

edited

Loading

vird commented Mar 21, 2024 •

edited

Loading

mayank-daga commented Apr 9, 2024 •

edited

Loading

mayank-daga commented Apr 10, 2024 •

edited

Loading

vird commented Apr 10, 2024 •

edited

Loading

mayank-daga commented Apr 17, 2024 •

edited

Loading

vird commented Apr 17, 2024 •

edited

Loading