MPI had an error

[Leni-Yeo]

Good afternoon,
I am trying to do a simulation for polycrystal grain growth. I saw the example that applies the polycrystalvariables and polycrystalkernel actions. I modified the polycrystalkernel action to include the following kernels: ADAllenCahn, ADTimeDerivative, and ADACInterfaceKobayashi2 for a basic simulation. The problem is that I get a "MPI had an error" when running the simulation. I cannot find the exact problem. I assumed it is in the action kernel but still could not solve it. Please help.

I attached my input file and the kernel action I am using to set up the kernels.

##################### I COPIED THE ERROR BELOW ##########################

Generic Warning: In ../Parallel/MPI/vtkMPICommunicator.cxx, line 69
MPI had an error

Invalid communicator, error stack:
MPI_Comm_test_inter(112): MPI_Comm_test_inter(comm=0xf2e8f3e0, flag=0x7ffcce88cfc0) failed
MPI_Comm_test_inter(70).: Invalid communicator

application called MPI_Abort(MPI_COMM_WORLD, 612613) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=612613
:
system msg for write_line failure : Bad file descriptor

Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

Generic Warning: In ../Parallel/MPI/vtkMPICommunicator.cxx, line 69
MPI had an error

Invalid communicator, error stack:
MPI_Comm_test_inter(112): MPI_Comm_test_inter(comm=0x4ce493e0, flag=0x7ffd8f4d8be0) failed
MPI_Comm_test_inter(70).: Invalid communicator

application called MPI_Abort(MPI_COMM_WORLD, 612613) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=612613
:
system msg for write_line failure : Bad file descriptor
Assertion failed in file src/mpi/comm/comm_test_inter.c at line 77: ((comm)&(0x03ffffff)) < MPIR_COMM_N_BUILTIN
/home/user/miniconda3/envs/moose/lib/libmpi.so.12(+0x1c7d93) [0x7fa949f80d93]
/home/user/miniconda3/envs/moose/lib/libmpi.so.12(MPL_backtrace_show+0x18) [0x7fa949f80e73]
/home/user/miniconda3/envs/moose/lib/libmpi.so.12(+0x15bb94) [0x7fa949f14b94]
/home/user/miniconda3/envs/moose/lib/libmpi.so.12(MPI_Comm_test_inter+0x2f0) [0x7fa949e27840]
/lib/x86_64-linux-gnu/libmpi_cxx.so.40(_ZN3MPI9IntracommC1EP19ompi_communicator_t+0x8b) [0x7fa9443623db]
/lib/x86_64-linux-gnu/libmpi_cxx.so.40(+0x1125a) [0x7fa94435e25a]
/lib64/ld-linux-x86-64.so.2(+0x11b8a) [0x7fa94e9abb8a]
/lib64/ld-linux-x86-64.so.2(+0x11c91) [0x7fa94e9abc91]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0xe5) [0x7fa94abed915]
/lib64/ld-linux-x86-64.so.2(+0x160bf) [0x7fa94e9b00bf]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88) [0x7fa94abed8b8]
/lib64/ld-linux-x86-64.so.2(+0x155fa) [0x7fa94e9af5fa]
/lib/x86_64-linux-gnu/libdl.so.2(+0x134c) [0x7fa94717734c]
/lib/x86_64-linux-gnu/libc.so.6(_dl_catch_exception+0x88) [0x7fa94abed8b8]
/lib/
internal ABORT - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1
:
system msg for write_line failure : Bad file descriptor
Generic Warning: In ../Parallel/MPI/vtkMPICommunicator.cxx, line 69
MPI had an error

Invalid communicator, error stack:
MPI_Comm_test_inter(112): MPI_Comm_test_inter(comm=0xa292c3e0, flag=0x7ffe75b7b580) failed
MPI_Comm_test_inter(70).: Invalid communicator

application called MPI_Abort(MPI_COMM_WORLD, 537483525) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=537483525
:
system msg for write_line failure : Bad file descriptor
Generic Warning: In ../Parallel/MPI/vtkMPICommunicator.cxx, line 69
MPI had an error

Invalid communicator, error stack:
MPI_Comm_test_inter(112): MPI_Comm_test_inter(comm=0x62a903e0, flag=0x7ffcd7088450) failed
MPI_Comm_test_inter(70).: Invalid communicator

files.zip

[GiudGiud]

Hi

Do you have a working version without this new action?
I dont see the new kernels or the new action in the input file you provided.

Guillaume

[Leni-Yeo]

Hi
Thank you for answering back.
The problem was fortunately solved. We had set up a wrong kernel apparently which messed up the IC and boundary conditions.
To answer your question, it was a modified version of the "grain_growth_2D_voronoi_newadapt.i" file, with the first kernel replaced by the "ACInterfaceKobayashi1".

[GiudGiud]

Great! Thanks for letting us know.