CUDA build fails

[elliottslaughter]

Looks like double2 is a native type in CUDA, so when Pennant defines it in Vec2.hh we run into trouble.

In file included from /usr/local/cuda-11.2/include/driver_types.h:59,
                 from /usr/local/cuda-11.2/include/builtin_types.h:59,
                 from /usr/local/cuda-11.2/include/cuda_runtime_api.h:147,
                 from /usr/local/cuda-11.2/include/thrust/system/cuda/detail/guarded_cuda_runtime_api.h:38,
                 from /usr/local/cuda-11.2/include/thrust/type_traits/is_trivially_relocatable.h:193,
                 from /usr/local/cuda-11.2/include/thrust/detail/complex/complex.inl:20,
                 from /usr/local/cuda-11.2/include/thrust/complex.h:1031,
                 from /users/eslaught/resilience/legion/install/include/mathtypes/complex.h:34,
                 from /users/eslaught/resilience/legion/install/include/legion/legion_redop.h:25,
                 from /users/eslaught/resilience/legion/install/include/legion.h:59,
                 from /users/eslaught/resilience/src/resilience.h:24,
                 from /users/eslaught/resilience/examples/pennant/src/Mesh.hh:21,
                 from /users/eslaught/resilience/examples/pennant/src/ExportGold.cc:21:
/usr/local/cuda-11.2/include/vector_types.h:334:49: error: redefinition of 'struct double2'
  334 | struct __device_builtin__ __builtin_align__(16) double2
      |                                                 ^~~~~~~
In file included from /users/eslaught/resilience/examples/pennant/src/ExportGold.cc:20:
/users/eslaught/resilience/examples/pennant/src/Vec2.hh:24:8: note: previous definition of 'struct double2'
   24 | struct double2
      |        ^~~~~~~

On Piz Daint with CUDA 11 and GCC 10, in case that makes any difference.

[elliottslaughter]

Looking at the code, I see it's all guarded by:

#ifndef __CUDACC__

So I'm not sure how we're hitting this. Maybe host code doesn't define __CUDACC__?

[eddy16112]

If I remember correctly, pennant does not need the complex redop from Legion, so for a quick fix, you can turn off the Legion_REDOP_COMPLEX.

[elliottslaughter]

I am trying to build Regent and C++ apps in the same build, so that's why I have Legion_REDOP_COMPLEX turned on. I guess I could split them up, but it would be a pain.

[elliottslaughter]

So the issue is that we're not even building this file with nvcc. We're using gcc because it's on the CPU sources list.

What CUDA header do I need to define this type? If we switch to checking USE_CUDA I hit undefined errors in some files.