Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update astc version (#951) errors on linking on a M1 Mac #965

Closed
RefuX opened this issue Mar 8, 2024 · 12 comments · Fixed by #970 · May be fixed by #969
Closed

Update astc version (#951) errors on linking on a M1 Mac #965

RefuX opened this issue Mar 8, 2024 · 12 comments · Fixed by #970 · May be fixed by #969
Labels
build This is relevant to the build system urgent Need to resolve as soon as possible

Comments

@RefuX
Copy link

RefuX commented Mar 8, 2024

System
OS: macOS 14.4
CPU: Apple M1

Issue
I cloned the repo and on running the command cmake --build build/mac --config Release --target vulkan_samples -j4 the linking failed with the below errors:

[100%] Linking CXX executable bin/Release/arm64/vulkan_samples
ld: warning: ignoring duplicate libraries: '../components/core/lib/Release/arm64/libvkb__core.a', '../components/unix/lib/Release/arm64/libvkb__unix_platform.a'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[23](astcenc_weight_quant_xfer_tables.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[22](astcenc_weight_align.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[21](astcenc_symbolic_physical.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[20](astcenc_quantization.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[19](astcenc_pick_best_endpoint_format.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[18](astcenc_percentile_tables.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[17](astcenc_partition_tables.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[16](astcenc_mathlib_softfloat.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[15](astcenc_mathlib.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[14](astcenc_integer_sequence.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[13](astcenc_image.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[12](astcenc_ideal_endpoints_and_weights.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[11](astcenc_find_best_partitioning.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[10](astcenc_entry.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[9](astcenc_diagnostic_trace.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[8](astcenc_decompress_symbolic.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[7](astcenc_compute_variance.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[6](astcenc_compress_symbolic.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[5](astcenc_color_unquantize.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[4](astcenc_color_quantize.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[3](astcenc_block_sizes.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[2](astcenc_averages_and_directions.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
Undefined symbols for architecture arm64:
  "astcenc_config_init(astcenc_profile, unsigned int, unsigned int, unsigned int, float, unsigned int, astcenc_config*)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
  "astcenc_context_free(astcenc_context*)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
  "astcenc_context_alloc(astcenc_config const*, unsigned int, astcenc_context**)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
  "astcenc_decompress_image(astcenc_context*, unsigned char const*, unsigned long, astcenc_image*, astcenc_swizzle const*, unsigned int)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [app/bin/Release/arm64/vulkan_samples] Error 1
make[2]: *** [app/CMakeFiles/vulkan_samples.dir/all] Error 2
make[1]: *** [app/CMakeFiles/vulkan_samples.dir/rule] Error 2
make: *** [vulkan_samples] Error 2

Workaround (for a fresh install)
My workout was to simply checkout the previous commit, thus avoiding the astc update.

git clone https://github.com/KhronosGroup/Vulkan-Samples.git
git checkout 5f86b120c45c4146f78faa98eb4a370f382837f1
git submodule update --init --recursive

@SaschaWillems
Copy link
Collaborator

SaschaWillems commented Mar 8, 2024

Thanks for bringing this up. We'll try to fix this asap.

@SaschaWillems SaschaWillems added build This is relevant to the build system urgent Need to resolve as soon as possible labels Mar 8, 2024
@SaschaWillems SaschaWillems mentioned this issue Mar 8, 2024
12 tasks
@jherico
Copy link
Contributor

jherico commented Mar 8, 2024

I don't have an M1 macbook to test on directly. I'll see if there's a cloud provider I can try to use to test, or if I can replicate the problem on a Raspberry Pi or something.

@SaschaWillems
Copy link
Collaborator

We do have CI for MacOS, but I guess it's x86 only. Maybe there is a way to add MacOS ARM support?

@tomadamatkinson
Copy link
Collaborator

I have an M1 air I can test on. I think GitHub released Apple M1/2 runners this year too so I can add these to the CI also

@SaschaWillems
Copy link
Collaborator

We can probably just replace the x86_64 CI builds with ARM.

@jherico
Copy link
Contributor

jherico commented Mar 8, 2024

I've started an AWS M2 instance and I'm working the problem. Hopefully I can find a solution that won't require upstream changes.

@jherico
Copy link
Contributor

jherico commented Mar 8, 2024

One possibility would be simply use CMake ARCH detection and preprocessor macros to exclude the ASTC encoder on ARM based macs, since they support ASTC natively and don't need to do the decoding.

Honestly that may be the shortest path to resolving this issue.

@tomadamatkinson
Copy link
Collaborator

Taking a look also. In terms of the CICD I think we just need to add macos-14 and we have Apple Silicon

Heres the docs

@jherico
Copy link
Contributor

jherico commented Mar 8, 2024

@tomadamatkinson I've pushed a (likely) fix PR and cherry-picked your CI changes into it. This changed allowed me to build on the EC2 M2 instance after having the same link error described in the issue.

@tomadamatkinson
Copy link
Collaborator

tomadamatkinson commented Mar 8, 2024

The commit you took wont work. You may need to remove your current commit and cherry pick again.

As the runners are in beta it looks like they have limited availability. They seem to take a while to provision https://github.com/KhronosGroup/Vulkan-Samples/actions/runs/8209479230?pr=969

Might be worth only running the Apple silicon task on main because of this

Looks like macos-latest will be moved to macos-14 later in the year https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

@jherico
Copy link
Contributor

jherico commented Mar 8, 2024

I've removed your CI commit and also added a comment to my change that points to the upstream bug, so the logic can be removed when it's fixed. Cheers.

As the runners are in beta it looks like they have limited availability. They seem to take a while to provision

Yeah, testing this required allocating an AWS dedicated host for at least 24 hours just so I could run an instance. Not a lot of call for cloud based ARM macs I guess, but unfortunately that's the only place where this bug would happen (on investigation I realized I wouldn't have had a problem on any arm64 arch EXCEPT Apple, because this hinges on the CMake OSX_ARCHITECTURES property of targets).

@tomadamatkinson
Copy link
Collaborator

Sounds good we can merge yours first and I can leave the Arm runner PR open for feedback. The CI looks to be working now but I think there are some considerations around the provisioning aspect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build This is relevant to the build system urgent Need to resolve as soon as possible
Projects
None yet
4 participants