[ML-252] Enable python package installation #253
Conversation
|
Could you separate this to different PR. one to only enable python package, another one to update scripts. don't mix them together. |
dev/prepare-build-deps-gpu.sh
Outdated
| @@ -75,6 +75,6 @@ patchelf --set-soname libirng.so $TARGET_DIR/libirng.so | |||
|
|
|||
| cp $CMPLR_ROOT/linux/compiler/lib/intel64_lin/libimf.so $TARGET_DIR | |||
| cp $CMPLR_ROOT/linux/lib/libOpenCL.so.1 $TARGET_DIR | |||
| cp $CMPLR_ROOT/linux/lib/libsycl.so.5 $TARGET_DIR | |||
| cp $CMPLR_ROOT/linux/lib/libsycl.so.6 $TARGET_DIR | |||
|
|
|||
There was a problem hiding this comment.
could you add some file existence check and report error for file missing?
I think we can build a list of files mapping from a specifc oneapi version to a list for files. such as oneapi_2023_1_libs=( libsycl.so.6 libintlc.so.5 ... ), then we just need to add a new version and update the file list when new oneapi come up.
| @@ -17,4 +17,4 @@ | |||
| #ADDRS="0","1","2","3","4","5","6","7" | |||
| #echo {\"name\": \"gpu\", \"addresses\":[\"$ADDRS\"]} | |||
|
|
|||
| echo {\"name\": \"gpu\", \"addresses\":[\"0\",\"1\",\"2\",\"3\"]} | |||
| echo {\"name\": \"gpu\", \"addresses\":[\"0\"]} | |||
There was a problem hiding this comment.
It's better to keep at least 2 GPUs as example. pls revise below as well. And could we move this file to a common place since it's common for all GPU examples.
|
|
||
| # Should run in standalone mode | ||
| time $SPARK_HOME/bin/spark-submit --master $SPARK_MASTER \ | ||
| time $SPARK_HOME/bin/spark-submit --master spark://kunpeng:7077 \ |
There was a problem hiding this comment.
don't use internal hostname
pyoap/setup.py
Outdated
There was a problem hiding this comment.
rename top level dir pyoap to python according to pyspark name structure
rename all pyoap in the file to oapmllib since oap is an umbrella project. oap mllib is only part of it.
|
please rebase to master the resolve comments. |
argentea
force-pushed
the
pip-package
branch
2 times, most recently
from
75b021f
to
c90ad22
Compare
|
@argentea could you rebase to master? |
argentea
force-pushed
the
pip-package
branch
2 times, most recently
from
dc737b8
to
c498587
Compare
Todo: