之前为了省事,没从源码安装slurm,在 Ubuntu 18.04 上根据这篇博客通过 apt 的方式直接装了slurm-wlm的17.11.02版本。
在安装pyslurm的时候,发现提示找不到slurm的安装路径,或者提示pyslurm版本和本机slurm版本不匹配。
创建软链接 ln -sf /usr/include/slurm-wlm/ /usr/include/slurm mkdir /usr/lib/slurm ln -sf /usr/lib/x86_64-linux-gnu /usr/lib/slurm/lib 安装对应版本pyslurm git clone -b 17-11-12 https://github.com/PySlurm/pyslurm.git cd pyslurm python setup.py build --slurm-inc=/usr/include/ --slurm-lib=/usr/lib/slurm
从github下载pyslurm,进入根目录,运行安装目录报错,提示找不到slurm的安装路径。
root@manager:~/pyslurm# python setup.py build -slurm=/usr/lib/x86_64-linux-gnu/slurm-wlm/ INFO: INFO: Building PySlurm (19.05.0.0) INFO: ------------------------------ INFO: INFO: Cython version 0.26.1 installed INFO: INFO: Clean - checking for objects to clean INFO: Clean - completed ERROR: Build - Cannot locate the Slurm include in /usr/include
由于我们安装的是slurm-wlm,搜索一下其存在的目录,发现是存在相关安装文件和动态链接库的,主要是pyslurm只能识别名为slurm的文件夹
root@manager:~# sudo find / -name slurm-wlm /usr/share/doc/slurm-wlm /usr/share/doc-base/slurm-wlm /usr/lib/x86_64-linux-gnu/slurm-wlm
因此,我们需要手动创建两个slurm软链接指向slurm-wlm的文件夹
ln -sf /usr/include/slurm-wlm/ /usr/include/slurm mkdir /usr/lib/slurm ln -sf /usr/lib/x86_64-linux-gnu /usr/lib/slurm/lib
重新执行安装命令,美滋滋再次报错pyslurm和已安装slurm版本不符
root@manager:~/pyslurm# python setup.py build --slurm-inc=/usr/include/ --slurm-lib=/usr/lib/slurm INFO: INFO: Building PySlurm (19.05.0.0) INFO: ------------------------------ INFO: INFO: Cython version 0.26.1 installed INFO: INFO: Clean - checking for objects to clean INFO: Clean - completed INFO: Build - Found Slurm header in /usr/include/ INFO: Build - Detected Slurm version - 0x110b02 (17.11.02) ERROR: Build - Incorrect slurm version detected, requires Slurm 19.05
主要是因为github直接clone的是最新的19.x版本,而apt安装的slurm只是17.x版本,所以需要下载对应的17.x版本pyslurm
在github找到对应tag之后,重新下载pyslurm
root@manager:~# git clone -b 17-11-12 https://github.com/PySlurm/pyslurm.git Cloning into 'pyslurm'... remote: Enumerating objects: 4658, done. remote: Total 4658 (delta 0), reused 0 (delta 0), pack-reused 4658 Receiving objects: 100% (4658/4658), 4.70 MiB | 1.25 MiB/s, done. Resolving deltas: 100% (3004/3004), done. Note: checking out 'cd4be4668cc8ff2e3da637ddd26f957ced1e502a'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name>
到这里终于下载好对应版本pyslurm了,重新运行安装命令
root@manager:~/pyslurm# python setup.py build --slurm-inc=/usr/include/ --slurm-lib=/usr/lib/slurm INFO: INFO: Building PySlurm (17.11.0.14) INFO: ------------------------------ INFO: INFO: Cython version 0.26.1 installed INFO: INFO: Clean - checking for objects to clean INFO: Clean - completed INFO: Build - Cannot locate the Slurm include in /usr/include/ PySlurm Help ------------ --slurm=PATH Where to look for Slurm, PATH points to the Slurm installation root (default=/usr) --slurm-lib=PATH Where to look for libslurm.so (default=/usr/lib64/slurm) --slurm-inc=PATH Where to look for slurm.h, slurm_errno.h and slurmdb.h (default=/usr/include) --bgq Enable support for BG/Q mode For help with building or installing PySlurm, please ask on the PySlurm Google group at https://groups.google.com/forum/#!forum/pyslurm. If you are sure that you have run into a bug, please report it at https://github.com/PySlurm/pyslurm/issues. INFO: Build - Detected Slurm version - 0x110b02 (17.11.02) INFO: Build - Found Slurm shared library in /usr/lib/slurm/lib INFO: Build - Generating pyslurm/bluegene.pxi file running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/pyslurm copying pyslurm/__version__.py -> build/lib.linux-x86_64-2.7/pyslurm copying pyslurm/__init__.py -> build/lib.linux-x86_64-2.7/pyslurm running build_ext cythoning pyslurm/pyslurm.pyx to pyslurm/pyslurm.c building 'pyslurm.pyslurm' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/pyslurm x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-gnDdqE/python2.7-2.7.17=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/ -I. -I/usr/include/python2.7 -c pyslurm/pyslurm.c -o build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOU RCE=2 -g -fdebug-prefix-map=/build/python2.7-gnDdqE/python2.7-2.7.17=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fde bug-prefix-map=/build/python2.7-gnDdqE/python2.7-2.7.17=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o -L/usr/lib/slurm/lib -L/usr/lib/slurm/ lib/slurm -Wl,-R/usr/lib/slurm/lib/ -Wl,-R/usr/lib/slurm/lib/slurm -lslurmdb -lslurm -o build/lib.linux-x86_64-2.7/pyslurm/pyslurm.so
到此,已经正常安装完pyclurm,可以import pyslurm开干了敢动。