安装新版transformers后,啪,报了个错
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /data/jcdu/miniconda3/envs/main/lib/python3.7/site-packages/tokenizers/tokenizers.cpython-37m-x86_64-linux-gnu.so)
解决方案:安装旧版tokenizers
升级glibc也行,但是不小心就会干掉系统,还是罢了
conda install -c huggingface transformers tokenizers=0.10.1
tokenizers 0.10.3需要glibc2.29
tokenizers 0.10.1对应的transformers版本是4.11.3
顺便记录下ubuntu 18.04 能正常用的环境版本,conda env export > py36.yaml
导出,conda env create -f py36.yaml
name: main channels: - pytorch - huggingface - defaults dependencies: - _libgcc_mutex=0.1=main - _openmp_mutex=5.1=1_gnu - abseil-cpp=20211102.0=hd4dd3e8_0 - aiohttp=3.8.1=py37h7f8727e_1 - aiosignal=1.2.0=pyhd3eb1b0_0 - arrow-cpp=8.0.0=py37h60b952e_0 - async-timeout=4.0.1=pyhd3eb1b0_0 - asynctest=0.13.0=py_0 - attrs=21.4.0=pyhd3eb1b0_0 - aws-c-common=0.4.57=he6710b0_1 - aws-c-event-stream=0.1.6=h2531618_5 - aws-checksums=0.1.9=he6710b0_0 - aws-sdk-cpp=1.8.185=hce553d0_0 - blas=1.0=mkl - boost-cpp=1.73.0=h7f8727e_12 - bottleneck=1.3.5=py37h7deecbd_0 - brotli=1.0.9=he6710b0_2 - brotlipy=0.7.0=py37h27cfd23_1003 - bzip2=1.0.8=h7b6447c_0 - c-ares=1.18.1=h7f8727e_0 - ca-certificates=2022.07.19=h06a4308_0 - certifi=2022.6.15=py37h06a4308_0 - cffi=1.15.0=py37hd667e15_1 - charset-normalizer=2.0.4=pyhd3eb1b0_0 - click=8.0.4=py37h06a4308_0 - cryptography=37.0.1=py37h9ce1e76_0 - cudatoolkit=11.3.1=h2bc3f7f_2 - dataclasses=0.8=pyh6d0b6a4_7 - datasets=2.4.0=py_0 - dill=0.3.4=pyhd3eb1b0_0 - ffmpeg=4.2.2=h20bf706_0 - filelock=3.6.0=pyhd3eb1b0_0 - freetype=2.11.0=h70c0345_0 - frozenlist=1.2.0=py37h7f8727e_0 - fsspec=2022.3.0=py37h06a4308_0 - gflags=2.2.2=he6710b0_0 - giflib=5.2.1=h7b6447c_0 - glog=0.5.0=h2531618_0 - gmp=6.2.1=h295c915_3 - gnutls=3.6.15=he1e5248_0 - grpc-cpp=1.46.1=h33aed49_0 - huggingface_hub=0.2.1=pyhd3eb1b0_0 - icu=58.2=he6710b0_3 - idna=3.3=pyhd3eb1b0_0 - importlib-metadata=4.11.3=py37h06a4308_0 - importlib_metadata=4.11.3=hd3eb1b0_0 - intel-openmp=2021.4.0=h06a4308_3561 - joblib=1.1.0=pyhd3eb1b0_0 - jpeg=9e=h7f8727e_0 - krb5=1.19.2=hac12032_0 - lame=3.100=h7b6447c_0 - lcms2=2.12=h3be6417_0 - ld_impl_linux-64=2.38=h1181459_1 - libboost=1.73.0=h28710b8_12 - libcurl=7.82.0=h0b77cf5_0 - libedit=3.1.20210910=h7f8727e_0 - libev=4.33=h7f8727e_1 - libevent=2.1.12=h8f2d780_0 - libffi=3.3=he6710b0_2 - libgcc-ng=11.2.0=h1234567_1 - libgfortran-ng=7.5.0=ha8ba4b0_17 - libgfortran4=7.5.0=ha8ba4b0_17 - libgomp=11.2.0=h1234567_1 - libidn2=2.3.2=h7f8727e_0 - libnghttp2=1.46.0=hce63b2e_0 - libopus=1.3.1=h7b6447c_0 - libpng=1.6.37=hbc83047_0 - libprotobuf=3.20.1=h4ff587b_0 - libssh2=1.10.0=h8f2d780_0 - libstdcxx-ng=11.2.0=h1234567_1 - libtasn1=4.16.0=h27cfd23_0 - libthrift=0.15.0=hcc01f38_0 - libtiff=4.2.0=h2818925_1 - libunistring=0.9.10=h27cfd23_0 - libvpx=1.7.0=h439df22_0 - libwebp=1.2.2=h55f646e_0 - libwebp-base=1.2.2=h7f8727e_0 - lz4-c=1.9.3=h295c915_1 - mkl=2021.4.0=h06a4308_640 - mkl-service=2.4.0=py37h7f8727e_0 - mkl_fft=1.3.1=py37hd3c417c_0 - mkl_random=1.2.2=py37h51133e4_0 - multidict=5.2.0=py37h7f8727e_2 - multiprocess=0.70.12.2=py37h7f8727e_0 - ncurses=6.3=h5eee18b_3 - nettle=3.7.3=hbbd107a_1 - numexpr=2.8.3=py37h807cd23_0 - numpy=1.21.5=py37h6c91a56_3 - numpy-base=1.21.5=py37ha15fc14_3 - openh264=2.1.1=h4ff587b_0 - openssl=1.1.1q=h7f8727e_0 - orc=1.7.4=h07ed6aa_0 - packaging=21.3=pyhd3eb1b0_0 - pandas=1.3.5=py37h8c16a72_0 - pillow=9.2.0=py37hace64e9_1 - pip=22.1.2=py37h06a4308_0 - protobuf=3.20.1=py37h295c915_0 - pyarrow=8.0.0=py37h992f0b0_0 - pycparser=2.21=pyhd3eb1b0_0 - pyopenssl=22.0.0=pyhd3eb1b0_0 - pyparsing=3.0.4=pyhd3eb1b0_0 - pysocks=1.7.1=py37_1 - python=3.7.13=h12debd9_0 - python-dateutil=2.8.2=pyhd3eb1b0_0 - python-xxhash=2.0.2=py37h7f8727e_0 - python_abi=3.7=1_cp37m - pytorch=1.12.0=py3.7_cuda11.3_cudnn8.3.2_0 - pytorch-mutex=1.0=cuda - pytz=2022.1=py37h06a4308_0 - pyyaml=6.0=py37h7f8727e_1 - re2=2022.04.01=h295c915_0 - readline=8.1.2=h7f8727e_1 - regex=2022.7.9=py37h5eee18b_0 - requests=2.28.1=py37h06a4308_0 - sacremoses=master=py_0 - scikit-learn=1.0.2=py37h51133e4_1 - scipy=1.7.3=py37hc147768_0 - setuptools=61.2.0=py37h06a4308_0 - six=1.16.0=pyhd3eb1b0_1 - snappy=1.1.9=h295c915_0 - sqlite=3.38.5=hc218d9a_0 - threadpoolctl=2.2.0=pyh0d69192_0 - tk=8.6.12=h1ccaba5_0 - tokenizers=0.10.1=py37_0 - torchaudio=0.12.0=py37_cu113 - torchvision=0.13.0=py37_cu113 - tqdm=4.64.0=py37h06a4308_0 - transformers=4.11.3=py_0 - typing-extensions=4.1.1=hd3eb1b0_0 - typing_extensions=4.1.1=pyh06a4308_0 - urllib3=1.26.9=py37h06a4308_0 - utf8proc=2.6.1=h27cfd23_0 - wheel=0.37.1=pyhd3eb1b0_0 - x264=1!157.20191217=h7b6447c_0 - xxhash=0.8.0=h7f8727e_3 - xz=5.2.5=h7f8727e_1 - yaml=0.2.5=h7b6447c_0 - yarl=1.6.3=py37h27cfd23_0 - zipp=3.8.0=py37h06a4308_0 - zlib=1.2.12=h7f8727e_2 - zstd=1.5.2=ha4553b6_0 prefix: /data/jcdu/miniconda3/envs/main