The environment of different server clusters in the lab is slightly different, and the first time mmdet3d is configured, many problems are encountered. The content of this article is mainly for your own notes. takeaway is the next two install sh； Write some pits in front.

anaconda installation and source change

The server does not have my administrator permission and the underlying environment is shared, so it is best to use conda installation package instead of apt get
Connecting conda and pip in China is too slow, especially in Shanghai. pip often loses connection when the network is bad.
Download conda: Download Anaconda from Anaconda's official website_ xxx. sh
Display conda source configuration: conda config --set show_channel_urls yes

. condarc into Tsinghua source

It's not easy to use, but it's faster:

ssl_verify: true
channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
show_channel_urls: true

pip source switching

Generally, add -i directly after pip install https://pypi.tuna.tsinghua.edu.cn/simple/

conda env configuration

Create a conda environment. You should use both python3.7 and 3.8.

conda create -n open-mmlab python=3.7
conda activate open-mmlab

pytorch and cudatoolkit

If used pytorch official website If you install conda, you need to install cudatoolkit. If you install it according to its pip, you don't need to, for example:

pip install torch==1.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html

Don't forget to install torchvision~
cuda warning will be available when mmdet or waymo is running. In fact, it is OK. If it is wrong, it is usually another problem.

Waymo open dataset installation

It can be officially installed according to mmdet, but I want to use the v1.3.1 feature, so I have to install the new version of the supporting library 1.4.7.

pip install waymo-open-dataset-tf-2-6-0==1.4.7 -i https://pypi.tuna.tsinghua.edu.cn/simple/

You can add – user so that it is not installed in the conda environment.
Note that camera/ops/py is missing from the pip package in 1.4.7_ camera_ model_ ops. Py, after installation, in waymo_ open_ dataset/third_ You can find it in party/camera/ops/ and directly copy it to the package installation directory. Generally

.local/lib/python3.7/site-packages/waymo_open_dataset/camera/ops

perhaps

anaconda3/envs/open-mmlab/lib/python3.7/site-packages/waymo_open_dataset/camera/ops

waymo asked the issue that there was a problem with 1.4.7. They told me that they fixed it and issued 1.4.8. As a result, there was a bug when installing dependency in pip... openexr in my environment could not be found. It is better to install 1.4.7.
It is better to install waymo first and then mmdet, so that you can install tensorflow first. In addition, you may need to install protobuf-3.20, because the installation of other packages may rely on the updated version of protobuf.
If you want to compile the evaluation tool of waymo for model evaluation, you need to waymo's github repo clone a corresponding version of the project, and then compile it with bazel. You can install bazel directly by conda install bazel.

Open mmlab series installation

Official tutorial for mmdet3d getting_started Also very good. But for the first time, I installed his beta v1.0.0rc2, and there were a lot of bug s_ ( ¦ 3」∠)_
Install mmcv, mmdet, mmsegmentation, ninja, and then mmdet3d
On the node series machines in Shanghai, the glibc version is relatively old. mmcv-1.5.1 can't be installed. mmcv-1.4.8 can be used. I haven't tried anything else.
I use mmdet3d v1.0.0rc2 to process waymo dataset s. See the previous article for specific methods. However, it was found that there were problems with the latest version of training and evaluation. Even the official fcos3d training was wrong, and loss could not be reduced. Therefore, a stable 0.17.3 was used.
In addition to mmdet3d, other direct pip s are sufficient. Mmdet3d needs to go to the corresponding version of git clone.
The specific mmlab Series Library version corresponding table can be checked on the official website. The following two configurations can be run.

If you want to use rc2 to train or test, you may encounter the error of pytorch multithreading. The solution is to add torch multiprocessing. set_ start_ method(‘fork’)

install_mmdet3d_0_17_3.sh

# conda activate open-mmlab
pip install waymo-open-dataset-tf-2-6-0==1.4.7 -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio==0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.2/index.html
pip install mmdet==2.19.1
pip install mmsegmentation==0.14.1
pip install ninja
#wget https://github.com/open-mmlab/mmdetection3d/archive/refs/tags/v0.17.3.zip
#or git clone and checkout 940e429
#If the git clone is not available, download it to the local computer first, and then scp it to the server
cd ~/mmdet3d/
pip install -v -e . -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip install protobuf==3.20
conda install bazel
# don't forget to copy py_camera_ops.oy to waymo dataset!

install_mmdet3d_rc2.sh

# conda activate open-mmlab
source ~/anaconda3/bin/activate
conda activate mmlab-rc2
pip install waymo-open-dataset-tf-2-6-0==1.4.7 -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip3 install torch==1.11.0+cu113 torchvision==0.12.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
pip install mmcv-full==1.4.8
pip install mmdet
pip install mmsegmentation
pip install ninja
## wget https://github.com/open-mmlab/mmdetection3d/archive/refs/tags/v1.0.0rc2.zip
#If the git clone is not available, download it to the local computer first, and then scp it to the server
cd ~/mmdetecion3d-rc2-waymo-dataonly
pip3 install -e . -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip install protobuf==3.20
conda install bazel
# don't forget to copy py_camera_ops.oy to waymo dataset!

Code synchronization between Shanghai and Beijing servers

Now take Beijing machine as git origin, and set the remote of Shanghai machine as Beijing to realize code synchronization between the two groups of servers.
Some necessary commands:

git clone ssh://$ServerName/~/mmdet3d
git remote add origin ssh://$ServerName/~/mmdet3d

It needs to be modified on the Beijing machine git/config:

[receive]
denyCurrentBranch = updateInstead

misc

If you use Tsinghua source conda to download pytorch, you need to go to Tsinghua source website Check the version correspondence. Otherwise, it is easy to download pytorch to the cpu version. Give an example:
conda install pytorch=1.11.0 cudatoolkit=11.3
Install torchvision:conda install torchvision -c pytorch
It seems that the package dependency analysis of Tsinghua source is not very good, and the pytorch source is better. However, the pytorch source is too slow in China, so pip is better