Sharp tools make good work. Before starting the journey of Databend contribution, let's configure our own development environment. express setup
To facilitate developers to quickly establish a development environment, Databend maintains a complex shell script located in scripts/setup/dev_setup.sh. Only one instruction is needed to complete the development environment configuration:
$ make setup -d
Note: this process will assist in the installation of some python environments, which may affect the local original development environment. It is recommended to execute the following commands in advance to create and enable a dedicated virtual environment.
$ python -m venv .databend $ source .databend/bin/activate
If you encounter the problem of missing dependencies, you can refer to the "step-by-step installation - Test prerequisites" section for installation. Step by step installation
Step by step installation
Here, take Fedora 36 as an example. Considering the differences between different systems and distributions, you may need to install gcc, python and openssl yourself.
1. Install Rust toolchain:
It is recommended to use rustup to manage Rust toolchain. Refer to https://rustup.rs/ Install. For MacOS and Linux users, execute:
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Databend usually uses the newly released nightly tool chain for development, and relevant information is recorded in trust toolchain Toml. Rustup will automatically reload the tool chain when it is used, and only the default configuration is required during installation.
$ cargo build info: syncing channel updates for 'nightly-2022-05-19-x86_64-unknown-linux-gnu' info: latest update on 2022-05-19, rust version 1.63.0-nightly (cd282d7f7 2022-05-18)
2. Installation prerequisites:
The following lists some key steps for installing, building and testing the necessary dependencies, and the descriptions and error messages are presented in the form of comments.
#Common hive meta store needs, thrift not found $ sudo dnf install thrift #OpenSSL sys requires Can't locate FindBin.pm, File/Compare.pm in @INC $ sudo dnf install perl-FindBin perl-File-Compare #is' cmake 'not installed? # The CMAKE_CXX_COMPILER: c++ is not a full PATH and was not found in the PATH., gcc-c++ and llvm will also be installed when installing clang $ sudo dnf install cmake clang
#Functional test and subsequent experience needs $ sudo dnf install mysql #It includes all Python dependencies required by current functional testing and lint $ cd tests $ pip install poetry $ poetry install $ poetry shell #sqllogic test needs (included in the above steps, as needed) (tests) $ cd logictest $ pip install -r requirements.txt #Fuzzy test needs (tests) $ cd fuzz $ pip install -r requirements.txt
Lint must have:
#taplo fmt needs $ cargo install taplo-cli
Editor - Visual Studio Code
- visit https://code.visualstudio.com , install Visual Studio Code.
1. Plug in Recommendation
- Author: The Rust Programming Language
- Provide Rust language support for Visual Studio Code.
- Author: Seray Uzgur
- Help Rust developers manage cargo Dependencies in toml. Only crites IO dependency.
- Author: Vadim Chugunov
- A native debugging tool driven by LLDB. Support debugging C++, Rust and other compiled languages.
Remote - Containers
- Author: Microsoft
- Open any folder or Repo in the Docker container and take advantage of the full functionality of Visual Studio Code.
2. Develop with Dev Containers (For Linux)
Install the "Remote - Containers" plug-in. After opening the Databend, you will see a pop-up window in the lower right corner and prompt "Reopen in Container".
Install and start the docker corresponding to your distribution according to Docker Docs - Install. Take Fedora 36 as an example, the steps are as follows:
#Remove old version docker $ sudo dnf remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-selinux \ docker-engine-selinux \ docker-engine #Set up repository $ sudo dnf -y install dnf-plugins-core $ sudo dnf config-manager \ --add-repo \ https://download.docker.com/linux/fedora/docker-ce.repo #Install Docker Engine $ sudo dnf install docker-ce docker-ce-cli containerd.io docker-compose-plugin
Add the current User to docker group reference Docker Docs - PostInstall In the manage docker as a non root user section, you may need to restart.
The steps are as follows
#Add docker user group $ sudo groupadd docker #Add users to the docker group $ sudo usermod -aG docker $USER #Activate changes $ newgrp docker #Change permissions to fix permission denied $ sudo chown "$USER":"$USER" /home/"$USER"/.docker -R $ sudo chmod g+rwx "$HOME/.docker" -R
$ sudo systemctl start docker
Click "open remote window" in the lower left corner and select "Reopen in Container" to experience.
Other utility recommendations
Here are some practical tools that may be helpful to the development of Databend, which can be selected according to the actual situation.
Lightweight, responsive, and infinitely customizable high-value terminal!
Refer to starship - installation for installation.
curl -sS https://starship.rs/install.sh | sh
Command line benchmarking tool.
reference resources hyperfine - installation Install.
cargo install hyperfine
Databend is an open-source, flexible, low-cost data warehouse that can do real-time analysis based on object storage. We look forward to your attention and explore cloud native data warehouse solutions to create a new generation of open source Data Cloud.
- Databend document: https://databend.rs/
- Twitter: https://twitter.com/Datafuse_...
- Slack: https://datafusecloud.slack.com/
- Wechat: Databend
- GitHub : https://github.com/datafusela...
The article was first published on official account: Databend