Install#
Hiperwalk offers multiple installation paths depending on your requirements. You can install Hiperwalk without High-Performance Computing (HPC) support, or opt for a Docker-based installation that includes HPC capabilities.
Choose your installation method:
Basic Installation without HPC support
-
Docker Installation
Local Installation (for advanced users)
Basic Installation#
Hiperwalk can be installed without HPC support. This installation relies on certain Python libraries.
Installation Steps#
Hiperwalk can be conveniently installed using pip. To begin, ensure that pip is installed on your system.
sudo apt install python3-pip
Hiperwalk has several Python dependencies, including numpy, scipy, networkx, and matplotlib.
Warning
If you have older versions of these packages, they will likely be updated. If you prefer not to have them updated, we recommend creating a virtual environment.
The following command will install Hiperwalk and its dependencies:
pip install hiperwalk
To verify the success of the installation, you can execute any example code available in the GitHub repository or any Jupyter notebook in the Examples section. Alternatively, you can proceed to the Tutorial section.
To update an older version of the hiperwalk package:
pip install hiperwalk --upgrade
To uninstall the hiperwalk package:
pip uninstall hiperwalk
HPC-Enabled Installation#
Hiperwalk supports HPC through either a Docker installation or a local installation. HPC capabilities can be leveraged using parallelism on multicore CPUs and, optionally, NVIDIA GPUs.
The following instructions will guide you through setting up CPU-based HPC, with optional GPU support.
GPU Prerequisites#
Skip this section if GPU support is not needed. To install the GPU driver, you can follow this tutorial for installing NVIDIA drivers. Below, we have outlined the essential steps.
Warning
If you have any concerns about the following commands, especially for installing the NVIDIA driver, please contact your local support team for assistance.
First, you’ll need to identify your GPU by running the following command:
lspci | grep -e VGA
You can then verify if the outputted GPU is CUDA compatible. If it is, execute the following command:
ubuntu-drivers devices
This will list the available drivers for your GPU. We recommend
installing the driver tagged with recommended at the end.
The driver’s name typically follows the format nvidia-driver-XXX
where XXX is a specific number.
For the subsequent steps in the installation process, substitute XXX
as required. To install the GPU driver, execute the following command:
sudo apt install nvidia-driver-XXX
Finally, reboot you computer. After rebooting, if the installation was successful, running the command
nvidia-smi
should display GPU information such as the name, driver version,
CUDA version, and so on. Alternatively, you can verify the
availability of the NVIDIA Settings application by
pressing the Super key on your keyboard and
typing nvidia settings.
Docker Installation#
Using Hiperwalk on Docker offers numerous benefits. Docker, a form of containerization, automatically includes Hiperwalk, its prerequisites, and all HPC-enabling software. It provides a lightweight, portable, and scalable environment, ensuring seamless deployment across different systems. Docker simplifies dependency management, updates, and configuration replication, enhancing consistency and reliability.
Installation Steps#
Single time configuration of Docker.
Start by updating the package lists:
sudo apt-get update
Step 1. Add Docker’s official GPG key:
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
Step 2. Add the repository to Apt sources:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Update the package lists again:
sudo apt-get update
Step 3. Install Docker:
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
Step 4. Add the user to Docker group:
sudo groupadd docker
sudo usermod -aG docker $USER
Step 5. Log out of your session and then log back in.
Step 6. Test your Docker installation:
docker run hello-world
NVIDIA Container#
Skip this section if GPU support is not needed. If the prerequisites are not installed, refer to GPU Prerequisites.
Single time configuration of NVDIA container toolkit.
Step 1. Configure the repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
Step 2. Update and install:
sudo apt-get update
sudo apt-get install -y nvidia-docker2
Step 3. Configure Docker to use NVIDIA in rootless mode:
nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json
systemctl --user restart docker
sudo nvidia-ctk config --set nvidia-container-cli.no-cgroups --in-place
It may be necessary to edit the following file and
change the value of no-cgroups to false:
sudo vi /etc/nvidia-container-runtime/config.toml
Then, execute:
sudo systemctl restart docker
Step 4. Test the GPU access:
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu20.04 nvidia-smi
Run Docker#
Create a folder where you want to save your notebooks. This folder will store all the examples you create in the browser.
Open a terminal and navigate to the folder you just created.
To run Hiperwalk in Docker with CPU-only support, execute the following command:
docker run --rm -v $(pwd):/home/jovyan/work -p 8888:8888 hiperwalk/hiperwalk:2.0.b0
To run Hiperwalk in Docker with GPU and CPU support, execute the following command:
docker run --rm --gpus all -v $(pwd):/home/jovyan/work -p 8888:8888 hiperwalk/hiperwalk:2.0.b0
Open your web browser and go to the URL that appears on the screen,
starting with 127.0.0.1. Alternatively, you can directly click
on the hyperlink.
In the Jupyter environment, access the work/ folder.
All your data and notebooks will be saved in the folder
you have created above
(external to the Docker container).
Local Installation#
This section and the ones that follow are intended for developers of the Hiperwalk package. Before proceeding, it is advisable to update and upgrade your Ubuntu packages. Execute the following commands:
sudo apt update
sudo apt upgrade
The steps described here will cover identifying the GPU, installing the GPU drivers, hiperblas-core, hiperblas-opencl-bridge, pyhiperblas, and all necessary Python libraries. Next, run the following commands to install the prerequisites:
sudo apt install git
sudo apt install g++
sudo apt install cmake
sudo apt install libgtest-dev
sudo apt install python3-distutils
sudo apt install python3-pip
pip install pytest
These newly installed programs serve the following purposes:
git: used to download hiperblas-core, hiperblas-opencl-bridge, pyhiperblas, and hiperwalk;
g++: used for compiling hiperblas-core, and hiperblas-opencl-bridge;
cmake: essential for compiling hiperblas-core, hiperblas-opencl-bridge;
libgtest-dev: verifies the successful installation of hiperblas-core, and hiperblas-opencl-bridge;
python3-distutils: aids in the installation of pyhiperblas;
python3-pip: necessary for installing Python libraries;
pytest: helps test pyhiperblas.
Although it’s not essential, we recommend installing FFmpeg, which is used for generating animations.
sudo apt install ffmpeg
NVIDIA Toolkit#
Skip this section if the GPU Prerequisites are not installed.
Once the GPU drivers have been successfully installed, it’s necessary to install the NVIDIA Toolkit, allowing hiperblas-core to use CUDA. To do this, access CUDA toolkit Downloads and select the options of Operating System, Architecture, Distribution, and Version, according to your machine, and the desired Installer Type. Then, follow the instructions of the Base Installer section.
To verify the correct installation of the NVIDIA Toolkit,
you can check if the nvcc compiler has been installed.
This can be simply done by running the following command:
nvcc --version
Hiperblas#
For HPC support, Hiperwalk uses hiperblas-core, hiperblas-opencl-bridge, and pyhiperblas.
The information in this guide is compiled from Paulo Motta’s blog, hiperblas-core github, and pyhiperblas github.
It is strongly recommended that hiperblas-core,
hiperblas-opencl-bridge, and pyhiperblas
are installed (i.e. cloned) in the same directory.
In this guide, we will install both projects into the home directory.
In Linux, the tilde (~) serves as an alias for the home directory.
hiperblas-core#
Firstly, clone the repository in the home directory.
cd ~
git clone https://github.com/hiperblas/hiperblas-core.git
Next, navigate to the hiperblas-core directory to compile and install the code.
cd ~/hiperblas-core
cmake .
make
sudo make install
sudo ldconfig
The ldconfig command creates a link for the newly installed hiperblas-core,
making it accessible for use by pyhiperblas.
Before moving forward, reboot your computer to
ensure that the ldconfig command takes effect.
After rebboting,
run the following ln command to create
a symbolic link to another directory.
sudo ln -s /usr/local/lib /usr/local/lib64
To verify the successful installation of hiperblas-core,
execute the vector_test and matrix_test tests.
cd ~/hiperblas-core
./vector_test
./matrix_test
hiperblas-opencl-bridge#
Skip this section if the GPU Prerequisites are not installed.
The installation of the hiperblas-opencl-bridge is very similar to the installation of hiperblas-core. To install hiperblas-opencl-bridge, first clone the repository into the same directory hiperblas-core was cloned. In this guide, we cloned hiperblas-core into the home directory.
cd ~
git clone https://github.com/hiperblas/hiperblas-opencl-bridge.git
Now, enter the new hiperblas-opencl-bridge directory to compile and
install the code.
cd hiperblas-opencl-bridge
cmake .
make
sudo make install
To verify the succesful installation of hiperblas-opencl-bridge, execute the tests
./vector_test
./matrix_test
pyhiperblas#
To install pyhiperblas, first clone the repository into the same directory hiperblas-core was cloned. In this guide, we cloned hiperblas-core into the home directory. Thus, execute:
cd ~
git clone https://github.com/hiperblas/pyhiperblas.git
Before installing pyhiperblas,
install numpy using the sudo command.
sudo pip install numpy
Next, navigate to the newly created pyhiperblas directory to install it.
cd pyhiperblas
sudo python3 setup.py install
To verify whether the installation was successful, run the following test:
python3 test.py
Hiperwalk#
To finish the local hiperwalk installation, issue the same commands of the Basic Installation section.