Environment variables

The following table gives some of the environment variables that can be set before running zCFD. These are generally for advanced users and can affect the way zCFD runs on your system.

Variable	Default	Description
ZCFD_GPUDIRECT	1 on GPU ranks	Set to 1 to pass GPU buffers directly to a GPU-aware MPI: Open MPI, Platform MPI or Cray MPI. When the `nv_peer_mem` kernel module is not loaded, zCFD sets it to 0 on a multi-node run, and on a 16-GPU node running more than eight ranks.
ZCFD_PRE_LAUNCH	unset	The path to a script that is sourced just before the solver starts, so that it can override the environment zCFD has set up.
CUDA_VISIBLE_DEVICES	unset	Restricts the GPUs zCFD can use. Typically used to exclude an incompatible GPU or the display GPU.
ZCFD_DISABLE_PINNED	unset	When set, GPU transfer buffers are allocated in ordinary host memory instead of pinned memory. Set it under WSL on Windows, where CUDA limits the amount of pinned memory.
ZCFD_NCCL	unset	Set to 1 to use the NVIDIA NCCL library for parallel exchanges, in builds that include NCCL.
PMPI_CUDA	set by zCFD under Platform MPI	Enables GPU-aware MPI transfers with Platform MPI (custom builds only), when `ZCFD_GPUDIRECT` is 1.
MPICH_GPU_SUPPORT_ENABLED	unset	Set to 1 to enable GPU-aware MPI transfers with Cray MPI (custom builds only), when `ZCFD_GPUDIRECT` is 1.
ZCFD_DISPATCH_SYNC	unset	When set, every kernel launch is synchronous.
ZCFD_DEV_OVERSUBSCRIBE_GPU	unset	Experimental. When set, several MPI ranks may share one GPU. When unset, each MPI rank is given its own GPU. Intended for debugging and testing with limited GPU resources.