Warning: Today's date is April 14th 2021, and therefore these instructions might no longer be applicable for you!
Too Long Didn't Read (TLDR): Getting the error message below after installing CUDA? Download the .run file and follow the non (optional) code snippets to update your NVIDIA driver.
I started off by installing Ubuntu and afterwards I PRECISELY followed these installation instructions from NVIDIA on how install CUDA. I recommend you do the same.
ERR_NVGPUCTRPERM: Permission issue with Performance Counters
NVIDIA has a page that addresses the issue. Unfortunately, the instructions didn't solve the problem for me. However, I found a solution from rameshgunjal in the NVIDIA forums.
You might have noticed that sudo nv-nsight-cu results in an error. Instead, you have to provide the path: sudo /usr/local/cuda/bin/nv-nsight-cu. The same applies for other applications as well: sudo /usr/local/cuda/bin/nsight-sys.