Setup a blender rendernode on a headless server with crowdrender addon

blender_cr_headless.md

GUIDE: How To create a headless blender renderserver with CrowdRender addon

Requirements

• any headless Linux server
• Blender v2.79 or v2.80 - v2.93 (from package manager or blender.org)
• latest version of CrowdRender addon
• optional: GPU drivers for GPU-rendering

install and activate addon

for blender <= 2.79

$ blender -b --python-console
(InteractiveConsole)
>>> import bpy
>>> bpy.ops.wm.addon_install(overwrite=True, filepath="/path/to/folder/addon.zip")
>>> bpy.ops.wm.addon_enable(module='crowdrender')
>>> bpy.ops.wm.save_userpref() 
>>> exit()

for blender >= 2.80 (thanks to @Marcus-Zhu)

$ blender -b --python-console
(InteractiveConsole)
>>> import bpy
>>> bpy.ops.preferences.addon_install(overwrite=True,filepath='/path/to/addon.zip') 
>>> bpy.ops.preferences.addon_enable(module='crowdrender')
>>> bpy.ops.wm.save_userpref()
>>> exit()

start server

for Crowdrender <= 0.1.6

$ blender -noaudio -b --python ~/.config/blender/2.79/scripts/addons/crowdrender/src/bl_2_79/serv_int_start.py -- -t "server_int_proc"
Crowd Render Server Interface Process ready

for blender v2.80 you will need to change the path to serv_int_start.py accordingly

for Crowdrender >= 0.1.7 / Blender <= 2.79

$ blender -noaudio -b --python ~/.config/blender/2.79/scripts/addons/crowdrender/src/py_3_5_3/serv_int_start.py -- -t "server_int_proc"

for Crowdrender >= 0.1.7 / Blender >= 2.80

$ blender -noaudio -b --python ~/.config/blender/2.80/scripts/addons/crowdrender/src/py_3_7/serv_int_start.py -- -t "server_int_proc"

for Crowdrender >= 0.3.0 / Blender >= 2.80

$ blender -noaudio -b --python ~/.config/blender/2.80/scripts/addons/crowdrender/src/cr/serv_int_start.py -- -t "server_int_proc"

now it should show the IP address of your server and you should be able to connect to the server via any client running in the same network

additional CLI arguments

• -p false - if you're using a virtual machine from a cloud provider, you may not want to permanently store it in your crowdrender account as if you do, each time you use a vm from a cloud provider, you'll get a different vm and they'll start to pile up in your account.
• -ak - called the access key, if you setup multiple vm's on your own servers, best give them each a unique access key. This can be literally anything, but its important if you're using containerisation or vms on your own hardware since if you do, chances are the hardware id of those vm's containers could be the same, which totally prevents you from connecting to more than on of them at once, defeating the purpose of using our software.
• -ct - authorisation token of your Crowdrender cloud account. Using this allows the vm's / containers to post to your cloud account on our servers when they are alive. This means you'll actually be able to see which of them are alive and able to be used and which are offline (or just not posting cause they're not logged in).

you can run the process in the background by using screen and detatch it with Ctrl + a + d

$ screen blender -noaudio -b [...]

create systemd service for automatic startup and restart (thanks to @jan-thoma)

(Blender 2.80)

You create a unit /etc/systemd/system/crowdrender.service

[Unit]
Description=CrowdRender service
After=network.target

[Service]
Type=simple
ExecStart=/path/to/blender -noaudio -b --python /home/example/.config/blender/2.80/scripts/addons/crowdrender/src/py_3_7/serv_int_start.py -- -t "server_int_proc"
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

for Blender 2.79/2.81 etc. you need to change the path to the addon in ExecStart

To start the demon you run:

systemctl start crowdrender.service

To start at boot you enable it:

systemctl enable crowdrender.service

Ok, if you're going to use the -ak option, you should be using the -ct option as well and putting in your CR token, which you can get from your cloud account profile page here -> discovery.crowd-render.com/profile.

All you need to do is login to your profile and then you'll be able to copy your auth token and use it to fulfil the -ct option (ct = crowdrender token).

This automatically populates the vm's to your cloud account, so you should then see them appear in your list of nodes on your master machine (your master machine needs to also be logged in for this to work as the master will query your account to get the list of nodes that are associated with your account).

Doing it this way will automatically populate the access keys for the nodes on your master machine, meaning your master will be able to connect since it has the right key for each machine.

You may also want to consider if you want the VMs to persist in your cloud account. If you do then no further config is necessary, If you'd rather they didn't get saved to your account, then you can use -p false which stops a node from persisting in your account, it will be forgotten about. Persistent nodes will always be shown in your list, even if they are off-line.

Thanks for the help, this did answer my question but now I am just running into performance issues: When I add another node there is a very noticeable latency increase when navigating blender, and when I try to open the project I am working on (probably a worst case scenario: 400mb .obj file) blender crashes. I am watching my resources and I should have plenty of RAM to spare, although maybe there is an error with my system monitor. I guess I can use multiple nodes when working on smaller projects, but I guess for this project I am stuck with only one node. One last thing, is there documentation somewhere with this information? I feel like the questions I am asking should be somewhere, but I have done a lot of googling and found nothing. Thanks again for your help!

Hi Jacob, what are your hardware specs for the machine you are using as the client/master?

Specs are listed below, but before I go into those I do have a bit more information now. I am able to render easily with multiple nodes and I found that the automatic load balancer works pretty well (Nodes finish within about 5% of each other at the end of a render). This was with a much smaller .obj file than the excessively large 400mb obj file, but I still found it weird that crowdrender was what crashed blender (I was able to open the file after removing CR). Maybe it was a case of the straw that broke the camel's back, but it would be nice if there was a failsafe where CR just stops working instead of crashing blender altogether but I don't know the feasibility of that. And now here are the specs of the hardware that I love to play around with too much:

The master:
OS: Manjaro kde kernel: 5.14.1
cpu: i5-7500 @3.8ghz
ram: 16G ddr4

The client(server1):
OS host: Proxmox
OS VM: Ubuntu 21.04
CPU: 2x Xeon X5672 (VM has access to full cpu)
RAM(VM): 16G ddr3

Client(server2):
OS host: Proxmox
OS VM: Ubuntu 21.04
CPU : Xeon X5687 (VM only has access to 6 of the 8 threads)
RAM(VM): 4G

If you don't know much about proxmox, I tend to get less than 5% performance loss in a vm

Great, thanks for the data! If blender crashes due to CR, there might be some information either in the terminal you launch Blender from, or from logging files that CR writes to the machines disk. It would be great if you could take a look and maybe share that here.

Or, if you prefer not to share data from your machine with the whole internet, then you can also submit them in a support request form here

Will take a look through these and see if anything jumps out as the cause.

I know there was nothing unique from the terminal I launched blender (except for killed or something like that) or the client terminals, but next time this happens I will submit a support request with the data from the logs. Thanks for the help!