HOW TO SETUP AN OPEN SOURCE RENDERFARM WITH WINDOWS


There must be some translation errors and I apologize for it. I'll try to correct them as soon as I can ;)

         Introduction         


For all the "CG artists" who wish like to improve the productivity of their creative process, the use of a "renderfarm" is highly recommended. A renderfarm is simply computers connected in network to help you calculate your image sequences.

The title of this article can be in a way considered as misleading. A real open source renderfarm would only use open source software running on an open source OS like Linux. Nevertheless, for all of those who needs to work with software like After Effect or Cinema4D a windows/Mac running system is the only way to go. So waiting for the Linux big jump, we can already use some Windows compatible open source solutions like Natron ( compositing ) and Afanasy (renderfarm manager).

This tutorial based on my own experience will show you how to set up a render farm running on windows and how to use Afanasy to launch and control your rendering process through the network.

renderfarm



         1. Network Settings        


    Step 1: configure the physical network    


Required equipment:
  1. a master computer called WORKSTATION.
  2. several computers called CLIENTS.
  3. "modem" and a "switch" to connect the whole network with ethernet cables.

Tip: I recommend to connect the modem to the switch rather than use the second ethernet port of your Workstation (if you have one). So the computers will all get an IP delivered by the modem and it will be easier to locate them on the network and control them by internet. Plus it will make possible the OS updates.

Your network should look like that:

network



     Step 2: Windows settings    


   1. Turn on "network discovery"   


 For every computer (workstation + clients) follow these steps:
  1. "Start => Control panel => network => advanced parameters".
  2. turn on "network discovery" and deactivate "protection by password", for Public and Home Network.
network discovery

Other computers available on the network should be listed in the left tab.

Tip: It is not necessary to create a "bridge" through a "home group". We just need to map all the computers on the network.

 

 

 

   2. Create a work folder   


Create a folder on your workstation (wherever you want). This folder will have to contain all the elements of the project, necessary for its rendering. Whether it is for Maya, After-Effect, Nuke, Natron ...etc., the logic remains the same: the access path of your files has to be exactly the same for all your computers. This folder that we gonna named "WORK", will contain:
  1. the project file
  2. the external references (files, images, videos) 
  3. the save folder (to set up in your export parameters)

So when your renderfarm manager will distribute a job to your clients, they will all use the same path and save your files to the same folder.

Do not forget to share this folder:
  1. Right click on it => Properties => ...
  2. In the drop-down menu select "Everyone" then "Add"
  3. select "Reading / writing" then "Share"
share folder



   3. Map network drive    


Once your folder " WORK " is created on your workstation it is necessary, for each computer (workstation and clients) to follow these steps:
  1. Open Windows explorer.
  2. Click "Map network drive" then choose an arbitrarily letter, let's say "X", and select your folder "WORK".
  3. "reconnect at logon" remains marked.
map network drive


A new network drive should appeared with your local disk (C:). It has to be named:

" Name-file (\\ Host-Name) (letter of your choice:) "


The "Host-Name" is your computer name given at Windows install (If you don't remember it go to "Control panel => System", then below on the right). So in our case we have :

"WORK (\\WORKSTATION-PC) (X:)"

          
Repeat this operation on every clients. Be careful to choose the same letter (X in our example) as well as the same folder (WORK through the network by WORKSTATION).

network drive

Every computers have the same network drive with the same name targeting towards the same folder. Your computers are now set up in renderfarm! Now you have to install your softwares and plugins on every one.




        2. Installing AFANASY on Windows         



To help you work with your renderfarm you will find a lot of different manager software capable of submitting, distribute and organize your jobs. Most are paying like Deadline, Royal Render, PipelineFX etc...
... However, some of them are OPEN SOURCE, such as AFANASY, which is part of an open source toolkit dedicated to graphic creation called CGRU (http://cgru.info/home) and available for all platforms Windows, Mac and Linux.

 [ CGRU is a CG tools pack includes AFANASY - Free Open Source Render Farm Manager. Name "CGRU" - came from CG - Rules. It has two meanings: computer graphics principles and computer graphics is a cool thing. Name "Afanasy" - came from a Greek name meaning "immortal". ]


Afanasy proposes "submission scripts" for these applications:

Afanasy looks like this:



Afanasy UI2

Afanasy UI3

 

 

   Step 1: server setup (workstation)  


The server is the computer that will dispatch automatically the different tasks to the other client computers. So Afanasy will be preferably installed on your workstation:

  1. Download the last release of CGRU for MS Windows here: http://cgru.info/downloads
  2. Unzip the folder and paste the content ("cgru-Windows") directly into the root of your local disk : C:\cgru-windows.
Afanasy folder

You're almost done! Two more things to specify:

  • The IP address of the server.
  • The path of the ".EXE" file of your softwares.

To do that you will need an embedded software called "KEEPER". Go to "cgru-Windows" folder and execute "start.cmd".  Keeper should now pop up in the notifications icons list. Then click right to get the options menu.

Keeper UI
  1. Right click => Afanasy => Set to server...
  2. Indicate the IP address of your workstation (to know it go to "Start => write "CMD" in the search field then "enter", in the console write " ipconfig " then "enter", your IP appears on the 8th line in front of "Address IPv4 in ....... ").
  3. Again Right click => Software => Setup Soft => select the one who concerns you (Maya, Blender ...) then indicate the path towards his .exe.
set server

Tip: indicate your IP rather than your host-name. For any reason I can't explain it works for me only with the IP address.

That's it! Nothing more to do. Now, to launch the server, it is necessary to reach the Afanasy commands. There are two options:
  • Go to C:\cgru-windows\start\AFANASY\
  • On Keeper Right click => Afanasy => [browse]
You will find all the commands necessary to make AFANASY work:
  • _afserver.cmd: launch the server.
  • _afstarter.cmd: launch a job.
  • Afwatch.cmd: to manage your Jobs, renders and users.
  • Render.cmd: launch the render.
Keeper commands
Follow these steps:
  1. Execute "_afserver.cmd"
  2. Then "afwatch.cmd" or on Keeper "Right click = > Afanasy = > Start Watch". A window will pop up with the tab "Renders" where you should see the name of your workstation. In my example : " workstation-pc " afwatch says that it's "Offline". It is normal, the render is not launched yet. The server communicates but render nothing, so it is necessary to launch also the render on my workstation to make it as a potential unit of calculation in afwatch.
  3. Execute "render.cmd" or on Keeper "Right click => Afanasy => Local Render..."

You should see:
  • Afwatch
afwatch UI1
  • _afserveur
afserver UI
  • Afwatch once the server launched
afwatch UI2

  • Once the server AND the render both launched

afwatch UI3


Your server is now configured and your workstation is ready to make any job. Do a test job and execute "_afstarter.cmd" (don't forget to give the path of the .exe file your software). You can alternatively use the submission scripts to send the jobs directly from your applications.

  


 

   Step 2: renders setup   


Render will "make the job". You can start it on clients AND workstation.

To install it on your clients, follow the same steps as on your workstation (be careful on step n°4):
  1. Copy-paste "cgru-Windows" folder directly into the root of your local disk (C:\cgru-Windows).
  2. Browse "cgru-Windows" double-click on "start.cmd".
  3. On Keeper: right click => Afanasy => Set server...
  4. INDICATE THE IP ADDRESS OF YOUR WORKSTATION!!!
  5. Again on Keeper: right click => Software => Setup Soft => select the software for which you want to use the render (indicate the path to its .exe, e.g. maya.exe or cinema4d.exe).
  6. Go to C:\cgru-windows\start\AFANASY\ and execute "render.cmd" or on Keeper "Right click => Afanasy => Local Render".

Verify that  the window command ends with " Render registered ". It is very important, it means that your client has just been correctly located by the server. Last check : go bacl to afwatch on the workstation : the new client should appear in the tab "Renders" tab.

render UI

Your client is now configured and your workstation is ready to make any job. Repeat this operation on many clients as you need. Then send your jobs by executing "_afstarter.cmd " or using the submission sripts.

afwatch UI4

Tip: no need to use Keeper in the future. You can instead use shortcuts of Afanasy's commands. To create an icon on your desktop: right click => Send to => Desktop (shortcuts). A better way to launch quickly the render on your clients.


If you meet specific concerns you can always refer to the on-line user manual at http://cgru.info/home and if you still don't get it you can also ask questions on the dedicated forum at http://rules.cgru.info/*/Ask_Questions_Here.



   Step 3: launch a Job    


Proceed as follows:
  1. On your workstation start "_afserver.cmd" then "render.cmd" then "afwatch.cmd".
  2. Start "render.cmd" on your clients.
  3. Back to the workstation execute "afstarter.cmd".
afstarter UI

  • File: select your project file (.mb, .ma, .aep, .ntp, .c4d, .nk ...etc).
  • Frames: indicate your "frame range".
  • FPT: "frames Per Task" i.e. the number of images calculated by task sent to the renders. If you enter 1 it means that there will be as many task as images to calculate. Knowing that every client will have first to load the project for every task it is better to increase this parameter at least to 10 (it actually depends on your project: if your project is too heavy it is better to decrease this parameter to avoid crashes. If your project is quite light with a big frame range then you can increase this parameter as you like).

Click "Refresh" to see the code sent to the server then "Start" to start the Job.

Tip : once afwatch is launched type "IDDQD" to change in "Super User" mode (don't write it in search tabs. Just type it over the window). afwatch must turn green. This will give you access to all available options when you'll right clic on Jobs, Renders and Users

super user mode

Instead of using afwatch you can use your web browser :
  • on Keeper => Right Clic => AFANASY => Web GUI
  • or type in your browser adress => your IP:51000
It's recomended to use Firefox with the CGRU addon : http://cgru.info/firefox/cgru-addon.xpi


Afanasy firefox2




        3. Tricks and tools:        


   Remote control via your internet connection   


Now that everything is set up, you gonna need an easy way to access your clients from the workstation to check if everything is going correctly.

At the moment, the simplest solution I found, not open source but FREE, is " Teamviewer ".It gives you the possibility to control your computers throught the internet, including via your smartphone (very usefull when you need to leave the office and allows to restart any calculations you want in case of error or crash).

Easy to use also with a free account, It just has to be installed on each computer. Type your admin and password to connect them together with the same account. You're done! You can open as many Teamviewer windows as clients you have.

teamviewer UI



   More Tips   


  • Desactivate your Windows updates to avoid your computers shuting down in the middle of a job.
  • To save energy power, certain motherboard models switch off your network card alimentation. This result in interruption cuts in the network connection. To resolve it go to: "Start => Configuration panel => network Center and sharing => Modify the parameters of the card =>  Right click on your connection => Properties => Configure ... ". Sometimes you also have to deactivate this option in your BIOS.
  • Verify that your firewall parameters is not blocking any of your softwares connections. "Start => Configuration panel => Firewall Windows => advanced Parameters".
  • To save time at startup, remove the password user account protection. "Start => Configuration panel => Users Accounts".



   Recognition of the network drive   


Under Windows 7, the automatic recognition of the network drive at startup is not perfect. To help there is a simple solution:
  1. Open any writer editor
  2. write: "explore X:\ " (without quotation marks, and instead of X, the letter you chosed as shared network drive) save it with the name you want and change manually its extension in ".bat ". Before that, verify that your display options allow you to see your files extensions. In Windows explorer click "Organize => Options of the file and research => tab Display => deseable: Mask the extensions ... ".
  3. Paste this file in your startup folder: "Start => All the programs => Right click on Startup => Open all users". You can paste here all the shortcuts of the programs that you wish to see running automatically at startup.



   Read images sequences   


Once your image sequence is calculated by your renderfarm, it is necessary to verify that nothing is missing and that it's what you expected. And thus, rather than make a preview with After-Effect (opening AE, import the image sequence, create composition, make a video etc.), there is a very practical open source soft which allows you to play directly in video an image sequence: DJV_VIEW

djv view UI


Install djv_view then select an image of your image sequence (not necessarily the first one) :
Right click => open with => djv_view




       The END        


Et voila !
I will continue to update this article according to your remarks and to my experiments.

JR
creature-studio

14 commentaires:

  1. Thanks for the amazing detail tutorial. It helped me alot in setting up my network rendering.
    I have a question though, i uses arnold with maya, however, when i send my job to the renderfarm, it only renders 1 frame per machine.
    Is there a way to have one machine process parts of a frame, so the speed of rendering can improve? instead of 1 frame per machine, to may 20% a frame to 5 machines?
    I have been reading up on documentation but cannot find information!
    Thanks, and any help is appreciateD!

    cheers
    -sen

    RépondreSupprimer
    Réponses
    1. Hello Sen,
      I think what you're looking for is "Distributed Rendering" (I don't know if arnold has this feature). You can search for "tile rendering" too. I use DR a lot with Vray, without Afanasy because Vray does the work by himself. DR is efficient with still images but for image sequence you don't really need it because it's the whole sequence you distribute on your renderfarm and not images parts.

      JR

      Supprimer
  2. Ce commentaire a été supprimé par l'auteur.

    RépondreSupprimer
  3. Hello! It works with solid angle arnold plugin for maya ?

    RépondreSupprimer
    Réponses
    1. I don't use it but yes Arnold is in the supported list : http://cgru.info/afstarter

      Supprimer
  4. Best tut on Network Rendering with CGRU Afanasy.

    RépondreSupprimer
  5. Hello Jonathan, thank you for the detailed runthrough. Can't wait to set up my own render farm.
    I have one question in the mean time. I am fighting with a current server that has a render farm attached to it. Storage server works like a charm, however often render jobs, when sent to render on the nodes, freeze up and do not continue. That is suing Afanasy for the scheduling and mostly Houdini for the actual work. The cpu indicator in Afanasy just degrades.
    Would you have any clue at what to look for, as truly the physical server setup is fine, could it be something with the software used for the rendering or the workstations themselves.
    Any direction could be helpful.

    Thank you again for the great post.
    Dimitrios K

    RépondreSupprimer
    Réponses
    1. Hi,
      Afanasy just send command lines through network. It must be related to your scene. Try with a simple cube. Also you should ask on the CGRU forum maybe someone more experienced in Houdini can help : http://forum.cgru.info/index.php.
      JR

      Supprimer
  6. Hi
    i'm trying to assign multiple hosts/clients to single Job which is added from after effect.
    but its only use one render/client/ user to one job
    can you please help

    RépondreSupprimer
    Réponses
    1. try this : first set your export sequence in after, save, then lanch your job with afstarter and don't forget to put your comp name in the Node/Camera parameter (with Frames sequence and FramePerTask). That should work...

      Supprimer
    2. Thank You so much, My above mentioned problem is solved.
      However I have encountered with another issue. I have rendered 900 image sequences. Then due to some reasons I have deleted around 15 frames in between from that sequence & lunched render/Job again. In reality my render was done in few minutes BUT in Afanasy UI it was showing that render process is still going on. Every client was showing Busy status even though they was rendering nothing.
      Is there any way that we can make this -Skip Existing frame process- Fast.

      Supprimer
    3. If the problem is that your clients continue to render without stopping while your renders are actually finished then search in Output tasks if there is something wrong (double clic on job then on task and then search in Output tab).
      In any case if you relaunch your render then all your sequence will be rewritten, normal, afanasy won't skip existing frames. But you can manually skip tasks you don't want to reprocess (right clic on it).

      Supprimer
  7. Hi
    I have checked Output tab and its showing...

    PROGRESS: (Skipping 00356) (356): 0 Seconds
    PROGRESS: (Skipping 00357) (357): 0 Seconds
    PROGRESS: (Skipping 00358) (358): 0 Seconds
    PROGRESS: (Skipping 00359) (359): 0 Seconds
    PROGRESS: (Skipping 00360) (360): 0 Seconds
    PROGRESS: 09-04-2018 20:46:07: Finished composition Animation_Video .
    PROGRESS: Total Time Elapsed: 1 Seconds

    means the actual render was done in just 1 second

    but it took 0:00:25 seconds to reflect in software/UI

    because of this issue my new items in render queue started late unnecessarily.

    RépondreSupprimer