The tutorial shows you how to integrate das element with your render farm.
In this example AWS Deadline is used. The concept however applies to every other render farm manager as well.
Watch this video tutorial to get started!
Follow these steps
create shared resources folder on the server
add environment variable DASELEMENT_RESOURCES to point to that shared folder. Either set it globally for the OS or use the variables.env file.
adjust transcoding templates
copy and paste the "executable" and "parameters" from the example files
can be found on the top of each Python files
if needed: add new "Path Values" for the "width" and "height"
Example files
The latest example files for renderfarm jobs can be found here:
https://github.com/das-element/resources
Python hooks
For each transcoding task, the Python hook for ‘pre_render’ (pre_render.py) and ‘post render' (post_render.py) gets the Job ID from the render farm. If the main task isn’t finished then there is nothing to render for the proxy tasks and they will probably fail.
It is why it is important to set a dependencies on the proxy tasks to the main task.
Pre Render Hook
The input is the resolver data dictionary
which will be used to resolve the path pattern. You can modify it here before the transcoding task gets processed.
You will need to return the same dictionary with your changes included.
In the example below, in order to resolve a path pattern like <custom.dependecy>
you need to add some value for the custom data. If that is not provided the resolve would otherwise fail if you re-render the proxies and there is no main task to provide the dependency which we normally would get from the post render hook after sending the task to the render farm.
Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/hooks/examples/deadline/pre_render.py
Post Render Hook
The input is the output of the process called by the custom command task.
You will have to return a Dictionary which will be added as custom
to the resolver data and can later be accessed in the Path Builder. return {'dependency': job_id}
can later be resolved with <custom.dependency>
Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/hooks/examples/deadline/post_render.py
Transcoding tasks
Examples for Nuke
Here are some example scripts and templates on how to use The Foundry Nuke for transcoding elements.
Check out the example script here:
https://github.com/das-element/resources/tree/main/scripts/custom/examples/nuke
Transcoding task: main element
Simple command line
# exec: # Linux: cp # Windows: copy params: \"<source.path>\" \"<path>\"
Python script
The source file(s) can be a single file, like a single frame or a movie file or a sequence of files.
That makes stuff a little bit more complicated if you want to have a generic file to deal with all these cases.
Find below an example script which was tested on Linux and Windows with Deadline.
Please make sure to install the fileseq Python package. Thank you to the developers!
Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/copy_main.py
Transcoding task: thumbnail
In this example we use FFmpeg to create the thumbnail.
Simple command line
It shows you a way how to do it without any custom scripting. However this might be not very practical in production, be cause e need to create two transcoding tasks. One for single image files and another for movies/sequences. They have different command lines because movies/sequences need to know which frame is the thumbnail.
# exec: /opt/Thinkbox/Deadline10/bin/deadlinecommand # params: # command line for single images -SubmitCommandLineJob -executable "/usr/bin/ffmpeg" -arguments "-i "<path>" -y -vframes 1 -an -s <width>x<height> "<path_thumbnail>"" -frames 1 -chunksize 1 -priority 50 -name "[das element] <name> - thumbnail image" -prop BatchName="[das element] <name>" -prop JobDependencies="<custom.dependency>" # command line for movie files/sequences (the additional time flag (-ss) is defined) -SubmitCommandLineJob -executable "/usr/bin/ffmpeg" -arguments "-i "<path>" -y -vframes 1 -an -s <width>x<height> -ss <frame_first> "<path_thumbnail>"" -frames 1 -chunksize 1 -priority 50 -name "[das element] <name> - thumbnail movie/sequence" -prop BatchName="[das element] <name>" -prop JobDependencies="<custom.dependency>"
Python script
Creating proxy thumbnail can also be done with a custom python script. FFmpeg is used to render the proxy files in this example.
There is a parameter that describes the type of media (<media_type>)
this is either image
, movie
or sequence
. This helps you to determine how to deal with the different file paths types. It can be included in the Custom Commands Parameters.
Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/create_thumbnail.py
Transcoding task: proxy movie
In this example we use FFmpeg to create proxy movie file.
Simple command line
# exec: /opt/Thinkbox/Deadline10/bin/deadlinecommand # params: -SubmitCommandLineJob -executable "/usr/bin/ffmpeg" -arguments "-i \"<path>\" -y -r 24 -vf \"scale=<width>:<height>:force_original_aspect_ratio=decrease,pad=<width>:<height>:(ow-iw)/2:(oh-ih)/2\" -vcodec libx264 -crf 23 -preset faster -tune film -pix_fmt yuv420p -framerate 24 -timecode 00:00:41:17 -acodec copy \"<path_proxy>\"" -frames 1 -chunksize 1 -priority 50 -name "[das element] <name> - proxy movie" -prop BatchName="[das element] <name>" -prop JobDependencies="<custom.dependency>"
Python script
Creating proxy movie files can also be done with a custom python script. FFmpeg is used to render the proxy files in this example.
There is a parameter that describes the type of media (<media_type>)
this is either image
, movie
or sequence
. This helps you to determine how to deal with the different file paths types. It can be included in the Custom Commands Parameters.
Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/create_proxy.py
Transcoding task: filmstrip
In this example we use FFmpeg to create a filmstrip for movie files and sequences of images.
Python script
This can be done with a custom python script. FFmpeg is used to render the filmstrip in this example.
There is a parameter that describes the type of media (<media_type>)
this is either image
, movie
or sequence
. This helps you to determine how to deal with the different file paths types. It can be included in the Custom Commands Parameters.
Check out the example script here:
https://github.com/das-element/resources/blob/main/scripts/custom/examples/create_filmstrip.py
List of useful tags
The keys get resolved during ingest/proxy generation time and will be replaced with the actual value.
You can use the post render hook to add custom values.
key | Description | Example value |
---|---|---|
| Custom additional values from the user that get returned from a post render hook script. (post_render.py) |
|
| Pixel Width from either the source file, database entity, the transcoding preset (like thumbnail) or the overwrite from the Path Values |
|
| Pixel Height from either the source file, database entity, the transcoding preset (like thumbnail) or the overwrite from the Path Values |
|
| Pixel Aspect Ratio from either the source file, the transcoding preset (like thumbnail) or the overwrite from the Path Values |
|
| First frame from either the source file, database entity or the overwrite from the Path Values |
|
| Last frame from either the source file, database entity or the overwrite from the Path Values |
|
| Name of the element in the database |
|
| Database ID of the element |
|
| The type of media which is either its |
|
| File path to element |
|
| File path to elements filmstrip |
|
| File path to elements proxy movie file |
|
| File path to elements thumbnail |
|
| File path to original source |
|
| Full file name of original source file(s) |
|
| Basename file name, without extension and frame counter, of original source file(s) |
|
| File extension of original source file(s) |
|
| Colorspace of the original source file(s) |
|