Category: Terragen2

TGD Batch for Terragen 2 – V7

Had an idea to make a community renderfarm system for Terragen 2.
I already have the TGD_Batch program and if I refactor it right, then it seems like a good start.
Renaming it to Batch_TG2 seems like a good idea. Maybe there is a better name to use…?

SO this is the refactor – later there will be the server to manage the community farm.

It lets you:
– Break up a large image into smaller tiles for spreading the render load
– Create Panorama images in Equirectangular projection for reflection and environment maps.
– Blend between values of parameters to create wedges. Makes single, two and three axis blends. Annotates these images and montage into grids so they can be easily understood.

You need to have a licensed copy of Terragen2 for the commandline version to work. This will not work on the free version.

What’s happened is:
– The older Pythoncard UI has been swapped out for a more modern QT based UI.
– General refactoring and cleanup has occurred especially the cruft to support notebooks in pythonCard (which did not work that well) has been removed.
– A render thread has been added so the UI doesn’t block when it goes off to render locally
– All renders are processed in two passes
– one gathers them all into a new Task class containing individual RenderActions for each and every image
– the second renders each image (RenderAction) one at a time.
– When its later extended to send Renderactions to the farm – they can be split up
– All scripts to recombine now work on these finished frames, once they all come in.
– Logging, guestimating finish times, and tracking of progress has all been added
– some refinements like; triaxial blends (interp 3 parameters at the same time) and allowing loading of the same TGD file several times with different settings, have been added.

The QT is a really nice and simple UI to build with. I am very pleased its so stable now. Also using Pyside as the python binding.

The following packages are needed to make it work:
– Pyside – the python wrapper for QT. downloading Pyside automatically brings in the QT dlls so this is all you need for the UI
– psutil – a simple Python module which is used when cancelling a render to kill the tgdcli process.
– Hugin – a very cool Panorama making program. We only use Nona.exe to stitch the panoramas together but its worth downloading the entire app, its so good.
http://hugin.sourceforge.net/
– Imagemagick – a commandline based image processing tool. It does the clipping, annotating and creates montages.

So here it is..

Installation:
– no clever installer instead you have to follow these simple instructions:

1. Choose 32 or 64 bit. Tested with 32bit winxp SP3 and 64bit Win7.
– IT IS CRITICAL that you choose the right packages below so they all match.
– if you choose 64bit you MUST choose python 2.7 (psutil is only available in 64bit Python 2.7)

2. Choose python 2.6 or 2.7 (not 3.0)
– if you already have Python 2.6 on your system. Then use it. Else 2.7 is recomended.
– Either will work on 32bit. You MUST choose 2.7 if you have a 64bit system

3. Download the following
– make sure to only get the proper version (2.6, or 2.7, and 32 or 64bit when downloading)
– Pyside
http://developer.qt.nokia.com/wiki/PySide_Binaries_Windows

– Psutil
http://code.google.com/p/psutil/downloads/list

– Hugin
32bit —- http://sourceforge.net/projects/hugin/files/hugin/hugin-2011.0/HuginSetu…
64bit —- http://sourceforge.net/projects/hugin/files/hugin/hugin-2011.0/HuginSetu…

– Imagemagick
http://www.imagemagick.org/script/binary-releases.php#windows
– 16 bit versions of Imagemagick supposedly support OpenEXR but cannot find a binary with it built in.
– This is unfortunate because if there was one – we could make EXR panoramas for HDRI lighting use in other programs.

If we had Ham, we could have Ham and Eggs.. if we had eggs….

4. Install them. Only critical order is Python first.

5. unzip the Batch_TG2 archive into a handy folder you make somewhere and put a shortcut to Batch_TG2.py on your desktop.

6. Test by running Batch_TG2.py
– Double click should work.
Choose a simpe TGD to start with so renders are short. You can override the width, height to 200,150 and lower the Quality to 0.1 for fast test renders.

Try to break it…
Tell me about the breaks…
http://forums.planetside.co.uk/index.php?topic=12769

Instructions:
I’m hoping the layout is self evident…..?
– Organise Tab is where you load and setup what you want to do.
– Render Tab is where you go to Render
– “Render Farm” and “Render Local” do the same thing right now…. Get used to using “Render Farm” 🙂

Batch_TG2-small

TGD Batch for Terragen 2 – V4

Here is a program to help automate the rendering of several aspects of Terragen 2 files.
Terragen 2 is a program from Planetside that renders very realistic outdoor naturalistic imagery.
The announcement and discussion about this app is here:
http://forums.planetside.co.uk/index.php?topic=5251.msg54258#msg54258

It will only work for purchased versions of TG2 as it requires the command line option to work.

This is Version 4. The previous versions are on the Planetside forums. Here: http://forums.planetside.co.uk/index.php?topic=2816.0

Features:
– Creation of panoramas (6 auto views with tiling)
– Tiled renders for very large images (with overlaps)
– batch renders for remote rendering (but no Renderfarm Manager)
– wedge tests for all parameters in TGD files
– tested in version 1.10.23.1
– renders animations one frame at a time (to facilitate remote batch rendering)

Please fell free to make suggestions for improvements of any kind.

Panoramas
– are generated using Nona – part of the Hugin project (which replaces PTStitcher with an opensource alternative)
– an Imagemagick script recombines all rendered views into a single image and Nona turns it into an equirectangular projection ready for Quicktime or Environment maps
– currently does not work for the HDRI EXR format because Imagemagick needs a custom compile.
However – using new version of hugin, manually with the 6 images, will create the EXR end result.

Big Ben supplied an excellent workflow and scripts using PTStitcher which I have learnt a lot from.
http://forums.planetside.co.uk/index.php?topic=853.0
http://forums.planetside.co.uk/index.php?topic=972.0
Thanks BigBen.

Tiled rendering:
– has overlaps to help deal with the GI problems and to ensure that antialiasing does not create edge artifacts.
– I use Imagemagick scripts to recombine the tiles into a single image.
– these scripts are automatically generated.
– the individual frames are well named for manual identification

Wedge Tests:
– Blends can be one dimensional or two dimensional
– The resulting images will be grouped into a single image grid
– the user can control the shape of the grid for one dimensional blends
– all blends are labeled with their parameters and values

Installing
Alas – I have not packaged it up in an installer. sorry about that 🙁
You need to load (in this order):
– Python (2.5.2 or 2.6)
– wxPython (version based on your version of Python)
– Pythoncard
– Imagemagick
– Hugin – if you want Panoramas (tested in 0.7 beta4 and release 0.7.0)

I built it using stable features of python 2.5.2 and wxpython 2.8
Later versions should work. Earlier versions may not. (You must not use wxpython 2.6 – notebook is faulty)

Get Python from http://www.python.org/download/
choose the latest . I tested in 2.5.2 and 2.6
Get wxpython from: http://www.wxpython.org/download.php#binaries
Use a matching installer for your version of Python
Get Hugin from: http://hugin.sourceforge.net/download/
we only use Nona (replaces PTStitcher for panoramas) but its all bundled together.
Get Imagemagick from http://www.imagemagick.org/script/binary-releases.php#windows
the 16bit version does not support exr 🙁 so choose either the 16bit or 8bit windows version
Get PythonCard 0.8.2 from http://pythoncard.sourceforge.net/
– Specifically download from: http://sourceforge.net/project/showfiles.php?group_id=19015
– Installation Instructions (seven steps including dependent packages) are here:
http://pythoncard.sourceforge.net/windows_installation.html

Install Python
Install wxpython
Install Pythoncard
Install Imagemagick
Install (just unpack really) Hugin
– see where Nona is located and manually enter its proper location into the top of Batch_TG2.py
– the location is defined in a global variable at the head of the file

Operation:
Once PythonCard is installed.
– Unzip the attached zipfile into any directory you like.
– Double click on the file Batch_TG2.py

It will open a dialog prompting you to select a TGD file (or files).
You may deactivate ones you do not wish to render.
Adding a Tile will allow you to tile or make a panorama
Scripts will be made that will composite the tiles into a single image.
– Scripts will be named _recombine.bat
Panoramas will create the recombine scripts, as well as a script that calls them all and create an Equirectangular mapped image for a panorama.
(Quicktime style but equally useful for environment maps)
– Scripts will be named _Collate_and_Build_Pano.bat
Panoramas can be viewed directly in panoglview (part of hugin install)

The log window will show you progress output from TGD as it runs in the background.
Animation is done by repeatedly opening the file and rendering single frames
(see http://forums.planetside.co.uk/index.php?topic=954.0)

You can edit:
– size of image, quality settings, antialiasing, GI Settings, motion blur
– if animating – range, step-size
– many key settings – see screenshot

The program will make a temporary copy of your tgd file so as not to modify your original. It will prefix all these new TGD files with the prefix “tmp” so they are easy to find and delete after rendering.

Choose a rendering destination directory to collect frames – then render.
To recombine the tiles – run the script: _recombine.bat (It will be in the image directory)
To recombine tiles and build a panorama – run the script: _Collate_and_Build_Pano.bat
Your original rendered images will not be edited or deleted.
Your original tgd files will not be edited or deleted.
You will have to remove the surplus files manually (its just safer this way and they are well named)

Using “Batch” instead of “Render” will create the files to run – but not execute them.

Summary:
I find it useful for:
– doing wedge tests (blends) of parameter ranges
– making up a batch of renders to run overnight (locally or on remote machines),
– setting off long animation sequences,
– making tiled renders for really large images (not HDR yet),
– Panoramas (not hdr yet)
Your original tgd files will not be edited or deleted.

User mods:
You may wish to change the default starting directory.
– Its defined at the head of the file: Batch_TG2.py
You may wish to change the “tmp” prefix for temporary TGD files.
– Its defined at the head of the file: Batch_TG2_class.py
You will want to set the default location of the Nona.exe (if you make Panoramas)
– Its defined at the head of the file: Batch_TG2.py

Next version may
– support HDR options for panoramas and tiled images
– bundle all referenced files into a zip file for easy remote rendering
I need help (see about box) to get a version of Imagemagick compiled with OpenEXR support. Any takers ?
If we can do that then I will alter it to make HDR panos and tiled images