[Radiance-general] rtrace parallelization

[Jack de Valpine]

Hi Marija,

The -PP option only allows for memory to be shared between processes on 
a multi-core machine, however this is not at all essential for 
parallelization for radiance jobs. The -PP option does not distribute 
the rays to be traced between multiple processes. That is whys currently 
you see only one process actually doing any work.

I think that there are probably a few different ways to distribute rays 
to multiple rtrace processes (using memory sharing or not). The basic 
idea is to take the view or grid points that you want to calculate and 
subdivide. This subdivision could be in terms of the number of cores and 
or machines available. So lets assume that you have 2 machines with 2 
cores available, thus 4 cores. So if you divide your grid 2x2, then you 
could do something like the following (lots of ways to run rtrace so 
there are other approaches):

FROM A SHARED NFS DIRECTORY on Machine 1 and 2 AND using memory sharing:

On Machine 1:

    rtrace [parameters] -PP m1.pp scene.oct  < points.1.1 >
    points.1.1.output
    rtrace [parameters] -PP m1.pp scene.oct  < points.1.2 >
    points.1.2.output

On Machine 2:

    rtrace [parameters] -PP m2.pp scene.oct  < points.2.1 >
    points.2.1.output
    rtrace [parameters] -PP m2.pp scene.oct  < points.2.2 >
    points.2.2.output

When all the jobs are complete then you will need to use other radiance 
tools to recombine your output into one file. Since you know the indices 
of each subset of points this should not be too difficult.

I think that this approach should work. We typically do something 
similar when distributing jobs using rpiece. I hope this helps.

-Jack

Marija Velickovic wrote:
> Hi Lars,
>
> I followed your advice but still first rtrace is running and other 
> rtrace processes are sleeping.
>
> When I run two separate rtrace processes without -PP option and with 
> different input data they both run in parallel without a problem.
> I suppose I'm doing something wrong, so other ideas are welcome!
>
> Marija
>
> On Mon, Dec 22, 2008 at 2:06 PM, Lars O. Grobe <akilog at nus.edu.sg 
> <mailto:akilog at nus.edu.sg>> wrote:
>
>     Hi!
>
>
>         I've found "-PP pfile" option but I'm not sure how to use it.
>         I supposed that "pfile" should be empty so program can write
>         into it some control variables. When I started rtrace with -PP
>         option 2 processes are created: one for my  rtrace and one
>         child rtrace process with the same parameters. When I tried to
>         explore running processes it seems that only the first one is
>         really running, while the second one is sleeping - so I don't
>         have real paralellization here.
>
>
>     You can attach more processes by starting them with the same
>     -PP-option. Sp if you had "rtrace (other rtrace-options) -PP
>     control.pfile" started, just start one more "rtrace -PP
>     control.file", but leave the other rtrace options empty as the
>     second (or third or forth) process will use the same as the first
>     one. Hopes that works, as it is a while since I did that and I
>     have no example to start here.
>
>     CU Lars.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Radiance-general mailing list
> Radiance-general at radiance-online.org
> http://www.radiance-online.org/mailman/listinfo/radiance-general
>   

-- 
# Jack de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://radiance-online.org/pipermail/radiance-general/attachments/20081222/3bd5008b/attachment.html