[Radiance-general] parallel dctimestep

[Greg Ward]

Ah, that's a different problem!  

How much memory is on your machine?  If there is less RAM than the combined size of the component images, then you won't see the benefit of disk caching.  You can try decreasing the memory required for caching by compacting your pictures (e.g., running "ra_rgbe -r" on each of them).

If you don't have as much free RAM as you have images and you're running off a hard disk rather than an SSD, that is the worst case for dctimestep and image processing.  Running in parallel may help you a bit, but you still need to think about the i/o requirements.  Having a bunch of processes fighting to access the same disk information might not do that much.

Cheers,
-Greg

> From: Santiago Torres <Santiago.Torres at arup.com>
> Date: September 22, 2017 9:13:50 AM PDT
> 
> Dear Sarith, Greg,
>  
> Thank you for your replies. In this case I am running the annual calculation of an image, and each time-step (daytime) is taking 3-4 minutes to run, so the total calculation takes more than 9 days. And I would like to increase the resolution to reduce noise…
>  
> So I thought if I run 10 dctimestep processes in parallel, it should take less than a day. I hope this makes sense. I will try it out and let you know if I find problems.
>  
> Thank you,
> Santiago
>  
>  
> From: Sarith Subramaniam [mailto:sarith at psu.edu] 
> Sent: 22 September 2017 16:53
>  
> Hi Santiago,
> 
> I pretty much did the exact same thing that you mentioned to run multiple instances of dctimestep with Python. The syntax is something like:
> import multiprocessing as mp
> 
> import os
> 
> #Run dctimesteps, 16 at at time.
> 
> #runCommand is just a call to os.system
> 
> #commands is a list of dctimestep commands like ["dctimestep dc.mtx sky0001.smx > res0001.smx",  "dctimestep dc.mtx sky0002.smx > res0002.smx" ...]
> 
> pools = mp.Pool(processes=16)
> 
> pools.map(runCommand, commands)
> 
> 
> For the above, you'd have to do some additional scripting to run rmtxop and compile all the results into a single file. There is also a method that was discussed by Wangda Zuo and Andy Mcneil in this paper: http://www.ibpsa.org/proceedings/BS2011/P_1155.pdf
> Sarith
> 
>   
> 
> On 9/22/2017 9:55 AM, Santiago Torres wrote:
> Dear all,
>  
> This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?
>  
> Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?
>  
> Thanks,
> Santiago
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.radiance-online.org/pipermail/radiance-general/attachments/20170922/955ab020/attachment.html>