[Radiance-dev] time needed for octree reading (was Re: setting buffer size on octree reads (setvbuf) )

[Peter Apian-Bennewitz]

Hi all,

The observed slow reading of frozen octrees happens if and only if:
- the geometric object (polygons in this case) each have their own, 
different modifier. This is probably rare, but  may happen if each 
polygon uses a texfunc to interpolate the normal.
- each of these modifiers have a unique name
It is only relevant if there are millions of such objects (e.g. 
tessellated surfaces imported from CAD).

Then the octree reading spends 96% of its time in otndx() (readoct.c), 
which handles a hash table. I would *guess* (not be field of expertise 
anyway) that the hash algorithm is slow under these extreme 
circumstances, since these unique modifier names are only inserted, but 
never ever re-used. Effectively this exotic scenario screws up the idea 
of having a hash table.

The solution (thanks Greg) was to use a modifier name for the texfunc 
that applies to all polygons with the same material, although each 
texfunc is different. This works since the texfunc is specified just 
prior to its polygon.
The decrease in loading time is noticeable: 18 versus 148 seconds for 
320MB of frozen octrees.

It is worth to note that, in a wider context and from a different 
perspective, this specific use of texfunc/polygons might be better 
served with the Radiance mesh primitive, resulting in less file and 
memory use and with faster reading and rendering.

As a general hint, reading of non-frozen octrees may be slower. Use of 
frozen octrees is recommended for speed (by Greg).

Regarding my suggestion to increase the stream buffer size for octree 
readings: The above timings use a 2MB buffer. Likely the 4k standard 
buffer size doesn't matter much for local discs, although I'm inclined 
to stick with the 2MB, because of the reduced. number of system calls. 
It may have a larger influence on NFS mounts, depending on the mount 
options. The only reason not to change the source is that *any* 
non-urgent change may introduce unforeseen side effects on some systems. 
Even if setvbuf seems to be a pretty much standardized call.

Thanks to Greg and the gprof profiler

cheers
Peter

-- 
 pab-opto, Freiburg, Germany, http://www.pab-opto.de
 [see web page to check digital email signature]