Maximum data size for scatteredDataToGrid3D(...)?

[gregcotten]

Is there a maximum size for the data set in scatteredDataToGrid? Using ctlrender I run into trouble with the CTL below but when I truncate most of the data it works fine.

The error I get with the large data set: Assertion failed: (sigma > .0f), function kernel, file /tmp/ctl-sQsqh1/CTL-ctl-1.5.2/lib/IlmCtlMath/CtlRbfInterpolator.cpp, line 77.

const float minLUT3D[3] = {0.0, 0.0, 0.0};
const float maxLUT3D[3] = {1.0, 1.0, 1.0};
const float data[][2][3] =
        {
            {{0.406317, 0.413001, 0.338628}, {0.406317, 0.413001, 0.338628}},
            {{0.456565, 0.469993, 0.423239}, {0.456565, 0.469993, 0.423239}},
            {{0.738231, 0.702113, 0.610636}, {0.738231, 0.702113, 0.610636}},
            {{0.381430, 0.375586, 0.340856}, {0.381430, 0.375586, 0.340856}},
            {{0.634546, 0.603052, 0.480674}, {0.634546, 0.603052, 0.480674}},
            {{0.446357, 0.445991, 0.383597}, {0.446357, 0.445991, 0.383597}},
            {{0.632502, 0.613642, 0.448280}, {0.632502, 0.613642, 0.448280}},
            {{0.497597, 0.492256, 0.466331}, {0.497597, 0.492256, 0.466331}},
            {{0.412024, 0.418799, 0.348653}, {0.412024, 0.418799, 0.348653}},
            {{0.428977, 0.428580, 0.357183}, {0.428977, 0.428580, 0.357183}},
            {{0.374533, 0.392370, 0.341527}, {0.374533, 0.392370, 0.341527}},
            {{0.520043, 0.498222, 0.412421}, {0.520043, 0.498222, 0.412421}},
            {{0.090929, 0.684276, 0.090929}, {0.090929, 0.684276, 0.090929}},
            {{0.214115, 0.191669, 0.190494}, {0.214115, 0.191669, 0.190494}},
            {{0.663981, 0.621118, 0.550607}, {0.663981, 0.621118, 0.550607}},
            {{0.363851, 0.380728, 0.339895}, {0.363851, 0.380728, 0.339895}},
            {{0.542260, 0.533425, 0.411887}, {0.542260, 0.533425, 0.411887}},
            {{0.476081, 0.477241, 0.395727}, {0.476081, 0.477241, 0.395727}},
            {{0.565087, 0.551614, 0.427649}, {0.565087, 0.551614, 0.427649}},
            {{0.544533, 0.540444, 0.448981}, {0.544533, 0.540444, 0.448981}},
            {{0.345373, 0.357000, 0.328435}, {0.345373, 0.357000, 0.328435}},
            {{0.601923, 0.560021, 0.467674}, {0.601923, 0.560021, 0.467674}},
            {{0.345190, 0.335653, 0.300679}, {0.345190, 0.335653, 0.300679}},
            {{0.533516, 0.538582, 0.465354}, {0.533516, 0.538582, 0.465354}},
            {{0.090929, 0.090929, 0.684276}, {0.090929, 0.090929, 0.684276}},
            {{0.508370, 0.418509, 0.374533}, {0.508370, 0.418509, 0.374533}},
            {{0.595163, 0.597330, 0.448356}, {0.595163, 0.597330, 0.448356}},
            {{0.389410, 0.407645, 0.342565}, {0.389410, 0.407645, 0.342565}},
            {{0.557656, 0.536019, 0.459174}, {0.557656, 0.536019, 0.459174}},
            {{0.334371, 0.348577, 0.307698}, {0.334371, 0.348577, 0.307698}},
            {{0.489021, 0.469108, 0.390021}, {0.489021, 0.469108, 0.390021}},
            {{0.465980, 0.468910, 0.378943}, {0.465980, 0.468910, 0.378943}},
            {{0.414679, 0.449714, 0.368704}, {0.414679, 0.449714, 0.368704}},
            {{0.497490, 0.482933, 0.396887}, {0.497490, 0.482933, 0.396887}},
            {{0.432166, 0.420340, 0.394965}, {0.432166, 0.420340, 0.394965}},
            {{0.425467, 0.418021, 0.377584}, {0.425467, 0.418021, 0.377584}},
            {{0.090929, 0.684276, 0.684276}, {0.090929, 0.684276, 0.684276}},
            {{0.509087, 0.450126, 0.408042}, {0.509087, 0.450126, 0.408042}},
            {{0.462669, 0.457984, 0.376547}, {0.462669, 0.457984, 0.376547}},
            {{0.512322, 0.497337, 0.466087}, {0.512322, 0.497337, 0.466087}},
            {{0.903532, 0.848753, 0.812863}, {0.903532, 0.848753, 0.812863}},
            {{0.331365, 0.342931, 0.276066}, {0.331365, 0.342931, 0.276066}},
            {{0.520989, 0.499199, 0.485878}, {0.520989, 0.499199, 0.485878}},
            {{0.354543, 0.357534, 0.335332}, {0.354543, 0.357534, 0.335332}},
            {{0.414405, 0.444663, 0.491325}, {0.414405, 0.444663, 0.491325}},
            {{0.453605, 0.468452, 0.488640}, {0.453605, 0.468452, 0.488640}},
            {{0.256001, 0.245762, 0.227680}, {0.256001, 0.245762, 0.227680}},
            {{0.443534, 0.431174, 0.431418}, {0.443534, 0.431174, 0.431418}},
            {{0.684276, 0.684276, 0.090929}, {0.684276, 0.684276, 0.090929}},
            {{0.444327, 0.450263, 0.470115}, {0.444327, 0.450263, 0.470115}},
            {{0.470832, 0.519127, 0.531014}, {0.470832, 0.519127, 0.531014}},
            {{0.430045, 0.436683, 0.459770}, {0.430045, 0.436683, 0.459770}},
            {{0.561944, 0.596551, 0.599496}, {0.561944, 0.596551, 0.599496}},
            {{0.403647, 0.425483, 0.387243}, {0.403647, 0.425483, 0.387243}},
            {{0.480720, 0.483192, 0.498314}, {0.480720, 0.483192, 0.498314}},
            {{0.477775, 0.457740, 0.408347}, {0.477775, 0.457740, 0.408347}},
            {{0.351324, 0.355947, 0.374868}, {0.351324, 0.355947, 0.374868}},
            {{0.236103, 0.240024, 0.227817}, {0.236103, 0.240024, 0.227817}},
            {{0.420645, 0.435584, 0.367514}, {0.420645, 0.435584, 0.367514}},
            {{0.659434, 0.595590, 0.600229}, {0.659434, 0.595590, 0.600229}},
            {{0.323857, 0.343130, 0.307485}, {0.323857, 0.343130, 0.307485}},
            {{0.388754, 0.412543, 0.459632}, {0.388754, 0.412543, 0.459632}},
            {{0.208026, 0.220600, 0.230396}, {0.208026, 0.220600, 0.230396}},
            {{0.337789, 0.332555, 0.343603}, {0.337789, 0.332555, 0.343603}},
            {{0.221195, 0.218799, 0.190295}, {0.221195, 0.218799, 0.190295}},
            {{0.328557, 0.345235, 0.268437}, {0.328557, 0.345235, 0.268437}},
            {{0.552544, 0.557916, 0.563210}, {0.552544, 0.557916, 0.563210}},
            {{0.264607, 0.328923, 0.325521}, {0.264607, 0.328923, 0.325521}},
            {{0.415030, 0.422049, 0.452445}, {0.415030, 0.422049, 0.452445}},
            {{0.230076, 0.231754, 0.236088}, {0.230076, 0.231754, 0.236088}},
            {{0.520150, 0.540627, 0.549859}, {0.520150, 0.540627, 0.549859}},
            {{0.452094, 0.388434, 0.355230}, {0.452094, 0.388434, 0.355230}},
            {{0.318883, 0.298497, 0.261097}, {0.318883, 0.298497, 0.261097}},
            {{0.516655, 0.516884, 0.532250}, {0.516655, 0.516884, 0.532250}},
            {{0.589014, 0.590661, 0.638239}, {0.589014, 0.590661, 0.638239}},
            {{0.223743, 0.230640, 0.244846}, {0.223743, 0.230640, 0.244846}},
            {{0.701259, 0.505852, 0.473487}, {0.701259, 0.505852, 0.473487}},
            {{0.434989, 0.434989, 0.434989}, {0.434989, 0.434989, 0.434989}},
            {{0.407874, 0.408545, 0.416770}, {0.407874, 0.408545, 0.416770}},
            {{0.547494, 0.545007, 0.589822}, {0.547494, 0.545007, 0.589822}},
            {{0.387549, 0.394614, 0.418219}, {0.387549, 0.394614, 0.418219}},
            {{0.579782, 0.584085, 0.624399}, {0.579782, 0.584085, 0.624399}},
            {{0.436576, 0.452445, 0.479637}, {0.436576, 0.452445, 0.479637}},
            {{0.282918, 0.252384, 0.262135}, {0.282918, 0.252384, 0.262135}},
            {{0.434989, 0.434989, 0.434989}, {0.434989, 0.434989, 0.434989}},
            {{0.438178, 0.440238, 0.471702}, {0.438178, 0.440238, 0.471702}},
            {{0.554009, 0.547204, 0.582010}, {0.554009, 0.547204, 0.582010}},
            {{0.425544, 0.436759, 0.468864}, {0.425544, 0.436759, 0.468864}},
            {{0.513176, 0.514702, 0.428305}, {0.513176, 0.514702, 0.428305}},
            {{0.434989, 0.434989, 0.434989}, {0.434989, 0.434989, 0.434989}},
            {{0.339300, 0.343267, 0.362417}, {0.339300, 0.343267, 0.362417}},
            {{0.333333, 0.272801, 0.282780}, {0.333333, 0.272801, 0.282780}},
            {{0.448096, 0.465522, 0.494377}, {0.448096, 0.465522, 0.494377}},
            {{0.321630, 0.249287, 0.249210}, {0.321630, 0.249287, 0.249210}},
            {{0.452064, 0.465385, 0.477485}, {0.452064, 0.465385, 0.477485}},
            {{0.330465, 0.301137, 0.305516}, {0.330465, 0.301137, 0.305516}},
            {{0.434989, 0.434989, 0.434989}, {0.434989, 0.434989, 0.434989}},
            {{0.426627, 0.446418, 0.490425}, {0.426627, 0.446418, 0.490425}},
            {{0.524559, 0.522332, 0.571725}, {0.524559, 0.522332, 0.571725}},
            {{0.294697, 0.310552, 0.339574}, {0.294697, 0.310552, 0.339574}},
            {{0.597543, 0.420935, 0.366033}, {0.597543, 0.420935, 0.366033}},
            {{0.434989, 0.434989, 0.434989}, {0.434989, 0.434989, 0.434989}},
            {{0.507607, 0.512856, 0.526040}, {0.507607, 0.512856, 0.526040}},
            {{0.322393, 0.270665, 0.283528}, {0.322393, 0.270665, 0.283528}}
        };

const float lut3D[33][33][33][3], scatteredDataToGrid3D (data, minLUT3D, maxLUT3D, lut3D);

void main
 (
 input varying float rIn,
 input varying float gIn,
 input varying float bIn,
 input varying float aIn,
 output varying float rOut,
 output varying float gOut,
 output varying float bOut,
 output varying float aOut
 )
 {
 float r = rIn;
 float g = gIn;
 float b = bIn;
 lookup3D_f(lut3D, minLUT3D, maxLUT3D, r, g, b, r, g, b);
 rOut = r;
 gOut = g;
 bOut = b;
 aOut = aIn;
 }

[michaeldsmith]

I tried to replicate the error you've reported using the CTL code you've pasted into this issue processed by ctlrender (using this file issue_56.ctl.rename.txt that you can rename from *.rename.txt to *.ctl) , but I do not get the error you've reported and the output image file appears to be valid.

I noticed that the pairs of input/output 3D coordinates in your large data[][2][3] array appear to have matching values for the input and output values. Is that intentional? I suppose it might be a good test that could be used to confirm the function scatteredDataToGrid3D () returns an identity 3D LUT that doesn't modify the image data when the input and output data value match. The only other effect of such a identity LUT processing should be limiting the processed input data to the range specified by minLUT3D and maxLUT3D, since values outside that range are not defined by the 3D LUT

[gregcotten]

@michaeldsmith I have slept 2,931 days since I created this issue and I haven't the foggiest what I was on about. Happy to see this one closed - I no longer use ctlrender anyway.

[michaeldsmith]

thanks for the response @gregcotten, I will close the issue

[michaeldsmith]

I happened to be doing some work with scatteredDataToGrid3D() this week and encountered the same issue originally reported by @gregcotten years ago Assertion failed: (sigma > .0f), function kernel, file /lib/IlmCtlMath/CtlRbfInterpolator.cpp, line 77.

It seems that the assert() does not cause failure in Windows with Release mode, but does cause failure when using the simple cmake .. command under Ubuntu. If I configure cmake to create a Release build in Ubuntu by using cmake .. -D CMAKE_BUILD_TYPE=Release then I don't get the failure under Ubuntu. But, there could be some other issue like divide-by-zero or something that the assert was trying to prevent. I will reopen this issue and investigate further.