[hist] Using TKDE::Fill works with empty tkde #14740

aaronj0 · 2024-02-15T13:32:49Z

This is happening because in when the parentheses operator overload TKDE::operator()(Double_t x) calls ReInit (const_cast<TKDE*>(this))->ReInit() it returns before setting the fKernel in the case of new data.

One approach is to call SetKernel here:

if (fNewData)  {
      InitFromNewData();
      SetKernel();
      return;
   }

or call it at the end of InitFromNewData().

When that happens, the fKernel is no longer null but this error is reproducible with both iterative options -
With Adaptive we get NaN and Fixed we get inf

This is because the weight calculation is using the nSum that is 0 when binning is not used
Inversing that gives the infinity in the Iteration:Fixed case

This fix:

adds the call to SetBinCountData() in InitFromNewData()
recreates the kernel fun pointer (previously fKernel ends up a nullptr)
calculating nSum as fNEvents if not binning in TKDE::TKernel::operator().

Results:

    auto kde = new TKDE(0, nullptr, 0, 5, "KernelType:Gaussian;Iteration:Adaptive;Mirror:noMirror;Binning:RelaxedBinning", 1);
    for (unsigned int i = 0; i < 100; i++) { kde->Fill(gRandom->Gaus(2,1)); }
    std::cout<<kde->GetValue(2)<<"\n";

Gives
0.487581

phsft-bot · 2024-02-15T13:32:53Z

Can one of the admins verify this patch?

lmoneta

Thank you for fixing the TKDE when updating the data with Fill.

I agree the SetKernel() is needed in the InitFromNewData function, since by adding the data both the fixed bandwidth and the adaptive one needs to be re-computed.
A possible optimisation would be to have a function UpdateKernel, which will re-use the existing kernel, but just changing the fixed bandwidth or the adaptive one, depending on the case.

lmoneta · 2024-02-15T14:24:25Z

@phsft-bot build

phsft-bot · 2024-02-15T14:24:32Z

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default
How to customize builds

phsft-bot · 2024-02-15T14:25:01Z

Build failed on ROOT-ubuntu2004/python3.
See console output.

phsft-bot · 2024-02-15T14:49:05Z

Build failed on ROOT-ubuntu2204/nortcxxmod.
Running on root-ubuntu-2204-2.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.hist.hist.test.gtest_hist_hist_test_testTKDE

phsft-bot · 2024-02-15T16:34:28Z

Build failed on windows10/default.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Failing tests:

projectroot.hist.hist.test.gtest_hist_hist_test_testTKDE

hist/hist/src/TKDE.cxx

phsft-bot · 2024-02-15T18:20:56Z

Build failed on ROOT-performance-centos8-multicore/soversion.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.hist.hist.test.gtest_hist_hist_test_testTKDE

phsft-bot · 2024-02-16T02:36:36Z

Build failed on mac12arm/cxx20.
Running on 194.12.161.128:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.hist.hist.test.gtest_hist_hist_test_testTKDE

github-actions · 2024-02-16T07:47:46Z

Test Results

20 files 20 suites 3d 6h 31m 6s ⏱️
3 063 tests 3 063 ✅ 0 💤 0 ❌
59 671 runs 59 671 ✅ 0 💤 0 ❌

Results for commit 9d189d2.

♻️ This comment has been updated with latest results.

Done

as suggested by lmoneta. This was causing a test to fail. The normalisation of the TKDE was wrong, because now nSum was (in case of unbinned, useCount=false) computed from all events, instead it should be computed from all the events in the range (see SetBinCounts).

as suggested by lmoneta

aaronj0 · 2025-06-30T11:32:02Z

Hello @ferdymercury, thanks for following up on this PR! I have rebased on master and squashed the last three commits that deal with the added test. @lmoneta It would be good to obtain a review and land this PR if the CI is happy

ferdymercury · 2025-06-30T13:06:54Z

@guitargeek see mac15 RPATH? failure with gfortran

guitargeek · 2025-06-30T13:30:02Z

Looks like the mac15 runners have inconsistent environments:

2025-06-30T11:37:44.3529390Z ld: warning: search path '/usr/local/gfortran/lib/gcc/aarch64-apple-darwin22/12.2.0' not found
2025-06-30T11:37:44.3541440Z ld: library 'emutls_w' not found

Indeed, the last build to produce the artifacts for the incremental PR builds was produced on macphsft31, which has gfortran from gcc 12, while the macphsft33 that was used for this PR has gcc 13.

@couet, can we bring macphsft31 also to gfortran 13?

couet · 2025-06-30T13:46:45Z

@guitargeek The gfortran version ready for download is 14.2. See https://github.com/fxcoudert/gfortran-for-macOS/releases . Is it ok if I update our nodes to this one ?

guitargeek · 2025-06-30T13:56:37Z

Yes, that would be nice. Thanks!

ferdymercury · 2025-06-30T14:00:09Z

Yes, that would be nice. Thanks!

Related question: what is the current gfortran version of mac13 and mac14 (where h2root fails) ? Just mentioning since I see in the gcc12 relnotes: WG5/N1942, "TS 29113 Further Interoperability of Fortran with C", is now fully supported. In addition to implementing previously missing functionality, such as support for character arguments of length greater than one in functions marked bind(c) and gaps in the handling for assumed-rank arrays, numerous other bugs have been fixed, and an extensive set of new conformance test cases has been added.

couet · 2025-06-30T14:08:31Z

Ok I already put 13.2 on macphsft31:

sftnight@macphsft31 ~ % gfortran --version
GNU Fortran (GCC) 13.2.0

But as its looks like for sequoia it is version 14, I will do the update of all our 15.5 machines

couet · 2025-06-30T14:17:34Z

gfortran is now 14.2.0 on the two MacOs 15.5 machines

couet · 2025-06-30T14:21:55Z

Both MacOS 13 machines are running gfortran 12.2.0

couet · 2025-06-30T14:45:13Z

The two MacOS 14 machines are now running gfortran 14.2.0

aaronj0 requested a review from lmoneta as a code owner February 15, 2024 13:32

lmoneta approved these changes Feb 15, 2024

View reviewed changes

hahnjo assigned lmoneta Feb 15, 2024

lmoneta previously requested changes Feb 15, 2024

View reviewed changes

hist/hist/src/TKDE.cxx Outdated Show resolved Hide resolved

ferdymercury added the in:Hist label Apr 17, 2024

ferdymercury requested review from lmoneta and guitargeek December 17, 2024 19:42

ferdymercury added the 1st Hackathon: the Fixhathon This issue can be tackled at a ROOT fixathon label Dec 18, 2024

ferdymercury requested a review from dpiparo December 19, 2024 17:47

aaronj0 and others added 4 commits June 30, 2025 13:24

[hist] make TKDE::GetValue work with fill on empty tkde

1611615

[hist] raise warning with zero sum

399f832

as suggested by lmoneta

[hist] add test for regression

9d189d2

aaronj0 force-pushed the tkde-fix branch from cf18add to 9d189d2 Compare June 30, 2025 11:31

aaronj0 added the clean build Ask CI to do non-incremental build on PR label Jun 30, 2025

aaronj0 closed this Jun 30, 2025

aaronj0 reopened this Jun 30, 2025

[hist] Using TKDE::Fill works with empty tkde #14740

Are you sure you want to change the base?

[hist] Using TKDE::Fill works with empty tkde #14740

Uh oh!

Conversation

aaronj0 commented Feb 15, 2024 • edited by ferdymercury Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phsft-bot commented Feb 15, 2024

Uh oh!

lmoneta left a comment

Choose a reason for hiding this comment

Uh oh!

lmoneta commented Feb 15, 2024

Uh oh!

phsft-bot commented Feb 15, 2024

Uh oh!

phsft-bot commented Feb 15, 2024

Uh oh!

phsft-bot commented Feb 15, 2024

Failing tests:

Uh oh!

phsft-bot commented Feb 15, 2024

Failing tests:

Uh oh!

Uh oh!

phsft-bot commented Feb 15, 2024

Failing tests:

Uh oh!

phsft-bot commented Feb 16, 2024

Failing tests:

Uh oh!

github-actions bot commented Feb 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

aaronj0 commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ferdymercury commented Jun 30, 2025

Uh oh!

guitargeek commented Jun 30, 2025

Uh oh!

couet commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

guitargeek commented Jun 30, 2025

Uh oh!

ferdymercury commented Jun 30, 2025

Uh oh!

couet commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

couet commented Jun 30, 2025

Uh oh!

couet commented Jun 30, 2025

Uh oh!

couet commented Jun 30, 2025

Uh oh!

Uh oh!

aaronj0 commented Feb 15, 2024 •

edited by ferdymercury

Loading

github-actions bot commented Feb 16, 2024 •

edited

Loading

aaronj0 commented Jun 30, 2025 •

edited

Loading

couet commented Jun 30, 2025 •

edited

Loading

couet commented Jun 30, 2025 •

edited

Loading