KeyError when updating a metric with CluStream (cluster module)

[qetdr]

Versions

river version: 0.15.0
Python version: 3.10.4
Operating system: macOS Ventura 13.2

Describe the bug

I tried to run a river's clusterer (CluStream, specifically) and update the metrics with each iteration. However, I received a key error. A fully-reproducible example is below.

Steps/code to reproduce

import pandas as pd
from river.cluster import CluStream
from river import stream
from river.metrics import Silhouette

# Import the data
s1 = pd.read_table('http://cs.uef.fi/sipu/datasets/s1.txt', 
                   sep = "\s+", 
                   names = ['x1', 'x2']).sample(5000, random_state = 42).reset_index(drop = True)

# Taking a random sample for a smaller batch of the data
n_samples = 500
df_first_batch = s1.sample(n_samples).reset_index(drop = True)

clusterer = CluStream(time_window=1,
                      max_micro_clusters=30,
                      n_macro_clusters=15,                      
                      seed=0,
                      halflife=0.4
                     )
metric = Silhouette()

for x, _ in stream.iter_pandas(df_first_batch):
    clusterer = clusterer.learn_one(x)
    y_pred = clusterer.predict_one(x)
    metric = metric.update(x = x, 
                           y_pred = y_pred, 
                           centers = clusterer.centers)

Here's the output:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[2], line 26
     24 clusterer = clusterer.learn_one(x)
     25 y_pred = clusterer.predict_one(x)
---> 26 metric = metric.update(x = x, 
     27                        y_pred = y_pred, 
     28                        centers = clusterer.centers)

File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/river/metrics/silhouette.py:71, in Silhouette.update(self, x, y_pred, centers, sample_weight)
     69 def update(self, x, y_pred, centers, sample_weight=1.0):
---> 71     distance_closest_centroid = math.sqrt(utils.math.minkowski_distance(centers[y_pred], x, 2))
     72     self._sum_distance_closest_centroid += distance_closest_centroid
     74     distance_second_closest_centroid = self._find_distance_second_closest_center(centers, x)

KeyError: 0

[MaxHalford]

Hey there @qetdr. I believe that's not a fully reproducible example; it seems you're missing the training loop. Could you update your post accordingly please?

[qetdr]

Hi @MaxHalford, thank you for the fast response. Based on what you wrote (and also how I understood the clusterer examples), should one:
(1) set up and run a loop where a batch of data is used to only train the model (using the learn_one() function on each data point),
(2) followed by predicting new data instances (predict_one()) and updating a given metric (e.g., Silhouette score) on each iteration?
Presently (e.g., in the script in original post), the learning, prediction, and metric update take place on each iteration in one loop.

Of note, the error does not seem to come up when using, say, STREAMKMeans (with the script from the original post where clustrer is changed to STREAMKMeans). In that case, metrics are produced on each iteration, i.e., by using the "learn_one() -> predict_one() -> metric.update()" steps. Hence, I am interested in whether the error in the original post might arise due to the necessity of different workflow configuration assumed by different algorithms.

Thank you for your help!

[MaxHalford]

Presently (e.g., in the script in original post), the learning, prediction, and metric update take place on each iteration in one loop.

That's the correct way to do online learning. There aren't separate training and learning phases.

Of note, the error does not seem to come up when using, say, STREAMKMeans (with the script from the original post where clustrer is changed to STREAMKMeans). In that case, metrics are produced on each iteration, i.e., by using the "learn_one() -> predict_one() -> metric.update()" steps. Hence, I am interested in whether the error in the original post might arise due to the necessity of different workflow configuration assumed by different algorithms.

Sure, I understand. Still, can you please share a fully copy/pastable script which reproduces the bug?

[qetdr]

Sorry, @MaxHalford, I'm a bit confused: the script in the original post does reproduce the error. Is the loop at the end of the script which includes the "learn->predict->update metric" process not what you mean?

[MaxHalford]

Arf I'm sorry, I didn't notice the script had a scroll-bar 🙈

[MaxHalford]

@hoanganhngo610 can you take a look? (if no answer in 7 days I'll take a look)

[qetdr]

Arf I'm sorry, I didn't notice the script had a scroll-bar 🙈

No problem! I wasn't aware either that the code was not fully displayed. :D Anyway, thanks for the help and I'll be looking forward to an update on this issue.

[qetdr]

Hi @MaxHalford. Just checking on the status of this issue - are you taking a look at it yourself or is @hoanganhngo610 planning to work on it? Thanks!

[MaxHalford]

I'll look at it today! Sorry for not looking into it earlier, I'm busy as usual :)

[MaxHalford]

So it seems that CluStream doesn't have any centers for the first 100 samples. Is this expected @hoanganhngo610?

@qetdr you can get around this issue for now by doing this:

for i, (x, _) in enumerate(stream.iter_pandas(df_first_batch)):
    clusterer = clusterer.learn_one(x)
    y_pred = clusterer.predict_one(x)
    if not clusterer.centers:
        continue
    metric = metric.update(x = x, 
                           y_pred = y_pred, 
                           centers = clusterer.centers)