Guidance for Integrating KBLaM with Food/Nutrition Knowledge Base

[maksvavken]

Hello KBLaM team,

First of all, thank you for making this project open and accessible — it's a really exciting approach to structured knowledge injection.

I'ma student working on a domain-specific use case involving a large food and nutrition database, which includes thousands of entities covering products, nutritional values, food groups, and other attributes such as:

• Macronutrients (e.g. ENERC, FAT, CHO, PROT)
• Micronutrients (e.g. VITC, CA, FE)
• Numeric values (e.g. 12.2 g sugar, 0.6 g saturated fat, 61.0 g carbohydrates)
• Categorization tags (e.g. "cereal products", "vegetables")

---

Training Setup

I trained the model for 2,000 steps using the Meta-Llama-3-8B-Instruct model. The food database was transformed into the expected KBLaM format like this:

{"name": "White pepper", "property": "FOOD_GROUP", "value": "Seasoning"}
{"name": "White pepper", "property": "CA", "value": "265.0"}
{"name": "Beef soup", "property": "THIA", "value": "0.022"}

Observed Issues

• Despite the training, I’m encountering several challenges:

• Poor retrieval quality for health-related queries
  e.g. “Which foods are good for diabetics?” often retrieves items with high sugar or refined carbs.

• Abbreviations like ENERC, FIBT, FASAT are not well understood
  (Note: I plan to map these to full names in the next training run.)

• Generated outputs are incoherent, sometimes repeating the user's prompt or hallucinating answers.

• Numeric values (e.g. 12.6g) appear particularly problematic for the model to use effectively.

Request for Guidance

Could you please advise on best practices for integrating a numeric-heavy, structured KB like this into KBLaM?

Specifically:

• Handling Numeric Data: Any suggestions for how to better encode or structure numerical values so that the model can reason over them effectively?

• Downstream Fine-Tuning: Should I augment the training with open-ended QA examples (e.g. “What foods are good for a Mediterranean diet?”)?

I understand that numerical information may be difficult for the current compression method, as noted in the paper, but any insights or advice would be greatly appreciated.

Thank you again for your great work and for your time!

[ti250]

What sort of questions are you currently training with? "Which foods are good for diabetics?" is a lot more complex than what we trained on in the paper, so you may find that we need additional tricks/additional complex questions of that sort. The KBLaM model in the paper also needed some extra training for apologising when the answer wasn't in its context.

As you seem to have observed, the sentence embedders used may or may not do well with numeric values. This was something we side-stepped in the paper by working on descriptive values, but it seems something we cannot avoid here. My personal expectation is that it would work for general magnitudes, but not exact numbers. Here, it seems that exact numbers would be important though and I'd be very interested to see what solutions you find!

The abbreviations not being well understood are also expected, given the sort of sentence encoders we're using to generate the key and value tokens. You may find it helpful to change the formatting as well, e.g. instead of "The CA of White pepper" for the key and "265.0" for the value, given that you have a closed schema, it would make sense to format this a little nicer, e.g. "White pepper calorie count", "265.0 calories", and corresponding, for each different property type.

2,000 steps also seems to be undertrained given our experience. While we don't have logging code here, if you add logging with some validation set, I think you'll find that you can train for quite a bit more.

Please let me know if you have any other questions!

[maksvavken]

Thank you for your reply!

What I’m Currently Doing

Right now, I’m only training on simple questions like this:

What is the value of PROT in {food name}? 
The value of PROT in {food name} is 2.6.

This is repeated for every food and every property in the dataset — so I have around 300K examples in total.

I trained the LLaMA 3 8B model for 5000 steps, and here’s what I’m seeing:

---

Problem with Numbers

Prompt:
What is the value of PROT in {food name}?

Model Output:
The value of PROT in {food name} is 4.8.

Ground Truth:
The value of PROT in {food name} is 6.8.

The model often gives the wrong number, but the top attended KB item is correct, which is a good sign.

---

Example of a More Complex Question

Prompt:
Is {food name} appropriate for people with high cholesterol?

Model Output:
{food name} appropriate for people with high cholesterol.

The answer is short and has grammar issues.
But again, the attended KBs are relevant — they include the right food properties and even similar items. That’s great.

---

Open-Ended Question

Prompt:
Which foods are appropriate for a Mediterranean diet?

Model Output:
Which foods are appropriate for a meditarranean diet?

The model just repeats the question and has a typo.
But the attended KBs are surprisingly good — they include foods and nutrients that make sense for this kind of diet.

---

Observations

At first, I hoped that by training on simple questions, the model would learn to also answer more open-ended ones. But that doesn’t seem to be working.

Still, I’m really happy with how well the model retrieves relevant knowledge — even when the answer it gives isn’t good. This makes me think KBLaM could work well as a retriever in a RAG pipeline, where another model would generate the final answer based on what KBLaM finds.

---

Questions I Still Have

I'm wondering:

• How should I change the training data to help the model give better answers — ones that are grammatically correct and make more sense?
• Since I want to support broader questions, do I need to train on those types of examples directly?
• Are there any tricks for making the model understand and use numeric values better?

Thanks again for all the help and for making this project available. I really appreciate your feedback!

[gwyong]

Hello, All.
Any updates?

[maksvavken]

Hello, unfortunately our team stopped working on this because it yielded no promising results (worse than our existing RAG).
The model couldnt form sentences, or formed grammarly incorrect sentences even after 20K steps of training. The problem with numbers also persisted.

[gwyong]

Hello, unfortunately our team stopped working on this because it yielded no promising results (worse than our existing RAG). The model couldnt form sentences, or formed grammarly incorrect sentences even after 20K steps of training. The problem with numbers also persisted.

Thank you for your response. May I ask two more questions?

1. When you did fine-tuning, did you training from scratch? or did you use pre-trained weights from the original synthetic_dataset?
2. When you did fine-tuning, did you observe the increasing KB_size as epoch increases? (I found that mine shows consistent KB_size during training)

Thanks,

[maksvavken]

We did training from scratch. I am sorry but i cannot answer the second question because I didn't observe kb_size.

[gwyong]

We did training from scratch. I am sorry but i cannot answer the second question because I didn't observe kb_size.

Thank you for your response. It helps me a lot.