chore: update paper trend data

Author: github-actions[bot]

data/full/acl-2023.json

@@ -23889,7 +23889,7 @@
     "title": "Low-Rank Updates of pre-trained Weights for Multi-Task Learning",
     "authors": [
       "Alexandre Audibert",
-      "Massih R Amini",
+      "Massih-Reza Amini",
       "Konstantin Usevich",
       "Marianne Clausel"
     ],

data/full/acl-2024.json

@@ -7496,7 +7496,7 @@
     "title": "Can Large Language Model Summarizers Adapt to Diverse Scientific Communication Goals?",
     "authors": [
       "Marcio Fonseca",
-      "Shay Cohen"
+      "Shay B. Cohen"
     ],
     "abstract": "In this work, we investigate the controllability of large language models (LLMs) on scientific summarization tasks. We identify key stylistic and content coverage factors that characterize different types of summaries such as paper reviews, abstracts, and lay summaries. By controlling stylistic features, we find that non-fine-tuned LLMs outperform humans in the MuP review generation task, both in terms of similarity to reference summaries and human preferences. Also, we show that we can improve the controllability of LLMs with keyword-based classifier-free guidance (CFG) while achieving lexical overlap comparable to strong fine-tuned baselines on arXiv and PubMed. However, our results also indicate that LLMs cannot consistently generate long summaries with more than 8 sentences. Furthermore, these models exhibit limited capacity to produce highly abstractive lay summaries. Although LLMs demonstrate strong generic summarization competency, sophisticated content control without costly fine-tuning remains an open problem for domain-specific applications.",
     "year": 2024,
@@ -7540,7 +7540,7 @@
     "title": "Can Large Language Models Follow Concept Annotation Guidelines? A Case Study on Scientific and Financial Domains",
     "authors": [
       "Marcio Fonseca",
-      "Shay Cohen"
+      "Shay B. Cohen"
     ],
     "abstract": "Although large language models (LLMs) exhibit remarkable capacity to leverage in-context demonstrations, it is still unclear to what extent they can learn new facts or concept definitions via prompts. To address this question, we examine the capacity of instruction-tuned LLMs to follow in-context concept annotation guidelines for zero-shot sentence labeling tasks. We design guidelines that present different types of factual and counterfactual concept definitions, which are used as prompts for zero-shot sentence classification tasks. Our results show that although concept definitions consistently help in task performance, only the larger models (with 70B parameters or more) have limited ability to work under counterfactual contexts. Importantly, only proprietary models such as GPT-3.5 can recognize nonsensical guidelines, which we hypothesize is due to more sophisticated alignment methods. Finally, we find that Falcon-180B-chat is outperformed by Llama-2-70B-chat is most cases, which indicates that increasing model scale does not guarantee better adherence to guidelines. Altogether, our simple evaluation method reveals significant gaps in concept understanding between the most capable open-source language models and the leading proprietary APIs.",
     "year": 2024,
@@ -25409,7 +25409,7 @@
     "authors": [
       "Michelle Lo",
       "Fazl Barez",
-      "Shay Cohen"
+      "Shay B. Cohen"
     ],
     "abstract": "Advances in model editing through neuron pruning hold promise for removing undesirable concepts from large language models. However, it remains unclear whether models have the capacity to reacquire pruned concepts after editing. To investigate this, we evaluate concept relearning in models by tracking concept saliency and similarity in pruned neurons during retraining for named entity recognition tasks. Our findings reveal that models can quickly regain performance post-pruning by relocating advanced concepts to earlier layers and reallocating pruned concepts to primed neurons with similar semantics. This suggests that models exhibit polysemantic capacities and can blend old and new concepts in individual neurons. While neuron pruning provides interpretability into model concepts, our results highlight the challenges of permanent concept removal for improved model *safety*. Monitoring concept reemergence and developing techniques to mitigate relearning of unsafe concepts will be important directions for more robust model editing. Overall, our work strongly demonstrates the resilience and fluidity of concept representations in LLMs post concept removal.",
     "year": 2024,

data/full/acl-2025.json

@@ -21352,7 +21352,7 @@
       "Varun R. Embar",
       "Yizhe Zhang",
       "Navdeep Jaitly",
-      "Shay B Cohen",
+      "Shay B. Cohen",
       "Benjamin Han"
     ],
     "abstract": "Recent advancements in long-context language models (LCLMs) promise to transform Retrieval-Augmented Generation (RAG) by simplifying pipelines. With their expanded context windows, LCLMs can process entire knowledge bases and perform retrieval and reasoning directly – a capability we define as In-Context Retrieval and Reasoning (ICR^2). However, existing benchmarks like LOFT often overestimate LCLM performance by providing overly simplified contexts. To address this, we introduce ICR^2, a benchmark that evaluates LCLMs in more realistic scenarios by including confounding passages retrieved with strong retrievers. We then propose three methods to enhance LCLM performance: (1) retrieve-then-generate fine-tuning, (2) retrieval-attention-probing, which uses attention heads to filter and de-noise long contexts during decoding, and (3) joint retrieval head training alongside the generation head. Our evaluation of five well-known LCLMs on LOFT and ICR^2 demonstrates significant gains with our best approach applied to Mistral-7B: +17 and +15 points by Exact Match on LOFT, and +13 and +2 points on ICR^2, compared to vanilla RAG and supervised fine-tuning, respectively. It even outperforms GPT-4-Turbo on most tasks despite being a much smaller model.",
@@ -47726,7 +47726,7 @@
       "Clara Vania",
       "Subhradeep Kayal",
       "Naila Khan",
-      "Shay B Cohen",
+      "Shay B. Cohen",
       "Emine Yilmaz"
     ],
     "abstract": "Large language models (LLMs) have advanced conversational AI assistants. However, systematically evaluating how well these assistants apply personalization—adapting to individual user preferences while completing tasks—remains challenging. Existing personalization benchmarks focus on chit-chat, non-conversational tasks, or narrow domains, failing to capture the complexities of personalized task-oriented assistance. To address this, we introduce PersonaLens, a comprehensive benchmark for evaluating personalization in task-oriented AI assistants. Our benchmark features diverse user profiles equipped with rich preferences and interaction histories, along with two specialized LLM-based agents: a user agent that engages in realistic task-oriented dialogues with AI assistants, and a judge agent that employs the LLM-as-a-Judge paradigm to assess personalization, response quality, and task success. Through extensive experiments with current LLM assistants across diverse tasks, we reveal significant variability in their personalization capabilities, providing crucial insights for advancing conversational AI systems.",
@@ -102391,7 +102391,7 @@
     "id": "2025.acl-long.1351",
     "title": "Evaluating Sequence Labeling on the basis of Information Theory",
     "authors": [
-      "Enrique Amigo",
+      "Enrique Amigó",
       "Elena Álvarez-Mellado",
       "Julio Gonzalo",
       "Jorge Carrillo-de-Albornoz"
@@ -151892,7 +151892,7 @@
       "Joshua Ong Jun Leang",
       "Giwon Hong",
       "Wenda Li",
-      "Shay B Cohen"
+      "Shay B. Cohen"
     ],
     "abstract": "The demand for synthetic data in mathematical reasoning has increased due to its potential to enhance the mathematical capabilities of large language models (LLMs). However, ensuring the validity of intermediate reasoning steps remains a significant challenge, affecting data quality. While formal verification via theorem provers effectively validates LLM reasoning, the autoformalisation of mathematical proofs remains error-prone. In response, we introduce *iterative autoformalisation*, an approach that iteratively refines theorem prover formalisation to mitigate errors, thereby increasing the execution rate on the Lean prover from 60% to 87%. Building upon that, we introduce *Theorem Prover as a Judge (TP-as-a-Judge)*, a method that employs theorem prover formalisation to rigorously assess LLM intermediate reasoning, effectively integrating autoformalisation with synthetic data generation. Finally, we present *Reinforcement Learning from Theorem Prover Feedback (RLTPF),* a framework that replaces human annotation with theorem prover feedback in Reinforcement Learning from Human Feedback (RLHF). Across multiple LLMs, applying *TP-as-a-Judge* and *RLTPF* improves benchmarks with only 3,508 samples, achieving 5.56% accuracy gain on Mistral-7B for MultiArith, 6.00% on Llama-2-7B for SVAMP, and 3.55% on Llama-3.1-8B for AQUA.",
     "year": 2025,

data/full/emnlp-2023.json

@@ -1011,7 +1011,7 @@
       "Zheng Zhao",
       "Yftah Ziser",
       "Bonnie Webber",
-      "Shay Cohen"
+      "Shay B. Cohen"
     ],
     "abstract": "We present an analysis tool based on joint matrix factorization for comparing latent representations of multilingual and monolingual models. An alternative to probing, this tool allows us to analyze multiple sets of representations in a joint manner. Using this tool, we study to what extent and how morphosyntactic features are reflected in the representations learned by multilingual pre-trained models. We conduct a large-scale empirical study of over 33 languages and 17 morphosyntactic categories. Our findings demonstrate variations in the encoding of morphosyntactic information across upper and lower layers, with category-specific differences influenced by language properties. Hierarchical clustering of the factorization outputs yields a tree structure that is related to phylogenetic trees manually crafted by linguists. Moreover, we find the factorization outputs exhibit strong associations with performance observed across different cross-lingual tasks. We release our code to facilitate future research.",
     "year": 2023,
@@ -36119,7 +36119,7 @@
       "Ashok Urlana",
       "Pinzhen Chen",
       "Zheng Zhao",
-      "Shay Cohen",
+      "Shay B. Cohen",
       "Manish Shrivastava",
       "Barry Haddow"
     ],
@@ -38762,14 +38762,14 @@
     "id": "2023.findings-emnlp.104",
     "title": "Remember what you did so you know what to do next",
     "authors": [
-      "Manuel Ciosici",
+      "Manuel R. Ciosici",
       "Alex Hedges",
       "Yash Kankanampati",
       "Justin Martin",
       "Marjorie Freedman",
       "Ralph Weischedel"
     ],
-    "abstract": "We explore using the 6B parameter GPT-J language model to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld, a text game simulator for elementary science experiments and for which previously published empirical work has shown large language models (LLM)s to be a poor fit (Wang et al., 2022). Using the Markov assumption, the LLM outperforms the state-of-the-art based on reinforcement learning by a factor of 1.4. When we fill the LLM’s input buffer with as many prior steps as will fit, improvement rises to 3.3x. Even when training on only 6.5% of the training data, we observe a 2.3x improvement over the state-of-the-art. Our experiments show that performance varies widely across the 30 classes of actions, indicating that averaging over tasks can hide significant performance issues.",
+    "abstract": "We explore using a moderately sized large language model (GPT-J 6B parameters) to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld, a text game simulator for elementary science experiments. Previously published empirical work claimed that large language models (LLMs) are a poor fit (Wang et al., 2022) compared to reinforcement learning. Using the Markov assumption (a single previous step), the LLM outperforms the reinforcement learning-based approach by a factor of 1.4. When we fill the LLM's input buffer with as many prior steps as possible, improvement rises to 3.5x. Even when training on only 6.5% of the training data, we observe a 2.2x improvement over the reinforcement-learning-based approach. Our experiments show that performance varies widely across the 30 classes of actions, indicating that averaging over tasks can hide significant performance issues.In work contemporaneous with ours, Lin et al. (2023) demonstrated a two-part approach (SwiftSage) that uses a small LLM (T5-large) complemented by OpenAI's massive LLMs to achieve outstanding results in ScienceWorld. Our 6-B parameter, single-stage GPT-J matches the performance of SwiftSage's two-stage architecture when it incorporates GPT-3.5 turbo which has 29-times more parameters than GPT-J.",
     "year": 2023,
     "conference": "emnlp",
     "conferenceName": "EMNLP",
@@ -38791,13 +38791,13 @@
         "id": "llms",
         "label": "LLMs & Foundation Models",
         "color": "#2563eb",
-        "score": 4
+        "score": 5
       },
       {
         "id": "rl_robotics",
         "label": "RL, Robotics & Control",
         "color": "#ea580c",
-        "score": 2
+        "score": 3
       },
       {
         "id": "domain_science",
@@ -53664,7 +53664,7 @@
     "title": "AMR Parsing is Far from Solved: G r APES , the Granular AMR Parsing Evaluation Suite",
     "authors": [
       "Jonas Groschwitz",
-      "Shay Cohen",
+      "Shay B. Cohen",
       "Lucia Donatelli",
       "Meaghan Fowlie"
     ],
@@ -63608,7 +63608,7 @@
       "Yftah Ziser",
       "Anna Korhonen",
       "Edoardo Ponti",
-      "Shay Cohen"
+      "Shay B. Cohen"
     ],
     "abstract": "Hallucinations pose a significant challenge to the reliability of neural models for abstractive summarisation. While automatically generated summaries may be fluent, they often lack faithfulness to the original document. This issue becomes even more pronounced in low-resource languages, where summarisation requires cross-lingual transfer. With the existing faithful metrics focusing on English, even measuring the extent of this phenomenon in cross-lingual settings is hard. To address this, we first develop a novel metric, mFACT, evaluating the faithfulness of non-English summaries, leveraging translation-based transfer from multiple English faithfulness metrics. Through extensive experiments in multiple languages, we demonstrate that mFACT is best suited to detect hallucinations compared to alternative metrics. With mFACT, we assess a broad range of multilingual large language models, and find that they all tend to hallucinate often in languages different from English. We then propose a simple but effective method to reduce hallucinations in cross-lingual transfer, which weighs the loss of each training example by its faithfulness score. This method drastically increases both performance and faithfulness according to both automatic and human evaluation when compared to strong baselines for cross-lingual transfer such as MAD-X. Our code and dataset are available at https://github.com/yfqiu-nlp/mfact-summ.",
     "year": 2023,
@@ -95269,12 +95269,12 @@
     "id": "2023.emnlp-main.848",
     "title": "The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis",
     "authors": [
-      "Pranav Venkit",
+      "Pranav Narayanan Venkit",
       "Mukund Srinath",
       "Sanjana Gautam",
       "Saranya Venkatraman",
       "Vipul Gupta",
-      "Rebecca Passonneau",
+      "Rebecca J. Passonneau",
       "Shomir Wilson"
     ],
     "abstract": "We conduct an inquiry into the sociotechnical aspects of sentiment analysis (SA) by critically examining 189 peer-reviewed papers on their applications, models, and datasets. Our investigation stems from the recognition that SA has become an integral component of diverse sociotechnical systems, exerting influence on both social and technical users. By delving into sociological and technological literature on sentiment, we unveil distinct conceptualizations of this term in domains such as finance, government, and medicine. Our study exposes a lack of explicit definitions and frameworks for characterizing sentiment, resulting in potential challenges and biases. To tackle this issue, we propose an ethics sheet encompassing critical inquiries to guide practitioners in ensuring equitable utilization of SA. Our findings underscore the significance of adopting an interdisciplinary approach to defining sentiment in SA and offer a pragmatic solution for its implementation.",