[REVIEW]: autoLM: AI-Driven Collective Response Synthesis

[editorialbot]

Submitting author: @LarsDeSloover (Lars De Sloover)
Repository: https://github.com/LarsDeSloover/autoLM
Branch with paper.md (empty if default branch):
Version: v1.0.0
Editor: @juanklopper
Reviewers: @tylermenezes, @KonradHoeffner
Archive: Pending
Paper kind: software

Status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/340ae8c64895c59a85262dcaa909f589"><img src="https://jose.theoj.org/papers/340ae8c64895c59a85262dcaa909f589/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/340ae8c64895c59a85262dcaa909f589/status.svg)](https://jose.theoj.org/papers/340ae8c64895c59a85262dcaa909f589)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@tylermenezes & @KonradHoeffner, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://openjournals.readthedocs.io/en/jose/reviewer_guidelines.html. Any questions/concerns please let @juanklopper know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

@tylermenezes, please create your checklist typing: @editorialbot generate my checklist

@KonradHoeffner, please create your checklist typing: @editorialbot generate my checklist

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Repository Analysis Report

Code breakdown

github.com/AlDanial/cloc v 1.98  T=0.02 s (1652.3 files/s, 142400.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
PHP                             23            634             93           1225
Markdown                         2             53              0            151
TeX                              1             15              0            132
JavaScript                       1              3              0             21
-------------------------------------------------------------------------------
SUM:                            27            705             93           1529
-------------------------------------------------------------------------------

Commit Count by Author

32	LarsDeSloover

Repository History

Repository age: First commit on December 20, 2024

Code distribution (top 3 48-hour windows):

Period	Insertions	% of Total	Signal	View Changes
Dec 18, 2024 - Dec 20, 2024	2549	99.3%	🔴	View diff
May 07, 2025 - May 09, 2025	18	0.7%	🟢	View diff

GitHub Activity Metrics

Metric	Count
GitHub stars	0
Forks	0
GitHub contributors	1
Releases	1
Total issues	0
Total pull requests	0
Unique commenters/reviewers (excluding authors)	0

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1095

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🔴 Failed to discover a valid open source license

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1080/15230406.2021.1949392 is OK
- 10.5281/zenodo.13986012 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: ChatGPT for good? On opportunities and challenges ...
- No DOI given, and none found for title: OpenAI GPT-4o mini API

❌ MISSING DOIs

- 10.1007/s10956-008-9140-4 may be a valid DOI for title: Technology-enhanced formative assessment: A resear...
- 10.1187/cbe.06-12-0205 may be a valid DOI for title: Clickers in the large classroom: Current research ...
- 10.1016/j.compedu.2020.103818 may be a valid DOI for title: The effect of using Kahoot! for learning–A literat...
- 10.1016/j.compedu.2019.02.015 may be a valid DOI for title: Comparing success and engagement in gamified learn...
- 10.1016/j.compedu.2009.05.001 may be a valid DOI for title: Examining the benefits and challenges of using aud...
- 10.1111/j.1365-2729.2004.00074.x may be a valid DOI for title: Increasing interactivity in lectures using an elec...
- 10.1016/j.ece.2023.10.001 may be a valid DOI for title: Utilising interactive applications as educational ...
- 10.3390/educsci10120348 may be a valid DOI for title: Student response systems: A multidisciplinary anal...
- 10.1016/j.caeai.2024.100206 may be a valid DOI for title: Automatic assessment of text-based responses in po...
- 10.1016/j.apgeog.2011.12.004 may be a valid DOI for title: An empirical evaluation of the spatial accuracy of...
- 10.1007/s10708-008-9186-0 may be a valid DOI for title: Volunteered geographic information: future researc...
- 10.18653/v1/2024.emnlp-industry.69 may be a valid DOI for title: Tell me what I need to know: Exploring LLM-based (...

❌ INVALID DOIs

- None

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[labarba]

@juanklopper 👋 — I assigned this submission to you to handle as editor. We already have two volunteer reviewers: @tylermenezes, @KonradHoeffner (Thank you both! 🙏 ), so your hardest task is overcome! Now, it shall be a matter of shepherding, and giving your own input to the author.

In particular, I want to ask the author @LarsDeSloover to help us out with the following. The editors are now discussing the journal's position with regards to AI assistance in submissions, and we are paying close attention to those that show large fractions of insertions in a short burst of commits. Your submission shows 99.3% insertions in a 48-hr period, which is a signal of machine-dominant work. We are not rejecting such submissions, but we ask that authors be forthcoming in specifying the history of the artifacts they submit for review and publication.

Your paper includes a "Story of the Project" section, which is one of the items we will be focusing on more. Thank you for including that. I would like to ask you to expand on it to help us understand the scholarship behind the artifact. For submissions with substantial machine-generated code we may ask the authors to describe their methodology to test and verify the work. Authors using AI extensively may need to focus their submission on the actual scholarship, and not the built artifact. What is the intellectual contribution of the author, beyond the mechanics of producing PHP code? If this was machine-generated it is not scholarship.

Please understand that these are new and ongoing conversations among the editors. We are needing to adapt and communicate our stance rather quickly. Submission volume is increasing very rapidly, and it is unsustainable to put the onus on the community to review mostly machine-generated artifacts. We ask for your cooperation.