Skip to content

Classify technical-debt category (blocked on the TD taxonomy/oracle) #39

Description

@khatchad

Follow-on to #11, blocked. Do not start yet. #11 builds the data-refactoring type classifier from a commit's change. This sibling issue is the parallel move on the other axis: classifying the technical-debt category a data refactoring addresses.

Why A Classifier, In This Domain

The ICSE '21 ML-systems study was mostly manual. Refactoring identification used no tools; the refactoring-type and technical-debt categories were open-coded into a hierarchy from the change plus developer evidence (issue-tracker discussion, developer communication), with RefactoringMiner only aiding classification in some cases and blind to the novel categories devised there. TD was never a classification target.

Two things make automated classification the right call here instead:

  • No detector exists for data refactorings. There is no RefactoringMiner equivalent, so even the type axis must be learned from the change (that is Build the evidence-grounded rule classifier #11).
  • The manual method's evidence is missing. Hugging Face dataset commits rarely link issue trackers or explain intent in their log messages, so the developer-communication evidence that made manual TD coding tractable is not available here. Classifying from the change is the scalable path, and the study already committed to it in Build the evidence-grounded rule classifier #11.

Why It Is Still Blocked

A classifier needs a TD-labeled oracle, and none exists yet. Producing one is a research-design step tracked off this repo. Nothing here can start until that oracle is committed.

It is also downstream of #11 in a stronger sense than the blocked-by relation shows: #11's stage 1 has to be scored, not merely underway, before this one starts.

Scope, When Unblocked

Mirror #11's staging, one stage at a time:

  1. Evidence-Grounded Rules. Message and diff signals, plus the refactoring type as a feature, scored per TD category against the TD-labeled oracle. As in Build the evidence-grounded rule classifier #11, this classifier extracts its own evidence; it does not call analysis/inspect_commit.py.
  2. Feature-Based ML. Only if it beats the rule baseline.
  3. Optional LLM. Only if it beats the rules, and disclosed as a threat to validity.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions