Skip to content

rfastlowess: High-Performance LOWESS Smoothing for R #769

Description

@thisisamirv

Submitting Author Name: Amir Valizadeh
Submitting Author Github Handle: @thisisamirv
Other Package Authors Github handles:
Repository: https://github.com/thisisamirv/lowess-project
Sub-directory: bindings/r
Version submitted: 1.3.0
Submission type: Stats
Badge grade: gold
Editor: TBD
Reviewers: TBD

Archive: TBD
Version accepted: TBD
Language: en


  • Paste the full DESCRIPTION file inside a code block below:
Package: rfastlowess
Title: High-Performance LOWESS Smoothing for R
Version: 1.3.0
Authors@R: person("Amir", "Valizadeh", email = "thisisamirv@gmail.com", role = c("aut", "cre", "fnd"), comment = c(ORCID = "0000-0001-5983-8527"))
Description: Provides high-performance LOWESS smoothing (Locally Weighted Scatterplot Smoothing) 
    using a 'Rust' backend. Supports various weight functions, robustness iterations, 
    streaming/online processing, cross-validation, and uncertainty quantification.
    Applicable to genomic data, time series, and general-purpose smoothing tasks.
License: MIT + file LICENSE | Apache License (== 2.0)
SystemRequirements: 'Rust' programming language (via 'rustup' or system tools)
Encoding: UTF-8
Roxygen: list(markdown = TRUE, roclets = c("namespace", "rd", "srr::srr_stats_roclet"))
biocViews: Preprocessing, StatisticalMethod, Regression, FunctionalPrediction, QualityControl, KEGG
Imports: BiocGenerics
Suggests: BiocStyle, knitr, rmarkdown, testthat (>= 3.0.0)
Config/testthat/edition: 3
VignetteBuilder: knitr
BugReports: https://github.com/thisisamirv/lowess-project/issues
URL: https://github.com/thisisamirv/lowess-project, https://lowess.readthedocs.io/
Config/rextendr/version: 0.5.0
Config/roxygen2/version: 8.0.0

Scope

  • Please indicate which of our statistical package categories this package falls under. (Please check one or more appropriate boxes below):

    Statistical Packages

    • Bayesian and Monte Carlo Routines
    • Dimensionality Reduction, Clustering, and Unsupervised Learning
    • Machine Learning
    • Regression and Supervised Learning
    • Exploratory Data Analysis (EDA) and Summary Statistics
    • Spatial Analyses
    • Time Series Analyses
    • Probability Distributions

Pre-submission Inquiry

  • A pre-submission inquiry has been approved in issue#<739>

General Information

  • Who is the target audience and what are scientific applications of this package?
    The target audience includes statisticians, bioinformaticians, and data scientists. Scientific applications include genomic data smoothing, time series analysis, and general-purpose robust trend estimation and filtering.

  • Paste your responses to our General Standard G1.1 here, describing whether your software is:

    • The first implementation of a novel algorithm; or
    • The first implementation within R of an algorithm which has previously been implemented in other languages or contexts; or
    • An improvement on other implementations of similar algorithms in R.

    Please include hyperlinked references to all other relevant software.

    Response:
    This package is an improvement on other implementations of similar algorithms in R, specifically the base R stats::lowess and Python's statsmodels.nonparametric.smoothers_lowess.lowess.

    It improves upon existing implementations by offering:

    1. Speed: Average of 2-3x faster than R's lowess and up to 327x faster than Python's statsmodels via a compiled, multi-threaded Rust backend.
    2. Algorithmic Robustness: Implements Median Absolute Deviation (MAD) for scale estimation rather than the standard median of absolute residuals (MAR), providing a breakdown-point-optimal estimator. It also introduces boundary padding policies (Extend, Reflect, Zero) to remove asymmetric bias near the edges.
    3. Feature Completeness: Includes features absent in base R, such as streaming/online modes, confidence and prediction intervals, k-fold cross-validation, auto-convergence, and 7 distinct kernel options.
  • (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

    Response: Not applicable. This is a general-purpose mathematical modeling and statistical smoothing tool. It does not contain or require human subject datasets.

Badging

  • What grade of badge are you aiming for? (bronze, silver, gold)

    Gold

  • If aiming for silver or gold, describe which of the four aspects listed in the Guide for Authors chapter the package fulfils (at least one aspect for silver; three for gold)

    This package fulfills the requirements for a Gold badge by satisfying the following aspects:

    1. Performance Enhancements via Compiled Code: The core computations are pushed to a memory-safe, multi-threaded Rust backend (via extendr), allowing for highly parallelized execution and significantly improved execution speed over traditional R/C implementations.
    2. Algorithmic Extensions: It vastly extends the baseline LOWESS algorithm by incorporating native support for prediction and confidence interval calculations, multiple robustness weighting and boundary padding policies, and automated cross-validation, moving it far beyond a simple wrapper.
    3. Larger-than-RAM / Streaming capabilities: The package implements streaming and online processing modes for LOWESS smoothing, allowing it to efficiently calculate and update estimations on large out-of-core datasets (such as long-running time-series or heavy genomic sequences) without memory exhaustion.

Technical checks

Confirm each of the following by checking the box.

This package:

Use of Generative AI

  • Generative AI tools were used to produce some of the material in this submission.

Description of AI Usage

Generative AI (Antigravity, an agentic AI assistant) was used in different aspects of the production and maintenance of this software. Specifically:

  • Design of the logo for the software.
  • Writing the foundation of the bindings scripts.
  • Identification and patching of security vulnerabilities (CVEs) in dependency stacks and hardening of FFI layers (Node.js, WASM).
  • Maintaining complex Makefiles and ensuring cross-platform build consistency.
  • Fixing broken workspace tests and improving test coverage.
  • Research and help find different alternative solutions to performance aspects of the software.
  • Design of different workflows for maintenance and release of the software.
  • Debugging and organizing the codebase.

Affirmation: All AI-generated material has been carefully reviewed, tested, and approved by the human authors. Mathematical algorithms were specifically written by the author, cross-checked with the references, and validated for different scenarios.

Publication options

  • Do you intend for this package to go on CRAN?
  • Do you intend for this package to go on Bioconductor?

Code of conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions