Support for Exporting Template and Namespaced Functions in Rcpp

[ManosPapadakis95]

Enhancement: Support for Exporting Template and Namespaced Functions in Rcpp

Rcpp has been an incredible tool for bridging R and C++, and its flexibility has inspired me to explore new ways to extend its functionality. Building on the existing framework, I have added support for exporting template functions and functions inside namespaces, expanding the possibilities for advanced C++ integration.

Key Features

1. Support for Exporting Template Functions

• The Rcpp::export attribute can now be applied to explicit template instantiations.
• The function must have an explicit instantiation in the .cpp file for it to be correctly recognized.
• Works seamlessly with Rcpp::interfaces(cpp).

2. Support for Exporting Functions Inside Namespaces

• Functions within namespaces can now be exported, as long as a forward declaration is provided with cpp_name in Rcpp::export.
• This ensures proper symbol resolution while allowing structured code organization.

3. Handling of Declaration Specifiers

• The system now accounts for one and only one declaration specifier:
  • template → Used for explicit template instantiations.
  • inline → Supports functions defined inside headers.
• Combinations like "static inline" are not supported (e.g., static is explicitly disallowed).
• An inline function can still be forward-declared without the inline keyword, and it will work seamlessly.

4. New includes Attribute for Header File Support

• Since Rcpp does not scan for header files, a new Rcpp::includes attribute has been introduced.
• This allows specifying one or more header file paths, ensuring necessary definitions are available in the generated .cpp file.
• This is especially important for template functions, as the compiler needs access to their full definitions in order to instantiate them correctly.

Example Usage

// [[Rcpp::includes("helpers.h")]]

// [[Rcpp::export(name = "A1", cpp_name = "A1")]]
template NumericVector A<NumericVector>(size_t n);

// [[Rcpp::export(name = "A2", cpp_name = "A2")]]
template NumericVector A<NumericVector>(NumericVector x);

// [[Rcpp::export(name = "A3", cpp_name = "A3")]]
NumericVector Test::G();

Generated Rcpp Export File

// A<NumericVector>
template NumericVector A<NumericVector>(size_t n);
RcppExport SEXP _testRcpp_A1(SEXP nSEXP) {
BEGIN_RCPP
    Rcpp::RObject rcpp_result_gen;
    Rcpp::RNGScope rcpp_rngScope_gen;
    Rcpp::traits::input_parameter< size_t >::type n(nSEXP);
    rcpp_result_gen = Rcpp::wrap(A<NumericVector>(n));
    return rcpp_result_gen;
END_RCPP
}

// A<NumericVector>
template NumericVector A<NumericVector>(NumericVector x);
RcppExport SEXP _testRcpp_A2(SEXP xSEXP) {
BEGIN_RCPP
    Rcpp::RObject rcpp_result_gen;
    Rcpp::RNGScope rcpp_rngScope_gen;
    Rcpp::traits::input_parameter< NumericVector >::type x(xSEXP);
    rcpp_result_gen = Rcpp::wrap(A<NumericVector>(x));
    return rcpp_result_gen;
END_RCPP
}

// Test::G
NumericVector Test::G();
RcppExport SEXP _testRcpp_A3() {
BEGIN_RCPP
    Rcpp::RObject rcpp_result_gen;
    Rcpp::RNGScope rcpp_rngScope_gen;
    rcpp_result_gen = Rcpp::wrap(Test::G());
    return rcpp_result_gen;
END_RCPP
}

static const R_CallMethodDef CallEntries[] = {
    {"_testRcpp_A1", (DL_FUNC) &_testRcpp_A1, 1},
    {"_testRcpp_A2", (DL_FUNC) &_testRcpp_A2, 1},
    {"_testRcpp_A3", (DL_FUNC) &_testRcpp_A3, 0},
    {NULL, NULL, 0}
};

Testing & Future Considerations

I have run extensive tests to ensure this new feature works correctly. The ability to export template and namespaced functions integrates well with the existing Rcpp framework. However, given the complexity of C++ templates and namespaces, there might still be edge cases that I haven’t encountered.

Would this be a feature that the maintainers would be interested in? If so, I would be happy to prepare a pull request.

[eddelbuettel]

Intriguing, and potentially very interesting. Paging @jjallaire who is of course the main author of the 'Attributes' code.

[Enchufa2]

Thanks, @ManosPapadakis95, this is a very interesting proposal. It sounds like you have some working prototype, but I cannot find it in your list of repositories. Would you mind pointing us to it? It would be great to have a look to be able to evaluate and discuss the implementation complexity before going for the PR.

[ManosPapadakis95]

Yes, of course. Sorry, but I forgot to push the commits. They are now available. All the commits I have can be found here.

[eddelbuettel]

Sorry for keeping you waiting on this! As you may have seen we were quite busy establishing a new baseline of C++11 removing some older (now unused) code paths from that compilation standard was not (widely or at all) availble.

Good news is your commit rebase cleanly. That may be something you want to do. (Also it may be easier for you to catch up / play along if you changes are on a branch and you treat your master as a pure copy of our master. I sometimes forget that to when working on PRs for other repos).

With the rebased package, compileAttributes() still works on your included test package which also installs. So the question is how do we best decompose this to review / discuss? Are there in fact multiple proposals here? As in your points 1 to 4 above?

[ManosPapadakis95]

Yes, of course, Dr. Eddelbuettel. I was in a rush to make the commits and forgot that I pushed to the master branch. I have already updated the link in my previous comment to include the new branch as you suggested.

I believe, but I am not entirely sure, that the best way to structure this is:

• cpp_name can work with simple forward-declared functions (including namespaced functions) and is standalone within Rcpp’s current framework. Here are three examples:

// [[Rcpp::export(name = "A3", cpp_name = "A3")]]
NumericVector Test::G(); //The cpp_name field is necessary

//or

// [[Rcpp::export(name = "A3", cpp_name = "A3")]]
NumericVector G(); //No need for the cpp_name but it is accepted

//or

// [[Rcpp::export(name = "A3", cpp_name = "A3")]]
NumericVector G(){} //No need for the cpp_name but it is accepted

• Explicit template instantiations require declaration specifiers, cpp_name, and Rcpp::includes to function correctly. Here is an example:

// [[Rcpp::includes("helpers.h")]]

// [[Rcpp::export(name = "A1", cpp_name = "A1")]]
template NumericVector A<NumericVector>(size_t n); //The cpp_name field and the Rcpp::includes are necessary

• Declaration specifiers only serve as a mechanism to identify and store keywords (e.g., template, inline) within the Function class.

I would say that these are not multiple separate proposals, but rather interdependent components of a single enhancement.

This is just my opinion, though. I'm happy to adjust things based on your suggestions.

[eddelbuettel]

That's helpful. But what might be even better is more examples. Even for the simple declaration. I see some of these used in your testpkg (in the initial version, may have changed) but having more examples may help. We may need them as unit tests anyway.

In short, I see nothing fundamental that stands in the way of a PR like this but it would be good to have more motivation.

[ManosPapadakis95]

That makes a lot of sense. I’ll prepare more examples and update the test package accordingly.

I completely understand if the motivation may not seem pressing from your perspective. In my case, I was exploring the idea of creating a new package that acts as a C++ header-only library. I wanted an easier way to expose template functions to R and use the Rcpp mechanism without writing the intermediate wrappers by hand. Using these enhancements, I was able to reduce the boilerplate in my future package by over 60%, which was a big win in terms of maintainability and clarity.

I’ll commit the additional examples and test updates and follow up here once that’s done.

[eddelbuettel]

Oh I completely see where it could be useful -- fully agreed. We have some use cases where Rcpp Modules, and some where it could not get used, and something like this could definitely help. As would an example packages of course.

[ManosPapadakis95]

Apologies for the silence. It took me some time to create a package with examples (including a mix of all the attribute fields), and it was helpful because I discovered two bugs related to Rcpp::includes and complex template function names. You can find the commits here.

I have also created two packages:

1. testRcpp: This package exports various functions using intermediate wrapped functions or directly with Rcpp::export.
2. testRcppNew: This package exports various functions using the new enhancements (for template or namespaced functions) or directly with Rcpp::export.

I will continue testing the new features, but for now, I wanted to share my progress with these small packages.

[eddelbuettel]

Thanks. Might it make sense to put the example package(s) into a separate repo for easier access and study?

[ManosPapadakis95]

As requested, the packages are placed in separate repositories..

1. testRcpp
2. testRcppNew

[eddelbuettel]

I sent you a quick PR for testRcpp. Feel free to ignore or to adapt partially, but I have a pretty strong dislike of using namespace ... in package code. Explicit use is preferred.

[ManosPapadakis95]

Please proceed with any modifications you deem necessary; I will implement the same changes in testRcppNew.

[Enchufa2]

(Sorry for disappearing, but I'll probably have some time next week to take a look)

[eddelbuettel]

I just noticed that when I run compileAttributes() in your testRcppNew I get a change to RcppExports.cpp that seems odd.

diff --git i/src/RcppExports.cpp w/src/RcppExports.cpp
index bdc1385..13359a3 100644
--- i/src/RcppExports.cpp
+++ w/src/RcppExports.cpp
@@ -3,7 +3,7 @@
 
 #include <Rcpp.h>
 #include "helpers.h"
-#include "Str.h"
+#include "helpers.h"#include "Str.h"
 
 using namespace Rcpp;
 

I am now under R 4.5.0 but I doubt that has an effect.