Reconsider bringing BigIntExportBehavior::Number back

[mackrol]

Since specta_typescript 0.0.11 BigIntExportBehaviour is no longer available and export fails if there is any i64 in the fields. I use i64 as timestamps in milliseconds extensively across many projects. Manually annotating them with different type i.e. #[specta(type = f64)] is a hack that will cause troubles in other exporters . We need a global configuration for i64 and other long precision primitives in the typescript exporter.

There is an explanation why this was removed in bigint-forbidden. But it feels too strict. I as a user have previously agreed to loss of precision and potentially bugs on the receiving end by using BigIntExportBehaviour::Number.

[oscartbeaumont]

Truefully I need to hear more about your usecase to be convinced but I also don't see the appeal of it.

To be clear what you are asking for here is a feature which allows you to write types which are possibly not going to be deserialized correctly in a way which doesn't make it clear the danger your putting yourself in? What happens when your time fields get big enough, your UI will be incorrect in an unpredictable way and that's fine?

Don't get me wrong I do think removing this global option is annoying unless your using a Specta framework but it's intentionally annoying because it makes writing dangerous code explicit and noticable. I am working on supporting bigints in Tauri Specta and TauRPC in a way which won't require overrides and will also preserve the number types losslessly between frontend and backend so if your using those frameworks you won't feel this issue but I know a lot of people do use Specta directly where we can't really have a built-in solution as we don't own the JS side.

I use i64 as timestamps in milliseconds extensively across many projects

By bring back BigIntExportBehaviour your saying your desired implementation is that numbers bigger than (2^53 - 1) should just be sliently truncated by your system. To me this goes against the principals most people pick Rust for which is explicit error handling and reliability.

Manually annotating them with different type i.e. #[specta(type = f64)] is a hack that will cause troubles in other exporters

Trouble with other exporters is definitely a good point. I will think about a better way of overriding but I also am not sure this needs to change because doing is #[specta(type = f64)] everywhere is not designed to be a good solution because it should only be used as a last resort.

We need a global configuration for i64 and other long precision primitives in the typescript exporter.

Part of removing BigIntExportBehaviour is that I think a global configuration is super dangerous. If your in a shared codebase and a developer sets BigIntExportBehaviour::Number and then another developers comes across and adds a new type depending on u64/etc and then they can't be aware of the tradeoff being made.

I as a user have previously agreed to loss of precision and potentially bugs on the receiving end by using BigIntExportBehaviour::Number.

You could define a function to remap the number types on the DataType layer like this which I am using as a temporary patch in Tauri Specta until I finish proper support for loseeless bigints in Tauri.

I would also reccomend just using Tauri Specta/TauRPC if your using Tauri. If your using HTTP we currently lack any maintained frameworks which sucks but at some point one will exist which will take care of this for you.

Another option would be to have DTO objects like User and UserDTO where the conversion from normal to DTO object truncates the numbers and can throw an error if it fails. This feels like something a 3rd party derive macro could make pleasant to work with.

Maybe i'm too harsh on this but I really struggle to see the desire of having a system with unpredictable behaviour that has such an easy opt-in. I know removing this is a bit of a cart before the horse in that even frameworks like Tauri Specta and TauRPC don't support bigints yet but the work on it is in progress and for now the DataType rewrites are maintaining compatibility for existing applications.

[mackrol]

Thank You for thorough response. We're using specta to export typescript bindings for REST services, so RPC framework is a no go for me. In the previous versions it was trivial to make i64 to number mapping (with all it's flaws on the JS side). Now specta out of the box makes generating binding with even one i64 impossible. The f64 trick will not pass code review. The code you pointed out looks promising, but it's still a global mapping right?. So we traded BigIntExportBehaviour for a pretty big block of code that has to maintained by users of specta instead of specta itself. Should I create a dedicated crate for that?

[oscartbeaumont]

NOTE: SOME OF THE LINKS ARE BROKEN PENDING RELEASE OF THE NEXT SPECTA VERSION.

Responding to both @mackrol (message above) and @JustinCMR (from message here).

One follow-up worth flagging in the same scenario: real-world export of serde_json::Value still fails, but the failure has moved. Type for serde_json::Number resolves to f64 | i64 | u64, so Typescript::default() aborts with Error::BigIntForbidden on the i64 / u64 variants:

The current guidance does lack anything on this specific senario so I will update it! This is a hard one as it's not really possible for a user to assert they know what they are doing when they don't control the implementation so this likely requires you to modify the DataType structure to replace the bigint-style types with non-bigint-style types (discussed later on).

The f64 trick will not pass code review.

I have added a specta_typescript::Number type which can be used instead to make it more explicit. I don't see why this wouldn't pass codereview when it's you explicitly asserting the tradeoff your making when you making it.

I view it similar to a Rust unsafe block. You put it around a block of code to say "I assert this is safe as the borrow checker is unable to".

It's definitely less clean but it's calling out an unsafety in your system.

The code you pointed out looks promising, but it's still a global mapping right?

Yes it is a global mapping. Frameworks like Tauri Specta and TauRPC needs to be able to control the global mapping as they need to make the types export as BigInt as they could have runtime and serializers powerful enought to make that work losslessly (my writeup on that rabbithole).

This is basically what you want but instead of remapping to BigInt you want to remap to number. Right now the BigInt transformation is going to be part of the semantic types system for suporting Date, Uint8Array and BigInt's so it's not useable directly.

So we traded BigIntExportBehaviour for a pretty big block of code that has to maintained by users of specta instead of specta itself. Should I create a dedicated crate for that?

Pretty much. I have however broken it out into a utility within specta_util as I do think remapping types can be generally useful for people wanting to do more advanced stuff with Specta.

Using the utility, what your trying to do would look something like the following:

let mut remapper = specta_util::Remapper::new();
for primitive in [
    Primitive::usize,
    Primitive::isize,
    Primitive::u64,
    Primitive::i64,
    Primitive::u128,
    Primitive::i128,
] {
    remapper = remapper.rule(DataType::Primitive(primitive), DataType::Primitive(Primitive::u32));
}

let types = remapper.remap_types(types)

// then use `types` for exporting.
//
// Note that if your using `specta_typescript::primitives` you must also use `Remapper:remap_dt`.

This is likely your best bet for now. Would be curious if this would pass your code review standards/your thoughts on it?

I do understand how this sort of thing can be extremely annoying so I am going to keep this discussion open and would love more feedback but also as the author of a crate used by a lot of people I want to push those people towards building good and reliable systems and I think at the end of the day there is a certain physics to this sort of stuff and it's fine for people to do their own risk analysis and ignore it but it's a really dangerous making it too easy to do, especially with with LLM driven development where it's too easy for it to be added, without anyone doing a proper risk analysis or concideration on the risks that it brings.