API support for string slice parameters instead of null-terminated strings

[bitshifter]

I discovered when trying to call ImGui from Rust that Rust strings are not null terminated and thus using there are some hurdles to jump through to produce a null terminated string to be use with ImGui.

I thought it would be convenient if I could use character data arrays instead of null terminated strings with ImGui.

I've made a branch with a proof-of-concept of this idea by replacing char* parameters in ImGui functions with ImStr - a struct containing a pointer to the start and the end of a string slice. It has a implicit char* constructor, so no calling code needs to be changed. I have a proof of concept branch at https://github.com/bitshifter/imgui/tree/imstr.

I haven't converted everything to ImStr, but it's enough to get the idea I think.

There may actually be some advantages in doing this for the C++ API:

• string_view is being added to C++ - if you aren't already familiar with it, string_view is simply a struct with a const char* pointer to the start and end of a char array, so you can use it to reference char data without needing a null terminator. I've used something similar in code bases that were suffering from severe std::string abuse to reduce a lot of string allocating and copying. It's pretty handy, until you need a null terminated string, which is why it would be nice if ImGui supported this idea
• Most C++ codebases use some kind of string class or classes, whether it's std::string or a custom string. They usually require calling c_str() or something to get a char*. With ImStr, users could add an implicit constructor that creates an ImStr from their string type, meaning they can easily use it with ImGui. String types usually know they length of the string, avoiding the need call strlen in ImGui
• For FFI it helps Rust, and probably other languages that either don't use null-terminated strings, or already know the length of their strings avoiding unnecessary strlen calls.
• On the down side, this probably isn't convenient for C users, but it could at least be hidden behind the C API.

Performance wise, it didn't seem to make a big difference in my profiling, I thought strlen might rise up in the profile but it didn't seem to for me at least.

If you are interested in this idea, I can try finish off converting the rest of the char* parameters a make a PR.

[MrSapps]

string_view will never be used because its C++11

[Cthutu]

Use std::string to produce your null terminated string: std::string(start,
end).c_str()

On Wed, 20 Jan 2016 at 07:36 Paul notifications@github.com wrote:

string_view will never be used because its C++11

—
Reply to this email directly or view it on GitHub
#494 (comment).

[bitshifter]

string_view will never be used because its C++11

I'm not suggesting that ImGui should use string_view and I haven't used it (or any other C++11) in this change. My point is that people who use ImGui may use string_view in their code and this change makes it easy use with ImGui, while still allowing people to use char*, or other sting types.

[bitshifter]

Use std::string to produce your null terminated string: std::string(start, end).c_str()

This will cause many many allocations and string copies, which can be avoided with this change.

[ocornut]

edited

@bitshifter I'll have to get back to you but give me some time. This is potentially a big change to follow on and I don't have sufficient bandwidth at the moment. One of my concern, which is arguably hard to measure, is that I don't want to give the impression to people skimming at the library that this is a high-level library that uses lots of mysterious and heavy data structures. Note that I am merely talking about the impression here. C/C++ developers are very reluctant to use libraries and ImGui aim to combat that. char* carry a natural feeling of accessibility to many devs. Now, your point are valid and I don't want to reject it. Interested in your progress and thoughts down the line.

@Cthutu : very slow very heavy. The whole point of slices is to avoid it.

[bitshifter]

@ocornut Yeah, I totally agree/understand your concern, so wouldn't be offended if the idea is ultimately rejected :)

I'll have a look at converting the remaining char* at some point, some of them might be a bit less straight forward than the labels so it would be interesting to see how it pans out.

[ocornut]

Some ideas:

• I'd suggest to make ImStr as simple as possible aka const char* Begin (as members but no accessor). I realise it seems odd but default compilation settings for Visual Studio in Debug make even trivial normally inlined function calls have measurable cost. As I aim to minimize CPU footprint even in the worse condition. I also found it very liberating to ditch unnecessary accessors and also ditch size_t in favor of int (but here we don't store either).
• This is something that can be investigated later (or by me) but I might like a system where the End point can be lazily evaluated, allowing for single pass on large blurb of texts instead of two passes (e.g. TextUnformatted is tuned to do single pass on all text that is clipped).
  That would mean the constructors would NOT do the strlen() but certain end points of the code would do it and store back only if they need it (defaulting to zero-terminated). (the act of storing back may imply passing ImStr by value for convenience in the various code using it).
  However I may be overthinking that, TextUnformatted already allows for End to be passed so I could drop the guarantee Begin-only version does one pass and request user to pass a proper slice if they want that guarantee, which is reasonable if you are going to submit kilobytes of texts. So maybe just ignore that paragraph for me, just laying out the idea out loud.

[ocornut]

A potential plan B (IF we can't get plan A to work) would be to make the necessary adjustments to master so that maintaining a fork is painless. I am also open to make that sort of work easier.

[bitshifter]

Good point about function call overhead in debug builds. I wonder if anything can be done about the function call overhead of the ImStr constructors themselves.

I've committed some updates to my branch based on your feedback:

• Simplified ImStr by removing all accessors and methods except for;
• Lazily evaluate End pointer when needed, added ImStr::CalcEnd() to do this

I think the lazy evaluation works well, as many method calls don't need End to be set anyway, e.g. if all they are doing is calculating a hash.

I didn't do anything specific with TextUnformatted, it currently does a CalcEnd(), it was already doing a strlen before so no change in behaviour here.

Regarding a fork, it seems reasonable to have gestate this in a separate branch and see if there's any demand for it.

I'm not sure what would make maintenance easier. Code wise, the changes are reasonably straight forward, the only problem areas were were the new function signature would match old code (e.g. the new CalcTextSize(ImStr, bool, float) function can be called with old CalcTextSize(char*, NULL) and the NULL will be used as the bool parameter). Some RenderText calls also had this problem. They're pretty easy to spot though. Of course there maybe other things I haven't noticed yet.

I think what would probably help most with maintaining a fork is probably more along the infrastructure side. If there was someway to automate merging and building commits to master against the imstr branch that would remove a lot of manual work. It's possible that github/travis already does this.

[ratchetfreak]

There is a bug in your ImStr::CalcEnd() if ImStr is constructed with NULL (as happens with some default parameters like for column).

And why doesn't ImStrDup use CalcEnd itself?

[bitshifter]

@ratchetfreak The intention is that ImStrdup and CalcEnd are only used internally by ImGui, which means it's easier to expect preconditions like ImStr.Begin being non-null before CalcEnd is called.

This is effectively the case in ImGui already (master, not my branch), often a text_end value is calculated by calling strlen on a char* text parameter, it is expected that the char* parameter is not null, but this is not checked at runtime.

Still, I am tempted to make CalcEnd a function which is completely internal to ImGui to prevent people from using it externally, since it is expected to be used in a specific way.

I've changed the signature of ImStrdup so hopefully it's clearer that it should be given a valid End pointer. The main reason I don't call CalcEnd in there is I think it's preferable that End is calculated as soon as possible if it's needed, to avoid repeatedly recalculating it in leaf functions like ImStrdup.

@ocornut I've made a few more commits porting more of the API over to use ImStr. It's mostly done now, just combo lists and list boxes and text filtering to go I think.

[ratchetfreak]

There is also formatting to be done

I suggest adding a ImFormatString(ImStr buffer, ImStr fmt, ...) that will check fmt.End and if it's null just pass to the char* version (because it expects a null terminated string anyway)

char* fmt_buf;
if(fmt.End){
    fmt_buf = alloca(fmt.End-fmt.Begin+1);
    memcpy(fmt_buf, fmt.Begin, fmt.End-fmt.Begin);
    fmtbuf[fmt.End-fmt.Begin] = 0;
} else {
    fmt_buf = fmt;
}
ImFormatString(buffer.Begin, buffer.End - buffer.Begin, fmt_buf, args);

[bitshifter]

Yes that's true. I was intentionally leaving the format strings as char*, but as you point out the format copy is only required if End is actually set to something, so that doesn't seem so bad.