Add API support for adjusting camera roll angle, extend pitch angle range

[ssokol]

User Story

As a developer I can programmatically adjust the roll angle of the camera from 0 (level) to +/- 180° and I can programmatically adjust the pitch angle from 0° (looking down) to 180° (looking directly up) so that I can accurately represent the spatial orientation of the observer when displaying maps with 3D terrain.

Rationale

I am looking to use MapLibre in an aviation application to implement what is commonly know as "Synthetic Vision" - a feature which provides a virtual "view from the cockpit". To be able to accurately represent the this view, I need to be able to adjust the roll angle of the camera to match the roll angle of the airplane. You can see a similar system at a distinct roll angle here:

https://blog.foreflight.com/wp-content/uploads/2014/12/800x600-refined-terrain.png

From what I can find, the transformCameraUpdate callback currently allows programmatic control of bearing, center, elevation, pitch, and zoom (with pitch limited to 0° - 60°). Perhaps this could be extended to include the roll parameter and to accept pitch values from 0 (straight down) to 180 (straight up).

It might also be useful to allow for all of these parameter to be incorporated into the parameter objects for map.flyTo() and map.easeTo(). The way the Synthetic Vision app works, a server feeds situation data (location, altitude, track, speed, etc.) over a WebSocket at between 10-20 Hz. The handler for the WebSocket moves the camera using a string of very short duration flyTo or easeTo operations. Adding the ability to update situational parameters directly would simplify the process.

Impact

This won't have any real impact on users who only need a basic down-and-forward level view, but it will go a long way towards making MapLibre more useful for certain simulations, games, and situational awareness applications.

[HarelM]

Thanks for taking the time to open this issue.
In general, I think this can be a nice (although niche) addition to this lib.
The main caveat I can think of is which tile to fetch and show.
But feel free to push this forward.

[ssokol]

Thanks!

I'm slowly making my way through the library - long time Javascript user, first time Typescript user. I hope to be able to submit a PR, but it will probably need some help from those more fluent in the language. One question for you...

I haven't been able to find a way to manually set the camera altitude in feet or meters above median sea level. I kind of expected the easeTo and flyTo functions to have an altitude parameter, indicating the camera altitude at the end of the move.

Is there an API call to set the camera altitude independent of the zoom level? I see references to altitude in the camera.ts and map.ts files, but the altitude seems to be an output based on the current zoom level rather than something that can be preset programmatically.

[HarelM]

Zoom level is highly tide to altitude, it has deep historic roots and causes a lot of grief with terrain related stuff.
You can use cameraFromTo I believe to calculate some stuff, IIRC.

[ssokol]

Thanks. I'll look into that. I was just reading over the discussion of deterministic values on #4688 and it seems like the camera controls and associated motion methods may have come from the purely 2D world, where 'zoom' makes far more sense than 'altitude', and where pitch simply isn't a concept.

In testing some ideas I've discovered that the "center" parameter has very different meanings depending on if there is a pitch angle set. With a zero pitch (essentially a 2D map), center places the camera exactly where I expect it to. With a pitch value set, the lib appears to set the "gaze" or "focal point" of the camera at the "center" location, rather than the camera itself. Probably useful for some things, but challenging to use for a PoV application.

I wonder if 3D / terrain apps might be better off with a different camera control model. This would make center, altitude, bearing, pitch, and roll fixed and deterministic - these position the camera. The "gaze" or "focal point" is then determined by the zoom level (and optionally some additional parameters if needed). For my app, zoom will be constant throughout since it emulates what a human would see looking out the front window of an airplane.

Do you think there might be interest in a large bounty (perhaps funded by several participants) for a project that builds out a more 3D-centric "mode" or set of APIs?

[HarelM]

I don't have a good answer to the question about if there is an interest, and if someone would like to pay for it.
One thing to note is that one can switch from 2D to 3D and the expectation is to be at the same "position", so a concept where center means different things in respect to terrain may be problematic.
API-wise, I think Cesium has implemented something with unreal engine that allows looking from the ground up, might be interesting to look at what they did.
The concept of zoom is very much coupled to level of details (vector rendering) so that needs to be taken into account as well...

[ssokol]

My company would be willing to pay a bounty for a set of enhancements that make the 3D behavior meet my our case. I can see if anyone else in my industry is interested in participating.

On the center issue.... Is the current behavior (that center = focal point) intentional, inadvertent but accepted, or erroneous?

From what I can tell, giving the map a pitch angle and a center value positions the camera such that it is "looking at" the center rather than being positioned directly on the center. The actual camera position appears to be determined by some combination of the zoom value and the pitch angle.

If that's a behavior other users rely on, I wouldn't want to break it but would suggest creating a new cameraPoint parameter which is deterministic. If you set the cameraPoint, the lib calculates the focal point based on pitch and zoom. If you set the center, the lib uses the current behavior and determines the camera placement based on the center, pitch, and zoom.

Does that sound reasonable, or would it be better to change center control camera placement?

[HarelM]

Regarding the bounty, I would suggest to advertise it in our slack channel, as not many people follow the github issues.
In order to avoid breaking changes a parameter can be added to the map initialization to define a different camera behavior, something like immersive: true or something similar, once this is set thing will behave differently.
I think @cigone-openindoor had similar requirements.

• 360 immersive #1136

[ssokol]

Working on a proposal. Here is a link to the Google Doc for anyone who is interested:

https://docs.google.com/document/d/1ulpxmkuYbLlslq0mv1rUHCLCgN2ytKWWc-Iv5GPhFgA/edit?usp=sharing

[HarelM]

I've read the doc, thanks for sharing!
It would be great if someone could join the next monthly meeting to talk about it.
It's worth noting that the concept of zoom is integrated deeply with the style specification and styling in general when it comes to vector tiles, so I don't see a way to work around that. Having said that, I know it has its limitations when it comes to terrain and below sea level stuff, panning on heavy terrain scene etc...

Another approach would be to see which parts are align with current mapping needs and which parts may be exposed using some APIs and events to allow a plugin architecture for this specific case.
If the plugin is widely used and there is a demand to incorporate features from it into MapLibre we can always consider this in the future and allow faster implementation outside the main code base.
These are just initial thoughts.

[ssokol]

You're very welcome.

I would love to attend the next meeting but unfortunately I will be out on vacation on the second Wednesday of October. I can try to join, but chances are good that it won't work as I will be in the mountains of west Texas. I'm more than happy to provide any additional information that would help the discussion.

I suspected that zoom is very much a core concept since it's a core element of the tile path. In playing with the current library, I don't think that the "accuracy" or "verisimilitude" of the altitude is all that critical. Humans - even pilots - live in a 2D world: nobody knows exactly what things look like from a specific altitude. The important thing is relative altitude: the location of the camera relative to the surrounding terrain, ground placed objects (runways in my case), and sprites (other aircraft). The exact size of roadways and trees doesn't matter for my application.

I suspect someone more familiar with the library that I could readily come up with a heuristic that takes an MSL altitude (i.e. 392 meters above sea level) and translates that into a zoom level given the current field of view setting. I've tried to do something like that in my proof of concept and it almost works. If the camera placement issue was solved (i.e. the center and pitch angle determine the camera placement rather than the focal point) it probably would work.

In terms of the approach taken to building out the functionality - the plugin vs. core modifications - is above my pay grade. The plug-in route sounds quicker in the short term, but I don't want to end up with the plug-ins falling out of sync with the library over time.

Please let me know if you see anything in the document that seems unrealistic or too far out of scope for MapLibre. I don't want to do anything that would push the project off of its core mission.

[HarelM]

It's currently a very high-level definitions, I'll need to see a software design document to be able to say if something is relevant to be in the core code vs new API vs plugin.
In general, we try and make the public API stable so that people will know what breaks and when (major releases etc).
But it feels a bit early to talk about it. Let's see how this progresses first.

[NathanMOlson]

@ssokol @HarelM @Samarth1696

The camera rotation MapLibre is currently defined by 2 Euler angles, bearing and pitch. To define an arbitrary camera rotation, we need 3 Euler angles (or an alternate rotation representation such as quaternions). To implement this proposed feature, we need to make an early decision on how to represent the camera rotation .

Here is how the camera rotation is currently achieved:

1. Point the camera straight down, North up.
2. Rotate the camera about its Z axis by bearing.
3. Rotate the camera about its X axis by pitch.

Assuming we stick with Euler angles (which I think we definitely should for the public API, but maybe not for the internals), the two options we have to define the third Euler angle are

4a. Rotate the camera about its Z axis by roll (this puts the singularity at pitch == 0, camera pointed straight down).

or

4b. Rotate the camera about its Y axis by roll (this puts the singularity at pitch== 90, camera pointed at the horizon).

With either rotation representation, we can express any arbitrary camera rotation. Since the camera is virtual, the Euler angle singularity is just a mathematical issue, not a physical one. It will only cause problems when doing things like linear interpolation of Euler angles near the singularity (we currently do linear interpolation of Euler angles in easeTo() and flyTo()).

Option 4a is a better fit for @ssokol's use case. It is closer to the Euler angles traditionally used for aircraft, and puts the singularity at nadir, where @ssokol is uninterested. It also offers a more intuitive expression of the camera rotation, and would be a simpler extension to the existing code.

Option 4b would be a better fit for something like this: #1980. It would also keep the singularity away from nadir. Nadir feels like an strange choice for the Euler angle singularity in a mapping library that has traditionally been geared toward nadir maps.

Regardless of which choice we make, I think easeTo() and flyTo() will have to be re-written to avoid interpolation of Euler angles, in order to be completely generic.

IMO Option 4a is a better choice overall. I'd love to hear others' thoughts on this.

[ssokol]

As you predicted, 4a works better for me. In my use case, if the camera is pointed at nadir, the user has more important things to look at than the screen. (Though there are aerobatic maneuvers that can result in a straight down pitch attitude - briefly.)

Our current attitude and heading subsystem are geared for Euler angles, so that would be the easiest approach. Our AHRS internally uses quaternions but the output to display systems is just Eulers plus a few other values (rate of turn, slip/skid, g-load).

Do we take a chance on messing other users up by revising the functionality of easeTo() / flyTo()? Or would it just be the internals that change with the behavior remaining consistent?+