Skip to content

scope should use rounding rather than truncating when computing pixel positions #343

@programmerjake

Description

@programmerjake

x = (int) (rect.width *(1+xa)*.499);
y = (int) (rect.height*(1-ya)*.499);
} else {
double gridPx = calc2dGridPx(rect.width, rect.height);
x=(int)(rect.width*.499+(v/plots.get(0).manScale)*gridPx+gridPx*manDivisions*(double)(plots.get(0).manVPosition)/(double)(V_POSITION_STEPS));
y=(int)(rect.height*.499-(yval/plots.get(1).manScale)*gridPx-gridPx*manDivisions*(double)(plots.get(1).manVPosition)/(double)(V_POSITION_STEPS));

using truncating conversion instead of rounding (using Math.round or similar) causes there to be twice as many numbers that convert to 0 compared to numbers that convert to any other integer, causing a flat spot in the graph when crossing the x axis.

with (int)pixels_as_double every value in (-1, 1) converts to 0 but values that convert to 1 are just [1, 2) which is half the size.
with Math.round every value in (-0.5, 0.5) converts to 0 which is basically the same size as those converting to 1 which is [0.5,1.5)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions