Filter distance fix for PSU Walk Study

[nmcneil]

Update study configuration so that filter distance defaults to "-1" (or equivalent fix to address intermittent data collection).

[JGreenlee]

Config changes:

• ✨ define deployment-level overrides on native configs: tracking, sync op-deployment-configs#166

Native code changes:

• Support deployment overrides to tracking config; Add is_fleet e-mission-data-collection#249

Phone changes:

• Rework Config Edits & UI improvements e-mission-phone#1208

[JGreenlee]

The changes above will have the following effects:

• The config for walk-study-psu now has

  "tracking": {
      "filter_distance": -1
  },

• The native code reads properties in tracking as overrides to the default config, so the distance filter gets turned off for walk-study-psu users, while all other deployments keep using the defaults (-1 for android, 1 meter for iOS)

• The UI for users to edit their own tracking config (Profile > Developer Zone) has been improved. However, the fields for deployment-level overrides are disabled and cannot be edited by users

Further deployment-level overrides to the tracking config could be added and applied the same way

[shankari]

These changes are now on staging. I am recording my testing here.

✅ ensure no regressions for other programs

Test	android pixel 4	iPhone
upgraded app to 2.0.1	✅	✅
did not change program (logged in to stage-program and stage-time-use)	✅	✅
app launched properly	✅	✅
diary was visible properly	✅	✅
developer zone settings were consistent with old defaults	✅	✅
new trip was recorded properly	✅	✅

Settings screenshots

iOS	android

Switch to the walk study by logging out and logging in with a new opcode

Test	android pixel 4	iPhone
app launched properly	✅	✅
diary was visible properly	✅	✅
surveys were launched properly	✅	✅
new trip was recorded properly	✅	✅
developer zone settings were consistent with deployment values	NA	❌

@JGreenlee on iOS, I am still seeing the filter distance as 1 (see screenshot), even though it is grayed out because it is set in the deployment.

Uninstall + reinstall and then login with the walk study

I'm going to uninstall + reinstall instead of logout + login. Ah that worked!

After logout from a regular study + login to PSU	After uninstall from a regular study + install and login to PSU

@nmcneil I think this should be sufficient for your immediate needs, since your users will not have the app installed with a prior study, but will be doing a fresh install

@JGreenlee as part of polishing, we should think about the expected behavior in the logout case; maybe the easiest option is to delete the sensor config when logging out

[JGreenlee]

I can confirm that I saw something similar in my testing this weekend. Logout/login did not work, but uninstall/reinstall did.

I looked at logs this morning to better understand why.
On log out, we do 3 things: i) clear local storage, ii) clear user cache (via BEMUserCache.clearAll()), iii) refresh the UI
Since we clear the user cache, there shouldn't be a stored sensor config left over from a previous opcode

from my logs:

in readFromCache, cached tracking config = (null)
in getDeploymentConfig, deployment config = {
    "admin_dashboard" =     {
        "additional_trip_columns" =         (
        );
        "data_trips" = 1;
        "data_trips_columns_exclude" =         (
        );
        "data_uuids" = 1;
        "data_uuids_columns_exclude" =         (
        );
        "map_bubble" = 1;
        "map_heatmap" = 1;
        "map_trip_lines" = 1;
        "options_emails" = 1;
        "options_uuids" = 1;
        "overview_active_users" = 1;
        "overview_signup_trends" = 1;
        "overview_trips" = 1;
        "overview_trips_trend" = 1;
        "overview_users" = 1;
        "push_send" = 1;
        "token_generate" = 1;
        "token_prefix" = nrelop;
    };
    "display_config" =     {
        "use_imperial" = 0;
    };
    intro =     {
        "deployment_partner_name" = "National Renewable Energy Laboratory (NREL)";
        "mode_studied" = "e-bike";
        "program_admin_contact" = "K. Shankari (k.shankari@nrel.gov)";
        "program_or_study" = program;
        "start_month" = 9;
        "start_year" = 2020;
        "translated_text" =         {
            en =             {
                "deployment_name" = "Staging environment for testing programs only";
                "deployment_partner_name" = "National Renewable Energy Laboratory (NREL)";
                "research_questions" =                 (
                    "Comparing the open-access group (no intervention) and programs that provide interventions.",
                    "General monitoring of mobility trends for travel behavior, including shifts to travel behavior and the introduction of new mobility modes."
                );
                "short_textual_description" = "Dev-ops best practices involve setting up a staging environment to test changes in the real world before they are pushed out to production. This environment is designed for use with staging or beta versions of the OpenPATH. NREL makes no guarantees about data retention on this environment. Please use the Open Access environment to understand your carbon footprint.";
                "summary_line_1" = "allows real-world testing of the app in study mode";
                "summary_line_2" = "before pushing changes to production";
                "summary_line_3" = "without any guarantees about retaining data";
                "why_we_collect" = "To ensure that the end-to-end system works properly on a variety of real-world phones";
            };
            es =             {
                "deployment_name" = "Entorno de ensayo solo para programas de prueba";
                "deployment_partner_name" = "Laboratorio Nacional de Energ\U00edas Renovables (NREL)";
                "research_questions" =                 (
                    "Comparando el grupo de acceso abierto (sin intervenci\U00f3n) y los programas que brindan intervenciones.",
                    "Monitoreo general de las tendencias de movilidad para el comportamiento de viaje, incluidos los cambios en el comportamiento de viaje y la introducci\U00f3n de nuevos modos de movilidad."
                );
                "short_textual_description" = "Las mejores pr\U00e1cticas de Dev-ops implican configurar un entorno de prueba para probar los cambios en el mundo real antes de que se env\U00eden a producci\U00f3n. Este entorno est\U00e1 dise\U00f1ado para usarse con versiones provisionales o beta de OpenPATH. NREL no garantiza la retenci\U00f3n de datos en este entorno. Utilice el entorno de acceso abierto para comprender su huella de carbono.";
                "summary_line_1" = "permite realizar pruebas en el mundo real de la aplicaci\U00f3n en modo de programa";
                "summary_line_2" = "antes de impulsar cambios en la producci\U00f3n";
                "summary_line_3" = "sin ninguna garant\U00eda sobre la retenci\U00f3n de datos";
                "why_we_collect" = "Para garantizar que el sistema de extremo a extremo funcione correctamente en una variedad de tel\U00e9fonos del mundo real";
                "why_we_collect_es" = "Para garantizar que el sistema de extremo a extremo funcione correctamente en una variedad de tel\U00e9fonos del mundo real";
            };
        };
    };
    joined =     {
        opcode = "[redacted]";
        "study_name" = "stage-program";
    };
    metrics =     {
        "include_test_users" = 1;
    };
    name = stage;
    "profile_controls" =     {
        "support_upload" = 0;
        "trip_end_notification" = 0;
    };
    "push_notifications" =     {
        message =         {
            en = "Custom push notification message from the dynamic config";
        };
        "recent_user_input_threshold" = 7;
        title =         {
            en = "Custom push notification title";
        };
    };
    server =     {
        "aggregate_call_auth" = "user_only";
        connectUrl = "https://openpath-stage.nrel.gov/api/";
    };
    "survey_info" =     {
        surveys =         {
            UserProfileSurvey =             {
                compatibleWith = 1;
                dataKey = "manual/demographic_survey";
                formPath = "json/demo-survey-v2.json";
                labelTemplate =                 {
                    en = Answered;
                    es = Contestada;
                };
                version = 1;
            };
        };
        "trip-labels" = MULTILABEL;
    };
    ts = 1655143472;
    version = 1;
}

That was the only log statement for getDeploymentConfig, meaning it was still using the config for stage (which is what I was on before)
So I think this was just due to the old deployment config being cached in memory

[JGreenlee]

Although we don't expect end users to switch deployments like this, it could present other problems.
When we update the config from the UI, we do something similar to refresh and get rid of cached instances of the old config version. This works for UI, but the native code will still have the old version cached until the app fully restarts

It would be cleanest if we can trigger a restart of the whole app rather than just refreshing the UI.
cordova-diagnostic-plugin's restart with cold=true might work (https://github.com/dpa99c/cordova-diagnostic-plugin/tree/master?tab=readme-ov-file#restart)

[JGreenlee]

Unfortunately, cold restart is only possible on Android and is not allowed on iOS https://stackoverflow.com/a/52278587

Other options:

1. don't cache the deployment config (or anything that depends on it, including the tracking config) in memory
2. cache in memory, but keep track of the places we are doing so and expose methods to programmatically clear them out

(1) may cause slowness since the tracking config is frequently referenced throughout multiple plugins
(2) is probably safer and more principled, but potentially annoying to implement and adhere to

[shankari]

I think we should go with (2), but I agree that it has the risk that we will forget to clear some of the cached objects.

The classic approach to this, IMHO, would be to have an explicit "in-memory cache" data structure that holds all the in-memory objects, have a clear method for it, and then, through code review, ensure that all in-memory config options are stored in the in-memory cache.

That also allows us to potentially monitor the size of the in-memory cache, swap out objects as needed ect.

An alternate approach with similar principles but a different implementation, would be for in-memory cached objects to register themselves with the config manager, and then the config manager can iterate over all of them and clear them out. Again, we would need the enforce the registration through code review.

[JGreenlee]

The version we have been testing includes the changes needed to onboard new users with the distance filter disabled, so we are submitting it to the app stores for release. Review takes a few days but it will hopefully be ready for PSU to test next week.

For testers who already have the app installed, you will need to:

• Update to the new version (which will be 2.0.2 once it is available)
• Click "Refresh App Configuration" (near the bottom of the Profile screen)
• Turn on tracking if you disabled it
• Manually restart the app (swipe OpenPATH away from your recent apps) and open it again

We are going to improve the config update process for existing users in a follow-up release (discussion in #1136)

We will close this issue once version 2.0.2 is published on both app stores.

[shankari]

2.0.2 is published on both app stores