Qgov master 2.11.2 cicd uplift etag

[duttonw]

Fixes ckan#6955

Proposed fixes:

Introduce ETag response and ability to respond that data has not changed
Introduce ability to disable cache: private, and set no cache headers enforcement (in situations where Vary: Cookie fails)

Features:

• includes tests covering changes
• includes updated documentation
• includes user-visible changes
• includes API changes
• includes bugfix for possible backport

Please [X] all the boxes above that apply

[codecov]

❌ 3 Tests Failed:

Tests completed	Failed	Passed	Skipped
3033	3	3030	3

View the top 3 failed test(s) by shortest run time

ckan/tests/views/test_cache.py::test_sets_private_cache_when_no_private_cache_env_present

Stack Traces | 0.931s run time

app = <ckan.tests.helpers.CKANTestApp object at 0x7fa28b1ee160>

    @pytest.mark.ckan_config("ckan.cache_expires", 7200)
    def test_sets_private_cache_when_no_private_cache_env_present(app: CKANTestApp):
        """Test that private cache is set when `__no_private_cache__` is not present but `__no_cache__` is present."""
        builder = EnvironBuilder(path='/', method='GET', headers={}, environ_overrides={
            "__no_cache__": True})
        env = builder.get_environ()
        Request(env)
        response = Response()  # dummy response
    
        with app.flask_app.request_context(env):  # only works if you have app.flask_app
            updated_response = views.set_cache_control_headers_for_response(response)
>       assert updated_response.cache_control.private is True
E       AssertionError: assert '*' is True
E        +  where '*' = <ResponseCacheControl private=None>.private
E        +    where <ResponseCacheControl private=None> = <Response 0 bytes [200 OK]>.cache_control

.../tests/views/test_cache.py:97: AssertionError

ckan/tests/views/test_cache.py::test_disables_cache_when_no_cache_env_present

Stack Traces | 0.936s run time

app = <ckan.tests.helpers.CKANTestApp object at 0x7f627ddc2ca0>

    @pytest.mark.ckan_config("ckan.cache_expires", 0)
    def test_disables_cache_when_no_cache_env_present(app: CKANTestApp):
        """Test that no-cache headers are set when `__no_cache__` is true and `__no_private_cache__` is true in the environment."""
        builder = EnvironBuilder(path='/', method='GET', headers={}, environ_overrides={
            "__no_cache__": True,
            '__no_private_cache__': True})
        env = builder.get_environ()
        Request(env)
        response = Response()  # dummy response
    
        with app.flask_app.request_context(env):  # only works if you have app.flask_app
            updated_response = views.set_cache_control_headers_for_response(response)
>       assert updated_response.cache_control.no_cache is True
E       AssertionError: assert '*' is True
E        +  where '*' = <ResponseCacheControl max-age='0' must-revalidate=None no-cache=None no-store=None>.no_cache
E        +    where <ResponseCacheControl max-age='0' must-revalidate=None no-cache=None no-store=None> = <Response 0 bytes [200 OK]>.cache_control

.../tests/views/test_cache.py:64: AssertionError

ckan/tests/views/test_cache.py::test_sets_private_cache_when___no_cache__is_set_and_no_private_cache_env_present

Stack Traces | 0.96s run time

app = <ckan.tests.helpers.CKANTestApp object at 0x7fdc71b57d00>

    @pytest.mark.ckan_config("ckan.cache_expires", 7200)
    def test_sets_private_cache_when___no_cache__is_set_and_no_private_cache_env_present(app: CKANTestApp):
        """Test that private cache is set when `__no_private_cache__` is absent but `__no_cache__` is present."""
        builder = EnvironBuilder(path='/', method='GET', headers={}, environ_overrides={
            "__no_cache__": True})
        env = builder.get_environ()
        Request(env)
        response = Response()  # dummy response
    
        with app.flask_app.request_context(env):  # only works if you have app.flask_app
            updated_response = views.set_cache_control_headers_for_response(response)
    
>       assert updated_response.cache_control.private is True
E       AssertionError: assert '*' is True
E        +  where '*' = <ResponseCacheControl private=None>.private
E        +    where <ResponseCacheControl private=None> = <Response 0 bytes [200 OK]>.cache_control

.../tests/views/test_cache.py:82: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[duttonw]

This is 2 commits on top of #217 which has work already included in upstream master

[duttonw]

backport from ckan upstream master: ckan#8781
need to fix a test that was highlighted that something was done incorrectly on enforcing attachments to only be downloaded except for pdf's which can be opened in the browser.

[ThrawnCA]

This comment is exactly the same as the previous test. It should say something more specific to this test.

[ThrawnCA]

@duttonw Can you uplift this to target 2.11.4?

[duttonw]

i'm going to need to redo this entire workload since upstream did not like the direction i took it :/