Skip to content

feat: allow to download results from all pages#787

Open
anesson-cs wants to merge 24 commits into
developfrom
search-results-pagination-generator
Open

feat: allow to download results from all pages#787
anesson-cs wants to merge 24 commits into
developfrom
search-results-pagination-generator

Conversation

@anesson-cs

@anesson-cs anesson-cs commented Aug 9, 2023 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Fixes #133

After searching some products, we can now download the ones from all pages of this search by setting exhaust to True in download_all() method parameters (False by default).

It works by saving some elements in several objects:

  • search params in SearchResults object to search next pages of the search
  • list of crunchers in SearchResults object to apply them on next pages and to skip a crunch if it has already been applied on the object
  • list of search params in EODataAccessGateway object to skip a page when it has already been requested

Then, the search_iter_page() is called to search products from all pages by skipping the one already found.
Finally, if there are other products, they are added to the initial SearchResults object.

@github-actions

github-actions Bot commented Aug 9, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Test Results

    4 files  ± 0      4 suites  ±0   6m 15s ⏱️ +4s
  558 tests + 3    555 ✅ + 3    3 💤 ±0  0 ❌ ±0 
2 232 runs  +12  2 130 ✅ +12  102 💤 ±0  0 ❌ ±0 

Results for commit 555375b. ± Comparison against base commit fad1677.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Aug 9, 2023

Copy link
Copy Markdown
Contributor

Code Coverage (Ubuntu)

File Coverage
All files 81% ✅

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against 480239a

@github-actions

github-actions Bot commented Aug 9, 2023

Copy link
Copy Markdown
Contributor

Code Coverage (Windows)

File Coverage
All files 76% ✅

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against 480239a

@sbrunato
sbrunato marked this pull request as draft January 24, 2024 14:01
@anesson-cs
anesson-cs force-pushed the search-results-pagination-generator branch from 480239a to 654017c Compare July 11, 2024 15:52
@github-actions

github-actions Bot commented Jul 11, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

badge

Code Coverage (Ubuntu)

Details 
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         303      50  83.50%   654-700, 802-853, 857
config.py                                      367      27  92.64%   82-84, 93, 101, 105-107, 175, 186, 452-454, 514-517, 560-561, 570-571, 676, 739-744, 746
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    764      74  90.31%   626, 670-673, 711, 746, 776, 823, 827-832, 858, 949, 1017, 1147, 1230-1242, 1278, 1280, 1316, 1320-1331, 1344-1350, 1433-1436, 1469-1489, 1541, 1558-1561, 1573-1576, 1598-1605, 1919, 1954-1960, 2213, 2217-2220, 2231-2233, 2265
api/search_result.py                            64       8  87.50%   69-72, 95, 104, 111, 125
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                675      81  88.00%   129-131, 226, 258-259, 305-306, 316-328, 330, 341, 347-359, 404-405, 442, 463-466, 489, 497-498, 574-575, 599-600, 606-609, 624-625, 774, 820, 991-996, 1123, 1137-1157, 1177, 1182, 1289, 1311, 1325, 1338-1357, 1396, 1448, 1480-1484, 1499
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             130      12  90.77%   104-109, 159, 197, 219, 223, 247, 281-282
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91       8  91.21%   150-152, 199-200, 226-228
plugins/apis/usgs.py                           180      31  82.78%   133, 235, 269, 304-306, 311, 337-338, 343, 373-380, 391-396, 418-424, 426-432, 456
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   34, 47
plugins/authentication/generic.py               14       2  85.71%   40, 50
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   132, 156-161
plugins/authentication/oauth.py                 13       7  46.15%   32-34, 38-41
plugins/authentication/openid_connect.py       183      17  90.71%   119, 133-158, 166, 320-323, 347
plugins/authentication/qsauth.py                34       1  97.06%   83
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 88      16  81.82%   79, 107, 109, 131-143, 198-202
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          19       5  73.68%   38, 44-46, 58
plugins/crunch/filter_date.py                   59      14  76.27%   49-54, 66, 75, 84, 87, 99-101, 110-112, 119
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   51-52, 68, 77-80, 82, 89-92
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 68-71, 78-81, 87, 95, 106-122
plugins/crunch/filter_property.py               30       7  76.67%   54-59, 62-63, 79-83
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        489     163  66.67%   266, 279, 346-349, 363-367, 409-411, 415, 447-448, 454-458, 487, 519, 523, 530, 560-568, 572, 604-612, 619-621, 655-729, 747-808, 819-824, 836-849, 874, 889-891, 894, 904-912, 920-933, 943-974, 981-993, 1031, 1057, 1102-1104, 1325
plugins/download/base.py                       275      51  81.45%   140, 168, 300-301, 318-324, 355-359, 365-366, 408, 411-425, 437, 441, 553-557, 587-588, 596-613, 620-628, 630-634, 677, 699, 721, 729
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       534     129  75.84%   196-208, 210-211, 243-246, 307-310, 312-313, 320-325, 343-358, 375-377, 389, 437, 444-450, 468, 482, 496, 504-506, 522-527, 538, 556, 598-602, 624, 664, 709, 723-729, 758-822, 840, 873-882, 904-905, 932-937, 943, 946, 962, 979-980, 1010-1011, 1018, 1079-1085, 1140-1141, 1147, 1157, 1193, 1229, 1247-1263, 1289-1291
plugins/download/s3rest.py                     116      24  79.31%   113, 149, 156, 191, 221-228, 231-233, 237, 248-254, 262-263, 266-270, 293, 314-317
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         128       9  92.97%   102, 106, 117, 255, 275, 331-332, 352, 361
plugins/search/build_search_result.py          181      23  87.29%   95, 136-137, 143, 154, 288-291, 320, 373-390, 450, 453, 463, 480, 508, 510
plugins/search/cop_marine.py                   197      47  76.14%   55, 63-65, 71-72, 88, 90, 93, 128-130, 142-143, 183-192, 196, 199, 203, 221, 249, 253, 268, 272, 276, 280, 284-288, 294-297, 300-314, 331, 354, 357, 363
plugins/search/creodias_s3.py                   55       3  94.55%   53, 71, 105
plugins/search/csw.py                          105      81  22.86%   58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          200      67  66.50%   90-93, 109, 120, 124-125, 136, 141, 146, 153, 166-169, 223-224, 228, 238-244, 249, 275-278, 286-297, 314, 316, 323-324, 326-327, 345-349, 382, 392, 403, 416, 422-437, 442
plugins/search/qssearch.py                     666      81  87.84%   388, 392-398, 406-407, 513-519, 565, 581, 591, 618, 620, 662-665, 739-740, 788, 807, 814, 826, 883, 904, 907-908, 917-918, 927-928, 937-938, 965, 1036-1041, 1045-1054, 1088, 1110, 1170, 1219, 1293-1297, 1357, 1360, 1366-1367, 1388, 1415-1427, 1434, 1466-1468, 1478-1484, 1514, 1537, 1552, 1574, 1681-1691
plugins/search/static_stac_search.py            72      10  86.11%   98-125, 141, 154
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  26       0  100.00%
rest/constants.py                                7       0  100.00%
rest/core.py                                   234      26  88.89%   269, 355, 454, 670, 677-725
rest/server.py                                 283      53  81.27%   108, 131-133, 246-248, 304-305, 317-333, 425-430, 458, 623-630, 659, 703-704, 727, 798-800, 817-822, 851, 853, 857-858, 862-863
rest/stac.py                                   464     120  74.14%   303, 325, 373-376, 400-426, 431-437, 457-459, 482, 514-515, 543, 556, 597-637, 659-675, 749-762, 769, 823-824, 885, 950-952, 1156, 1164-1176, 1187-1209, 1220-1265, 1424-1425
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     179       5  97.21%   232-236, 289, 292, 360
rest/types/queryables.py                        56       1  98.21%   164
rest/types/stac_search.py                      126       7  94.44%   129, 175, 190-192, 200, 204
rest/utils/__init__.py                          93      12  87.10%   108-109, 128-130, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              114      14  87.72%   53, 70, 129-132, 199, 213-222, 232, 253, 266
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              500      37  92.60%   85, 90, 194-195, 204-231, 234, 248, 328-332, 406-410, 429-431, 510, 525, 561-562, 926-929, 937-938, 976-977, 1148
utils/constraints.py                           119      37  68.91%   89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             35       1  97.14%   93
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                29       1  96.55%   121
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 88, 90, 92, 94, 96, 112, 120-122, 130
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9604    1632  83.01%

Diff against develop

Filename                    Stmts    Miss  Cover
------------------------  -------  ------  --------
api/core.py                    +6       0  +0.07%
api/search_result.py          +13      +4  -4.66%
plugins/crunch/base.py         +9      +4  -16.32%
plugins/download/base.py      +22       0  +1.61%
types/download_args.py         +1       0  +100.00%
TOTAL                         +51      +8  +0.01%

Results for commit: 555375b

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@github-actions

github-actions Bot commented Jul 11, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

badge

Code Coverage (Windows)

Details 
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         303      50  83.50%   654-700, 802-853, 857
config.py                                      367      28  92.37%   82-84, 93, 101, 105-107, 175, 186, 452-454, 514-517, 560-561, 570-571, 676, 707, 739-744, 746
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    764      74  90.31%   626, 670-673, 711, 746, 776, 823, 827-832, 858, 949, 1017, 1147, 1230-1242, 1278, 1280, 1316, 1320-1331, 1344-1350, 1433-1436, 1469-1489, 1541, 1558-1561, 1573-1576, 1598-1605, 1919, 1954-1960, 2213, 2217-2220, 2231-2233, 2265
api/search_result.py                            64       8  87.50%   69-72, 95, 104, 111, 125
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                675      81  88.00%   129-131, 226, 258-259, 305-306, 316-328, 330, 341, 347-359, 404-405, 442, 463-466, 489, 497-498, 574-575, 599-600, 606-609, 624-625, 774, 820, 991-996, 1123, 1137-1157, 1177, 1182, 1289, 1311, 1325, 1338-1357, 1396, 1448, 1480-1484, 1499
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       3  85.71%   48, 55, 68
plugins/manager.py                             130      12  90.77%   104-109, 159, 197, 219, 223, 247, 281-282
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91       8  91.21%   150-152, 199-200, 226-228
plugins/apis/usgs.py                           180      31  82.78%   133, 235, 269, 304-306, 311, 337-338, 343, 373-380, 391-396, 418-424, 426-432, 456
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   34, 47
plugins/authentication/generic.py               14       2  85.71%   40, 50
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   132, 156-161
plugins/authentication/oauth.py                 13       7  46.15%   32-34, 38-41
plugins/authentication/openid_connect.py       183      17  90.71%   119, 133-158, 166, 320-323, 347
plugins/authentication/qsauth.py                34       1  97.06%   83
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 88      16  81.82%   79, 107, 109, 131-143, 198-202
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          19       5  73.68%   38, 44-46, 58
plugins/crunch/filter_date.py                   59      14  76.27%   49-54, 66, 75, 84, 87, 99-101, 110-112, 119
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   48-53, 66-111
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 68-71, 78-81, 87, 95, 106-122
plugins/crunch/filter_property.py               30       7  76.67%   54-59, 62-63, 79-83
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        489     163  66.67%   266, 279, 346-349, 363-367, 409-411, 415, 447-448, 454-458, 487, 519, 523, 530, 560-568, 572, 604-612, 619-621, 655-729, 747-808, 819-824, 836-849, 874, 889-891, 894, 904-912, 920-933, 943-974, 981-993, 1031, 1057, 1102-1104, 1325
plugins/download/base.py                       275      53  80.73%   140, 168, 235-237, 300-301, 318-324, 355-359, 365-366, 408, 411-425, 437, 441, 553-557, 587-588, 596-613, 620-628, 630-634, 677, 699, 721, 729
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       534     130  75.66%   196-208, 210-211, 243-246, 307-310, 312-313, 320-325, 343-358, 375-377, 389, 437, 444-450, 468, 482, 496, 504-506, 522-527, 538, 556, 598-602, 624, 664, 709, 723-729, 758-822, 840, 873-882, 904-905, 932-937, 943, 946, 962, 979-980, 993, 1010-1011, 1018, 1079-1085, 1140-1141, 1147, 1157, 1193, 1229, 1247-1263, 1289-1291
plugins/download/s3rest.py                     116      24  79.31%   113, 149, 156, 191, 221-228, 231-233, 237, 248-254, 262-263, 266-270, 293, 314-317
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         128      14  89.06%   102, 106, 117, 255, 275, 331-332, 352, 355-363, 365
plugins/search/build_search_result.py          181      30  83.43%   95, 136-137, 143, 154, 288-291, 320, 373-390, 450, 453, 463, 480, 500-515
plugins/search/cop_marine.py                   197      47  76.14%   55, 63-65, 71-72, 88, 90, 93, 128-130, 142-143, 183-192, 196, 199, 203, 221, 249, 253, 268, 272, 276, 280, 284-288, 294-297, 300-314, 331, 354, 357, 363
plugins/search/creodias_s3.py                   55       3  94.55%   53, 71, 105
plugins/search/csw.py                          105      81  22.86%   58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          200      67  66.50%   90-93, 109, 120, 124-125, 136, 141, 146, 153, 166-169, 223-224, 228, 238-244, 249, 275-278, 286-297, 314, 316, 323-324, 326-327, 345-349, 382, 392, 403, 416, 422-437, 442
plugins/search/qssearch.py                     666     109  83.63%   388, 392-398, 406-407, 513-519, 565, 568, 581, 591, 610-625, 662-665, 739-740, 788, 807, 814, 826, 883, 904, 907-908, 917-918, 927-928, 937-938, 965, 1036-1041, 1045-1054, 1088, 1110, 1170, 1219, 1293-1297, 1357, 1360, 1366-1367, 1388, 1415-1427, 1434, 1466-1468, 1478-1484, 1514, 1537, 1552, 1574, 1640-1711
plugins/search/static_stac_search.py            72      10  86.11%   98-125, 141, 154
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  26       4  84.62%   34-36, 68
rest/constants.py                                7       0  100.00%
rest/core.py                                   234     149  36.32%   160-241, 260-314, 327-365, 396-428, 443-459, 477-488, 497-533, 550, 592-631, 670, 677-725
rest/server.py                                 283     283  0.00%    18-874
rest/stac.py                                   464     393  15.30%   123-129, 136-161, 187-192, 219, 236-359, 369-438, 453-492, 505-558, 566-589, 597-637, 659-675, 685, 699-702, 718-788, 804-840, 863-888, 895-911, 918-923, 932-959, 968-970, 977-979, 991-993, 1005-1022, 1030-1051, 1059-1081, 1088-1105, 1124-1147, 1156, 1164-1176, 1187-1209, 1220-1265, 1272-1289, 1297-1455
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     179      15  91.62%   122, 232-236, 269-271, 289, 292, 298, 302, 360, 372-375
rest/types/queryables.py                        56      13  76.79%   51-52, 59-60, 67-68, 94-99, 108-109, 164
rest/types/stac_search.py                      126      18  85.71%   127-129, 153-154, 159-160, 175, 190-192, 200, 204, 251-256
rest/utils/__init__.py                          93      30  67.74%   79-85, 105, 108-109, 128-130, 143, 150, 175-183, 190-211
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       6  73.91%   40, 43-44, 48, 60, 62
types/__init__.py                              114      39  65.79%   53, 66-70, 81-93, 120-122, 129-132, 172, 199, 209-225, 230, 232, 253, 258, 266, 276
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              500      39  92.20%   85, 90, 194-195, 204-231, 234, 248, 328-332, 406-410, 429-431, 510, 525, 561-562, 926-929, 937-938, 976-977, 1006, 1148, 1309
utils/constraints.py                           119      37  68.91%   89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             35       1  97.14%   93
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                29       1  96.55%   121
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 88, 90, 92, 94, 96, 112, 120-122, 130
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9604    2428  74.72%

Diff against develop

Filename                    Stmts    Miss  Cover
------------------------  -------  ------  --------
api/core.py                    +6       0  +0.07%
api/search_result.py          +13      +4  -4.66%
plugins/crunch/base.py         +9      +4  -16.32%
plugins/download/base.py      +22       0  +1.68%
types/download_args.py         +1       0  +100.00%
TOTAL                         +51      +8  +0.05%

Results for commit: 555375b

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@anesson-cs
anesson-cs force-pushed the search-results-pagination-generator branch 2 times, most recently from 9c58e01 to e08e53c Compare July 16, 2024 08:46
@anesson-cs
anesson-cs marked this pull request as ready for review July 16, 2024 08:51
@anesson-cs
anesson-cs force-pushed the search-results-pagination-generator branch from e08e53c to 3f5d1a1 Compare July 24, 2024 15:51
anesson-cs added 18 commits August 2, 2024 17:55
@anesson-cs
feat: allow to download results from all pages
6e47b95
@anesson-cs
fix: add 'exhaust' parameter to download configuration file
b478bac
@anesson-cs
fix: add docstrings about 'exhaust' parameter to 'Api' and 'Download'…
4d5b90f 
… classes
@anesson-cs
refactor: shorten other products to download initialization
3a68f6a 
'SearchResult' class could not be imported before the rebase due to a circular import
@anesson-cs
fix: remove a log and an authentication not right anymore
2588ce0
@anesson-cs
fix: transform search kwargs record into a 'SearchResults' instance f…
166a0c6 
…rom download_all() input products
@anesson-cs
fix: change the way parameters are removed before search
9f8d382
@anesson-cs
fix: remove comment not useful anymore
befdc4e
@anesson-cs
fix: add the provider of products in search arguments
f44c7e1
@anesson-cs
fix: add clear() method to 'SearchResult' class
3e13b59
@anesson-cs
refactor: optimize the way crunchers are added to 'SearchResult' inst…
2ff912b 
…ances
@anesson-cs
fix: add comments and logs before and after applying crunchers to new…
66f428e 
… results
@anesson-cs
fix: change 'SearchResult' class docstrings
6e64115
@anesson-cs
fix: add search kwargs lost due to rebase
123899b
@anesson-cs
fix: use f-strings instead of percentages
26f7108
@anesson-cs
fix: add a log for crunched results
bddbba3
@anesson-cs
fix: change spelling of first download_all() logs
469ef03
@anesson-cs
fix: make tests using 'Crunch' class works
2c615dc
@anesson-cs
anesson-cs force-pushed the search-results-pagination-generator branch from 3f5d1a1 to 5bedf51 Compare August 5, 2024 15:27
@anesson-cs
anesson-cs force-pushed the search-results-pagination-generator branch from 5bedf51 to 555375b Compare August 6, 2024 08:31
@sbrunato sbrunato mentioned this pull request Jul 2, 2025
Closed
@sbrunato sbrunato modified the milestone: 4.0.0.dev Jul 2, 2025
@sbrunato

sbrunato commented Oct 31, 2025

Copy link
Copy Markdown
Collaborator

Following #1745, this can now be updated to use SearchResult.next_page()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@anesson-cs anesson-cs

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

handle search results pagination with a generator

2 participants

@anesson-cs @sbrunato