Skip to content

Commit 8bd8530

Browse files
[core] feat: add fake report fetcher
* Update FakeApiClient to support initializating from csv and json files * Add FakeApiReportFetcher class and expose with with `fake` alias * Ensure that default parsers is DictParser for all ApiReportFetchers * Add tests for fake report fetcher and api client * Bump library to version 0.1.0
1 parent 3485717 commit 8bd8530

File tree

13 files changed

+468
-43
lines changed

13 files changed

+468
-43
lines changed

libs/garf_core/README.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,13 @@
88

99
These abstractions are designed to be as modular and simple as possible:
1010

11-
* `BaseApiClient` - an interface for connecting to APIs.
12-
* `BaseQuery` - encapsulates SQL-like request.
11+
* `BaseApiClient` - an interface for connecting to APIs. Check [default implementations](docs/builtin-functionality.md#apiclients)
12+
* `BaseParser` - an interface to parse results from the API. Check [default implementations](docs/builtin-functionality.md#parsers)
13+
* `ApiReportFetcher` - responsible for fetching and parsing data from reporting API. [Default implementations](docs/builtin-functionality.md#apireportfetchers)
14+
1315
* `QuerySpecification` - parsed SQL-query into various elements.
14-
* `BaseParser` - an interface to parse results from the API. Have a couple of default implementations:
15-
* `ListParser` - returns results from API as a raw list.
16-
* `DictParser` - returns results from API as a formatted dict.
17-
* `NumericDictParser` - returns results from API as a formatted dict with converted numeric values.
16+
* `BaseQuery` - protocol for all class based queries.
1817
* `GarfReport` - contains data from API in a format that is easy to write and interact with.
19-
* `ApiReportFetcher` - responsible for fetching and parsing data from reporting API.
2018

2119
## Installation
2220

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# `garf_core` built-in funtionality
2+
3+
Apart from defining such interfaces as `BaseApiClient`, `BaseParser`, `ApiReportFetcher`, etc.
4+
`garf_core` comes with several built-in implementations.
5+
6+
## Parsers
7+
8+
* `ListParser` - returns results from API as a raw list.
9+
* `DictParser` - returns results from API as a formatted dict.
10+
* `NumericDictParser` - returns results from API as a formatted dict with converted numeric values.
11+
12+
## ApiClients
13+
14+
* `FakeApiClient` - initializes API responses based on some predefined set of data. \
15+
You can either pass data to it directly (as a list of dicts) or load them from JSON / CSV file.
16+
17+
```python
18+
from garf_core.api_clients import FakeApiClient
19+
20+
21+
api_client = FakeApiClient(data=[{'field': 'value'}]
22+
api_client = FakeApiClient.from_csv('path/to/csv')
23+
api_client = FakeApiClient.from_json('path/to/json')
24+
```
25+
26+
* `RestApiClient` - gets data from remote / local API endpoint.
27+
```python
28+
from garf_core.api_clients import RestApiClient
29+
30+
31+
api_client = RestApiClient(endpoint='http://localhost:3000/api/resource')
32+
```
33+
34+
## ApiReportFetchers
35+
36+
* `FakeApiReportFetcher` - fetches data based on provided data or load them from csv / json file.
37+
38+
```python
39+
from garf_core.fetchers import FakeApiReportFetcher
40+
41+
42+
fetcher = FakeApiApiReportFetcher(data=[{'field': 'value'}]
43+
fetcher = FakeApiApiReportFetcher.from_csv('path/to/csv')
44+
fetcher = FakeApiApiReportFetcher.from_json('path/to/json')
45+
```
46+
* `RestApiReportFetcher` - fetches data from specified API endpoint.
47+
48+
```python
49+
from garf_core.fetchers import RestApiReportFetcher
50+
51+
52+
fetcher = RestApiApiReportFetcher(endpoint='http://localhost:8000/api/resource')
53+
```

libs/garf_core/garf_core/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,4 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15-
__version__ = '0.0.12'
15+
__version__ = '0.1.0'

libs/garf_core/garf_core/api_clients.py

Lines changed: 88 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,14 @@
1616
from __future__ import annotations
1717

1818
import abc
19+
import contextlib
20+
import csv
1921
import dataclasses
22+
import json
23+
import os
24+
import pathlib
2025
from collections.abc import Sequence
26+
from typing import Any
2127

2228
import requests
2329
from typing_extensions import override
@@ -74,13 +80,93 @@ def get_response(
7480
class FakeApiClient(BaseClient):
7581
"""Fake class for specifying API client."""
7682

77-
def __init__(self, results: Sequence) -> None:
83+
def __init__(self, results: Sequence[dict[str, Any]], **kwargs: str) -> None:
7884
"""Initializes FakeApiClient."""
7985
self.results = list(results)
86+
self.kwargs = kwargs
8087

8188
@override
8289
def get_response(
83-
self, request: GarfApiRequest = GarfApiRequest()
90+
self, request: GarfApiRequest = GarfApiRequest(), **kwargs: str
8491
) -> GarfApiResponse:
8592
del request
8693
return GarfApiResponse(results=self.results)
94+
95+
@classmethod
96+
def from_file(cls, file_location: str | os.PathLike[str]) -> FakeApiClient:
97+
"""Initializes FakeApiClient from json or csv files.
98+
99+
Args:
100+
file_location: Path of file with data.
101+
102+
Returns:
103+
Initialized client.
104+
105+
Raises:
106+
GarfApiError: When file with unsupported extension is provided.
107+
"""
108+
if str(file_location).endswith('.json'):
109+
return FakeApiClient.from_json(file_location)
110+
if str(file_location).endswith('.csv'):
111+
return FakeApiClient.from_csv(file_location)
112+
raise GarfApiError(
113+
'Unsupported file extension, only csv and json are supported.'
114+
)
115+
116+
@classmethod
117+
def from_json(cls, file_location: str | os.PathLike[str]) -> FakeApiClient:
118+
"""Initializes FakeApiClient from json file.
119+
120+
Args:
121+
file_location: Path of file with data.
122+
123+
Returns:
124+
Initialized client.
125+
126+
Raises:
127+
GarfApiError: When file with data not found.
128+
"""
129+
try:
130+
with pathlib.Path.open(file_location, 'r', encoding='utf-8') as f:
131+
data = json.load(f)
132+
return FakeApiClient(data)
133+
except FileNotFoundError as e:
134+
raise GarfApiError(f'Failed to open {file_location}') from e
135+
136+
@classmethod
137+
def from_csv(cls, file_location: str | os.PathLike[str]) -> FakeApiClient:
138+
"""Initializes FakeApiClient from csv file.
139+
140+
Args:
141+
file_location: Path of file with data.
142+
143+
Returns:
144+
Initialized client.
145+
146+
Raises:
147+
GarfApiError: When file with data not found.
148+
"""
149+
try:
150+
with pathlib.Path.open(file_location, 'r', encoding='utf-8') as f:
151+
reader = csv.DictReader(f)
152+
data = []
153+
for row in reader:
154+
data.append(
155+
{key: _field_converter(value) for key, value in row.items()}
156+
)
157+
return FakeApiClient(data)
158+
except FileNotFoundError as e:
159+
raise GarfApiError(f'Failed to open {file_location}') from e
160+
161+
162+
def _field_converter(field: str):
163+
if isinstance(field, str) and (lower_field := field.lower()) in (
164+
'true',
165+
'false',
166+
):
167+
return lower_field == 'true'
168+
with contextlib.suppress(ValueError):
169+
return int(field)
170+
with contextlib.suppress(ValueError):
171+
return float(field)
172+
return field
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Built-in fetchers."""
16+
17+
from garf_core.fetchers.fake import FakeApiReportFetcher
18+
from garf_core.fetchers.rest import RestApiReportFetcher
19+
20+
__all__ = [
21+
'FakeApiReportFetcher',
22+
'RestApiReportFetcher',
23+
]
Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Copyright 2025 Google LLf
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# pylint: disable=C0330, g-bad-import-order, g-multiple-import
16+
17+
"""Getting fake data from memory or a file."""
18+
19+
from __future__ import annotations
20+
21+
import logging
22+
import os
23+
24+
from garf_core import (
25+
api_clients,
26+
parsers,
27+
query_editor,
28+
report_fetcher,
29+
)
30+
31+
logger = logging.getLogger(__name__)
32+
33+
34+
class FakeApiReportFetcher(report_fetcher.ApiReportFetcher):
35+
"""Returns simulated data."""
36+
37+
def __init__(
38+
self,
39+
data: list[dict[str, parsers.ApiRowElement]] | None = None,
40+
parser: parsers.BaseParser = parsers.DictParser,
41+
query_specification_builder: query_editor.QuerySpecification = (
42+
query_editor.QuerySpecification
43+
),
44+
data_location: str | os.PathLike[str] | None = None,
45+
csv_location: str | os.PathLike[str] | None = None,
46+
json_location: str | os.PathLike[str] | None = None,
47+
**kwargs: str,
48+
) -> None:
49+
if not data and not (
50+
data_location := json_location or csv_location or data_location
51+
):
52+
raise report_fetcher.ApiReportFetcherError(
53+
'Missing fake data for the fetcher.'
54+
)
55+
api_client = (
56+
api_clients.FakeApiClient(data)
57+
if data
58+
else api_clients.FakeApiClient.from_file(data_location)
59+
)
60+
super().__init__(api_client, parser, query_specification_builder, **kwargs)
61+
62+
@classmethod
63+
def from_csv(
64+
cls, file_location: str | os.PathLike[str]
65+
) -> FakeApiReportFetcher:
66+
"""Initialized FakeApiReportFetcher from a csv file."""
67+
return FakeApiReportFetcher(csv_location=file_location)
68+
69+
@classmethod
70+
def from_json(
71+
cls, file_location: str | os.PathLike[str]
72+
) -> FakeApiReportFetcher:
73+
"""Initialized FakeApiReportFetcher from a json file."""
74+
return FakeApiReportFetcher(json_location=file_location)
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Copyright 2025 Google LLf
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# pylint: disable=C0330, g-bad-import-order, g-multiple-import
16+
17+
"""Module for getting data from Rest APIs based on a query."""
18+
19+
from __future__ import annotations
20+
21+
import logging
22+
23+
from garf_core import (
24+
api_clients,
25+
parsers,
26+
query_editor,
27+
report_fetcher,
28+
)
29+
30+
logger = logging.getLogger(__name__)
31+
32+
33+
class RestApiReportFetcher(report_fetcher.ApiReportFetcher):
34+
"""Fetches data from an REST API endpoint.
35+
36+
Attributes:
37+
api_client: Initialized RestApiClient.
38+
parser: Type of parser to convert API response.
39+
"""
40+
41+
def __init__(
42+
self,
43+
endpoint: str,
44+
parser: parsers.BaseParser = parsers.DictParser,
45+
query_specification_builder: query_editor.QuerySpecification = (
46+
query_editor.QuerySpecification
47+
),
48+
**kwargs: str,
49+
) -> None:
50+
"""Instantiates RestApiReportFetcher.
51+
52+
Args:
53+
endpoint: URL of API endpoint.
54+
parser: Type of parser to convert API response.
55+
query_specification_builder: Class to perform query parsing.
56+
"""
57+
api_client = api_clients.RestApiClient(endpoint)
58+
super().__init__(api_client, parser, query_specification_builder, **kwargs)

0 commit comments

Comments
 (0)