|
1 | | -# Combine fetching and saving with `ApiQueryExecutor` |
| 1 | +#ApiExecutor |
| 2 | + |
| 3 | +If your job is to execute query and write it to local/remote storage you can use `ApiQueryExecutor` to do it easily. |
2 | 4 |
|
3 | 5 | ## Install |
4 | 6 |
|
5 | 7 | Ensure that `garf-executors` library is installed: |
6 | 8 |
|
7 | | -``` |
| 9 | +```bash |
8 | 10 | pip install garf-executors |
9 | 11 | ``` |
| 12 | +## Run |
10 | 13 |
|
11 | | -## Initialize |
12 | | - |
13 | | -If your job is to execute query and write it to local/remote storage you can use `ApiQueryExecutor` to do it easily. |
14 | | -> When reading query from file `ApiQueryExecutor` will use query file name as a name for output file/table. |
15 | | -/// tab | Python |
16 | | -```python |
17 | | -from garf_core.fetchers import FakeApiReportFetcher |
18 | | -from garf_executors import api_executor |
19 | | -from garf_io import reader |
| 14 | +Let's take an example of working with [YouTube Data API fetcher](../fetchers/youtube-data-api.md) to get some stats on YouTube video. |
20 | 15 |
|
| 16 | +!!! important |
| 17 | + Make sure that corresponding library for interacting with YouTube Data API is installed |
21 | 18 |
|
22 | | -# initialize query_executor to fetch report and store them in local/remote storage |
23 | | -fake_report_fetcher = FakeApiReportFetcher(data=[{'campaign': {'id': 1}}]) |
| 19 | + ```bash |
| 20 | + pip install garf_youtube_data_api |
| 21 | + ``` |
24 | 22 |
|
25 | | -query_executor = api_executor.ApiQueryExecutor(fetcher=fake_report_fetcher) |
| 23 | +/// tab | bash |
26 | 24 |
|
27 | | -context = api_executor.ApiExecutionContext(writer='csv') |
28 | | -``` |
29 | | -/// |
| 25 | +```bash |
30 | 26 |
|
31 | | -## Run |
| 27 | +echo " |
| 28 | +SELECT |
| 29 | + id, |
| 30 | + snippet.publishedAt AS published_at, |
| 31 | + snippet.title AS title |
| 32 | +FROM videos" > query.sql |
32 | 33 |
|
33 | | -/// tab | bash |
34 | 34 |
|
35 | | -``` |
36 | | -garf <QUERIES> --source <API_SOURCE> \ |
37 | | - --output <OUTPUT_TYPE> \ |
38 | | - --source.params1=<VALUE> |
| 35 | +garf query.sql --source youtube_data_api \ |
| 36 | + --output console \ |
| 37 | + --source.ids=VIDEO_ID |
39 | 38 | ``` |
40 | 39 |
|
41 | 40 | where |
42 | 41 |
|
43 | | -* `<QUERIES>`- local or remote path(s) to files with queries. |
44 | | -* `<API_SOURCE>`- type of API to use. Based on that the appropriate report fetcher will be initialized. |
45 | | -* `<OUTPUT_TYPE>` - output supported by [`garf-io` library](../garf_io/README.md). |
46 | | - |
47 | | -If your report fetcher requires additional parameters you can pass them via key value pairs under `--source.` argument, i.e.`--source.regionCode='US'` - to get data only from *US*. |
48 | | -> Concrete `--source` parameters are dependent on a particular report fetcher and should be looked up in a documentation for this fetcher. |
| 42 | +* `query` - local or remote path(s) to files with queries. |
| 43 | +* `source`- type of API to use. Based on that the appropriate report fetcher will be initialized. |
| 44 | +* `output` - output supported by [`garf-io` library](https://google.github.io/garf/usage/writers/). |
49 | 45 |
|
50 | 46 | /// |
51 | 47 | /// tab | Python |
52 | 48 | ```python |
53 | | -query_text = "SELECT campaign.id AS campaign_id, FROM campaign" |
| 49 | +from garf_executors import api_executor |
| 50 | + |
| 51 | + |
54 | 52 |
|
55 | | -# execute query and save results to `campaign.csv` |
56 | | -query_executor.execute(query=query_text, title="campaign", context=context) |
| 53 | +query_executor = api_executor.ApiQueryExecutor.from_fetcher_alias( |
| 54 | + 'youtube-data-api' |
| 55 | +) |
| 56 | +context = api_executor.ApiExecutionContext(writer='csv') |
57 | 57 |
|
58 | | -# execute query from file and save to results to `query.csv` |
59 | | -reader_client = reader.FileReader() |
60 | | -query_path="path/to/query.sql" |
| 58 | +query_text = """ |
| 59 | +SELECT |
| 60 | + id, |
| 61 | + snippet.publishedAt AS published_at, |
| 62 | + snippet.title AS title |
| 63 | +FROM videos |
| 64 | +""" |
61 | 65 |
|
62 | 66 | query_executor.execute( |
63 | | - query=reader_client.read(query_path), |
64 | | - title=query_path, |
65 | | - context=context |
| 67 | + query=query_text, |
| 68 | + title="query", |
| 69 | + context=context |
66 | 70 | ) |
67 | 71 | ``` |
68 | 72 | /// |
0 commit comments