Skip to content

Commit 908df78

Browse files
quantopiancalquantopiancal
authored
Separated the logic to group by market cap in the notebook (#294)
New groupby logic
1 parent e9e209b commit 908df78

File tree

2 files changed

+41
-69
lines changed

2 files changed

+41
-69
lines changed

‎notebooks/tutorials/3_factset_alphalens_lesson_4/notebook.ipynb

Lines changed: 35 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,16 @@
1313
"1. Grouping assets by market cap, then analyzing each cap type individually.\n",
1414
"2. Writing group neutral strategies.\n",
1515
"3. Determining an alpha factor's decay rate.\n",
16-
"4. Dealing with a common Alphalens error named MaxLossExceededError.\n",
17-
"\n",
18-
"**All sections of this lesson will use the data produced by the Pipeline created in the following cell. Please run it.**\n",
16+
"4. Dealing with a common Alphalens error named MaxLossExceededError."
17+
]
18+
},
19+
{
20+
"cell_type": "markdown",
21+
"metadata": {},
22+
"source": [
23+
"## Grouping By Market Cap\n",
1924
"\n",
20-
"**Important note**: Until this lesson, we passed the output of `run_pipeline()` to `get_clean_factor_and_forward_returns()` without any changes. This was possible because the previous lessons' Pipelines only returned one column. This lesson's Pipeline returns two columns, which means we need to *specify the column* we're passing as factor data. Look for commented code near `get_clean_factor_and_forward_returns()` in the following cell to see how to do this."
25+
"The following code defines a universe and creates an alpha factor within a pipeline. It also returns a classifier by using the quantiles() function. This function is useful for grouping your assets by an arbitrary column of data. In this example, we will group our assets by their market cap, and analyze how effective our alpha factor is among the different cap types (small, medium, and large cap)."
2126
]
2227
},
2328
{
@@ -54,19 +59,17 @@
5459
"\n",
5560
" factor_to_analyze = (ciwc_processed + spwc_processed).zscore()\n",
5661
"\n",
57-
" # The following columns will help us group assets by market cap. This will allow us to analyze\n",
58-
" # whether our alpha factor's predictiveness varies among assets with different market caps.\n",
5962
" market_cap = factset.Fundamentals.mkt_val.latest\n",
60-
" is_small_cap = market_cap.percentile_between(0, 100)\n",
61-
" is_mid_cap = market_cap.percentile_between(50, 100)\n",
62-
" is_large_cap = market_cap.percentile_between(90, 100)\n",
63+
" \n",
64+
" # .quantiles(), when supplied with bins=3, tells you which third that the assets value places in.\n",
65+
" # for example, in 2018, Apple is in the third bin, because it has a large market cap.\n",
66+
" # A different asset with a smaller market cap would probably be in the first or second bin.\n",
67+
" cap_type = market_cap.quantiles(bins=3, mask=base_universe)\n",
6368
"\n",
6469
" return Pipeline(\n",
6570
" columns = {\n",
6671
" 'factor_to_analyze': factor_to_analyze, \n",
67-
" 'small_cap_filter': is_small_cap,\n",
68-
" 'mid_cap_filter': is_mid_cap,\n",
69-
" 'large_cap_filter': is_large_cap,\n",
72+
" 'cap_type': cap_type\n",
7073
" },\n",
7174
" screen = (\n",
7275
" base_universe\n",
@@ -75,29 +78,14 @@
7578
" )\n",
7679
" )\n",
7780
"\n",
78-
"\n",
81+
"# Create the pipeline data\n",
7982
"pipeline_output = run_pipeline(make_pipeline(), '2013-1-1', '2014-1-1')\n",
80-
"pricing_data = get_pricing(pipeline_output.index.levels[1], '2013-1-1', '2014-3-1', fields='open_price')\n",
8183
"\n",
82-
"# To group by market cap, we will follow the following steps.\n",
84+
"# Replace the quantile values in the cap_type column for added clarity\n",
85+
"pipeline_output['cap_type'].replace([0, 1, 2], ['small_cap', 'mid_cap', 'large_cap'], inplace=True)\n",
8386
"\n",
84-
"# Convert the \"True\" values to ones, so they can be added together\n",
85-
"pipeline_output[['small_cap_filter', 'mid_cap_filter', 'large_cap_filter']] *= 1\n",
86-
"\n",
87-
"# If a stock passed the large_cap filter, it also passed the mid_cap and small_cap filters.\n",
88-
"# This means we can add the three columns, and stocks that are large_cap will get a value of 3,\n",
89-
"# stocks that are mid cap will get a value of 2, and stocks that are small cap will get 1.\n",
90-
"pipeline_output['cap_type'] = (\n",
91-
" pipeline_output['small_cap_filter'] + pipeline_output['mid_cap_filter'] + pipeline_output['large_cap_filter']\n",
92-
")\n",
93-
"\n",
94-
"# drop the old columns, we don't need them anymore\n",
95-
"pipeline_output.drop(['small_cap_filter', 'mid_cap_filter', 'large_cap_filter'], axis=1, inplace=True)\n",
96-
"\n",
97-
"# rename the 1's, 2's and 3's for clarity\n",
98-
"pipeline_output['cap_type'].replace([1, 2, 3], ['small_cap', 'mid_cap', 'large_cap'], inplace=True)\n",
87+
"pricing_data = get_pricing(pipeline_output.index.levels[1], '2013-1-1', '2014-3-1', fields='open_price')\n",
9988
"\n",
100-
"# the final product\n",
10189
"pipeline_output.head(5)"
10290
]
10391
},
@@ -111,7 +99,9 @@
11199
"\n",
112100
"You can group assets by any classifier, but sector and market cap are most common. The Pipeline in the first cell of this lesson returns a column named `cap_type`, whose values represent the assets market capitalization. All we have to do now is pass that column to the `groupby` argument of `get_clean_factor_and_forward_returns()`\n",
113101
"\n",
114-
"**Run the following cell, and notice the charts at the bottom of the tear sheet showing how our factor performs among different cap types.**"
102+
"**Run the following cell, and notice the charts at the bottom of the tear sheet showing how our factor performs among different cap types.**\n",
103+
"\n",
104+
"**Important note**: Until this lesson, we passed the output of `run_pipeline()` to `get_clean_factor_and_forward_returns()` without any changes. This was possible because the previous lessons' Pipelines only returned one column. This lesson's Pipeline returns two columns, which means we need to *specify the column* we're passing as factor data. Look for commented code near `get_clean_factor_and_forward_returns()` in the following cell to see how to do this."
115105
]
116106
},
117107
{
@@ -125,7 +115,7 @@
125115
"from alphalens.tears import create_returns_tear_sheet\n",
126116
"\n",
127117
"factor_data = get_clean_factor_and_forward_returns(\n",
128-
" factor=pipeline_output['factor_to_analyze'],\n",
118+
" factor=pipeline_output['factor_to_analyze'], # This is how you pass a single column from pipeline_output\n",
129119
" prices=pricing_data,\n",
130120
" groupby=pipeline_output['cap_type'],\n",
131121
")\n",
@@ -247,26 +237,26 @@
247237
"metadata": {},
248238
"outputs": [],
249239
"source": [
250-
"new_pipeline_output = run_pipeline(\n",
240+
"pipeline_output = run_pipeline(\n",
251241
" make_pipeline(),\n",
252242
" start_date='2013-1-1', \n",
253243
" end_date='2014-1-1' # *** NOTE *** Our factor data ends in 2014\n",
254244
")\n",
255245
"\n",
256-
"new_pricing_data = get_pricing(\n",
246+
"pricing_data = get_pricing(\n",
257247
" pipeline_output.index.levels[1], \n",
258248
" start_date='2013-1-1',\n",
259249
" end_date='2015-2-1', # *** NOTE *** Our pricing data ends in 2015\n",
260250
" fields='open_price'\n",
261251
")\n",
262252
"\n",
263-
"new_factor_data = get_clean_factor_and_forward_returns(\n",
264-
" new_pipeline_output['factor_to_analyze'], \n",
265-
" new_pricing_data,\n",
253+
"factor_data = get_clean_factor_and_forward_returns(\n",
254+
" pipeline_output['factor_to_analyze'], \n",
255+
" pricing_data,\n",
266256
" periods=range(1,252,20) # Change the step to 10 or more for long look forward periods to save time\n",
267257
")\n",
268258
"\n",
269-
"mean_information_coefficient(new_factor_data).plot()"
259+
"mean_information_coefficient(factor_data).plot()"
270260
]
271261
},
272262
{
@@ -293,23 +283,23 @@
293283
],
294284
"metadata": {
295285
"kernelspec": {
296-
"display_name": "Python 3",
286+
"display_name": "Python 2",
297287
"language": "python",
298-
"name": "python3"
288+
"name": "python2"
299289
},
300290
"language_info": {
301291
"codemirror_mode": {
302292
"name": "ipython",
303-
"version": 3
293+
"version": 2
304294
},
305295
"file_extension": ".py",
306296
"mimetype": "text/x-python",
307297
"name": "python",
308298
"nbconvert_exporter": "python",
309-
"pygments_lexer": "ipython3",
310-
"version": "3.7.0"
299+
"pygments_lexer": "ipython2",
300+
"version": "2.7.12"
311301
}
312302
},
313303
"nbformat": 4,
314304
"nbformat_minor": 2
315-
}
305+
}

‎notebooks/tutorials/3_factset_alphalens_lesson_5/notebook.ipynb

Lines changed: 6 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -60,16 +60,13 @@
6060
" # The following columns will help us group assets by market cap. This will allow us to analyze\n",
6161
" # whether our alpha factor's predictiveness varies among assets with different market caps.\n",
6262
" market_cap = factset.Fundamentals.mkt_val.latest\n",
63-
" is_small_cap = market_cap.percentile_between(0, 100)\n",
64-
" is_mid_cap = market_cap.percentile_between(50, 100)\n",
65-
" is_large_cap = market_cap.percentile_between(90, 100)\n",
63+
" cap_type = market_cap.quantiles(bins=3, mask=base_universe)\n",
6664
"\n",
6765
" return Pipeline(\n",
6866
" columns = {\n",
69-
" 'factor_to_analyze': factor_to_analyze, \n",
70-
" 'small_cap_filter': is_small_cap,\n",
71-
" 'mid_cap_filter': is_mid_cap,\n",
72-
" 'large_cap_filter': is_large_cap,\n",
67+
" 'factor_to_analyze': factor_to_analyze,\n",
68+
" 'cap_type': cap_type\n",
69+
" \n",
7370
" },\n",
7471
" screen = (\n",
7572
" base_universe\n",
@@ -78,26 +75,11 @@
7875
" )\n",
7976
" )\n",
8077
"\n",
81-
"# To group by market cap, we will follow the following steps.\n",
82-
"\n",
83-
"# Convert the \"True\" values to ones, so they can be added together\n",
84-
"pipeline_output[['small_cap_filter', 'mid_cap_filter', 'large_cap_filter']] *= 1\n",
85-
"\n",
86-
"# If a stock passed the large_cap filter, it also passed the mid_cap and small_cap filters.\n",
87-
"# This means we can add the three columns, and stocks that are large_cap will get a value of 3,\n",
88-
"# stocks that are mid cap will get a value of 2, and stocks that are small cap will get 1.\n",
89-
"pipeline_output['cap_type'] = (\n",
90-
" pipeline_output['small_cap_filter'] + pipeline_output['mid_cap_filter'] + pipeline_output['large_cap_filter']\n",
91-
")\n",
92-
"\n",
93-
"# drop the old columns, we don't need them anymore\n",
94-
"pipeline_output.drop(['small_cap_filter', 'mid_cap_filter', 'large_cap_filter'], axis=1, inplace=True)\n",
95-
"\n",
78+
"pipeline_output = run_pipeline(make_pipeline(), '2015-1-1', '2016-1-1')\n",
9679
"# rename the 1's, 2's and 3's for clarity\n",
9780
"pipeline_output['cap_type'].replace([1, 2, 3], ['small_cap', 'mid_cap', 'large_cap'], inplace=True)\n",
9881
"\n",
99-
"# the final product\n",
100-
"pipeline_output.head(5)"
82+
"pricing_data = get_pricing(pipeline_output.index.levels[1], '2015-1-1', '2016-6-1', fields='open_price')"
10183
]
10284
},
10385
{

0 commit comments

Comments
 (0)