Skip to content

Commit 380fb13

Browse files
committed
up
1 parent bea900f commit 380fb13

35 files changed

Lines changed: 4187 additions & 73 deletions

body-temp/00-sample-codes.ipynb

Lines changed: 660 additions & 0 deletions
Large diffs are not rendered by default.

body-temp/G00-notes.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
2+
## 可视化简介
3+
4+
- [国家基础科学公共科学数据中心](https://nbsdc.cn/general/newQualityProductiveForce)
5+
6+
## 各类主要图形的绘制
7+
8+
- Johansson, R. (2024). Numerical Python. Apress. [Link](https://doi.org/10.1007/979-8-8688-0413-7), [PDF](https://link.springer.com/content/pdf/10.1007/979-8-8688-0413-7.pdf), [Google](<https://scholar.google.com/scholar?q=Numerical Python>). [github](https://github.com/Apress/Numerical-Python-3rd-ed).
9+
- [Chapter 4: Plotting and Visualization](https://github.com/lianxhcn/Numerical-Python-3rd-ed/blob/main/ch04-code-listing.ipynb)
10+
11+
112
## 可视化-Python 绘图
213

314
- file:///D:/Rbook/DSFinance/books/VanderPlas_2023_PDSH_Python_Data_Science_Handbook-2E.pdf#page=281.11

body-temp/graph_00_principles.ipynb

Lines changed: 300 additions & 16 deletions
Large diffs are not rendered by default.

body/00_intro.ipynb

Lines changed: 26 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -76,24 +76,26 @@
7676
"- Hilpisch Y., **Python for Finance**. 2019. [-PDF-](https://www.sea-stat.com/wp-content/uploads/2021/05/Yves-Hilpisch-Python-for-Finance_-Mastering-Data-Driven-Finance-Book-OReilly-2018.pdf#page=225.11), [github](https://github.com/yhilpisch/py4fi2nd)\n",
7777
"\n",
7878
"- Machine Learning for Algorithmic Trading, 2nd edition. [github](https://github.com/stefan-jansen/machine-learning-for-trading), [Website](https://ml4trading.io/)\n",
79-
"\n",
79+
"\n"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"id": "cebf60fc",
85+
"metadata": {},
86+
"source": [
8087
"### 因果推断和机器学习\n",
8188
"\n",
82-
"- Alves, Matheus Facure. **2022**, **Causal Inference for The Brave and True**. [Online Read](https://matheusfacure.github.io/python-causality-handbook/landing-page.html), [-github-](https://github.com/matheusfacure/python-causality-handbook)\n",
83-
" - 基本上覆盖了目前文献中使用多的多数因果推断方法,包括 IV, DID, SDID, PSM, Matching, Panel, SCM, RDD \n",
84-
" - 提供了完整的 Python 代码,可以 Fork [-github-](https://github.com/matheusfacure/python-causality-handbook) 仓库,然后在本地运行 **.ipynb** 文档 (Jupyter Notebook)\n",
85-
" - 书中使用了 `causalml` 和 `dowhy` 两个包,前者是作者开发的一个包,后者是微软开发的一个包\n",
86-
"\n",
87-
"- &#x1F34E; **ISLP** &emsp; James, G., D. Witten, T. Hastie, R. Tibshirani. **An introduction to statistical learning**: with Applications in Python (ISLP)[M]. Springer, **2023**, [website](https://www.statlearning.com/), [Resources](https://www.statlearning.com/resources-python), [github](https://github.com/intro-stat-learning/ISLP_labs), [-PDF-](https://bayanbox.ir/view/1060725898744657072/An-Introduction-to-Statistical-Learning-with-Applications-in-Python.pdf)\n",
88-
" - [ISLP documentation](https://intro-stat-learning.github.io/ISLP/):书中数据文件的详细说明\n",
89-
" - [各章 Python 实操部分](https://intro-stat-learning.github.io/ISLP/labs/Ch02-statlearn-lab.html)\n",
90-
" - [github-Notebooks](https://github.com/ogulcancicek/An-Introduction-to-Statistical-Learning-Python)&emsp; |&emsp; [Excercises and Solultions](https://github.com/hardikkamboj/An-Introduction-to-Statistical-Learning)\n",
91-
" \n",
92-
"- Tatsat, H., Puri, S., & Lookabaugh, B. (2020). **Machine Learning and Data Science Blueprints for Finance**. O'Reilly Media. [-PDF-](https://soclibrary.futa.edu.ng/books/Machine%20Learning%20and%20Data%20Science%20Blueprints%20for%20Finance%20(Hariom%20Tatsat,%20Sahil%20Puri,%20Brad%20Lookabaugh)%20(Z-Library).pdf), [github-2022](https://github.com/tatsath/fin-ml), [githu-new-2024](https://github.com/alecontuIT/ml_for_finance)\n",
93-
" - 分成监督学习和非监督学习两大部分,包含了常用的机器学习方法\n",
94-
" - 13 cases,涉及债券市场,股票市场分析等\n",
95-
" - 书里边的所有案例对应的 Python 代码可以不用本地安装,而在作者提供的 [在线平台](https://mybinder.org/v2/gh/tatsath/fin-ml/master) 上直接运行。\n",
96-
" - 用的 Jupyter Notebook"
89+
"- Facure, Matheus (2022). Causal Inference for The Brave and True. Online: https://matheusfacure.github.io/python-causality-handbook/landing-page.html. GitHub: https://github.com/matheusfacure/python-causality-handbook.(覆盖 IV、DID、SDID、PSM、Matching、Panel、SCM、RDD;包含完整 Jupyter Notebook;使用 causalml 与 dowhy。)\n",
90+
"\n",
91+
"- James, G., Witten, D., Hastie, T., & Tibshirani, R. (2023). An Introduction to Statistical Learning: With Applications in Python (ISLP). Springer. Website: https://www.statlearning.com/. Python 资源与实验:https://intro-stat-learning.github.io/ISLP/,GitHub: https://github.com/intro-stat-learning/ISLP_labs。PDF: https://bayanbox.ir/view/1060725898744657072/An-Introduction-to-Statistical-Learning-with-Applications-in-Python.pdf\n",
92+
"\n",
93+
"- Tatsat, H., Puri, S., & Lookabaugh, B. (2020). Machine Learning and Data Science Blueprints for Finance. O'Reilly Media. PDF: https://soclibrary.futa.edu.ng/books/Machine%20Learning%20and%20Data%20Science%20Blueprints%20for%20Finance%20(Hariom%20Tatsat,%20Sahil%20Puri,%20Brad%20Lookabaugh)%20(Z-Library).pdf。GitHub: https://github.com/tatsath/fin-ml,案例可在 Binder 在线运行: https://mybinder.org/v2/gh/tatsath/fin-ml/master。\n",
94+
"\n",
95+
"- Chollet, François (2021). Deep Learning with Python (2nd ed.). Manning. ISBN: 9781617296864. 代码示例: https://www.manning.com/books/deep-learning-with-python-second-edition。GitHub: https://github.com/fchollet/deep-learning-with-python-notebooks。\n",
96+
"- Buduma, N., Buduma, N., & Papa, J. (2022). Fundamentals of deep learning. [-PDF-](https://webfiles.amrita.edu/2025/02/deep-learning-material-dept-ece-ase-blr-1.pdf)\n",
97+
"\n",
98+
"- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. URL: <http://www.deeplearningbook.org>, [TensorFlow-Excercises](https://www.tensorflow.org/tutorials?hl=zh-cn), [Slides](https://www.deeplearningbook.org/lecture_slides.html), [-PDF1-](http://alvarestech.com/temp/deep/Deep%20Learning%20by%20Ian%20Goodfellow,%20Yoshua%20Bengio,%20Aaron%20Courville%20(z-lib.org).pdf), [-PDF2-](https://github.com/janishar/mit-deep-learning-book-pdf/blob/master/complete-book-pdf/Ian%20Goodfellow%2C%20Yoshua%20Bengio%2C%20Aaron%20Courville%20-%20Deep%20Learning%20(2017%2C%20MIT).pdf)\n"
9799
]
98100
},
99101
{
@@ -122,6 +124,8 @@
122124
" - 数据科学和机器学习领域的知名网站,提供了大量的资源和信息。\n",
123125
"- [Kaggle Datasets](https://www.kaggle.com/datasets)\n",
124126
" - 全球知名的数据科学与机器学习社区,用户可以在平台上获取数据集、参与竞赛、分享与学习代码、交流讨论。\n",
127+
"- [Mendeley Data](https://data.mendeley.com/)\n",
128+
" - 一个开放的数据集存储库,用户可以上传和分享数据集。诸多学术期刊的作者在此分享他们的数据集。\n",
125129
"- [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php) \n",
126130
" - 机器学习领域最经典的数据集仓库,涵盖分类、回归、聚类等多种任务,适合教学和算法测试。\n",
127131
"- [Google Dataset Search](https://datasetsearch.research.google.com/) \n",
@@ -139,6 +143,12 @@
139143
"- [awesome-public-datasets](https://github.com/awesomedata/awesome-public-datasets)\n",
140144
" - GitHub 开放数据集列表 \n",
141145
"\n",
146+
"### 国内数据平台\n",
147+
"- [中国科学院科学数据中心](https://www.casdc.cn/home)\n",
148+
"- [国家生态科学数据中心](https://www.nesdc.org.cn/about)\n",
149+
"### 学术期刊\n",
150+
"\n",
151+
"- [Data in Brief]()\n",
142152
"\n",
143153
"### 学校图书馆\n",
144154
"\n",

body/01_04_markdown.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
#
1+
# Markdown 和 Marp
22

33
## 何谓 Markdown?
44

body/01_05_github.ipynb

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "d3200b85",
6+
"metadata": {},
7+
"source": [
8+
"# github\n",
9+
"\n",
10+
"参考资料:\n",
11+
"\n",
12+
"- YouTube: [Git, GitHub, & GitHub Desktop for beginners](https://www.youtube.com/watch?v=8Dd7KRpKeaE)\n",
13+
"- YouTube: [Create Repository, Commit, and Push using GitHub Desktop](https://www.youtube.com/watch?v=PvUexC0-D2s)\n",
14+
"- YouTube: [Using Git & GitHub in VSCode: Stage, Commit, and Push](https://www.youtube.com/watch?v=z5jZ9lrSpqk)"
15+
]
16+
}
17+
],
18+
"metadata": {
19+
"language_info": {
20+
"name": "python"
21+
}
22+
},
23+
"nbformat": 4,
24+
"nbformat_minor": 5
25+
}

body/ML_deepML_00_notes.md

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
2+
## 参考资料
3+
4+
- Greg Durrett, 2021-2024, "CS388 Natural Language Processing course materials", retrieved from https://www.cs.utexas.edu/~gdurrett/courses/online-course/materials.html
5+
6+
- Philipp Krähenbühl, 2020-2024, "AI394T Deep Learning course materials", retrieved from https://www.philkr.net/dl_class/material and https://ut.philkr.net/deeplearning/
7+
8+
- Philipp Krähenbühl, 2025, "AI395T Advances in Deep Learning course materials", retrieved from https://ut.philkr.net/advances_in_deeplearning/
9+
10+
11+
- François Fleuret, The Little Book of Deep Learning, [-PDF-](https://fleuret.org/public/lbdl.pdf)
12+
- 梯度下降部分介绍的很好
13+
14+
15+
## ANN
16+
17+
- CS229 Lecture Notes, 2023, Stanford University, [7.2 Neural networks](https://cs229.stanford.edu/main_notes.pdf#page=81.24)
18+
- 写得非常清楚
19+
20+
21+
## 简单 ANN
22+
23+
### 激活函数
24+
25+
> Smets, B. M. N. (2024). Mathematics of Neural Networks (Lecture Notes Graduate Course) (Version 1). arXiv. [Link](https://doi.org/10.48550/arXiv.2403.04807) (rep), [PDF](https://arxiv.org/pdf/2403.04807.pdf), [Google](<https://scholar.google.com/scholar?q=Mathematics of Neural Networks (Lecture Notes Graduate Course) (Version 1)>).
26+
27+
28+
We list some commonly used scalar activation functions which are also illustrated in Figure 1.2.
29+
30+
![20250910151737](https://fig-lianxh.oss-cn-shenzhen.aliyuncs.com/20250910151737.png)
31+
32+
![20250910151811](https://fig-lianxh.oss-cn-shenzhen.aliyuncs.com/20250910151811.png)
33+
34+
> Figure 1.2.
35+
36+
- Rectified Linear Unit (ReLU): arguably the most used activation function in modern neural networks, it is calculated as
37+
38+
$$
39+
\sigma(\lambda)=\operatorname{ReLU}(\lambda):=\max \{0, \lambda\} .
40+
$$
41+
42+
- Sigmoid (also known as logistic sigmoid or soft-step):
43+
44+
$$
45+
\sigma(\lambda):=\frac{1}{1+e^{-\lambda}} .
46+
$$
47+
48+
49+
The sigmoid was commonly used as activation function in early neural networks, which is the reason that activations functions in general are still often labeled with a $\sigma$.
50+
- Hyperbolic tangent: very similar to the sigmoid, it is given by
51+
52+
$$
53+
\tanh (\lambda):=\frac{e^\lambda-e^{-\lambda}}{e^\lambda+e^{-\lambda}} .
54+
$$
55+
56+
- Swish: a more recent choice of activation function that can be thought of as a smooth variant of the ReLU. It is given by the multiplication of the input itself with the sigmoid function:
57+
58+
$$
59+
\operatorname{swish}_\beta(\lambda):=\lambda \sigma(\beta \lambda):=\frac{\lambda}{1+e^{-\beta \lambda}},
60+
$$
61+
62+
where $\beta>0$. The $\beta$ parameter is usually chosen to be 1 but could be treated as a trainable parameter if desired. In case of $\beta=1$, this function is also called the sigmoid-weighted linear unit or SiLU.
63+
64+
### 多变量激活函数
65+
66+
Activation functions need not be scalar, we list some common multivariate functions.
67+
- Softmax, also known as the normalized exponential function: softmax : $\mathbb{R}^n \rightarrow[0,1]^n$ is given by
68+
69+
$$
70+
\operatorname{softmax}\left(\left[\begin{array}{c}
71+
x_1 \\
72+
x_2 \\
73+
\vdots \\
74+
x_n
75+
\end{array}\right]\right):=\frac{1}{\sum_{i=1}^n e^{x_i}}\left[\begin{array}{c}
76+
e^{x_1} \\
77+
e^{x_2} \\
78+
\vdots \\
79+
e^{x_n}
80+
\end{array}\right]
81+
$$
82+
83+
84+
Softmax has the useful property that its output is a discrete probability distribution, i.e. each value is a non-negative real in the range [ 0,1 ], and all the values in its output add up to exactly 1 .
85+
- Maxpool: here each output is the maximum of a certain subset of the inputs: the function $\operatorname{maxpool}: \mathbb{R}^n \rightarrow \mathbb{R}^m$ is given by
86+
87+
$$
88+
\text { maxpool }\left(\left[\begin{array}{c}
89+
x_1 \\
90+
x_2 \\
91+
\vdots \\
92+
x_n
93+
\end{array}\right]\right):=\left[\begin{array}{cc}
94+
\max _{j \in I_1} & x_j \\
95+
\max _{j \in I_2} & x_j \\
96+
\vdots & \\
97+
\max _{j \in I_m} & x_j
98+
\end{array}\right] .
99+
$$
100+
101+
102+
Where for each $i \in\{1, \ldots, m\}$ we have a $I_i \subset\{1, \ldots, n\}$ that specifies over which inputs to take the maximum for each output. Maxpooling can easily be generalised by replacing the max operation with $\min$, the average, the mean, etc.
103+
104+
- Normalization, sometimes it is desirable to re-center and re-scale a signal:
105+
106+
$$
107+
\text { normalize }\left(\left[\begin{array}{c}
108+
x_1 \\
109+
x_2 \\
110+
\vdots \\
111+
x_n
112+
\end{array}\right]\right):=\left[\begin{array}{c}
113+
\frac{x_1-\mu}{\sigma} \\
114+
\frac{x_2-\mu}{\sigma} \\
115+
\vdots \\
116+
\frac{x_n-\mu}{\sigma}
117+
\end{array}\right] \text {, }
118+
$$
119+
120+
where $\mu=\mathbb{E}[x]$ and $\sigma^2=\operatorname{Var}(x)$. There are many variants on normalization where the difference is how $\mu$ and $\sigma$ are computed: over time, over subsets of the incoming signals, etc.
121+
122+
All the previous examples of activation functions are deterministic, but stochastic activation functions are also used.
123+
124+
- Dropout is a stochastic function that is often used during the training process but is removed once the training is finished. It works by randomly setting individual values of a signal to zero with probability $p$ :
125+
126+
$$
127+
\left(\operatorname{dropout}_p(x)\right)_i:= \begin{cases}0 & \text { with probability } p \\ x_i & \text { with probability } 1-p\end{cases}
128+
$$
129+
130+
- Heatbath is a scalar function that outputs 1 or -1 with a probability that depends on the input:
131+
132+
$$
133+
\text { heatbath }(\lambda):=\left\{\begin{aligned}
134+
1 & \text { with probability } \frac{1}{1+e^{-\lambda}} \\
135+
-1 & \text { otherwise. }
136+
\end{aligned}\right.
137+
$$
138+
139+
140+
All the activation functions we seen are essentially fixed functions, swish and dropout have a parameter but it is usually fixed to some chosen value. That means that the trainable parameters of a neural network are usually the linear weights and biases. There is however no a-priori reason why that needs to be the case, in fact we will see a class of non-linear operators with trainable parameters at the end of Chapter 3. Regardless, having parameters in the non-linear part of a network is somewhat rare in practice at the time of this writing.
141+
142+
143+
## Gradient Desent
144+
145+
146+
147+
148+
![20250910171507](https://fig-lianxh.oss-cn-shenzhen.aliyuncs.com/20250910171507.png)
149+
150+
> <https://www.cs.cmu.edu/~ggordon/MCMC/ggordon.MCMC-tutorial.pdf>
151+
152+
- https://www.cs.cornell.edu/courses/cs4780/2022sp/notes/LectureNotes11.html
153+
154+
155+
![20250910172841](https://fig-lianxh.oss-cn-shenzhen.aliyuncs.com/20250910172841.png)
156+
> https://bayen.berkeley.edu/sites/default/files/lecture101.pdf
157+
158+
#### 重点参考
159+
160+
- Ryan Tibshirani Scribes: Cong Lu/Yu Zhao, 2012, [Lecture 5: Gradient Desent Revisited](https://www.cs.cmu.edu/~ggordon/10725-F12/scribes/10725_Lecture5.pdf)
161+
162+
163+
- Ryan Tibshirani Scribes: Cong Lu/Yu Zhao, 2012, [Lecture 2: Optimization](https://www.cs.cmu.edu/~ggordon/10725-F12/scribes/10725_Lecture2.pdf)
164+
Gradient descent is the grandfather of first order methods. It simply starts at an initial point and then repeatedly takes a step opposite to the gradient direction of the function at the current point. The gradient descent algorithm to minimize a function $f(x)$ is as follows:
165+
166+
$
167+
\begin{aligned}
168+
& \text { for } k=0,1,2, \ldots \text { do } \\
169+
& \qquad g_k \leftarrow \nabla f\left(x_k\right) \\
170+
& \quad x_{k+1} \leftarrow x_k-t_k g_k \\
171+
& \text { end for }
172+
\end{aligned}
173+
$
174+
175+
176+
### 随机梯度下降
177+
178+
> <https://ubc-mds.github.io/DSCI_572_sup-learn-2/lectures/03_sgd-intro-to-nn.html>
179+
180+
- But if we have large $n$ and/or $\mathbf{w}$, gradient descent becomes very computationally expensive (when we get to deep learning, we'll have models where the number of weights to optimize is in the millions)
181+
- Say $\mathrm{n}=1,000,000$, we have 1000 parameters to optimize, and we do 1000 iterations $=O\left(10^{12}\right)$ computations
182+
- For each data point, we need to compute gradients with respect to each of model parameter.
183+
- For each iteration, it'll have to compute $10^6 \times 10^3=10^9$ gradients
184+
- We can reduce this workload by using just a fraction of our dataset to update our parameters each iteration (rather than using the whole data set)
185+
- This is called stochastic gradient descent [1]
186+
187+
188+
## Python codes
189+
190+
- [financial-data-science-notebooks](https://github.com/lianxhcn/financial-data-science-notebooks/tree/main) / [6.3\_deep\_learning.ipynb](https://github.com/lianxhcn/financial-data-science-notebooks/blob/main/6.3_deep_learning.ipynb)

0 commit comments

Comments
 (0)