优化NLU样本、改进训练pipline

Author: jiangdongguo

ReadMe.md

@@ -5,9 +5,13 @@
 **RASA 开发中文指南系列博文：**
 
 - [Rasa中文聊天机器人开发指南(1)：入门篇](https://jiangdg.blog.csdn.net/article/details/104328946)
-- Rasa中文聊天机器人开发指南(2)：NLU篇
+- [Rasa中文聊天机器人开发指南(2)：NLU篇](https://jiangdg.blog.csdn.net/article/details/104328946)
 - Rasa中文聊天机器人开发指南(3)：Core篇
-- Rasa中文聊天机器人开发指南(4)：RasaX篇
+- Rasa中文聊天机器人开发指南(4)：RasaX与模型评估
+- Rasa中文聊天机器人开发指南(5)：浅析Mitie、spaCy和CRF实体识别器
+- Rasa中文聊天机器人开发指南(6)：浅析Mitie、Sklearn和Embedding意图分类器
+
+**注：本系列博客翻译自[Rasa官方文档](https://rasa.com/docs/rasa/)，并融合了自己的理解和项目实战，同时对文档中涉及到的技术点进行了一定程度的扩展，目的是为了更好的理解Rasa工作机制和相关技术要点。与本系列博文配套的项目GitHub地址：[ChitChatAssistant](https://github.com/jiangdongguo/ChitChatAssistant)，欢迎`star`和`issues`，我们共同讨论、学习！**
 
 
 
@@ -57,11 +61,20 @@ pip install jieba
 
 # 2. 训练模型  
 
- 当所有样本和配置文件准备好后，接下来就是训练模型了，打开命令终端执行下面的命令，该命令会同时训练NLU和Core模型，具体如下：
+ 当所有样本和配置文件准备好后，接下来就是训练模型了，打开命令终端执行下面的命令，该命令会同时训练NLU和Core模型。
+
+- 使用MITIE
+
 ```shell
 python -m rasa train --config configs/config.yml --domain configs/domain.yml --data data/
 ```
 
+- 使用Supervised_Embedding
+
+```bash
+python -m rasa train --config configs/zh_jieba_supervised_embeddings_config.yml --domain configs/domain.yml --data data/
+```
+
 # 3. 运行服务  
 
 **（1）启动Rasa服务**
@@ -91,24 +104,32 @@ Python -m rasa run actions --port 5055 --actions actions --debug
 python server.py
 ```
 
-当**Rasa Server**、**Action Server**和**Server.py**运行后，在浏览器输入：
+当**Rasa Server**、**Action Server**和**Server.py**运行后，在浏览器输入测试：
 
 ` http://127.0.0.1:8088/ai?content="查询广州明天的天气"`
 
-返回的结果为：
+终端调用效果为：
+
+![](https://img-blog.csdnimg.cn/20200227153932228.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0FuZHJFeHBlcnQ=,size_16,color_FFFFFF,t_70)
+
 
-![入门7](https://img-blog.csdnimg.cn/20200215154508497.png)
 
 # 4. 更新日志
 
 
 
-**（1）V1.0.0.2020.02.15**
+**（1）V1.0.2020.02.15**
 
 - 创建项目，模型训练成功；
 - 前端访问Rasa服务器正常响应；
 - 对接图灵闲聊机器人、心知天气API，便于测试；
 
+**（2）V1..1.2020.02.27**
+
+- 优化NLU样本，尝试使用同义词、正则、查找表；
+- 改进supervised_embeddings，实体提取和意图识别明显提高，训练速度加快很多；
+- 完成`Rasa中文聊天机器人开发指南(2)：NLU篇`文章撰写；
+
 
 
 # 5. License

actions/action.py

@@ -15,6 +15,39 @@
 )
 
 
+class PhoneForm(FormAction):
+
+    def name(self) -> Text:
+        """Unique identifier of the form"""
+
+        return "phone_form"
+
+    @staticmethod
+    def required_slots(tracker: Tracker) -> List[Text]:
+        """A list of required slots that the form has to fill"""
+
+        return ["phone_number", "business"]
+
+    def submit(
+            self,
+            dispatcher: CollectingDispatcher,
+            tracker: Tracker,
+            domain: Dict[Text, Any],
+    ) -> List[Dict]:
+        """Define what the form has to do
+            after all required slots are filled"""
+        business = tracker.get_slot('business')
+        number = tracker.get_slot('phone_number')
+
+        if business == "机主":
+            dispatcher.utter_message("您要查询的号码{}属于隔壁老蒋".format(number))
+        elif business == "余额":
+            dispatcher.utter_message("您要查询的号码{}账户余额为66666元".format(number))
+        else:
+            dispatcher.utter_message("暂不支持查询{}业务".format(business))
+        return [Restarted()]
+
+
 class WeatherForm(FormAction):
 
     def name(self) -> Text:

configs/config.yml

@@ -4,6 +4,7 @@ pipeline:
 - name: "MitieNLP"
   model: "data/total_word_feature_extractor_zh.dat"
 - name: "JiebaTokenizer"
+  dictionary_path: "data/dict"
 - name: "MitieEntityExtractor"
 - name: "EntitySynonymMapper"
 - name: "RegexFeaturizer"

configs/domain.yml

@@ -7,16 +7,23 @@ intents:
   - whoareyou
   - whattodo
   - request_weather
+  - request_phone_business
 
 slots:
   date-time:
     type: unfeaturized
   address:
     type: unfeaturized
+  phone_number:
+    type: unfeaturized
+  business:
+    type: unfeaturized
 
 entities:
   - date-time
   - address
+  - phone_number
+  - business
 
 actions:
   - utter_answer_affirm
@@ -28,10 +35,13 @@ actions:
   - utter_answer_whattodo
   - utter_ask_date-time
   - utter_ask_address
+  - utter_ask_phone_number
+  - utter_ask_business
   - action_default_fallback
 
 forms:
   - weather_form
+  - phone_form
 
 responses:
   utter_answer_affirm:
@@ -72,5 +82,11 @@ responses:
   utter_ask_address:
     - text: "请问您要查下哪里的天气？"
 
+  utter_ask_phone_number:
+    - text: "请问您要查的电话号码是多少？"
+
+  utter_ask_business:
+    - text: "请问您要查询什么业务呢？"
+
   utter_default:
     - text: "没听懂，请换种说法吧~"

configs/zh_jieba_supervised_embeddings_config.yml

@@ -0,0 +1,24 @@
+language: "zh"
+
+pipeline:
+- name: "JiebaTokenizer"
+  dictionary_path: "data/dict"
+- name: "RegexFeaturizer"
+- name: "CRFEntityExtractor"
+- name: "EntitySynonymMapper"
+- name: "CountVectorsFeaturizer"
+- name: "CountVectorsFeaturizer"
+  analyzer: "char_wb"
+  min_ngram: 1
+  max_ngram: 4
+- name: "EmbeddingIntentClassifier"
+
+policies:
+  - name: KerasPolicy
+    epochs: 500
+    max_history: 5
+  - name: FallbackPolicy
+    fallback_action_name: 'action_default_fallback'
+  - name: MemoizationPolicy
+    max_history: 5
+  - name: FormPolicy

data/dict/userdict.txt

@@ -0,0 +1,2 @@
+谁的 5 n
+属于谁 5 n

data/lookup_tables/DataPackage.txt

@@ -0,0 +1,6 @@
+腾讯视频流量包
+爱奇艺会员流量包
+网易免流包
+抖音免流包
+流量月包
+酷狗定向流量包

data/nlu/nlu.md

@@ -114,4 +114,47 @@
 -  [长沙](address)的天气
 - [深圳](address)[明天](date-time)的天气
 - 查下[今天](date-time)[上海](address)的天气
-- 帮我查查[佛山](address)这[周六](date-time)的天气
+- 帮我查查[佛山](address)这[周六](date-time)的天气
+
+## intent:request_phone_business
+- 查电话[19820618425](phone_number)
+- 我想知道电话号码为[19860612425](phone_number)
+- 查[11160222425](phone_number)
+- 查电话号码[19800222425](phone_number)
+- [机主](business)
+- 号码是[谁的](business)
+- 这个号码是[属于谁](business)
+- 谁是这个号码的[拥有者](business)
+- 查下[机主信息](business)
+- [机主](business)是谁
+- 我要查这个号码的[账户余额](business)
+- 帮我查[余额](business)
+- 查[话费](business)
+- 能告诉我现在的[话费余额](business)还剩多少
+- 我想查电话号码[19860618422](phone_number)的[账户余额](business)
+- 我要查下[19822618425](phone_number)的[机主](business)是谁
+- 你好！请帮我查询一下电话[12260618425](phone_number)的[账户余额](business)
+- 查一下手机号码[19862228425](phone_number)的[机主信息](business)
+- 帮我查个手机号[19860612222](phone_number)的[余额](business)
+- [19860222425](phone_number)是[谁的](business)
+
+
+## synonym:机主
+- 机主信息
+- 机主
+- 拥有者
+- 谁的
+- 属于谁
+
+## synonym:余额
+- 余额
+- 话费
+- 话费余额
+- 账户余额
+
+
+## regex:phone_number
+- ((\d{3,4}-)?\d{7,8})|(((\+86)|(86))?(1)\d{10})
+
+## lookup: mobile_data_package
+data/lookup_tables/DataPackage.txt

data/stories/stories.md

@@ -67,4 +67,10 @@
 * request_weather
     - weather_form
     - form{"name": "weather_form"}
+    - form{"name": null}
+    
+## happy path
+* request_phone_business
+    - phone_form
+    - form{"name": "phone_form"}
     - form{"name": null}