简介
最近各大厂商有关深度学习神经网络的快速模型构建的组件包，层出不穷，Uber也不甘其后，推出了基于TensorFlow的工具箱: Ludwig
该工具集已经发布到了GitHub
2019-02-18 15_12_04-uber_ludwig_ Ludwig is a toolbox built on top of TensorFlow that allows to train.png
特点
这个工具箱甚至允许不写任何代码就能够训练神经网络模型，示例网页：Examples
2019-02-18 15_13_15-Examples - Ludwig.png
支持的应用包括：
- Text Classification
- Named Entity Recognition Tagging
- Natural Language Understanding
- Machine Translation （支持Attention机制）
- Chit-Chat Dialogue Modeling through Sequence2Sequence（支持Attention机制）
- Sentiment Analysis
- Image Classification
- Image Captioning
- One-shot Learning with Siamese Networks
- Visual Question Answering
- Kaggle's Titanic: Predicting survivors
- Time series forecasting
- Movie rating prediction
- Multi-label classification
-
Multi-Task Learning
看到支持的类型，我的心情是这样的：
duang.jpg
安装
ludwig依赖的组件包：
Cython>=0.25
h5py>=2.6
matplotlib>=3.0
numpy>=1.15,<1.16
pandas>=0.19
scipy>=0.18
scikit-image
scikit-learn
seaborn>=0.7
spacy>=2.0
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
tqdm
tabulate>=0.7
tensorflow>=1.12.0
PyYAML>=3.12
pytest
安装的方式如下，说明ludwig是依赖Spacy的
pip install ludwig
python -m spacy download en
初试牛刀
实现模型的训练，最简单的方式，准备训练集，如情感分析，数据集是如下的csv:
2019-02-18 15_23_34-movie_reviews.csv - Excel.png
假如路径为：./model/movie_reviews.csv
然后准备模型的描述YAML文件，文件路径为：./model/movie_sentiment_model_definition.yaml
内容如下：
需要特别说明的是：
情感分析的例子，按照官方说明的默认配置：
input_features:
-
name: review
type: text
encoder: parallel_cnn
level: word
output_features:
-
name: sentiment
type: category
效果是非常差的，训练3次之后，准确率在50%之后，就没有提升了，这与抛硬币也没什么区别啊。。。
2019-02-18 17_02_20-Photos.png
经过实践证明，type设置为sequence，encoder设置为rnn是相对靠谱的，训练5次之后，准确率如下：
2019-02-18 17_02_50-Photos.png
训练10次之后，准确率如下：
2019-02-18 17_14_22-Photos.png
所以对于文本方面的处理，按照如下设置，训练才有一定意义。
input_features:
-
name: review
type: sequence
encoder: rnn
cell_type: lstm
bidirectional: true
num_layers: 2
reduce_output: None
output_features:
-
name: sentiment
type: category
training:
epochs: 100
dropout: True
dropout_rate: 0.1
early_stop: 100
可以发现，这个配置是RNN文本分类模型相关，并且设置为BI-LSTM机制
安装ludwig完毕之后，训练这个模型，只需要在控制台录入：
ludwig experiment --data_csv ./model/movie_reviews.csv --model_definition_file ./model/movie_sentiment_model_definition.yaml
然后就搞定模型的训练了。
训练过程中，对于训练集中的文本，会生成与训练集同名的文本结构描述文件，如：movie_reviews.json（因为内容太大，所以截图结构折叠后的样子）：
2019-02-18 15_47_17-D__reviews.json - Notepad+.png
会自动在项目路径生成如下目录结构：results_run_0
此目录下会有一个模型描述json文件：results_run_0\description.json
我们可以在这里找到我们所熟悉的模型相关的超参数：
{
"command": "ludwig experiment ",
"dataset_type": "generic",
"input_data": "./model/movie_reviews.csv",
"ludwig_version": "0.1.0",
"model_definition": {
"combiner": {
"type": "concat"
},
"input_features": [
{
"bidirectional": true,
"cell_type": "lstm",
"encoder": "rnn",
"name": "review",
"num_layers": 2,
"reduce_output": "None",
"tied_weights": null,
"type": "sequence"
}
],
"output_features": [
{
"dependencies": [],
"loss": {
"class_distance_temperature": 0,
"class_weights": 1,
"confidence_penalty": 0,
"distortion": 1,
"labels_smoothing": 0,
"negative_samples": 0,
"robust_lambda": 0,
"sampler": null,
"type": "softmax_cross_entropy",
"unique": false,
"weight": 1
},
"name": "sentiment",
"reduce_dependencies": "sum",
"reduce_input": "sum",
"top_k": 3,
"type": "category"
}
],
"preprocessing": {
"bag": {
"fill_value": "",
"format": "space",
"lowercase": 10000,
"missing_value_strategy": "fill_with_const",
"most_common": false
},
"binary": {
"fill_value": 0,
"missing_value_strategy": "fill_with_const"
},
"category": {
"fill_value": "<UNK>",
"lowercase": false,
"missing_value_strategy": "fill_with_const",
"most_common": 10000
},
"force_split": false,
"image": {
"missing_value_strategy": "backfill"
},
"numerical": {
"fill_value": 0,
"missing_value_strategy": "fill_with_const"
},
"sequence": {
"fill_value": "",
"format": "space",
"lowercase": false,
"missing_value_strategy": "fill_with_const",
"most_common": 20000,
"padding": "right",
"padding_symbol": "<PAD>",
"sequence_length_limit": 256,
"unknown_symbol": "<UNK>"
},
"set": {
"fill_value": "",
"format": "space",
"lowercase": false,
"missing_value_strategy": "fill_with_const",
"most_common": 10000
},
"split_probabilities": [
0.7,
0.1,
0.2
],
"stratify": null,
"text": {
"char_format": "characters",
"char_most_common": 70,
"char_sequence_length_limit": 1024,
"fill_value": "",
"lowercase": true,
"missing_value_strategy": "fill_with_const",
"padding": "right",
"padding_symbol": "<PAD>",
"unknown_symbol": "<UNK>",
"word_format": "space_punct",
"word_most_common": 20000,
"word_sequence_length_limit": 256
},
"timeseries": {
"fill_value": "",
"format": "space",
"missing_value_strategy": "fill_with_const",
"padding": "right",
"padding_value": 0,
"timeseries_length_limit": 256
}
},
"training": {
"batch_size": 64,
"bucketing_field": null,
"decay": false,
"decay_rate": 0.96,
"decay_steps": 10000,
"dropout": true,
"dropout_rate": 0.1,
"early_stop": 100,
"epochs": 100,
"gradient_clipping": null,
"increase_batch_size_on_plateau": 0,
"increase_batch_size_on_plateau_max": 512,
"increase_batch_size_on_plateau_patience": 5,
"increase_batch_size_on_plateau_rate": 2,
"learning_rate": 0.001,
"learning_rate_warmup_epochs": 5,
"optimizer": {
"beta1": 0.9,
"beta2": 0.999,
"epsilon": 1e-08,
"type": "adam"
},
"reduce_learning_rate_on_plateau": 0,
"reduce_learning_rate_on_plateau_patience": 5,
"reduce_learning_rate_on_plateau_rate": 0.5,
"regularization_lambda": 0,
"regularizer": "l2",
"staircase": false,
"validation_field": "combined",
"validation_measure": "loss"
}
},
"random_seed": 42
}
训练所得的模型结构如下：
2019-02-18 15_38_01-Photos.png
对于模型的预测，训练完毕之后，假定有如下数据需要预测：
./data/test_data.csv
2019-02-18 15_44_37-test_data.csv - Excel.png
因为已经获得训练之后模型的路径了，只需要在控制台录入：
ludwig predict --data_csv ./data/test_data.csv --model_path ./results/_run_0/model
之后，Ludwig又会给我们生成一个results_0目录，其中就是预测的结果，这个预测结果是根据之前那个不合理的配置生成的模型，得到的，因此只有预览的意义：
2019-02-18 15_49_58-Q-Dir 6.17.png
预测的类别名称：
2019-02-18 15_50_33-sentiment_predictions.csv - Excel.png
预测的概率：
2019-02-18 15_50_55-sentiment_probability.csv - Excel.png
之所以看起来预测结果这么不靠谱，是因为只训练了两次，真正业务训练，对于这种大小的训练集，至少也应该训练100次以上了。
通过python调用Ludwig API
这个就很简单了，按照如下代码写即可，加入logging_level=logging.DEBUG这一句的目的是能够正常输出训练模型的日志。
"""
@version: 0.1
@author: Blade He
@site:
@software: PyCharm
@file: main.py
@time: 2019/02/18
"""
from ludwig import LudwigModel
import yaml
import logging
def startjob(csv_file_path = r'./model/movie_reviews.csv',
model_file = r'./model/movie_sentiment_model_definition.yaml',
test_file = r'./data/test_data.csv'):
with open(model_file, encoding='utf-8', mode='r') as file:
model_definition = yaml.load(file.read())
print(model_definition)
ludwig_model = LudwigModel(model_definition)
train_stats = ludwig_model.train(data_csv=csv_file_path,
logging_level=logging.DEBUG)
print(train_stats)
predictions = ludwig_model.predict(data_csv=test_file,
logging_level=logging.DEBUG)
print(predictions)
ludwig_model.close()
if __name__ == '__main__':
startjob()
小结
Uber这次真的是把神经网络的应用，做成工具包了，是真正意义上的，无需写代码，通过命令行+模型配置文件的形式，就可以训练与使用神经网络模型。
这应该就是趋势吧
