Fix bug in config_parse.py when batch_norm layer is used in RecurrentLayerGroup #966
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #961
解决方法:
在 config_parse.py里构造 BatchNormLayer的另外两个Input()时,不经过函数 https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer/config_parser.py#L503
错误原因:
产生原因是 BatchNorm Layer里自动为moving average参数添加了Input():
因为调用顺序原因,这里引用的名字
inputs[0].input_layer_name
, 在RecurrentLayerGroup里是被函数MakeLayerNameInSubmodel()编码之后的名字,即input_layer_name + "@" + submodel_name
, 导致出错。其他解决办法:
如果在config_parse.py外面, 由用户自定添加,例如下面方式,这样两个Input(1)和Input(2)的name经过config_parse.py内部编码会与 Input(0) 保持一致, 不会出错。
这个可以在
trainer_config_helpers
里重新包装, 但是部分还是用老配置的用户,得手动自己添加,较麻烦。其他办法:
是否可以在config_parse.py里只添加 Parameter(), 而不添加Input()?
但Layer.cpp里init函数https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/gserver/layers/Layer.cpp#L57 解析时一个input对应一个参数,所以应该不可行。
可否合并为同一个参数?
因为 .w0 .w1, .w2的一些操作不同,也无法创建同一片空间,在Layer.cpp里用offset的方式构造。