bojone / crf Goto Github PK
View Code? Open in Web Editor NEWkeras implementation of conditional random field
Home Page: https://kexue.fm/archives/5542
keras implementation of conditional random field
Home Page: https://kexue.fm/archives/5542
如题,因为通常序列会在后面补一定数量的 0, 能忽略这个么?
请问,keras搭建层结构,model.compile(loss=CRF.loss, optimizer='adam',metrics=[CRF.accuracy])必须得有y_pred,应该怎么修改
希望能支持 tf.keras
改成python3需要更改三个地方才能跑起来.
1.np.random.shuffle(sents) # 打乱语料,以便后面划分验证集 sents = sents[0] chars = {} # 统计字表def max_in_dict(d): # 定义一个求字典中最大值的函数 key, value = list(d.items())[0] for i, j in list(d.items())[1:]: if j > value: key, value = i, j return key, value卡在那不动,是生成器的时候,遍历数据的时候,数据为空,所以需要sents = sents[0], 后面两处是验证的时候需要更改的.
我测试环境是Python3.6+tensorflow1.14/tensorflow2.1
其实把split('\r\n')改成split('\n')就可以了
Originally posted by @liujingqiao in #5 (comment)
苏神你好,
init_state包一层
Line 61 in 6bd55d9
step_function
里面解开
Line 31 in 6bd55d9
作为新状态又包一层
Line 36 in 6bd55d9
根据文档对initial_states的定义
initial_states: Tensor with shape (samples, state_size) (no time dimension), containing the initial values for the states used in the step function. In the case that state_size is in a nested shape, the shape of initial_states will also follow the nested structure.
但这里的initial_state.shape=(1,samples, state_size)
的这个是有特殊考虑吗?
crf_keras.py 文件的loss如下:
def loss(self, y_true, y_pred): # 目标y_pred需要是one hot形式
if self.ignore_last_label:
mask = 1 - y_true[:, :, -1:]
else:
mask = K.ones_like(y_pred[:, :, :1])
看了一下ConditionalRandomField的代码,https://github.com/bojone/bert4keras/blob/master/bert4keras/layers.py
def dense_loss(self, y_true, y_pred):
"""y_true需要是one hot形式
"""
# 导出mask并转换数据类型
mask = K.all(K.greater(y_pred, -1e6), axis=2, keepdims=True)
mask = K.cast(mask, K.floatx())
dense_loss计算mask用的y_pred, y_pred的值应该都是大于-1000000(-1e6),mask全是1 能起到什么作用呢?
不是应该用y_true来计算mask吗?类似crf_keras.py中的 mask = 1 - y_true[:, :, -1:]
实在没想通,望大神拨冗解惑一下
我想知道, 计算viterbi 的时候, 为什么不取 log ,而是直接把概率累加?
假设 P(AB) = P(A) * P(B|A) 只有取 log 了以后,才能 log(P(A) + log(P(B|A)) 吧。
不知道是否使用了一些特别的技巧?
您好,首先感谢您的分享。
我对CRF了解不多,想用您提供的语料分析的例子,观察一下CRF的执行过程。然后我发现在crf_keras.py文件中,有一个loss函数和一个accuracy函数,然后我希望能够像在tensorflow中那样,获取这两个函数中的每一个变量,然后sess.run掉,观察其中的内容。但是我对Keras不是很熟悉,请问是否可以获取loss函数或者accuracy函数中的变量(如loss函数中的init_states),观察其内容?
PS:网上说过获取keras模型中间层输出的方法,如https://blog.csdn.net/u010420283/article/details/80303231,但难点是您程序里的init_states等变量似乎并不是keras模型里的变量,所以这种方法似乎不能解决这个问题。
多谢指点!
` def log_norm_step(self, inputs, states):
"""递归计算归一化因子
要点:1、递归计算;2、用logsumexp避免溢出。
技巧:通过expand_dims来对齐张量。
"""
states = K.expand_dims(states[0], 2)
inputs = K.expand_dims(inputs, 2)
trans = K.expand_dims(self.trans, 0)
output = K.logsumexp(states+trans+inputs, 1)
return output, [output] #`
这里上个时刻的结果 + 转移矩阵 + 这个时刻标签的分值然后再取e指数吧,望指正
WARNING:tensorflow:From D:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/10
就这样
您好,我在看到您实现crf的时候,转移概率是通过两个错位的labels相乘得到,这一点我不是特别理解,如果label是one-hot向量,两个错位的label相乘的结果应该全为0了?用numpy模拟了一下确实结果为0,希望能予以解答。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.