Comments (2)
from tensorflow-2.x-tutorials.
The demo code from keras-bert should still work if you rename a few dependencies.
Delete the period in "from .layers import" in bert.py and try running this:
from tensorflow import keras
from bert import get_base_dict, get_model, gen_batch_inputs
# A toy input example
sentence_pairs = [
[['all', 'work', 'and', 'no', 'play'], ['makes', 'jack', 'a', 'dull', 'boy']],
[['from', 'the', 'day', 'forth'], ['my', 'arm', 'changed']],
[['and', 'a', 'voice', 'echoed'], ['power', 'give', 'me', 'more', 'power']],
]
# Build token dictionary
token_dict = get_base_dict() # A dict that contains some special tokens
for pairs in sentence_pairs:
for token in pairs[0] + pairs[1]:
if token not in token_dict:
token_dict[token] = len(token_dict)
token_list = list(token_dict.keys()) # Used for selecting a random word
# Build & train the model
model = get_model(
token_num=len(token_dict),
head_num=5,
transformer_num=12,
embed_dim=25,
feed_forward_dim=100,
seq_len=20,
pos_num=20,
dropout_rate=0.05,
)
model.summary()
def _generator():
while True:
yield gen_batch_inputs(
sentence_pairs,
token_dict,
token_list,
seq_len=20,
mask_rate=0.3,
swap_sentence_rate=1.0,
)
model.fit_generator(
generator=_generator(),
steps_per_epoch=1000,
epochs=100,
validation_data=_generator(),
validation_steps=100,
callbacks=[
keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)
],
)
# Use the trained model
inputs, output_layer = get_model(
token_num=len(token_dict),
head_num=5,
transformer_num=12,
embed_dim=25,
feed_forward_dim=100,
seq_len=20,
pos_num=20,
dropout_rate=0.05,
training=False, # The input layers and output layer will be returned if `training` is `False`
trainable=False, # Whether the model is trainable. The default value is the same with `training`
output_layer_num=4, # The number of layers whose outputs will be concatenated as a single output.
# Only available when `training` is `False`.
)
I'm pushing a main.py for lesson29-BERT soon so hopefully that'll make it more convenient to use.
from tensorflow-2.x-tutorials.
Related Issues (20)
- GCN中utils文件的preprocess_features函数的注释问题
- Any new project? HOT 1
- Can't download the pre-trained weights HOT 1
- Will the English tutorials be updated with Chinese ones? HOT 1
- Gradients do not exist for variables HOT 5
- 13-DCGCN error HOT 1
- 20-GCN Data HOT 2
- 电子书 9.8 过拟合问题实战部分代码不完整
- google drive的链接404了
- 关于WGAN-gp源码的两点问题 HOT 2
- 3.2模型构建--利用TensorFlow自动下载MNIST数据集,引入tensorflow.keras会报错 HOT 1
- Fix Google drive link for FasterRCNN weights
- 如何判断训练数据是有效的 HOT 3
- error about fasterRcnn
- 10.7池化层 最大输出判断错误
- Question about Faster RCNN pretrained model HOT 2
- 你好,请问nlp 中文文本怎么处理?
- Faster-RCNN只有resnet101的预训练模型嘛,请问有resnet50的预训练模型嘛
- Faster-RCNN能訓練自己數據集?
- Can Faster-RCNN train its own data set?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorflow-2.x-tutorials.