Using pip package sentinfer: ModuleNotFoundError: No module named 'sentinfer.main.functional' about pyabsa HOT 24 CLOSED

please try use from sentinfer import * instead

from pyabsa.

Thanks that worked. My train script looks like:

from sentinfer import *
param_dict = {'model_name':'slide_lcf_bert', 'lcf':'cdw', 'batch_size': 16}

# public datasets can be found in the other branch
train_set_path = 'datasets/acl-14-short-data/train.raw'  
model_path_to_save = './'
infermodel = train(param_dict, train_set_path, model_path_to_save)

--

Is this expected output? dataset files say laptop whereas i mentioned the acl data file path for training:
Sorted by free memory size
Downloading: 100%|█████████████████████████████████████████████████████████████████| 570/570 [00:00<00:00, 505kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████| 440M/440M [00:06<00:00, 64.0MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 2.01MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████| 28.0/28.0 [00:00<00:00, 27.3kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████| 466k/466k [00:00<00:00, 3.30MB/s]
buliding word indices...
n_trainable_params: 119521539, n_nontrainable_params: 0

SRD: 2
batch_size: 16
dataset: laptop
distance_aware_window: True
dropout: 0.1
l2reg: 1e-05
lcf: cdw
initializer: <function xavier_uniform_ at 0x7f1d2da3d0d0>
learning_rate: 2e-05
max_seq_len: 80
model_name: slide_lcf_bert
num_epoch: 3
optimizer: <class 'torch.optim.adam.Adam'>
pretrained_bert_name: bert-base-uncased
use_bert_spc: True
use_dual_bert: False
window: lr
seed: 996
device: cpu
distance_aware_windows: True
embed_dim: 768
hidden_dim: 768
polarities_dim: 3
train_dataset_path: datasets/acl-14-short-data/train.raw
model_path_to_save: .//slide_lcf_bert
model_class: <class 'sentinfer.models.slide_lcf_bert.SLIDE_LCF_BERT'>
dataset_file: {'main': './datasets/semeval14/Laptops_Train.xml.seg', 'test': './datasets/semeval14/Laptops_Test
_Gold.xml.seg'}
inputs_cols: ['text_bert_indices', 'spc_mask_vec', 'lcf_vec', 'left_lcf_vec', 'right_lcf_vec', 'left_asp_dist_w
', 'right_asp_dist_w']
repeat: 0

from pyabsa.

Yes, the model you trained are based on acl, I am sorry that mis-print is a bug need to be fixed, I will fix as soon as I can.

from pyabsa.

Ok no worries thank you! I am just waiting for it to finish training but it just reached epoch 1. Seems slower than expected.. maybe because im using a single K380 gpu. Will keep you posted in case I find anything else.

from pyabsa.

And thanks for your questions, I will optimize the code and package continuously.

from pyabsa.

Inferring on the text data line throws this error - Error while processing.

391it [24:17, 3.73s/it, epoch 0]
391it [24:58, 3.83s/it, epoch 1]
391it [24:27, 3.75s/it, epoch 2]
saved: .//slide_lcf_bert
Error while processing: [PADDING] everything is always cooked to perfection , the [ASP]service[ASP] is excellent ,
the decor cool and understated . [PADDING]

Will try the batch_infer also

from pyabsa.

I run and get a result, maybe I should review the code carefully.

from pyabsa.

Please comment the try-except block and see what the detail exception is, I will fix it, thank you

from pyabsa.

Maybe the reason is the text is incorrectly cut by '\n'

from pyabsa.

Yes it was because of the added \n. My bad!! It worked now. Thanks!
Had a couple of questions regarding your model.

1. Can it be trained on multiple sentences in the same example? Like:
   a) "$TUniversity researcher$T is training ABSA models for fun. He hopes to complete the exercise in a couple of days at a minimum. Sucks!!" Polarity for target: Negative
   b) "$T University researcher$T is training ABSA models for fun. He hopes to complete in a couple days minimum. Bravo!" Polarity for target: Positive.
   There are 3 sentence breaks in the above two examples.
2. If Bert is pretrained on specific domain corpus - like laptops, restaurants, or financial tweets. Would that reduce required size of training examples?

from pyabsa.

For batch inference, there is no result print and cannot see a predictions file. Is there a config that can be passed with the results predictions path?

from pyabsa.

For batch inference, there is no result print and cannot see a predictions file. Is there a config that can be passed with the results predictions path?

The batch infer function will print and return the result, I am not sure what happened. However your advice is great, the inference results should be stored optional.

from pyabsa.

Yes it was because of the added \n. My bad!! It worked now. Thanks!
Had a couple of questions regarding your model.

1. Can it be trained on multiple sentences in the same example? Like:
   a) "$TUniversity researcher$T is training ABSA models for fun. He hopes to complete the exercise in a couple of days at a minimum. Sucks!!" Polarity for target: Negative
   b) "$T University researcher$T is training ABSA models for fun. He hopes to complete in a couple days minimum. Bravo!" Polarity for target: Positive.
   There are 3 sentence breaks in the above two examples.
2. If Bert is pretrained on specific domain corpus - like laptops, restaurants, or financial tweets. Would that reduce required size of training examples?

1. No, this repo provides some simple functions currently, but will be updated according to useful advice.
2. There is no definite clue for how much data are needed on fine-tuning BERT trained specific domain corpus, but I believe that the more training data, the better performance.

from pyabsa.

I just publish a package fixing mix-print of parameters, you can update sentinfer by pip.

from pyabsa.

Thanks!

1. No, this repo provides some simple functions currently, but will be updated according to useful advice.

So if I create a training dataset with such examples on my own and use with your model. It will still not work? Most single tweets in real world are multiple sentences.

Would also be great to keep the build from source option available as before - in case somebody wants to tweak any code. I wanted to extract prediction classification probabilities apart from just the class, as that will indicate confidence of the prediction.

from pyabsa.

Thanks!

1. No, this repo provides some simple functions currently, but will be updated according to useful advice.

So if I create a training dataset with such examples on my own and use with your model. It will still not work? Most single tweets in real world are multiple sentences.

Would also be great to keep the build from source option available as before - in case somebody wants to tweak any code. I wanted to extract prediction classification probabilities apart from just the class, as that will indicate confidence of the prediction.

Good suggestions. Maybe I misunderstood you, of course you can train (and infer) on your own dataset, only if according to the data format of provided training datasets. Let me konw if I misunderstand you again, thanks

from pyabsa.

Multiple sentences

Yes it was because of the added \n. My bad!! It worked now. Thanks!
Had a couple of questions regarding your model.

1. Can it be trained on multiple sentences in the same example? Like:
   a) "$TUniversity researcher$T is training ABSA models for fun. He hopes to complete the exercise in a couple of days at a minimum. Sucks!!" Polarity for target: Negative
   b) "$T University researcher$T is training ABSA models for fun. He hopes to complete in a couple days minimum. Bravo!" Polarity for target: Positive.
   There are 3 sentence breaks in the above two examples.
2. If Bert is pretrained on specific domain corpus - like laptops, restaurants, or financial tweets. Would that reduce required size of training examples?

1. No, this repo provides some simple functions currently, but will be updated according to useful advice.
2. There is no definite clue for how much data are needed on fine-tuning BERT trained specific domain corpus, but I believe that the more training data, the better performance.

i.e. When constructing new dataset, multiple sentences are suggested to be regarded as independent smaples, and should be formated into three lines(text, aspect, polarity). However, if sentences are really inter-related, we can regrad them as one sample.

No matter how many aspects exist in a sample, each asepct should have three lines in case of training conviniently.

from pyabsa.

Thanks for your replies. What about inferring from source code instead of pip - to get the prediction class probabilities also.

For multiple sentences:
You mean when creating my own dataset. All samples should contain same number of sentences, and approximate length? What if I have a mixed sample?

from pyabsa.

That features could be available in future.

I mean, how to construct a dataset is somewhat intutive and has no standard, as long as follow the data format. And, each aspect show be regared as a training unit. Pack the sentences in to a line, the code will dynamic truncate the combined sentence before traing in order to save memory.

from pyabsa.

Thanks! I was able to build from source from your last repo and worked fine.

For the batch_infer function.. only see the following output. Could be a bug on my end but I doublechecked the infer script. Can do more troubleshooting as you suggest:

Choosing the GPU device has largest free memory...
Sorted by free memory size
buliding word indices...

from pyabsa.

Maybe because it is too slow to preprocess dataset. If you use sentinfer==0.1.2alpha0, please try install the new package pyabsa. The data preprocess is faster in the pyabsa.

from pyabsa.

I find the bug that contribute to the low speed in data processing, and fix it. now the inference preprocessing can be finished in within seconds. I will upload the new package and detach this tool into a independent repo for easy debugging and mantianing.

from pyabsa.

Thanks, I saw that the github repo was reverted to source code usage like before.. with some file updates.
I was trying this one out but came across more bugs while inferring I think. Is it possible to use the repo from march 2021on github? I still have it but even in that one of the inferring script files seems to have a bug. Let me raise the current bug in a separate issue, and please feel free to close this.

from pyabsa.

Maybe because it is too slow to preprocess dataset. If you use sentinfer==0.1.2alpha0, please try install the new package pyabsa. The data preprocess is faster in the pyabsa.

Got it will try using pyabsa for now.

from pyabsa.