petrochukm / simple-qa-emnlp-2018 Goto Github PK
View Code? Open in Web Editor NEWCode for my EMNLP 2018 paper "SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach"
Home Page: http://aclweb.org/anthology/D18-1051
Code for my EMNLP 2018 paper "SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach"
Home Page: http://aclweb.org/anthology/D18-1051
Hi, I'm a student studying in QA with deeplearning.
I'm following your github 'readme' and have some problems.
first, i'm first time in use PostgreSQL, so not sure how I can do thrid dot.
โ Create and populate a PostgreSQL table named fb_two_subject_name with notebooks/Simple QA KG to PostgreSQL DB/fb_two_subject_name.csv.gz
second, where I should put the SimpleQuestions v2 dataset?
third, I'm trying to run the "FB5M & FB2M KG to DB", but I got errors. The first one I got was 'FileNotFoundError', and it says I need '.pass', how I can get it or do something?
when runing Relation Classifier RNN Model.ipynb ,there has a problem in function get_iterator of Dataset Iterators part. In get_iterator ,when transfer NoisySortedBatchSampler,the dataset is not matched with NoisySortedSampler,and product a bug.
like this:
~/anaconda3/envs/QA/lib/python3.6/site-packages/torchnlp/samplers/noisy_sorted_batch_sampler.py in init(self, data, batch_size, drop_last, sort_key, sort_key_noise, last_batch_first, shuffle)
64 self.shuffle = shuffle
65 super().init(
---> 66 NoisySortedSampler(data=data, sort_key=sort_key, sort_key_noise=sort_key_noise),
67 batch_size, drop_last)
68
~/anaconda3/envs/QA/lib/python3.6/site-packages/torchnlp/samplers/noisy_sorted_sampler.py in init(self, data, sort_key, sort_key_noise)
29 for i, row in enumerate(self.data):
30 value = self.sort_key(row)
---> 31 noise_value = value * sort_key_noise
32 noise = random.uniform(-noise_value, noise_value)
33 value = noise + value
TypeError: unsupported operand type(s) for *: 'dict' and 'float'
i'sorry,I don't know how you add noise to the dataset.
hi~ there are some questions i want to ask.thanks
when runing Relation Classifier RNN Model.ipynb ,there has a problem in function get_iterator of Dataset Iterators part. In get_iterator ,when transfering **NoisySortedBatchSampler,**the dataset is not matched with NoisySortedSampler,then producted a bug.
like this:
~/anaconda3/envs/QA/lib/python3.6/site-packages/torchnlp/samplers/noisy_sorted_batch_sampler.py in init(self, data, batch_size, drop_last, sort_key, sort_key_noise, last_batch_first, shuffle)
64 self.shuffle = shuffle
65 super().init(
---> 66 NoisySortedSampler(data=data, sort_key=sort_key, sort_key_noise=sort_key_noise),
67 batch_size, drop_last)
68
~/anaconda3/envs/QA/lib/python3.6/site-packages/torchnlp/samplers/noisy_sorted_sampler.py in init(self, data, sort_key, sort_key_noise)
29 for i, row in enumerate(self.data):
30 value = self.sort_key(row)
---> 31 noise_value = value * sort_key_noise
32 noise = random.uniform(-noise_value, noise_value)
33 value = noise + value
TypeError: unsupported operand type(s) for *: 'dict' and 'float'
i'sorry,I don't know how you add noise to the dataset
Hi.
Looks like we are unable to run through the end-to-end pipeline currently due to missing model checkpoints in pretrained_model folder.
../../pretrained_models/relation_classifier.02_02_07:59:28/1746.pt
../../pretrained_models/relation_classifier.02_02_13:31:11/189.pt
../../pretrained_models/relation_classifier.02_02_13:23:02/223.pt
When I ran the third code cell in notebooks/Simple QA End-To-End/Step 1 - Predict Subject Name.ipynb, I got the following error.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-ab905cab6d01> in <module>
1 import importlib
2 import lib.import_notebook
----> 3 from allennlp.models.archival import load_archive
4 from allennlp.service.predictors import Predictor
5
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/models/__init__.py in <module>
4 """
5
----> 6 from allennlp.models.archival import archive_model, load_archive
7 from allennlp.models.crf_tagger import CrfTagger
8 from allennlp.models.decomposable_attention import DecomposableAttention
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/models/archival.py in <module>
15 from allennlp.common import Params
16 from allennlp.common.file_utils import cached_path
---> 17 from allennlp.models.model import Model, _DEFAULT_WEIGHTS
18
19 logger = logging.getLogger(__name__) # pylint: disable=invalid-name
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/models/model.py in <module>
13 from allennlp.common.params import Params
14 from allennlp.common.registrable import Registrable
---> 15 from allennlp.data import Instance, Vocabulary, Dataset
16 from allennlp.nn import util
17 from allennlp.nn.regularizers import RegularizerApplicator
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/data/__init__.py in <module>
----> 1 from allennlp.data.dataset import Dataset
2 from allennlp.data.dataset_readers.dataset_reader import DatasetReader
3 from allennlp.data.fields.field import DataArray, Field
4 from allennlp.data.instance import Instance
5 from allennlp.data.iterators.data_iterator import DataIterator
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/data/dataset.py in <module>
11 import tqdm
12
---> 13 from allennlp.data.instance import Instance
14 from allennlp.data.vocabulary import Vocabulary
15 from allennlp.common.checks import ConfigurationError
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/data/instance.py in <module>
1 from typing import Dict
2
----> 3 from allennlp.data.fields.field import DataArray, Field
4 from allennlp.data.vocabulary import Vocabulary
5
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/data/fields/__init__.py in <module>
5
6 from allennlp.data.fields.field import Field
----> 7 from allennlp.data.fields.array_field import ArrayField
8 from allennlp.data.fields.index_field import IndexField
9 from allennlp.data.fields.label_field import LabelField
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/data/fields/array_field.py in <module>
9
10
---> 11 class ArrayField(Field[numpy.ndarray]):
12 """
13 A class representing an array, which could have arbitrary dimensions.
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/allennlp/data/fields/array_field.py in ArrayField()
44 return tensor if cuda_device == -1 else tensor.cuda(cuda_device)
45
---> 46 @overrides
47 def empty_field(self): # pylint: disable=no-self-use
48 return ArrayField(numpy.array([], dtype="float32"))
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/overrides/overrides.py in overrides(method, check_signature, check_at_runtime)
86 """
87 if method is not None:
---> 88 return _overrides(method, check_signature, check_at_runtime)
89 else:
90 return functools.partial(
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/overrides/overrides.py in _overrides(method, check_signature, check_at_runtime)
112 return wrapper # type: ignore
113 else:
--> 114 _validate_method(method, super_class, check_signature)
115 return method
116 raise TypeError(f"{method.__qualname__}: No super class method found")
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/overrides/overrides.py in _validate_method(method, super_class, check_signature)
133 and not isinstance(super_method, property)
134 ):
--> 135 ensure_signature_is_compatible(super_method, method, is_static)
136
137
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/overrides/signature.py in ensure_signature_is_compatible(super_callable, sub_callable, is_static)
91
92 if super_type_hints is not None and sub_type_hints is not None:
---> 93 ensure_return_type_compatibility(super_type_hints, sub_type_hints, method_name)
94 ensure_all_kwargs_defined_in_sub(
95 super_sig, sub_sig, super_type_hints, sub_type_hints, is_static, method_name
~/anaconda3/envs/emnlp/lib/python3.6/site-packages/overrides/signature.py in ensure_return_type_compatibility(super_type_hints, sub_type_hints, method_name)
286 if not _issubtype(sub_return, super_return) and super_return is not None:
287 raise TypeError(
--> 288 f"{method_name}: return type `{sub_return}` is not a `{super_return}`."
289 )
TypeError: ArrayField.empty_field: return type `None` is not a `allennlp.data.fields.field.Field`.
I think it is due to the latest overrides package as discussed in here. If acceptable to you, I would be most grateful if you fix this error.
I noticed that the .pt model was missed , Can I train the model via the given code ?
Him in notebooks/Simple QA KG to PostgreSQL DB/Full Freebase to DB.ipynb
i see:
The dump of Freebase can be found here: https://developers.google.com/freebase
Should i download 30Gb of Freebase ? it is 250 gb unzipped
I created a new environment "test" with Python 3.6.
When I ran
$ python -m pip install -r requirements.txt
I got the following error.
Collecting git+git://github.com/PetrochukM/allennlp (from -r requirements.txt (line 14))
Cloning git://github.com/PetrochukM/allennlp to /tmp/pip-req-build-932446ln
Running command git clone -q git://github.com/PetrochukM/allennlp /tmp/pip-req-build-932446ln
Resolved git://github.com/PetrochukM/allennlp to commit ceab2e7a51d7d8791fdf35c7f6c46105a7d0adf8
ERROR: Command errored out with exit status 1:
command: /home/kato8966/anaconda3/envs/test/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-932446ln/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-932446ln/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-t14y823d
cwd: /tmp/pip-req-build-932446ln/
Complete output (7 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-932446ln/setup.py", line 54, in <module>
from allennlp import version
File "/tmp/pip-req-build-932446ln/allennlp/__init__.py", line 10, in <module>
import spacy, torch, numpy # pylint: disable=multiple-imports
ModuleNotFoundError: No module named 'spacy'
----------------------------------------
WARNING: Discarding git+git://github.com/PetrochukM/allennlp. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
I am afraid to say that the install process mentioned in README.md might not be sufficient enough. If acceptable to you, I would be most grateful if you give me a more detailed process.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.