17zuoye / pyirt Goto Github PK
View Code? Open in Web Editor NEWA python library of IRT algorithm
License: MIT License
A python library of IRT algorithm
License: MIT License
@junchenfeng
Hello.
When I executed a simple code, I found duplicate logs as follows. As you see, half of these are unnecessary logs.
2021-07-07 14:56:12 INFO: start loading data
start loading data
2021-07-07 14:56:12 INFO: data loaded
data loaded
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 281.77it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 321.18it/s]
2021-07-07 14:56:12 DEBUG: E step runs for 322.238 sec
E step runs for 322.238 sec
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 238.91it/s]
2021-07-07 14:56:12 DEBUG: M step runs for 153.022 sec
M step runs for 153.022 sec
2021-07-07 14:56:12 DEBUG: score calculating
score calculating
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 3794.03it/s]
2021-07-07 14:56:12 DEBUG: score calculated.
score calculated.
2021-07-07 14:56:12 DEBUG: 0.6018582902998789
0.6018582902998789
2021-07-07 14:56:12 DEBUG: stop condition runs for 120.155 sec
stop condition runs for 120.155 sec
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 222.29it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 102.90it/s]
2021-07-07 14:56:13 DEBUG: E step runs for 339.617 sec
E step runs for 339.617 sec
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 172.94it/s]
2021-07-07 14:56:13 DEBUG: M step runs for 169.493 sec
M step runs for 169.493 sec
2021-07-07 14:56:13 DEBUG: score calculating
score calculating
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2362.32it/s]
2021-07-07 14:56:13 DEBUG: score calculated.
score calculated.
2021-07-07 14:56:13 DEBUG: 0.6018582902998789
0.6018582902998789
2021-07-07 14:56:13 INFO: EM converged at iteration 2.
EM converged at iteration 2.
2021-07-07 14:56:13 DEBUG: stop condition runs for 114.175 sec
stop condition runs for 114.175 sec
2021-07-07 14:56:13 INFO: parameter estimated
parameter estimated
2021-07-07 14:56:13 INFO: parameter retrieved
parameter retrieved
This is the code I executed:
from pyirt import irt
input = [
('user_0', 'item_0', 0),
('user_1', 'item_0', 1)
]
item_param, user_param = irt(input)
您好,看到您写的这个irt算法,想使用下,可是不知道从哪里下手,不清楚源数据是哪些,都是什么格式,输出数据是什么,是什么格式,希望您有时间可以帮助下,谢谢。
hi ,there is a problem which confuse me a few of days .i run code as follows:
res = irt(data)
i check many items in the data. when the beta is smaller such as -1.9,-2 , but the liked rate of this item is very low which i think it should be more large.when the beta is bigger such as 1.9 , 2 ,but the liked rate of this item is very high which i think it should be smaller. could anyone explain sth to me when the beta is large ,the liked rate should be large or smaller. note: the liked rate equals the number of "1" divide all numbers of items .
the format of the data is as follows :
customer_id | product_id | liked |
---|---|---|
1086992 | 14 | 1 |
1112695 | 14 | 0 |
1112695 | 17 | 1 |
803222 | 18 | 1 |
1110363 | 19 | 0 |
273880 | 19 | 1 |
445253 | 19 | 0 |
I'm trying to use PyIRT, however it seems that the library has some issues with how it uses multiprocessing
. I guess it used to be fine some time ago, I'm currently using Python 3.9.1.
It seems that essentially PyIRT passes a local update
function as a target
to Process
. In effect, when multiprocessing
tries to fork and spawn a new process, it tries to dump a pickle of the function, which fails with AttributeError: Can't pickle local object 'IRT_MMLE_2PL.__update_theta_distr.<locals>.update'
.
Does anyone have a solution or a workaround to this issue?
Hi Junchen Feng,
I'm not sure if my comment is a real problem. I just want to contribute your code.
In the file: solver/model.py, in the lines: 256 and 269, you 'change' the values of the variables 'last_item_param_dict' and 'item_param_dict'.
Line 256 in c8f8c8e
Line 269 in c8f8c8e
However, the way the code is written, python will keep the same memory region being accessed by these two variables.
If you run
id(last_item_param_dict) == id(item_param_dict)
..the answer will be: True
.
I am in doubt whether this is a bug, or if you would like to keep these two variables pointing to the same structure in memory?
I did some testing and found that if I make an explicit copy of the dictionaries the result will be (a little) different.
The lines below show my test:
line 256: self.item_param_dict = self.last_item_param_dict.copy()
line 269: self.last_item_param_dict = self.item_param_dict.copy()
Thank you.
MongoDB integration is a huge advantage of this repo but is not documented in the Readme.md file.
Please add at least a little bit of documentation about how to use this - this is important for large datasets (e.g. >6GB or over 50,000,000 responses is outside of the memory capability of most laptops and necessitates use of a memory-optimized EC2 instance).
Hi ,this is a really good tool to use. But I encounter some problem, I feed data with colmuns=[user_ID,item_ID,choice], but get error. However, I choose the part of data, It sometimes works. I find that if a item is all aright or all false pyirt will return error.
`2018-08-13 16:09:20 INFO: data loaded
data loaded
data loaded
data loaded
data loaded
data loaded
data loaded
data loaded
data loaded
data loaded
100%|████████████████████████████████████████████████████████████████████████| 10000/10000 [00:55<00:00, 179.29it/s]
0%| | 0/61615 [00:00<?, ?it/s]
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
--- Logging error ---
Traceback (most recent call last):
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 992, in emit
msg = self.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 838, in format
return fmt.format(record)
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 575, in format
record.message = record.getMessage()
File "/home/lb/anaconda3/lib/python3.6/logging/init.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 394, in __get_expect_count
procs = procs_operator(procs, 7200, 0.1)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 27, in procs_operator
p.start()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
code = process_obj._bootstrap()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 373, in update
self.logger.critical("Unexpected error:", str(e))
Message: 'Unexpected error:'
Arguments: ('174',)
Process Process-168:
Traceback (most recent call last):
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 374, in update
raise e
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 367, in update
map_user_idx_vec = self.dao.get_map(item_idx, ['1', '0'])
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in get_map
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
File "/home/lb/pyirt-master/pyirt/dao.py", line 150, in
return [self.database.item2user_map[str(ans_key)][item_idx] for ans_key in ans_key_list]
KeyError: 174
Traceback (most recent call last):
File "", line 1, in
File "/home/lb/pyirt-master/pyirt/_pyirt.py", line 46, in irt
mod.solve_EM()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 101, in solve_EM
self._exp_step()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 161, in _exp_step
self.__get_expect_count()
File "/home/lb/pyirt-master/pyirt/solver/model.py", line 399, in __get_expect_count
self.item_expected_right_by_theta[:, item_idx] = procs_repo[item_idx][1]
File "", line 2, in getitem
File "/home/lb/anaconda3/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
raise convert_to_error(kind, result)
KeyError: 174
`
Just importing the package throws this error
/usr/local/lib/python3.6/site-packages/pyirt/util/clib.pyx:12:9: Compiler crash in AnalyseDeclarationsTransform
ModuleNode.body = StatListNode(clib.pyx:3:0)
StatListNode.stats[2] = StatListNode(clib.pyx:6:0)
StatListNode.stats[0] = DefNode(clib.pyx:6:0,
modifiers = [...]/0,
name = 'log_likelihood_2PL',
np_args_idx = [...]/0,
num_required_args = 5,
py_wrapper_required = True,
reqd_kw_flags_cname = '0')
File 'Nodes.py', line 427, in analyse_declarations: StatListNode(clib.pyx:12:4,
is_terminator = True)
File 'Nodes.py', line 495, in analyse_declarations: CDefExternNode(clib.pyx:12:9,
include_file = 'math.h')
Compiler crash traceback from this point on:
File "/usr/local/lib/python3.6/site-packages/Cython/Compiler/Nodes.py", line 495, in analyse_declarations
env.add_include_file(self.include_file, self.verbatim_include, late)
TypeError: add_include_file() takes 2 positional arguments but 4 were given
/tmp/pyximport/temp.macosx-10.13-x86_64-3.6/pyrex/pyirt/util/clib.c:1:2: error:
Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^
1 error generated.
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'clang' failed with exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pyximport/pyximport.py", line 215, in load_module
inplace=build_inplace, language_level=language_level)
File "/usr/local/lib/python3.6/site-packages/pyximport/pyximport.py", line 191, in build_module
reload_support=pyxargs.reload_support)
File "/usr/local/lib/python3.6/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
dist.run_commands()
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/usr/local/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 194, in build_extensions
self.build_extension(ext)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'clang' failed with exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "IRT_data_analyser.py", line 1, in
from pyirt import irt
File "/usr/local/lib/python3.6/site-packages/pyirt/init.py", line 2, in
from ._pyirt import irt
File "/usr/local/lib/python3.6/site-packages/pyirt/_pyirt.py", line 2, in
from .solver import model
File "/usr/local/lib/python3.6/site-packages/pyirt/solver/model.py", line 19, in
from ..util import clib, tools
File "/usr/local/lib/python3.6/site-packages/pyirt/util/init.py", line 8, in
from . import clib
File "/usr/local/lib/python3.6/site-packages/pyximport/pyximport.py", line 458, in load_module
language_level=self.language_level)
File "/usr/local/lib/python3.6/site-packages/pyximport/pyximport.py", line 231, in load_module
raise exc.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/pyximport/pyximport.py", line 215, in load_module
inplace=build_inplace, language_level=language_level)
File "/usr/local/lib/python3.6/site-packages/pyximport/pyximport.py", line 191, in build_module
reload_support=pyxargs.reload_support)
File "/usr/local/lib/python3.6/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
dist.run_commands()
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/usr/local/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 194, in build_extensions
self.build_extension(ext)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
ImportError: Building module pyirt.util.clib failed: ["distutils.errors.CompileError: command 'clang' failed with exit status 1\n"]
Hi there,
Very good work; this is definitely the easiest IRT library existing for python. I'm using this library to determine item and student params for vocabulary learning - that is, a student will often revisit the same word over and over again.
Is this library designed for multiple student-item interactions? Or is it solely for a test-based assessment?
Thanks
I have a dataset where there are 5 concepts and each concept has multiple items and there are student observations for the items ie if they attempted it correctly or not. Now while running pyirt I have these questions.
Is there any way to set the default values of alpha, beta, theta initially? From what I could learn that you have just mentioned bounds.
How is the algorithm running? I mean does it run on the entire database which has multiple concepts or is it running on individual concepts that have multiple items.
I ran the algorithm at concept level for beta range of 0-1 and found that beta values come out mostly 0. Do you know any reason why this could be happening?
How did you come up with the default values and what is the explanation for it?
adding our data for actual vs predicted output by the pyirt library -
Item ID | Actual Discrimination | Actual Difficulty | Predicted Discrimination | Predicted Difficulty |
---|---|---|---|---|
item1 | 92.177 | 0.41404 | 9.696222165393651 | 0.03189432377525194 |
item2 | 74.092 | 0.39517 | 23.581584962133306 | 0.0 |
item3 | 31.352 | 0.40759 | 1.2933333754068872 | 0.6941432833146606 |
Add documentation of build process.
For example:
'''
INSTALLATION
To install, please run python setup.py build
followed by python setup.py install
. NOTE: if sudo access is required and system has a different Python than the one you want to use (e.g. Anaconda on Mac OS, Ubuntu, or EC2 with Amazon Linux), you can use sudo -s
before running python setup.py install
because just running sudo python setup.py install
will use the system's Python instead of the user's Python.
'''
Hi, could you please update the PyPI release? I was unable to reproduce the demo when installing with pip, but everything went fine when I installed from source. The error that I am having is exactly the one from issue #13, it was already fixed, but PyPI is outdated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.