ktw5691 / psychtm Goto Github PK
View Code? Open in Web Editor NEWText Mining for Psychological Research
License: Other
Text Mining for Psychological Research
License: Other
How can models fitted with/without topics be compared?
For instance, in the following code:
with_category = gibbs_sldax(log(HoursActual) ~ log(HoursEstimate)+Category+ProjectCode, data = cl_info,
m = 450, burn = 300,
docs = docs_vocab$documents,
V = vocab_len,
K = 10, model = "sldax", display_progress = TRUE)
what would be the impact on goodness of fit, from just fitting:
log(HoursActual) ~ log(HoursEstimate)+Category+ProjectCode
?
That is, how can the impact of topics be separated out from the other covariates?
A quality of fit metric, such as AIC, would be useful output from summary
, assuming that upstream functions generate the necessary information.
Currently, gibbs_sldax()
checks for missing data in ANY variable in the data frame supplied to the data
argument. It will stop and warn the user about missing values even if the variables being used for modeling have no missing data.
Being clueless, I typed in the code from the help page, i.e., iter=5
, and it worked, or rather it produced numbers without any complaint.
Then I noticed that the output from summary contained a chain value (for my code, 1), so I tried that. It produced different output.
Then I noticed that in the example, the call to gibbs_mlr
included the argument m=5
. My
call to gibbs_sldax
has m=450
, and waic_all
produces different output when given 450.
The second parameter of waic_all
, l_pred
, is an 'iter' x D matrix.
The value of iter can be obtained from the argument passed. Why is the first parameter, iter
, needed?
The help is not helpful with regard to legitimate values of iter
. Not giving one does produce an error message.
bigmemory R package?
Bad initialization of regression coefficients can start the algorithm with infinite values from which it never recovers.
Having to do stop-word removal and stemming is a hassle.
Be helpful, like the textProcessor
function in the stm package, do it automatically (by default).
First release:
usethis::use_cran_comments()
Title:
and Description:
@return
and @examples
Authors@R:
includes a copyright holder (role 'cph')Prepare for release:
devtools::build_readme()
urlchecker::url_check()
devtools::check(remote = TRUE, manual = TRUE)
devtools::check_win_devel()
rhub::check_for_cran()
rhub::check(platform = 'ubuntu-rchk')
rhub::check_with_sanitizers()
Submit to CRAN:
usethis::use_version('minor')
devtools::submit_cran()
Wait for CRAN...
usethis::use_github_release()
usethis::use_dev_version()
Provide vignette to illustrate package use.
Package checking using the goodpractice
R package (which uses the cyclocomp
package) flagged the gibbs_*()
functions as having high cyclometric complexity. This could be reduced by writing more helper functions to take up tasks in the gibbs_*()
functions.
See https://adv-r.hadley.nz/s4.html#s4-classes Section 15.3.5
With verbose = TRUE
and model = "lda"
, output:
1 eta: 0
500 zbar d1: 0 6.9351e-310 nan
1000 zbar d1: 0 6.9351e-310 nan
...
Should be able to adapt code for sLDA sampler
The check in prep_docs
: if (any(is.na(data))) stop("Cannot handle missing values in 'data'")
is too broad. Only the columns specified in col
need to be checked.
Having to remove NA
from columns that are not referenced by the called psychtm
function is a pain.
Currently, the default contrasts computed in post_regression
use equal weighting for all reference topics being tested against for each topic effect. It would be nice to implement Piepel-type (1986) contrasts to take into account possible range restriction in the empirical topic proportions as well as their sampling variability.
# Example usage
post_regression(sldax_fit, ctype = c("equal", "piepel"))
Expand unit tests: next target is 25%.
tinytest
for C++ code unit tests.The message:
"Each document (row) in 'docs' must contain
at least 2 positive integers"
is not helpful. Why not say: "The text must contain at least two words."
Even better, why not automatically remove rows that don't contain some minimum number of words?
It took me 30 minutes to figure out which columns I needed to remove in the original data. For code+data see: https://shape-of-code.com/2022/01/23/including-natural-language-text-topics-in-a-regression-model/
Suggested by @alexbrodersen
A Bayesian
# Example usage
bayes_r2(sldax_fit)
Rcpp v.1.0.8 adds support for alternative degrees of full functionality from Rcpp:
See changed files for specific differences.
Set up a website for the package using packagedown
.
If they mismatch:
Error in .gibbs_sldax_cpp(docs, V, m, burn, thin, K, model, y, x, mu0, ย : join_rows() / join_horiz(): number of rows must be the same
Currently they are calculated based on the full model, not the collapsed model
I think this would be doable. The posterior mean estimates should just be weighted sums that could be updated at each non-thinned iteration after burn-in.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.