Giter VIP home page Giter VIP logo

gorse's People

Contributors

alitrack avatar amaaazing avatar asdine avatar ccfish86 avatar cyjaysong avatar dankinder avatar davewang avatar deepsource-autofix[bot] avatar dependabot-preview[bot] avatar dependabot[bot] avatar deslum avatar europaer avatar imlewc avatar justinlongdev avatar lbw114007 avatar mehmetcansahin avatar mezbaul-h avatar mrtztg avatar nmathar avatar rhorber avatar rustfix avatar somiacao avatar testwill avatar ujwachuku avatar varunvora avatar winwill2012 avatar wisperdin avatar zhang555 avatar zhangzhenghao avatar zhenghaoz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gorse's Issues

请问release-0.2

请问release-0.2这个分支是所有接口功能都完备的吗?

Rest API Recommends Not Found

Hello, when I add dummy data and then list the recommends, I get an error like "3 not found". But when I list users, I see 3 id
users in the list. I can see the feedback of 3 id users. So why am I getting such an error even though there are 3 id users?

gorse.db.zip

config.toml

# This section declares settings for the server.
[server]
host = "127.0.0.1"      # server host
port = 8888             # server port

# This section declares setting for the database.
[database]
file = "gorse.db"       # database file

# This section declares settings for recommendation.
[recommend]
model = "bpr"           # recommendation model
cache_size = 100        # the number of cached recommendations
update_threshold = 10   # update model when more than 10 ratings are added
check_period = 1        # check for update every one minute
similarity = "pearson"  # similarity metric for neighbors
fit_jobs = 1            # concurrent jobs for fitting

# This section declares hyperparameters for the recommendation model.
[params]
n_factors = 10          # the number of latent factors
reg = 0.01              # regularization strength
lr = 0.05               # learning rate
n_epochs = 100          # the number of learning epochs
init_mean = 0.0         # the mean of initial latent factors initilaized by Gaussian distribution
init_std = 0.001        # the standard deviation of initial latent factors initilaized by Gaussian distribution

添加新数据是否需要重新训练模型

我打算在go的项目中使用您的包来进行完成推荐的部分,但文档中这部分却很少我未找到我需要的资料。在这里冒昧询问,希望可以得到您的回复

  1. 我在加载数据之后,可以得到 相关的排行数据,如果产生了新的数据,是不是要重新计算这个模型,包括之前的旧数据;
    // 数据添加之后是否需要重新执行下面这个过程
    data := core.LoadDataFromCSV("test/recommend/games.csv", ",", true)
    train, _ := core.Split(data, 0.2)
    // Create model
    knn := model.NewKNNImplicit(base.Params{
    base.NFactors: 10,
    base.Reg: 0.01,
    base.Lr: 0.05,
    base.NEpochs: 100,
    base.InitMean: 0,
    base.InitStdDev: 0.001,
    })
    knn.Fit(train, nil)
    rank, _ := core.Neighbors(data, "222900", 10, engine.LoadSimilarity("msd"))

关于要清除item的需求

在商业应用中,一些应用,如作品已发布到gorse,然后异步鉴黄作品后,作品没有通过鉴黄,需要在gorse中清除数据。这个功能如何做?谢谢!

请问是否可以支持自定义?

你好,大神:
在单机测试了下你做的这个项目,挺不错的。👍
现在我几个问题想请教下,先谢谢了。

  1. 可以自定义数据结构么,目前项目是基于rate以及关联item进行推荐。有没有可能不仅仅基于rate,比如用户在某个内容发表评论、点赞、打标签等等这些指标呢?
  2. 看了下api.go,里面有两个数据录入接口/items和/feedback,是需要json根式的数据么,大概样式是什么样的呢?
  3. 在测试/random/?number=3的时候,返回结果有时少于3个。
  4. 如果把数据存在mongo db里,是否需要自己用代码实现?

再次感谢分享这个项目👍

Unknown model knn_implicit

After running the test
gorse test knn_implicit --load-csv games.csv --csv-sep ',' --eval-precision --eval-recall --eval-ndcg --eval-map --eval-mrr
I am getting an error like this:
Unknown model knn_implicit

I tried both, with -tags='avx2' and without
gorse version
0.1.3

[bug] bucket not found

After import Steam Dataset, request for http://127.0.0.1:8080/user/xxxxxx will return an error bucket not found. However, it works for MovieLens datasets.

Need example Save and Load model

Example save - load model

package main

import (
	"fmt"
	"github.com/zhenghaoz/gorse/base"
	"github.com/zhenghaoz/gorse/core"
	"github.com/zhenghaoz/gorse/model"
)

func main() {
	// Load dataset
	data := core.LoadDataFromBuiltIn("ml-100k")
	// Split dataset
	train, test := core.Split(data, 0.2)
	// Create model
	svd := model.NewSVD(base.Params{
		base.Lr:       0.007,
		base.NEpochs:  100,
		base.NFactors: 80,
		base.Reg:      0.1,
	})
	// Fit model
	svd.Fit(train)
	if err := core.Save("test", svd); err!=nil{
		fmt.Println(err)
	}

	var newSVD model.SVD


	if err := core.Load("test", &newSVD);err!=nil{
		fmt.Println(err)
	}

	scores := core.EvaluateRating(&newSVD, test, core.RMSE, core.MAE)
	fmt.Printf("RMSE = %.5f\n", scores[0])
	fmt.Printf("RMSE = %.5f\n", scores[1])
	// Predict a rating
	fmt.Printf("Predict(4,8) = %.5f\n", newSVD.Predict(4, 8))
}

请问能推荐相似的电影吗?

比如在电影详情页,推荐一些相似的电影,或者是大家都爱看的且与当前这一部相关的。
我理解的是现在是只能推荐给用户他可能会喜欢的电影。
对这些算法不太了解,非常感谢作者的付出。

2.0 怎么跑起来?

先启动master的时候报错(应该是数据表没有)

gorse git:(release-0.2) ✗ ./gorse-master -c config.toml              
INFO[0000] master: load config from config.toml         
INFO[0000] master: start rpc server 127.0.0.1:8086      
INFO[0000] master: start loop (period = 30 min)         
FATA[0000] master: failed to pull dataset for ranking (the Database field must be set on Operation) 

用gorse-cli导入数据时候 提示必须先起master :(

 ./gorse-cli -h                             
FATA[0000] cli: failed to load master config (rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:8086: connect: connection refused") 

Prevent recommending an item for an user who provided feedback

I have been trying the gorse with bpr model to recommend content to users, whom have option to give feedback which are fed into gorse.

I'm using gorse as standalone web server and accessing with it's own apis.

What I'm trying to accomplish is preventing the engine from recommending content the user already provided feedback. Bu I have looked both into the docs and the code but nothing really looked like what I wanted. There is somewhat related #33 but it recommends an entry only once and never again.

Is there any way to accomplish what I want.

Possible to combine subsets?

I want to create a paged recommender system, but this means I need to exclude the results from past pages.

I can currently get a list of items the user has rated to not show with excludeItems := set.User(username), as per the documentation. However, I don't see a way to combine the past item IDs I've been recommending (which are strings) with the subset provided by excludeItems.

gorse error in Docker container

goroutine 1 [running]:
xr/svd-cooker/vendor/github.com/zhenghaoz/gorse/core.init.0()
	/go/src/xr/svd-cooker/vendor/github.com/zhenghaoz/gorse/core/built_in.go:75 +0x2a
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x75be6a]

https://github.com/zhenghaoz/gorse/blob/master/core/built_in.go

func init() {
	usr, _ := user.Current()
	gorseDir := usr.HomeDir + "/.gorse"
	downloadDir = gorseDir + "/download"
	dataSetDir = gorseDir + "/datasets"
	TempDir = gorseDir + "/temp"
}

  1. Need return err in user.Current()
  2. Need create path without being tied to the user

Strings for user and item IDs

Hi there,

Currently it is not possible to use User IDs other than integers,
for example 8b3876c0-69b5-444e-9118-c5fcd575169f, cannot be converted with str.Atoi(), which results in an error.

Is it possible to support strings for IDs?

Cheers,

希望restful api 能增加分页功能

首先很感谢作者分享此项目,特别是提供了http方式可以简便的进行操作。

但目前发现api不支持游标和分页,只支持number进行获取,在使用过程中有诸多不便。
希望能增加分页功能,谢谢

docker-compose and dummy dataset of products/items

Hi @zhenghaoz ,

Hope you are all well !

I am posting this issue as I was wondering if it could be possible to add a docker-compose file for bootstrapping gorse with all its components ?

Also, I was wondering if you could provide a dummy dataset of products (e-commerce) to show case how to import and query for collaborative filtering scores. If you want, I can create such dataset if you give me the columns definition of the csv or jsonl file. :-)

Thanks for any insights or inputs on these questions.

Cheers,
Luc Michalski

Go get not work

After update

root@pc:/home/deslum/src# go get github.com/zhenghaoz/gorse
package github.com/zhenghaoz/gorse: no Go files in /home/deslum/src/github.com/zhenghaoz/gorse

Potential code quality issues found

I forked this repo a while ago and ran a DeepSource analysis on it. DeepSource found a variety of different issues categorized based on their types and severity which you can view here.

Brief description -
Anti-Patterns: 38
Bug Risks: 12
Security Issues: 1

You can find a detailed description and fixes for some of them here.

Please let me know what issues you'd like to focus on fixing and I'd be happy to take a look into it. Also, you can choose to hide certain types of issues too (if you wish to ignore them or you believe it is a false positive). I'd also be happy to send a patch with the DeepSource configuration file with the required fixes.

You can find the required configuration file for DeepSource here.

Invalid model name results in segmentation violation

When running gorse serve -c config/gorse.toml the command crashes:

2020/02/05 10:58:07 training model
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x917733]

goroutine 5 [running]:
github.com/zhenghaoz/gorse/engine.UpdateRecommends(0xc000104050, 0xc, 0xc004565110, 0x32, 0x1, 0xb7cdc0, 0xc0000de000, 0xc0000105a0, 0x0, 0x2)
        /root/gorse/engine/offline.go:89 +0x193
github.com/zhenghaoz/gorse/engine.Update(0xc00002bfc0, 0x7, 0x1f90, 0xc000104010, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /root/gorse/engine/offline.go:135 +0x1d8
github.com/zhenghaoz/gorse/cmd.watch(0xc00002bfc0, 0x7, 0x1f90, 0xc000104010, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /root/gorse/cmd/serve.go:85 +0x36b
created by github.com/zhenghaoz/gorse/cmd.glob..func7
        /root/gorse/cmd/serve.go:31 +0x30c

After some investigation I found out that I used an invalid model (knn-implicit instead of knn_implicit).

IMHO an invalid model should be reported instead of causing a segmentation fault.

Error in example

Error in example

package main

import (
	"fmt"
	"github.com/zhenghaoz/gorse/base"
	"github.com/zhenghaoz/gorse/core"
	"github.com/zhenghaoz/gorse/model"
)

func main() {
	// Load dataset
	data := core.LoadDataFromBuiltIn("ml-100k")
	// Split dataset
	train, test := core.Split(data, 0.2)
	// Create model
	svd := model.NewSVD(base.Params{
		base.Lr:       0.007,
		base.NEpochs:  100,
		base.NFactors: 80,
		base.Reg:      0.1,
	})
	// Fit model
	svd.Fit(train)
	// Evaluate model
	fmt.Printf("RMSE = %.5f\n", core.RMSE(svd, test))
	// Predict a rating
	fmt.Printf("Predict(4,8) = %.5f\n", svd.Predict(4, 8))
}package main

import (
	"fmt"
	"github.com/zhenghaoz/gorse/base"
	"github.com/zhenghaoz/gorse/core"
	"github.com/zhenghaoz/gorse/model"
)

func main() {
	// Load dataset
	data := core.LoadDataFromBuiltIn("ml-100k")
	// Split dataset
	train, test := core.Split(data, 0.2)
	// Create model
	svd := model.NewSVD(base.Params{
		base.Lr:       0.007,
		base.NEpochs:  100,
		base.NFactors: 80,
		base.Reg:      0.1,
	})
	// Fit model
	svd.Fit(train)
	// Evaluate model
	fmt.Printf("RMSE = %.5f\n", core.RMSE(svd, test))
	// Predict a rating
	fmt.Printf("Predict(4,8) = %.5f\n", svd.Predict(4, 8))
}

./main.go:25:39: not enough arguments in call to core.RMSE
have (*model.SVD, *core.DataSet)
want (core.Model, *core.DataSet, *core.DataSet)

Dependabot can't resolve your Go dependency files

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

go: gonum.org/v1/[email protected]: unrecognized import path "gonum.org/v1/gonum": parse https://gonum.org/v1/gonum?go-get=1: no go-import meta tags ()

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

分页功能:offset/limit

像 popular items / recommended items 之类的接口,现在只能传入一个 number 来限定返回个数。
希望能加入类似 offset/limit 或者 cursor 这样的分页功能,更具实用性。

recommend 配置中once参数不生效

配置文件如下:config.toml

This section declares settings for recommendation.

[recommend]
model = "knn_implicit" # recommendation model
cache_size = 200 # the number of cached recommendations
update_threshold = 5 # update model when more than 10 ratings are added
check_period = 1 # check for update every one minute
similarity = "implicit" # similarity metric for neighbors
once = true # recommend once
fit_jobs = 10 # concurrent jobs for fitting

设置once为true后,调用api接口recommends/100?number=10,每次返回的10条数据都是一样的。
请问下,配置中的once = true 是指一个item在同一个user的推荐中只出现一次么?如果是的话,返回的数据和预期的不一致

关于Restful server 有一些问题。

如果用这个做个内容推荐,类似推特
实现的效果:用户每刷新一次,请求一次推荐,每次的推荐的内容不同。
根据用户的浏览点击和点赞来打分,把事实的数据同步给gorse(新发的推文,浏览量,点赞量打分更新等),
1.每一次请求出不同的推荐结果(已经推荐过的内容不会再推荐)?
2.是否可以控制每次最大推荐的数量(比如一次最多推荐20条条推文) ?

Neighbors endpoint in apidocs not functioning.

In http://127.0.0.1:8087/apidocs/ using an ID from an import of the github.sql tutorial the url generated doesn't seem to be correct and returns an empty json. It is making it uncertain as to what endpoint to use:

Example:
Item ID of 007:hashcash-js results in:

http://127.0.0.1:8087/api/neighbors/{item-id}?item-id=007%3Ahashcash-js

Selection_517

在Ubuntu18.04上go get命令报错

root@VM-0-16-ubuntu:~# go get github.com/zhenghaoz/gorse/...

go.etcd.io/bbolt

/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:13:59: flock redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:13:59
/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:46:22: funlock redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:45:22
/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:56:27: mmap redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:50:27
/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:76:21: munmap redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:72:21
root@VM-0-16-ubuntu:~# cd /usr/local/go
root@VM-0-16-ubuntu:/usr/local/go# go get github.com/zhenghaoz/gorse/...

go.etcd.io/bbolt

/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:13:59: flock redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:13:59
/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:46:22: funlock redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:45:22
/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:56:27: mmap redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:50:27
/home/go/src/go.etcd.io/bbolt/bolt_unix_aix.go:76:21: munmap redeclared in this block
previous declaration at /home/go/src/go.etcd.io/bbolt/bolt_unix.go:72:

请教下如何安装gorse,不胜感激

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.