The rake.go from afjoseph

Hardcoded stop-list path

Hi.

You send us a pull request here avelino/awesome-go#1229.

I've start my review, but I see that hardcoded stopPath should cause a fail here should cause a fail here if file not found, but it should not exist on linux/windows.

Performance degradation caused by commit b66ca2f2b6bf9f4d84f82946e670799b8d2b2e46

This commit fixes #1 but adds pretty heavy loop and result should be cached or stored as a []string but it isn't https://github.com/Obaied/RAKE.Go/blob/b66ca2f2b6bf9f4d84f82946e670799b8d2b2e46/stopwords.go#L584

README currently has invalid import

Easy enough to figure out, but worth mentioning

README.md Example: Stopwords are not Ignored

I believe your commit 6bf7f9f, 'Load stop-words as a string slice instead of splitting a large string', broke stopwords from working. If I rollback to version prior to that commit, everything works fine (git checkout 7df06d19b2795d3b3101a8da3b79efad4c2ce7be).

Running the README.md example (with the import fixed), stopwords are not removed.

package main

import (
	"fmt"

	rake "github.com/afjoseph/RAKE.Go"
)

func main() {
	text := `The growing doubt of human autonomy and reason has created a state of moral confusion where man is left without the guidance of either revelation or reason. The result is the acceptance of a relativistic position which proposes that value judgements and ethical norms are exclusively matters of arbitrary preference and that no objectively valid statement can be made in this realm... But since man cannot live without values and norms, this relativism makes him an easy prey for irrational value systems.`

	candidates := rake.RunRake(text)

	for _, candidate := range candidates {
		fmt.Printf("%s --> %f\n", candidate.Key, candidate.Value)
	}

	fmt.Printf("\nsize: %d\n", len(candidates))
}

a relativistic position --> 9.000000
objectively valid statement --> 9.000000
an easy prey --> 9.000000
value judgements --> 4.000000
human autonomy --> 4.000000
the acceptance --> 4.000000
growing doubt --> 4.000000
relativism makes --> 4.000000
the guidance --> 4.000000
either revelation --> 4.000000
be made --> 4.000000
arbitrary preference --> 4.000000
exclusively matters --> 4.000000
this realm --> 4.000000
moral confusion --> 4.000000
since man --> 3.500000
ethical norms --> 3.500000
norms --> 1.500000
man --> 1.500000
proposes --> 1.000000
irrational --> 1.000000
left --> 1.000000
created --> 1.000000
reason --> 1.000000
state --> 1.000000
values --> 1.000000
result --> 1.000000
systems --> 1.000000
live --> 1.000000
that --> 1.000000

size: 30

I am running:

go version go1.13.1 darwin/amd64

Does it work with foreign languages and utf8?

Does not split words with accents

Try the text "Problème avec les accents".

accents   1
probl   1
è   0

afjoseph / rake.go Goto Github PK

rake.go's People

Contributors

Stargazers

Watchers

Forkers

rake.go's Issues

Hardcoded stop-list path

Performance degradation caused by commit b66ca2f2b6bf9f4d84f82946e670799b8d2b2e46

README currently has invalid import

README.md Example: Stopwords are not Ignored

Does it work with foreign languages and utf8?

Does not split words with accents

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent