hklemp / dotnet-stop-words Goto Github PK
View Code? Open in Web Editor NEWGet list of common stop words in various languages in dotnet
License: BSD 3-Clause "New" or "Revised" License
Get list of common stop words in various languages in dotnet
License: BSD 3-Clause "New" or "Revised" License
I have been using your library which seems to work fine locally, but when I push it to my microsoft azure app service and try to use it I'm getting a strange error that I've not been able to debug yet. I've found no google results related to the specific "iv" language and I'm not sure where to proceed from here.
I should clarify the first part of the error the serialization exception is from my code trying to decode what was supposed to be a json result but the api returned "there was an error processing your request" which is also part of my code and then the specific exception thrown which appears to be from the stopword package.
[07/13/2020 13:25:31 > d26972: INFO] Newtonsoft.Json.JsonSerializationException: Error converting value "There was an error processing your request: System.ArgumentException: The language iv is not supported
[07/13/2020 13:25:31 > d26972: INFO] at StopWord.StopWords.MapLanguage(String shortLanguageName)
[07/13/2020 13:25:31 > d26972: INFO] at StopWord.StopWordsExtension.Remove(String s, String shortLanguageName)
Our specific use case is this:
string sPunctRemoved = Regex.Replace(sDefn, "[^a-zA-Z0-9]+", " ");
sFilterDefn = StopWord.StopWordsExtension.RemoveStopWords(sPunctRemoved.ToLower());
In this code, the line with the linq => s = s.Split(.....
If the entire input string of 's' is stop words (e.g. 'cannot find'), then the Aggregate call returns null, so it throws an exception on the .ToString().
private static string Remove(string s,string shortLanguageName)
{
var stopWordList = StopWords.GetStopWords(shortLanguageName);
s = s.Split(' ').Where(x => !stopWordList.Contains(x)).DefaultIfEmpty().Aggregate((current, next) => current + " " + next).ToString();
return s;
}
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
.github/workflows/ci.yml
actions/checkout v3
actions/setup-dotnet v3
src/src.csproj
Newtonsoft.Json 13.0.3
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.