Comments (6)
Cannot reproduce.
I opened http://drugoi.livejournal.com/3971967.html in Firefox and did
copy/paste of all the text into a UTF8 file, then ran
./compact_lang_det_test_chrome0122_2 should_not_be_unk_chrome_8.utf8
and got
ExtLanguage RUSSIAN(80% 1027p), UKRAINIAN(2% 450p), INDONESIAN(0% 637p), 40/45 KB of non-tag letters, Summary: RUSSIAN
SummaryLanguage RUSSIAN at 0 of 46701 2617us (17 MB/sec), should_not_be_unk_chrome_8.utf8
If you are not getting that result, please rerun in your context, setting
kCLDFlagEcho as the flag value in the call to ExtDetectLanguageSummary and send
me stderr (not post or email, which open the possibility of various
svn/web/mail/browser software changing the exact bytes), or run with flags
kCLDFlagHtml | kCLDFlagCr
and send me stderr, or compare to the attached file of the output that I got.
Is it possible that there is an encoding problem and you are not passing clean
UTF-8 to CLD2?
Original comment by [email protected]
on 5 Mar 2014 at 6:26
Attachments:
from cld2.
Seems like we are still using R84. Would this explain the difference?
Original comment by [email protected]
on 6 Mar 2014 at 4:19
from cld2.
No R84 does not explain the difference. Please capture the actual bytes sent to
CLD2. Thanks, /dick
Original comment by [email protected]
on 6 Mar 2014 at 9:54
from cld2.
FWIW, I am planning to roll Chromium to the latest CLD2 in the Very Near(TM)
future.
Original comment by [email protected]
on 11 Mar 2014 at 12:44
from cld2.
Re #4: please try the subject URL http://drugoi.livejournal.com/3971967.html
and send the requested debugging output fomr #1 if the detected language is
Unknown. /dick
Original comment by [email protected]
on 11 Mar 2014 at 6:33
from cld2.
current version of Chrome Version 38.0.2125.104 (64-bit) detects Russian and
translates correctly. Closing as Fixed.
Original comment by [email protected]
on 23 Oct 2014 at 8:18
- Changed state: Fixed
from cld2.
Related Issues (20)
- SIGBUS on ARM32 in utf8statetable.cc:517 HOT 13
- compact_lang_det.h: loadDataFromRawAddress should use types from stdint.h instead of "int" HOT 3
- Can't link "dynamic" and "full" HOT 3
- CLD2DynamicDataLoader calls delete instead of delete[] on array types HOT 4
- The ISO 639 code for Hebrew is outdated HOT 1
- Consider declaring dynamic data methods unconditionally HOT 3
- CLD2 result chunk vector omits portions of input file HOT 6
- Dynamic data loading should not use iostream HOT 5
- Windows build fails: undeclared identifier 'close' HOT 6
- Support mmap-ing dynamic data on win32 HOT 5
- Build warning on Windows with clang HOT 2
- Eliminate redundancy and/or simplify default case for compiling unittest_data.h HOT 4
- Missing include in cld2_dynamic_data_loader.cc HOT 1
- cld2_dynamic_data.cc and cld2_dynamic_data_loader.cc problems on Win32 HOT 10
- Enable dynamic data for 20141015 release HOT 1
- New GCC 5.0 hits problem with narrowing in list-initializers
- CLD should check result of "new" in all use cases
- please use CFLAGS CXXFLAGS CPPFLAGS and LDFLAGS HOT 3
- please provide a SONAME HOT 8
- cld2 testsuite failures HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cld2.