Comments (4)
unittest_data.h has two copies of all the data, one in UTF-8 encoding and one
in standard C-string hex escape sequences.
The UTF-8 encoded data is often easier for humans to read. For compilers that
do not support UTF-8 strings, compile with
-Davoid_utf8_string_constants
Did I miss something here, or did you fail to read the comments at the front of
unittest_data.h ? /dick
Original comment by [email protected]
on 23 Oct 2014 at 8:13
- Changed state: WontFix
from cld2.
I missed the comment at the top of the file. Thanks.
Devil's advocate question:
Given that -Davoid_utf8_string_constants should work on all compilers while the
default (without the define) might break, and that the stated reason for the
UTF-8 is for human readability, why not just turn all the UTF-8 strings into
single-line comments and be done with it? Alternatively, why not invert the
#ifdef logic? There doesn't seem to be an obvious reason to have both copies of
the data at all.
OTOH, we could use the redundant copy to verify that indeed your compiler did
compile the UTF-8 text correctly, i.e. that there is a perfect binary match
between both copies of the string.
My vote would be to invert the ifdef so that the default case works everywhere.
I'll set this back to assigned for now, but change defect to enhancement since
and priority to low since it is working as intended :)
Original comment by [email protected]
on 24 Oct 2014 at 7:08
- Added labels: Type-Enhancement, Priority-Low
- Removed labels: Type-Defect, Priority-Medium
from cld2.
Original comment by [email protected]
on 24 Oct 2014 at 7:08
- Changed state: New
from cld2.
Original comment by [email protected]
on 24 Oct 2014 at 7:09
- Changed title: Eliminate redundancy and/or simplify default case for compiling unittest_data.h
from cld2.
Related Issues (20)
- SIGBUS on ARM32 in utf8statetable.cc:517 HOT 13
- compact_lang_det.h: loadDataFromRawAddress should use types from stdint.h instead of "int" HOT 3
- Can't link "dynamic" and "full" HOT 3
- CLD2DynamicDataLoader calls delete instead of delete[] on array types HOT 4
- The ISO 639 code for Hebrew is outdated HOT 1
- Consider declaring dynamic data methods unconditionally HOT 3
- CLD2 result chunk vector omits portions of input file HOT 6
- Dynamic data loading should not use iostream HOT 5
- Windows build fails: undeclared identifier 'close' HOT 6
- Support mmap-ing dynamic data on win32 HOT 5
- Build warning on Windows with clang HOT 2
- Missing include in cld2_dynamic_data_loader.cc HOT 1
- cld2_dynamic_data.cc and cld2_dynamic_data_loader.cc problems on Win32 HOT 10
- Enable dynamic data for 20141015 release HOT 1
- New GCC 5.0 hits problem with narrowing in list-initializers
- CLD should check result of "new" in all use cases
- please use CFLAGS CXXFLAGS CPPFLAGS and LDFLAGS HOT 3
- please provide a SONAME HOT 8
- cld2 testsuite failures HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cld2.