clab / fast_align Goto Github PK
View Code? Open in Web Editor NEWSimple, fast unsupervised word aligner
License: Apache License 2.0
Simple, fast unsupervised word aligner
License: Apache License 2.0
Hi,
I followed the steps in issue 33 and saved fwd_params, fwd_err, rev_params and rev_err after running fast_align on the training corpus. Later I compared the alignment results by running fast_align from scratch and running force_align.py with the saved params on that training corpus.
I was expecting fast_align produces exactly the same alignments either from scratch or by force_align, but I got different alignments.
I'm sure I followed the correct steps to run these comparisons, so I'm wondering why the alignments are differnt.
fast_align would function well for English-English alignment?
Hi,
I'm trying to run the fast align on UN parallel dataset. I have tokenized it and running the below command:
./fast_align -i /home/ubuntu/mayub/datasets/raw/un/fastalign_input.txt -d -o -v > /home/ubuntu/mayub/datasets/raw/un/forward_align_en_es.align
it runs for a while with the below output and gives error on Iteration 2: saying it cant read the input file.
Any idea why it is failing ? looping some folks for help. @armatthews @mjdenkowski
Thanks !
Mohammed Ayub
I have installed sparsehash without any troubles. While trying to build fast_align getting the following error:
error:/Users/shreesh/Project/fast_align-master/src/hashtables.h:5:10: fatal error: 'google/sparse_hash_map' file
not found
#include <google/sparse_hash_map>
^~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/fast_align.dir/src/fast_align.cc.o] Error 1
make[1]: *** [CMakeFiles/fast_align.dir/all] Error 2
make: *** [all] Error 2
I tried manually changing the paths in hashtables.h but still couldn't get it to work.
hey , Thank you very much for developing such a useful tool, but I wonder if there is any way to call it without corresponding commands? Can it be called from the python command line?
thank you man .
The log is
.................................................. [20300000]
.................................................. [20350000]
............
expected target length = source length * 0.934537
Killed
how to train this model on a very large data ?
Thanks for creating and maintaining this useful tool!
I'd like to report a bug in the following lines: fast_align.cc#L261~L264
.
Line 264 in 7c2bbca
src.size()
is zero, inf
is added to *tot_len_ratio
. This, in turn, sets mean_srclen_multiplier
to inf
(fast_align.cc#L291), and subsequently log_prob
to inf
.
I think this can be avoided by adding a continue;
statement when src.size()
is zero.
Hope this helps!
How to train this model in a new corpus? I use this model in en->ch, but I find this result is not good in fact, so I abtain a large bilingual corpus and want to train this model again.
Is there a way to trim (pruning) an alignment model?
I have many computational and RAM constraints, so I need to reduce the size of the models.
I tried to reduce the corpus size, but I would like to do it in a more efficient way in terms of quality.
Thanks
Hi,
How many iterations does it run ?
Any idea how long does this take to run finish on a t2.xlarge instance ?
@armatthews
Thanks !
Mohammed Ayub
Hello I have a large parallel corpus on which I used fast-align to get alignments but I have a few other sentences which are different from the original corpus for which I need to get alignment info. Is there a way so that I can use the knowledge learnt from large parallel corpus to directly get alignments for the smaller set of sentences.(Similar to saving checkpoint so that I can load checkpoint and get alignments).
Thanks in advance.
Is there a way to output grow-diag-final-and symmetrization from the fast_align output?
I would like to know what algorithm is basically used to create this tool.
Is there any specific publication that I can read and find more about ?
da.h:58 const double z = ComputeZ(i, n, m, alpha);
this should be changed to const double z = ComputeZ(i, m, n, alpha);
I've already trained on large corpus in parallel to get word alignments. How could I further infer with the word alignments to get the translation probability for the new corpus?
Hello,
we are trying to run force_align after using the fast_align, the fast_align is working fine and we get the model.
After running force_align, we get Traceback (most recent call last):
File "/home/christine/fast_align/build/force_align.py", line 93, in
main()
File "/home/christine/fast_align/build/force_align.py", line 87, in main
sys.stdout.write('{}\n'.format(aligner.align(line.strip())))
File "/home/christine/fast_align/build/force_align.py", line 30, in align
self.fwd_align.stdin.write('{}\n'.format(line))
TypeError: a bytes-like object is required, not 'str'
we are running on python 3.6.9.
we do not know where the problem can be from, we are using the right format of sentence:
The mechanic gave the clerk a present because it was his birthday. ||| La mecánica dio al Cirque durargo, porque era su cumpleaños.
please advise!!
I have problem with larger dataset while using ./fast_align
I found that I could not deal with the data more than 1000 lines maybe. Have you ever met the same problem or how do you usually run the script?
For me it is:
./fast_align -i ~/sockeye_autopilot/systems/wmt14_en_de/data/bpe/new_combined_srctrgaa -d -o -v > forward.align_endea
Thanks a lot!
While training a chinese corpus, my model doesn't seems to train. It is giving the following heuristic at the end of each iteration.
log_2 likelihood: nan
cross entropy: -nan
perplexity: -nan
posterior p0: 0
posterior al-feat: 0
size counts: 33102
Some lines from training corpus:
3月 6日 , 安理会 举行 了 一 次 非公开 会议 ( 第4286 次 会议 ) 南斯拉夫 联盟 共和国 总理 佐兰 · 日日奇 参加 了 会议 。 ||| On 6 March, the Council held a private meeting (4286th) with the participation of the Prime Minister of the Federal Republic of Yugoslavia, Zoran Žižić.
为 了 实现 这个 目标 , 实现 千 年 发展 目标 , 我们 认为 拥有 资源 的 国 家 必须 努力 提供 与 此 挑战 相 适宜 的 资金 。 ||| To achieve that objective and to attain the Millennium Development Goals, we believe that the countries possessing the resources must make a financial effort commensurate with the challenge.
㈡ 在 政府 间 论坛 上 对 咨询 服务 表示 满意 的 机构 的 数目 ||| (ii) Number of institutions expressing satisfaction with advisory services in intergovernmental forums
本 文件 的 增编 详细 介绍 了 中心 在 1995 - 1996年 发挥 的 作用 。 ||| Details of the Centre's role during the period 1995-1996 is provided in an addendum to the present document.
I'm using thulac for tokenizing chinese corpus.
anyone can tell me what four options of atools command mean?
I don't quite understand.
thank you.
Hi! Is there any way to get alignments of new sentences and score of alignment using force_alignment?
Hello!
I'm having troubles aligning a en-jap file. I've seen online people mention using fast align on japanese, couldn't find any guidance tho! Has anyone tried it out?
ARG=i ARG=d ARG=o ARG=v INITIAL PASS Error in line 1 Select a file to blacklist||| 除外 リスト に 追加 する ファイル を 選択 Error in line 2 Clear Usage Data||| 利用 履歴 データ を クリア Error in line 3 Delete records of which files and applications were used:||| ファイル と アプリケーション を 利用 し た 記録 の 削除 : Error in line 4 In the past hour||| 1 時間 以内 expected target length = source length * 0 ITERATION 1 Error in line 4 Select a file to blacklist||| 除外 リスト に 追加 する ファイル を 選択 Error in line 4 Clear Usage Data||| 利用 履歴 データ を クリア Error in line 4 In the past hour||| 1 時間 以内 Error in line 4 Delete records of which files and applications were used:||| ファイル と アプリケーション を 利用 し た 記録 の 削除 : log_e likelihood: 0 log_2 likelihood: 0 cross entropy: -nan perplexity: -nan posterior p0: -nan posterior al-feat: -nan size counts: 4 ITERATION 2 Error in line 4 In the past hour||| 1 時間 以内 Error in line Error in line 4 Delete records of which files and applications were used:||| ファイル と アプリケーション を 利用 し た 記録 の 削除 : 4 Select a file to blacklist||| 除外 リスト に 追加 する ファイル を 選択 Error in line 4 Clear Usage Data||| 利用 履歴 データ を クリア log_e likelihood: 0 log_2 likelihood: 0 cross entropy: -nan perplexity: -nan posterior p0: -nan posterior al-feat: -nan size counts: 4 1 model al-feat: -nan (tension=4) 2 model al-feat: -nan (tension=-nan) 3 model al-feat: -nan (tension=-nan) 4 model al-feat: -nan (tension=-nan) 5 model al-feat: -nan (tension=-nan) 6 model al-feat: -nan (tension=-nan) 7 model al-feat: -nan (tension=-nan) 8 model al-feat: -nan (tension=-nan) final tension: -nan ITERATION 3 Error in line 4 Clear Usage Data||| 利用 履歴 データ を クリア Error in line 4 Delete records of which files and applications were used:||| ファイル と アプリケーション を 利用 し た 記録 の 削除 : Error in line 4 In the past hour||| 1 時間 以内 Error in line 4 Select a file to blacklist||| 除外 リスト に 追加 する ファイル を 選択 log_e likelihood: 0 log_2 likelihood: 0 cross entropy: -nan perplexity: -nan posterior p0: -nan posterior al-feat: -nan size counts: 4 1 model al-feat: -nan (tension=-nan) 2 model al-feat: -nan (tension=-nan) 3 model al-feat: -nan (tension=-nan) 4 model al-feat: -nan (tension=-nan) 5 model al-feat: -nan (tension=-nan) 6 model al-feat: -nan (tension=-nan) 7 model al-feat: -nan (tension=-nan) 8 model al-feat: -nan (tension=-nan) final tension: -nan ITERATION 4 Error in line 4 Select a file to blacklist||| 除外 リスト に 追加 する ファイル を 選択 Error in line 4 Clear Usage Data||| 利用 履歴 データ を クリア Error in line 4 In the past hour||| 1 時間 以内 Error in line 4 Delete records of which files and applications were used:||| ファイル と アプリケーション を 利用 し た 記録 の 削除 : log_e likelihood: 0 log_2 likelihood: 0 cross entropy: -nan perplexity: -nan posterior p0: -nan posterior al-feat: -nan size counts: 4 1 model al-feat: -nan (tension=-nan) 2 model al-feat: -nan (tension=-nan) 3 model al-feat: -nan (tension=-nan) 4 model al-feat: -nan (tension=-nan) 5 model al-feat: -nan (tension=-nan) 6 model al-feat: -nan (tension=-nan) 7 model al-feat: -nan (tension=-nan) 8 model al-feat: -nan (tension=-nan) final tension: -nan ITERATION 5 (FINAL) Error in line 4 In the past hour||| 1 時間 以内 Error in line 4 Delete records of which files and applications were used:||| ファイル と アプリケーション を 利用 し た 記録 の 削除 : Error in line 4 Clear Usage Data||| 利用 履歴 データ を クリア Error in line 4 Select a file to blacklist||| 除外 リスト に 追加 する ファイル を 選択 log_e likelihood: 0 log_2 likelihood: 0 cross entropy: -nan perplexity: -nan posterior p0: -nan posterior al-feat: -nan size counts: 4
I've tried to align the file with and without spaces on the japanese sentences.
Thanks for the amazing free tool :-)
Best, José
The first issue is that Visual Studio doesn't have getopt.h
. I manually imported it from here. But even then, there are a bunch of errors and tons of warnings left.
Errors
'conditional_probability_filename': unknown override specifier C:\tools\fast_align\src\fast_align.cc 59
'Dict::d': uses 'Dict', which is being defined C:\tools\fast_align\src\fast_align.cc 36
'Dict::options': array bound cannot be deduced from a default member initializer C:\tools\fast_align\src\fast_align.cc 92
'i': index variable in OpenMP 'for' statement must have signed integral type C:\tools\fast_align\src\ttables.h 85
'i': index variable in OpenMP 'for' statement must have signed integral type C:\tools\fast_align\src\ttables.h 102
'i': index variable in OpenMP 'for' statement must have signed integral type C:\tools\fast_align\src\ttables.h 160
'input': unknown override specifier C:\tools\fast_align\src\fast_align.cc 58
'input_model_file': unknown override specifier C:\tools\fast_align\src\fast_align.cc 60
missing type specifier - int assumed. Note: C++ does not support default-int C:\tools\fast_align\src\fast_align.cc 31
missing type specifier - int assumed. Note: C++ does not support default-int C:\tools\fast_align\src\fast_align.cc 38
missing type specifier - int assumed. Note: C++ does not support default-int C:\tools\fast_align\src\fast_align.cc 58
missing type specifier - int assumed. Note: C++ does not support default-int C:\tools\fast_align\src\fast_align.cc 124
syntax error: ')' C:\tools\fast_align\src\corpus.h 19
syntax error: ')' C:\tools\fast_align\src\corpus.h 66
syntax error: '=' C:\tools\fast_align\src\fast_align.cc 59
syntax error: '=' C:\tools\fast_align\src\fast_align.cc 60
syntax error: 'string' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\assert.h 14
syntax error: 'string' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\assert.h 14
syntax error: 'using namespace' C:\tools\fast_align\src\fast_align.cc 28
syntax error: identifier 'vector' C:\tools\fast_align\src\fast_align.cc 224
syntax error: identifier 'vector' C:\tools\fast_align\src\fast_align.cc 238
syntax error: missing ')' before ';' C:\tools\fast_align\src\corpus.h 66
syntax error: missing ',' before '&' C:\tools\fast_align\src\fast_align.cc 38
syntax error: missing ',' before '<' C:\tools\fast_align\src\fast_align.cc 31
syntax error: missing ',' before '<' C:\tools\fast_align\src\fast_align.cc 124
unexpected end-of-file found C:\tools\fast_align\src\fast_align.cc 492
unexpected token(s) preceding ':'; skipping apparent function body C:\tools\fast_align\src\corpus.h 19
unexpected token(s) preceding ';' C:\tools\fast_align\src\corpus.h 66
unexpected token(s) preceding ';' C:\tools\fast_align\src\fast_align.cc 28
unexpected token(s) preceding ';' C:\tools\fast_align\src\fast_align.cc 59
unexpected token(s) preceding ';' C:\tools\fast_align\src\fast_align.cc 60
unexpected token(s) preceding '{'; skipping apparent function body C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\assert.h 14
unexpected token(s) preceding '{'; skipping apparent function body C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\assert.h 14
Warnings (mainly OS specific)
Description File Line
'=': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 120
'=': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 120
'=': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\tools\fast_align\src\getopt.h 309
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1062
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1064
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1066
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1068
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1152
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1156
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1374
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1376
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1378
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1380
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1062
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1064
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1066
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1068
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1152
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1156
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1374
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1376
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1378
'=': conversion from 'int' to 'unsigned long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1380
'argument': conversion from 'const int' to 'const unsigned int', signed/unsigned mismatch C:\tools\fast_align\src\ttables.h 58
'argument': conversion from 'const int' to 'const unsigned int', signed/unsigned mismatch C:\tools\fast_align\src\ttables.h 59
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1625
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1691
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1720
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1741
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1874
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1940
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1969
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1990
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2520
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2533
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1625
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1691
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1720
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1741
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1874
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1940
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1969
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1990
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2520
'argument': conversion from 'int' to 'size_t', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2533
'argument': conversion from 'std::_Atomic_integral_t' to 'long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\memory 778
'argument': conversion from 'std::_Atomic_integral_t' to 'long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\memory 778
'Dict': '3' bytes padding added after data member 'Dict::force_align' C:\tools\fast_align\src\fast_align.cc 74
'Dict': '4' bytes padding added after data member 'Dict::d' C:\tools\fast_align\src\fast_align.cc 61
'Dict': '4' bytes padding added after data member 'Dict::favor_diagonal' C:\tools\fast_align\src\fast_align.cc 65
'Dict::TTable': '2' bytes padding added after data member 'Dict::TTable::probs_initialized_' C:\tools\fast_align\src\ttables.h 176
'return': conversion from 'std::_Atomic_integral_t' to 'long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\memory 823
'return': conversion from 'std::_Atomic_integral_t' to 'long', signed/unsigned mismatch C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\memory 823
'std::basic_filebuf<char,std::char_traits<char>>': '2' bytes padding added after data member 'std::basic_filebuf<char,std::char_traits<char>>::_Wrotesome' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\fstream 742
'std::basic_filebuf<char,std::char_traits<char>>': '3' bytes padding added after data member 'std::basic_filebuf<char,std::char_traits<char>>::_Closef' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\fstream 744
'std::basic_ifstream<char,std::char_traits<char>>': '4' bytes padding added after data member 'std::basic_ifstream<char,std::char_traits<char>>::_Filebuffer' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\fstream 1000
'std::basic_ios<char,std::char_traits<char>>': '7' bytes padding added after data member 'std::basic_ios<char,std::char_traits<char>>::_Fillch' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\ios 192
'std::basic_ios<char,std::char_traits<char>>': '7' bytes padding added after data member 'std::basic_ios<char,std::char_traits<char>>::_Fillch' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\ios 192
'std::basic_ios<wchar_t,std::char_traits<wchar_t>>': '6' bytes padding added after data member 'std::basic_ios<wchar_t,std::char_traits<wchar_t>>::_Fillch' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\ios 192
'std::basic_ios<wchar_t,std::char_traits<wchar_t>>': '6' bytes padding added after data member 'std::basic_ios<wchar_t,std::char_traits<wchar_t>>::_Fillch' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\ios 192
'std::basic_iostream<char,std::char_traits<char>>': '4' bytes padding added after base class 'std::basic_ostream<char,std::char_traits<char>>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<char,std::char_traits<char>>': '4' bytes padding added after base class 'std::basic_ostream<char,std::char_traits<char>>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<char,std::char_traits<char>>': '4' bytes padding added after data member '<alignment member>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<char,std::char_traits<char>>': '4' bytes padding added after data member '<alignment member>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<wchar_t,std::char_traits<wchar_t>>': '4' bytes padding added after base class 'std::basic_ostream<wchar_t,std::char_traits<wchar_t>>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<wchar_t,std::char_traits<wchar_t>>': '4' bytes padding added after base class 'std::basic_ostream<wchar_t,std::char_traits<wchar_t>>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<wchar_t,std::char_traits<wchar_t>>': '4' bytes padding added after data member '<alignment member>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_iostream<wchar_t,std::char_traits<wchar_t>>': '4' bytes padding added after data member '<alignment member>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 970
'std::basic_istream<char,std::char_traits<char>>': '4' bytes padding added after data member 'std::basic_istream<char,std::char_traits<char>>::_Chcount' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 879
'std::basic_istream<char,std::char_traits<char>>': '4' bytes padding added after data member 'std::basic_istream<char,std::char_traits<char>>::_Chcount' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 879
'std::basic_istream<wchar_t,std::char_traits<wchar_t>>': '4' bytes padding added after data member 'std::basic_istream<wchar_t,std::char_traits<wchar_t>>::_Chcount' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 879
'std::basic_istream<wchar_t,std::char_traits<wchar_t>>': '4' bytes padding added after data member 'std::basic_istream<wchar_t,std::char_traits<wchar_t>>::_Chcount' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\istream 879
'std::basic_ofstream<char,std::char_traits<char>>': '4' bytes padding added after data member '<alignment member>' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\fstream 1236
'std::basic_ofstream<char,std::char_traits<char>>': '4' bytes padding added after data member 'std::basic_ofstream<char,std::char_traits<char>>::_Filebuffer' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\fstream 1236
'std::codecvt<char,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 921
'std::codecvt<char,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 921
'std::codecvt<char,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 921
'std::codecvt<char,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 921
'std::codecvt<char16_t,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1266
'std::codecvt<char16_t,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1266
'std::codecvt<char16_t,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1266
'std::codecvt<char16_t,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1266
'std::codecvt<char32_t,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1533
'std::codecvt<char32_t,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1533
'std::codecvt<char32_t,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1533
'std::codecvt<char32_t,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1533
'std::codecvt<unsigned short,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2030
'std::codecvt<unsigned short,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2030
'std::codecvt<unsigned short,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2030
'std::codecvt<unsigned short,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2030
'std::codecvt<wchar_t,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1781
'std::codecvt<wchar_t,char,_Mbstatet>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1781
'std::codecvt<wchar_t,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1781
'std::codecvt<wchar_t,char,_Mbstatet>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 1781
'std::codecvt_base': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 797
'std::codecvt_base': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 797
'std::codecvt_base': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 797
'std::codecvt_base': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 797
'std::ctype<char>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2539
'std::ctype<char>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2539
'std::ctype<char>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2539
'std::ctype<char>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2539
'std::ctype<unsigned short>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2983
'std::ctype<unsigned short>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2983
'std::ctype<unsigned short>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2983
'std::ctype<unsigned short>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2983
'std::ctype<wchar_t>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2760
'std::ctype<wchar_t>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2760
'std::ctype<wchar_t>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2760
'std::ctype<wchar_t>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2760
'std::ctype_base': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2078
'std::ctype_base': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2078
'std::ctype_base': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2078
'std::ctype_base': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 2078
'std::ctype_byname<char>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 3028
'std::ctype_byname<char>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 3028
'std::ctype_byname<char>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 3028
'std::ctype_byname<char>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 3028
'std::ios_base': '4' bytes padding added after data member 'std::ios_base::_Ploc' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xiosbase 647
'std::ios_base': '4' bytes padding added after data member 'std::ios_base::_Ploc' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xiosbase 647
'std::locale': '3' bytes padding added after base class 'std::_Crt_new_delete' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 484
'std::locale': '3' bytes padding added after base class 'std::_Crt_new_delete' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 484
'std::locale::_Locimp': '3' bytes padding added after data member 'std::locale::_Locimp::_Xparent' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 243
'std::locale::_Locimp': '3' bytes padding added after data member 'std::locale::_Locimp::_Xparent' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 243
'std::numpunct<char>': '2' bytes padding added after data member 'std::numpunct<char>::_Kseparator' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 265
'std::numpunct<char>': '2' bytes padding added after data member 'std::numpunct<char>::_Kseparator' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 265
'std::numpunct<char>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<char>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<char>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<char>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<wchar_t>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<wchar_t>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<wchar_t>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::numpunct<wchar_t>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 267
'std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<wchar_t,std::istreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<wchar_t,std::istreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<wchar_t,std::istreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_get<wchar_t,std::istreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1220
'std::num_put<char,std::ostreambuf_iterator<char,std::char_traits<char>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<char,std::ostreambuf_iterator<char,std::char_traits<char>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<char,std::ostreambuf_iterator<char,std::char_traits<char>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<char,std::ostreambuf_iterator<char,std::char_traits<char>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<wchar_t,std::ostreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<wchar_t,std::ostreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<wchar_t,std::ostreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::num_put<wchar_t,std::ostreambuf_iterator<wchar_t,std::char_traits<wchar_t>>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocnum 1664
'std::pair<const _Kty,_Ty>': '4' bytes padding added after data member 'std::pair<const _Kty,_Ty>::first' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\utility 293
'std::pair<std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<_Ty>>>,bool>': '3' bytes padding added after data member 'std::pair<std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<_Ty>>>,bool>::second' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\utility 294
'std::_Generic_error_category': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': move assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': move assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': move constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Generic_error_category': move constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 498
'std::_Hash<std::_Umap_traits<_Kty,_Ty,std::_Uhash_compare<_Kty,_Hasher,_Keyeq>,_Alloc,false>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xhash 1178
'std::_Hash<std::_Umap_traits<_Kty,_Ty,std::_Uhash_compare<_Kty,_Hasher,_Keyeq>,_Alloc,false>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xhash 1178
'std::_Iostream_error_category': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': move assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': move assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': move constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_Iostream_error_category': move constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 526
'std::_System_error_category': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': move assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': move assignment operator was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': move constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_System_error_category': move constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\system_error 574
'std::_Tree<std::_Tset_traits<_Kty,_Pr,_Alloc,false>>': copy constructor was implicitly defined as deleted C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xtree 2420
'std::_Yarn<char>': '3' bytes padding added after data member 'std::_Yarn<char>::_Nul' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xutility 4231
'std::_Yarn<char>': '3' bytes padding added after data member 'std::_Yarn<char>::_Nul' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xutility 4231
'std::_Yarn<wchar_t>': '2' bytes padding added after data member 'std::_Yarn<wchar_t>::_Nul' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xutility 4231
'std::_Yarn<wchar_t>': '2' bytes padding added after data member 'std::_Yarn<wchar_t>::_Nul' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xutility 4231
'TpSetCallbackCleanupGroup': pointer or reference to potentially throwing function passed to extern C function under -EHc. Undefined behavior may occur if this function throws an exception. C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winbase.h 7383
'TTable': '2' bytes padding added after data member 'TTable::probs_initialized_' C:\tools\fast_align\src\ttables.h 176
'_WIN32_WINNT_WIN10_RS1' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8916
'_WIN32_WINNT_WIN10_RS1' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8961
'_WIN32_WINNT_WIN10_RS1' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8969
'_WIN32_WINNT_WIN10_RS1' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 14178
'_WIN32_WINNT_WIN10_RS2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8932
'_WIN32_WINNT_WIN10_RS2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8942
'_WIN32_WINNT_WIN10_RS2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 9159
'_WIN32_WINNT_WIN10_RS2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13631
'_WIN32_WINNT_WIN10_RS2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13684
'_WIN32_WINNT_WIN10_RS3' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8951
'_WIN32_WINNT_WIN10_RS3' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13648
'_WIN32_WINNT_WIN10_RS3' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13665
'_WIN32_WINNT_WIN10_RS3' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13803
'_WIN32_WINNT_WIN10_RS4' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8958
'_WIN32_WINNT_WIN10_RS4' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8964
'_WIN32_WINNT_WIN10_RS4' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 9726
'_WIN32_WINNT_WIN10_RS4' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13953
'_WIN32_WINNT_WIN10_RS4' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 13966
'_WIN32_WINNT_WIN10_TH2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8910
'_WIN32_WINNT_WIN10_TH2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8921
'_WIN32_WINNT_WIN10_TH2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8925
'_WIN32_WINNT_WIN10_TH2' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif' C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um\winioctl.h 8946
'__std_exception_data': '3' bytes padding added after data member '__std_exception_data::_DoFree' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\vcruntime_exception.h 25
'__std_exception_data': '3' bytes padding added after data member '__std_exception_data::_DoFree' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\vcruntime_exception.h 25
'__std_type_info_data': '3' bytes padding added after data member '__std_type_info_data::_DecoratedName' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\vcruntime_typeinfo.h 44
'__std_type_info_data': '3' bytes padding added after data member '__std_type_info_data::_DecoratedName' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\vcruntime_typeinfo.h 44
expected 'newline'; found ';' C:\tools\fast_align\src\getopt.h 59
ignoring unknown option '-g' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\cl 1
ignoring unknown option '-g' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\cl 1
ignoring unknown option '-O3' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\cl 1
ignoring unknown option '-O3' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\cl 1
ignoring unknown option '-std=c++11' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\cl 1
ignoring unknown option '-std=c++11' C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\cl 1
Informational: catch(...) semantics changed since Visual C++ 7.1; structured exceptions (SEH) are no longer caught C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 326
Informational: catch(...) semantics changed since Visual C++ 7.1; structured exceptions (SEH) are no longer caught C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 352
Informational: catch(...) semantics changed since Visual C++ 7.1; structured exceptions (SEH) are no longer caught C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 326
Informational: catch(...) semantics changed since Visual C++ 7.1; structured exceptions (SEH) are no longer caught C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include\xlocale 352
nonstandard extension used: zero-sized array in struct/union C:\tools\fast_align\src\fast_align.cc 92
not enough actual parameters for macro 'max' C:\tools\fast_align\src\corpus.h 19
I'm not a C (or its siblings) developer, so I don't know where to start on this. But it's definitely a bug. Windows support would be nice.
I followed the steps mention to setup fast align in my system. However, i am facing some difficulties.
Can anyone tell me exact procedure?
Do I have to train first and then calculate alignment on my test data?
Any python based approach?
Hi,
I'm trying to run this on some English-spanish datasets. if you could please explain the below statistics I'm getting. Not sure if this means good results or bad...
What do these stats means -
log_e likelihood: -7.98726e+09
log_2 likelihood: -1.15232e+10
cross entropy: 29.8974
perplexity: 1e+09
posterior p0: 0.08
posterior al-feat: -0.165795
size counts: 30503
Thanks !
Mohammed Ayub
Hi,
When running the symmetrization with atools and grow-diag-final-and heuristics on the following links, the symmetrized links show an additional token (the last one, 27).
Forward
0-0 1-1 2-2 3-3 4-4 5-5 7-6 6-7 8-8 9-9 10-10 11-11 12-13 13-14 14-15 15-16 18-17 16-18 17-19 19-20 20-21 21-22 22-23 22-24 22-25 23-26
Reversed
0-0 1-1 2-2 3-3 4-4 5-5 6-7 7-6 8-8 9-9 10-10 11-11 12-13 13-14 14-15 15-16 16-17 17-19 18-18 19-20 20-21 21-22 22-25 23-26
Symmetrized
0-0 1-1 2-2 3-3 4-4 5-5 6-7 7-6 8-8 9-9 10-10 11-11 12-13 13-14 14-15 15-16 16-17 16-18 17-19 18-18 18-20 19-20 19-21 20-22 21-23 22-24 22-25 22-26 23-27
This does not happen to all lines, though. Is this a bug or it is something to be expected in certain cases?
Thanks for the amazingly fast alignment tool, I understand that fast_align gives forward and backwards word alignments.
A short tutorial-like answer to the 2 questions would help a lot for users to use fast_align in the MT toolkit pipelines.
I have an English->German corpus with ~7GB built from (Pattr, Europarl and News Comments), max sentence length of 80 chars. fast_align crashes in iteration 1.
ITERATION 1
.................................................. [50000]
.................................................. [100000]
.................................................. [150000]
.................................................. [200000]
.................................................. [250000]
...
.................................................. [5650000]
Killed: 9
It is a memory problem. Process gets killed because of the memory consumption.
I avoided the problem by using a machine with a lot of memory.
The document linked in README:
Chris Dyer, Victor Chahuneau, and Noah A. Smith. (2013). A Simple, Fast, and Effective Reparameterization of IBM Model 2. In Proc. of NAACL.
http://www.ark.cs.cmu.edu/cdyer/fast_valign.pdf
is giving 404
When running this library the stdout looks similar to the following:
ARG=i
ARG=v
ARG=d
ARG=o
INITIAL PASS
.................................................. [50000]
.................................................. [100000]
.................................................. [150000]
.................................................. [200000]
expected target length = source length * 3.7347
Is there an option to hide this?
shouldn't line 225 use j+1 instead of j?
emp_feat += DiagonalAlignment::Feature(j, i, trg.size(), src.size()) * p;
in the surrounding loops, i ranges from 1 to src.size() inclusive and j from 0 to trg.size() exclusive. Further up in the code, j+1 is used as well, and I think the feature calculation assumes the same.
This bug would bias it towards a flatter distribution I think, but I'm not sure.
Hi, I'm using fast_align to align a small corpus but it keeps showing this error message after run fast_align
:
Error in line 200000
||| 奥巴马 主义 的 长期 影响 需要 更 长时间 才能 评估 , 但是 , 随着 11 月 大选 的 临近 , 奥巴马 似乎 在 与其 外交政策 的 反对者 的 斗争 中 占据 了 上风 。
Error in line 200000
||| 现在 , 日本 对 美国 的 出口 在 下降 然而 日元 却 在 走强 。
Error in line 200000
||| 而且 , 对 一个 所有 石油 需求 都 依靠 进口 的 国家 , 高油价 以及 商业利润 和 信心 的 下降 也 在 将 日本 推入 衰退 。
In the end,
log_e likelihood: nan
log_2 likelihood: nan
cross entropy: -nan
perplexity: -nan
posterior p0: 0
posterior al-feat: 0
size counts: 5047
What might go wrong here?
Does this mean that my corpus is not well aligned and thus I cannot learn a good alignment model?
Thank you!
According to the help message (fast_align.cc:307) the option -c outputs a conditional probability table, but the actual option appears to be -p (fast_align.cc:89).
The link that is supposed to get to paper "A Simple, Fast, and Effective Reparameterization of IBM Model 2" is expired. It's now showing 404 error.
Is it possible to convert the alignment output into a phrase table:
source ||| target
source ||| target
source ||| target
?
I don‘t know how to get the alignment reference and training data reported in the paper, could you give me some clue ?
./fast_align -i train.en-zh -d -o -v > forward.align
bash: ./fast_align: 没有那个文件或目录(There is no such file or directory)
why?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.