ykaliuta / fidogate Goto Github PK

System is configured with 2 AKAs in the same network: 2:463/1331 & 2:463/0 and BEST_AKA option is enabled during fidogate build time.
EchoMail packet is created like this:

$ cat text.txt | ftnoutpkt -f "SysOp@2:463/0" -s "Test Subject" -A HUGAYDA.TEST "All@2:463/0"

debug shows best AKA works:

...
Select best AKA: 2:463/0.0  Uplink: 0:0/0.0
...

However, when runinc and then ftntoss processes the packet, it's being discarded due to circular path:

...
Select Z2 AKA: 2:463/1331.0  Uplink: 0:0/0.0
SEEN+BY: 2:463/0
Path   : 2:463/0
SEEN+BY: 2:463/0 2:463/1331
Path   : 2:463/0 2:463/1331
New    : 2:463/1331.1
May 18 13:07:33 ftntoss circular path echomail area HUGAYDA.TEST from 2:463/0.0 to 2:463/0.0
...

This will work, if I change order of addresses so 2:463/0 will come as first & default AKA. However, then failures will appear for packets made with 2nd AKA.

I suggest to add into the messages that are copied to the carbon group the Followup-To: header in which there will be no carbon group included. This will make it possible to not crosspost in it when answering from the NNTP client.

Hello! Is it possible to install this package fidogate to centos 7 ?
I'll try to: rpm -i fidogate-5.12-1.x86_64.rpm but have a problem with perl:
error: Failed dependencies:
perl(getopts.pl) is needed by fidogate-5.12-1.x86_64
I have installed perl, but no idea what they want....

It seems, impossible to send message from nntp to ftn, if areaname contain "-" symbol.
E.g. 5020-723.LOCAL. Logs said:

Oct 11 14:20:06 ftntoss node 2:5034/10.999 not authorized to create area 5020|723.LOCAL (config restriction)
Oct 11 14:20:06 ftntoss unknown area 5020|723.LOCAL from 2:5034/10.999

However, the area aleady exist as 5020-723.LOCAL. Echomail from ftn to nntp looks good.

Seems, if message has multi-part structure and if one of parts were lost:
/usr/local/fido/gate/bin/runinc missing: 21 Jun 16 03:20:44 @770/3 00000 02 / 02 message <> incomplete!

Is it possible to generate the full thread chain in the References field (header) when doing ftn2rfc?
RFC2822 says:

The "References:" field will contain the contents of the parent's "References:" field (if any) followed by the contents of the parent's "Message-ID:" field (if any). If the parent message does not contain a "References:" field but does have an "In-Reply-To:" field containing a single message identifier, then the "References:" field will contain the contents of the parent's "In-Reply-To:" field followed by the contents of the parent's "Message-ID:" field (if any). If the parent has none of the "References:", "In-Reply-To:", or "Message-ID:" fields, then the new message will have no "References:" field.

I know that the References field's message-ids chain cannot be included unsing just only the FTN messages (because there are none of them), so it should be looked up from the innd grepgistory/sm like the xct wrapper does with "From:" fields. So, the best way I see is to include one more processing wrapper into the innd batches which will complement the full References: chain with message-ids from parent messages.

How does it work at the moment?
Only the parent message-id is included in the References field, without contents of the parent's "References" fields.

Why this issue is important?
With the full References in RFC messages path there will be no more teared and broken threads in NNTP clients with bad threads building alghorithms.

I suggest to add fourth optional parameter into charset translation schemas that will set up the default fallback charset for rfc2ftn gating. Charset should be assumed as fallback when Content-Type header is in RFC article. There are some groups where default charset is UTF-8, but some old clients are using old 8-bit charsets without Content-Type header. In russian groups is is koi8-r or windows-1251, in english groups it can be us-ascii, iso8859-1 or windows-1252.

Example:
DefaultCharset cp866:cp866:utf-8:koi8-r
In this case:
rfc2ftn:

1. FTN message without @chrs should be threated as written in CP866 and recoded to UTF-8
2. FTN message with @chrs should be threated as written in charset that in @chrs kludge and recored to UTF-8

ftn2rfc:

1. RFC message without charset in Content-Type header should be threated as KOI8-R and recoded to CP866
2. RFC message with charset in Content-Type header should be threated as written in that charset and recoded to CP866

Also, I suggest to add special "passthru" charset for rfc2ftn and ftn2rfc schemas which would indicate that no transcoding is needed, only correct headers and kludges.

Examples:
cp866:utf-8:passthru:utf-8
FTN message with LATIN-1 CHRS kludge should not be recoded to UTF-8 when gating ftn2rfc but iso-8859-1 charset should be added to Content-Type header.
Also, ASCII -> us-ascii, LATIN-5 -> iso-8859-9, etc. Unknown charsets should be threated as written in CP866 and recoded as usual.

cp866:passthru:utf-8:utf-8
RFC message with iso-8859-1 in Content-Type header shold not be recoded to CP866 when gating rfc2ftn but CHRS: LATIN-1 kludge should be added.
Also, us-ascii -> ASCII, ibm866 or cp866 -> CP866, etc. Unknown charsets should be threated as written in UTF-8 and recoded as usual.

If option -b is present in the areas config file, for example:
"" fido. -8 -H -z 2 -b -a 2:5034/10.999
so during encoding, headers will contain additional empty lines:
[skip]
Organization: =?windows-1251?B?SODh7vAg7e7i+/Ug6+jt6u7iIC0gaHR0cDovL2FsdHluY2x1Yi5ydS9maWRv?=

=?windows-1251?B?bmV0Lmh0bWwK?=
Lines: 13
X-Gateway: FIDO 5034.ru [FIDOGATE 5.2.4ds]
X-FTN-From: Dima Bargamov @ 2:5020/570.0
X-FTN-Flags:
X-FTN-Tearline: GoldED-NSF (altyn.local)
X-FTN-Origin: =?windows-1251?B?SODh7vAg7e7i+/Ug6+jt6u7iIC0gaHR0cDovL2FsdHluY2x1Yi5ydS9maWRv?=

=?windows-1251?B?bmV0Lmh0bWwgKDI6NTAyMC81NzApCg==?=
[skip]

as result it cause incorrect display messages in some newsreaders (for example Mozilla Thunderbird)

sniffer result:
http://s11.radikal.ru/i183/1701/b7/374f3dffab28t.jpg

When gating in both sides, date, time and timezone should use timezone from headers, not the timezone of the Fidogate server, when UseTZUTCKludge option is set.

Actual behaviour

RFC:
Date: Mon, 17 Feb 2020 07:32:23 +0400
FTN:
Date: 2020-02-17 06:32:22
@TZUTC: 0300

Expected behaviour

RFC:
Date: Mon, 17 Feb 2020 07:32:23 +0400
FTN:
Date: 2020-02-17 07:32:23
@TZUTC: 0400

Configuration

Timezone on Fidogate server: Europe/Moscow (UTC+0300)
Timezone of NNTP client: Europe/Samara (UTC+0400)
Timezone of FTN client: Europe/Samara (UTC+0400)

For subjects 29-35 characters long part of mime encoding left, for longer the real subject is replaced with "(no subject)"

Related to issue #5 , the long subjects handling is still broken.

charset.bin was not overwritten after "make install" command

When ftn2rfc is gating from cyrillic CP866 charset into UTF-8 encoded with base64 in cases of long strings the headers are splitting into several lines. It is ok, but Fidogare often splits these lines in the wrong place: in the middle of the UTF8 multibyte sequence, which means that the next line not begin with an UTF sequence when decoding from base64. This causes some email clients and newsreaders to cut the Headers like Subject or Organization when decoding from base64. It would be necessary to fix this algorithm in accordance with RFC2047 and not break header lines in the middle of the letters.
RFC 2047 specifically forbids this:

Each 'encoded-word' MUST encode an integral number of octets. The
'encoded-text' in each 'encoded-word' must be well-formed according
to the encoding specified; the 'encoded-text' may not be continued in
the next 'encoded-word'. (For example, "=?charset?Q?=?=
=?charset?Q?AB?=" would be illegal, because the two hex digits "AB"
must follow the "=" in the same 'encoded-word'.)

Each 'encoded-word' MUST represent an integral number of characters.
A multi-octet character may not be split across adjacent 'encoded-
word's.

Examples:

• FTN Subject:
  А почему не пошла в магазин и не купила телячью отбивную?
  RFC:
  Subject: =?utf-8?B?0JAg0L/QvtGH0LXQvNGDINC90LUg0L/QvtGI0LvQsCDQsiDQvNCw?=
=?utf-8?B?0LPQsNC30LjQvSDQuCDQvdC1INC60YPQv9C40LvQsCDRgtC10LvRj9GH0YzR?=
=?utf-8?B?jiDQvtGC0LHQuNCy0L3Rg9GOPwo=?=
• FTN Subject:
  Это ваше ФИДО (переиздание)
  RFC:
  Subject: =?utf-8?B?0K3RgtC+INCy0LDRiNC1INCk0JjQlNCeICjQv9C10YDQtdC40LfQ?=
=?utf-8?B?tNCw0L3QuNC1KQo=?=
• FTN Origin:
  А Ларчик просто открывался...
  RFC:
  Organization: =?utf-8?B?0JAg0JvQsNGA0YfQuNC6INC/0YDQvtGB0YLQviDQvtGC0LrR?=
=?utf-8?B?gNGL0LLQsNC70YHRjy4uLgo=?=

`

areas config file has undocumented options: -b and -Q
For example:
"" fido. -8 -H -z 2 <-b or -Q> -a 2:5034/10.999