sgg / brokaw Goto Github PK
View Code? Open in Web Editor NEWA modern Usenet library in Rust 📰
License: MIT License
A modern Usenet library in Rust 📰
License: MIT License
Add the ability to post articles. Now would probably be a good time to introduce some integration tests!
POST
is particularly tricky because (like IHAVE
) it is a two stage command. If we model this incorrectly we could end up with an inconsistent stream state. The workflow is as follows:
POST
NntpClient
so that users do not have to deal with the stateful nature themselvesNntpClient
NntpConnection
NntpConnection
w/ typed ResponsesFor better isolation we could spin up innd
in a container as reaching out to public servers is brittle (and potentially annoying for the owners)
Implement Astraweb-style compression. This could be provided as part of overviews in the NntpClient and as separate deserialization functions.
Astraweb-style header compression leverages two custom commands, XZVER
and XZHDR
.
The closes thing I've found to documentatoin is an old post from their helpdesk (pasted below).
============================================================
We are proposing two new commands as an extension to the NNTP protocol: XZVER and XZHDR as counterparts to XOVER and XHDR respectively.
These two new optional commands will appear in the list returned from the "HELP" command.
Request: XZVER [range]
Response:
224 compressed data follows
=ybegin line=128 size=-1
=yend crc=FFFFFFFF
.
Request: XZHDR header [range]
Response
221
compressed data follows
=ybegin line=128 size=-1
=yend crc=FFFFFFFF
.
Where [range] is in the standard article number range as per XOVER.
Notes:
- The multi-line responses conform to standard NNTP protocol with a single period "." on a line by itself.
- Line terminals are \r\n
- The actual XOVER / XHDR response is compressed with zlib, then yEnc encoded.
- The compression conforms with zlib's deflate() method and there is NO gzip done on it. ie. Just pure gzip'ed data with no extra padding info to make it gzip-file compliant.
- The yEnc spec requires that the "size" parameter in "=ybegin line=123 size=456" be present, but for efficiency and performance sake, the total size is not known ahead of time so it has been set to some arbitrary value that should be ignored. The yEnc CRC32 value should be sufficient to validate the payload. On top of that, zlib has builtin adler32 which is an alternate form of CRC32.
It is assumed that the implementation for XZVER be duplicated from XOVER, with two additional steps at the end, that being: yEnc decode the data, then zlib inflate the data.
============================================================
Brokaw supports NNTP over TLS (also known as "NNTPS") but it does not support opportunistic TLS (STARTTLS
) as described in RFC 4642.
Giganews (and decedents) use compression via an XFEATURES
extension. This extension does not seem to be documented publically but some old newsposts indicate some folks Giganews might have a spec.
I have done some reverse engineering and will eventually put the notes here.
We allocate a buffer JIT if NntpConnection
encounters a multiline response. We should preallocate this and store it with the connection.
Building, testing, rustfmt, etc via GH Actions.
All (?) of Brokaw's iterators are over fixed collections so they could ExactSizeIterator
.
Implement the compression as described in RFC 8054. Humorously none of the servers I use even seem to use this.
A REPL-like Newsreader CLI would make for a cute example
Capabilities are currently string based. The common ones could be typed allowing for feature fallback/selection in NntpClient
without a bunch of String look ups in the hashmap.
Much of the commands
module is boilerplate code and couple be replaced w/ a couple proc-macros
The client half of the types provided by Brokaw more or less boils down to two things:
ToString
Encode
)TryFrom<RawResponse>
)One could extend Brokaw to support server-side connections by implementing (1) command decoding and (2) response encoding.
It might be worthwhile to reconsider the use of ToString
as a trait for command encoding as it prevents Brokaw from supporting non UTF-8 messages (tracked in #26)
Command serialization is currently built on Display
. This implicitly requires that commands be entirely UTF-8. To support i18n and binary payloads we should decouple serialization from Display
.
HEAD
, BODY
and STAT
LIST
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.