Current code has hardcoded default values for invoice (see here)

We should allow to override those default values by using environment variables and use hardcoded values only if no env vars are provided.

As per this issue there is now a release of the relay pagination implementation.

This issue is to replace the mere copy of the code with the use of the external crate

The demo should implement a user system that would allow to build auth guarded endpoint in order to restrict some operations to the authenticated users.

With roles added we expect each role to have access to different features.
Therefore, it would be useful to provide the authed user scope as part of the sign-in action, so front-end interfaces can build an ACL on top of it.

This could be done by generating an additional cookie that would provide the scope of current user.

Licence shall be MIT or a licence that would keep the code open-source and forbid any private patenting on the feature.

Another interesting mechanism could be to wrap a whole API endpoint with the paywall.

In order to do so, it could be expected from client to provide a header field that would be the payment request which would then be processed by the API to ensure the payment is effective.

We would need the following elements for implementation :

• #12
• #13
• #14

Current global paywall goes on top of queries paywall.

The PoC should provide a queries paywall go-through.

A quick way to do it is to split the get_post in different methods for each endpoint case.

Allowing for invoice expiring should be an optional flag into payment table in order to allow the business logic to invalidate an invoice based on time-availability.

the current feature implementation should also implement the business logic onto the HTTP file access endpoint

We fixed the CI using stable v1.57 of Rust.

As it seems that it works this means we might not need nightly builds anymore and more over that we could step ahead to latest stable versions. We should give a try about it

The current local_cache calls the api_payment generation on each call.

Rocket documentation's specifies that the local_cache should call the f method ONLY if local had no previous state.
This is probably due to the async property of the api_payment and invoice generation.

To fix this we should call local_cache_async method instead.

Current version of the server only allows for file upload when logged-in.

It'd be useful to provide also tools to edit media's metadata and allow media removal.

A hard coded value is pushed to generate invoice for post :
https://github.com/Asone/graphQLN/blob/c86a9c2dca57ba69835775d7a8765dd30c7e5c7f/src/lnd/invoice.rs#L90

We might consider handling a specific expiry for each post by adding a column in the post table that would provide such value.

As it is not implemented - yet - we should rather use the default LN invoice expiry provided through env vars instead of an hardcoded one.

Current field errors returned through extensions are quite limited in its design.

When a request should return data about payment, it should, aside the payment request field if a new invoice is generated, provide also an invoice state information, otherwise there is a potential bottleneck when handling query responses and conditional rendering on client side

See log

The HTTP paywall guard uses an api_payment table that registers the generated invoices.

As using a table for GraphQL paywall seems necessary to ensure the client provided invoice fits a requested data query, it might not be an absolute necessity for the global HTTP paywall.

We could remove the table and just check the provided invoice exists and has been paid through the LND service.

However, doing so would induce that the HTTP server could never be 100% sure that the provided invoice has been generated in a HTTP request context. Then so, providing any invoice that exists in the LND service could allow to go through the paywall. A few checks could be made to reduce the capabilities of passing through with non related invoice like checking the invoice value or the memo.

I currently have no clue if it would be a good idea and will probably leave the mechanism as it is, but want to keep the question opened for further options.

Having a query to request an invoice for a specific content would be a good hint on the required design to have a clean paywall mechanism.

Prior to #7, the problem that current design induces is that there is no way to quickly cache the LN payment request related to a specific post. It still could be doable, but probably not in a clean and fashion way.

To understand why it'd be complicated to handle it this way is to look how apollo handles its cache. It would require us to catch the data in the cache, store it aside of the apollo cache and make us write a middleware that would catch first the getPost request to inject the payment request.

This seems an overkill approach.

Instead we could create a getInvoiceForPost query that would generate an invoice for a post and return it to the user.
Using the cache-first fetch policy of Apollo we could then ensure that we do not rerequire a new invoice everytime we call this query.
Note that such query would need to return an expiresAt field which would allow us to effectively require a new invoice when previous one has expired.

The flow would the following one with Apollo :

User accesses to the post front page for the first time :

• The client requests get getInvoiceForPost with cache-first policy.
• Client finds no data in cache, so calls the API with such request
• API returns a response with an invoice payment request and an expiration field
• Client caches the response and then requests the getPost providing the payment request
• The API should return an extension error mentioning that the Invoice is awaiting payment.

User accesses the post front page a second time before invoice expires :

• The client requests get getInvoiceForPost with cache-first policy.
• Client finds data in cache and returns the cached response
• The client requests the getPost providing the payment request
• If payment is still awaiting, the user gets the invoice payment request displayed, if it has been paid, the content is retrieved.

User accesses the post front page a second time after invoice expires :

• The client requests get getInvoiceForPost with cache-first policy.
• Client finds data in cache and returns the cached response
• The client request getPost a second time to get a new invoice
• If payment is still awaiting, the user gets the invoice payment request displayed, if it has been paid, the content is retrieved.
• Client caches the response and then requests the getPost providing the payment request
• The API should return an extension error mentioning that the Invoice is awaiting payment.

Current implementation does not have any query that allow us to retrieve a list of users.

The query should be protected in order to be accessible only to admin.

Disclaimer : This issue and the associated branch is mostly a personal analytic note around file handling with Rust, Rocket & Juniper more than a formal implementation.

File upload and multipart/form-data specification

File upload on an HTTP requires an implementation of the RFC-7578 which describe multipart/form-data formatted HTTP requests.

GraphQL and file upload specification

Official GraphQL specification does not provide any specification around file upload.

When thinking about it it makes sense as GQL is made to provide a query language more than a protocol whereas file upload seems to be more tied to protocol specifications.

Therefore Juniper does not implement file upload handling and does not intend to do it as per the comment of this issue.

This does not mean there is no approach that could be built to handle file upload with GraphQL. Two main ways tends to be described :

Attach a REST HTTP endpoint

The first one is to create an HTTP REST endpoint that will handle file upload and retrieve an Id or data that will be later attached to a mutation GraphQL request in order to point out to the server which files should be manipulated.

It is an easy solution to implement and quite compatible with Rocket as per the library described in the File upload with rocket paragraph.

There is some trade-off however. As the file upload and the file processing gets splitted, if no mutation request the uploaded file will lie dead on the filesystem unless a mechanism gets implemented to regularly clean the temporary folder from dead files.

Use unofficial specification

Even if there is no official specification, many GraphQL clients and servers uses an unofficial specification to allow handling such feature.

Apollo, Altair and async-graphql seem to rely on this spec to handle uploading files to the server and process said files.

The main trade-off of this implementation is that as being non-standard it can not offer warranty that all gql clients will handle file upload the same way.

Also it bring some technical complexity on top of the original multipart/form-data implementation complexity due to the described structure as we'll see later in the technical solution analysis.

File upload with Rocket

Rocket does not currently provide an easily usable native implementation of multipart handling. This topic is subject to an issue in rocket repo.

The crate rocket-multipart-form-data exists trying to ease file upload handling.

What the above crate allows is to declare specific fields expected for a multipart/form-data request and parse the declared fields to fetch the provided data.

Any data attached to the form that won't have been declared won't be parsed and will be ignored.

Current choice of implementation

I decided to give a try into implementing the unofficial spec implementation.

A few reasons for that :

• Avoiding to have dead files on the filesystem or to have the necessity of building a clean-up job
• Build a file handler that corresponds to the initial main current PoC which was about LN over GQL more than REST itself.
• Implementation seems quite more challenging, so more fun to break my teeth onto such challenge 😄 .

Implementation complexities & Solution approach

Understanding how Juniper handles requests

To provide such implementation we must first understand how Juniper handles requests and the provided content-type.
Scrapping the source code of Juniper we the following code, we can find a FromData guard that pre-processes the request :

https://github.com/graphql-rust/juniper/blob/64fb83f5aa865962527cf4ff691ef277f7147b84/juniper_rocket/src/lib.rs#L345-L367

The first block of interest is the following one :

let is_json = match content_type {
    Some(("application", "json")) => true,
    Some(("application", "graphql")) => false,
    _ => return Box::pin(async move { Forward(data) }).await,
};

We must note how if content-type header is not application/json or application/graphql the request guard forwards to the next guard.

The second block of interest is this one, a few lines below :

Success(GraphQLRequest(if is_json {
      match serde_json::from_str(&body) {
          Ok(req) => req,
          Err(e) => return Failure((Status::BadRequest, format!("{}", e))),
      }
  } else {
      GraphQLBatchRequest::Single(http::GraphQLRequest::new(body, None, None))
  }))

When provided with a request, if the content-type is application/json, Juniper will parse the provided body and forward it to the GraphQLRequest. If the content-type is application/graphql the body will be provided as native request to the GraphQLRequest object. Any other declared content type will be forwarded to the next data guard, which includes our multipart/form-data request.

This is important for our implementation as it shows 2 things :

1. There is no direct way for the fromData to handle the multipart form data request, no matter the content of said request.
2. We will have to provide a GraphQLRequest instance with the part of our request that represents the graphQL query and make the other parts of the request ( i.e : the uploaded files ) accessible during query execution to allow us to process the files.

Unofficial specification data structure

To implement a data guard that will allow us to handle our request we need to look first at what will be the structure of our data sent through the multipart form.

According to the specification, when calling a multipart/form-data request we should be provided with a request that transcripted to json typed should look like this :

{
   "operations" : object,
   "map" : object,
   [key: string | number] : binary
}

• operations field is the field that should contain our graphQL query, providing the queries and/or mutations, the query variables and the operation name(s).
• map should be an object that describes the mapping to the other fields containing the files
• the part annotated as [key: string | number] should be the fields that contains the binary data of the files

Implementation process

We will split the implementation process :

1. Create a parser based on the rocket-multipart-form-data process that will allow us to provide the operations field to the GraphQLRequest object
2. Find a mechanism that will allow us to attach either the raw data or raw pointers to the temp file(s) so the files can be manipulated during the query execution
   3. Find a way to implement safety checks onto the files to avoid processing files that should be rejected automatically ( file size, file type, etc).

GraphiQL is enabled by default on the root path of the server. It could be convenient to disable the tool when going on some environments.

Therefore an option should be provided to replace the access to the tool to some other content ( maybe static content ? )

Current Media object would be default and each kind of media would have specific metadata.
Traits could be useful here.

There's currently no unit testing onto the API demo.

At some point, it'd be nice to have those as it would :
1/ Ensure that the demo works as expected
2/ Give example of the different use cases handling to users
3/ Show how to unit test that kind of app

The first step would be to build simple unit test covering :
1/ Juniper
2/ Rocket
3/ Diesel
4/ Lnd Mock

As we implement more logic with help of third-parties, more errors are prone to be triggered which can be very different in the way they should be handled.

A thought should be given on handling the different kind of errors with more clarity

Roles for users would allow to add constraints on some actions.

Ideally we would have :

• Admin role : Can perform any action : CRUD Medias and Users
• Publisher role : Can upload medias and edit its uploaded medias
• Moderator : Can CRUD Medias

Currently, when a browser tries to reach the API it obviously emits an HTTP Options request first to ensure CORS validation.

If no options handler is provided, rocket will return a default 404 HTTP response.
If the OPTIONS request fails, the POST request will also fail.

Without this it is not possible to use the API outside testing tools ( like GraphiQL or POSTMAN ).

We'd need to handle OPTIONS requests setting the needed headers .

This will speedup CI speed.

A possible action is here