aerogear / offix Goto Github PK

Feature Request

Use docusaurus for the documentation as suggested in #68
Target of this issue is to migrate existing docs to docusaurus.

Feature Request

Check if client works with Prisma and if possible to extend Prisma to include conflict resolution feature on top of prisma

Bug Report

• Affected Package(s):
• Package version(s):
• Platform (e.g. Android/iOS):
• Platform Version:
• Cordova Version:
• Node.js / npm versions:

/packages/offix-cache/package.json has [email protected] as a dev-dependency but this version has not been publicly released.
Perhaps [email protected] omitted by mistake from last release cycle?

Using offix 0.5.0. This is a basic react-native expo project as discussed in #23. With react-apollo 2.5.5 <ApolloConsumer> fails to find the client on the context. Pinning react-apollo to 2.5.2 works. I'm not sure if this is caused by a react-apollo bug or by offix. apollographql/react-apollo#2900 discusses the issue but only when using MockedProvider and it appears a fix was put in for that.

Hash state implementation is missing on the client

Target

Provide comprehensive support for Offline and Conflict resolution on top of Apollo GraphQL. Form community for best practices and patterns for building offline first applications on top of the Apollo GraphQL

Current Focus

• Non-blocking mutation functions #2
• Global listeners
• Cache helpers #9
• Modular architecture #4
• Documentation and website #22

Comming Next

• Extended capabilities for conflict resolution (out of the box conflicts)
• Extended cache capabilities (working with client-side db's)
• Storage improvements

Community suggestions

Feel free to create a new feature request and suggestion.

Bug Report

See #67

Some of the types have changed that prevent from compiling the source code.
The workaround, for now, is to use as any for MutationOptions
See #59

Bug Report

Currently getUpdateFunction here: https://github.com/aerogear/offix/blob/master/packages/offix-cache/src/cache/mutations.ts#L77 does not have a default value for opType. This is bad and can be breaking.

• Affected Package(s):
  offix-cache
• Package version(s):
  1.6.1

Feature Request

React integration is needed to deliver better UX.
We can have separate components like OfflineMutation etc.

Looking for feedback.

Feature Request

Offline changes can be saved as diffs between cache and data passed to mutation.
When working with web frameworks users will pass all data to mutations despite what fields were actually changed. We can have an additional layer between user data and mutation that will calculate diff between what UI have and what was stored in cache.

Assumptions

Mutations accept diffs rather than full objects as arguments.
Mutations can return full objects to make sure that local cache is up to date
Subscriptions will return diffs that will be automatically merged into existing objects.

Benefits

• fewer data to query
• fewer data to store in offline queue
• much improved network reliability
• ability to perform fully featured conflict resolution on client and server
• ability to perform subscriptions that returning only data that was changed.

Limitations

a diff-based approach will prevent us from having required fields.

For example:

updateTask(id: ID!, required: String!)
or 
type Task {
 id: ID!
 required: String!
}

Will not work with this approach as it has required a field that may not have changed (it will not be part of the diff)

Technical approach

TODO

Feature Request

Support advanced method for creating client where user can create client themselves and just supply offline and conflicts links on the chain. Currently, the client is being created with many defaults making it hard to use it for generic use cases.

See also #4 that will be required to do this properly.

Feature Request

Provide helper that will support generating update functions.
For example:

client.mutate(
     update: customFn
)

can become

For example:

client.mutate(
     updateQueries: [{name: "getItem", variables: {}] 
)

As the title says I upgraded to 0.5.0 and it crashes here

[Unhandled promise rejection: SyntaxError: JSON Parse error: Unexpected identifier "object"]
* [native code]:null in parse
- node_modules/offix-client/dist/offline/OfflineStore.js:79:45 in <unknown>
- node_modules/offix-client/dist/offline/OfflineStore.js:32:27 in step
- node_modules/offix-client/dist/offline/OfflineStore.js:7:13 in <unknown>

It looks like the getOfflineData function doesn't properly handle invalid data from the store. I'm using 0.5.0 on react-native (expo) with AsyncStorage.

Edit: It actually crashes using AsyncStorage no matter what now. It isn't just with old data. I cleared the data on the simulator and the result is the same. This worked in the previous version without problems.

When offline requests is finished with success we need to process 2 different things:

• Check and remap local id's
• Check fields for versioning

After chatting with the community we found out that there could be another use cases for updating results. We should abstract processing results in an offline link to ResultProcessors and use them to apply conflict and id mapping.

We're making some really great progress with Offix, however I think some work needs to be done around testing. I'd like this issue to focus solely on unit tests. There is plenty of scope for integration tests also, but I think we can start simple and cover those in a separate issue.

There are several components in Offix that perform some complex and tricky logic. Cache operations, data structure manipulations. Let's try to collaborate and identify the areas where we will get the most value out of unit tests.

I see this initiative being tackled as lots of small PRs. Contributions in the form of PRs, and suggestions/discussion are welcome. Myself and other collaborators are here to help!

1. Use separate repository for website. Create a proper landing page.
2. Improve documentation. Fix styling issues etc.

Client do not allow to pass initial data to the store and seed it with initial objects. We simply can pass an additional parameter to InMemoryCache

Feature Request

Related discussion here: darahayes/offix-example-react#1

Is your feature request related to a problem? Please describe.

The main problem right now is that the flow to create a client looks something like this.

let client = new OfflineClient(config);
await client.init();
let apolloClient = client.apolloClient

The apolloClient is really what we're looking for but it doesn't become available to us until after we await client.init(). I noticed while trying to build a react application that the main way to pass an ApolloClient around a React application is to use an ApolloProvider either from react-apollo-hooks or react-apollo.

Example:

import React from 'react';
import { render } from 'react-dom';

import { ApolloProvider } from 'react-apollo-hooks';

const client = ... // create Apollo client

const App = () => (
  <ApolloProvider client={client}>
    <MyRootComponent />
  </ApolloProvider>
);

render(<App />, document.getElementById('root'))

Now useApolloClient can be used within components.

The problem is that ApolloProvider must be passed a client and you can't really await for things during a component render.

The hacky solution I found so far was to create a regular apollo client, inject that into my application, and use a hook to swap out the regular client with the one coming from Offix after the init has completed.

Describe the solution you'd like
Ideally we would have one of two fixes:

• OfflineClient.init() synchronously returns the ApolloClient.
• OfflineClient extends ApolloClient and somehow everything can be synchronously set up in the constructor.

I think I would actually prefer the first option for now because it's less of a drastic change.

We ignore MutationHelperOptions.idFIeld provided by the user and use the hardcoded id field in IDProcessor - Going to create a separate issue for that bug.

Originally logged in #74

Using apollo client mutation I was able to use update method to provide my custom function to update cache. Thanks for "mutationCacheUpdates" by offix that handle the standard situations.
Whereas , in many cases the data could be nested or overlapped, where some other updates cache should be performed on developer side.

In this example below, PateintsMediaFile will contain some MediaFiles (overlap), Whenever a new MediaFile is created PateintsMediaFile should be updated and recalculate some values manually.

And other cases could be nested data , also should update the cache manually.

We need to make sure that we have a differentiation between oflineMutate and mutation. We have onlineOnly but since we dropping now support for client.mutate() we can think about disabling all functionalities when client.mutate is used.

Feature Request

Currently optimistic responses are persisted for offline queue and it is possible to replicate them by supplying cache update functions globally.

We can use a custom cache persistence layer on top of Apollo cache persist that will save and manage optimist responses in cache and remove the need of supplying complex cache update functions globally.

How this will work

Optimistic responses can be still hold using apollo cache but we can have the entire layer persisted and restored upon application restart.

Feature Request

Split conflict and offline features with client creation so developers can pick them separately and integrate with their apps.

Rebase Feature for Conflicts

Currently any subsequent change on the same object while offline can cause conflict. This due to the fact that any next subsequent change will be based on an optimistic response.

This causes network inefficiency - an additional roundtrip is needed to server. The server needs to fetch server state

Approaches for resolving this problem

Rebasing offline queue for each conflicted change

When conflict happens we know that any subsequent change in the same type will cause another conflict. This means that we can effectively query the operations for the same type and try to apply the latest server-side data on top of that using conflict strategy. Using conflict strategy before hitting the server will allow us to avoid going to a server in cases where we know that conflict will always occur.

We had similar functionality before that was using ConflictProcessor abstraction to iterate through the queue.

Diff based offline storage

Storing diff for the offline mutation instead of the entire object will allow us to avoid the problem however it can be a controversial change.
This is due to the fact that offline change will be saved as diff.

If the client wins approach will be taken then we may even skip saving the base leading to better storage efficiency.

When we can expect 0.7.0 release?
Can you do fix to support for Apollo 2.6 - types are breaking now.
I like more examples for conflict with actual database and react integration.
It is hard to use it in React. Do you plan to provide offix components?

Feature Request

Currently, when calling mutate functions users will need to pass generated optimistic response and also supply global cache updates. UX of that approach is quite restrictive and prevents developers from taking things even further.

client.mutate({ 
     ...devOptions, 
     *optimisticResponse: createOptimisticResponse()*
     update: updateFn
})

Proposal

Provide wrapper for client.mutate that will provide extra capabilities:

Variant 1

Example parameters for wrapped mutate function:

client.mutate(...devOptions, 
  // Ability to control how offline state is reported
  offlineStrategy: 'failWhenOffline', 
  // Add ability to update Query cache automatically
  updateQuery: "getItems",
  // Adds automatic optimistic response based on payload
  generateOptimisticResponse: "getItems")
}

Variant 2

client.mutate(...devOptions,
  // Supplies argument that will be used to build 
  // Optimistic response and query Updates
  // Devs will be able to override them by 
  // supplying original parameters to function. 
  affectedQueries: ["getItems"]
  // Many other arguments possible
}

Function internally can

• Check if we are offline online
• For offline it will save changes locally
• Register update functions that will be triggered to see optimistic responses.
• Allow providing sensible defaults and better support for offline that is not limited to Apollo Link/Apollo client API.

Pros/Cons

Pros

Doing this will give us many benefits:

• Awesome UX and Offline support out of the box.
• Simplify entire implementation (no holding mutations etc.)
• Ability to work with other clients depending on the community interest (not only Apollo)
• Ability to autogenerate cache updates (update fn) if dev did not supply them
• Ability to autogenerate optimisticReponse if it was not supplied
• Ability to supply extra parameters. For example, we can check if the client is offline and then return result instantly

Cons

• A custom client without the ability to reuse links inside
  Offline Link will no longer be sufficient to integrate that with other projects.

Alternatives

Keep existing implementation without the wrapper and provide a toolbox that will generate some parameters:

const offlineArguments = offlineHelper({affectedQueries}
client.mutate(...devOptions, ...offlineArguments)

Add the build process using circle

Definition of the problem

Offline requests block promises.
This will mean that code in then will never be called until the device becomes online.

return this.apollo.mutate<Task>().then((retry)=>{
	// Never called when offline.
	// This will be resolved only when online
}).catch((error)=>{
	// Never called when offline
	// This will be resolved only when online
});

Proposal 1

Return error when change is enqueued offline and resubmit every change using client.

return this.apollo.mutate<Task>().then((retry)=>{
   // Never called when offline.
   // This will be resolved only when online
}).catch((error)=>{
  if(error.conflict){
    // User will get error with conflict
  }
});

Proposal 2

Return another observable or promise in error

return this.apollo.mutate<Task>().then((retry)=>{
   // Never called when offline.
   // This will be resolved only when online
}).catch((error)=>{
  if(error.offlinePromise){
    offlinePromise.then((data)=>{
       // Will be called when we become back online
    });
  }
});

Bug Report

OfflineQueueListener can be supplied to the client to provide updates.
However, if developers want to listen to updates in different places they cannot configure that without providing some wrappers.
Using observables here will be the natural choice as they can be hooked into Angular and React.js land.

Reported by a community member.

Currently we are using temporary package names.
We should use offix package names everywhere

I think the best way to start with this is simply to enable the application to Query specific data on the server and Subscribe for results when:

• Application is starting
• The application became online and it is on the foreground.

We currently have OfflineMutationsHandler that gives the capability to resend offline mutations.

Proposal for client

The client can have new methods for registering queries/subscriptions:

client.registerOnlineQuery(new OnlineQuery({gql,variables}))
client.registerOnlineSubscription(new OnlineSubscription({gql,variables}))

OnlineQuery/OnlineSubscription apart from having all required fields to perform mutation will contain metadata used to decide when to call mutate.

For example:

// Wait with request after becoming online
public initialDelay: number = 0; 
// Interval used for pooling
public interval?: 
// Even some extra metadata
public requiresWifi: boolean

Developers will be able to trigger Query Refresh manually (and force subscriptions to reconnect:

client.forceOnlineRefresh();

Related work

0. Extend OfflineMutationsHandler to handle subscriptions
1. Create abstraction for OnlineQuery and OnlineSubscription
2. Create Registry of OnflineQueryes/Subscriptions.
   Expose methods to register OnlineQueries and OnlineSubscriptions to registry
3. Connect NetworkState interface to interact with the registry and execute depending on medatadata in OnlineQuery and OnlineSubscription
4. Write unit tests and integration tests for this feature.

Open for comments, opinions and contributions
We can create individual issues once the community will agree on the flow.

Discussion in #6 for more information

Definition of the problem

Currently, when performing offline mutate operation we going to apply optimistic response updates twice. This approach also requires users to pass the global cache update function, which is not convenient.

Solution

We can return error to the user and check network state directly in the helper making. Offline link then will be able to enqueue the changes as always but this time we going to know that this operation was marked as offline.
This will help us to introduce server state aware offline cases where developers can specify some function that will quality changes to be enqueued to offline queue after network request.

How its possible to update the cache after delete mutation?

Repo
I do not know if I'm doing this right.
But, I have already created very simple demo to play with offix until I feel ready to move for production.
It is a pleasure if you check it Offix-angular-demo.

The conflict processor is designed to update state information for offline queue. We should check if the object actually contains state.
When dealing with objects without the state we currently going to apply a new state by default which will mean that we going to get conflict resolution logic for the changes we do not want to be included in conflict resolution.

Very simple fix will include check if state exist before applying new version

Offline mutation scheduler is now part of the Apollo-link.
This has some disadvantages as processing on link requires client.mutate to be scheduled. Ideally, our scheduler should act before client.mutate is triggered and persist the related data

See #125

Create an entire chapter about extensions we offer and describe them.
See ApolloOfflineClient interface for more information.

Documentation it's a little bit confusing, can you provide us with samples ?

EDITED BY @wtrocki

Target of this issue is to document way developers can specify Apollo Cache Redirects.
Those redirects needs to be specified when creating InMemoryCache
https://github.com/aerogear/offix/blob/master/packages/offix-client/src/OfflineClient.ts#L102

Currently there is no way to do so

graphql version conflicts with user graphql when using without latest version of the library. We should make sure that we have all packages configured using peer-dependencies and require people to pass versions in their apps.

Bug Report

• Affected Package(s):
• Package version(s):
• Platform (e.g. Android/iOS):
• Platform Version:
• Cordova Version:
• Node.js / npm versions:

Bug Report

Currently, conflict processor is creating an issue where it updates issues in the queue meaning conflicts (that should be triggered) are not.

Also, the strategy interface should be extended to allow the passing of additional necessary data.

The default implementation is also incorrect and has the potential to cause data loss as server data is overwritten with base data. To combat this the base should not be used in the resolution as it is not needed. Instead we can apply the clientDiff data to the server data returned with the conflict. This ensures only up to date data is passed with the next mutation in the queue.

Currently working on this.

• Affected Package(s): offix-offline

Feature Question

Hi

I came across your libraries while looking for an offline Apollo solution BUT your offix-cache is also of interest because I have would like a simple/clean method to update the cache when you have multiple occurrences of the same query in your cache that differ on their parameter values.

an example:

• You have a parent entity with a list containing children.
• The parent with its children is referenced by many queries.
• We delete one of the children.
• We now need to update all queries that reference that parent in order for its list of children to be updated.

Reading your documentation, I get the impression that in order to identify queries in the cache, you need to provide the values that those queries used. Am I correct?

Our use cases would include dynamic user values being used within queries, so it may not be possible to provide parameter values.

Any clarification appreciated.

It looks like there are just a couple little missing pieces to work with react native. I'm not sure exactly how to integrate them properly into the codebase so I will put the snippets here. We just need a NetworkStatus implementation and to pass AsyncStorage for the storage engine when creating the client. IMO these should just be defaults when react native is the environment.

Detecting react-native

if (typeof navigator != 'undefined' && navigator.product == 'ReactNative') {
  // I'm in react-native, use the ReactNativeNetworkStatus and AsyncStorage
}

React native network status implementation

import {  NetInfo } from "react-native"
import { NetworkStatus, NetworkStatusChangeCallback } from "apollo-offline-client"

class ReactNativeNetworkStatus implements NetworkStatus {
  public onStatusChangeListener(callback: NetworkStatusChangeCallback): void {
    const listener = (connected: boolean) => {
      callback.onStatusChange({online: connected})
    };
    NetInfo.isConnected.addEventListener('connectionChange', listener)
  }

  public async isOffline(): Promise<boolean> {
    const isConnected = await NetInfo.isConnected.fetch()
    return !isConnected
  }
}

Config for react native

import { AsyncStorage } from "react-native"
import ReactNativeNetworkStatus from "./react-native-network-status.ts"

let config = {
  httpUrl: "http://localhost:4000/graphql",
  wsUrl: "ws://localhost:4000/graphql",
  storage: AsyncStorage,
  networkStatus: new ReactNativeNetworkStatus()
} 

edit: oops it looks like AsyncStorage might not be just drop in. Types don't line up but I think it will work regardless.

When refactoring projects to different packages I found out that having 3 words in name is not a good idea. Package names become really long and confusing:

apollo-offline-client-client
apollo-offline-client-link-server

Additionally, we probably may avoid using Apollo GraphQL as it is a trademarked name and I can be confusing as developers may think that these packages come from Apollo team.

Picking the name

Propositions:

1. 🔬 OfflineLab (GraphQL)
2. 🚀 OfflineLink (GraphQL)
3. 📶 OfflineX
4. 💾 LocalGraphQL
5. Offix ;)
   

Is it possible to use the variable id (which is auto-generated on client-side) in optimistic response rather than "client:xxxxxx".

In case of nested data, this approach will cause an issues. Then, my solution is always to generate an id on client side and offix should accept this variable.

Architecture of the offix client

Currently offix-client is very complex set of the links that perform many different operations (non offline related). We trying to keep links separated internally, but this do not change the fact that our link implementation gets more complex over the time.

Solution

To prevent from complexity but still promote reusability we can have a separate package that will be purely an offline and conflict link and offix-client that will provide required piping like cache implementation and config. This will keep our architecture reusable and pluggable but is also going to hide the complexity of internal offline and conflict links