Comments (20)
See org.dataloader.registries.ScheduledDataLoaderRegistry
in the lastest 3.x versions for some of the answers to the above
It allows the DataLoader to "tick" in the background and decide if it is going to dispatch or not.
The predicates you provide can dispatch on time or depth or both. Or make up you own predicates.
This may help people compose together dataloaders such that dispatch is eventually called regardless of the field tracking in graphql say
from java-dataloader.
I've run into the same limitation and ScheduledDataLoaderRegistry
didn't work for me. I think ScheduledDataLoaderRegistry
serves a different use case: to make the overall dispatching strategy less eager by pushing dispatch attempts into the future. This still relies on dispatchAll
to be called first, which doesn't happen in the scenario with nested loaders.
So my ugly hack current approach is to have a separate scheduled task periodically check all inflight data loaders and forcefully dispatch them if they haven't been dispatched within a preset time window (e.g. 500ms). This works to unstuck nested data loaders, at the cost of naively triggering dispatches too early for long running data loading tasks. I am not sure what the implications of those are, but my API has been working fine so far so I'm happy 😎
@Component
@Slf4j
public class ScheduledDataLoaderDispatcher {
Queue<DataLoaderRegistry> globalRegistries = new ConcurrentLinkedQueue<>();
Duration timeToDispatch;
public ScheduledDataLoaderDispatcher(@Value("${app.dataLoader.timeToDispatch:500}") Integer timeToDispatch) {
this.timeToDispatch = Duration.ofMillis(timeToDispatch);
}
public void addRegistry(DataLoaderRegistry dataLoaderRegistry) {
globalRegistries.add(dataLoaderRegistry);
}
public void removeRegistry(DataLoaderRegistry dataLoaderRegistry) {
globalRegistries.remove(dataLoaderRegistry);
}
@Scheduled(fixedRateString = "${app.dataLoader.dispatchTickRate:100}")
public void dispatchAll() {
globalRegistries.stream()
.map(DataLoaderRegistry::getDataLoaders)
.flatMap(Collection::stream)
.filter(this::isDispatchNeeded)
.forEach(DataLoader::dispatch);
}
private boolean isDispatchNeeded(DataLoader dataLoader) {
return timeToDispatch.compareTo(dataLoader.getTimeSinceDispatch()) < 0;
}
}
from java-dataloader.
This works to unstuck nested data loaders, at the cost of naively triggering dispatches too early for long running data loading tasks.
This is pretty much how the JS tick
works for JavaScript data loaders. They can dispatch too early a well but never miss composed loaders because eventually control is passed back and tick
will happen.
One thing I will say about the above us - since DataLoaders are per request, your scheduler Queue will grow to to the size of the number of concurrent requests * the number of dataloaders per request.
It's good that you have a removeRegistry
because otherwise this would get unwieldy quick with enough load
from java-dataloader.
Currently we dont have a way to do what you are after.
CompleteableFutures give us no way to know how deep they are nested and hence how many times dispatch must be called
This is currently and unsolved problem
from java-dataloader.
CompleteableFutures might do something else completely so even if you had a way of inspecting their "nestedness", you still wouldn't know that their completion leads to another load(), or even multiple.
I see two possible solutions:
- Whenever load() is called, start a timer to call dispatchAll() after 1 ms. If the timer is already running, delay it a bit.
- Whenever a batchload completes, complete all the futures and then call dispatchAll().
Both can be made optional using an extra parameter. And of course dispatchAll() should finish quickly when nothing needs to be done.
from java-dataloader.
The solution to this problem is to modify loader1
so that nothing needs to be loaded in the .then()
block. In a traditional relational database this would mean that loader1
would do some extra joins to obtain all the required data.
from java-dataloader.
@vojtapol In all honesty, if you modify the DataLoader to eagerly optimize fetching, you can also modify the original resolver function in the exact same way and drop DataLoader completely.
from java-dataloader.
The solution to this problem is to modify
loader1
so that nothing needs to be loaded in the.then()
block.
That would solve the immediate problem but there might be other ways to obtain id2. Then that load would have to do a similar join, not taking advantage of the already loaded data2 so that would decrease the effectiveness of the cache and require more memory.
from java-dataloader.
I noticed #46 which will solve this transparently.
from java-dataloader.
One potential solution is to dispatch all pending data loaders, wait for the futures returned from the dispatched data loaders to complete and then repeat the process to dispatch new pending data loaders that come from the thenCompose
chaining. This process can be repeated until all levels of pending data loaders are dispatched. A code example to do this would be something like:
public class Dispatcher {
private final List<DataLoader<?, ?>> dataLoaders;
Dispatcher(List<DataLoader<?, ?>> dataLoaders) {
this.dataLoaders = dataLoaders;
}
private int depth() {
return dataLoaders.stream()
.mapToInt(DataLoader::dispatchDepth)
.sum();
}
void dispatchAllAndJoin() {
while (depth() > 0) {
// Dispatch all data loaders. This will kickoff all batched tasks.
CompletableFuture<?>[] futures = dataLoaders.stream()
.filter(dataLoader -> dataLoader.dispatchDepth() > 0)
.map(DataLoader::dispatch)
.toArray(CompletableFuture[]::new);
// Wait for the futures to complete.
CompletableFuture.allOf(futures).join();
}
}
}
In every round of dispatch, Dispatcher#dispatchAllAndJoin
will be able to batch all tasks whose dependencies have been resolved by previous dispatches. This logic could potentially live in DataLoaderRegistry
but clients can also just implement their own dispatching logic without changing the dataloader library.
from java-dataloader.
@hahooy I know, I'm actually using a fully asynchronous version of this: #46 (comment)_
from java-dataloader.
For the record just dispatching until the depth is <= 0 will work however it will have the opposite effect. It will cause fields that that COULD be batched together to be eagerly dispatched. So you have "UNDER BATCHING" in this situation.
The real trick is that you need to know WHEN a good time to dispatch is and unlike say javascript there is no "nextTick" time in a JVM.
Actually I have done testing on node.js and they can also UNDER BATCH based on next tick firing before fields compete.
from java-dataloader.
For the record just dispatching until the depth is <= 0 will work however it will have the opposite effect. It will cause fields that that COULD be batched together to be eagerly dispatched. So you have "UNDER BATCHING" in this situation.
I don't think that is true since the code waits for all dispatchers to terminate before starting a new round. This guarantees that no new calls will be done on the dataloaders. Unless of course multiple dispatchAllAndJoin() loops run in parallel...
Another inefficiency in this method is that a new round has to wait for the slowest dataloader. Some dispatches could have started earlier.
I imagine some heuristics where each dataloader that has received requests waits some time before dispatching. The closer the number of requests is to the maximum batchsize, the earlier the dispatch. The optimal mapping from number of waiting keys to wait time could be hand tuned or "learned" automatically.
from java-dataloader.
I tried swapping the default DataLoaderRegistry
with ScheduledDataLoaderRegistry
in my graphql application and ran a simple loader1.load(id1).thenCompose(id2 -> loader2.load(id2))
test with the following configuration
DispatchPredicate depthOrTimePredicate =
DispatchPredicate.dispatchIfDepthGreaterThan(10)
.or(DispatchPredicate.dispatchIfLongerThan(Duration.ofMillis(200)))
ScheduledDataLoaderRegistry.newScheduledRegistry()
.dispatchPredicate(depthOrTimePredicate)
.schedule(Duration.ofMillis(10))
.register(...)
.build()
When executing a query that triggered the data fetcher calling the test, the server hung after calling loader1.load(id1)
never calling loader2.load(id2)
Has anyone gotten an example this or an alternative to work?
from java-dataloader.
Thank you for your reply! Correct, I also wrote an instrumentation to clean up the globalRegistries
after each request. And even with that mechanism in place, the queue can grow rapidly under heavy load. 500 milliseconds is a long time when your API is processing tens or hundreds of calls simultaneously. I still have to performance test my API to make sure this setup functions under load.
@Component
@RequiredArgsConstructor
public class ScheduledDataLoaderInstrumentation extends SimpleInstrumentation {
private final ScheduledDataLoaderDispatcher scheduledDataLoaderDispatcher;
@Override
public InstrumentationContext<ExecutionResult> beginExecution(InstrumentationExecutionParameters parameters) {
return new SimpleInstrumentationContext<>() {
@Override
public void onCompleted(ExecutionResult result, Throwable t) {
Optional.ofNullable(parameters.getExecutionInput())
.map(ExecutionInput::getDataLoaderRegistry)
.ifPresent(scheduledDataLoaderDispatcher::removeRegistry);
}
};
}
}
from java-dataloader.
Just checking in. Has anyone come up with a workable solution to this problem yet?
from java-dataloader.
Trying to find one.
from java-dataloader.
This not composing dataloader calls per say - however this PR may help others who want to write custom dispatchers
from java-dataloader.
I have created a variant on ScheduledDataLoaderRegistry
called ticker mode that will reschedule the dispatch()
calls continuously in the background.
This will allow chained calls to complete however perhaps the batching windows will be as efficient as possible.
See PR : #131
from java-dataloader.
Closing this thread after pull requests #128 and #131
from java-dataloader.
Related Issues (20)
- invalid automatic module name when used with Java modules
- Version 3.0.0 and later is no OSGi bundle anymore HOT 2
- Automatic Module Name clashes with graphql-java Module HOT 2
- Add ReactiveStreams and Kotlin Coroutines variants for DataLoader HOT 1
- Support Flow or Channel as Subscription return type HOT 1
- Failed to build.gradle HOT 1
- Querying for length of dataloader queue HOT 2
- Avoid allocations when dataloader has no work
- ScheduledDataLoaderRegistry.newScheduledRegistry() creates new ScheduledExecutorService on every invocation
- ScheduledDataLoaderRegistry can leak scheduler threads HOT 1
- When dispatchPredicate condition evaluates to false, shouldDispatch should return false HOT 3
- License question HOT 4
- Use of LongAdder in StatisticsCollector HOT 2
- Test - please ignore
- Upgrading dataloader to Java 11 HOT 1
- Is the data loader's per-key context allowed to be null?
- Java DataLoader: Mismatches results HOT 1
- Null pointer in dispatchQueueBatch v2.1.1 HOT 1
- ValueCache: batch function isn't always triggered on cache miss HOT 5
- DataLoader 3.x will not be upgraded in graphql-java 17.0 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from java-dataloader.