calvinmetcalf / catiline Goto Github PK

Is it not possible to use a variable for the importScripts url when creating a worker?

I've been using shapefile-js for a while, recently needed to start handling some larger files and trying to implement cataline. Maybe I'm missing something here but both of the 'no' examples below crash the app.

var url = '.' + require.toUrl('./libs/shapefile.js');
console.log(url); // ./js/utility/libs/shapefile.js

var url2 = require.toUrl('./libs/shapefile.js');
console.log(url2); // /js/utility/libs/shapefile.js

var worker = new libs.catiline.Worker(function (data) {
  importScripts('./js/utility/libs/shapefile.js'); // yes

  // importScripts('/js/utility/libs/shapefile.js'); // yes

  // importScripts(url); // no

  // importScripts(url2); // no

  return shp.parseZip(data);
});

add minification

I'm working on an interactive app where the user manipulates a lot of data on screen.
I have workers on the background, constantly reading that new data (in queue mode) and then displaying it in processed form.
This process can take several seconds and I often have to discard the processed data as outdated because the user changed something.
With an enthusiastic user, the queue is soon filled up with calls and the app is on it's knees.
I can not cancel the entire queue because I may have several independent callers.
It would be useful here if I could assign a handler for specific queue calls and cancel or terminate those calls when they are no longer up to date.

Here is the test case: http://jsfiddle.net/QHz2A/1/

There is rather resource-demanding worker that is called 20 times with the same parameters. Evaluation time of each call is printed to console.

Here is the typical output in Win7 Chrome 26.0.1410.43:

281, 311, 287, 262, 258, 269, 260, 264, 265, 282, 299, 1910, 2019, 1951, 1914, 1905, 1894, 1908, 1898, 1894

Note, that last 9 calls are significantly longer than first 11.

There are no such effect in FireFox (I can't check other browsers right now). And I've tried to use native API with postMessage() - everything works ok.

you can use cw instead of communist it should be discussed somewhere

"Important: you're going to want to it yourself, on the same domain as your page, if you can't or need to bundle it and still want it to work on IE10, Opera, and Safari. You need to host the file "SHIM_WORKER.js" on the same domain as the html file and set the path to it in a global variable SHIM_WORKER_PATH before you load this script."

I'm sorry but I have no idea when to host SHIM_WORKER.js and when not to from the above statement. This uncertainty fills me with greater fear than I once had for west Berlin. Please clarify kind comrade.

The worker object, the callback in the worker function, and maybe the object methods should have on, off, and fire methods

Replace all the w and c variables with more descriptive names

Default function should be one where you can add a function with a name and then call it.

close should clear all intervals and timeouts in socialist

currently it isn't

breaking the non-script import part of #66 into it's own issue, basically this

  testWorker = cw (input, callback) ->
    console.log 'i am the worker ',input
    callback input+5

  testWorker.data(2).then (res) -> console.log res

  setTimeout ->
    testWorker.data(3).then (res) -> console.log res
  , 1000

  # Listen for socket event and update the
  # scope with the new data as processed by the above function
  $socket.on 'dataProcessingTrend', (theThing) ->  
    console.log 'chart update recieved'
    testWorker.data(4).then (res) -> console.log res

produces

   i am the worker 2 
   7 
   i am the worker 3 
   8 
   chart update recieved 
   i am the worker 4 
   chart update recieved 
   i am the worker 4 

@Fresheyeball what happends when you add a second callback to the worker inside the socket, ie

success = (data)->
   console.log 'yes ', data
failure = (data)->
    console.log 'no ', data
testWorker.data(4).then success,failure

Create a more CouchDB like map reduce that allows multiple things to be returned (or emited one might say) from the map function

cancel the queues both the array of waiting promises and for the smart queue, the array of data waiting to be sent.

look into using iframes for the fakeworker like operative does

Having addEventListener dangle as a global is not ideal.

Please use window.addEventListener to provide more context to static analysis tooling.

Hello,

I found a bug in the replacement of the importScript.
On my minified code the webworker initialization failed and I started to investigate why:

My not minified code:
init: function(){
importScripts("libs/mylib.js");
this.somevar="init value";
...

my minified code:
init: function(){
importScripts("libs/mylib.js"),
this.somevar="init value",
...

and after the movement of the importScripts a comma , is left in the init function and I get a syntax error.

My solution was the following in catiline regexImports function ([;|,] instead of ;):
match = rest.match(/(importScripts(.?)[;|,]?)/);
rest = rest.replace(/(importScripts(\s(?:['"].?['"])?\s)[;|,]?)/,'\n');
if(match){
match[0].replace(/importScripts(\s_(['"]._?['"])?\s*)[;|,]?/g,loopFunc);
}

Cheers,
Beni

operative can pass variable to web worker.
Is possible to add this feature or with a plugin?

var myWorker = operative({
    something: 123,
    doStuff: function() {
        this.something += 456;
    }
});

Would it be possible to publish a new release with the comma fix?

I am having two problems with the current release of communist and ie9

1. When using importScripts js dies silently. When wrapping importScripts in a try catch I get : ReferenceError: 'importScripts' is undefined
2. .then of the promise is not getting called in ie9 depending on the wrapper function. This one is hard to describe. Here is some coffeescript from my project

which produces the following logs in ie9

   i am the worker 2 
   7 
   i am the worker 3 
   8 
   chart update recieved 
   i am the worker 4 
   chart update recieved 
   i am the worker 4 

Throws this error.
Uncaught Error: an unsupported module was defined, expecteddefine(name, deps, module)instead got:1arguments to define

The issue Ember has with the form of module use is described here: ngot/devicejs#7

Hi,
Catiline support transfarable object?

currently, the quickest way to put some data and a function into another eventloop is:

var worker = cw(myFunc)
var response = worker.data(YOUR DATA);

succincter and imo clearer:

var wp = new WorkerPromise(myFunc, myMessage);
wp.then(success)

There's a more popular library with a similar premise — Parallel.js.
https://github.com/adambom/parallel.js

It would be nice if the Readme mentioned the library and the main differences of Communist compared to it.

for some reason the opera tests are failing in strange ways, I think this might be an issue with opera and saucelabs and or mocha so will have to investigate.

there have been some complaints about the name of the library, should it have a shortcut function? full on renaming?

Checked http://calvinmetcalf.github.io/leaflet-fractal with different browsers at Windows 7:

• FF, Chrome - works without errors!
• Opera 12.14 - Unhandled Error: Cannot convert 's.URL' to object
• IE10 - SCRIPT5022: SecurityError communist.js, line 113 char 3
• Safari 5.1.7 - TypeError: 'undefined' is not an object (evaluating 'c.URL.createObjectURL') line 113.

the callbacks are getting screwed up/crossed when used in the sync version

currently I have long multiline strings , I should do something better.

What's the best way to handle if the browser doesn't support web workers at all?

p.s. this lib is uber cool 👍

it should work with node

maybe though defs or grunt-defs

When an event is fired in a queued wormer it should say which worker fired it

the ajax method is exactly how workers shouldn't be used

Create an object to hold the callbacks then add a random number for each bit of sent data so the correct cb can be associated with the right data

make it possible for different functions to have importScripts calls that all get combined into one deduped one.

I couldn't find any mention of it, so I assume this is not supported yet. Would be cool if transferable objects are supported for more high-performance needs.

Firefox fails in sauce lab tests but not when done manually, looks to be some sort of global leak.

Errors in the callbacks are suppressed by the try/catch block in execute(), i.e.:

worker.testFunc(args).then(
  function (returnValue) {
    if (returnValue == bad) {
      throw new Error("Return value is bad!"); //this error would get suppressed
    }
  },
  function (errMsg) {
    throw new Error(errMsg); //this error would also get suppressed
  }
);

It's possible to make the errors throw by escaping the context, i.e.:

worker.testFunc(args).then(
  function (returnValue) {
    if (returnValue == bad) {
      setTimeout(function(){
        throw new Error("Return value is bad!"); //this error would get thrown
      },0);
    }
  },
  function (errMsg) {
    setTimeout(function(){
      throw new Error(errMsg); //this error would also get thrown
    },0);
  }
);

I am planning to put together a PR that would use nextTick in execute to fix this--is that the right approach or am I barking up the wrong tree entirely?

Hey Calvin,

Thanks for creating this library. It's been super handy :)

Recently I've encountered some weird problem with importScripts when I try to do this:

var momentUrl = "url/to/moment.js";
importScripts(momentUrl);

When I run it the page just locks up while the console is still functional (a bit laggy though).

I installed catiline through bower, and the version is 2.9.3.

I remember being able to use variables before like this:

var componentUrl = "url/to/components";
importScripts(componentUrl+"moment/moment.js");

It never locked the page up, and I haven't updated the bower component.

In addition, I can't have line breaks in importScripts, I'll just get a syntax error when I do.

Could you help me figure out what's going on here? Would this be caused by the regex matching method?

any reason why you use gh-pages as your default branch?

I would use master and write a quick script that published to gh-pages. Such as: https://github.com/pinf/pinf-loader-js/blob/master/Makefile#L29-L32

I'm blanking on how else to effectively have RSVP in as both a global in the browser and a commonjs in node

var p2;
var port = cw(function(data,cb){
    this.x = new MessageChannel();
    this.on('mess',function(a){
        this.x.port1.postMessage(a);
    });
    cb(this.x.port2,[this.x.port2]);
});

port.data(1).then(function(a){
    console.log('yes');
    p2=a;
},function(a){
    console.log(a);
});

p2.onmessage=function(a){
    console.log('msg ',a.data);
}

port.fire('mess','hello there')

this could be a polyfill for sub workers in chrome

I need to make sure the tests are cleaning up after themselves. The Queue and Dumb Queue tests are timing out when run together with the other tests, but not by themselves.

it's about time

Code snippet:

				`var worker = cw(function(a,callback){
				    callback(a[0]+a[1]);
				});
			        worker.data([2,5]).then(function(a){
				    console.log("One worker: " + a + " -- " + new Date());
				})//prints 7
			        worker.close();//close it up`

and no errors are getting shown as well.