Hello,

I wrote this sample test code:

  1 var PythonShell = require('python-shell');
  2 var pyshell = new PythonShell('test_json.py', { mode: 'json'});
  3 
  4 
  5 pyshell.on('message', function (message) {
  6   // received a message sent from the Python script (a simple "print" statement) 
  7   console.log('received: ' + JSON.stringify(message));
  8   console.log('a: ' + message.command);
  9 }).on('error', function (err) {
 10   console.log(err);
 11 });
 12 
 13 // sends a message to the Python script via stdin 
 14 pyshell.send({ command: "do_stuff", args: [1, 2, 3] });
 15 
 16 pyshell.on('close', function () {
 17   console.log('close');
 18 });

And when i run it with node app.js it is stuck, it doesn't output anything.

On the other hand, if I add

 22   pyshell.end(function (err) {
 23     if (err) throw err;
 24     console.log('finished');
 25   });}

I receive all the messages.

I don't get it. Why do I need to end the python script to get the messages?
I thought python-shell enabled live communication between nodejs and python script.

Am I wrong?
Thank you for you reply,
Emmanuel

PS: python script

  1 import sys, json
  2 
  3 # simple JSON echo script
  4 for line in sys.stdin:
  5   message = json.loads(line)
  6   print json.dumps(message)
  7   if message['command'] == 'do_stuff':
  8     message['command'] = 'do_it_yourself' 
  9     print json.dumps(message) 
 10  

Hi, I'm trying to pass script parameters through python-shell but I'm not quite sure how I should set them. The setup I use can execute normal scripts and catch the responses fine but the command I'm trying to pass here seems a bit trickier.

The commandline version I have tested and works is as follows: mpiexec -n 5 --hosts pi@raspi1, p2@raspi2 python MPI

Currently this is how I set the options:
var options = {
mode: 'text',
pythonPath: 'usr/bin/python',
pythonOptions: ['mpiexec','-n 5','--hosts','pi@raspi1','p2@raspi2'],
scriptPath: 'MPI',
args: [null]
};
var pyshell = new PythonShell('testcalc.py',options);

I've tried to play with where the options belong but I'm not making any progress.
Any thoughts?

Syntax for mpiexec:
mpiexec [global_args] local_args1 [: local_args2 [...]]
mpiexec [global_args] -configfile

(Man pages for mpiexec http://linuxcommand.org/man_pages/mpiexec1.html)

test.js file content:
var PythonShell = require('python-shell'); //external package
PythonShell.run('HelloWorld.py');

HelloWorld.py has following statement
print "Hello World"

When executing "node test.js" I'm getting following error. Any help is appreciated.

/home/pi/node_modules/python-shell/index.js:149
return callback(null, output.length ? output : null);
^
TypeError: undefined is not a function
at null._endCallback (/home/pi/node_modules/python-shell/index.js:149:16)
at ChildProcess. (/home/pi/node_modules/python-shell/index.js:99:35)
at ChildProcess.emit (events.js:110:17)
at Process.ChildProcess._handle.onexit (child_process.js:1074:12)

When I installed the module, I have the following error:

I think you should tell that: the default script path is in "python" directory.

I'm calling a python script that outputs JSON if everything goes right, but outputs an error message otherwise. When using {mode: 'json'}, a JSON parsing error is thrown instead of being returned through the callback.

SyntaxError: Unexpected token o
    at Object.parse (native)
    at asJson (...\node_modules\python-shell\index.js:142:21)
    at ...\node_modules\python-shell\index.js:232:35
    at Array.forEach (native)
    at PythonShell.receive (...\node_modules\python-shell\index.js:231:11)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:153:18)
    at Socket.Readable.push (_stream_readable.js:111:10)
    at Pipe.onread (net.js:531:20)

Hi,

I'm trying to send data to a python script that controls motors on my Raspberry Pi, but I'm confused about the recommended way of doing this. Up until now I was using something like:

function runMotor (speed) {
  var shell = new PythonShell('./script.py');
  shell.send(speed);
  shell.end( function(err) {
      if (err) throw err;
    }
  );
}

But I was getting some latency between sending the command and the motor moving, which I think is because there's a heavy overhead with instantiating a new instance of PythonShell each time and importing the modules every time on the python side.

I tried moving the shell creation outside of the function, and not calling shell.end() immediately, but then it blocks, I assume because it only starts reading the input once you call shell.end(). You also can't instantiate the shell and call send() and end() multiple times, so what is the proper way to do this?

I saw you mention using var shell = PythonShell.run('./script.py') instead in this issue: #16 (comment), but when I try that it complains "Error: write after end".

Is there a way to keep the script 'open' so that I don't incur an overhead from calling the constructor over and over and re-importing the modules the python script needs over and over?

Hi~
I have just run the word2vec model by using genesis module and save the model. And I want to call python in my sails app and load the model I have generated. And I got the following error. I have spending so much time on that but still couldn't figure out what's run.
Thanks ahead for your help!

here is my code:
word2vec.py
import collections
import gensim, logging
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)
model = gensim.models.Word2Vec.load('course_word2vec')

CourseController.js
var PythonShell = require('python-shell');
var courseid=req.param('courseid');
sails.log.debug(coursed);
var options = {
args: [courseid]
};
PythonShell.run('word2vec.py', options, function (err, results) {
if (err) throw err;
console.log('results: %s', results[0]);
});
return res.ok();
and the error is

`/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/api/controllers/SimilarcourseController.js:41
if (err) throw err;
^
Error: 2016-08-09 20:51:11,742 : INFO : loading Word2Vec object from course_word2vec
2016-08-09 20:51:11,804 : INFO : setting ignored attribute syn0norm to None
2016-08-09 20:51:11,804 : INFO : setting ignored attribute cum_table to None

at PythonShell.parseError (/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/node_modules/python-shell/index.js:190:17)
at terminateIfNeeded (/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/node_modules/python-shell/index.js:98:28)
at ChildProcess.<anonymous> (/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/node_modules/python-shell/index.js:88:9)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)

`

> [email protected] test /usr/local/lib/node_modules/python-shell
> mocha -R spec



  PythonShell
    #ctor(script, options)
      ✓ should spawn a Python process (75ms)
      ✓ should spawn a Python process with options (55ms)
      ✓ should spawn a Python process with script arguments (53ms)
    #run(script, options)
      1) should run the script and return output data
      ✓ should try to run the script and fail appropriately (47ms)
      ✓ should run the script and fail with an extended stack trace (57ms)
      ✓ should run multiple scripts and fail with an extended stack trace for each of them (446ms)
      2) should run multiple scripts and return output data for each of them
    .send(message)
      ✓ should send string messages when mode is "text" (61ms)
      ✓ should send JSON messages when mode is "json" (63ms)
      ✓ should use a custom formatter (60ms)
      ✓ should write as-is when mode is "binary" (51ms)
    .receive(data)
      ✓ should emit messages as strings when mode is "text" (60ms)
      ✓ should emit messages as JSON when mode is "json" (62ms)
      ✓ should properly buffer partial messages (59ms)
      ✓ should not be invoked when mode is "binary" (54ms)
      ✓ should use a custom parser function (61ms)
    .end(callback)
      ✓ should end normally when exit code is zero (57ms)
      ✓ should emit error if exit code is not zero (57ms)
      ✓ should emit error when data is written to stderr (53ms)
    .parseError(data)
      ✓ should extend error with context properties (51ms)
      ✓ should extend err.stack with traceback (54ms)


  20 passing (2s)
  2 failing

  1) PythonShell #run(script, options) should run the script and return output data:
     Uncaught TypeError: Cannot read property 'have' of undefined
      at test/test-python-shell.js:43:47
      at null._endCallback (index.js:166:16)
      at terminateIfNeeded (index.js:116:35)
      at ChildProcess.<anonymous> (index.js:88:9)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

  2) PythonShell #run(script, options) should run multiple scripts and return output data for each of them:
     Uncaught TypeError: Cannot read property 'have' of undefined
      at test/test-python-shell.js:102:51
      at null._endCallback (index.js:166:16)
      at terminateIfNeeded (index.js:116:35)
      at Socket.<anonymous> (index.js:78:9)
      at endReadableNT (_stream_readable.js:903:12)

Any idea how to resolve?

Sometimes I get the close/end event before all messages arrive - it happens rarely, but it happens, usually when I run many scripts in parallel.

here is an example:

    const runningScript = new PythonShell("script.py", scriptOptions);
    runningScript.on('message', (message) => {
      console.log("message", message);
    });
    runningScript.end(() => {
      console.log("close"); // sometimes logged before messages arrive
      console.log(runningScript.terminated); // always logs true
    });
    runningScript.stdout.on('end', () => {
      console.log(runningScript.terminated); // always logs true
      console.log("stdout ended");
    })

My workaround for this is to listen to the stdout 'end' event for now

Thanks for building out this tool! I use it all the time.

One issue I've run into time and time again is that I will have a bug in some logic I've written that's dependent on a message event firing.

The message event fires, my code has a bug in it, an error gets thrown; standard operating procedure.

Except, python-shell catches the error, with a log saying that the message itself is broken.

Needless to say, this makes it super difficult to track down the source of the bug!

The obvious solution is to just emit the event, and let the standard error handling system take over from there, so it can give the user some good information about where the source of the error is.

If you really want, you could add in a little warning in that error chain saying "This error might be caused by an invalid message from your python shell.", but still letting the rest of the error handling system take place.

Again, thanks for the useful tool! I'd love to hear your thoughts on error handling. If my approach doesn't make sense for this module, let me know and I'll just operate off my own fork.

Hey! This is a really cool module! Nice work!

How can I make it use Python 3 instead of Python 2?

Thanks!

Edit: nevermind, it seems django manage.py puts everything into stdout

Hi
I found a solution to return values : http://stackoverflow.com/questions/18231415/best-way-to-return-a-value-from-a-python-script
The best solution is :
import ...
def main():
# calculcate stuff
return [1,2,3]
How can I do to get the returned values ?
Thanks for your help.

This is my Node.js code:

var shell = new PythonShell('script.py');

shell.on('message', function(msg){
    console.log(msg);
});

shell.send('Hello');
shell.end();

And this is my Python script:

tmp = raw_input()
print tmp
# just get input and output the same message

I want to send the message and get message from Python script.
But it always get message from Python after shell.end()
If I remove shell.end(), the Python script will never run.
And I will not get the message from Python script.
Is there any way to run script before shell.end()

Is it possible to use PythonShell.run together with mode:JSON?
What is correct options structure?

Hi !
First thanks for the library it's helpful.
If I run this python script from the python command window :

   # application main loop
    for i in range(0,1000000):
      if i % 9893 == 0:
        print i
     # heavy crunching
      z = 1
      for j in range(0,100):
        z = z*j

I get one result after another (0 then it's crunching then 9893...)
But if I call this script from node & this python-shell module and use console.log with the message event, I only get the results (what's written in stdout, the line print i) once the program is done crunching numbers..

Do you have any idea how I might have async stdout with python-shell ?
Thanks in advance!

I'm using the following to call my python script:

var PythonShell = require('python-shell');
var pyshell = new PythonShell.run('/home/pi/Development/Sensor/python/SenseHat.py');

// get message back
pyshell.on('message', function (message) {
  // received a message sent from the Python script (a simple "print" statement) 
  console.log(message);
});

// end the input stream and allow the process to exit 
pyshell.end(function (err) {
  if (err) throw err;
  console.log('exit');
});

If I use the full path to the script it fails with the error below:

/home/pi/Development/Sensor/node/pythonTest.js:14
  if (err) throw err;
           ^
Error: python: can't open file 'home/pi/Development/Sensor/python/SenseHat.py': [Errno 2] No such file or directory

    at PythonShell.parseError (/home/pi/node_modules/python-shell/index.js:190:17)
    at terminateIfNeeded (/home/pi/node_modules/python-shell/index.js:98:28)
    at Socket.<anonymous> (/home/pi/node_modules/python-shell/index.js:78:9)
    at emitNone (events.js:72:20)
    at Socket.emit (events.js:166:7)
    at endReadableNT (_stream_readable.js:903:12)
    at doNTCallback2 (node.js:439:9)
    at process._tickCallback (node.js:353:17)

If I use the following path to the script it all works ok:
../python/SenseHat.py
and so does:
../../../../../home/pi/Development/Sensor/python/SenseHat.py

Hi,
I'm trying to create a module that uses python-shell, however I'm running into problems with referencing the python file I need to call.

I'm placing all the files into node_modules/test_module folder and in there, I've got a test_file.py and index.js.

Whatever I reference test_file.py with either directly or ./test_file.py, the application can't find the file. The only way is to reference it via the apps directory structure and in that case my module needs to refer to the file as: node_modules/test_module/test_file.py
Is there a better way to do this ?

Thanks

is there any way to get & kill a Python process already running?

I'm very new with node.js and I'm trying to make a web service for some Django command line tools.
In the python code of the utilities they use the input() for some questions.

For testing I've created a basic script:

hello.py

 print "Hello Mike!!"
 print "Hello Mike!!"
 print "Hello Mike!!"
 print "Hello Mike!!"
 print "Hello Mike!!"
 print "Hello Mike!!"
 print "Hello Mike!!"
 print "Hello Mike!!"
 x = input("Please")
 print "You're welcome %s" % x

I've copied some code of the demos an modified them a bit to check how they works:

var PythonShell = require('python-shell');
var options = {
    scriptPath: __dirname,
    mode: "text"
}
var pyshell = new PythonShell('hello.py', options);


var i = 0;
pyshell.on('message', function (message) {  
  console.log("Message #" + i, message);
  i++;
  if(message === "Please") {
    pyshell.send("Mike\n");
  }
});

pyshell.end(function (err) {
  if (err) {throw err;}
  console.log('finished');
});

I'm getting some troubles with this, and this is the error stack that I'm receiving:

$ node terminal.js                                                                                                                                      
Message #0 Hello Mike!!
Message #1 Hello Mike!!
Message #2 Hello Mike!!
Message #3 Hello Mike!!
Message #4 Hello Mike!!
Message #5 Hello Mike!!
Message #6 Hello Mike!!
Message #7 Hello Mike!!

/Users/usuario/node/nodeTerminal/terminal.js:28
  if (err) {throw err;}
                  ^
Error: EOFError: EOF when reading a line
    at PythonShell.parseError (/Users/usuario/node_modules/python-shell/index.js:131:17)
    at ChildProcess.<anonymous> (/Users/usuario/node_modules/python-shell/index.js:67:28)
    at ChildProcess.EventEmitter.emit (events.js:98:17)
    at Process.ChildProcess._handle.onexit (child_process.js:797:12)
    ----- Python Traceback -----
    File "/Users/usuario/node/nodeTerminal/hello.py", line 9, in <module>
      x = input("Please")

It would be nice to know how to make this work.
Thanks.

i'm trying to pass by params 2 base64:

var options = {
mode: 'json',
args: [databaseImage, clientImage] // Base64 encode Image
},
deferred = new Deferred();
pythonShell.run('./scripts/matchFace.py', options, function (err, result) {
if (err) {
deferred.reject();
} else {
deferred.resolve(result[0]);
}
});
return deferred.promise;

But i get this error: spawn ENAMETOOLONG .

My question: pythonShell have a size restriction in the arguments? or it's problem of the node?

Thanks and sorry for my bad english.

Can I know how to pass the args to the scripts? Is that pass to the python or global variable? Can give an example?

For some reason, this command can't find the file open-door.py:

python: can't open file '../py/open-door.py': [Errno 2] No such file or directory

Here is the code I use to run the python script:

PythonShell.run("open-door.py", { scriptPath: '../py' }, (err, result) => {
    if (err) {
        console.log("Script had an error: " + err);
        return;
    }

As you can see, open-door.py is inside the /py folder.

The script calling PythonShell.run(... is inside the /build folder, (or /ts folder, /ts builds into /build)

I cannot see whats wrong! I am using OS X El Capitan. Could that be the reason?

Here is the project if anyone wants to try or check it out:
https://github.com/dolanmiu/Automated-Chicken-Coop-2

I am continuously getting this memory issue after executing consecutive scripts, even when ran synchronously

Hi

i have a problem with your plugins.

// run a simple script
I have this script in node js
PythonShell.run('us.py', { mode: 'text'}, function (err, results) {
console.log(results);
response.send(results);
response.end();
});

And this print value in my python script
print "%.2f" % distance1
print "%.2f" % distance2

The console.log of the nodejs script is :
[ '42.38', '3.38' ]

And on the firefox web browser i obtain this error (firebug)
SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data at line 1 column 6 of the JSON data
json_data = jQuery.parseJSON($.trim(data));

I tried different thing but nothing good
change option (mode: json)
change my output on python

I think the problem become because there is a space between [ and ' but i m not sure.

If you have a solution it will be very cool :)

pyshell.on('message', function (message) {
console.log(message);
// once python sends a output it exits, but i has to look for event emission again right. forgive if i didn't understand " on " method properly.
});

will on keep on push the output instead of exit ? kind of polling

This is the example for exchanging data between Node and Python:

var PythonShell = require('python-shell');
var pyshell = new PythonShell('my_script.py');

// sends a message to the Python script via stdin
pyshell.send('hello');

pyshell.on('message', function (message) {
  // received a message sent from the Python script (a simple "print" statement)
  console.log(message);
});

// end the input stream and allow the process to exit
pyshell.end(function (err) {
  if (err) throw err;
  console.log('finished');
});

Would kindly give me an example of the python part. How does python receives the message sent from node and do something with it?

Hi,

How can I terminate prematurely the python script that's executed through the shell? Or is it a feature request?

Thanks a lot!

Many programs utilize carriage return in the output updates to the console without pushing a new line. It would be sweet if python-shell could handle these gracefully and fire a new event like 'crdata' so the consumer can gracefully handle these line updates.

stream-splitter seems like a good module that can help to accomplish this: https://github.com/samcday/node-stream-splitter

Thanks for this cool tool!

when i run this script i got a error message like this,

C:\nodejs\node.exe test_py.js

events.js:72
throw er; // Unhandled 'error' event
^
Error: spawn ENOENT
at errnoException (child_process.js:1001:11)
at Process.ChildProcess._handle.onexit (child_process.js:792:34)

Process finished with exit code 8

Could provide me a solution for this, or how could i run the sample program

Have a possible to execute a Python script but no wait a response and no wait a python script finish ?

I am calling yowsup-cli from node using python-shell and it fails
If I call it from a Bash Shell, it works.
Any clue ?

Asking yowsup for help : >>> tgalal/yowsup#1702

Nodejs code

app.post( '/enviar_msg_whatsapp/ParamTfNum=:req_tf_num', function (req, res) {

 var WhatsApp_Tf_Number = req.params.req_tf_num ;
 console.log( '>>> Menu enviar msg WhatsApp via python. Tf param is (' + WhatsApp_Tf_Number + '). ' ) ;

var python_options = {
mode: 'text',
pythonPath: '/usr/bin/python',
pythonOptions: ['-u'],
scriptPath: '/usr/local/bin',
args: ['']
} ;

 PythonShell.run( 'yowsup-cli', python_options, function( err, results ) {
      if ( err ) throw err ;
      console.log( '(+) Snd WhatsApp Python results are (%j).', results ) ; // results is an array consisting of messages collected during execution
      var sndRC = String( results ) ;                           // convert to string

      szResultat = '+++ whatsapp msg sent. RC ('+ sndRC + ').' ;
      res.status( 200 ).send( szResultat ) ;
 } ) ; // run

} ) ; // enviar mensage whatsapp

Execution result 👍

Menu enviar msg WhatsApp via python. Tf param is ("666777888").

/home/pi/semafor/1_sem.js:453
if ( err ) throw err ;
^

Error: process exited with code 1
at terminateIfNeeded (/home/pi/semafor/node_modules/python-shell/index.js:100:23)
at ChildProcess. (/home/pi/semafor/node_modules/python-shell/index.js:88:9)
at emitTwo (events.js:100:13)
at ChildProcess.emit (events.js:185:7)
at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)
pi@raspberrypi-llovet:~/semafor $

Thanks.

Solved using

args: [ 'demos', '-c', '/usr/local/bin/mydetails', '-s', '34666777888', 'Envio des el NODEJS' ]

Thanks so much for python-shell ! My web-app use python-shell as backend. It allows user to send their python code, run on my server, and return the stdout for them.

However, sometimes, users may make mistakes, like writing an infinite loop. So I try to set a timeout, for example 5s, i.e. If a user's code cannot exit within 5s, the python-shell will force to end running his code.

I wonder how this could be possible?

Hi,

I try to write a simple script to read the data, like in your example, but nothing is happening
(I've checked and script is producing data on stdout...), what did I missed ?

`module.exports = function(RED) {
function AccelerometerNode(config) {
var PythonShell = require('python-shell');

    RED.nodes.createNode(this,config);
    var node = this;

    script="accelerometer.py";
    //scriptPath=__dirname
    args=["--sensor_update_rate","3"];


    var ls = new PythonShell(script,{ scriptPath: __dirname});

    ls.on('data', function (data) {
      node.log("data recieved : " + data)
    });

    ls.end(function (err) {
      if (err) throw err;
      node.log('finished');
    });

}
RED.nodes.registerType("accelerometer",AccelerometerNode);

}`

Is it possible to run script with superuser privileges?
Like:
exec("sudo python some_script.py", function (error, stdout, stderr) {
//do smth;
});

Can i run script above using your python-shell?

If I'm right python-shell works for one task per time, with pysh.end end the shell and get returned data. Well, is there a way using this for long time communication and finally end the shell by pysh.send some special message such as 'shell:end' (binary/text) or {shell:'end'} (json mode)?

in order to run python -m mymodule using python shell I had to bend the expectations on how it is used.

        var shellOptions = {
            scriptPath: 'press',
            cwd: __dirname + '/scripts',
            pythonOptions: ['-m']
        };

        PythonShell.run('', shellOptions, function (err) {

I had to set the scriptPath to be the module name, I then had to set the cwd to be the path to the module, and setting the pythonOptions for -m was pretty normal. When running this module, I had to pass an empty string to PythonShell.run.

Am I abusing this or is it just not designed to work friendly with modules?

In function PythonShell.prototype.receive(), when data contains only one line, it is saved to lastLine by

var lastLine = lines.pop();

and lines[0] is undifined.

Code line 177

lines[0] = (this._remaining || '') + lines[0];

set lines[0] to be string 'undefined'. This will cause error to be throw at JSON.parse below.

Propose: change line 177 to

if (this._remaining) {
      lines[0] = this._remaining + lines[0];
}

I used the example given for sending and receiving data from a python script:

var PythonShell = require('python-shell');
var pyshell = new PythonShell('my_script.py');

// sends a message to the Python script via stdin 
pyshell.send('hello');

pyshell.on('message', function (message) {
  // received a message sent from the Python script (a simple "print" statement) 
  console.log(message);
});

// end the input stream and allow the process to exit 
pyshell.end(function (err) {
  if (err) throw err;
  console.log('finished');
});

When i replace my_script.py with my python file name, the python script is in the same folder as the javascript file, when i run it, i get this error:

app.js:14 Uncaught Error: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'livestreamerApp.py': [Errno 2] No such file or directory

How do i fix this?

NPM somehow holds a 0.3.0 version. I don't see that anywhere in the repo, neither does the git commit in package.json on NPM.

Latest commit also failing dfc6721

¯\_(ツ)_/¯

  19 passing (214ms)
  1 failing

  1) PythonShell #run(script, options) should run the script and return output data:
     Uncaught TypeError: Cannot read property 'have' of undefined
      at test/test-python-shell.js:43:47
      at null._endCallback (index.js:149:16)
      at ChildProcess.<anonymous> (index.js:99:35)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

Hi,

Is it possible to execute script from node.js as root?

Thanks

Getting the following error:

ERROR in ./~/python-shell/index.js
Module not found: Error: Cannot resolve module 'child_process' in /path/to/file
 @ ./~/python-shell/index.js 4:12-36

When in json mode, if a message received is larger than the buffer, the listener thinks it has received the last message and tries to parse and emit the object, which results in a json exception.

This is the code here (index.js, line ~171):
PythonShell.prototype.receive = function (data) {
var self = this;
var lines = (''+data).split(/\n/g);
var lastLine = lines.pop();

    // fix the first line with the remaining from the previous iteration of 'receive'
    lines[0] = (this._remaining || '') + lines[0];
    // keep the remaining for the next iteration of 'receive'
    this._remaining = lastLine;

    lines.forEach(function (line) {
        if (self.mode === 'json') {
            try {
                self.emit('message', JSON.parse(line));
            } catch (err) {
                self.emit('error', extend(
                    new Error('invalid JSON message: ' + data),
                    { inner: err, data: data}
                ));
            }
        } else {
            self.emit('message', line);
        }
    });
};

Say the buffer is twenty characters. If stdout gets something like:
{test1:'abc'}\n{test2:'defghijklmnopqrstuvwxyz'}\n
At first, it gets "{test1:'abc'}\n{test", emits "{test1:'abc'}", and stores "{test" in lastLine. So far so good.
Next, it gets "2:'defghijklmnopqrst", prepends lastLine such that "lines[0]" is "{test2:'defghijklmnopqrst", and then proceeds to try to parse it, instead of realizing that the object isn't done and waiting for the rest.

To get around this, I had to replace lines 176-179 with this:
if (lines.length === 0) {
// Only one line of text was received, and it did not contain a \n character, so the rest of the object is
// still in the buffer.
// keep the remaining for the next iteration of 'receive'
this._remaining = this._remaining + lastLine;
} else {
// fix the first line with the remaining from the previous iteration of 'receive'
lines[0] = (this._remaining || '') + lines[0];
this._remaining = lastLine;
}

When it receives a message that doesn't contain a \n it means it isn't a complete json object yet, and "lines" becomes empty after the .pop() call. In it's current state, "lines" is instantly repopulated, causing it to try to run JSON.parse() on this._remaining + lastLine, effectively. In the fix, "lines" stays empty if the data did not contain \n, and therefore the issue is avoided.

Hey - I'm trying to the send/message functionality on python-shell to exchange data between a long-running python script and node, but I'm seeing that the messages don't get handled till I end the input stream. For example, the following sample code does not invoke the 'message' callback (or 'close' or 'error):

var pyshell = new PythonShell('python/helloworld.py', options);

pyshell.send({'hello': 'world'});

pyshell.on('message', function(message) {
  console.log("message - ", message);
}).on('close', function (result) {
  console.log("close - ", result);
}).on('error', function (err) {
  console.log("error")
});

The python is the same as your echo script.

import sys, json

# simple JSON echo script
for line in sys.stdin:
  print json.dumps(json.loads(line))

Now, if I close the connection after the above code, then I get all the messages:

pyshell.end(function (err) {
  if (err) throw err;
  console.log('finished');
});

This won't work for me as my python script maintains state and I don't want to re-run the script every time. I'd rather have a long running child process I could pass messages back and forth from. This seems like a bug. Any suggestions?

Is there any options to run the scripts inside a virtualenv context ? to be able to see the modules installed into the environment and execute the binaries inside ?

First, thanks for the great work. I'm using your python-shell often.

I'm running a sub command in my python script using subprocess.call. The thing is this call is returning a warning, yet no matter what I do it is getting returned as an error to python-shell so that I can never see my results in the node script.

{ [Error: Warning 1: Value 14155551400 of field Shape_Area of feature 0 not successfully written. Possibly due to too larger number with respect to field width
Warning 1: Value 11811799300 of field Shape_Area of feature 1 not successfully written. Possibly due to too larger number with respect to field width
]
  executable: 'python',
  options: null,
  script: 'python/percentOverlay.py',

Hi~

I came across problem in the #52 again, this time I wanted to import keras in my python program. And when I import the keras, it will show 'Using Theano backend' and it will be regarded as an error. @extrabacon has mentioned in #52 that to override PythonShell.prototype.parseError, could you @extrabacon explain more details about how to implement that. Also is there any other method? such as avoid kears from printing that, because I do not want to lose python traceback
Thank you ahead for your help!

`/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/api/controllers/NextcourseController.js:30
if (err) throw err;
^

Error: Using Theano backend.

at PythonShell.parseError (/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/node_modules/python-shell/index.js:190:17)
at terminateIfNeeded (/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/node_modules/python-shell/index.js:98:28)
at ChildProcess.<anonymous> (/Users/luoyuetian/Desktop/junior/summerproject/course_recommendation_interface/node_modules/python-shell/index.js:88:9)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)`