CODEWITHSUNDEEP
javascriptpythonnode.js
mamur
mamur

Reputation: 371

how to send encoded data from python to nodejs

I want to send some string data from Python3 to nodeJs. The string is Korean characters and I am encoding it to utf8.(Cause I don't know other ways sending data safely.) When I send(from python) it is ByteStream and in nodeJs I receive it as Array. I convert this Array to String. But now I cannot decode string back to original Korean characters. Here are some codes I am using.

python

input = sys.argv[1]
d = bot.get_response(input)
data = str(d).encode('utf8')

print(data)

nodeJs

var utf = require('utf8');
var python = require('python-shell');

var pyt = path.normalize('path/to/my/python.exe'),
        scrp = path.normalize('path/to/my/scriptsFolder/'),


    var options = {
        mode: 'text',
        pythonPath: pyt,
        pythonOptions: ['-u'],
        scriptPath: scrp,
        encoding: 'utf8',
        args: [message]
    };

    python.run('test.py', options, function (err, results) {

        //here I need to decode 'results'
        var originalString = utf.encode(results.toString());// that code is not working for me

    });

I have used several libs like utf8 to decode but didn't help. Can someone please give some idea how to make it work.

EDIT

I have to edit with some more info. I have tried @smarx approach but did not work.

I have two cases:

1. if I send data as string from python here is what I get in nodeJs b'\xec\x95\x88\xeb\x85\x95\xed\x95\x98\xec\x8b\xad\xeb\x8b\x88\xea\xb9\x8c? \xec\x9d\xb4\xed\x9a\xa8\xec\xa2\x85 \xea\xb3\xa0\xea\xb0\x9d\xeb\x8b\x98! \xeb\x8f\x99\xec\x96\x91\xeb\xa7\xa4\xec\xa7\x81\xec\x9e\x85\xeb\x8b\x88\xeb\x8b\xa4

2. if I encode data and send. I get �ȳ��Ͻʴϱ�? ��ȿ�� ������! �

Upvotes: 2

Views: 2678

Answers (4)

Theia17
Theia17

Reputation: 1

I had same trouble in using data(string) from python in node js.

I solved this problem in this way:

Try change default code page of Windows Console to UTF-8, if your code page of Windows Console is not UTF-8. (In my case default code page was CP949.)

In my case:

I got message like ������ 2���� ��������.

I tried encoding on online (http://code.cside.com/3rdpage/us/url/converter.html)

then I found my strings encoded cp949 -> decoded utf-8.

Upvotes: 0

Sol Han
Sol Han

Reputation: 21

I had the totally same issue on my project and now I finally found the answer.

I solved my problem by using these codes.

It works on windows (macOS and Linux, their default system encoding it 'utf8' so the issue doesn't happen).

I hope it might help you, too!

#in the python file that your javascript file will call by python-shell module put those code
import sys    
sys.stdout.reconfigure(encoding='utf-8')

I found the hints from python-shell description.

feature >Simple and efficient data transfers through stdin and stdout streams

Upvotes: 2

cswu
cswu

Reputation: 361

I have the same issue when using python-shell.

Here is my solution:

The string after .encode('utf-8') is a binary string. So you need to print it on stdout directly.

in test.py, it print a utf-8 json which include some chinese char:

sys.stdout.buffer.write(json.dumps({"你好":"世界"}, ensure_ascii=False).encode('utf8'))
print() # print \n at ending to support python-shell in json mode

in main.js

let opt = {mode: 'json', pythonOptions: ['-u'], pythonPath: 'python', encoding: 'utf8'}

let pyshell = new PythonShell('lyric.py', opt);
pyshell.on('message', function (message) {

    console.log(message);  //*** The console msg may still wrong (still ���)
    let json = JSON.stringify(message);
    let fs = require('fs');
    fs.writeFile('myjsonfile.json', json, 'utf8', function () {
    }); //*** The output json file will be correct utf8 output
});

result: enter image description here This shows the msg is correctly receive in utf-8, because the json output is correct. However console.log output apparently failed. I don't know is there any way to fix console.log output. (Windows 10)

Upvotes: 0

user94559
user94559

Reputation: 60153

I'm still not sure what python.run does, since you won't share that code, but here's my version of the code, which is working fine:

test.py

print("안녕 세상")

app.js

const { exec } = require('child_process');

exec('python3 test.py', function (err, stdout, stderr) {
    console.log(stdout);
});

// Output:
// 안녕 세상

Upvotes: 1

Related Questions