CODEWITHSUNDEEP
deno
Andrius Solopovas
Andrius Solopovas

Reputation: 1057

How to re-create an equivalent to linux bash statement in Deno with Deno.Run

How to re-create an equivalent to following Linux bash statement in Deno?

docker compose exec container_name -uroot -ppass db_name < ./dbDump.sql

I have tried the following:

    const encoder = new TextEncoder
    const p = await Deno.run({
        cmd: [
            'docker',
            'compose',
            'exec',
            'container_name',
            'mysql',
            '-uroot',
            '-ppass',
            'db_name',
        ],
        stdout: 'piped',
        stderr: 'piped',
        stdin: "piped",
    })
    
    await p.stdin.write(encoder.encode(await Deno.readTextFile('./dbDump.sql')))
    await p.stdin.close()
    await p.close()

But for some reason whenever I do it this way I get an error ERROR 1064 (42000) at line 145: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version which does not happen when I perform the exact same command in the bash.

Could someone please explain me how it has to be done properly?

Upvotes: 0

Views: 249

Answers (2)

danopia
danopia

Reputation: 441

Without a sample input file, it's impossible to be certain of your exact issue.

Given the context though, I suspect that your input file is too large for a single proc.stdin.write() call. Try using the writeAll() function to make sure the full payload goes through:

import { writeAll } from "https://deno.land/[email protected]/streams/conversion.ts";
await writeAll(proc.stdin, await Deno.readFile(sqlFilePath));

To show what this fixes, here's a Deno program pipe-to-wc.ts which passes its input to the Linux 'word count' utility (in character-counting mode):

#!/usr/bin/env -S deno run --allow-read=/dev/stdin --allow-run=wc
const proc = await Deno.run({
  cmd: ['wc', '-c'],
  stdin: 'piped',
});

await proc.stdin.write(await Deno.readFile('/dev/stdin'));
proc.stdin.close();
await proc.status();

If we use this program with a small input, the count lines up:

# use the shebang to make the following commands easier
$ chmod +x pipe-to-wc.ts

$ dd if=/dev/zero bs=1024 count=1 | ./pipe-to-wc.ts
1+0 records in
1+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000116906 s, 8.8 MB/s
1024

But as soon as the input is big, only 65k bytes are going through!

$ dd if=/dev/zero bs=1024 count=100 | ./pipe-to-wc.ts
100+0 records in
100+0 records out
102400 bytes (102 kB, 100 KiB) copied, 0.0424347 s, 2.4 MB/s
65536

To fix this issue, let's replace the write() call with writeAll():

#!/usr/bin/env -S deno run --allow-read=/dev/stdin --allow-run=wc
const proc = await Deno.run({
  cmd: ['wc', '-c'],
  stdin: 'piped',
});

import { writeAll } from "https://deno.land/[email protected]/streams/conversion.ts";
await writeAll(proc.stdin, await Deno.readFile('/dev/stdin'));
proc.stdin.close();
await proc.status();

Now all the bytes are getting passed through on big inputs :D

$ dd if=/dev/zero bs=1024 count=1000 | ./pipe-to-wc.ts
1000+0 records in
1000+0 records out
1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0854184 s, 12.0 MB/s
1024000

Note that this will still fail on huge inputs once they exceed the amount of memory available to your program. The writeAll() solution should be fine up to 100 megabytes or so. After that point you'd probably want to switch to a streaming solution.

Upvotes: 1

jsejcksn
jsejcksn

Reputation: 33749

First, a couple of notes:

  1. Deno currently doesn't offer a way to create a detached subprocess. (You didn't mention this, but it seems potentially relevant to your scenario given typical docker compose usage) See denoland/deno#5501.

  2. Deno's subprocess API is currently being reworked. See denoland/deno#11016.

Second, here are links to the relevant docs:

Now, here's a commented breakdown of how to create a subprocess (according to the current API) using your scenario as an example:

module.ts:

const dbUser = 'actual_database_username';
const dbPass = 'actual_database_password'
const dbName = 'actual_database_name';

const dockerExecProcCmd = ['mysql', '-u', dbUser, '-p', dbPass, dbName];

const serviceName = 'actual_compose_service_name';

// Build the run command
const cmd = ['docker', 'compose', 'exec', '-T', serviceName, ...dockerExecProcCmd];

/**
 * Create the subprocess
 *
 * For now, leave `stderr` and `stdout` undefined so they'll print
 * to your console while you are debugging. Later, you can pipe (capture) them
 * and handle them in your program
 */
const p = Deno.run({
  cmd,
  stdin: 'piped',
  // stderr: 'piped',
  // stdout: 'piped',
});

/**
 * If you use a relative path, this will be relative to `Deno.cwd`
 * at the time the subprocess is created
 *
 * https://doc.deno.land/deno/stable/~/Deno.cwd
 */
 const sqlFilePath = './dbDump.sql';

// Write contents of SQL script to stdin
await p.stdin.write(await Deno.readFile(sqlFilePath));

/**
 * Close stdin
 *
 * I don't know how `mysql` handles `stdin`, but if it needs the EOT sent by
 * closing and you don't need to write to `stdin` any more, then this is correct
 */
p.stdin.close();

// Wait for the process to finish (either OK or NOK)
const {code} = await p.status();
console.log({'docker-compose exit status code': code});

// Not strictly necessary, but better to be explicit
p.close();

Upvotes: 1

Related Questions