How to run multiple packages in development inside a Monorepo

Question

I've a monorepo using lerna and yarn workspaces for Frontend applications and libs. I add series of npm scripts at root package.json, to manage each package (app, lib, plugin) like the code at end of this post. The problem is that my root package.json is getting much large and I just started. Other problem is if I need to develop a component used in the CRM, inside the components lib, that uses i18n lib I need to start 3 packages in 3 separate terminal window and it is possible that I have more dependant packages to develop simultaneously.

I think in put start:crm for example in each package (using --parallel) that is installed in the CRM, not a good idea, I think.

And I see an example that the files are being imported from another packages through the file system (becoming part of the file watch off running application), ignoring the package root folder and I have fear of this approach because it breaks the versioning.

How you guys are handling these problems?

"/** CRM SCRIPTS */": "",
"start:crm": "npx lerna run start --scope @kovi-cx-frontend/crm",
"build:crm": "npx lerna run build --stream --scope @kovi-cx-frontend/crm",
"test:crm": "npx lerna run test --stream --scope @kovi-cx-frontend/crm",
"lint:crm": "npx lerna run lint --stream --scope @kovi-cx-frontend/crm",
"codegen:crm": "npx lerna run codegen --scope @kovi-cx-frontend/crm",
"start:crm:storybook": "npx lerna run start:storybook --stream --scope @kovi-cx-frontend/crm",
"build:crm:storybook": "npx lerna run build:storybook --stream --scope @kovi-cx-frontend/crm",

Answer 1

Appropriate tooling is a must here.

I had similar problem: Yarn workspaces with tens of packages depending on each other deeply. And my package.json looked like this:

{
  "scripts": {
    "all": "run-s \"generic all\" \"all:apps all\" --",
    "all:apps": "run-p --aggregate-output \"eslint-config {@}\" \"apps1 {@}\" \"apps2 {@}\" --",
    "apps1": "run-s \"apps1:deps {@}\" \"apps1:c {@}\" --",
    "apps1:c": "run-p --aggregate-output \"hatsy:c {@}\" --",
    "apps1:deps": "run-s \"route-match:c {@}\" --",
    "apps2": "run-p --aggregate-output \"siteparts:c {@}\" \"realworld-app:c {@}\" \"examples:c {@}\" --",
    "build": "run-s \"generic build {@}\" \"all:apps build {@}\" --",
    "lint": "run-p --aggregate-output \"generic lint {@}\" \"all:apps lint {@}\" --",
    "test": "run-p --aggregate-output \"generic test {@}\" \"test:apps test {@}\" --",
    "test:apps": "run-p --aggregate-output \"apps1:deps {@}\" \"apps1:c {@}\" \"siteparts:c {@}\" \"realworld-app:c {@}\" --",
    "a-iterable": "run-s \"call-thru {@}\" \"a-iterable:c {@}\" --",
    "a-iterable:c": "run-s -l \"a-iterable:x {@}\" --",
    "a-iterable:x": "yarn workspace @proc7ts/a-iterable",
    "call-thru": "run-s \"primitives:c {@}\" \"call-thru:c {@}\" --",
    "call-thru:c": "run-s -l \"call-thru:x {@}\" --",
    "call-thru:x": "yarn workspace @proc7ts/call-thru",
    "context-values": "run-s \"call-thru {@}\" \"context-values:deps {@}\" \"context-values:c {@}\" --",
    "context-values:c": "run-s -l \"context-values:x {@}\" --",
    "context-values:deps": "run-p --aggregate-output \"a-iterable:c {@}\" \"fun-events:c {@}\" --",
    "context-values:x": "yarn workspace @proc7ts/context-values",
    "delta-set": "run-s \"delta-set:c {@}\" --",
    "delta-set:c": "run-s -l \"delta-set:x {@}\" --",
    "delta-set:x": "yarn workspace @proc7ts/delta-set",
    "eslint-config": "run-s \"eslint-config:c {@}\" --",
    "eslint-config:c": "run-s -l \"eslint-config:x {@}\" --",
    "eslint-config:x": "yarn workspace @proc7ts/eslint-config",
    "fun-events": "run-s \"call-thru {@}\" \"fun-events:c {@}\" --",
    "fun-events:c": "run-s -l \"fun-events:x {@}\" --",
    "fun-events:x": "yarn workspace @proc7ts/fun-events",
    "hatsy": "run-s \"hatsy:deps {@}\" \"hatsy:c {@}\" --",
    "hatsy:c": "run-s -l \"hatsy:x {@}\" --",
    "hatsy:deps": "run-p --aggregate-output \"route-match {@}\" \"http-header-value:c {@}\" --",
    "hatsy:x": "yarn workspace @hatsy/hatsy",
    "http-header-value": "run-s \"http-header-value:c {@}\" --",
    "http-header-value:c": "run-s -l \"http-header-value:x {@}\" --",
    "http-header-value:x": "yarn workspace @hatsy/http-header-value",
    "route-match": "run-s \"primitives:c {@}\" \"route-match:c {@}\" --",
    "route-match:c": "run-s -l \"route-match:x {@}\" --",
    "route-match:x": "yarn workspace @hatsy/route-match",
    "input-aspects": "run-s \"input-aspects:deps0 {@}\" \"input-aspects:deps1 {@}\" \"input-aspects:c {@}\" --",
    "input-aspects:c": "run-s -l \"input-aspects:x {@}\" --",
    "input-aspects:deps0": "run-p --aggregate-output \"call-thru {@}\" \"namespace-aliaser:c {@}\" \"render-scheduler:c {@}\" --",
    "input-aspects:deps1": "run-p --aggregate-output \"a-iterable:c {@}\" \"fun-events:c {@}\" \"delta-set:c {@}\" --",
    "input-aspects:x": "yarn workspace @proc7ts/input-aspects",
    "namespace-aliaser": "run-s \"namespace-aliaser:c {@}\" --",
    "namespace-aliaser:c": "run-s -l \"namespace-aliaser:x {@}\" --",
    "namespace-aliaser:x": "yarn workspace @proc7ts/namespace-aliaser",
    "primitives": "run-s \"primitives:c {@}\" --",
    "primitives:c": "run-s -l \"primitives:x {@}\" --",
    "primitives:x": "yarn workspace @proc7ts/primitives",
    "render-scheduler": "run-s \"render-scheduler:c {@}\" --",
    "render-scheduler:c": "run-s -l \"render-scheduler:x {@}\" --",
    "render-scheduler:x": "yarn workspace @proc7ts/render-scheduler",
    "style-producer": "run-s \"style-producer:deps0 {@}\" \"style-producer:deps1 {@}\" \"style-producer:c {@}\" --",
    "style-producer:c": "run-s -l \"style-producer:x {@}\" --",
    "style-producer:deps0": "run-p --aggregate-output \"call-thru {@}\" \"namespace-aliaser:c {@}\" \"render-scheduler:c {@}\" --",
    "style-producer:deps1": "run-p --aggregate-output \"a-iterable:c {@}\" \"fun-events:c {@}\" --",
    "style-producer:x": "yarn workspace @proc7ts/style-producer",
    "examples": "run-s \"generic {@}\" \"examples:c {@}\" --",
    "examples:c": "run-s -l \"examples:x {@}\" --",
    "examples:x": "yarn workspace @wesib/examples",
    "generic": "run-s \"wesib {@}\" \"generic:deps {@}\" \"generic:c {@}\" --",
    "generic:c": "run-s -l \"generic:x {@}\" --",
    "generic:x": "yarn workspace @wesib/generic",
    "generic:deps": "run-p --aggregate-output \"http-header-value:c {@}\" \"input-aspects:c {@}\" \"style-producer:c {@}\" --",
    "realworld-app": "run-s \"generic {@}\" \"realworld-app:c {@}\" --",
    "realworld-app:c": "run-s -l \"realworld-app:x {@}\" --",
    "realworld-app:x": "yarn workspace @wesib/realworld-app",
    "wesib": "run-s \"wesib:deps {@}\" \"wesib:c {@}\" --",
    "wesib:c": "run-s -l \"wesib:x {@}\" --",
    "wesib:x": "yarn workspace @wesib/wesib",
    "wesib:deps": "run-p --aggregate-output \"context-values {@}\" \"namespace-aliaser:c {@}\" \"render-scheduler:c {@}\" --",
    "siteparts": "run-s \"generic {@}\" \"siteparts:deps {@}\" \"siteparts:c {@}\" --",
    "siteparts:c": "run-s -l \"siteparts:x {@}\" --",
    "siteparts:deps": "run-s \"route-match:c {@}\" \"hatsy:c {@}\" --",
    "siteparts:x": "yarn workspace @surol/siteparts"
  }
}

This is definitely not a way to go.

So I've created run-z.

package.json looks like this now:

{
  "scripts": {
    "each": "run-z ...proc7ts ...hatsy ...wesib ...siteparts",
    "each:p": "run-z --bap ...dev-kit ...run-z ...each",
    "+dev-kit/*": "run-z +z ./dev-kit...dev-kit",
    "hatsy/*": "run-z +z ./hatsy// ./hatsy/kit/packages//",
    "proc7ts/*": "run-z +z ./proc7ts//",
    "+run-z/*": "run-z +z ./run-z//",
    "siteparts/*": "run-z +z ./siteparts",
    "wesib/*": "run-z +z ./wesib//",
    "z": "run-z +z:clean +z:test",
    "z:clean": "run-z +each:p/clean",
    "z:test": "run-z +each:p/lint,+each:p/cmd:jest/--runInBand"
  }
}

So, now I can:

• run yarn z clean build --all to rebuild all packages,
• run yarn z build test --only hatsy to build and test a named subset of packages,
• run yarn build --with-deps inside particular package directory to build it along with its dependencies,
• allow some tasks to run in parallel (e.g. lint and test),

...and many more.

run-z takes some time to learn, but the result is satisfying.

Btw, you can try it without installing it. E.g. the following command

npx run-z build:crm build:crm:storybook test:crm,lint:crm start:crm,start:crm:storybook

Would build CRM and storybook, then test and lint (in parallel to each other), then start both CRM and storybook (again, in parallel to each other).