16

We're starting to adopt a monorepo setup using yarn workspaces and we'd like to have our firebase functions inside it. The repo structure is something like:

repo
    node_modules <- all dependencies
    packages
        core
        commom
        functions <- firebase functions

So, I have 2 problems with this setup:

  1. The dependencies of the functions don't live on the same folder as the entry file from functions
  2. The functions depends on other packages such as core and commom that are in the repo so yarn symlinks from node_modules to the packages in the repo.

Is there anyway I can handle this?

tk421
  • 5,288
  • 6
  • 22
  • 32
Thiago Nascimento
  • 935
  • 9
  • 31

3 Answers3

3

The solution I found for this is Yarn's nohoist option in your root package.json file.

By default Yarn hoists dependencies to the root directory so they can be shared between your packages. Unfortunately this will not work with Firebase. This means you need to tell Yarn not to hoist the dependencies used by your Firebase functions.

The documentation for nohoist is less than ideal, but here is an official blog post about it here: https://yarnpkg.com/blog/2018/02/15/nohoist/

You probably want something like this:

{
  "workspaces": {
    "packages": [
      "packages/*"
    ],
    "nohoist": [
      "functions/core",
      "functions/common",
      "functions/**"
    ]
  }
}

Keep in mind that this uses the name field used in the package.json files of each workspace package. So in this example, it is assume that the functions directory has a package.json with "functions" as it's name.

functions/** tells yarn not to hoist any of the dependencies specified in packages/functions/package.json. This doesn't work for your shared yarn packages though, so functions/core and functions/common need to be specified separately.

You also need to include your workspaces as dependencies in your functions project, so add them to your package.json:

{
  "name": "functions",
  "dependencies": {
    "core": "*",
    "common": "*",
  }
}

Once you have added all that, you should delete your packages/functions/node_modules directory and run yarn install. After doing this, you should see all your dependencies included in packages/functions/node_modules (not symlinks).

twiz
  • 6,335
  • 5
  • 35
  • 67
  • I tried this approach but was unable to deploy the functions. The firebase process is unable to install the dependencies (since they're local) – Thiago Nascimento Jan 05 '20 at 17:12
  • @ThiagoNascimento hmmm. I did this a while ago, and had forgot to post this answer. I may be forgetting another step in the process. I'll take a look. Thanks for letting me know. I'll update it if I figure anything out. – twiz Jan 05 '20 at 19:15
  • I've created a repository to demonstrate the use of this technique, however I seem to having issues when deploying the functions to Firebase, even though the web (hosting) aspect works fine. Any help would be greatly appreciated - I've added the errors to the Readme https://github.com/cjmyles/firebase-monorepo – Craig Myles Apr 03 '20 at 14:46
  • 1
    I also wasn't able to get this to work with cloud functions. While node_modules contains the shared core package, the deployment of cloud functions doesn't use node_modules. So, I was left with creating a packaged version of my shared dependency and then updating packages.json. – R. Wright May 06 '20 at 13:58
0

I am not sure I understand the question exactly, but I could give you my two cents on yarn workspaces based on whatever I understood from your question and from my experience using it.

Yarn workspaces consolidate all your dependencies into the node_modules present in project root as well as in a single package-lock.json to reduce conflicts and enables yarn to optimize the installation process giving you a faster yarn install. And also another advantage of it is, with a single pass yarn install can install dependencies of all packages under the workspace.

Edit: I think for some reason yarn link is not being called and instead only yarn install is being run, which will search the npm registries and throws the error mentioned in comment since it can't find the mentioned package on npm registry. So for a solution try creating an entry in the firebase's package.json like

"dependencies": {
  "a": "file:../dependency-package-name/",
}
PrivateOmega
  • 1,708
  • 10
  • 23
  • I know how yarn workspaces work and I'm currently using in the project. The problem that I'm having is that, when importing another local package from the same project, yarn takes care of symlinking it so instead of pointing to node_modules it will actually import from the local package. And that's the issue, firebase functions doesn't seem to work with symlinked packages that aren't in node_modules – Thiago Nascimento Jun 18 '19 at 17:29
  • Now the question is much clearer for me. Yes yarn symlinks to package in same workspace. But what issue are you facing exactly. It should work just like an npm installed package. – PrivateOmega Jun 18 '19 at 17:30
  • when deploying to firebase functions, I get an error that some X package (which is local) wasn't found – Thiago Nascimento Jun 18 '19 at 21:49
  • In the project, I am guessing yarn.lock file is there because if it is not, Google cloud installs using npm which might not do the symlinking I guess. – PrivateOmega Jun 19 '19 at 01:37
  • Yes, yarn.lock is there. Does Google cloud even support yarn? Maybe that's the point – Thiago Nascimento Jun 19 '19 at 01:57
  • Yes Google cloud supports yarn. We might be missing something though. Did you check the logs for the command executions. Maybe it would shed a light on what is being done underneath when you deploy it. – PrivateOmega Jun 19 '19 at 02:00
  • The error is the following: `Module @local/XXXX not found in npm registry`. I checked and the yarn.lock file is in the root of the directory and not in the functions folder. Could that be the issue? – Thiago Nascimento Jun 19 '19 at 15:21
  • @ThiagoNascimento I think I got the problem, try the solution I mentioned in the Edit. – PrivateOmega Jun 20 '19 at 03:29
0

With Yarn 2 node_modules aren't fetched and placed into in the respective functions directory (as it would be the case with calling npm i in the functions directory). So when calling firebase deploy --project default --only function the node_modules folder is missing and firebase will complain about this and abort the deployment process with the following error (or similar):

Error parsing triggers: Cannot find module [...]
Try running "npm install" in your functions directory before deploying.

There are two github issues that are tracking this issue at the moment:

In the two issues above, several clever workarounds are presented by firebase users, e.g. using webpack to create a build that contains all the local packages in the release or using rsync or other tools that rewire the packages before release.

Another solution is not hoisting your project packages, if that is possible. You can do this, be adding the following two directives to your .yarnrc.yml file.

# yarnrc.yml

# disables yarn's plugnplay style and uses node_modules instead
nodeLinker: node-modules
# makes sure the node_modules are not hoisted to the (monorepo) project root
nmHoistingLimits: "dependencies"

The two directives above are explained in the yarnrc configuration docs as follows:

nmHoistingLimits Defines the highest point where packages can be hoisted. One of workspaces (don't hoist packages past the workspace that depends on them), dependencies (packages aren't hoisted past the direct dependencies for each workspace), or none (the default, packages are hoisted as much as possible). This setting can be overriden per-workspace through the installConfig.hoistingLimits field.

nodeLinker Defines what linker should be used for installing Node packages (useful to enable the node-modules plugin), one of: pnp, node-modules.

B12Toaster
  • 8,944
  • 5
  • 48
  • 48