Performance improvements on complex Meteor apps

performance-analysis.md

Performance improvements on complex Meteor apps

We are developing what will become a fairly complex Meteor application that is replacing an existing Ruby/Rails application. This application has been in development for approximately one year, from Meteor 1.2.1 version to 1.4.2.3, which is the one we currently use. We use React.

The staging instance, hosted on Galaxy, is available here: http://admin.staging.pathable.com. Feel free to create a community.

Currently we face a few critical issues related to application size and initial load times. While the performance in development has recently become just barely tenable, the performance of the deployed, production application on Galaxy is not.

App and package infrastructure

First, here is a brief overview of our application structure and approach. You are welcome to have access to our GitHub repository access should you want to review our code directly.

There are two main applications named app and admin. These include both NPM dependencies and Meteor packages.

According to what we understood to be the recommended best practices at the time, we tried to compose our application from focused, independent Meteor packages. While we have 7 packages currently in total (with plans for more) the main ones are:

• api - define the common methods and publications.
• collections - define the collections, models and factories.
• schema - define the schema for the models and methods, publications and components signature.
• ui - define the visual and data components shared on the apps.
• utilities - define the helpers, constants, and so on.
• others for styles, build plugins,...

In effect, our apps and packages are setup like this:

app or admin
|_ NPM & Meteor dependencies
|_ api
|_ collections
|_ schema
|_ ui
|_ utilities

api
|_ NPM & Meteor dependencies
|_ collections
|_ schema
|_ utilities

collections
|_ NPM & Meteor dependencies
|_ schema
|_ utilities

schema
|_ NPM & Meteor dependencies
|_ utilities

ui
|_ NPM & Meteor dependencies
|_ api
|_ collections
|_ utilities

Critical Performance Issues

Below are timing profiles (via Chrome Tools) loading our application after it's been packaged and deployed to Galaxy:

For admin app,

For app app,

As you can see the times spent on both are quite similar but surprisingly high. The next graphic shows the time to download packages. Our application is as yet relatively small so the large size is surprising.

And finally the next picture shows the timeline for the admin case (app is similar but it gets less time to download and load app.js since it implements less content).

Going through the analysis we have noted the large amount of time spent in both downloading and loading the packages and the app itself. The app execution could be also something to look into, it involves subscription and other network tasks related, however the time spent before feels more considerable and we want to understand why, since our app is small.

Possible causes and solutions

After getting some time analyzing everything we have noted some possible causes. They are related on how we have designed our Meteor packages, but the solutions already take up some issues, for that we'd want to know how Meteor usually is approached on large projects.

NPM dependencies are loaded multiple times

The way meteor packages handle NPM dependencies, is that all of them, whether they are included directly at package.json or using Npm.depends, are bundled on the package itself, which produces increasing our apps in both size and loading. As you can check in the next graphic there libs are being included one time per package that uses, .

Clearly, loading react, or other common libs like lodash multiple times is wasteful.

Solutions

As Meteor doesn't have a way to handle peer dependencies between the apps and packages as NPM does, we've been researched that the way to tackle that is to move all NPM dependencies from the packages to the app level. However that produces to move all our package tests to a separate application and running them there, which make our delivering process slower and more complex. And it also requires a way to handle the requirements on the NPM versions that each package needs, check-npm-version has not resulted to be worth for us, we've faced so many issues like #1, and others run failures due to dependencies needed on server and between packages that depend on other packages. That approach feels like quite buggy and the project seems not to provide the solutions.

• Do you happen to know other ideas or ways to handle peer NPM dependencies on your projects?

Other solution could be to create our packages as plain NPM modules and have a custom way to build and test (through webpack or other env setup tool), and then we could benefit from NPM advantages to handle peer dependencies and so on. The problem here is that some of our modules need to include meteor package dependencies. So,

• How can we import meteor or other meteors atmosphere package functionality in our NPM modules?

We've heard that this is on your roadmap, So,

• How is this transition going to work?

Dependencies eagerly loaded

All our package dependencies are importing all their contents automatically though the api.mainModule definition. This causes everything to be imported even if that content is never directly used.

  api.mainModule('api.client.js', 'client');
  api.mainModule('api.server.js', 'server');

Then api.client.js or api.server.js exports all the logic for each end. We probably want to have every collection definition on the correspond package but we don't want it to be loaded in all our apps since it won't be used is some of them.

Solutions

Researching over the Internet, a possible solution could be to stop using an api.mainModule to expose our logic modules. So a possible alternative could be to export our package modules lazily using api.addFiles(..., ..., { isAsset: true }). So as example,

file structure

collections
|_ some
    |_ index.js
    |_ collection.js

on the package.js:

api.addFiles(['./some/index.js', './some/collection.js'], ['client', 'server'], { isAsset: true });

at app/other package context:

import { someCollection } from 'meteor/collections/some';

That is likely to not have any impact on the package size, but it will probably have on some apps that doesn't need such extra logic. I guess another thing is to rely on some logic to be defined on app level, but as long as they are needed in other apps they will be moved to the package. So having a way to import the functionality on demand could be good.

Questions here

• Would that way to expose and import by demand work?
• Is that a common pattern used in Meteor packages for improving the performance? Or what is?

Other causes

We are making a lot of work trying to figure out other causes, like some possible heavy dependencies that could be imported different or alternatives, check that every server-only logic is not delivered to the client, possible circular dependencies handled badly on javascript, etc.

But as stated at the beginning, we need expert advice. There may exist other causes that you have faced with other Meteor users that could drive us to check them and could be causing major impact. We really need the feedback you could provide us about meteor designing and best practices on apps and packages.