N+1 Queries, Batch Loading and Active Model Serializers

Gemfile

# ...
# https://github.com/exAspArk/batch-loader
gem 'batch-loader'

init-code.rb

class PostsController < ApplicationController  
  def index    
    posts = Post.all     
     
    render json: posts  
  end
end

class Post
  belongs_to :author, class_name: 'User'
end

class PostSerializer < ActiveModel::Serializer  
  attributes :id, :title, :details
  belongs_to :author 
end

post.rb

class Post < ApplicationRecord
  # ...
  
  def get_author_lazily
    BatchLoader.for(self).batch do |posts, batch_loader|
      User.where(:_id.in => posts.pluck(:author_id)).each do |user|
        # Modify the user through a given block, say, for serialization.
        modified_user = block_given? ? yield(user) : user
        batch_loader.call(posts.detect { |p| p.author_id == user._id.to_s }, modified_user)
      end
    end
  end
  # ...
end

posts_controller.rb

class PostsController < ApplicationController
  def index
    # Can't do Post.includes(:author) beacuse the author (User object)
    # is stored in an entirely different database: a MongoDB instance.
    posts = Post.all

    render json: posts
  end
end

PostSerializer.rb

class PostSerializer < ActiveModel::Serializer
  attributes :id, :title, :details, :author

  def author
    object.get_author_lazily do |author|
      # Serialize the author after it has been loaded.
      ActiveModelSerializers::SerializableResource.new(author).as_json[:user]
    end
  end
  # ...
end

user.rb

class User
  include Mongoid::Document
  include Mongoid::Timestamps
  # ...
end

user_serializer.rb

class UserSerializer < ActiveModel::Serializer
  # ....
end

Nice implementation! Also, good post around the subject. The application of this with AMS is really useful, since most graphql new approaches are heavy influenced by the data loader technic.

I was playing a little with your code and it works, except for a drawback with AMS. This batch behaviour to postpone the attribute will not take advantage of include directive for choosing the JSON depth/attributes and response performance.

For exemple: If the post User has a list of User as friends and I don't want to load them, I would specify:

render json: posts, include: :user

And if I want it: render json: posts, include: [user: :friends]

But with batch loader its not possible maybe because ActiveModelSerializers::SerializableResource.new(author).as_json[:user] will be constructed afterwards to apply the include? I'm thinking here what could be a solution but can't figured out if is a limitation of AMS or the BathLoader.

@luccasmaso yes, include works only for ORM-defined database relations, not custom attributes, which is what user has become in our case.
We can pass custom parameters and access them with @instance_options to achieve our goal:

# PostSerializer.rb
attribute :user, if: -> { @instance_options.key?(:with_user) && @instance_options[:with_user] }
# ...

render json: posts, with_user: true

You can also stick with include, use @instance_options[:include] and check if user was specified. But I'd probably not do this because it sort of goes against what the include option is supposed to be for. Also, in a way our point was to avoid include since it forced the n+1 queries to run. Having said that, there's nothing essentially wrong with using @instance_options[:include].

As far as nested associations are concerned, you can pass them to the explicit call to AMS;

ActiveModelSerializers::SerializableResource.new(user, include: [some: :nested_stuff]).as_json[:user]

Sorry for replying late. Don't know why I didn't get an email!

Hey, I digged in this topic a bit and created a plugin for ActiveModelSerializers - https://github.com/Bajena/ams_lazy_relationships

It eliminates the problem that @lucasmaso mentioned in his comment :)