`WP_Query` subclass that takes a non-paginated query and split into different paginated queries offering a transparent interface to "standard loop" usage.

AutoPaginatedQuery.php

<?php
namespace GM;

/**
 * `WP_Query` subclass that takes a non-paginated query and split into different
 * paginated queries offering a transparent interface to "standard loop" usage.
 *
 * The class is not 100% transparent:
 * - the var `$posts` and the method `get_posts()`, that are not used directly
 *   in standard loop usage, here don't return array of all posts, but only posts
 *   for the current query. Otherwise we would not be able to reduce memory usage
 * - in `\WP_Query` the var `$post_count` is equal to `$post_found` for non-paginated queries,
 *   but in this class `$post_count` will contain the post number of current query
 * - all the `posts_*` filters related to SQL query are skipped to avoid they
 *   act on pagination, breaking class workflow
 *
 * Besides of those points all other variables and methods should be transparent
 * to user.
 *
 * @author  Giuseppe Mazzapica <[email protected]>
 * @license http://opensource.org/licenses/MIT MIT
 */
class AutoPaginatedQuery extends \WP_Query
{
    /**
     * Set in constructor, it is used as `posts_per_page` argument for each query.
     * E.g. for a total of 1000 posts, if this is set to 100, the class will perform 10 queries
     *
     * @var int
     */
    protected $postsPerQuery = 0;

    /**
     * A flag that tells if the class is performing auto-pagination or not.
     * When false all class methods fallback to `WP_Query` for 100% compatibility.
     * It's false for already paginated queries, singular queries and when `$postsPerQuery` <= 0.
     *
     * @var bool
     */
    protected $doSplit = false;

    /**
     * Because some query vars originally provided are changed to enforce pagination, the original
     * values are backup in this property, so they can be retrieved using `get()` so improving
     * compatibility with `WP_Query`. However, direct access to `$query` or `$query_vars` will
     * retrieve the paginated arguments.
     *
     * @var array|null
     */
    protected $varsBackup = null;

    /**
     * Because of the original non-paginated query is split in more paginated queries, this property
     * holds the 1-based index for the current query being performed.
     *
     * @var int
     */
    protected $queryIndex = 1;

    /**
     * Because of we enforce `$current_post` variable to be consistent as post index during the loop,
     * we need a post index related to the current query being looped.
     *
     * @var int
     */
    protected $current_query_post = - 1;

    /**
     * @param array|string $query         Query arguments, 100% compatible with `WP_Query`
     * @param int          $postsPerQuery Used as `posts_per_page` argument for each query
     */
    public function __construct($query, $postsPerQuery = 0)
    {
        $this->postsPerQuery = is_numeric($postsPerQuery) ? (int) $postsPerQuery : 0;
        parent::__construct($query);
    }

    /**
     * @inheritdoc
     */
    public function init()
    {
        $this->current_query_post = - 1;
        parent::init();
    }

    /**
     * For maximum compatibility, vars that are changed to enforce pagination
     * are pulled from original query vars.
     * For all the other vars, we use `WP_Query` method.
     *
     * @inheritdoc
     */
    public function get($query_var, $default = '')
    {
        if (
            $this->doSplit 
            && $this->varsBackup
            && array_key_exists($query_var, $this->varsBackup)
        ) {
            return $this->varsBackup[$query_var];
        }

        return parent::get($query_var, $default);
    }

    /**
     * Because of we split the non-paginated query into different queries,
     * the method return true either in the case that current query have posts
     * or in the case that there are query to be performed
     *
     * @return true
     */
    public function have_posts()
    {
        // split is not done, just use `WP_Query` method
        if (! $this->doSplit) {
            return parent::have_posts();
        }

        if ($this->current_post + 1 < $this->found_posts) {
            return true;
        }

        // last post of last query query: loop ended
        if ($this->current_post + 1 === $this->found_posts) {
            do_action_ref_array('loop_end', array( &$this ));
            $this->rewind_posts();
        }

        $this->in_the_loop = false;

        return false;
    }

    /**
     * The method has to increment both the index of current query and the overall post index.
     * When last post of current query is reached, we need to perform next query to get next post.
     *
     * @return \WP_Post
     */
    public function next_post()
    {
        if (! $this->doSplit) {
            return parent::next_post();
        }

        $this->current_post ++;
        $this->current_query_post ++;

        if ($this->current_query_post === $this->post_count && $this->current_post < $this->found_posts) {
            $this->queryIndex ++;
            $this->posts = [];
            $this->get_posts();
            $this->current_query_post = 0;
        }

        $this->post = $this->posts[ $this->current_query_post ];

        return $this->post;
    }

    /**
     * Reset overall post index, current query post index, and query index.
     * Finally perform `get_posts` because we have to ensure that `$post` variable
     * points to the first post of the first query
     */
    public function rewind_posts()
    {
        if (! $this->doSplit) {
            return parent::rewind_posts();
        }

        $this->current_post       = - 1;
        $this->current_query_post = - 1;
        $this->queryIndex         = 1;
        $this->posts              = [ ];
        $this->get_posts();
        if ($this->post_count) {
            $this->post = $this->posts[ 0 ];
        }
    }

    /**
     * The actual posts retrieval is done with `WP_Query` method, however, before to perform the query,
     * we enforce pagination arguments.
     * SQL filters are turned off to avoid they break pagination.
     * First time method is called, we check that the query need to be auto-paginated, for instance
     * singular queries and query that are already paginated are not auto-paginated.
     * When there's no auto-pagination, this and other methods in this class just fallback to
     * `WP_Query` methods.
     *
     * @return \WP_Post[]
     */
    public function get_posts()
    {
        $first = false;
        
        $this->parse_query();

        // First time this is called, let's backup some original query vars
        // to be used by `get()`
        if (is_null($this->varsBackup)) {
            $first            = true;
            $this->varsBackup = [
                'nopaging'       => $this->query_vars[ 'nopaging' ],
                'posts_per_page' => $this->query_vars[ 'posts_per_page' ],
                'is_paged'       => false,
                'paged'          => null,
            ];
        }

        if ($first) {
            // First time method is called, we check if to split query
            // and do it only for non-singular, non-paginated queries
            // and only if a pagination threshold have been set
            $this->doSplit =
                $this->postsPerQuery > 0
                && ! $this->is_singular
                && (
                    (int) $this->query_vars[ 'posts_per_page' ] === - 1
                    || $this->query_vars[ 'nopaging' ]
                );
        }

        // If 'posts_per_page' have a value, so we are not going to auto-paginate the query because
        // already paginated, we use `$postsPerQuery` as pagination, if it exists.
        if (
            $first
            && ! $this->doSplit
            && $this->query_vars[ 'posts_per_page' ] > 0
            && $this->postsPerQuery > 0
            && $this->postsPerQuery !== (int)$this->query_vars['posts_per_page' ]
        ) {
            $this->query_vars[ 'posts_per_page' ] = $this->postsPerQuery;
        }

        // No auto-pagination: just use `WP_Query` method.
        // It will call `parse_query()` again which will not hurt functionality, but performance a bit
        // so avoid to use this class for query that are already paginated...
        if (! $this->doSplit) {
            return parent::get_posts();
        }

        // Let's enforce pagination
        $this->query_vars[ 'paged' ]          = $this->queryIndex;
        $this->query_vars[ 'nopaging' ]       = false;
        $this->query_vars[ 'posts_per_page' ] = $this->postsPerQuery;

        // We need to know total rows count only first time the method is called
        $this->query_vars[ 'no_found_rows' ] = ! $first;

        // We need to be opinionated here.
        // There are a lot of SQL filters that may affect pagination, breaking
        // our workflow and there's no way to prevent it happen if filters
        // are parsed. The only viable way is to remove filters.
        $this->query_vars[ 'suppress_filters' ] = true;

        // Actually query posts
        parent::get_posts();

        // Because, you know, WordPress
        $first and $this->found_posts = (int) $this->found_posts;

        return $this->posts;
    }
}

Usage

// trasparently creates more `WP_Query` where each one has `posts_per_page` set to 100
$query = new GM\AutoPaginatedQuery(['posts_per_page' => -1], 100);

while($query->have_posts()) {
    $query->the_post(); // <-- REQUIRED
    // loop code here
}

wp_reset_postdata();

Uh… all this fine logic bound to WP_Query. Now do it again for WP_User_Query. And comment query, and… (Just kidding, please don't)

Minor typo: $query->the_post(); without the s or did you mean $query->get_posts()?

@franz-josef-kaiser right, fixed, thanks

@dnaber-de