Base on swipeRefreshLayoutRecyclerView.java of Mandhor . I fixed bug SwipeRefreshLayout catches scroll up too early - not on top of the list when use with recyclerview. But in some cases It disable SwipeRefreshLayout I can't scroll up . I fixed it with code below

gistfile1.txt

//we have to keep current scroll value somewhere in our fragment
private int overallYScroll = 0;

recyclerView.setOnScrollListener(new RecyclerView.OnScrollListener() {
    @Override
    public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
        super.onScrolled(recyclerView, dx, dy);

        overallYScroll = overallYScroll + dy;

        if (overallYScroll <= 0) {
          //enable swipeRefreshLayout
            swipeRefreshLayout.setEnabled(true);
        } else {
            //disable
            swipeRefreshLayout.setEnabled(false);
        }

    }
});

....
///Combine with code