Pagination

Description

Pagination is used to split data across several pages. It is a well known pattern to optimise performance on large dataset.

DSG works with pagination the same way DRF Pagination does. It uses the Django Pagination system and its Paginator class.

That means DSG supports the default pagination class from DRF like PageNumberPagination, LimitOffsetPagination and CursorPagination. It also supports creating your own Pagination for advanced pagination functionalities.

Pagination can be enabled globally for the all the project or for only some services.

To use pagination you may use metadata or request depending on the PAGINATION_BEHAVIOR setting, See example for how to use it.

Using it globally on all List and Stream actions

To enable pagination globally you need to set the DEFAULT_PAGINATION_CLASS settings to the pagination class you want to use.

# settings.py
GRPC_FRAMEWORK = {
    ...
    "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination"
    ...
}

Using it by service on List action

To enable pagination only for one or some service you can override the pagination_class attribute.

# server
# quickstart/services.py
from django_socio_grpc import generics
from quickstart.models import Post
from quickstart.serializer import PostProtoSerializer
from rest_framework.pagination import PageNumberPagination


# This service will have all the CRUD actions
class PostService(generics.AsyncModelService):
    queryset = Post.objects.all()
    serializer_class = PostProtoSerializer
    pagination_class = PageNumberPagination

Using it on specific action when using filter in message

If your PAGINATION_BEHAVIOR setting is set to REQUEST_STRUCT_STRICT or METADATA_AND_REQUEST_STRUCT and you want to use filtering for your custom action by message and not metadata (See Using It section) you need to use the PaginationGenerationPlugin as demonstrated below (See Generation Plugin documentation):

# server
# quickstart/services.py
from django_socio_grpc import generics
from quickstart.models import Post
from quickstart.serializer import PostProtoSerializer
from rest_framework.pagination import PageNumberPagination
from django_socio_grpc.decorators import grpc_action
from django_socio_grpc.protobuf.generation_plugin import ListGenerationPlugin, PaginationGenerationPlugin


# This service will have all the CRUD actions
class PostService(generics.GenericService):
    queryset = Post.objects.all()
    serializer_class = PostProtoSerializer
    pagination_class = PageNumberPagination

    @grpc_action(
        request=[],
        response=PostProtoSerializer,
        use_generation_plugins=[ListGenerationPlugin(request=True), PaginationGenerationPlugin()],
    )
    async def CustomListWithPagination(self, request, context):
        queryset = self.filter_queryset(self.get_queryset())
        page = self.paginate_queryset(queryset)
        if page is not None:
            serializer = self.get_serializer(page, many=True)
            if hasattr(serializer.message, "count"):
                serializer.message.count = self.paginator.page.paginator.count
            return serializer.message
        else:
            serializer = self.get_serializer(queryset, many=True)
            return serializer.message

Using it

You can use metadata or _pagination request field to make the filters work out of the box.

For more example you can see the client in DSG example repo

# client
import asyncio
import grpc
import json

async def main():
    ##########################################################################################################
    # Working if PAGINATION_BEHAVIOR settings is equal to "METADATA_STRICT" or "METADATA_AND_REQUEST_STRUCT" #
    ##########################################################################################################
    async with grpc.aio.insecure_channel("localhost:50051") as channel:
        quickstart_client = quickstart_pb2_grpc.PostControllerStub(channel)

        request = quickstart_pb2.PostListRequest()

        # Getting the 11 to 20 elements following backend ordering
        pagination_as_dict = {"page": 2, "page_size": 10}

        metadata = (("pagination", (json.dumps(pagination_as_dict))),)

        response = await quickstart_client.List(request, metadata=metadata)


    ################################################################################################################
    # Working if PAGINATION_BEHAVIOR settings is equal to "REQUEST_STRUCT_STRICT" or "METADATA_AND_REQUEST_STRUCT" #
    ################################################################################################################
    async with grpc.aio.insecure_channel("localhost:50051") as channel:
        quickstart_client = quickstart_pb2_grpc.PostControllerStub(channel)

        # Getting the 11 to 20 elements following backend ordering
        pagination_as_dict = {"page": 2, "page_size": 6}
        pagination_as_struct = struct_pb2.Struct()
        pagination_as_struct.update(pagination_as_dict)

        # _pagination field is only generated if you set PAGINATION_BEHAVIOR to the correct options. Think to regenerate proto after changing it.
        request = quickstart_pb2.PostListRequest(_pagination=pagination_as_struct)

        response = await quickstart_client.List(request)

if __name__ == "__main__":
    asyncio.run(main())

Using page size

As specified in the DRF Pagination documentation, you need to override the PageNumberPagination class to use page size.

# server
# quickstart/pagination.py
from rest_framework.pagination import PageNumberPagination

class CustomPageNumberPagination(PageNumberPagination):
    page_size = 10
    page_size_query_param = 'page_size'
    max_page_size = 1000

# quickstart/services.py
...
from quickstart.pagination import CustomPageNumberPagination
...

class PostService(generics.GenericService):
    ...
    pagination_class = CustomPageNumberPagination
    ...

Web Example

For web usage of the client see How to web: Using JS client

import { Struct } from "@bufbuild/protobuf";
// See web usage to understand how to use the client.
const postClient = createPromiseClient(PostController, transport);

const paginationStruct = Struct.fromJson({page: 2});
const res = await postClient.list({ Pagination: paginationStruct }); // _pagination is transformed to Pagination in buf build used by connect
console.log(res)

Warning

The following example is the deprecated way of using pagination. Please use the example above. Note that the examples work depending on the PAGINATION_BEHAVIOR setting settings.

// See web usage to understand how to use the client.
const postClient = createPromiseClient(PostController, transport);

headers = {pagination: JSON.stringify({page: 2})}

const res = await postClient.list({}, {headers})
console.log(res)