API Design: Best Practices for Filtering, Searching, Sorting, Paging

Restful APIs usually return either a single entity or multiple entities. Returning a single entity is often not an issue but returning multiple entities needs careful API design. When designing such APIs it is important to consider features like filtering, searching, sorting, and paging.

Generally, an application can have tens of thousands of entities.

Filtering allows users to narrow down search results by defining specific criteria, while sorting allows results to be ordered in a particular manner. Pagination enables the API to return a subset of data, reducing the amount of data transferred and improving performance.

Restful APIs often return a list of entities or objects. Returning everything is sometimes unnecessary and leads to performance issues. Thus introducing paging in API results to get the results quickly, and the application can query any particular page by giving the page numbers.

Filtering

Filtering is a useful feature that allows users to refine their search results with specific criteria. By selecting certain filters, users can narrow down their search and get more precise results tailored to their needs. Ex. Price > 50

Conditions should match exactly with the given criteria.
Operators - Using short codes instead of special characters Ex. gte instead of ">". If we use something like price>=50 in the API path, it isn't easy to parse because browsers will always use
- Equal to (=) --> No shortcodes (or) eq
- Not equal to (!=) --> neq
- Less than (<) --> lt
- Less than or equal to (<\=) --> lte
- Greater than (>) --> gt
- Greater than or equal to (>=) --> gte

You can think about other possible scenarios and adjust conditions accordingly. Ex. price=100&category=Electronics

GET /products?price=50 //Equal 
GET /products?price=neq:50 //Not Equal
GET /products?price=lt:50 //Less than
GET /products?price=lte:50 //Less than or equal
GET /products?price=gt:50 //Greater than
GET /products?price=gte:50 //Greater than or equal
GET /products?price=gte:10&price=lte:50
GET /items?price=20-60 //Range - from 20 to 60
//a
GET /products?category=Electronics
GET /products?price=eq:50&category=Electronics

Searching

Apart from filtering the individual columns, sometimes you want to search the entities with multiple columns like full-text search. You may use some popular third-party APIs or your logic to build the search.

GET /products?q=shirt

In the above example, the search term will be used to search the products with either single or multiple columns like product name, description, etc. You can define the columns you would like to search in your API logic.

Sorting (ascending/descending)

An API user may want to sort (arrange) by either one or multiple columns while getting the data from the endpoint. Sorting can be done by ascending or descending.

You can prefix sorting columns with + for ascending and - for descending.

GET /products?sort=+name,-price

The above example returns the products sorted by name (ascending) and price (descending).

Paging

Paging divides large results into smaller pages and returns only the particular page. Clients should send page number and number of results per page to the endpoint.

GET /products?page=2&pagesize=10

In the above example, the items per page are defined as 10, and page 2 will be returned. Internally maximum page size should be defined to avoid large results.

Paging is important in API design to limit the number of results to the client, hence avoiding network traffic and reducing bandwidth.

Example results with paging

Example: 1 (Returns entities with metadata)

{
  "data": [
    {
      "id": 21,
      "name": "Product 21",
      "description": "Description of product 21",
      "price": 10.99
    },
    {
      "id": 22,
      "name": "Product 22",
      "description": "Description of product 22",
      "price": 15.99
    },
    ...
  ],
  "metadata": {
    "total_count": 100,
    "limit": 10,
    "offset": 20
  }
}

Example: 2 (PaginatedResult abstract class)

{
  "has_next": true,    // do we have a another page after?
  "has_prev": true,   // do we have a page before?
  "total": 100,        // how many dogs we have in total in the db
  "page": 3,           // current page
  "per_page": 20,      // how many items per page the user requested
  "pages": 5,          // total number of pages
  "results": [         // the results (20 in this case)
             ... 
             { "id": 64, "name": "Rocky", "ownerName": "Lenny", "favoriteToy": "Ball"},
             ...
           ]
}

Use PaginatedResult abstract class with results as generics support. List-based endpoints can use this type to return the data. Ex. PaginatedResult<Products>.

Using a common PaginatedResult<T> type in all necessary places will avoid code duplicates.
If we want to change anything in the above result type (ex. renaming "pages" to "total_pages"), it is possible to do it in only one place.

References:

REST API Design: Filtering, Sorting, and Pagination (atatus.com)

REST API Design: Filtering, Sorting, and Pagination | Moesif Blog

RESTful API Designing guidelines — The best practices | HackerNoon

Best Practices for Designing a Pragmatic RESTful API | Vinay Sahni

https://blog.nitzano.com/generic-swagger-pagination

Rajasekar Blog