In modern API-driven applications, efficiently retrieving data is foundational for creating responsive and user-friendly experiences. As applications scale and user requirements become more nuanced, dynamic filtering becomes a vital feature for APIs. Dynamic filtering allows clients to specify precisely what data they need, reducing the need for hardcoded backend logic and enabling more scalable and maintainable solutions.
Using a standardized filtering pattern, such as [field:operator]=value or [ParentObject][field:operator]=value, provides a clear and consistent approach for querying data. This method not only simplifies the development process but also enhances the flexibility and power of your API.
In this article, we'll explore how to implement dynamic filtering in Laravel, including handling related data filters and extending the system with custom filters. By the end of this guide, you'll have a robust filtering mechanism that can handle complex queries and scale with your application.
Laravel’s Eloquent ORM and query builder provide a solid foundation for implementing dynamic filtering. Here's a basic outline of how to build a filtering system that includes handling related data filters.
To manage dynamic filters, including those for related data, we need to build a system that can interpret and apply these filters effectively. Here’s how we can approach this:
The filtering system will receive parameters in the URL query string. For example, a request might look like:
/api/products?filter[price:gt]=100&filter[Category][name:contains]=ElectronicsIn this example, filter[price:gt]=100 filters products where the price is greater than 100, and filter[category][name:contains]=Electronics filters products based on the name of the related category model.
The filter parameters need to be converted into query builder methods. For the basic fields, such as price:gt=100, this is straightforward. For related data, such as filter[category][name:contains]=Electronics, we need to apply a more complex query involving relationships.
Handling related data filters involves using Laravel’s Eloquent relationship methods. For instance, if you want to filter products based on a related category model, you need to apply the filter within a whereHas clause.
Now, let's create a class named ExtendQueryBuilder that will handle the dynamic filtering logic.
<?php
namespace App\Utilities;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Database\Eloquent\Model;
use ReflectionClass;
use ReflectionMethod;
class ExtendQueryBuilder
{
/**
* Create a query builder instance for the given model and apply filters, sorting, and grouping.
*
* @param string $modelName The model name (e.g., "User")
* @param array $params The parameters for the query (select, filter, sort, group_by)
* @return Builder The resulting query builder
*/
public static function for(string $modelName, array $params): Builder
{
// Initialize the model
$model = self::initializeModel($modelName);
// Start building the query
$query = $model->newQuery();
// Apply SELECT clause
$query->select($params['select'] ?? $model->getTable() . '.*');
// Apply filters
if (isset($params['filter'])) {
$query = self::applyFilters($model, $query, $params['filter']);
}
// Apply sorting
if (isset($params['sort'])) {
$query = self::applySort($query, $params['sort']);
}
// Apply grouping
$disableGrouping = isset($filters['group_by']) && $filters['group_by'] === false;
if (!$disableGrouping)
$query->groupBy($filters['group_by'] ?? $model->getTable() . '.id');
return $query;
}
/**
* Initialize a model instance based on the provided class name.
*
* @param string $modelName The model name (e.g., "User")
* @return Model The initialized model instance
*/
private static function initializeModel(string $modelName): Model
{
$modelClass = "App\\Models\\$modelName";
if (!class_exists($modelClass) || !is_subclass_of($modelClass, Model::class)) {
throw new \InvalidArgumentException("Model class $modelClass does not exist or is not a valid model.");
}
return new $modelClass();
}
/**
* Apply filters to the query.
*
* @param Model $model The model instance
* @param Builder $query The query builder
* @param array $filters The filters to apply
* @return Builder The query builder with applied filters
*/
private static function applyFilters(Model $model, Builder $query, array $filters): Builder
{
// Resolve custom filter class if exists
$filterClass = self::resolveModelFilterClass($model);
if ($filterClass) {
$filterObject = (new $filterClass($query));
$filters = $filterObject->prepareFiltersArray($filters);
$query = $filterObject->apply($filters);
}
// Apply joins based on relationships in filters
$relations = self::getJoinsFromFilters($model, $filters);
if (count($relations) > 0) {
$query = self::addJoinsToBuilder($query, $relations);
}
// Loop through each filter and apply it
foreach ($filters as $field => $value) {
if (is_array($value)) {
// Handle nested filters (related models)
$nestedModelClass = "App\\Models\\$field";
$query = self::applyFilters(new $nestedModelClass, $query, $value);
} else {
// Apply filter for the current model
$query = self::applyQueryFor($model, $query, $field, $value);
}
}
return $query;
}
/**
* Apply a single filter to the query based on the field and value.
*
* @param Model $model The model instance
* @param Builder $query The query builder
* @param string $field The field to filter
* @param mixed $value The value to filter by
* @return Builder The query builder with the applied filter
*/
private static function applyQueryFor(Model $model, Builder $query, string $field, $value): Builder
{
// Determine the operator (default to 'eq' if none is provided)
[$field, $operator] = strpos($field, ':') !== false ? explode(':', $field) : [$field, 'eq'];
// Ensure the field is fillable before applying the filter
if (!in_array($field, $model->getFillable())) {
return $query;
}
return self::applyQuery($model, $query, $field, $operator, $value);
}
/**
* Apply a query filter based on the operator.
*
* @param Model $model The model instance
* @param Builder $query The query builder
* @param string $field The field to filter
* @param string $operator The operator (e.g., 'gt', 'lt', 'like')
* @param mixed $value The value to filter by
* @return Builder The query builder with the applied filter
*/
private static function applyQuery($modelClass,Builder $query, string $field, string $operator, $value): Builder
{
$tableName = ($modelClass::getInstance())->getTable();
switch ($operator) {
case 'gt':
return $query->where($tableName.'.'.$field, '>', $value);
case 'lt':
return $query->where($tableName.'.'.$field, '<', $value);
case 'gte':
return $query->where($tableName.'.'.$field, '>=', $value);
case 'lte':
return $query->where($tableName.'.'.$field, '<=', $value);
case 'like':
return $query->where($tableName.'.'.$field, 'LIKE', "%$value%");
case 'not':
return $query->where(function ($query) use ($tableName,$field,$value){
$query->where($tableName . '.' . $field, '<>', $value)
->orWhereNull($tableName . '.' . $field) ;
});
case 'isnotnull':
return $query->whereNotNull($tableName.'.'.$field);
case 'isnull':
return $query->whereNull($tableName.'.'.$field);
case 'notin':
return $query->whereNotIn($tableName.'.'.$field, explode(',',$value));
case 'in':
return $query->whereIn($tableName.'.'.$field, explode(',',$value));
default:
return (count(explode(',',$value)) >1) ?
$query->whereIn($tableName.'.'.$field, explode(',',$value)) :
$query->where($tableName . '.' . $field, '=', $value);
}
}
/**
* Apply sorting to the query.
*
* @param Builder $query The query builder
* @param string $sort The sort parameter (e.g., '-name' for descending)
* @return Builder The query builder with applied sorting
*/
private static function applySort(Builder $query, string $sort): Builder
{
$direction = strpos($sort, '-') === 0 ? 'desc' : 'asc';
$field = ltrim($sort, '-');
return $query->orderBy($field, $direction);
}
/**
* Resolve the custom filter class for the model, if exists.
*
* @param Model $model The model instance
* @return string|null The filter class or null if not found
*/
private static function resolveModelFilterClass(Model $model): ?string
{
$filterClass = "App\\Filters\\" . class_basename($model) . "Filter";
return class_exists($filterClass) ? $filterClass : null;
}
/**
* Get relationships from filters and prepare joins.
*
* @param Model $model The model instance
* @param array $filters The filters array
* @return array The relations for joining
*/
private static function getJoinsFromFilters(Model $model, array $filters): array
{
$joins = [];
foreach ($filters as $relatedModel => $filter) {
if (is_array($filter) && class_exists("App\\Models\\$relatedModel")) {
$relationType = self::getRelationType($model, $relatedModel);
if ($relationType) {
$joins[] = [
'type' => $relationType,
'current_model' => class_basename($model),
'related_model' => $relatedModel,
];
}
}
}
return $joins;
}
/**
* Add joins to the query builder.
*
* @param Builder $query The query builder
* @param array $joins The array of joins to apply
* @return Builder The query builder with applied joins
*/
private static function addJoinsToBuilder(Builder $query, array $joins): Builder
{
foreach ($joins as $join) {
$currentTable = (new ("App\\Models\\" . $join['current_model']))->getTable();
$relatedTable = (new ("App\\Models\\" . $join['related_model']))->getTable();
if ($join['type'] === 'BelongsTo') {
$query = self::addBelongsToJoin($query, $currentTable, $relatedTable, $join['related_model']);
} elseif ($join['type'] === 'HasMany') {
$query = self::addHasManyJoin($query, $currentTable, $relatedTable, $join['current_model']);
}
}
return $query;
}
/**
* Add a BelongsTo relationship join.
*
* @param Builder $query The query builder
* @param string $currentTable The current table name
* @param string $relatedTable The related table name
* @param string $relatedModel The related model name
* @return Builder The query builder with applied join
*/
private static function addBelongsToJoin(Builder $query, string $currentTable, string $relatedTable, string $relatedModel): Builder
{
$foreignKey = self::fromCamelToDash($relatedModel) . '_id';
if (!self::hasJoinedBefore($query, "$relatedTable as $relatedTable")) {
$query->leftJoin("$relatedTable as $relatedTable", "$currentTable.$foreignKey", '=', "$relatedTable.id");
}
return $query;
}
/**
* Add a HasMany relationship join.
*
* @param Builder $query The query builder
* @param string $currentTable The current table name
* @param string $relatedTable The related table name
* @param string $currentModel The current model name
* @return Builder The query builder with applied join
*/
private static function addHasManyJoin(Builder $query, string $currentTable, string $relatedTable, string $currentModel): Builder
{
$foreignKey = self::fromCamelToDash($currentModel) . '_id';
if (!self::hasJoinedBefore($query, "$relatedTable as $relatedTable")) {
$query->leftJoin("$relatedTable as $relatedTable", "$relatedTable.$foreignKey", '=', "$currentTable.id");
}
return $query;
}
/**
* Determine the relation type (BelongsTo, HasMany, etc.) between two models.
*
* @param Model $model The model instance
* @param string $relatedModel The related model name
* @return string|null The relation type or null if not found
*/
private static function getRelationType(Model $model, string $relatedModel): ?string
{
$relatedModelClass = "App\\Models\\$relatedModel";
$methodName = lcfirst($relatedModel);
// Check for relation method or relation by table name
foreach ((new ReflectionClass($model))->getMethods(ReflectionMethod::IS_PUBLIC) as $method) {
if ($method->name === $methodName || $method->name === self::dashesToCamelCase((new $relatedModelClass())->getTable())) {
$relation = $method->invoke($model);
return (new ReflectionClass($relation))->getShortName();
}
}
return null;
}
/**
* Convert a camelCase string to a snake_case string.
*
* @param string $input The input string in camelCase
* @return string The converted string in snake_case
*/
private static function fromCamelToDash(string $input): string
{
return strtolower(preg_replace('/([a-z])([A-Z])/', '$1_$2', $input));
}
/**
* Convert a snake_case string to a camelCase string.
*
* @param string $string The input string in snake_case
* @param bool $capitalizeFirstCharacter Should the first character be capitalized?
* @return string The converted string in camelCase
*/
private static function dashesToCamelCase(string $string, bool $capitalizeFirstCharacter = false): string
{
$str = str_replace(' ', '', ucwords(str_replace('_', ' ', $string)));
if (!$capitalizeFirstCharacter) {
$str = lcfirst($str);
}
return $str;
}
/**
* Check if the table has already been joined in the query.
*
* @param Builder $query The query builder
* @param string $table The table name to check
* @return bool Whether the table has already been joined
*/
public static function hasJoinedBefore(Builder $query, string $table): bool
{
$joins = $query->getQuery()->joins;
if (!is_array($joins) && !is_object($joins)) {
return false;
}
foreach ($joins as $join) {
if (trim($join->table) === trim($table)) {
return true;
}
}
return false;
}
}
ExtendQueryBuilder Class with Default Service and Repository PatternNow that we have the ExtendQueryBuilder class, we can integrate it into our application using a default service and repository pattern. This approach provides a cleaner separation of concerns, making the code more maintainable and testable.
First, let's define the repository class that will interact with the ExtendQueryBuilder class to fetch data.
<?php
namespace App\Repositories;
use App\Utilities\ExtendQueryBuilder;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Support\Collection;
use Illuminate\Pagination\LengthAwarePaginator;
abstract class DefaultRepository implements DefaultRepositoryInterface
{
protected $model;
/**
* DefaultRepository constructor.
*
* @param Model $model
*/
public function __construct(Model $model)
{
$this->model = $model;
}
/**
* Find a list of items by applying filters with pagination.
*
* @param array $filters
* @return LengthAwarePaginator
*/
public function index(array $filters = []): LengthAwarePaginator
{
// Extract perPage from filters if available, default to 15
$perPage = $filters['per_page'] ?? 15;
// Remove per_page from filters to avoid affecting the query
unset($filters['per_page']);
// Build the query using ExtendQueryBuilder
$query = ExtendQueryBuilder::for($this->model::class, $filters);
// Apply pagination
return $query->paginate($perPage);
}
/**
* Find a list of items by applying filters.
*
* @param array $filters
* @return Collection
*/
public function find(array $filters = []): Collection
{
// Build the query using ExtendQueryBuilder
$query = ExtendQueryBuilder::for($this->model::class, $filters);
// Execute the query and return results
return $query->get();
}
/**
* Find the first item by applying filters.
*
* @param array $filters
* @return Model|null
*/
public function findFirst(array $filters = []): ?Model
{
// Build the query using ExtendQueryBuilder
$query = ExtendQueryBuilder::for($this->model::class, $filters);
// Execute the query and return the first result
return $query->first();
}
/**
* Count items by applying filters.
*
* @param array $filters
* @return int
*/
public function count(array $filters = []): int
{
// Build the query using ExtendQueryBuilder
$query = ExtendQueryBuilder::for($this->model::class, $filters);
// Execute the query and return the count
return $query->count();
}
/**
* Sum a specific column by applying filters.
*
* @param string $column
* @param array $filters
* @return float
*/
public function sum(string $column, array $filters = []): float
{
// Build the query using ExtendQueryBuilder
$query = ExtendQueryBuilder::for($this->model::class, $filters);
// Execute the query and return the sum
return $query->sum($column);
}
}
Next, we will create a default service class that provides common service methods for interacting with the repository.
<?php
namespace App\Services;
use App\Repositories\DefaultRepositoryInterface;
use Illuminate\Pagination\LengthAwarePaginator;
use Illuminate\Support\Collection;
abstract class DefaultService
{
protected $repository;
/**
* DefaultService constructor.
*
* @param DefaultRepositoryInterface $repository
*/
public function __construct(DefaultRepositoryInterface $repository)
{
$this->repository = $repository;
}
/**
* Get paginated list of items with filters.
*
* @param array $filters
* @return LengthAwarePaginator
*/
public function index(array $filters = []): LengthAwarePaginator
{
return $this->repository->index($filters);
}
/**
* Get a list of items with filters.
*
* @param array $filters
* @return Collection
*/
public function find(array $filters = []): Collection
{
return $this->repository->find($filters);
}
/**
* Get the first item matching the filters.
*
* @param array $filters
* @return Model|null
*/
public function findFirst(array $filters = []): ?Model
{
return $this->repository->findFirst($filters);
}
/**
* Count the number of items matching the filters.
*
* @param array $filters
* @return int
*/
public function count(array $filters = []): int
{
return $this->repository->count($filters);
}
/**
* Sum a specific column of items.
*
* @param string $column
* @param array $filters
* @return float
*/
public function sum(string $column, array $filters = []): float
{
return $this->repository->sum($column, $filters);
}
}
Now, create a ProductService class that extends the DefaultService. This class will be empty for now but is ready for future customizations.
<?php
namespace App\Services;
use App\Repositories\DefaultRepositoryInterface;
class ProductService extends DefaultService
{
// Currently, ProductService is empty and extends DefaultService
}
Finally, use the ProductService class in our controller to handle data retrieval with dynamic filtering. We will validate the request parameters and pass them to the service.
<?php
namespace App\Http\Controllers;
use App\Services\ProductService;
use Illuminate\Http\Request;
class ProductController extends Controller
{
protected $productService;
/**
* ProductController constructor.
*
* @param ProductService $productService
*/
public function __construct(ProductService $productService)
{
$this->productService = $productService;
}
public function index(Request $request)
{
// Validate request parameters
$params = $request->validate([
'page' => 'nullable|int|min:1',
'per_page' => 'nullable|int|min:1',
'filter' => 'array',
'sort' => 'nullable|string|min:1'
]);
// Get paginated products using the service
$products = $this->productService->index($filters);
// Return results as JSON
return response()->json($products);
}
}
To further enhance the filtering capabilities, you can add custom filters. This involves creating a base filter class and extending it for specific models. Custom filters allow for more granular control over query modifications.
First, create a base filter class that will handle common filter logic and provide methods for applying filters.
<?php
namespace App\Filters;
use App\Utilities\ExtendQueryBuilder;
use Illuminate\Database\Eloquent\Builder;
class BaseFilter
{
protected $query;
public function __construct(Builder $query)
{
$this->query = $query;
}
/**
* Apply filters to the query.
*
* @param array $filters
* @return Builder
*/
public function apply(array $filters): Builder
{
foreach ($filters as $filter => $value) {
if (method_exists($this, $filter)) {
$this->$filter($value);
}
}
return $this->query;
}
/**
* Prepare filters array (could be overridden by subclasses).
*
* @param array $filters
* @return array
*/
public function prepareFiltersArray(array $filters): array
{
return $filters;
}
/**
* Check if a table has been joined before.
*
* @param string $table
* @return bool
*/
protected function hasJoinedBefore(string $table): bool
{
return ExtendQueryBuilder::hasJoinedBefore($this->query, $table);
}
}
For each model, you can create specific filter classes that extend the BaseFilter class and implement custom filtering logic. This allows you to encapsulate filter logic for each model in its own class.
For example, if you have a Product model, you could create a ProductFilter class:
<?php
namespace App\Filters;
use Illuminate\Database\Eloquent\Builder;
class ProductFilter extends BaseFilter
{
protected function search($searchTerm)
{
$this->query->where(function (Builder $query) use ($searchTerm) {
$query->where('name', 'like', '%' . $searchTerm . '%')
->orWhere('description', 'like', '%' . $searchTerm . '%');
});
}
}
In your repository or service, you can use these custom filter classes to apply specific filters to your queries.
To test the filtering system, you can use tools like Postman or cURL to send requests to your API endpoint. Here are some example requests:
/api/products?filter[price:gt]=100/api/products?filter[category][name:contains]=Electronics/api/products?filter[search]=mobile/api/products?sort=-priceEach request will dynamically filter and return the results based on the parameters provided.
Implementing dynamic filtering in Laravel using the ExtendQueryBuilder class along with the service and repository pattern provides a flexible and powerful way to manage complex queries. By supporting filters for related data and allowing for dynamic sorting and grouping, this approach can greatly enhance the responsiveness and versatility of your API. This method scales well with the growth of your application and helps maintain a clean and maintainable codebase.
Feel free to extend and customize the ExtendQueryBuilder class, the service, and repository classes to suit your specific needs and business logic. With this system in place, you'll be well-equipped to handle a wide range of data retrieval scenarios in your Laravel applications.