Lazy, Filtered and Sorted Vaadin Grid using External Data Source
In large enterprise application development projects, it's usual to code use cases that, as the main requirement, define to show the user huge amounts of data in a tabular way. For this kind of scenarios, there is a particular web component that comes really handy: a Grid. Given that we like Vaadin's framework, we are going to describe how to configure it in such a way that doesn't restrict you about performance issues when doing that.
This post assumes that you have some knowledge of web application development using the Vaadin Framework and the Java Platform. But we’ll try to explain everything as much as possible. The version that we used to write this application is Vaadin 13, but this part of the API is almost the same since Vaadin 8, so it should be possible to use the same approach in older versions.
Setting up a Vaadin Grid, is really easy, just create an instance and then customize the columns, or even skip that part if your Pojo is sufficiently self-descriptive.
As an example, let’s assume that we want to show a grid with data for a person, which we can define as a simplified entity like this:
public class Person { private String name; private String lastName; private Integer age; public Person(String name, String lastName, Integer age) { super(); this.name = name; this.lastName = lastName; this.age = age; } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public Integer getAge() { return age; } public void setAge(Integer age) { this.age = age; } }
And then we have a service that will have the responsibility of retrieving this information from a backend, like a database or a remote API. Let’s call it PersonService.
Suppose we want to show those entities in a Vaadin Grid, we would do this:
List<Person> persons = getPersonService().fetchPersons(); Grid<Person> personsGrid = new Grid<>(Person.class); personsGrid.setItems(persons);
This is fine for small amounts of data, but when dealing with large data sets some features, such as lazy loading, filtering and sorting, must be left to the backend, implementing that all at once can be a complex task, so let's review them.
Lazy Loading
First let’s define what we are talking about when we use this term. Usually when we are handling a list of items, we can choose to obtain all of them, or just the ones that we want to display or handle in some way. Obviously it depends on the amount. If we are obtaining one hundred of them, it is something that we can manage directly in memory, but if we are handling millions, we have to deal with them in a different way.
This is the case when a mechanism such as lazy loading makes things easier.
Vaadin components, like Grid or ComboBox, already support this kind of behavior, without the need of doing something special, but it only will take care of retrieving what is needed in a lazy loading fashion between the web browser and your web application.
Given that Vaadin only takes care of the presentation layer of your application, you need to do something to apply this behavior from your underlying data source (database, REST API, etc.).
According to the official documentation, you need to ask to your backend two questions:
Can you tell me the total amount of items that we are dealing with?
Of those, can you only give me the first N items, skipping the previous M items?
You don’t need to calculate N and M, those numbers are provided by the framework. They are basically the limit (N) and offset (M). You answer those two questions by providing two lambda expressions to the method DataProvider.fromCallbacks(), something like this:
DataProvider<Person, Void> dataProvider = DataProvider.fromCallbacks( // First callback fetches items based on a query query -> { // The index of the first item to load int offset = query.getOffset(); // The number of items to load int limit = query.getLimit(); List<Person> persons = getPersonService() .fetchPersons(offset, limit); return persons.stream(); }, // Second callback fetches the number of items for a query query -> getPersonService().getPersonCount()); ); Grid<Person> grid = new Grid<>(); grid.setDataProvider(dataProvider);
Now we added two new different methods to our service class.
But let’s continue with another relevant requirement.
Filtering
It’s a rather common request to narrow down the size of the queried items, and one typical way of doing that is to make grids filterable.
But before digging into this, we need to define what is a Filter. For us it’s just a class that will contain values that we want to be taken into account when deciding if a given item will be returned or not.
In our case, let’s assume that we only want to filter people by their name and last name:
public class PersonFilter { private String nameFilter = null; private String lastNameFilter = null; public PersonFilter() { } public PersonFilter(String nameFilter, String lastNameFilter) { this.setNameFilter(nameFilter); this.setLastNameFilter(lastNameFilter); } public String getNameFilter() { return nameFilter; } public String getLastNameFilter() { return lastNameFilter; } public void setNameFilter(String nameFilter) { this.nameFilter = nameFilter; } public void setLastNameFilter(String lastNameFilter) { this.lastNameFilter = lastNameFilter; } }
Now that we have our filter, how to do the actual filtering?
Instead of calling DataProvider.fromCallbacks(), we are going to use DataProvider.fromFilteringCallbacks(), makes sense, right?
DataProvider<Person, PersonFilter> dataProvider = DataProvider.fromFilteringCallbacks( query -> { Optional<PersonFilter> filter = query.getFilter(); return getPersonService().fetchPersons( query.getOffset(), query.getLimit(), filter.map(f -> f.getNameFilter()).orElse(null), filter.map(f -> f.getLastNameFilter()).orElse(null) ); }, query -> { Optional<PersonFilter> filter = query.getFilter(); return getPersonService().getPersonCount( filter.map(f -> f.getNameFilter()).orElse(null), filter.map(f -> f.getLastNameFilter()).orElse(null) ); } );
In this case, what we are doing, is just to obtain the filter from the query, and send the filter values to the backend class. We cannot do the filtering by ourselves because if the underlying backend is a database, then those filters have to be taken into account when building the SQL queries. If we do the filtering by ourselves, then it’s impossible to calculate the final count of the items, without retrieving all of the entities from the backend, causing our lazy loading mechanism to fail its goal (not to retrieve everything). The same applies when calling a remote API.
But let’s continue, if we want to change the filters dynamically, then we need to be able to configure the filters, for that we need to call the method withConfigurableFilter(), that will return an enhanced DataProvider with a method that will allow us to set a given Filter instance:
PersonFilter gridFilter = new PersonFilter(); ConfigurableFilterDataProvider<Person,Void,PersonFilter> dp = dataProvider.withConfigurableFilter(); dp.setFilter(gridFilter);
We can change the filter later, or just save a reference to this instance and modify it’s values. After doing that a call to the method refreshAll() in the DataProvider will trigger the loading of the data with the new filter values.
Now, what about the visual part? … let’s see:
HeaderRow hr = personsGrid.prependHeaderRow(); TextField nameFilterTF = new TextField(); nameFilterTF.addValueChangeListener(ev->{ gridFilter.setNameFilter(ev.getValue()); dp.refreshAll(); }); hr.getCell(personsGrid.getColumnByKey("name")).setComponent(nameFilterTF); TextField lastNameFilterTF = new TextField(); lastNameFilterTF.addValueChangeListener(ev->{ gridFilter.setLastNameFilter(ev.getValue()); dp.refreshAll(); }); hr.getCell(personsGrid.getColumnByKey("lastName")).setComponent(lastNameFilterTF);
First we are creating a HeaderRow, that is a row that will show at the top of our fancy grid, that will contain components that will filter our data.
Then we are creating a TextField for holding the strings to filter the name and last name of our people.
Finally we are adding some value change listeners that will actually apply the filters when changing the value.
That’s great! … now we are loading filtered data in a lazy way using multiple filters.
But what if we need to sort the data? … let’s continue.
Sorting
Similarly to our filtering case, we need a class to store each sorting information that we need:public class PersonSort { private String propertyName; private boolean descending; public String getPropertyName() { return propertyName; } public void setPropertyName(String propertyName) { this.propertyName = propertyName; } public boolean isDescending() { return descending; } public void setDescending(boolean descending) { this.descending = descending; } }
The class is pretty self-explained. Just holding the property that is currently being sorted, and a boolean for specifying if we sort up or down.
You might think why we are constructing these classes (PersonFilter and PersonSort), given that we could just pass the values directly to the service. The main reason is to have framework agnostic classes to hold these kind of information and then pass them to the backend, making a cleaner separation of layers.
In our example, we are going to provide a new parameter to our PersonService: a list of PersonSort (sort orders). This is a list, because the order of the sort criteria is important, you can establish this order in the Grid component, by calling personsGrid.setMultiSort(true). Let’s review the changes in our code:
DataProvider<Person, PersonFilter> dataProvider = DataProvider.fromFilteringCallbacks( query -> { Optional<PersonFilter> filter = query.getFilter(); List<PersonSort> sortOrders = query.getSortOrders().stream().map(sortOrder->new PersonSort(sortOrder.getSorted(),sortOrder.getDirection().equals(SortDirection.ASCENDING))).collect(Collectors.toList()); return getPersonService().fetchPersons( query.getOffset(), query.getLimit(), filter.map(f -> f.getNameFilter()).orElse(null), filter.map(f -> f.getLastNameFilter()).orElse(null), sortOrders ); }, query -> { Optional<PersonFilter> filter = query.getFilter(); return getPersonService().getPersonCount( filter.map(f -> f.getNameFilter()).orElse(null), filter.map(f -> f.getLastNameFilter()).orElse(null) ); } );
Now we are assembling this list of PersonSort, using query.getSortOrders().
The implementation of PersonService is irrelevant, because it's up to the backend to provide the data requested using the information provided:
- Lazy loading information: offset and limit
- Filtering information: name and last name filters
- Sorting information: list of PersonSort
If the backend obtains the data from a relational database, then a SQL clause has to be created dynamically based on that information. For example:
SELECT * FROM PERSONS P WHERE P.NAME LIKE %FILTERNAME% AND P.LASTNAME LIKE %LASTNAME% ORDER BY P.NAME DESC, P.LASTNAME ASC LIMIT OFFSET,LIMIT
Here's a small animation showing our grid in action:
You can play around with this example, by checking this GitHub project with the sources.
Have fun!
