How do I allow search in android that works with character accents aswell?

Question

I have implemented a search mechanism in my app, such that when I search for a name or email, it displays the string with the matching character. However, there are some accented strings in my list and when I search with a regular character pertaining to that specific accent, say if i have string "àngela" and I search "angela" it doesnt display the string unless i search with exact string " àngela".

I am trying to get it to work regardless of the accent or not, say if i type in "à" , it should show all strings containing " à" and "a" and vice versa. Any idea how to go about this? I looked up a bunch of articles online, for example: How to ignore accent in SQLite query (Android) " and tried normalizer too but it partially works, if i search "a", it does show the accented letters with regular letters aswell but if I search with accented letters, it doesnt show anything.

Here's my code for filter:

 @Override
    public Filter getFilter() {
        return new Filter() {
            @Override
            protected FilterResults performFiltering(CharSequence charSequence) {
                String charString = charSequence.toString();
                if (charString.isEmpty()) {
                    mSearchGuestListResponseListFiltered = mSearchGuestListResponseList;
                } else {
                    List<RegisterGuestList.Guest> filteredList = new ArrayList<>();
                    for (RegisterGuestList.Guest row : mSearchGuestListResponseList) {

                        // name match condition. this might differ depending on your requirement
                        // here we are looking for name or phone number match
                        String firstName = row.getGuestFirstName().toLowerCase();
                        String lastName = row.getGuestLastName().toLowerCase();
                        String name = firstName + " " +lastName;
                        String email = row.getGuestEmail().toLowerCase();
                        if ( name.trim().contains(charString.toLowerCase().trim()) ||
                                email.trim().contains(charString.toLowerCase().trim())){
                            filteredList.add(row);
                            searchText = charString.toLowerCase();
                        }
                    }

                    mSearchGuestListResponseListFiltered = filteredList;
                }

                FilterResults filterResults = new FilterResults();
                filterResults.values = mSearchGuestListResponseListFiltered;
                return filterResults;
            }

            @Override
            protected void publishResults(CharSequence charSequence, FilterResults filterResults) {
                mSearchGuestListResponseListFiltered = (ArrayList<RegisterGuestList.Guest>) filterResults.values;
                notifyDataSetChanged();
            }
        };
    }

Here's the entire adapter class if anyone's interested : https://pastebin.com/VxsWWMiS Here's the corresponding activity view:

searchView.setOnQueryTextListener(new SearchView.OnQueryTextListener() {
            @Override
            public boolean onQueryTextSubmit(String query) {
                mSearchGuestListAdapter.getFilter().filter(query);

                return false;
            }

            @Override
            public boolean onQueryTextChange(String newText) {
                mSearchGuestListAdapter.getFilter().filter(newText);
                mSearchGuestListAdapter.notifyDataSetChanged();
                mSearchGuestListAdapter.setFilter(newText);

                if(mSearchGuestListAdapter.getItemCount() == 0){


                    String sourceString = "No match found for <b>" + newText + "</b> ";
                    mNoMatchTextView.setText(Html.fromHtml(sourceString));
                } else {
                    mEmptyRelativeLayout.setVisibility(View.GONE);
                    mRecyclerView.setVisibility(View.VISIBLE);
                }
                return false;
            }
        });

Happy to share any details if necessary. Also, randomly I do get the indexoutofboundexception onBind() method while searching (use recyclerview for list):

java.lang.IndexOutOfBoundsException: Index: 7, Size: 0
        at java.util.ArrayList.get(ArrayList.java:437)

Any idea how to go about this?

Answer 1

In general, I would recommend using a Collator with a strength setting of Collator.PRIMARY to compare strings containing accents and varied cases (e.g., N vs n and é vs e). Unfortunately, Collator does not have a contains() function.

So we will make our own.

private static boolean contains(String source, String target) {
    if (target.length() > source.length()) {
        return false;
    }

    Collator collator = Collator.getInstance();
    collator.setStrength(Collator.PRIMARY);

    int end = source.length() - target.length() + 1;

    for (int i = 0; i < end; i++) {
        String sourceSubstring = source.substring(i, i + target.length());

        if (collator.compare(sourceSubstring, target) == 0) {
            return true;
        }
    }

    return false;
}

This iterates over the source string, and checks whether each substring with the same length as the search target is equal to the search target, as far as the Collator is concerned.

For example, let's imagine our source string is "This is a Tèst" and we are searching for the word "test". This method will iterate over every four-letter substring:

This
his 
is i
s is
 is 
is a
s a 
 a T
a Tè
 Tès
Tèst

And will return true as soon as it finds a match. Since the strength is set to Collator.PRIMARY, the collator considers "Tèst" and "test" to be equal, and so our method returns true.

It's quite possible that there's are more optimizations to be made to this method, but this should be a reasonable starting point.

Edit: One possible optimization is to leverage collation keys as well as known details of the implementation of RuleBasedCollator and RuleBasedCollationKey (assuming you have Google's Guava in your project):

private static boolean containsBytes(String source, String target) {
    Collator collator = Collator.getInstance();
    collator.setStrength(Collator.PRIMARY);

    byte[] sourceBytes = dropLastFour(collator.getCollationKey(source).toByteArray());
    byte[] targetBytes = dropLastFour(collator.getCollationKey(target).toByteArray());

    return Bytes.indexOf(sourceBytes, targetBytes) >= 0;
}

private static byte[] dropLastFour(byte[] in) {
    return Arrays.copyOf(in, in.length - 4);
}

This is considerably more fragile (probably doesn't work for all locales), but in my tests it is somewhere between 2x and 10x faster.

Edit: To support highlighting, you should convert contains() to indexOf(), and then use that information:

private static int indexOf(String source, String target) {
    if (target.length() > source.length()) {
        return -1;
    }

    Collator collator = Collator.getInstance();
    collator.setStrength(Collator.PRIMARY);

    int end = source.length() - target.length() + 1;

    for (int i = 0; i < end; i++) {
        String sourceSubstring = source.substring(i, i + target.length());

        if (collator.compare(sourceSubstring, target) == 0) {
            return i;
        }
    }

    return -1;
}

And then you could apply it like this:

String guestWholeName = guest.getGuestFirstName() + " " + guest.getGuestLastName();
int wholeNameIndex = indexOf(guestWholeName, searchText);

if (wholeNameIndex > -1) {
    Timber.d("guest name first : guest.getGuestFirstName() %s", guest.getGuestFirstName());
    Timber.d("guest name last : guest.getGuestLastName() %s", guest.getGuestLastName());

    int endPos = wholeNameIndex + searchText.length();

    Spannable spannable = new SpannableString(guestWholeName);
    Typeface firstNameFont = Typeface.createFromAsset(context.getAssets(), "fonts/Graphik-Semibold.otf");
    spannable.setSpan(new CustomTypefaceSpan("", firstNameFont), wholeNameIndex, endPos, Spannable.SPAN_EXCLUSIVE_EXCLUSIVE);
    Objects.requireNonNull(guestName).setText(spannable);
} else {
    Objects.requireNonNull(guestName).setText(guestWholeName);
}