Remove duplicate values from a string in java

Question

Can anyone please let me know how to remove duplicate values from

String s="Bangalore-Chennai-NewYork-Bangalore-Chennai";

and output should be like

String s="Bangalore-Chennai-NewYork-";

using Java..

Any help would be appreciated.

Sure you want the `"-"` at the end of the result? – Martijn Courteaux Jul 22 '11 at 13:39 — Martijn Courteaux, Jul 22 '11 at 13:39

Bohemian · Accepted Answer · 2015-01-15T07:52:46.540

This does it in one line:

public String deDup(String s) {
    return new LinkedHashSet<String>(Arrays.asList(s.split("-"))).toString().replaceAll("(^\\[|\\]$)", "").replace(", ", "-");
}

public static void main(String[] args) {
    System.out.println(deDup("Bangalore-Chennai-NewYork-Bangalore-Chennai"));
}

Output:

Bangalore-Chennai-NewYork

Notice that the order is preserved :)

Key points are:

split("-") gives us the different values as an array
Arrays.asList() turns the array into a List
LinkedHashSet preserves uniqueness and insertion order - it does all the work of giving us the unique values, which are passed via the constructor
the toString() of a List is [element1, element2, ...]
the final replace commands remove the "punctuation" from the toString()

This solution requires the values to not contain the character sequence ", " - a reasonable requirement for such terse code.

Java 8 Update!

Of course it's 1 line:

public String deDup(String s) {
    return Arrays.stream(s.split("-")).distinct().collect(Collectors.joining("-"));
}

Regex update!

If you don't care about preserving order (ie it's OK to delete the first occurrence of a duplicate):

public String deDup(String s) {
    return s.replaceAll("(\\b\\w+\\b)-(?=.*\\b\\1\\b)", "");
}

Cutting of the brackets would I do using substring(), and do it on two lines instead. — Martijn Courteaux, Mar 20 '13 at 12:06
@MartijnCourteaux you could break this up into many lines, but this answer is also a demonstration of how it can be done in just one line. — Bohemian, Mar 20 '13 at 20:03

score 4 · Answer 2 · answered Jul 22 '11 at 13:33

public static String removeDuplicates(String txt, String splitterRegex)
{
    List<String> values = new ArrayList<String>();
    String[] splitted = txt.split(splitterRegex);
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < splitted.length; ++i)
    {
        if (!values.contains(splitted[i]))
        {
            values.add(splitted[i]);
            sb.append('-');
            sb.append(splitted[i]);
        }
    }
    return sb.substring(1);

}

Usage:

String s = "Bangalore-Chennai-NewYork-Bangalore-Chennai";
s = removeDuplicates(s, "\\-");
System.out.println(s);

Prints:

Bangalore-Chennai-NewYork

score 2 · Answer 3 · answered Jul 22 '11 at 13:31

2

You could add your strings to a HashSet.

Split the strings on a "-".
Store the individual words in Array. i.e arr[]

Sinppet :

Set<String> set = new HashSet<String>();

    for(int i=0; i < arr.length; i++){
      if(set.contains(arr[i])){
        System.out.println("Duplicate string found at index " + i);
      } else {
        set.add(arr[i]);
      }

answered Jul 22 '11 at 13:31

sgokhales

49,762
33
123
158

2

Or a LinkedHashSet to keep the original order – Maurice Perry Jul 22 '11 at 13:33
4

C'mon are you for real? You're checking to see if the String is already in the Set. But the whole point of a Set is that it doesn't contain duplicates. That's redundant code!!! – Aaron Chambers Jul 23 '11 at 18:14

score 1 · Answer 4 · edited Jul 01 '16 at 11:55

1

static String RemoveDuplicateCharInString(String s){
    for (int i = 0; i < s.length(); i++) {
        if((s.substring(i+1)).indexOf(s.charAt(i))!=-1){
            s=s.substring(0,i+1)+(s.substring(i+1)).replaceAll(""+s.charAt(i),"");
        }
    }
    return s;
}

edited Jul 01 '16 at 11:55

Dinidu Hewage

1,977
6
39
45

answered Jul 01 '16 at 10:59

raj neeraj

11
1

1

You should say a few words about your code solution and how it answers the OPs question so that they can understand it, rather than just posting code. – hypern Jul 01 '16 at 11:14

score 1 · Answer 5 · edited Jul 22 '11 at 14:06

1

Just the idea:

parse the string and split the tokens using separator "-"
load the tokens into a Collection
iterate the Collection and erase duplicates
use the result Collection to build the new string

The most tricky part should be 3, but not impossible. If you use a Set, you can skip this step.

EDIT maybe you can substitute 2&3 with a presence check before adding the element

edited Jul 22 '11 at 14:06

Qwerky

17,564
6
43
76

answered Jul 22 '11 at 13:31

ascanio

1,492
1
9
17

score 1 · Answer 6 · answered Jul 22 '11 at 13:34

Create array of string by spliting by - and then create a hashSet from it.

String s="Bangalore-Chennai-NewYork-Bangalore-Chennai"; 
String[] strArr = s.split("-");
Set<String> set = new HashSet<String>(Arrays.asList(strArr));

If you want back it as string array then do following:

String[] result = new String[set.size()];
set.toArray(result);

Here is a sample code to do this:

String s="Bangalore-Chennai-NewYork-Bangalore-Chennai"; 
String[] strArr = s.split("-");
Set<String> set = new LinkedHashSet<String>(Arrays.asList(strArr));
String[] result = new String[set.size()];
set.toArray(result);
StringBuilder res = new StringBuilder();
for (int i = 0; i < result.length; i++) {
    String string = result[i];
    if(i==result.length-1)
        res.append(string);
    else
        res.append(string).append("-");
}
System.out.println(res.toString());

Output:-

Bangalore-Chennai-NewYork

score 0 · Answer 7 · edited Apr 23 '16 at 16:04

0

public static void main(String[] args) {
    String str="Bangalore-Chennai-Newyork-Bangalore-Chennai";
    String output="";
    String [] arr=str.split("-");

    LinkedHashSet<String> lhs=new LinkedHashSet<String>();
    for (int i = 0; i < arr.length; i++) {
        lhs.add(arr[i]);
    }
    for(String s:lhs){
        output=output+s+"-";
    }

    System.out.println(output);
}

edited Apr 23 '16 at 16:04

Tunaki

116,530
39
281
370

answered Apr 23 '16 at 15:56

RAJESHPODDER007

633
1
7
12

It'd be good to explain what the solution would be, rather than just pasting the code of the solution. – Jose Gómez Apr 23 '16 at 16:14

score 0 · Answer 8 · answered Oct 12 '16 at 18:30

A little late to the game, but I would simply use a HashMap. It's easy to understand and has quick lookups on the keys, might not be the best way but it's still a good answer IMO. I use it all the time when I need to format quick and dirty:

                    String reason = "Word1 , Word2 , Word3";
                    HashMap<String,String> temp_hash = new HashMap<String,String>();
                    StringBuilder reason_fixed = new StringBuilder();
                    //in:
                    for(String word : reason.split(",")){
                        temp_hash.put(word,word);
                    }
                    //out:
                    for(String words_fixed : temp_hash.keySet()){
                        reason_fixed.append(words_fixed + " , ");
                    }
                    //print:
                    System.out.println(reason_fixed.toString());

score 0 · Answer 9 · answered Jan 30 '17 at 11:05

public class RemDuplicateWordFromString {
public static void main(String[] args) {
    String s1 = "Hello India Hello India Hello India Hello India";
    countWords(s1);
}
public static void countWords(String s1) {
    String[] s2 = s1.split(" ");
    for (int i = 0; i < s2.length; i++) {
        for (int j = i + 1; j < s2.length; j++) {
            if (s2[i].equals(s2[j])) {
                if (i != j) {
                    s2[i] = "";
                }
            }
        }
    }
    for (int i = 0; i < s2.length; i++) {
        if (s2[i] != "") {
            System.out.print(s2[i] + " ");
        }

    }

}

}

score 0 · Answer 10 · answered Jul 18 '18 at 01:29

0

StringBuilder builderWord = new StringBuilder(word);
for(int index=0; index < builderWord.length(); index++) {
    for(int reverseIndex=builderWord.length()-1; reverseIndex > index;reverseIndex--) {
        if (builderWord.charAt(reverseIndex) == builderWord.charAt(index)) {
            builderWord.deleteCharAt(reverseIndex);
        }
    }
}
return builderWord.toString();

answered Jul 18 '18 at 01:29

venkat

1

Please don't just post code blocks without context - include an explanation as well – CertainPerformance Jul 18 '18 at 01:59

score 0 · Answer 11 · answered Jul 22 '11 at 13:36

0

wordsArray = s.split("-");
List<String> wordsList = new Arrays.asList(wordsArray);
Set<String>  wordsSet  = new LinkedHashSet<String>(wordsList);

String[] noDuplicates = new String[wordsSet.size()];
wordsSet.toArray(noDuplicates);

answered Jul 22 '11 at 13:36

Hunter McMillen

52,839
21
105
154

score 0 · Answer 12 · answered Jul 22 '11 at 13:42

I'd prefer this which is simpler than all of the above.

public void removeDuplicates() {
  String myString = "Bangalore-Chennai-NewYork-Bangalore-Chennai";

  String[] array = myString.split("-");

  Set<String> hashSet = new HashSet<String>(Arrays.asList(array));

  String newString = StringUtils.join(hashSet, "-");        
}

score -1 · Answer 13 · edited Oct 31 '14 at 13:41

import java.util.HashSet;

public class SplitString {
    public static void main(String[] args) {
        String st = new String("New Delhi-Chennai-New York-Bangalore-Chennai-New Delhi-Chennai-New York");
        StringBuffer stb = new StringBuffer();

        HashSet<String> hashset = new HashSet<String>();
        for (String a : st.split("-"))
            hashset.add(a);

        Object[] str = (Object[]) hashset.toArray();

        for (int i = 0; i < str.length; i++) {
            stb.append(str[i]);

            if (i < str.length - 1)
                stb.append("-");

        }

        System.out.println(stb);
    }
}

score -2 · Answer 14 · edited Mar 31 '15 at 09:12

import java.util.*;

public class RemoveDuplicateWord {

    public static void main(String[] args) {
        String str = "Hai hello Hai how hello are how you";
        removeDupWord(str);
    }

    public static void removeDupWord(String input) {
        List<String> list = Arrays.asList(input.split(" "));
        LinkedHashSet<String> lhs = new LinkedHashSet<String>(list);
        for(String s : lhs) {
            System.out.print(s+" ");
        }                   
    }
}

Remove duplicate values from a string in java

14 Answers14

Java 8 Update!

Regex update!

Linked

Related