Importing array lists from a txt and filter numbers out

Question

I am a M.D., PhD candidate who wants to learn some programming in java.

I am conducting a research on the heart rate variability analysis and I want to make a simple program that would allow me to input an array list from a .txt file, filter some data out, and export the filtered data to a new .txt file. I have made the program in QBasic :), but I want to do it in java too.

I am able to get the first part, how to import an array:

public class random {
    public static void main (String[] args) throws FileNotFoundException
{
  Scanner s = new Scanner(new File("c:\\data.txt"));
  ArrayList<String> list = new ArrayList<String>();
  while (s.hasNext()){
      list.add(s.next());
  }
  s.close();

The program should filter repeated double type numbers from the txt file and only print out a unique values to a new txt document. Here you have my original question, where you can see what I mean:

Extraction of unique values form a array list

A user suggested this code, which works perfectly if I insert the numbers by hand.

 int[] input = new int[]{0, 0, 0, 0, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 5, 5, 5, 5, 8, 8, 10, 10, 2, 2, 2, 3, 3, 7, 7};
 int current = input[0];
 boolean found = false;

 for (int i = 1; i < input.length; i++) {
     if (current == input[i] && !found) {
         found = true;
     } else if (current != input[i]) {
         System.out.print(" " + current);
         current = input[i];
         found = false;
     }
 }
 System.out.print(" " + current);

Does anyone have suggestions how could I, instead of inserting the numbers by hand, import the array list from a text file and filter the data using the for loop in the previous code?

So your question is - *how to populate input array from data in a file*. right? — TheLostMind, Sep 22 '14 at 13:49
How will you manage double/int list in .txt file ??? Line by line or delimited with comma (,) ??? — Wundwin Born, Sep 22 '14 at 13:51
You can use a Set, look here: http://stackoverflow.com/questions/14887976/sortable-java-collection-without-duplicates — MarGar, Sep 22 '14 at 13:53
I am not sure if sets would be very useful (or maybe it's my ignorance), because there's a possibility that one number would be repeated after a certain amount of numbers. That repeated number shouldn't be excluded. For example: 0,0,1,1,2,2,3,3,3,4,4,4,6,6,0,0,2,2 should be printed out as: 0,1,2,3,4,6,0,2 — Ivn Bubrov, Sep 22 '14 at 15:07

score 2 · Answer 1 · edited May 23 '17 at 11:49

It depends on how the numbers are present in the file. Assuming you have a file with a number per line,

You can read the file line by line, instead of looping over the array.

//This is needed because unlike your array solution we do not know the first element already.
//If your file can have -1, use a number that you are sure will not be in the file
//If you are not sure that there is such a nunmber, use the wrapper class Integer and check for null
int current = -1; 

BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) 
{
   try
   {
       int i = Integer.parseInt(line); //use Double.parseDouble if the numbers can be decimals.
   }
   catch(NumberFormatException e)
   {
       //The line is not a number. So, some currupt data
       System.out.println("Encountered a non-number value in a line, ignoring value);
       e.printStacktrace();
   }
   if(current == -1)
   {
       current = i;
   }
  if (current == input[i] && !found) {
      found = true;
  ...
  ...

}
br.close();

Note: There are multiple assumptions here:

That the file has numbers one per line. If the file is delimited, say by a comma, you need to change your approach. Reading comma separated values (CSV) with Java: Java - Read CSV with Scanner().
You can use the Scanner class as in the linked answer above to read line by line as well.
The "duplicate check" logic you have assumes that the data in the file is sorted (so the duplicates are one after the other. If this is not the case, you need to use a Set or a similar structure to check for duplicates. See Remove Duplicate Lines from Text using Java for an example, but note that the answer there is for removing duplicates, you do not necessarily have to remove.

Yes, the numbers are presented in one column, one per line. I will try out your code — Ivn Bubrov, Sep 22 '14 at 14:00
@IvnBubrov also try to understand what the code is doing :-) — Nivas, Sep 22 '14 at 14:01
of course :D the whole point is to learn to write this code on my own :) thanks :D — Ivn Bubrov, Sep 22 '14 at 14:03

ares · Answer 2 · 2014-09-22T19:19:27.817

1

Since you're trying to remove duplicates here, the easiest way to achieve that is to use a Set.

    Scanner s = new Scanner(new File("c:\\data.txt"));
    HashSet<Double> set = new HashSet<Double>();



    while(s.hasNextDouble()){
        set.add(s.nextDouble());
    }

    System.out.println(set);

}

EDIT: Removing immediate dupliactes

This is not as simple as using a Set but not complicated either

Scanner s = new Scanner(new File("c:\\data.txt"));
ArrayList<Double> list = new ArrayList<Double>();

double next = s.nextDouble();
list.add(next);
double previous = next;
while(s.hasNextDouble()){
    next = s.nextDouble();
    if(next!=previous){
        list.add(next);
        previous = next;
    }

}

edited Sep 22 '14 at 19:19

answered Sep 22 '14 at 13:56

ares

3,921
4
31
54

Yes, I want to remove duplicates, but there's a possibility that I will get some numbers that occured before, later in the list. for example: 0,0,0,1,1,1,2,2,2,0,0 should be printed out as: 0,1,2,0 – Ivn Bubrov Sep 22 '14 at 14:54
This is perferct!! You are really a genius :) I added this part to output the data into a txt document: String fileName = "processed_data.txt"; PrintWriter outFile = new PrintWriter(new FileWriter(fileName)); outFile.print(list); outFile.close(); What would you change to get the output in a column instead of a array [...]? – Ivn Bubrov Sep 23 '14 at 00:13
You can iterate over the values: for(double x : list){outFile.println(x);} outFile.close(); – ares Sep 23 '14 at 06:43
Works perfectly!! Thank you very very much! – Ivn Bubrov Sep 23 '14 at 13:35

score 0 · Answer 3 · answered Sep 22 '14 at 13:54

s.hasNext returns a boolean, not a string, so your ArrayList is filled with boolean values and not numbers.

Do this instead;

public class Main {

    public static void main (String[] args) throws FileNotFoundException
    {
  Scanner s = new Scanner(new File("//data.txt"));
  ArrayList<String> list = new ArrayList<String>();

  while (s.hasNext()){
      String str = s.nextLine();
      list.add(str);
  }
  s.close();
  System.out.println(list
          );
   }

}

score 0 · Answer 4 · answered Sep 22 '14 at 14:03

Because arrays have fixed length, I suggest you to use a Collection like TreeSet:

  Scanner scanner = new Scanner("c:\\data.txt");
  TreeSet<Double> list = new TreeSet<Double>();
  while (scanner.hasNextDouble()) {
     double nextDouble = scanner.nextDouble();
     Double lastFound = list.last();
     if(lastFound != null && lastFound.doubleValue() != nextDouble) {
        list.add(new Double(nextDouble));
     }
  }

I searched for double-values instead of integer.

Importing array lists from a txt and filter numbers out

4 Answers4