I am unable to store the csv files in a hashmap of hashmap due to OutOfMemoryError.
CSV file format: word,count
Outer Hashmap: HashMap> - For storing language code and hashmap
Inner Hashmap: HashMap - For storing word and count
I am trying to loop through a list of .csv files(51) of total memory size 145mb in Tomcat server, Eclipse. But I'm getting OutOfMemoryError. Is there any API to convert csv to hashmap in java instead of iterating the csv file line by line?
private static HashMap<String, HashMap<String, Long>> wordCountAllLang = new HashMap<String, HashMap<String, Long>>();
public static void fetchWordCount(){
HashMap<String, Long> wordCountMap;
String language, line;
String[] lineArr;
BufferedReader in = null;
try{
final File folder = new File(ProofingConstants.DICTIONARY.WORD_COUNT_DATA_FOLDER_PATH);
for (File fileEntry : folder.listFiles()) {
language = fileEntry.getName().replace(".csv", ""); //No i18n
LOGGER.log(Level.WARNING, "Language:###"+language);
if (!isDictionaryLanguage(language)){
continue;
}
//Reading the file contents and putting it in the hash map
wordCountMap = new HashMap<String, Long>();
try{
in = new BufferedReader(new FileReader(fileEntry));
while ((line = in.readLine()) != null) {
lineArr = line.split(",");
wordCountMap.put(lineArr[0], Long.parseLong(lineArr[1]));
}
}
catch(Exception e) {
LOGGER.log(Level.WARNING, "SEVERE_ERROR: Exception while reading csv file data into hashmap", e);
}
finally{
in.close();
}
//Storing the hashmap in another hash map using language as its key
wordCountAllLang.put(language, wordCountMap);
LOGGER.log(Level.WARNING, "Language completed");
}
}
catch(Exception e) {
LOGGER.log(Level.WARNING, "SEVERE_ERROR: Exception while getting word count from csv file", e);
}
}
"exception: OutOfMemoryError"