Java program to count word frequency and sentence occurrence from a text file

[Franco]

hello everyone, thanks for taking ur time reading this po
i need help with something to do with java, designing an application to read sentences & words from a txt file, and create a new txt file, with all the words (in single) from the original txt file, next to each word, containing the frequency of how often each word appeared in the txt file and the number of sentences appeared. Assuming each sentence is paused with a full-stop. commas, question-marks, etc can be ignored.

Example:
Input file:
This is a simple simple example test. Another test.

Output file:
this 1 1
is 1 1
a 1 1
simple 2 1
test 2 2
example 1 1
another 1 1


so far i only have written my script as...


import java.io.*;
import java.util.*;

public class Analysis {

public static void main(String args[]) throws IOException {

File inputFile = null;
File outputFile = null;


inputFile = new File("Analysis_output.txt");
outputFile = new File("Analysis_source.txt");

FileReader in = new FileReader(inputFile);
FileWriter out = new FileWriter(outputFile);

int c;




in.close();
out.close();
}
}



i'm not sure how to convert sentences into arraylists (use charAt, seeking for fullstop?)
converting each words from a sentence into a sub-arraylist (use charAt, seeking for spaces in between?)

and the original txt file contains more than 1 line...
not all sentences stuffed into 1 line



THX FOR READING

 

[so-crates]

Use the StringTokenizer class.

 

[abhishek]

Use the StringTokenizer class.

From the http://java.sun.com/j2se/1.4.2/docs/api/java/util/StringTokenizer.html :

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

 

[ramollari]

From the http://java.sun.com/j2se/1.4.2/docs/api/java/util/StringTokenizer.html :

I've been using StringTokenizer myself with jsdk1.5 and it works fine. You just initialize it with the String and with the separator and the class does the job for you.
As suggestion use ". " as separator for sentences and " " for words. Also to improve your functionality during the separation of words, if the word contains a punctuation mark such as "." "," "?" etc. in the end, remove it.

 

[abhishek]

I've been using StringTokenizer myself with jsdk1.5 and it works fine. You just initialize it with the String and with the separator and the class does the job for you.

Yes, it will work fine. The text I quoted implies that clearly. I'm only pointing out that Sun would rather have you use split or regex.