Comp Sci Twitter API finding the most common word in a tweet

Click For Summary
SUMMARY

The discussion focuses on a Java implementation for finding the most common word in tweets using the Twitter API. The class TJTwitter contains methods for splitting tweets into words, converting iterators to lists, and removing common English words from the list of terms. The issue arises in the removeCommonEnglishWords() method, where the terms list remains unchanged despite attempts to remove common words. The problem is identified as a misunderstanding of how to properly manipulate the iterator and list.

PREREQUISITES
  • Java programming fundamentals
  • Understanding of the Twitter API
  • Basic file handling in Java
  • Knowledge of iterators and collections in Java
NEXT STEPS
  • Review Java Collections Framework, focusing on List and Iterator interfaces
  • Learn about Java file I/O operations, specifically using Scanner and File classes
  • Explore best practices for removing elements from a collection while iterating
  • Investigate the usage of Java's Stream API for filtering collections
USEFUL FOR

Java developers, software engineers working with the Twitter API, and anyone interested in text processing and data manipulation in Java.

JessicaHelena
Messages
188
Reaction score
3
Homework Statement
This method queries the tweets of a particular user's handle.
Find the most common word in the user's tweets and also return the frequency.
Relevant Equations
N/A
So in the class TJTwitter, I have the following defined:
Java:
private Twitter twitter;
private PrintStream consolePrint;
private List<Status> statuses;
private List<String> terms;
private String popularWord;
private int frequencyMax;

and in the same class, I have a few methods that are meant to help me achieve the goal of the problem.
1. I have a splitIntoWords() method that takes each status and splits them into individual words, which are then stored in List terms.
2. I have an iteratorToList method that turns an iterator into a List:
Java:
181       public static <String> List<String> iteratorToList(Iterator<String> iterator)
182 {
183 List<String> list = new ArrayList<>();
184 iterator.forEachRemaining(list::add);
185 return list;
186       }

3. I have the removeCommonEnglishWords(), which is supposed to remove common English words from the list of terms. (I am given a file called commonWords.txt, which is in the same folder as this java file). The method should also not throw an exception:
Java:
194       @SuppressWarnings("unchecked")
195 private void removeCommonEnglishWords()
196 {
197 Scanner sc = null;
198
199 try
200 {
201 sc = new Scanner(new File("commonWords.txt"));
202 }
203 catch(Exception e)
204 {
205 System.out.println("The file is not found");
206 }
207
208 List<String> commonWords = new ArrayList<String>();
209 int count = 0;
210 while(sc.hasNextLine())
211 {
212 count++;
213 commonWords.add(sc.nextLine());
214 }
215
216 Iterator<String> termIt = terms.iterator();
217 while(termIt.hasNext())
218 {
219 String term = termIt.next();
220 for(String word : commonWords)
221 if(term.equalsIgnoreCase(word))
222 termIt.remove();
223 }
224 //terms = iteratorToList(termIt);
225       }

However, this, when run, does not alter the terms at all. But when I uncomment the last line (terms = listToIterator(termIt);), I get an empty term, I think. What am I doing wrong?
(When I check the terms List at the start of removeCommonEnglishWords(), the terms List is not empty.)
 
Last edited by a moderator:
Physics news on Phys.org
Please look up the syntax for the IF statement (as used in lns. 221-223).
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 2 ·
Replies
2
Views
4K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 12 ·
Replies
12
Views
2K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 10 ·
Replies
10
Views
11K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 1 ·
Replies
1
Views
758
  • · Replies 3 ·
Replies
3
Views
8K