Creating index vs adding document to index differences

  • Topic: Comp Sci 
  • Thread starter shivajikobardan
  • Start date
  • Tags Tags
    Index
1 reply · 2K views · last reply
shivajikobardan
Messages
637
Reaction score
54
Homework Statement
Lucene indexing
Relevant Equations
none
Block diagram titled Architecture of Lucene: indexing flows from raw content through acquire content, build document, analyze document and index document into a central Index; searching flows from user through search user interface and search index, with query and result arrows.

These are the steps of indexing in Lucene given in our syllabus-:
Screenshot of a course handout titled Creating an Index (IndexWriter Class) with three numbered steps and a Java line creating an IndexWriter named indexWriter with index-directory, StandardAnalyzer and true.

Screenshot of a course document titled Parsing the Documents (Analyzer Class), listing four Lucene analyzers: StandardAnalyzer, WhitespaceAnalyzer, StopAnalyzer and SnowballAnalyzer, followed by the Document class and a line of Java code calling doc.add with a Field.

The first step says that it is creating an index whereas the last step says that it's adding document to index.
What's the difference between these two? Can I get an example.

Here's what I think it should happen-:
1) Collect all words from each documents. Lists it like-;

doc1=>word1,word2,WORD3….wordn
doc2=>word1,WORD2,word3….wordn
And so on.

2) Analyse the words and remove various types of words as per analyzer, process them as per analyzer.

Say now what remains is-:
doc1=>word1,word3,...word(n-1)
doc2=>word2,...word(n-3)

3) Done. Now you can make inverted index as well by converting this to inverted index.

But it's done bit differently, which I'm not 100% clear about.
 
shivajikobardan said:
whereas the last step says that it's adding document to index.
No it doesn't, what you are calling the "last" step simply creates a document; adding it to the index is another step.

shivajikobardan said:
What's the difference between these two? Can I get an example.
Can you get an example of the difference between creating a thing and adding something to that thing? Are you serious?

shivajikobardan said:
Here's what I think it should happen-:
...
This is all done behind the scenes, you don't have to worry about any of this to use Lucene, you just need to learn how to use the API. A good place to learn that is the API documentation itself: https://lucene.apache.org/core/9_2_0/core/index.html
 
  • Like
Likes   Reactions: jim mcnamara