Optimizing Topic Modeling: Saving and Loading Models for Faster Processing

  • Context: Python 
  • Thread starter Thread starter EngWiPy
  • Start date Start date
  • Tags Tags
    Modeling Topic
Join the discussion
Ask a follow-up here, or get your own question answered by working scientists, mathematicians and engineers — people, not an autocomplete.
Real named experts · corrections over time · the nuance an AI answer skips
1 reply · 1K views
EngWiPy
Messages
1,361
Reaction score
61
Hello,

I am running a program that takes relatively long time to read the corpus of some documents, and create the topic model. The code to do this is:

Code:
from gensim import models, corpora

corpus = corpora.BleiCorpus('./data/ap/ap.dat', './data/ap/vocab.txt')

#Creating the topic model
model = models.ldamodel.LdaModel(corpus, num_topics = 100, id2word = corpus.id2word)

I am testing different things with the code, and it's a little slow to run the code several times because of the above code's segment. How can I overcome this issue?

Thanks
 
Physics news on Phys.org
It turned out the model can be saved and loaded. Problem solved.
 
  • Like
Likes   Reactions: jim mcnamara and berkeman