Hubbry Logo
Topic modelTopic modelMain
Open search
Topic model
Community hub
Topic model
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Topic model
Topic model
Topic model
View on Wikipedia
from Wikipedia

In statistics and natural language processing, a topic model is a type of statistical model for discovering the abstract "topics" that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden semantic structures in a text body. Intuitively, given that a document is about a particular topic, one would expect particular words to appear in the document more or less frequently: "dog" and "bone" will appear more often in documents about dogs, "cat" and "meow" will appear in documents about cats, and "the" and "is" will appear approximately equally in both. A document typically concerns multiple topics in different proportions; thus, in a document that is 10% about cats and 90% about dogs, there would probably be about 9 times more dog words than cat words. The "topics" produced by topic modeling techniques are clusters of similar words. A topic model captures this intuition in a mathematical framework, which allows examining a set of documents and discovering, based on the statistics of the words in each, what the topics might be and what each document's balance of topics is.

Topic models are also referred to as probabilistic topic models, which refers to statistical algorithms for discovering the latent semantic structures of an extensive text body. In the age of information, the amount of the written material we encounter each day is simply beyond our processing capacity. Topic models can help to organize and offer insights for us to understand large collections of unstructured text bodies. Originally developed as a text-mining tool, topic models have been used to detect instructive structures in data such as genetic information, images, and networks. They also have applications in other fields such as bioinformatics[1] and computer vision.[2]

History

[edit]

An early topic model was described by Papadimitriou, Raghavan, Tamaki and Vempala in 1998.[3] Another one, called probabilistic latent semantic analysis (PLSA), was created by Thomas Hofmann in 1999.[4] Latent Dirichlet allocation (LDA), perhaps the most common topic model currently in use, is a generalization of PLSA. Developed by David Blei, Andrew Ng, and Michael I. Jordan in 2002, LDA introduces sparse Dirichlet prior distributions over document-topic and topic-word distributions, encoding the intuition that documents cover a small number of topics and that topics often use a small number of words.[5] Other topic models are generally extensions on LDA, such as Pachinko allocation, which improves on LDA by modeling correlations between topics in addition to the word correlations which constitute topics. Hierarchical latent tree analysis (HLTA) is an alternative to LDA, which models word co-occurrence using a tree of latent variables and the states of the latent variables, which correspond to soft clusters of documents, are interpreted as topics.

Animation of the topic detection process in a document-word matrix through biclustering. Every column corresponds to a document, every row to a word. A cell stores the frequency of a word in a document, with dark cells indicating high word frequencies. This procedure groups documents, which use similar words, as it groups words occurring in a similar set of documents. Such groups of words are then called topics. More usual topic models, such as LDA, only group documents, based on a more sophisticated and probabilistic mechanism.

Topic models for context information

[edit]

Approaches for temporal information include Block and Newman's determination of the temporal dynamics of topics in the Pennsylvania Gazette during 1728–1800. Griffiths & Steyvers used topic modeling on abstracts from the journal PNAS to identify topics that rose or fell in popularity from 1991 to 2001 whereas Lamba & Madhusushan [6] used topic modeling on full-text research articles retrieved from DJLIT journal from 1981 to 2018. In the field of library and information science, Lamba & Madhusudhan [6][7][8][9] applied topic modeling on different Indian resources like journal articles and electronic theses and resources (ETDs). Nelson [10] has been analyzing change in topics over time in the Richmond Times-Dispatch to understand social and political changes and continuities in Richmond during the American Civil War. Yang, Torget and Mihalcea applied topic modeling methods to newspapers from 1829 to 2008. Mimno used topic modelling with 24 journals on classical philology and archaeology spanning 150 years to look at how topics in the journals change over time and how the journals become more different or similar over time.

Yin et al.[11] introduced a topic model for geographically distributed documents, where document positions are explained by latent regions which are detected during inference.

Chang and Blei[12] included network information between linked documents in the relational topic model, to model the links between websites.

The author-topic model by Rosen-Zvi et al.[13] models the topics associated with authors of documents to improve the topic detection for documents with authorship information.

HLTA was applied to a collection of recent research papers published at major AI and Machine Learning venues. The resulting model is called The AI Tree. The resulting topics are used to index the papers at aipano.cse.ust.hk to help researchers track research trends and identify papers to read, and help conference organizers and journal editors identify reviewers for submissions.

To improve the qualitative aspects and coherency of generated topics, some researchers have explored the efficacy of "coherence scores", or otherwise how computer-extracted clusters (i.e. topics) align with a human benchmark.[14][15] Coherence scores are metrics for optimising the number of topics to extract from a document corpus.[16]

Algorithms

[edit]

In practice, researchers attempt to fit appropriate model parameters to the data corpus using one of several heuristics for maximum likelihood fit. A survey by D. Blei describes this suite of algorithms.[17] Several groups of researchers starting with Papadimitriou et al.[3] have attempted to design algorithms with provable guarantees. Assuming that the data were actually generated by the model in question, they try to design algorithms that probably find the model that was used to create the data. Techniques used here include singular value decomposition (SVD) and the method of moments. In 2012 an algorithm based upon non-negative matrix factorization (NMF) was introduced that also generalizes to topic models with correlations among topics.[18]

In 2017, neural network has been leveraged in topic modeling to make it faster in inference,[19] which has been extended weakly supervised version.[20]

In 2018 a new approach to topic models was proposed: it is based on stochastic block model.[21]

Because of the recent development of LLM, topic modeling has leveraged LLM through contextual embedding[22] and fine tuning.[23]

Applications of topic models

[edit]

To quantitative biomedicine

[edit]

Topic models are being used also in other contexts. For examples uses of topic models in biology and bioinformatics research emerged.[24] Recently topic models has been used to extract information from dataset of cancers' genomic samples.[25] In this case topics are biological latent variables to be inferred.

To analysis of music and creativity

[edit]

Topic models can be used for analysis of continuous signals like music. For instance, they were used to quantify how musical styles change in time, and identify the influence of specific artists on later music creation.[26]

See also

[edit]

References

[edit]

Further reading

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Topic model

View on Grokipedia
from Grokipedia
A topic model is a type of algorithm designed to automatically discover and annotate latent topics, or thematic patterns, within a large collection of documents, such as text corpora, by analyzing co-occurrences of words without requiring . These models treat documents as mixtures of topics and topics as distributions over vocabulary terms, enabling the extraction of interpretable summaries that reveal underlying structures in unstructured text. Probabilistic topic models, the dominant paradigm in this field, formalize the discovery process through generative statistical frameworks that simulate how documents are produced from hidden topic distributions. The foundational method, , introduced in 2003, posits a three-level hierarchical Bayesian model where each document is a finite mixture over a fixed number of topics, and each topic is an infinite mixture over word probabilities drawn from a Dirichlet prior. in LDA typically employs variational methods or sampling techniques like to approximate posterior distributions, allowing for scalable application to massive datasets. Topic modeling has evolved from earlier techniques like (LSA) in the 1990s, which used for , and probabilistic latent semantic indexing (pLSA) in 1999, which introduced mixture models but suffered from overfitting on unseen documents. LDA addressed these limitations by incorporating Bayesian priors, paving the way for extensions such as dynamic topic models for time-evolving corpora, supervised variants for classification tasks, and nonparametric models like hierarchical Dirichlet processes that infer the number of topics automatically. Applications of topic models span , , and beyond, including , recommendation systems, and exploratory analysis of or streams. In recent developments as of 2025, neural topic models integrate architectures, such as variational autoencoders and Large Language Models (LLMs), to handle short texts and embeddings, improving coherence and scalability on modern hardware. Despite challenges in evaluation—often relying on metrics or human judgments—these techniques remain essential for making sense of vast digital archives.

Introduction

Definition and Core Concepts

A topic model is a statistical technique for discovering latent thematic structure in a collection of documents, representing each document as a of topics and each topic as a over words in a fixed . These models operate under the bag-of-words assumption, treating documents as unordered collections of words where the order and specific positions do not influence the thematic representation, focusing instead on word frequencies to capture semantic content. Latent topics serve as hidden variables that explain the observed words, enabling the model to infer underlying themes without explicit supervision or annotations. At the core of topic models is a generative probabilistic framework, which posits an imaginary by which documents are produced: first, a distribution over topics is selected for the ; then, for each word position, a topic is drawn from this distribution, and a word is sampled from the corresponding topic's word distribution. This treats topics as unobserved components that generate the observable word sequences, allowing the model to reverse-engineer the latent structure from the data. For a single document in this framework, the basic generative process follows a Dirichlet-multinomial model, where the topic proportions θ\theta are drawn from a parameterized by α\alpha; for each word, a topic assignment zz is sampled from a multinomial distribution governed by θ\theta; and the word ww is then drawn from a over the vocabulary conditioned on the selected topic's parameters ϕz\phi_z: θ∼Dir(α),z∼Mult(θ),w∼Mult(ϕz).\begin{align*} \theta &\sim \mathrm{Dir}(\alpha), \\ z &\sim \mathrm{Mult}(\theta), \\ w &\sim \mathrm{Mult}(\phi_z). \end{align*} As an illustration, consider a corpus of news articles: a topic model might uncover a "" topic with high probabilities for words like "," "," and "," alongside a "sports" topic featuring terms such as "," "," and "score," thereby revealing thematic clusters across the collection.

Role in Natural Language Processing

Topic models play a pivotal role in (NLP) by enabling the unsupervised discovery of latent semantic structures within large collections of unstructured text data. This capability bridges the gap between raw textual inputs and structured representations, such as topic distributions over documents, allowing for interpretable insights into thematic content without requiring labeled training data. By modeling text as mixtures of topics—where each topic is a distribution over words—topic models facilitate the extraction of hidden patterns that reflect underlying themes, making them essential for handling the high volume and variability of . In information retrieval, topic models enhance performance through topic-based indexing, which captures document semantics beyond simple term matching and improves relevance ranking in ad-hoc search tasks. For instance, by representing documents as mixtures of topics rather than sparse bag-of-words vectors, these models enable more effective smoothing and query expansion, leading to higher retrieval precision. Similarly, in sentiment analysis, topic models contribute by disentangling sentiment from topical content, as seen in joint sentiment-topic frameworks that simultaneously infer polarity and themes, thereby improving the accuracy of opinion mining in reviews or social media. A key advantage of topic models in NLP is their ability to reduce dimensionality, transforming high-dimensional vocabulary spaces—often exceeding 10,000 terms—into compact K-dimensional topic representations, where K is typically much smaller (e.g., 50–200 topics). This reduction not only mitigates the curse of dimensionality but also serves as a foundational step for downstream tasks like document clustering, where topic vectors enable efficient grouping of similar texts based on shared themes. For example, topic models have been applied to , automatically identifying thematic categories such as "work-related" or "promotions" from unlabeled inboxes, which supports rule-based organization and spam detection without manual annotation.

Historical Development

Origins in

The foundations of topic modeling trace back to early (IR) systems developed in the , which emphasized structured representations of text to improve search efficiency. The , pioneered by Gerard Salton at , introduced term-document matrices as a core mechanism for indexing and retrieving documents based on weighted term frequencies, laying essential groundwork for later latent structure techniques. These matrices captured associations between terms and documents but struggled with synonymy and , highlighting the need for methods that could uncover deeper semantic relationships beyond exact term matching. During the and 1980s, models emerged as a dominant paradigm in IR, representing documents and queries as vectors in a high-dimensional space where similarity was measured via cosine distance or dot products. This approach, formalized by Salton and colleagues, enabled ranking based on term co-occurrences but revealed limitations in handling semantic nuances, such as related terms not explicitly co-occurring, which spurred interest in to reveal latent topical structures. Conferences like the ACM SIGIR, with its early meetings in the fostering key discussions on these challenges, played a pivotal role in driving innovations toward more sophisticated retrieval models. A landmark advancement came in 1990 with the introduction of Latent Semantic Indexing (LSI) by Deerwester et al., which applied (SVD) to term-document matrices for , thereby capturing implicit associations among terms and documents to enhance retrieval accuracy. LSI addressed some shortcomings by approximating latent semantic factors, yet its deterministic nature lacked a probabilistic interpretation, limiting its ability to model uncertainty in term distributions and motivating subsequent probabilistic extensions. This transition toward probabilistic frameworks built directly on LSI's insights into latent structures.

Evolution from Latent Semantic Analysis

Latent Semantic Indexing (LSI), a deterministic matrix method for uncovering latent topics in document collections, laid the groundwork for subsequent probabilistic approaches by addressing synonymy and in . However, LSI's reliance on lacked a statistical foundation for generative modeling, prompting the development of probabilistic alternatives in the late . In 1999, Thomas Hofmann introduced Probabilistic Latent Semantic Analysis (pLSA), also known as Probabilistic Latent Semantic Indexing, as a statistical extension of LSI that incorporates a , termed the aspect model, to generate word- co-occurrences probabilistically. The aspect model posits that each word in a is generated by first selecting a latent topic zz conditioned on the document, followed by sampling the word from the topic's distribution, enabling a likelihood-based framework fitted via expectation-maximization that outperformed LSI in retrieval tasks. Despite these advances, pLSA suffered from due to its maximum-likelihood estimation without hierarchical priors, resulting in parameters that scaled linearly with the training corpus size (kV+kMkV + kM, where kk is the number of topics, VV the vocabulary size, and MM the number of documents) and poor generalization to unseen documents, as it lacked a proper generative process for new data. This led to a pivotal shift in 2003 with the introduction of (LDA) by David M. Blei, Andrew Y. Ng, and , which established a fully generative Bayesian model for topic discovery by imposing Dirichlet priors on topic distributions to promote sparsity and coherence while fixing the parameter count independent of corpus size. LDA's hierarchical structure—drawing document-topic proportions θ\theta from a and topic-word distributions ϕ\phi similarly—enabled scalable inference through variational methods or sampling, transitioning topic modeling from deterministic approximations to stochastic, exchangeable processes that better captured corpus-level regularities. Published in the , this work marked a key milestone in enabling broader applications beyond retrieval, such as visualization and summarization. Following LDA, the integration of Bayesian nonparametrics after further evolved topic models by allowing the number of topics to grow adaptively with data, as seen in extensions like the Hierarchical Dirichlet Process, which influenced scalable, infinite mixtures for dynamic corpora without fixed hyperparameters.

Mathematical Foundations

Probabilistic Frameworks

Topic models operate within a probabilistic framework that conceptualizes documents as observed data generated from mixtures of hidden latent topics. In this setup, each document is represented as a distribution over topics, and each topic as a distribution over words, enabling the model to capture the underlying thematic structure of a corpus through processes. This approach draws on Bayesian principles to infer the posterior distribution of hidden variables—such as topic assignments and mixture proportions—given the observed words, providing a principled way to handle in topic discovery. A key aspect of this framework is the use of conjugate priors to ensure computational tractability. The serves as the for the multinomial distributions governing topic mixtures and word distributions, allowing for efficient posterior updates in . This choice facilitates the integration of prior knowledge about sparsity and smoothness in topic assignments, which is crucial for modeling real-world text data where topics are often sparse. Graphical models, often depicted using , compactly represent the generative process by illustrating dependencies and repetitions across documents and words; for instance, plates denote replication over multiple documents (D) and words within each document (N). The hierarchical structure distinguishes corpus-level from document-level distributions, enabling shared topics across the entire collection while allowing topic mixtures to vary per document. In models like (LDA), the per-topic word distributions φ are drawn once from a corpus-level Dirichlet prior parameterized by β, promoting coherence across documents, whereas per-document topic proportions θ are drawn independently from a document-level Dirichlet prior parameterized by α. This setup captures both global thematic consistency and document-specific emphases. The full joint distribution over the observed words W, latent topic assignments Z, document-topic distributions θ, and topic-word distributions φ, given hyperparameters α and β, is given by: p(W,Z,θ,ϕ∣α,β)=∏k=1Kp(ϕk∣β)∏d=1D[p(θd∣α)∏n=1Ndp(zd,n∣θd)p(wd,n∣zd,n,ϕzd,n)],\begin{aligned} p(W, Z, \theta, \phi \mid \alpha, \beta) = \prod_{k=1}^{K} p(\phi_k \mid \beta) \prod_{d=1}^{D} \left[ p(\theta_d \mid \alpha) \prod_{n=1}^{N_d} p(z_{d,n} \mid \theta_d) p(w_{d,n} \mid z_{d,n}, \phi_{z_{d,n}}) \right], \end{aligned} where is the number of topics, D the number of documents, and N_d the number of words in document d; here, p(φ_k | β) is Dirichlet, p(θ_d | α) is Dirichlet, p(z_{d,n} | θ_d) is multinomial, and p(w_{d,n} | z_{d,n}, φ) is multinomial. This formulation encapsulates the generative process and serves as the foundation for inference in probabilistic topic models.

Matrix Factorization Approaches

Matrix factorization approaches to topic modeling provide a non-probabilistic framework for discovering latent topics by decomposing the term-document matrix into lower-rank factors, offering deterministic alternatives to probabilistic methods. In this paradigm, the term-document matrix W∈Rm×nW \in \mathbb{R}^{m \times n}, where mm is the vocabulary size and nn is the number of documents, is approximated as W≈UVW \approx U V, with U∈Rm×kU \in \mathbb{R}^{m \times k} representing the topic-word matrix (where columns are topic distributions over words) and V∈Rk×nV \in \mathbb{R}^{k \times n} the document-topic matrix (where rows are topic mixtures for documents), for kk topics. This decomposition uncovers topics as coherent groups of terms and assigns documents to mixtures of these topics without assuming generative processes. The cornerstone of these approaches is (NMF), introduced by Lee and Seung in 1999, which enforces non-negativity constraints on UU and VV to yield interpretable, parts-based representations. Unlike unconstrained factorizations such as , NMF's non-negativity ensures that topics emerge as additive combinations of word features, promoting intuitive and human-readable results, as demonstrated in early applications to text data where semantic features naturally arise. NMF is optimized by minimizing the Frobenius norm of the reconstruction error: min⁡U,V≥0∥W−UV∥F2\min_{U, V \geq 0} \| W - U V \|_F^2 subject to U≥0U \geq 0 and V≥0V \geq 0, typically solved using multiplicative update rules that iteratively refine the factors while preserving non-negativity: U←U⊙WVTUVVT,V←V⊙UTWUTUV,U \leftarrow U \odot \frac{W V^T}{U V V^T}, \quad V \leftarrow V \odot \frac{U^T W}{U^T U V}, where ⊙\odot denotes element-wise multiplication. These updates converge to a local minimum, enabling efficient computation for large-scale text corpora. NMF offers distinct advantages in topic modeling, including inherent sparsity in the factor matrices, which reduces noise and highlights key terms per topic, and facilitates visualization by allowing topics to be represented as weighted sums of basis elements. For instance, sparse UU columns emphasize a subset of words defining each topic, aiding interpretability in document clustering tasks. Extensions such as Archetypal Analysis build on NMF by further constraining the factors to lie within the convex hull of the data points, representing archetypes as extreme mixtures that enhance extremal topic discovery. Introduced by Cutler and Breiman in 1994, this method modifies the NMF objective to emphasize boundary points, proving useful for identifying pure topic prototypes in diverse datasets. In contrast to probabilistic frameworks that model uncertainty through distributions, matrix factorization approaches like NMF prioritize optimization-based decompositions for scalable, reproducible topic extractions.

Key Algorithms and Models

Latent Dirichlet Allocation

Latent Dirichlet Allocation (LDA) is a generative probabilistic model for collections of discrete data such as text corpora, formulated as a three-level hierarchical Bayesian model. In this framework, documents are generated as mixtures of latent topics, where each topic is itself a mixture of words drawn from a shared . This hierarchical structure assumes that both the mixing proportions for topics within documents and the distributions over words within topics follow Dirichlet priors, enabling the discovery of coherent thematic patterns across large document sets. The generative process underlying LDA operates at multiple levels. Globally, for each topic k=1,…,Kk = 1, \dots, K, the topic-word distribution ϕk\phi_k is drawn from a : ϕk∼Dir(β)\phi_k \sim \text{Dir}(\beta). For each document m=1,…,Mm = 1, \dots, M, the document-topic mixture θm\theta_m is drawn from θm∼Dir(α)\theta_m \sim \text{Dir}(\alpha). Then, for each word position n=1,…,Nmn = 1, \dots, N_m in document mm, a topic assignment zm,nz_{m,n} is sampled from the zm,n∼Mult(θm)z_{m,n} \sim \text{Mult}(\theta_m), and the observed word wm,nw_{m,n} is drawn from wm,n∼Mult(ϕzm,n)w_{m,n} \sim \text{Mult}(\phi_{z_{m,n}}). This process models documents as bags of words exchangeably, capturing the latent topical structure through the assignments ZZ and parameters θ,ϕ\theta, \phi. Key hyperparameters in LDA include α\alpha and β\beta, which shape the resulting distributions. The parameter α\alpha governs the sparsity of the document-topic mixtures θm\theta_m, where smaller values of α\alpha encourage sparser representations with fewer dominant topics per . Similarly, β\beta controls the smoothness of the topic-word distributions ϕk\phi_k, with smaller values leading to more peaked (less smooth) distributions that concentrate probability mass on fewer words per topic. In practice, the number of topics KK is typically set between 50 and 100 when modeling large text corpora, balancing and interpretability. Inference in LDA aims to estimate the posterior distribution over the latent topic assignments ZZ, document mixtures θ\theta, and topic-word distributions ϕ\phi given the observed words WW: p(Z,θ,ϕ∣W,α,β)∝p(W,Z,θ,ϕ∣α,β)p(Z, \theta, \phi \mid W, \alpha, \beta) \propto p(W, Z, \theta, \phi \mid \alpha, \beta) This posterior lacks a closed-form solution owing to the intricate dependencies introduced by the Dirichlet priors and multinomial likelihood, requiring approximate methods for computation.

Probabilistic Latent Semantic Analysis

(pLSA), also referred to as (pLSI), is an probabilistic technique for discovering latent topics in a collection of documents. Introduced by Thomas Hofmann in 1999, it extends by incorporating a statistical to capture the probabilistic relationships between words and documents through unobserved latent variables representing topics or aspects. In pLSA, documents are viewed as mixtures of these latent topics, and topics are distributions over words, enabling the model to represent the co-occurrence patterns in text data more flexibly than deterministic methods. The core formulation of pLSA, known as the aspect model, posits that the probability of observing a word ww in a document dd is generated through a latent topic zz: P(w∣d)=∑zP(z∣d)P(w∣z)P(w \mid d) = \sum_z P(z \mid d) P(w \mid z) Here, P(z∣d)P(z \mid d) represents the mixing proportions of topics in document dd, while P(w∣z)P(w \mid z) denotes the probability of word ww under topic zz. This generative process treats each word occurrence as independently drawn from one of the topics associated with its document, assuming a fixed number of topics ZZ. To estimate the model parameters, pLSA employs the Expectation-Maximization (EM) algorithm, which iteratively maximizes the log-likelihood of the observed word-document data: log⁡P(W∣D)=∑d,wlog⁡∑zP(z∣d)P(w∣z)\log P(W \mid D) = \sum_{d,w} \log \sum_z P(z \mid d) P(w \mid z) The E-step computes posterior probabilities over latent topics, and the M-step updates the topic mixtures and word distributions accordingly. Despite its foundational role in probabilistic topic modeling, pLSA has notable limitations. The model lacks a proper generative story for new documents, making it unsuitable for assigning probabilities to unseen documents without retraining, as parameters are tied directly to the training corpus. Additionally, without regularization mechanisms like priors, pLSA is prone to , particularly as the number of parameters scales linearly with the training set size, leading to poor on sparse data. These issues motivated extensions such as , which introduces Bayesian priors to mitigate them.

Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) decomposes a non-negative input matrix V∈R≥0n×mV \in \mathbb{R}_{\geq 0}^{n \times m} into two lower-rank non-negative matrices W∈R≥0n×rW \in \mathbb{R}_{\geq 0}^{n \times r} and H∈R≥0r×mH \in \mathbb{R}_{\geq 0}^{r \times m}, such that V≈WHV \approx WH, where r≪min⁡(n,m)r \ll \min(n, m). In topic modeling, the columns of WW serve as basis vectors representing topics as distributions over words, while the rows of HH indicate the proportions of each topic in the corresponding documents. The non-negativity constraint promotes an additive parts-based representation, enhancing interpretability by ensuring that data points are reconstructed from localized, non-overlapping components rather than holistic or subtractive mixtures. For instance, when applied to pixel images of faces, NMF learns basis images corresponding to distinct facial parts like eyes, noses, and mouths. Similarly, in text corpora, it identifies coherent groups of words forming semantic topics, such as terms related to chemistry (e.g., "aluminum," "copper," "iron") or (e.g., "constitution," "court," "rights"). NMF was originally proposed in 1999 as a method for of object parts, with demonstrations on both image and text data for discovering semantic features. Extensions in 2001 focused on practical algorithms for computing the , enabling its broader adoption in topic modeling tasks. Common algorithms for NMF include multiplicative updates and alternating , both of which guarantee non-negativity and converge to a local minimum. Multiplicative updates, derived as diagonally rescaled , minimize objectives like the Frobenius norm ∥V−WH∥F2\|V - WH\|_F^2 through iterative element-wise . The update rules are: Haμ←Haμ(WTV)aμ(WTWH)aμH_{a\mu} \leftarrow H_{a\mu} \frac{(W^T V)_{a\mu}}{(W^T W H)_{a\mu}} Wia←Wia(VHT)ia(WHHT)iaW_{ia} \leftarrow W_{ia} \frac{(V H^T)_{ia}}{(W H H^T)_{ia}} A small positive constant ϵ\epsilon can be added to denominators for numerical stability. Analogous rules apply for minimizing the generalized Kullback-Leibler divergence. Alternating least squares (ALS) alternately optimizes WW and HH by solving non-negative least squares subproblems, often using active-set methods for efficiency in high dimensions.

Inference Methods

Variational Inference Techniques

Variational inference approximates the intractable posterior distributions in probabilistic topic models by selecting a tractable variational distribution q(Z,θ,ϕ)q(Z, \theta, \phi) that minimizes the Kullback-Leibler (KL) divergence to the true posterior p(Z,θ,ϕ∣W)p(Z, \theta, \phi \mid W). This process equivalently maximizes the (ELBO), which provides a tractable lower bound on the marginal log-likelihood of the observed data WW. The ELBO is formulated as L(q)=Eq[log⁡p(W,Z,θ,ϕ)−log⁡q(Z,θ,ϕ)],L(q) = \mathbb{E}_q \left[ \log p(W, Z, \theta, \phi) - \log q(Z, \theta, \phi) \right], where the expectation is taken with respect to qq, and maximizing L(q)L(q) tightens the bound while facilitating optimization. A common approach within variational inference employs a mean-field approximation, which factorizes the variational distribution to assume conditional independence among latent variables, such as q(Z,θ,ϕ)=q(θ∣γ)∏d∏nq(zdn∣ϕdn)q(Z, \theta, \phi) = q(\theta \mid \gamma) \prod_d \prod_n q(z_{dn} \mid \phi_{dn}), with Dirichlet priors on θ\theta and multinomial forms for the topic assignments ZZ. For Latent Dirichlet Allocation (LDA), inference proceeds via a coordinate ascent algorithm that iteratively optimizes the variational parameters. In the expectation (E) step, the variational posterior over topic assignments for each word is updated as q(zdn=k)∝exp⁡(ψ(γdk)+ψ(βkwdn)−ψ(∑wβkw))q(z_{dn}=k) \propto \exp\left( \psi(\gamma_{dk}) + \psi(\beta_{k w_{dn}}) - \psi\left( \sum_w \beta_{kw} \right) \right), where ψ\psi denotes the digamma function, γdk\gamma_{dk} parameterizes the per-document topic proportions, and βkw\beta_{kw} relates to the topic-word distributions. The maximization (M) step then refines the hyperparameters, such as the Dirichlet parameters α\alpha and β\beta, by maximizing the resulting ELBO. These variational techniques scale efficiently to large corpora, supporting over millions of documents through deterministic optimization that avoids the high variance of sampling-based alternatives. This scalability comes at the cost of introducing approximation bias in the posterior estimates, prioritizing computational speed over the unbiased but slower convergence of methods like MCMC.

Sampling-Based Methods

Sampling-based methods for inference in topic models primarily rely on Markov Chain Monte Carlo (MCMC) techniques to approximate the posterior distribution over latent variables, such as topic assignments, by generating samples from the joint distribution. These approaches are particularly valuable for models like (LDA), where exact inference is intractable due to the high-dimensional parameter space. Unlike deterministic approximations, MCMC methods provide asymptotically exact samples from the posterior, enabling better quantification of uncertainty in topic assignments and model parameters. A cornerstone of these methods is collapsed Gibbs sampling, which integrates out the continuous parameters (topic proportions θ and word distributions φ) to sample directly from the conditional distribution over topic assignments z. In LDA, the process iteratively samples the topic z_i for each word position i from its full conditional distribution, excluding the current assignment to avoid self-influence: P(zi=k∣z−i,w,α,β)∝(nd,k−i+α)nk,t−i+βnk,⋅−i+VβP(z_i = k \mid z_{-i}, w, \alpha, \beta) \propto (n_{d,k}^{-i} + \alpha) \frac{n_{k,t}^{-i} + \beta}{n_{k,\cdot}^{-i} + V \beta} Here, d is the document of word i, t is the observed word type, n_{d,k}^{-i} is the number of words in document d assigned to topic k excluding i, n_{k,t}^{-i} is the number of times word t is assigned to topic k excluding i, n_{k,\cdot}^{-i} is the total assignments to topic k excluding i, V is the vocabulary size, and α, β are Dirichlet hyperparameters. This sampling is repeated across all word positions in a sweep, with multiple sweeps continued until the chain reaches stationarity, as indicated by convergence diagnostics. The method was notably implemented and applied to scientific abstracts by Griffiths and Steyvers in , demonstrating its efficacy for discovering coherent topics. To ensure reliable , MCMC chains require periods to discard initial samples biased by starting values, allowing the chain to converge to the ; for instance, the first 1,000 iterations are often discarded in LDA applications. , or subsampling the chain at regular intervals (e.g., every 100 iterations), further reduces between samples, improving the effective sample size for estimating posterior expectations like topic-word distributions. While these techniques enhance accuracy, they increase computational cost compared to faster approximations. For efficiency, extensions incorporate the to sample from the multinomial conditionals in amortized O(1) time per draw by precomputing alias tables for the , as introduced in AliasLDA, which reduces the per-iteration complexity from O() to O(1) for topics. Overall, sampling-based methods excel in capturing posterior uncertainty but remain computationally intensive, often requiring thousands of iterations for large corpora.

Evaluation and Metrics

Intrinsic Measures

Intrinsic measures assess the quality of topic models internally, using only the model's parameters and the underlying corpus, without external tasks or human judgments. These metrics primarily evaluate how well the model captures the statistical structure of the data, focusing on fit and predictive generalization. Key examples include perplexity and held-out likelihood, which are derived from probabilistic principles and are applicable to models like (LDA). Perplexity quantifies the model's predictive power on held-out test data by measuring how surprised the model is by unseen words, with lower values indicating better performance. It is computed as the exponential of the negative average log-likelihood per word across the test set: perplexity(Dtest)=exp⁡(−∑d=1Mlog⁡p(wd)∑d=1MNd)\text{perplexity}(D_{\text{test}}) = \exp\left( -\frac{\sum_{d=1}^M \log p(w_d)}{\sum_{d=1}^M N_d} \right) where DtestD_{\text{test}} consists of MM documents, wdw_d denotes the sequence of words in document dd, and NdN_d is the length of dd in words. This metric originates from language modeling and has been adapted for topic models to gauge generalization, as demonstrated in early LDA evaluations where it outperformed unigram baselines. Despite its utility, perplexity has notable limitations in evaluating semantic quality, as it emphasizes likelihood-based fit over human-interpretable topic coherence or diversity. For instance, models with high may still produce meaningful topics, while low- models can yield semantically poor distributions. Held-out likelihood forms the basis for , directly estimating the probability of unseen documents under the model, p(w∣θ,ϕ,α)p(w \mid \theta, \phi, \alpha), where θ\theta are document-topic distributions, ϕ\phi are topic-word distributions, and α\alpha are hyperparameters. Due to the intractability of exact in LDA, approximations such as or bridge sampling are employed. Log-likelihood on the training data measures in-sample fit but tends to favor overparameterized models due to , making it less reliable for . To address this, the combines training and held-out likelihoods, approximating the as the over posterior samples z(s)z^{(s)}: p(w∣θ,ϕ,α)≈(1S∑s=1S1p(w∣z(s),ϕ))−1,p(w \mid \theta, \phi, \alpha) \approx \left( \frac{1}{S} \sum_{s=1}^S \frac{1}{p(w \mid z^{(s)}, \phi)} \right)^{-1}, where SS is the number of samples drawn from p(z∣w,θ,ϕ,α)p(z \mid w, \theta, \phi, \alpha). This estimator balances fit and generalization but can suffer from high variance. As a representative example, LDA models trained on the 20 Newsgroups dataset—a collection of approximately 20,000 documents across 20 categories—often yield perplexity scores around 1068 for 128 topics, providing a benchmark for comparing methods and hyperparameters.

Extrinsic Measures

Extrinsic measures assess the practical utility and semantic quality of topic models by evaluating their performance in downstream applications and alignment with human judgments, rather than solely internal statistical properties. These metrics emphasize interpretability and effectiveness in real-world tasks, such as enhancing document classification or information retrieval systems. By focusing on external criteria, extrinsic evaluations help determine how well topics support broader NLP objectives, including user-facing applications where coherent and diverse topics improve outcomes like recommendation accuracy or search relevance. A primary extrinsic metric is topic coherence, which quantifies the semantic relatedness among the top words representing a topic, serving as a proxy for interpretability. Coherence scores are derived from co-occurrence patterns in a reference corpus, such as , and have been validated against annotations where evaluators rate topics on scales from coherent to incoherent. For instance, automatic coherence measures achieve Spearman rank correlations of up to 0.78 with judgments on datasets like news articles and books, approaching inter-annotator agreement levels of 0.79–0.82. annotations typically involve multiple raters assessing 200–300 topics from models like LDA, providing gold-standard benchmarks for tuning and comparison. Prominent coherence variants include the UMass measure and Normalized Pointwise Mutual Information (NPMI). The UMass coherence computes the sum over pairs of top words of the log of their conditional co-occurrence probability, normalized by the total number of pairs: UMass=1N(N−1)∑i=1N∑j>iNlog⁡P(wj∣wi)P(wj),\text{UMass} = \frac{1}{N(N-1)} \sum_{i=1}^{N} \sum_{j>i}^{N} \log \frac{P(w_j \mid w_i)}{P(w_j)}, where NN is the number of top words per topic, P(wj∣wi)P(w_j \mid w_i) is the fraction of documents containing wiw_i that also contain wjw_j, and P(wj)P(w_j) is the fraction of documents containing wjw_j. This asymmetric measure favors word pairs that frequently co-occur in documents, promoting interpretable topics. NPMI extends by normalization to handle sparsity: NPMI(wi,wj)=log⁡P(wi,wj)P(wi)P(wj)−log⁡P(wi,wj),\text{NPMI}(w_i, w_j) = \frac{\log \frac{P(w_i, w_j)}{P(w_i) P(w_j)}}{-\log P(w_i, w_j)}, yielding values between -1 and 1, with higher scores indicating stronger semantic association based on joint and marginal probabilities from a large reference corpus. NPMI often outperforms UMass in correlating with human ratings due to its symmetry and normalization. Another widely adopted measure is C_v coherence, which combines document-level co-occurrence with pairwise word similarities derived from co-occurrence statistics (functioning as distributional embeddings) to compute an average indirect cosine similarity across topic words. This hybrid approach captures both topical proximity in documents and broader semantic links, achieving Pearson correlations of up to 0.859 with human evaluations on benchmarks like the 20 Newsgroups dataset. C_v is particularly effective for models producing diverse, human-readable topics, as it balances local context with global word relations. Topic coherence metrics are instrumental in hyperparameter tuning, such as selecting the optimal number of topics KK, by plotting coherence scores against KK and identifying peaks that indicate balanced granularity. Studies have demonstrated that maximizing semantic coherence during inference, such as through asymmetric priors in LDA, can improve the proportion of interpretable topics compared to standard settings. Recent advances include LLM-based metrics, such as Contextualized Topic Coherence (CTC), which leverage large language models to evaluate topic interpretability by considering contextual patterns and embeddings, achieving higher correlations with human judgments than traditional measures. Beyond coherence, extrinsic evaluations often examine integration in downstream tasks, where topic distributions serve as features for classifiers or retrieval systems. In , topic-enhanced models have shown improvements in F1-scores on tasks like , as topics provide compact, interpretable representations that capture latent themes missed by bag-of-words approaches. Similarly, in , topics boost precision at rank 10 by aligning queries with thematic document clusters, enhancing in large corpora. To ensure non-redundancy, diversity metrics complement coherence by quantifying topic overlap, typically via the average pairwise between topic-word probability vectors: TD=1−1K(K−1)∑i≠jcos⁡(θi,θj),\text{TD} = 1 - \frac{1}{K(K-1)} \sum_{i \neq j} \cos(\theta_i, \theta_j), where θi\theta_i and θj\theta_j are topic distributions and KK is the number of topics; values closer to 1 indicate greater diversity. High diversity prevents topics from converging on similar terms, supporting comprehensive coverage in applications like corpus exploration.

Applications

Text Corpus Analysis

Topic models have been widely applied to general text processing tasks, enabling the organization and exploration of large document collections through the discovery of latent themes. In document clustering, topic models project documents into a lower-dimensional space of topic distributions, facilitating the grouping of similar texts based on shared thematic content rather than exact word matches. This approach improves clustering accuracy by capturing semantic similarities, as demonstrated in integrations of topic modeling with traditional clustering algorithms like k-means, where topic weights serve as features for partitioning documents. For browsing, topic models support interactive navigation by providing summaries of document sets via topic proportions, allowing users to drill down into relevant subsets without exhaustive reading. A prominent application is in digital libraries, where topic models enhance search and discovery in vast archives. For instance, JSTOR's Topicgraph tool employs topic modeling to generate visual overviews of books, highlighting key topics and linking them to specific pages for efficient exploration of long-form content. This facilitates scholarly browsing by revealing thematic structures in monographs and journals, scaling to millions of digitized texts. Trend analysis in represents another key use, particularly for detecting evolving discussions over time. On platforms like , topic models identify emerging topics from , tracking shifts in public discourse such as event-driven conversations. Dynamic topic models extend this by modeling topic evolution across time slices, capturing how themes like political events or cultural trends change in large corpora, as introduced in the seminal work on dynamic topic models applied to historical document collections. Illustrative examples include visualizations of topic models on the Annotated Corpus, where interfaces allow users to explore article themes through interactive topic maps, revealing patterns in journalistic coverage over decades. Joint sentiment-topic models further enrich analysis by simultaneously inferring topics and associated polarities, enabling nuanced insights into opinion dynamics within text corpora, such as product reviews or news comments. to massive datasets is achieved through online variants of LDA, which update topic distributions incrementally as new documents arrive, processing millions of documents efficiently without requiring full-batch recomputation. This makes topic modeling viable for real-time applications on web-scale text, maintaining model quality while reducing computational demands.

Biomedical and Scientific Literature

Topic modeling has been extensively applied to biomedical and scientific literature to uncover latent themes and trends in vast collections of research articles, particularly from databases like . By analyzing abstracts and full texts, methods such as (LDA) enable the identification of evolving research foci, including disease mechanisms, treatment advancements, and interdisciplinary connections. For instance, LDA applied to large corpora of millions of articles has revealed temporal shifts in research emphasis, such as the progression of studies on disease trajectories from basic to clinical interventions. These applications facilitate quantitative by grouping related publications, aiding researchers in synthesizing knowledge without manual curation. A notable example in involves the use of survival-linked LDA (survLDA), which integrates data with outcomes to characterize cancer subtypes. In a 2012 study, survLDA was employed to model heterogeneous patterns in cancer datasets, identifying prognostic subtypes by linking topic distributions to patient rates, thereby providing interpretable biomarkers for . This approach highlights how topic models extend beyond text to multimodal biomedical data, enhancing subtype discovery in . Similarly, topic-based leverages these models to aggregate evidence across studies; for example, LDA clusters publications by thematic similarity, enabling systematic reviews of treatment efficacy in sparse or heterogeneous datasets like rare diseases. Integration of topic modeling with network analysis has advanced drug discovery by mapping relationships between drugs, pathways, and genes in scientific literature. A pathway-based LDA variant analyzes PubMed texts to infer probabilistic associations, constructing networks that reveal potential drug targets and repurposing opportunities, such as linking off-target effects to novel therapeutic pathways. This method outperforms traditional keyword searches by capturing contextual co-occurrences in biomedical narratives. Biomedical texts often feature sparse medical terms and domain-specific jargon, posing challenges for standard topic models due to high dimensionality and rarity of specialized vocabulary. To address this, advanced variants like multiple kernel fuzzy topic modeling (MKFTM) incorporate fuzzy membership and kernel functions to handle sparsity, improving topic coherence in PubMed abstracts by reducing noise from infrequent terms while preserving semantic relevance. Additionally, specialized priors, such as those in Graph-Sparse LDA, enforce structured sparsity based on biomedical ontologies or graphs, enabling more interpretable topics that align with known biological relationships and mitigate overfitting in jargon-heavy corpora. These adaptations ensure robust performance in quantitative analyses of scientific literature, where evaluation metrics like topic coherence are crucial for validating domain-specific insights.

Creative and Multimedia Domains

Topic modeling extends to creative and multimedia domains, enabling the of stylistic , genre patterns, and collaborative influences in music, art, and . In music, these methods process and symbolic representations like files to discover latent themes and s. For example, applying BERTopic to 537,553 English song from diverse s such as , rock, , and R&B uncovered 541 topics, revealing thematic shifts over 70 years—from romantic motifs like "tears_heart_wish" dominant in the –1980s to increased and , such as the "nigga_niggas_bitch" topic comprising 37.88% of rap since the 1990s—thus highlighting genre-specific evolutions akin to those in chart analyses. Similarly, BERTopic on 3,455 song from 14 artists generated 215 topic clusters, measuring artist similarity via shared topics (e.g., hip-hop artists like , 2Pac, and overlapping in 5–6 emotional and event-based themes), which supports modeling collaborative patterns in creative works. Symbolic music data, such as sequences, benefits from specialized topic models that account for temporal structure. The Variable-gram Topic Model integrates latent topics with a to learn probabilistic representations of melodic sequences within genres, outperforming standard LDA in next-note prediction on datasets like 264 Scottish and Irish folk reels by distinguishing musically meaningful regimes such as keys (e.g., vs. ) and tempos. This approach models topics, as in , by capturing contextual dependencies in sequential phrases, facilitating analysis of creative processes like spontaneous variation in solos. In , topic models aid author attribution for artistic texts, treating authorship as a latent stylistic topic. The Disjoint Author-Document Topic model (DADT), an extension of LDA, projects authors and documents into separate topic spaces, achieving state-of-the-art accuracy (e.g., 93.64% on small sets and 28.62% on large corpora with 19,320 authors) by capturing genre-agnostic stylistic markers applicable to literary . Multimedia extensions incorporate correlated topic models to handle images alongside text in creative analysis. The Topic Correlation Model (TCM) jointly models textual topics via LDA and image features via bag-of-visual-words (e.g., SIFT descriptors), enabling cross-modal retrieval on datasets like TVGraz; supporting stylistic studies in . Unique to arts applications, topic models address sequential and multimodal data through dynamic variants. The Document Influence Model, a dynamic topic extension, analyzes 24,941 songs (1922–2010) to track topic evolution over time slices, using time-decay kernels to quantify how influential tracks (e.g., innovative ones from the 1970s) shape subsequent genre topics, thus modeling stylistic progression in music corpora. TCM further integrates sequential image-text pairs for multimodal creativity, such as correlating narrative descriptions with artistic visuals in digital archives.

Recent Advances and Challenges

Neural and Deep Learning Integrations

One significant advancement in neural topic modeling emerged with ProdLDA, introduced in 2017, which adapts (LDA) using a framework to enable scalable inference through amortized optimization. This model replaces traditional multinomial priors with a product of experts prior, allowing end-to-end training where document embeddings are learned via an encoder-decoder , resulting in more coherent topics compared to standard LDA, as measured by automated coherence scores on benchmark datasets like 20 Newsgroups. ProdLDA's amortized inference approximates the posterior distribution efficiently during training, addressing limitations in classical LDA by integrating neural components for better representation of topic-document relationships without requiring collapsed . Building on such foundations, BERTopic, developed in 2020, leverages transformer-based embeddings from BERT combined with class-based TF-IDF (c-TF-IDF) to generate dynamic and interpretable topics from document clusters. The approach first embeds documents using BERT to capture contextual semantics, then applies via UMAP followed by HDBSCAN clustering, and finally represents topics with c-TF-IDF weighted by cluster assignments, enabling the model to handle evolving topics over time without retraining the entire pipeline. This integration has shown superior performance in topic diversity and coherence on short-text corpora, such as posts, where traditional bag-of-words models struggle due to sparsity. Neural topic models have further improved short-text handling and enabled zero-shot topic discovery by incorporating pre-trained language models, allowing inference on unseen domains without fine-tuning. For instance, contextualized embeddings from multilingual transformers facilitate cross-lingual topic extraction in zero-shot settings, outperforming non-neural baselines by up to 12% in F1 scores on classification tasks derived from topics. Recent developments from 2023 to 2025 have extended these to multimodal settings, such as neural topic models for text-image pairs, where joint variational inference on visual and textual features enhances topic interpretability in datasets such as artwork collections, achieving up to 174.8% improvement in recommendation accuracy over unimodal baselines. In applications with large language models (LLMs), neural topic models support interpretable prompting by providing structured topic representations that guide zero-shot generation, as seen in frameworks where LLMs rival traditional methods for topic assignment on long-context inputs. Scalability is bolstered through transformer architectures, enabling efficient processing of massive corpora via parallelizable embeddings and amortized inference, which reduces computational overhead by orders of magnitude compared to sampling-based alternatives. These integrations facilitate end-to-end training, where topic discovery and downstream tasks like classification are optimized jointly, promoting broader adoption in dynamic environments.

Scalability and Interpretability Issues

Scalability remains a primary challenge in topic modeling, particularly for applications where corpora exceed millions of documents. Traditional inference methods, such as sampling, suffer from high computational costs and slow convergence on large-scale datasets, often requiring days or weeks for training. To mitigate this, online variational Bayes approaches enable by processing documents in mini-batches, allowing models like to scale to massive without full recomputation. Similarly, distributed frameworks for hierarchical topic models distribute computation across clusters, achieving linear speedup for corpora up to billions of tokens while maintaining topic quality. Interpretability in topic models is hindered by issues of stability and diversity, where topics must be consistent across multiple runs and sufficiently distinct to provide meaningful insights. Instability arises from random initializations leading to varied topic-word distributions, complicating reliable analysis; metrics like normalized assess stability by comparing topic similarity over reruns. Diversity ensures topics capture broad corpus aspects without overlap, evaluated through measures like topic-word exclusivity, which penalizes redundant themes. Neural topic models exacerbate these challenges, as opaque embeddings can produce less human-readable topics compared to classical methods. Efforts to enhance interpretability include regularization techniques that promote coherent and diverse topics, such as constraints in variational autoencoders. Neural topic models integrating word embeddings inherit biases from pre-trained representations, resulting in skewed topics that amplify societal prejudices, such as stereotypes in word co-occurrences. For instance, embeddings trained on web corpora often associate professional terms with masculine attributes, leading to biased topic clusters in downstream applications like . Post-2020 studies employing topic modeling on AI literature have revealed ethical issues in biased topic discovery, including the reinforcement of discriminatory narratives in analysis and the need for debiasing interventions to ensure equitable outcomes. These biases pose risks of perpetuating inequities, prompting calls for fairness-aware training in topic discovery pipelines. Future directions in topic modeling emphasize hybrid symbolic-neural architectures, which combine neural s for with symbolic rules for explicit reasoning, improving both and interpretability in complex domains. Real-time streaming topic models, leveraging online updates and embedding spaces, enable dynamic topic on live data feeds like , supporting applications in monitoring. of evaluation remains crucial, with ongoing efforts to develop unified benchmarks for coherence, diversity, and downstream utility to facilitate reproducible comparisons across models.

References

  1. https://www.ccs.neu.edu/home/vip/teach/DMcourse/5_topicmodel_summ/LDA_TM/UIntrotoTopicModelsBlei2011-5.pdf
  2. https://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf
  3. https://arxiv.org/abs/2401.15351
  4. https://towardsdatascience.com/advanced-topic-modeling-with-llms/
  5. https://www.cs.columbia.edu/~blei/papers/Blei2012.pdf
  6. https://www.semanticscholar.org/paper/LDA-based-document-models-for-ad-hoc-retrieval-Wei-Croft/94a8ace25d5112e22f7235bbba26570b008a73e9
  7. https://ciir.cs.umass.edu/pubfiles/ir-464.pdf
  8. https://dl.acm.org/doi/10.1145/1645953.1646003
  9. https://ieeexplore.ieee.org/document/7237060
  10. https://sigir.org/files/museum/pub-06/I-1.pdf
  11. https://dl.acm.org/doi/10.1145/361219.361220
  12. http://www.sigir.org/sigir2007/history.html
  13. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9
  14. https://dl.acm.org/doi/10.1145/312624.312649
  15. https://cacm.acm.org/research/probabilistic-topic-models/
  16. https://www.nature.com/articles/44565
  17. https://papers.nips.cc/paper/1861-algorithms-for-non-negative-matrix-factorization
  18. https://www.tandfonline.com/doi/abs/10.1080/00401706.1994.10485840
  19. https://pmc.ncbi.nlm.nih.gov/articles/PMC8534395/
  20. https://arxiv.org/pdf/1204.1956
  21. http://www.cs.columbia.edu/~blei/fogm/2020F/readings/LeeSeung1999.pdf
  22. https://proceedings.neurips.cc/paper_files/paper/2000/file/f9d1152547c0bde01830b7e8bd60024c-Paper.pdf
  23. https://www.pnas.org/doi/10.1073/pnas.0307752101
  24. http://archive.gersteinlab.org/meetings/s/2014/08.28/kdd2014-i0kdd-meeting-materials/docs/p891.pdf
  25. https://dl.acm.org/doi/10.1145/1553374.1553515
  26. https://www.cs.cmu.edu/~rsalakhu/papers/etm.pdf
  27. https://arxiv.org/pdf/1502.03630
  28. https://aclanthology.org/N10-1012/
  29. https://mimno.infosci.cornell.edu/nips2013ws/nips2013tm_submission_7.pdf
  30. https://aclanthology.org/D11-1024/
  31. https://aclanthology.org/2024.findings-eacl.123/
  32. https://www.sciencedirect.com/science/article/pii/S2949719123000419
  33. https://www.researchgate.net/publication/381696091_Leveraging_Topic_Modelling_to_Analyze_Biomedical_Research_Trends_from_the_PubMed_Database_Using_LDA_Method
  34. https://pmc.ncbi.nlm.nih.gov/articles/PMC5028368/
  35. https://arxiv.org/abs/1202.5999
  36. https://doi.org/10.1016/j.compbiolchem.2014.08.019
  37. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-022-04780-1
  38. https://ojs.aaai.org/index.php/AAAI/article/view/9603
  39. https://arxiv.org/pdf/2409.15949
  40. https://aile3.ijs.si/dunja/SiKDD2022/Papers/SiKDD2022_paper_5269.pdf
  41. https://icml.cc/2012/papers/585.pdf
  42. https://direct.mit.edu/coli/article/40/2/269/1479/Authorship-Attribution-with-Topic-Models
  43. https://proceedings.mlr.press/v28/shalit13.pdf
  44. https://arxiv.org/abs/1703.01488
  45. https://arxiv.org/abs/2203.05794
  46. https://maartengr.github.io/BERTopic/index.html
  47. https://aclanthology.org/2021.eacl-main.143.pdf
  48. https://telecom-paris.hal.science/hal-03628242/file/preprint_zeroberto.pdf
  49. https://arxiv.org/abs/2403.17308
  50. https://www.mdpi.com/2079-9292/13/2/302
  51. https://arxiv.org/pdf/2510.03174
  52. https://www.annualreviews.org/content/journals/10.1146/annurev-statistics-112723-034123
  53. https://apps.dtic.mil/sti/tr/pdf/AD1028715.pdf
  54. http://www.vldb.org/pvldb/vol11/p826-chen.pdf
  55. https://dl.acm.org/doi/10.1145/3623269
  56. https://arxiv.org/html/2412.17338v1
  57. http://www.jmlr.org/papers/volume18/17-069/17-069.pdf
  58. https://dl.acm.org/doi/10.1145/3351095.3372843
  59. https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00325/96463/Topic-Modeling-in-Embedding-Spaces
  60. https://www.sciencedirect.com/science/article/pii/S0160791X25003173
  61. https://arxiv.org/html/2411.04383v1
  62. https://arxiv.org/abs/2509.01051
  63. https://www.cambridge.org/core/journals/political-science-research-and-methods/article/beyond-standardization-a-comprehensive-review-of-topic-modeling-validation-methods-for-computational-social-science-research/1D92CCD7C51491C9F6BE490BA5B434C4
  64. https://link.springer.com/article/10.1140/epjs/s11734-025-01874-8
Add your contribution
Related Hubs
Contribute something
User Avatar
No comments yet.