Topic modelling of farmer comments in online agricultural consultation groups using LDA

Text Mining and LDA Topic Modelling for Agriculture and Plantations

Field: Agriculture and PlantationsLevel: Doctoral dissertation (S3)Method: Text Mining and LDA Topic Modelling

0 views

Extracting topics from farmer consultation comments with LDA and relating them to comment sentiment score and number of replies, illustrated with synthetic data. Text mining turns text into word data, and Latent Dirichlet Allocation (LDA) finds hidden topics as word mixtures and each document as a topic mixture. It suits theses analysing reviews, social media comments or narrative documents.

Topic modelling of farmer comments in online agricultural consultation groups using LDA: result sheet

Project info

Field of studyAgriculture and Plantations
LevelDoctoral dissertation (S3)
Method typeData Exploration
Estimated build time± 7 working days
Toolstidytext::unnest_tokens, tidytext::cast_dtm, topicmodels::LDA, topicmodels::perplexity, ggplot2
PriceUS$31 - US$191

Research goal

Extracting topics from farmer consultation comments with LDA and relating them to comment sentiment score and number of replies, illustrated with synthetic data.

Variables

Role	Variable	Unit
Main	Farmer consultation comments	documents
Supporting 1	Comment sentiment score	score
Supporting 2	Number of replies	count

Example result

460Documents
32Terms (after stopwords)
14.614Perplexity (k = 3)
0.733Mean max topic share
<0.001Kruskal-Wallis p-value
0.64Spearman sent.-rating

A three-topic LDA on 460 documents (32 terms after stopword removal, perplexity 14.6) separates topics led by hama, pupuk, panen. Mean number of replies is highest for dominant topic T2: pupuk (24.76) and lowest for T1: hama (15.24); the Kruskal-Wallis test (p < 0.001) shows a significant difference.

Computed from illustrative synthetic data, not from a real study.

About Text Mining and LDA Topic Modelling

Text mining turns text into word data, and Latent Dirichlet Allocation (LDA) finds hidden topics as word mixtures and each document as a topic mixture. It suits theses analysing reviews, social media comments or narrative documents.

Steps

1Collect the texts and tokenise them with tidytext::unnest_tokens
2Remove Indonesian stopwords and count word frequencies
3Build the document-term matrix with tidytext::cast_dtm
4Fit LDA with k = 3 (topicmodels::LDA, Gibbs sampling) and compute perplexity
5Interpret top terms per topic (beta) and topic shares per document (gamma)
6Compare ratings across dominant topics with the Kruskal-Wallis test

Assumptions to check

Documents can be treated as unordered bags of words
The number of topics k is set in advance and supported by perplexity or interpretability
Preprocessing (stopwords, spelling normalisation) is applied consistently

Reading and reporting the results

Name each topic from its highest-beta terms and report the perplexity and the mean dominant topic share. Link the topics to other variables, such as mean rating per dominant topic with a Kruskal-Wallis test, to show which topics go with low ratings.

Example R code

A short starting script with this project's variable names. The delivered version is adapted to your data and fully commented.

Show code

# Contoh kode R: Text Mining and LDA Topic Modelling
suppressPackageStartupMessages({
  library(ggplot2)
  library(patchwork)
  library(dplyr)
  library(tidytext)
  library(topicmodels)
})
set.seed(42)
# ---- Gaya grafik ----
pal <- c("#1f6feb", "#e8590c", "#2f9e44", "#ae3ec9", "#f59f00")
warna <- list(bg = "#f4f6fa", panel = "#ffffff", teks = "#1f2933", redup = "#5f6b78", grid = "#e3e8ee", bar = "#1d3557")
tata <- 1
theme_set(theme_minimal(base_size = 11) + theme(legend.position = "bottom"))

# ---- Data sintetis: korpus dokumen pendek dari campuran 3 topik ----
n_dok <- sample(seq(300, 500, by = 10), 1)
kata_topik <- list(c("layanan", "cepat", "ramah", "petugas", "antrian", "pelayanan"), c("harga", "mahal", "murah", "biaya", "diskon", "tarif"), c("aplikasi", "error", "login", "update", "fitur", "lambat"))        # kata khas tiap topik
kata_henti <- c("yang", "dan", "di", "ke", "dari", "ini", "itu", "untuk", "dengan", "saya", "kami", "sudah", "tidak",
                "ada", "juga", "karena", "tapi", "jadi", "sangat", "bisa", "akan", "lagi", "masih", "atau", "pada",
                "sih", "aja", "ya", "lebih", "kalau", "sekali", "banget", "mau", "harus", "kok", "nya")
kata_umum <- sample(c("bagus", "kurang", "mohon", "semoga", "baru", "sering", "hari", "cukup", "pengalaman", "tolong",
                     "kali", "minggu", "pertama", "kemarin", "terus", "puas", "kecewa", "lumayan", "biasa", "oke"),
                   sample(8:14, 1))                     # kosakata umum yang muncul lintas topik
gamma_rand <- function(a) { g <- rgamma(length(a), a); g / sum(g) }   # Dirichlet lewat gamma
campuran <- t(replicate(n_dok, gamma_rand(rep(runif(1, 0.25, 0.45), 3))))
buat_dok <- function(th) {
  L <- sample(10:25, 1)
  jenis <- sample(c("topik", "henti", "umum"), L, replace = TRUE, prob = c(0.58, 0.32, 0.10))
  kata <- character(L)
  for (i in seq_len(L)) {
    kata[i] <- switch(jenis[i],
                      topik = { tp <- sample(1:3, 1, prob = th); sample(kata_topik[[tp]], 1, prob = 6:1) },
                      henti = sample(kata_henti, 1),
                      umum = sample(kata_umum, 1))
  }
  paste(kata, collapse = " ")
}
teks <- apply(campuran, 1, buat_dok)
efek <- sample(c(-1, 0, 1)) * runif(3, 0.7, 1.2)            # efek tiap topik terhadap sentimen/rating
laten <- as.numeric(scale(campuran %*% efek))
sentimen <- 50 + 10 * (0.85 * laten + rnorm(n_dok, 0, 0.55))
rating <- 30 + 6 * (0.75 * laten + rnorm(n_dok, 0, 0.6))
rating <- if (30 <= 5) pmin(pmax(round(rating), 1), 5) else round(rating, 1)   # bintang 1-5 bila skala kecil
d <- data.frame(farmer_consultation_comments = sprintf("D%03d", seq_len(n_dok)), teks = teks,
                comment_sentiment_score = round(sentimen, 2), number_of_replies = rating)
write.csv(d, "data_contoh.csv", row.names = FALSE)

# ---- Pra-proses: tokenisasi, buang kata henti, frekuensi kata ----
tok <- tidytext::unnest_tokens(data.frame(dokumen = d$farmer_consultation_comments, teks = d$teks), kata, teks)
tok <- dplyr::filter(tok, !kata %in% kata_henti)
frek <- dplyr::count(tok, kata, sort = TRUE)
print(head(frek, 15))

# ---- Document-term matrix dan LDA (k = 3, Gibbs sampling, alpha tetap 0.3 untuk dokumen pendek) ----
hit <- dplyr::count(tok, dokumen, kata)
dtm <- tidytext::cast_dtm(hit, dokumen, kata, n)
print(dtm)
lda <- topicmodels::LDA(dtm, k = 3, method = "Gibbs",
                        control = list(seed = sample.int(100000, 1), alpha = 0.3, delta = 0.1, burnin = 300, iter = 1000))
post <- topicmodels::posterior(lda)
beta <- post$terms                                   # p(kata | topik)
gam <- post$topics                                   # proporsi topik per dokumen
perpleks <- topicmodels::perplexity(lda, dtm)
print(topicmodels::terms(lda, 6))
kata_atas <- apply(beta, 1, function(b) colnames(beta)[which.max(b)])
label_topik <- sprintf("T%d: %s", 1:3, kata_atas)

# ---- Topik dominan per dokumen dan hubungannya dengan rating ----
dg <- data.frame(dokumen = rownames(gam), gam, check.names = FALSE)
names(dg)[2:4] <- label_topik
dg <- merge(d, dg, by.x = "farmer_consultation_comments", by.y = "dokumen")
dg$topik <- factor(label_topik[max.col(as.matrix(dg[label_topik]), ties.method = "first")], levels = label_topik)
print(aggregate(cbind(comment_sentiment_score, number_of_replies) ~ topik, data = dg, FUN = mean))
kw <- kruskal.test(number_of_replies ~ topik, data = dg)
print(kw)
rerata_rating <- tapply(dg$number_of_replies, dg$topik, mean)
rerata_rating <- rerata_rating[!is.na(rerata_rating)]
r_sr <- cor(dg$comment_sentiment_score, dg$number_of_replies, method = "spearman")


# ---- Grafik ----
warna_topik <- setNames(pal[1:3], label_topik)
bt <- do.call(rbind, lapply(1:3, function(j) {
  o <- order(beta[j, ], decreasing = TRUE)[1:6]
  data.frame(topik = label_topik[j], kata = colnames(beta)[o], beta = beta[j, o])
}))
bt$topik <- factor(bt$topik, levels = label_topik)
p1 <- ggplot(bt, aes(beta, tidytext::reorder_within(kata, beta, topik), fill = topik)) +
  geom_col(alpha = 0.9, width = 0.75, show.legend = FALSE) +
  tidytext::scale_y_reordered() +
  facet_wrap(~topik, scales = "free_y", nrow = 1) +
  scale_fill_manual(values = warna_topik) +
  scale_x_continuous(breaks = scales::breaks_pretty(3)) +
  labs(title = "Top terms per topic (beta)", x = "Beta = P(term | topic)", y = NULL)
p2 <- ggplot(dg, aes(topik, number_of_replies, colour = topik)) +
  geom_boxplot(outlier.shape = NA, fill = NA, width = 0.55, linewidth = 0.6) +
  geom_jitter(width = 0.18, height = if (30 <= 5) 0.15 else 0, alpha = 0.3, size = 1) +
  stat_summary(fun = mean, geom = "point", shape = 18, size = 3.6, colour = warna$teks) +
  scale_colour_manual(values = warna_topik, guide = "none") +
  labs(title = if (kw$p.value < 0.001) "Rating by dominant topic (KW p < 0.001)" else
                 sprintf("Rating by dominant topic (KW p = %.3f)", kw$p.value),
       x = "Dominant topic", y = "number_of_replies")
dg$bin <- if (length(unique(dg$number_of_replies)) <= 7) dg$number_of_replies else
  ave(dg$number_of_replies, cut(dg$number_of_replies, unique(quantile(dg$number_of_replies, seq(0, 1, 0.2))), include.lowest = TRUE))
pr <- do.call(rbind, lapply(label_topik, function(tj) {
  a <- aggregate(dg[[tj]], by = list(bin = dg$bin), FUN = mean)
  data.frame(bin = a$bin, prop = a$x, topik = tj)
}))
pr$topik <- factor(pr$topik, levels = label_topik)
p3 <- ggplot(pr, aes(bin, prop, colour = topik)) +
  geom_line(linewidth = 1) +
  geom_point(size = 2) +
  scale_colour_manual(values = warna_topik, name = NULL) +
  scale_y_continuous(labels = scales::percent) +
  labs(title = "Mean topic share across rating levels", x = "number_of_replies", y = "Mean topic proportion")
p_hasil <- if (tata == 1) (p1 / (p2 | p3)) + plot_layout(heights = c(1, 1.15)) else
  ((p2 | p3) / p1) + plot_layout(heights = c(1.15, 1))

print(p_hasil)

The project file you can download

The original project file is available. It is not just the picture above: you get the working R project. It can be downloaded after payment, and we send it by WhatsApp or email.

ZIP: analisis.R + data_contoh.csv + BACA-DULU.txt

What is inside the ZIP

The complete analysis script (analisis.R), commented step by step, ready to run in RStudio.
A sample dataset (data_contoh.csv) so the script runs straight away.
A short guide (BACA-DULU.txt): which packages to install, how to run it and how to swap in your own data.

Packages and prices

Choose how much help you need. Each higher package includes everything in the lower ones. Prices exclude 11% VAT.

Package 1

Code and Explanation

US$31

excl. 11% VAT

Duration: 7 working days

Output: R files + result figures

Deliverables

Fully commented R code (.R) for Text Mining and LDA Topic Modelling
Example data and step-by-step run instructions
Ready-to-use result figures and metrics table
Short explanation of the method, its assumptions and how to read the results

Choose Package 1

Package 2 · most chosen

Your Data Analysis + Interpretation

US$88

excl. 11% VAT

Duration: 13 working days

Output: Code + analysis + interpretation notes

Includes all of Package 1, plus

Adapting the code to your own research dataset (CSV or Excel)
Assumption checks, result validation and data cleaning
Tables and figures formatted for your manuscript
Interpretation notes for the methods and discussion sections, for you to write up in your own words

Choose Package 2

Package 3

Full Guidance

US$191

excl. 11% VAT

Duration: 18-21 working days

Output: Code + analysis + guidance until ready to present

Includes all of Packages 1 & 2, plus

Comparison against one alternative method with the reasoning for the choice
Two 60-minute online coaching sessions so you fully understand the code and results
Revisions following your supervisor feedback (up to 3 rounds)
Practice Q&A to prepare for your seminar and defence
Full documentation (README, R package list (sessionInfo)) so the results can be reproduced

Choose Package 3

Payment: 50% down payment before work starts, balance after the deliverables are handed over
Tax: Prices exclude 11% VAT
Data and academic integrity: Results are computed from your own research data; you write and stand behind your manuscript, we help with the analysis and your understanding of it
Revisions: Up to 2 revisions are included in the price (Package 3: 3 rounds)

Frequently asked questions

Do I get the real project file?

Yes. The original project file is a ZIP containing the complete analisis.R script, a data_contoh.csv sample dataset and a BACA-DULU.txt guide. It can be downloaded after payment, and we send it by WhatsApp or email. For your own data (Package 2 and 3) the script is adapted to your dataset.

Is this suitable for a Doctoral dissertation (S3)?

Yes. Text Mining and LDA Topic Modelling is commonly used at this level in Agriculture and Plantations. We adapt the analysis to your research question, your own data and your supervisor's requirements.

Is the result sheet real data?

No. The sheet is computed from illustrative synthetic data so you can see the kind of output R Studio gives. For your order we run the same method on your own data.

Will I understand the code?

That is the point of the packages. Every script is commented, and Package 3 includes two coaching sessions plus practice questions so you can explain the method and the results at your seminar and defence.

How is the price decided?

Each project has three packages: Package 1 (code and explanation) US$31, Package 2 (your data analysis and interpretation) US$88 and Package 3 (full guidance) US$191, excluding 11% VAT. Larger datasets, extra methods or tight deadlines adjust the quote after a short consultation.

Similar projects

See all

PreviousMeta-analysis of plyometric training effects on vertical jump height in young athletes

NextLong-run relation of happiness index, income per capita and unemployment using VECM

The result sheet uses illustrative synthetic data. R is a programming language of the R Foundation project; RStudio and Posit are trademarks of Posit PBC.

Back to R Studio portfolio

Memuat