Mastering Text Mining with R

Author: Ashish Kumar;Avinash Paul  

Publisher: Packt Publishing‎

Publication year: 2016

E-ISBN: 9781782174707

P-ISBN(Paperback): 9781783551811

Subject: TP39 computer application

Keyword: 计算机的应用,自动化技术、计算机技术,计算技术、计算机技术

Language: ENG

Access to resources Favorite

Disclaimer: Any content in publications that violate the sovereignty, the constitution or regulations of the PRC is not accepted or approved by CNPIEC.

Description

Master text-taming techniques and build effective text-processing applications with R About This Book • Develop all the relevant skills for building text-mining apps with R with this easy-to-follow guide • Gain in-depth understanding of the text mining process with lucid implementation in the R language • Example-rich guide that lets you gain high-quality information from text data Who This Book Is For If you are an R programmer, analyst, or data scientist who wants to gain experience in performing text data mining and analytics with R, then this book is for you. Exposure to working with statistical methods and language processing would be helpful. What You Will Learn • Get acquainted with some of the highly efficient R packages such as OpenNLP and RWeka to perform various steps in the text mining process • Access and manipulate data from different sources such as JSON and HTTP • Process text using regular expressions • Get to know the different approaches of tagging texts, such as POS tagging, to get started with text analysis • Explore different dimensionality reduction techniques, such as Principal Component Analysis (PCA), and understand its implementation in R • Discover the underlying themes or topics that are present in an unstructured collection of documents, using common topic models such as Latent Dirichlet Allocation (LDA) • Build a baseline sentence completing application • Perform entity extraction and named entity recognition using R In Detail Text Mini

Chapter

Chapter 1: Statistical Linguistics with R

Probability theory and basic statistics

Probability space and event

Theorem of compound probabilities

Conditional probability

Bayes' formula for conditional probability

Independent events

Random variables

Discrete random variables

Continuous random variables

Probability frequency function

Probability distributions using R

Cumulative distribution function

Joint distribution

Binomial distribution

Poisson distribution

Counting occurrences

Zipf's law

Heaps' law

Lexical richness

Lexical variation

Lexical density

Lexical originality

Lexical sophistication

Language models

N-gram models

Markov assumption

Hidden Markov models

Quantitative methods in linguistics

Document term matrix

Inverse document frequency

Words similarity and edit-distance functions

Euclidean distance

Cosine similarity

Levenshtein distance

Damerau-Levenshtein distance

Hamming distance

Gunning frog index

R packages for text mining

OpenNLP

Rweka

RcmdrPlugin.temis

tm

languageR

koRpus

RKEA

maxent

lsa

Summary

Chapter 2: Processing Text

Accessing text from diverse sources

File system

PDF documents

Microsoft Word documents

HTML

XML

JSON

HTTP

Databases

Processing text using regular expressions

Tokenization and segmentation

Word tokenization

Operations on a document-term matrix

Sentence segmentation

Normalizing texts

Lemmatization and stemming

Stemming

Lemmatization

Synonyms

Lexical diversity

Analyse lexical diversity

Calculate lexical diversity

Readability

Automated readability index

Language detection

Summary

Chapter 3: Categorizing and Tagging Text

Parts of speech tagging

POS tagging with R packages

Hidden Markov Models for POS tagging

Basic definitions and notations

Implementing HMMs

OpenNLP chunking

Chunk tags

Collocation and contingency tables

Extracting co-occurrences

Surface Co-occurrence

Textual co-occurrence

Syntactic co-occurrence

Co-occurrence in a document

Detailed analysis on textual collocations

Feature extraction

Synonymy and similarity

Multiwords, negation, and antonymy

Concept similarity

Path length

Resnik similarity

Lin similarity

Jiang – Conrath distance

Summary

Chapter 4: Dimensionality Reduction

The curse of dimensionality

Distance concentration and computational infeasibility

Dimensionality reduction

Principal component analysis

Using R for PCA

Understanding the FactoMineR package

Amap package

Proportion of variance

Scree plot

Reconstruction error

Correspondence analysis

Canonical correspondence analysis

Multiple correspondence analysis

Implementation of SVD using R

Summary

Chapter 5: Text Summarization and Clustering

Topic modeling

Latent Dirichlet Allocation

Correlated topic model

Model selection

R Package for topic modeling

Latent semantic analysis

R Package for latent semantic analysis

Illustrative example of LSA

Text clustering

Document clustering

Feature selection for text clustering

Mutual information

Statistic Chi Square feature selection

Frequency-based feature selection

Sentence completion

Summary

Chapter 6: Text Classification

Text classification

Document representation

Feature hashing

Classifiers – inductive learning

Tree-based learning

Bayesian classifiers: Naive Bayes classification

K-Nearest neighbors

Kernel methods

Support vector machines

Kernel Trick

How to apply SVM on a real world example?

Number of instances is significantly larger than the number of dimensions.Maximum entropy classifier

Maxent implemenation in R

RTextTools: a text classification framework

Model evaluation

Confusion matrix

ROC curve

Precision-recall

Bias–variance trade-off and learning curve

Bias-variance decomposition

Learning curve

Dealing with reducible error components

Cross validation

Leave-one-out

k-Fold

Bootstrap

Stratified

Summary

Chapter 7: Entity Recognition

Entity extraction

The rule-based approach

Machine learning

Sentence boundary detection

Word token annotator

Named entity recognition

Training a model with new features

Summary

Index

The users who browse this book also browse