Data Mining and Knowledge Engineering

Faran Kazi ¹, Shraddha Joshi ¹, Sahista Machchhar ¹, Krupa Mandaviya ¹

Affiliations
1 Department of Computer Engineering, MEF Group of Institutions, Rajkot-360003, IN

Abstract

Social network, online forums, blogs and various sites where people can hold conversation in the form of messages, is recently become a more valuable resource for mining in various fields like customer relationship management, public opinion tracking and other text mining entities. The knowledge obtained from these public forums is extremely valuable for marketing research companies. This paper also include information of various tools available for crawling data from online forums, sites or blogs. In first part emotional polarity for extracted text is obtained using python script. In second part combined approach using EM (Expectation Maximization) clustering and SVM classification algorithm is applied to detect weather given forum is hotspot or non-hotspot for given time window. This novel approach gives better result than previous approaches to detect hotspot forums. Among them EM gives better result instead of K-means Clustering Technique.

Keywords

SVM (Support Vector Machine), EM (Expectation Maximization), Sentiment Analysis, Hotspot Detection, K-Means.

Full Text

     

Priyanka D. Vaghasiya ¹, Sahista Machchhar ², Komal S. Sahedani ³

Affiliations
1 Department of Computer Engineering, Marwadi Education Foundation, Rajkot, IN
2 Marwadi Education Foundation, Rajkot, IN
3 C.E. Department, R.K. University, Rajkot, IN

Abstract

Modern years are taking great interest in knowing failure of an organization and also about the factors responsible for the failure of an institutes/organizations. Thus, organizations should take appropriate steps to improve the services given by them and also satisfies their employees. In educational organizations, students and teachers are most valuable assets. Results of students affects much more to this organizations. Hence, organizations are aiming to find risks factors behind the failure of students. So, in order to identify the risk factors various data mining techniques are applied and required result is formed. In this work, Experiments attempts to improve accuracy for predicting which student might fail or take drop out, using all available attributes, selecting best attributes using attribute selection methods available in WEKA and then sampling the data for experimentation and finally regression method is applied on whole data set. The outcomes are compared and modeled with the best result is shown.

Keywords

Educational Data Mining, Attribute Selection Methods Preprocessing, Stratified Sampling, Logistic Regression, Prediction.

Full Text

     

Priyanka D. Vaghasiya ¹, Sahista Machchhar ¹

Affiliations
1 Department of Computer Engineering, Marwadi Education Foundation, Rajkot, IN

Abstract

In recent year, huge amount of data related to education and particularly of students are stored in Database. To deal with these much data research communities showing greatest interest in data mining in Educational field. The goal of any educational organization is to increase the academic performance of students and ultimately progress of institute. To extract knowledge hidden within the dataset by means of analytical method is not easy. So the data mining techniques helps to transform knowledge into some human understable form. This paper provides literature available for educational data mining, what are the attributes which may affect the student's performance, how attribute selection methods are useful to select best attribute from available attributes, different data mining techniques used to know student's academic performance.

Keywords

Attribute Selection Methods, Data Mining, Educational Data Mining, Prediction.

Full Text

     

Triveedee Sandhya ¹, Sahista Machchhar ¹

Affiliations
1 Department of Computer Engineering, Marwadi Education Foundation Group of Institutes, Rajkot, IN

Abstract

Text summarization is the task of reducing a text document with help of a software in order to generate a summary that retains the most important parts of the original document. It is not simple for individuals to manually summarize expansive reports of content information, Because of the a lot of content information being produced and associations expanded quickly with the accessibility of Big Data platforms, there is no enough time to peruse and perceive all document and make judgments based on text stuffing. Therefore, there is a great demand for summarizing text documents to deliver a demonstrative substitute for the novel documents. There are two techniques to summarize a text document 1) extractive summarization and 2) abstractive summarization. It is not easy task for people to manually summarize large documents of text data. An extractive summarization technique Chooses vital sentence, content etc. from the original text document and joined them into shorter structure to form outline. An abstractive summarization technique comprehend the original text data and retelling it in fewer words to generate outline. In this survey paper, we exhibit an overview and Comparative Analysis of Unstructured Text Summarization Approach in the Age of Big Data.

Keywords

Text Mining, Text Summarization, Information Retrieval, Big Data Platform.

Full Text

     

Apeksha Charola ¹, Sahista Machchhar ²

Affiliations
1 Department of Computer Engineering, Marwadi Education Foundation Group of Institute, Rajkot, IN
2 Department of Computer Engineering, Marwadi Education Foundation Group of Institutions, Rajkot, IN

Abstract

This paper presents a novel technique by combining regular clustering techniques with information extracted from WordNet. There are two approaches for traditional clustering algorithms utilize in documents clustering area. First approach work with documents as bag of words and consider each term as independent (means ignore semantic relationships between words). Second approach can determine semantics using WordNet. The proposed technique isutilizing second approach with different (identity, synonym, direct hypernym and meronym relation) & weighted (identity>synonym>direct hypernym>meronym) semantic relation. Concepts are weighted by generating concepts chain of related concepts. It utilizes the WordNet in turn to create low dimensional vector space which allows to build an efficient clustering technique. The proposed technique can improve cluster quality as well as achieve low dimensional vector space compared to other techniques.

Keywords

Document Clustering, K-Means Algorithm, WordNet, Concept Weighting, Synonym, Hypernym, Meronym.

Full Text

     

Anil Pandya ¹, Sahista Machchhar ², Glory Shah ²

Affiliations
1 Marwadi Education Foundation Group of Institute, Gujarat Technological University, Ahmedabad, Gujarat, IN
2 Department of Computer Engineering, Marwadi Education Foundation Group of Institutions, Rajkot, IN

Abstract

Currently, organizations are distributed geographically. Normally, all the sites locally store its day-to-day data, which is being updated. Centralized data mining algorithms can’t be used in such type of organizations for discovering useful patterns as merging of datasets from different sites is not feasible as well as it causes large network communication costs. Data mining in distributed form has emerged as an active sub-domain of data mining research. In distributed association rule mining algorithm, one of the major challenges is to reduce the communication overhead. Data sites are required to exchange lot of information in the data mining process which may generates communication overhead. This report proposes an association rule mining algorithm which minimizes the communication overhead among the participating data sites. Instead of transmitting all itemsets and their counts, The algorithm transmits a binary vector of frequently large itemsets using Message Passing Interface (MPI) technique. Another challenge is to reduce number of database scan and generate the frequent itemsets from the database. Hence an algorithm term as "Efficient Distributed dynamic itemset counting" is proposed. This algorithm reduces the time of scan of partition database which increases the performance of the algorithm.

Keywords

Association Rules, Distributed Environment, Minimum Communication Cost, Dynamic Itemset Counting, Frequent Pattern Growth, Support and Confidence.

Full Text

     

Apeksha Charola ¹, Sahista Machchhar ²

Affiliations
1 Marwadi Education Foundation Group of Institute, Gujarat Technological University, Ahmedabad, Gujarat, IN
2 Department of Computer Engineering, Marwadi Education Foundation Group of Institutions, Rajkot, IN

Abstract

With the problem of increased utilization of internet and the huge amount of text documents,the necessity of having efficient document clustering technology. In the field of Text mining, most work going on information retrieval and document summarization, very lessattention in domain of document clustering. Traditional text mining represents document as bag of words which has some limitation, as this method does not consider semantic relationship among the texts. Semantic text mining can overcome this limitation-using Ontology. Using ontology document represents as vector of weighted concepts. In this paper, present two type of survey. First is, Survey on pre-clustering approach. Second is, Documents clustering techniques.

Keywords

Document Clustering, Ontology, Pre-Clustering, Semantic, Text Mining, Weighted Concepts.

Full Text

     

Sahista Machchhar ¹, Madhuri Vaghasia ¹, Chintan Bhatt ²

Affiliations
1 Marwadi Education Foundation, Rajkot, Gujarat, IN
2 U & P U Patel Department of Computer Engineering, Charotar University of Science and Technology, Changa, Gujarat, IN

Abstract

Clustering of inherent sequential natured data sets is useful for various purposes. Over the years, many methods have been developed for clustering objects having sequential nature according to their similarity. However, these methods tend to have a computational complexity that is at least quadratic on the number of sequences. Also, clustering algorithms often require that the entire dataset be kept in the computer memory. In this paper, we present novel algorithm for Mining of constraint based clustered sequential patterns (CBCSP) algorithm for clustering only user interesting sequential data using recency, monetary and compactness constraints. So, the algorithm generates a compact set of clusters of sequential patterns according to user interest by applying constraints in mining process. It minimizes the I/O cost involved. The proposed algorithm basically applies the well known K-means clustering algorithm along with Prefix-Projected Database construction to the set of sequential patterns. In this approach, the method first performs clustering based on a novel similarity function and then captures the sequential patterns of which are only user interesting in each cluster using a sequential pattern mining algorithm which employs pattern growth method not. The proposed work results in reduced search space as user intended sequential patterns tend to be discovered in the resulting list. Through experimental evaluation under various simulated conditions, the proposed method is shown to deliver excellent performance and leads to reasonably good clusters.

Keywords

Data Clustering, Projected Database, Sequential Patterns, K-Means.

Full Text

     

Sahista Machchhar ¹, C. K. Bhensdadia ², A. M. Ganatra ³

Affiliations
1 Dharamsinh Desai University, Nadiad, Gujarat, IN
2 Dharamsinh Desai Institute of Technology, Professor and Head of Department, IN
3 CHARUSET University, IN

Abstract

As database resources get complex and bulky it not only becomes difficult to access specific information but also to extract relevant information from them. A way to address this issue is through sequential pattern mining technique. Sequential pattern mining is new trend in the domain of data mining and has many useful and exciting applications. In the sequential pattern mining approach, we mainly deal with attempting to discover a pattern that is sequential in nature. This helps us to predicting next event after a sequence or sequence-of-event(s). The success of such techniques lies in the design of their algorithm. Today, there are several competitive and efficient algorithms that cope with the popular and computationally expensive task of sequential pattern mining. Actually, these algorithms are more or less described on their own. This paper mainly focuses on the need, merits and demerits of different sequential rule mining algorithms and categorizing them according to their mining method, search method adopted, database formatting employed and other constraints as applied to the database. The basic inspiration to undertake this study is to provide a single platform-of-information that will serve as a ready reference for both the researchers and practitioners interested in the designing and implementation of sequential pattern mining algorithms depending upon categorized databases.

Keywords

Sequential Pattern Mining, Database Formatting, Mining with Constraints, Pattern-Growth Method.

Full Text