Web Usage Mining Algorithms

Data preprocessing is considered as an important phase of Web usage mining due to unstructured, heterogeneous and noisy nature of log data. Complete and e ective data pre- processing insures the eciency and scalability of algorithms used in pattern discovery phase of Web usage mining. Data preprocessing generally includes the steps- Data fusion, Data cleaning, User identi cation, Session identi cation, Path com- pletion etc. Data cleaning is the initial and important step in preprocessing to extract cleaned data for further process- ing. It is important to apply data extraction before data cleaning on raw log data in analysis of speci c time-duration i.e. one day, one week or one month etc. In this paper we have mainly focused on data fusion, data extraction and data cleaning steps of preprocessing and proposed an algo- rithm for data extraction which extracts log data according to analysis of time duration. This algorithm also sorts log entries according to their date and time which will be further used in prediction of browsing sequence of user. After that we have applied data cleaning algorithm on extracted real Web server log. In data cleaning almost all irrelevant les, irrelevant HTTP methods and wrong HTTP status codes are considered and after experiment it is analyzed that raw log data reduces to almost 80% which shows the importance of initial phases of data preprocessing.

Internet is the boon in modern era as every organization uses it for dissemination of information and e-commerce related applications. Sometimes people of organization feel delay while accessing internet in spite of proper bandwidth. Prediction model of web caching and prefetching is an ideal solution of this delay problem. Prediction model analysing history of internet user from server raw log files and determine future sequence of web objects and placed all web objects to nearer to the user so access latency could be reduced to some extent and problem of delay is to be solved. To determine sequence of future web objects, it is necessary to determine proximity of one web object with other by identifying proper distance metric technique related to web caching and prefetching. This paper studies different distance metric techniques and concludes that bio informatics based distance metric techniques are ideal in context to Web Caching and Web Prefetching.

RapidMiner is a software for machine learning, data mining, predictive analytics, and business analytics. The server will record large web log files when user visits the website. Extracting knowledge from such huge data demands for new methods. In this paper, we propose a web usage mining method with RapidMiner. At first, the redundant files in log file are deleted by Matlab and then we mines web log which has been pretreated with RapidMiner, it obtains the custom of different user to visit the website by processing and analyzing log file, and mines unusual rules, and provides the reference for the policy decision and construction of website. Experimental result analysis show that, applying RapidMiner in web usage mining, will obtain frequent model which user visits the website, manage to optimize the website structure and recommends for users.

The internet has progressed with a wide range of information over last few eras. This also brings the issue of information flooding and may cause the interruption to the user when accessing pertinent information. So it becomes a challenging task for the webmasters to provide the search options according to the individual user. Therefore, Web usage mining is an efficient technique where the helpful information is extracted from the user's history stored in the server logs and clusters those logs into a variety of labels. Hence the proposed system used the CHAMELEON clustering algorithm which processes the search logs of the user and cluster those logs into separate clusters based on the search category. Finally the chameleon algorithm is compared with other data clustering algorithms in terms of the Sum of Score Squared Error (SSE), rand index and F-measure respectively.

Le Web Usage Mining est l’application des techniques du datamining aux données d’usage du web dans le but de découvrir les réelles attentes des utilisateurs d’un site web. Ce dernier est de plus en plus vu comme une source de fort revenu pour l’annonceur et donc pour le propriétaire du site qui
dispose d’un large champ d’action pour rentabiliser son site, tout en s’alignant aux comportements observés chez les visiteurs de son site web.

Ce travail se propose de fournir au webmaster un modèle statistique mettant en relief les variables qui
réaliseraient le mieux un objectif avancé par le propriétaire du site web. Le but est de guider un webmaster à agir correctement, non arbitrairement, pour atteindre son objectif, souvent lucratif.

A partir d’un échantillon donné de visites effectuées sur un site web étudié, nous avons construit un modèle de régression logistique, qui a pour ambition de modéliser la probabilité d’atteindre l’objectif souhaité en fonction de certaines variables décrivant la visite, le visiteur, et le site lui-même.

Dans un premier temps, le modèle a été choisi par une sélection pas à pas descendante. Ensuite, un algorithme de validation croisée a été appliqué pour décider du pouvoir prédictif du modèle. La lecture des résultats a permis d’énoncer quelques recommandations au webmaster.

Data preprocessing is considered as an important phase of
Web usage mining due to unstructured, heterogeneous and
noisy nature of log data. Complete and eective data pre-
processing insures the eciency and scalability of algorithms
used in pattern discovery phase of Web usage mining. Data
preprocessing generally includes the steps- Data fusion, Data
cleaning, User identication, Session identication, Path com-
pletion etc. Data cleaning is the initial and important step
in preprocessing to extract cleaned data for further process-
ing. It is important to apply data extraction before data
cleaning on raw log data in analysis of specic time-duration
i.e. one day, one week or one month etc. In this paper we
have mainly focused on data fusion, data extraction and
data cleaning steps of preprocessing and proposed an algo-
rithm for data extraction which extracts log data according
to analysis of time duration. This algorithm also sorts log
entries according to their date and time which will be further
used in prediction of browsing sequence of user. After that
we have applied data cleaning algorithm on extracted real
Web server log. In data cleaning almost all irrelevant les,
irrelevant HTTP methods and wrong HTTP status codes
are considered and after experiment it is analyzed that raw
log data reduces to almost 80% which shows the importance
of initial phases of data preprocessing.

Web is a wide, various and dynamic environment in which different users publish their documents. Web-mining is one of data mining applications in which web patterns are explored. Studies on web mining can be categorized into three classes: application mining, content mining and structure mining. Today, internet has found an increasing significance. Search engines are considered as an important tool to respond users' interactions. Among algorithms which is used to find pages desired by users is page rank algorithm which ranks pages based on users' interests. However, as being the most widely used algorithm by search engines including Google, this algorithm has proved its eligibility compared to similar algorithm, but considering growth speed of Internet and increase in using this technology, improving performance of this algorithm is considered as one of the web mining necessities. Current study emphasizes on Ant Colony algorithm and marks most visited links based on higher amount of pheromone. Results of the proposed algorithm indicate high accuracy of this method compared to previous methods. Ant Colony Algorithm as one of the swarm intelligence algorithms inspired by social behavior of ants can be effective in modeling social behavior of web users. In addition, application mining and structure mining techniques can be used simultaneously to improve page ranking performance.

The one of the most time consuming steps for association rule mining is the computation of the frequency of the occurrences of itemsets in the database. The hash table index approach converts a transaction database to an hash index tree by scanning the transaction database only once. Whenever user requests for any Uniform Resource Locator (URL), the request entry is stored in the Log File of the server. This paper presents the hash index table structure, a general and dense structure which provides web page set extraction from Log File of server. This hash table provides information about the original database. Web Page set mining (WPs-Mine) provides a complete representation of the original database. This approach works well for both sparse and dense data distributions. Web page set mining supported by hash table index shows the performance always comparable with and often better than algorithms accessing data on flat files. Incremental update is feasible without reaccessing the original transactional database.

The one of the most time consuming steps for association rule mining is the computation of the frequency of the occurrences of itemsets in the database. The hash table index approach converts a transaction database to an hash index tree by scanning the transaction database only once. Whenever user requests for any Uniform Resource Locator (URL), the request entry is stored in the Log File of the server. This paper presents the hash index table structure, a general and dense structure which provides web page set extraction from Log File of server. This hash table provides information about the original database. Web Page set mining (WPs-Mine) provides a complete representation of the original database. This approach works well for both sparse and dense data distributions. Web page set mining supported by hash table index shows the performance always comparable with and often better than algorithms accessing data on flat files. Incremental update is feasible without reaccessing the original transactional database.