Data Mining Technology

Data mining tools are used to judge future trends and behaviors allowing businesses to make knowledge driven, proactive decisions. The data mining provides automated, prospective analyses move which leaves behind the analyses of past events provided by retrospective tools of decision support systems. Data mining tools can answer business questions that were time consuming to resolve. They search databases for hidden patterns and finding predictive information that experts may miss.

Data mining techniques can be implemented rapidly on existing hardware and software platforms to increase the value of existing information resources and it can be easily integrated with new products and systems.

How Data Mining Works
1) Preparing the Information:
Data has to be properly organized for effective information processing. In data mining, 70 percent to 80 percent of the time is spent on sorting and summarizing the information before mining efforts actually start. Data is prepared on the basis of desired information objectives.

2) Modeling:
Model is created depending upon many factors such as database size, number of known variables and which kind of data mining algorithms to be employed. Successful models help companies to study and effectively identify their target market. Data mining models are designed against massive transaction detail data warehouses of 10 TB to 20 TB. These models are often used as front end by intelligent segmentation capabilities to enable the derivation of actionable customer segments.

3) Scoring Customers:
The best way to access models viability is to test it against an existing data where the answer to a particular question is already known. Score is the output of a model, which is number between 0 and 1 as the probability of answer to a specific question.

4) Dynamic Scoring :
In this a scoring process is determined by another software application with aim to use this score for some other purposes. For up to date results and eliminating the need to score an entire database, only required record subsets are scored.

Scope of Data Mining
Data mining gains its name from the similarities between searching for valuable business information in a large database. Data mining finds exact information either sifting through an immense amount of material or intelligently probing it. Data mining technology can generate new business opportunities by providing these facilities:

1) Automated prediction of trends and behaviors :
Data mining automatically finds predictive information in large databases. The questions, which require extensive analysis can now be answered directly from the data quickly. Predictive problem is a typical example of targeted marketing. Data mining uses data on past mailings to identify the targets and provide maximize return on investment in future mailings.

2) Automated discovery of previously unknown patterns :
Data mining tools search databases and identify previously hidden patterns in one step. Pattern discovery is example of the analysis of retail sales data to identify seemingly unrelated products that are often purchased together.

Data mining tools can analyze massive databases in minutes if these tools are implemented on high performance parallel processing systems. Faster processing allows users to automatically experiment with more models to understand complex data.

Databases can be larger in both depth and width:
a)More columns: Analysts must provide limited the number of variables during analysis due to time constraints. Data mining allows users to explore the full depth of a database without selecting a subset of variables.

b) More rows: Larger samples cause lower estimation errors, variance and allow users to make inferences about small, important segments of a population.
The most commonly used techniques in data mining are:
i)Artificial neural networks: it provides non-linear predictive models that learn through training and resemble biological neural networks in structure.
ii)Decision trees: It is tree-shaped structures which represent sets of decisions. This decision tree generates rules for the classification of a dataset. Some decision tree methods include Classification and Regression Trees (CART).
iii)Genetic algorithms: It provides optimization techniques that use process such as genetic combination, mutation, and natural selection in a design based on the concepts of evolution.
iv)Rule induction: Useful if-then rules are based on statistical significance.

Architecture for Data Mining
Data mining is fully integrated with a data warehouse and flexible interactive business analysis tools. Many data mining tools currently work outside the data warehouse and it requires extra steps for importing, extracting and analyzing the data. While new approach requires operational implementation, integration with the warehouse simplifies the application from data mining. The resulting analytic data warehouse can be useful to improve business processes throughout the organization in promotional campaign management, new product rollout and fraud detection.

Data warehouse is the ideal starting point containing a combination of internal data tracking all customer contact coupled with external market data about competitor activity. For prospecting background information on potential customers provides an excellent basis. This warehouse can be implemented in Sybase, Oracle, Redbrick, and so on, and should be optimized for flexible and fast data access.

When navigating the data warehouse an OLAP (On-Line Analytical Processing) server enables a more sophisticated end-user business model. These multidimensional structures allow the user to analyze data as per their view. The Data Mining Server is fully integrated with the data warehouse and OLAP server to embed ROI focused business analysis. Metadata template which is process-centric defines the data mining objectives for specific business issues like campaign management, prospecting and promotion optimization. Integration with the data warehouse allows operational decisions to be directly implemented and tracked. With new decisions and results the warehouse grows and the organization can continually mine the best practices and apply them to future decisions.

Applications
Data mining application areas are:
1)A pharmaceutical company can examine its recent sales force activity and their results to improve which marketing activities will have the greatest impact in the next few months. These data needs to include information about the local health care systems as well as competitor market activity. By using a wide-area network the results can be distributed to the sales force that enables the representatives to review the recommendations from the perspective of the key attributes in the decision process. This dynamic analysis of the data warehouse allows best practices from throughout the organization to be applied in specific sales situations.
2)A credit card company can force its vast warehouse of customer transaction data to identify customers to be interested in a new credit product. Using a small test mailing can identify the attributes of customers with an affinity for the product.
3)A large consumer package goods company applies data mining to improve its sales process to its retailers. Data collected from consumer panels, shipments, and competitor activity can be used to determine the reasons for brand and store switching. By using this analysis, the manufacturer select best strategies that reach their target customer segments.