Data Mining And Anomaly Detection Techniques

Data anomalies normally arise when a database is poorly designed or has a redundancy in the data. This often occurs if there is an inconsistency of more than one data appearance in several areas of the database. Below shows an example of a data anomaly from my lab project. As you can see, Mary Smith is enrolled in more than one classes on a different date and her student records are reproduced more than ones.

Builds topologies map (Every node knows how to reach to its directly connected neighbors and by making sure that the total of this acknowledge is distributed to every node then every node will have enough information to build a complete map of the network).

There were some anomalies found in the data but which were not included in the final results. These anomalies included negative velocities and numbers that were much larger than the ones before.

Step 1: Construct a network diagram for the project. (NOTE: EF for activity H should be 19)

nodes in a data structure, determined by node type or class. The advantage of this pattern

System Driven Anomaly: In this case, the deviation is due to the limitations that are inherent in the system. In this case, Red Planet would, after thorough study, inform Cecil County about the same and present the possible impacts of the anomaly. This would be followed by a detailed discussion with Cecil County to find a workaround to meet the stated requirement.

In order to observe whether there exists a data point that particularly differs from other data, we use boxplot to analyze the outliers:

Anomaly-based detection method is the process of comparing definitions of what activity is normal against observed events to identify substantial

Data Analyzing: Analysis of data includes the removal of the outliers. These are the experimental error which will leads to unsatisfactory results. It will deviate the actual result from the expected one so they must be

Before a data set can be mined, it first has to be ?cleaned?. This cleaning process removes errors, ensures consistency and takes missing values into account. Next, computer algorithms are used to ?mine? the clean data looking for unusual patterns. Finally, the patterns are interpreted to produce new knowledge.3

Both host and network based intrusions could be detected to some extent using an Anomaly based IDS. In this, a

Passive network monitoring uses some devices to inspect the traffics as they pass by. An effective algorithm proposed by Binkley and Singh combines TCP-based anomaly

The relationships and summaries derived are referred to as models or patterns. Examples include linear equations, rules, clusters, graphs, tree structures and recurrent patterns in time series.

The use of objective criteria enables automated classification of the nodes. Although in defining the shared features and the distinctive ones between the nodes a high degree of flexibility is permitted, unequivocal results require precision. Fuzzy similarities or dissimilarities between the nodes may still be captured with advanced artificial intelligence (AI) tools \cite{c8}.

Detecting outlier is one of the important tasks. A failure to detect outliers or their ineffective handling can have serious ramifications on the strength of the inferences drained from the exercise [4]. Outlier detection has direct applications in a wide variety of domains such as mining for anomalies to detect network intrusions, fraud detection in mobile phone industry and recently for detecting terrorism related activities [5].Outliers are found using the filters which is offered by data mining tools. Liver disorder is also referred to as