1. for future research. Published in: ISAI/IFIS 1996.

1. Selecting
training instances for supervised classification –


We Will Write a Custom Essay Specifically
For You For Only $13.90/page!

order now

Several experimental studies have tested the relative merits of
various supervised machine learning models. Comparisons have been made along
dimensions that include model complexity, prediction accuracy, training set
size, and training time. Only limited work has been done to study the effect of
training set exemplar typicality on model performance. We present experimental
results obtained in testing C4.5, SX-WEB, a backpropagation newal network and
linear discriminant analysis using a real-valued and a mixed form of a medical
data set. We generated training sets of highly typical, widely-varied and
atypical exemplars for both data sets. We tested the classification accuracy of
each model using the generated training sets. Test set accuracy levels ranged between
76% and 86% when each model was trained with typical or varied training sets.
The accuracy levels for C4.5, backpropagation neural net and discriminant
analysis dropped significantly when atypical training sets were used. In
contrast, with the exception of one test, SX-WEB was unaffected by training set
choice. When comparing the correctness of each model, SX WEB showed the best
overall performance. We conclude this paper with directions for future

Published in: ISAI/IFIS 1996. Mexico-USA Collaboration in Intelligent
Systems Technologies. Proceedings

Date of Conference: 12-15 Nov. 1996

Date Added to IEEE Xplore: 06 August 2002


2. Automatic
target recognition using higher order neural network –


Translational rotational scaling invariant (TRSI) pattern
recognition is an important problem in the automatic target recognition (ATR)
field. Recent research has shown that the higher order neural networks (HONN)
have numerous advantages over other neural network approaches in respect of the
object recognition with invariant of the object’s position size, and in-plane
rotation. The major limitation of HONNs is that the number of connected weights
is too large to store on most machines. For N/spl times/N image, the memory
needed to store the connections is proportional to N/sup 6/. This huge memory
requirement limits the HONN’s application to large scale images. In this paper,
we have developed an integrated method which combines the bi-directional
log-polar mapping and HONN pattern recognizer. It reduces the HONN memory
requirement from O(N/sup 6/) to O(N/sup 2/). The proposed method has been
successfully verified. Finally, the results are compared with those of
coarse-coding method, traditional log-polar method.

Published in: Aerospace and Electronics Conference, 1996. NAECON 1996.,
Proceedings of the IEEE 1996 National

Date of Conference: 20-22 May 1996

Date Added to IEEE Xplore: 06 August 2002


Query answering using discovered rules –


Research has been done in discovering rules from databases and
in applying these rules to intensional answers to database queries, semantic
query optimization, etc. However, rules discovered by one group of users may
not be used by other groups of users in their applications due to certain
mismatches. In this paper, we address the problems of using discovered rules
for query answering, and then propose algorithms for rewriting, applying and
maintaining these rules.

Published in: Data Engineering, 1996. Proceedings of the Twelfth International
Conference on

Date of Conference: 26 Feb.-1 March 1996

Date Added to IEEE Xplore: 06 August 2002

Print ISBN: 0-8186-7240-4

Print ISSN: 1063-6382

4. On
atypical database transactions: identification of probable frauds using machine
learning for user profiling –


The paper proposes a framework for deriving users’ profiles of
typical behaviour and detecting atypical transactions which may constitute
fraudulent events or simply a change in user’s behaviour. The anomaly detection
problem is presented and previous attempts to address it are discussed. The
proposed approach proves that individual user profiles can be constructed and
provides an algorithm that derives user profiles and an algorithm to identify
atypical transactions. Lower and upper bounds for the number of
misclassifications are also provided. An evaluation of this approach is
discussed and some issues for further research are outlined.

Published in: Knowledge and Data Engineering Exchange Workshop, 1997.

Date of Conference: 4-4 Nov. 1997

Date Added to IEEE Xplore: 06 August 2002

5. FUZZ:
a fuzzy-based concept formation system that integrates human categorization and
numerical clustering –


Recently, psychologists proposed the prototype theory of concept
representation, in which a concept is organized around a best example or
so-called prototype. Most proponents of the prototype theory conceive that
objects may fall in a concept to some degree rather than the all-or-none
membership in the classical theory. Fuzzy-set theory is compatible with the
basic premises of the prototype theory of concept representation. Concept
formation is defined as a machine learning task that captures concepts through
categorizing the observation of objects and also uses them in classifying
future experiences. A reasonable computational model of concept formation must
reflect the characteristics of human concept learning and categorization. In
this paper, the design and implementation of a fuzzy-set based concept
formation system (FUZZ) is presented. The main feature of the FUZZ is that the
concept hierarchy is nondisjoint, in which an instance may belong to two
categories in different memberships. An information-theoretic evaluation
measure called category-binding to direct-searches in the FUZZ is proposed. The
learning and classification algorithms of the FUZZ are also given. In order to
examine FUZZ’s behavior, the results of some experiments are examined.

Published in: IEEE Transactions on Systems, Man, and Cybernetics, Part B
(Cybernetics) ( Volume: 27, Issue: 1, Feb 1997 )

Page(s): 79 – 94

Date of Publication: Feb 1997 

6. Text
image restoration using cellular neural networks-


Optical character recognition (OCR) is a machine process that
recognizes writing symbols from an image and converts these symbols into a
machine readable form. In this paper it is proposed that a cellular neural
network (CNN) be trained to process distorted text and improve the accuracy of
an OCR processor. The results of this test will be compared to the results for
an ANN trained by back-propagation (BP) and an ANN trained by an extended
Kalman filter (EKF) by examining how this preprocessing affects the accuracy of
an OCR processor. The two proposed methods using ANN were successful but the
training times were long. Improvement of the distortion was distinct in both
cases. Continued research on both of these methods and new research on the use
of a CNN for this problem is continuing.

Published in: Circuits and Systems, 1997. ISCAS ’97., Proceedings of
1997 IEEE International Symposium on

Date of Conference: 12-12 June 1997

Date Added to IEEE Xplore: 06 August 2002

Print ISBN: 0-7803-3583-X

7. Factorial
HMMs for acoustic modeling


In the machine learning research field several extensions of
hidden Markov models (HMMs) have been proposed. In this paper we study their
possibilities and potential benefits for the field of acoustic modeling. We
describe preliminary experiments using an alternative modeling approach known
as factorial hidden Markov models (FHMMs). We present these models as
extensions of HMMs and detail a modification to the original formulation which
seems to allow a more natural fit to speech. We present experimental results on
the phonetically balanced TIMIT database comparing the performance of FHMMs
with HMMs. We also study alternative feature representations that might be more
suited to FHMMs.

Published in: Acoustics, Speech and Signal Processing, 1998. Proceedings
of the 1998 IEEE International Conference on

Date of Conference: 15-15 May 1998

Date Added to IEEE Xplore: 06 August 2002

Print ISBN: 0-7803-4428-6

Print ISSN: 1520-6149

8. On-line
tracking abilities of neural networks with graded responses


This paper analyzes the tracking performance of neural networks
with graded analog responses when the weights of a target network change slowly
with time. We first study the performance of a tracker consisting of a single
neuron and then discuss two-layer networks. The target network weights are
described by a stochastic difference equation with weights changing slowly with
time. The tracker weights follow the Least Mean Square (LMS) gradient descent
algorithm. We use a Gaussian error function: g(x)=1//spl radic/(2/spl pi/)/spl
int//sub -x//sup x/e(-t/sup 2//2)dt, as this activation function makes the
analysis tractable and the function closely approximates the standard sigmoidal
nonlinearity (g(x)=tanh(x)). For a weight drift of rate y and appropriately chosen
step size /spl mu/ the mean squared generalization error is proportional to
/spl gamma//sup 2///spl mu/. The paper then formulates an approach to analyzing
a two-layer soft committee machine and concludes by discussing extensions to
this research.

Published in: Circuits and Systems, 1998. ISCAS ’98. Proceedings of the
1998 IEEE International Symposium on

Date of Conference: 31 May-3 June 1998

Date Added to IEEE Xplore: 06 August 2002

Print ISBN: 0-7803-4455-3

9. A
physiological neuro fuzzy learning algorithm for medical image recognition-


In the recent trend of medical image recognition research, there
is a vigorous interest in applying artificial neural networks (ANNs) and fuzzy
theory beyond medical image recognition. However, we may face the problems of
oscillation at local minima, high price in training, misidentification to
degrade the efficiency of recognition etc. as well as inability to regulate
rules using respectively existing ANNs and fuzzy theory. The methods can be
fused into one in order to get over the problems. We propose a physiological
neuro fuzzy algorithm to improve the recognition rate of medical images. The
learning algorithm in the paper is proposed to capture two aspects of the
brain-its physiological neuronal structure and its function. That is, the
inhibition and excitation mechanism of the synapses found in physiological
studies is implemented with the cooperation of a neural network and fuzzy
logic. In order to evaluate the proposed algorithm, we applied them in
bronchogenic cancer cell recognition. The results of the simulation have shown
that the proposed algorithm is very effective for medical image recognition and
guarantee the convergence in the training phrase. The algorithm may contribute
to further elaborating studies such as medical expert systems, automatic
control, and machine vision in the real world.

Published in: Fuzzy Systems Conference Proceedings, 1999. FUZZ-IEEE ’99.
1999 IEEE International

Date of Conference: 22-25 Aug. 1999

Date Added to IEEE Xplore: 06 August 2002

10. A
problem of selecting optimal subset of fuzzy-valued features-


Feature subset selection refers to a data mining enhancement
technique which aims to reduce the number of features to be used. This
reduction is expected to improve the performance of data mining algorithms to
be used, in aspects of speed, accuracy and simplicity. Although there has been
some work on feature subset selection, research into the theoretically
computational complexity of this problem and on the optimal selection of
fuzzy-valued feature subsets has not been carried out. This paper focuses on a problem
called optimal fuzzy-valued feature subset selection (OFFSS) which is regarded
as being important but difficult in machine learning and pattern recognition.
The measure of the quality of a set of features is defined by the overall
overlapping degree between two classes of examples and the size of feature
subset. The main contributions of this paper are that: (1) the concept of fuzzy
extension matrix is introduced; (2) the computational complexity of OFFSS is
proved to be NP-hard; (3) a simple but powerful heuristic algorithm for OFFSS
is given; and (4) the feasibility and simplicity of the proposed algorithm are
demonstrated via applications of OFFSS to input selection of neuro-fuzzy
systems and to fuzzy decision tree induction.

Published in: Systems, Man, and Cybernetics, 1999. IEEE SMC ’99
Conference Proceedings. 1999 IEEE International Conference on

Date of Conference: 12-15 Oct. 1999

Date Added to IEEE Xplore: 06 August 2002

11. Multi-resolution
support vector machine


The support vector machine (SVM) is a new learning methodology
based on Vapnik-Chervonenkis (VC) theory (Vapnik, 1982, 1995). SVM has recently
attracted growing research interest due to its ability to learn classification
and regression tasks with high-dimensional data. The SVM formulation uses
kernel representation. The existing algorithm leaves the choice of the kernel
type and kernel parameters to the user. This paper describes an important
extension to the SVM method: the multiresolution SVM (M-SVM) in which several
kernels of different scales can be used simultaneously to approximate the
target function. The proposed M-SVM approach enables ‘automatic’ selection of
the ‘optimal’ kernel width. This usually results in better prediction accuracy
of SVM models.

Published in: Neural Networks, 1999. IJCNN ’99. International Joint
Conference on

Date of Conference: 10-16 July 1999

Date Added to IEEE Xplore: 06 August 2002

12. Performance
and efficiency: recent advances in supervised learning-


This paper reviews recent advances in supervised learning with a
focus on two most important issues: performance and efficiency. Performance
addresses the generalization capability of a learning machine on randomly
chosen samples that are not included in a training set. Efficiency deals with
the complexity of a learning machine in both space and time. As these two
issues are general to various learning machines and learning approaches, we
focus on a special type of adaptive learning systems with a neural
architecture. We discuss four types of learning approaches: training an
individual model; combinations of several well-trained models; combinations of
many weak models; and evolutionary computation of models. We explore advantages
and weaknesses of each approach and their interrelations, and we pose open
questions for possible future research.

Published in: Proceedings of the IEEE ( Volume: 87, Issue: 9, Sep 1999 )

Page(s): 1519 – 1535

Date of Publication: Sep 1999 

13. Towards
the use of problem knowledge in training neural networks for image processing


If a human can perform an image processing task then, given
sufficient time and determination an expert can often develop a machine vision
system to emulate this performance. The work reported here is part of a
long-term project that is aimed at the development of software systems that can
learn by example and emulate human performance. The non-algorithmic approach of
using a neural network based window filter (NNWF) has been used. General access
is provided to the results of this research via Adobe Plugin Protocol functions
available for use in image processing packages. This paper reports the use of
task specific knowledge to initialise the network weights prior to training.
Supervised neural network training is a high dimensional optimisation problem
and the initial conditions of the search are critical to the quality of the
solutions found (local or global optima) and the speed of convergence to the
solution. These initial conditions should be problem specific but standard
training methods, backpropagation, Levin-Marquadt etc. start with the initial
weights set to small, random numbers. The weight set of a trained network
embodies knowledge of the task; it is in some sense a description of the task.
The initial weight set is, by this reasoning, a partial description of the task
and could be derived from the available problem knowledge. It is often possible
to partially describe an image processing task as a set of rules. These rules
may be fuzzy in nature, incomplete and ambiguous but still provide a useful
guide for a human attempting the task. A mapping scheme has been developed that
can be used to map simple Boolean rules to a fuzzy neural network whose
architecture reflects the structure of IF THEN rules. The fuzzy neural network
(FuNN) architecture is described.

Published in: Image Processing And Its Applications, 1999. Seventh
International Conference on (Conf. Publ. No. 465)

Date of Conference: 13-15 July 1999

Date Added to IEEE Xplore: 06 August 2002

Print ISBN: 0-85296-717-9

Print ISSN: 0537-9989

14. Neural
networks: the state of the art


Artificial neural networks (ANNs) have demonstrated their
success in many applications due to their ability to solve some problems with
relative ease of use and the model-free property they enjoy. ANNs can solve
problems without the need to understand or learn the analytical and statistical
properties of the problem nor the solution steps. Research in ANNs has resulted
in a variety of models and learning algorithms. In this paper, a brief review
of recent advances in the field is presented. The paper then focuses on the
recent work conducted by the author’s group on modular neural networks. In
particular, the paper discusses the different modular structures, modes of
interactions, capabilities, co-operation among modules and fusion of their
decisions. Performance of these models has proven to be superior to nonmodular
neural networks.

Published in: Microelectronics, 1999. ICM ’99. The Eleventh
International Conference on

Date of Conference: 22-24 Nov. 1999

Date Added to IEEE Xplore: 06 August 2002

Print ISBN: 0-7803-6643-3

15. Le