Section 1 of 5 - Prev - Next
All sections - 1 - 2 - 3 - 4 - 5

Archive-name: ai-faq/neural-nets/part4
Last-modified: 2002-11-24
URL: ftp://ftp.sas.com/pub/neural/FAQ4.html
Maintainer: saswss@unx.sas.com (Warren S. Sarle)

Copyright 1997, 1998, 1999, 2000, 2001, 2002 by Warren S. Sarle, Cary, NC,
USA. Reviews provided by other authors as cited below are copyrighted by
those authors, who by submitting the reviews for the FAQ give permission for
the review to be reproduced as part of the FAQ in any of the ways specified
in part 1 of the FAQ. 

This is part 4 (of 7) of a monthly posting to the Usenet newsgroup
comp.ai.neural-nets. See the part 1 of this posting for full information
what it is all about.

========== Questions ========== 
********************************

Part 1: Introduction
Part 2: Learning
Part 3: Generalization
Part 4: Books, data, etc.

   Books and articles about Neural Networks?
      The Best
         The best of the best
         The best popular introduction to NNs
         The best introductory book for business executives
         The best elementary textbooks
         The best books on using and programming NNs
         The best intermediate textbooks on NNs
         The best advanced textbook covering NNs
         The best book on neurofuzzy systems
         The best comparison of NNs with other classification methods
      Other notable books
         Introductory
         Bayesian learning
         Biological learning and neurophysiology
         Collections
         Combining networks
         Connectionism
         Feedforward networks
         Fuzzy logic and neurofuzzy systems
         General (including SVMs and Fuzzy Logic)
         History
         Knowledge, rules, and expert systems
         Learning theory
         Object oriented programming
         On-line and incremental learning
         Optimization
         Pulsed/Spiking networks
         Recurrent
         Reinforcement learning
         Speech recognition
         Statistics
         Time-series forecasting
         Unsupervised learning
      Books for the Beginner
      Not-quite-so-introductory Literature
      Books with Source Code (C, C++)
      The Worst
   Journals and magazines about Neural Networks?
   Conferences and Workshops on Neural Networks?
   Neural Network Associations?
   Mailing lists, BBS, CD-ROM?
   How to benchmark learning methods?
   Databases for experimentation with NNs?
      UCI machine learning database
      UCI KDD Archive
      The neural-bench Benchmark collection
      Proben1
      Delve: Data for Evaluating Learning in Valid Experiments
      Bilkent University Function Approximation Repository
      NIST special databases of the National Institute Of Standards And
      Technology:
      CEDAR CD-ROM 1: Database of Handwritten Cities, States, ZIP Codes,
      Digits, and Alphabetic Characters
      AI-CD-ROM
      Time series
      Financial data
      USENIX Faces
      Linguistic Data Consortium
      Otago Speech Corpus
      Astronomical Time Series
      Miscellaneous Images
      StatLib

Part 5: Free software
Part 6: Commercial software
Part 7: Hardware and miscellaneous

------------------------------------------------------------------------

Subject: Books and articles about Neural Networks?
==================================================

The following search engines will search many bookstores for new and used
books and return information on availability, price, and shipping charges:

   AddAll: http://www.addall.com/
   Bookfinder: http://www.bookfinder.com/

Clicking on the author and title of most of the books listed in the "Best"
and "Notable" sections will do a search using AddAll.

There are many on-line bookstores, such as:

   Amazon: http://www.amazon.com/
   Amazon, UK: http://www.amazon.co.uk/
   Amazon, Germany: http://www.amazon.de/
   Barnes & Noble: http://www.bn.com/
   Bookpool: http://www.bookpool.com/
   Borders: http://www.borders.com/
   Fatbrain: http://www.fatbrain.com/

The neural networks reading group at the University of Illinois at
Urbana-Champaign, the Artifical Neural Networks and Computational Brain
Theory (ANNCBT) forum, has compiled a large number of book and paper reviews
at http://anncbt.ai.uiuc.edu/, with an emphasis more on cognitive science
rather than practical applications of NNs. 

The Best
++++++++

The best of the best
--------------------

Bishop (1995) is clearly the single best book on artificial NNs. This book
excels in organization and choice of material, and is a close runner-up to
Ripley (1996) for accuracy. If you are new to the field, read it from cover
to cover. If you have lots of experience with NNs, it's an excellent
reference. If you don't know calculus, take a class. I hope a second edition
comes out soon! For more information, see The best intermediate textbooks on
NNs below. 

If you have questions on feedforward nets that aren't answered by Bishop,
try Masters (1993) or Reed and Marks (1999) for practical issues or Ripley
(1996) for theortical issues, all of which are reviewed below. 

The best popular introduction to NNs
------------------------------------

Hinton, G.E. (1992), "How Neural Networks Learn from Experience", Scientific
American, 267 (September), 144-151 (page numbers are for the US edition).
Author's Webpage: http://www.cs.utoronto.ca/DCS/People/Faculty/hinton.html
(official)
and http://www.cs.toronto.edu/~hinton (private)
Journal Webpage: http://www.sciam.com/
Additional Information: Unfortunately that article is not available there.

The best introductory book for business executives
--------------------------------------------------

Bigus, J.P. (1996), Data Mining with Neural Networks: Solving Business
Problems--from Application Development to Decision Support, NY:
McGraw-Hill, ISBN 0-07-005779-6, xvii+221 pages.
The stereotypical business executive (SBE) does not want to know how or why
NNs work--he (SBEs are usually male) just wants to make money. The SBE may
know what an average or percentage is, but he is deathly afraid of
"statistics". He understands profit and loss but does not want to waste his
time learning things involving complicated math, such as high-school
algebra. For further information on the SBE, see the "Dilbert" comic strip. 

Bigus has written an excellent introduction to NNs for the SBE. Bigus says
(p. xv), "For business executives, managers, or computer professionals, this
book provides a thorough introduction to neural network technology and the
issues related to its application without getting bogged down in complex
math or needless details. The reader will be able to identify common
business problems that are amenable to the neural netwrk approach and will
be sensitized to the issues that can affect successful completion of such
applications." Bigus succeeds in explaining NNs at a practical, intuitive,
and necessarily shallow level without formulas--just what the SBE needs.
This book is far better than Caudill and Butler (1990), a popular but
disastrous attempt to explain NNs without formulas. 

Chapter 1 introduces data mining and data warehousing, and sketches some
applications thereof. Chapter 2 is the semi-obligatory
philosophico-historical discussion of AI and NNs and is well-written,
although the SBE in a hurry may want to skip it. Chapter 3 is a very useful
discussion of data preparation. Chapter 4 describes a variety of NNs and
what they are good for. Chapter 5 goes into practical issues of training and
testing NNs. Chapters 6 and 7 explain how to use the results from NNs.
Chapter 8 discusses intelligent agents. Chapters 9 through 12 contain case
histories of NN applications, including market segmentation, real-estate
pricing, customer ranking, and sales forecasting. 

Bigus provides generally sound advice. He briefly discusses overfitting and
overtraining without going into much detail, although I think his advice on
p. 57 to have at least two training cases for each connection is somewhat
lenient, even for noise-free data. I do not understand his claim on pp. 73
and 170 that RBF networks have advantages over backprop networks for
nonstationary inputs--perhaps he is using the word "nonstationary" in a
sense different from the statistical meaning of the term. There are other
things in the book that I would quibble with, but I did not find any of the
flagrant errors that are common in other books on NN applications such as
Swingler (1996). 

The one serious drawback of this book is that it is more than one page long
and may therefore tax the attention span of the SBE. But any SBE who
succeeds in reading the entire book should learn enough to be able to hire a
good NN expert to do the real work. 

The best elementary textbooks
-----------------------------

Fausett, L. (1994), Fundamentals of Neural Networks: Architectures,
Algorithms, and Applications, Englewood Cliffs, NJ: Prentice Hall, ISBN
0-13-334186-0. Also published as a Prentice Hall International Edition, ISBN
0-13-042250-9. Sample software (source code listings in C and Fortran) is
included in an Instructor's Manual. 
Book Webpage (Publisher): http://www.prenhall.com/books/esm_0133341860.html
Additional Information: The mentioned programs / additional support is not
available. Contents:
Ch. 1 Introduction, 1.1 Why Neural Networks and Why Now?, 1.2 What Is a
Neural Net?, 1.3 Where Are Neural Nets Being Used?, 1.4 How Are Neural
Networks Used?, 1.5 Who Is Developing Neural Networks?, 1.6 When Neural Nets
Began: the McCulloch-Pitts Neuron;
Ch. 2 Simple Neural Nets for Pattern Classification, 2.1 General Discussion,
2.2 Hebb Net, 2.3 Perceptron, 2.4 Adaline;
Ch. 3 Pattern Association, 3.1 Training Algorithms for Pattern Association,
3.2 Heteroassociative Memory Neural Network, 3.3 Autoassociative Net, 3.4
Iterative Autoassociative Net, 3.5 Bidirectional Associative Memory (BAM);
Ch. 4 Neural Networks Based on Competition, 4.1 Fixed-Weight Competitive
Nets, 4.2 Kohonen Self-Organizing Maps, 4.3 Learning Vector Quantization,
4.4 Counterpropagation;
Ch. 5 Adaptive Resonance Theory, 5.1 Introduction, 5.2 Art1, 5.3 Art2; 
Ch. 6 Backpropagation Neural Net, 6.1 Standard Backpropagation, 6.2
Variations, 6.3 Theoretical Results;
Ch. 7 A Sampler of Other Neural Nets, 7.1 Fixed Weight Nets for Constrained
Optimization, 7.2 A Few More Nets that Learn, 7.3 Adaptive Architectures,
7.4 Neocognitron; Glossary. 

Review by Ian Cresswell: 

   What a relief! As a broad introductory text this is without any doubt
   the best currently available in its area. It doesn't include source
   code of any kind (normally this is badly written and compiler
   specific). The algorithms for many different kinds of simple neural
   nets are presented in a clear step by step manner in plain English. 

   Equally, the mathematics is introduced in a relatively gentle manner.
   There are no unnecessary complications or diversions from the main
   theme. 

   The examples that are used to demonstrate the various algorithms are
   detailed but (perhaps necessarily) simple. 

   There are bad things that can be said about most books. There are
   only a small number of minor criticisms that can be made about this
   one. More space should have been given to backprop and its variants
   because of the practical importance of such methods. And while the
   author discusses early stopping in one paragraph, the treatment of
   generalization is skimpy compared to the books by Weiss and
   Kulikowski or Smith listed above. 

   If you're new to neural nets and you don't want to be swamped by
   bogus ideas, huge amounts of intimidating looking mathematics, a
   programming language that you don't know etc. etc. then this is the
   book for you. 

   In summary, this is the best starting point for the outsider and/or
   beginner... a truly excellent text. 

Smith, M. (1996). Neural Networks for Statistical Modeling, NY: Van Nostrand
Reinhold, ISBN 0-442-01310-8. 
Apparently there is a new edition I haven't seen yet:
Smith, M. (1996). Neural Networks for Statistical Modeling, Boston:
International Thomson Computer Press, ISBN 1-850-32842-0.
Book Webpage (Publisher): http://www.thompson.com/
Publisher's address: 20 Park Plaza, Suite 1001, Boston, MA 02116, USA.
Smith is not a statistician, but he has made an impressive effort to convey
statistical fundamentals applied to neural networks. The book has entire
brief chapters on overfitting and validation (early stopping and
split-sample validation, which he incorrectly calls cross-validation),
putting it a rung above most other introductions to NNs. There are also
brief chapters on data preparation and diagnostic plots, topics usually
ignored in elementary NN books. Only feedforward nets are covered in any
detail.
Chapter headings: Mapping Functions; Basic Concepts; Error Derivatives;
Learning Laws; Weight Initialization; The Course of Learning: An Example;
Overfitting; Cross Validation; Preparing the Data; Representing Variables;
Using the Model. 

Weiss, S.M. and Kulikowski, C.A. (1991), Computer Systems That Learn, 
Morgan Kaufmann. ISBN 1-55860-065-5. 
Author's Webpage: Kulikowski: 
http://ruccs.rutgers.edu/faculty/kulikowski.html
Book Webpage (Publisher): http://www.mkp.com/books_catalog/1-55860-065-5.asp
Additional Information: Information of Weiss, S.M. are not available.
Briefly covers at a very elementary level feedforward nets, linear and
nearest-neighbor discriminant analysis, trees, and expert sytems,
emphasizing practical applications. For a book at this level, it has an
unusually good chapter on estimating generalization error, including
bootstrapping.

1 Overview of Learning Systems 
    1.1 What is a Learning System? 
    1.2 Motivation for Building Learning Systems 
    1.3 Types of Practical Empirical Learning Systems 
        1.3.1 Common Theme: The Classification Model 
        1.3.2 Let the Data Speak
    1.4 What's New in Learning Methods 
        1.4.1 The Impact of New Technology
    1.5 Outline of the Book 
    1.6 Bibliographical and Historical Remarks

2 How to Estimate the True Performance of a Learning System 
    2.1 The Importance of Unbiased Error Rate Estimation 
    2.2. What is an Error? 
        2.2.1 Costs and Risks
    2.3 Apparent Error Rate Estimates 
    2.4 Too Good to Be True: Overspecialization 
    2.5 True Error Rate Estimation 
        2.5.1 The Idealized Model for Unlimited Samples 
        2.5.2 Train-and Test Error Rate Estimation 
        2.5.3 Resampling Techniques 
        2.5.4 Finding the Right Complexity Fit
    2.6 Getting the Most Out of the Data 
    2.7 Classifier Complexity and Feature Dimensionality 
        2.7.1 Expected Patterns of Classifier Behavior
    2.8 What Can Go Wrong? 
        2.8.1 Poor Features, Data Errors, and Mislabeled Classes 
        2.8.2 Unrepresentative Samples
    2.9 How Close to the Truth? 
    2.10 Common Mistakes in Performance Analysis 
    2.11 Bibliographical and Historical Remarks

3 Statistical Pattern Recognition 
    3.1 Introduction and Overview 
    3.2 A Few Sample Applications 
    3.3 Bayesian Classifiers 
        3.3.1 Direct Application of the Bayes Rule
    3.4 Linear Discriminants 
        3.4.1 The Normality Assumption and Discriminant Functions 
        3.4.2 Logistic Regression
    3.5 Nearest Neighbor Methods 
    3.6 Feature Selection 
    3.7 Error Rate Analysis 
    3.8 Bibliographical and Historical Remarks

4 Neural Nets 
    4.1 Introduction and Overview 
    4.2 Perceptrons 
        4.2.1 Least Mean Square Learning Systems 
        4.2.2 How Good Is a Linear Separation Network?
    4.3 Multilayer Neural Networks 
        4.3.1 Back-Propagation 
        4.3.2 The Practical Application of Back-Propagation
    4.4 Error Rate and Complexity Fit Estimation 
    4.5 Improving on Standard Back-Propagation 
    4.6 Bibliographical and Historical Remarks

5 Machine Learning: Easily Understood Decision Rules 
    5.1 Introduction and Overview 
    5.2 Decision Trees 
        5.2.1 Finding the Perfect Tree 
        5.2.2 The Incredible Shrinking Tree 
        5.2.3 Limitations of Tree Induction Methods
    5.3 Rule Induction 
        5.3.1 Predictive Value Maximization
    5.4 Bibliographical and Historical Remarks

6 Which Technique is Best? 
    6.1 What's Important in Choosing a Classifier? 
        6.1.1 Prediction Accuracy 
        6.1.2 Speed of Learning and Classification 
        6.1.3 Explanation and Insight
    6.2 So, How Do I Choose a Learning System? 
    6.3 Variations on the Standard Problem 
        6.3.1 Missing Data 
        6.3.2 Incremental Learning
    6.4 Future Prospects for Improved Learning Methods 
    6.5 Bibliographical and Historical Remarks

7 Expert Systems 
    7.1 Introduction and Overview 
        7.1.1 Why Build Expert Systems? New vs. Old Knowledge
    7.2 Estimating Error Rates for Expert Systems 
    7.3 Complexity of Knowledge Bases 
        7.3.1 How Many Rules Are Too Many?
    7.4 Knowledge Base Example 
    7.5 Empirical Analysis of Knowledge Bases 
    7.6 Future: Combined Learning and Expert Systems 
    7.7 Bibliographical and Historical Remarks

Reed, R.D., and Marks, R.J, II (1999), Neural Smithing: Supervised Learning
in Feedforward Artificial Neural Networks, Cambridge, MA: The MIT Press,
ISBN 0-262-18190-8.
Author's Webpage: Marks: http://cialab.ee.washington.edu/Marks.html
Book Webpage (Publisher): 
http://mitpress.mit.edu/book-home.tcl?isbn=0262181908
After you have read Smith (1996) or Weiss and Kulikowski (1991), consult
Reed and Marks for practical details on training MLPs (other types of neural
nets such as RBF networks are barely even mentioned). They provide extensive
coverage of backprop and its variants, and they also survey conventional
optimization algorithms. Their coverage of initialization methods,
constructive networks, pruning, and regularization methods is unusually
thorough. Unlike the vast majority of books on neural nets, this one has
lots of really informative graphs. The chapter on generalization assessment
is slightly weak, which is why you should read Smith (1996) or Weiss and
Kulikowski (1991) first. Also, there is little information on data
preparation, for which Smith (1996) and Masters (1993; see below) should be
consulted. There is some elementary calculus, but not enough that it should
scare off anybody. Many second-rate books treat neural nets as mysterious
black boxes, but Reed and Marks open up the box and provide genuine insight
into the way neural nets work. 

One problem with the book is that the terms "validation set" and "test set"
are used inconsistently. 

Chapter headings: Supervised Learning; Single-Layer Networks; MLP
Representational Capabilities; Back-Propagation; Learning Rate and Momentum;
Weight-Initialization Techniques; The Error Surface; Faster Variations of
Back-Propagation; Classical Optimization Techniques; Genetic Algorithms and
Neural Networks; Constructive Methods; Pruning Algorithms; Factors
Influencing Generalization; Generalization Prediction and Assessment;
Heuristics for Improving Generalization; Effects of Training with Noisy
Inputs; Linear Regression; Principal Components Analysis; Jitter
Calculations; Sigmoid-like Nonlinear Functions 

The best books on using and programming NNs
-------------------------------------------

Masters, T. (1993), Practical Neural Network Recipes in C++, Academic
Press, ISBN 0-12-479040-2, US $45 incl. disks.
Book Webpage (Publisher): 
http://www.apcatalog.com/cgi-bin/AP?ISBN=0124790402&LOCATION=US&FORM=FORM2
Masters has written three exceptionally good books on NNs (the two others
are listed below). He combines generally sound practical advice with some
basic statistical knowledge to produce a programming text that is far
superior to the competition (see "The Worst" below). Not everyone likes his
C++ code (the usual complaint is that the code is not sufficiently OO) but,
unlike the code in some other books, Masters's code has been successfully
compiled and run by some readers of comp.ai.neural-nets. Masters's books are
well worth reading even for people who have no interest in programming. 
Chapter headings: Foundations; Classification; Autoassociation; Time-Series
Prediction; Function Approximation; Multilayer Feedforward Networks; Eluding
Local Minima I: Simulated Annealing; Eluding Local Minima II: Genetic
Optimization; Regression and Neural Networks; Designing Feedforward Network
Architectures; Interpreting Weights: How Does This Thing Work; Probabilistic
Neural Networks; Functional Link Networks; Hybrid Networks; Designing the
Training Set; Preparing Input Data; Fuzzy Data and Processing; Unsupervised
Training; Evaluating Performance of Neural Networks; Confidence Measures;
Optimizing the Decision Threshold; Using the NEURAL Program. 

Masters, T. (1995) Advanced Algorithms for Neural Networks: A C++
Sourcebook, NY: John Wiley and Sons, ISBN 0-471-10588-0
Book Webpage (Publisher): http://www.wiley.com/
Additional Information: One has to search.
Clear explanations of conjugate gradient and Levenberg-Marquardt
optimization algorithms, simulated annealing, kernel regression (GRNN) and
discriminant analysis (PNN), Gram-Charlier networks, dimensionality
reduction, cross-validation, and bootstrapping. 

Masters, T. (1994), Signal and Image Processing with Neural Networks: A
C++ Sourcebook, NY: Wiley, ISBN 0-471-04963-8.
Book Webpage (Publisher): http://www.wiley.com/
Additional Information: One has to search.

The best intermediate textbooks on NNs
--------------------------------------

Bishop, C.M. (1995). Neural Networks for Pattern Recognition, Oxford:
Oxford University Press. ISBN 0-19-853849-9 (hardback) or 0-19-853864-2
(paperback), xvii+482 pages.
Book Webpage (Author): http://research.microsoft.com/~cmbishop/nnpr.htm
Book Webpage (Publisher): http://www.oup.co.uk/isbn/0-19-853864-2
This is definitely the best book on feedforward neural nets for readers
comfortable with calculus. The book is exceptionally well organized,
presenting topics in a logical progression ideal for conceptual
understanding. 

Geoffrey Hinton writes in the foreword:
"Bishop is a leading researcher who has a deep understanding of the material
and has gone to great lengths to organize it in a sequence that makes sense.
He has wisely avoided the temptation to try to cover everything and has
therefore omitted interesting topics like reinforcement learning, Hopfield
networks, and Boltzmann machines in order to focus on the types of neural
networks that are most widely used in practical applications. He assumes
that the reader has the basic mathematical literacy required for an
undergraduate science degree, and using these tools he explains everything
from scratch. Before introducing the multilayer perceptron, for example, he
lays a solid foundation of basic statistical concepts. So the crucial
concept of overfitting is introduced using easily visualized examples of
one-dimensional polynomials and only later applied to neural networks. An
impressive aspect of this book is that it takes the reader all the way from
the simplest linear models to the very latest Bayesian multilayer neural
networks without ever requiring any great intellectual leaps." 

Chapter headings: Statistical Pattern Recognition; Probability Density
Estimation; Single-Layer Networks; The Multi-layer Perceptron; Radial Basis
Functions; Error Functions; Parameter Optimization Algorithms;
Pre-processing and Feature Extraction; Learning and Generalization; Bayesian
Techniques; Symmetric Matrices; Gaussian Integrals; Lagrange Multipliers;
Calculus of Variations; Principal Components. 

Hertz, J., Krogh, A., and Palmer, R. (1991). Introduction to the Theory of
Neural Computation. Redwood City, CA: Addison-Wesley, ISBN 0-201-50395-6
(hardbound) and 0-201-51560-1 (paperbound)
Book Webpage (Publisher): http://www2.awl.com/gb/abp/sfi/computer.html
This is an excellent classic work on neural nets from the perspective of
physics covering a wide variety of networks. Comments from readers of
comp.ai.neural-nets: "My first impression is that this one is by far the
best book on the topic. And it's below $30 for the paperback."; "Well
written, theoretical (but not overwhelming)"; It provides a good balance of
model development, computational algorithms, and applications. The
mathematical derivations are especially well done"; "Nice mathematical
analysis on the mechanism of different learning algorithms"; "It is NOT for
mathematical beginner. If you don't have a good grasp of higher level math,
this book can be really tough to get through."

The best advanced textbook covering NNs
---------------------------------------

Ripley, B.D. (1996) Pattern Recognition and Neural Networks, Cambridge:
Cambridge University Press, ISBN 0-521-46086-7 (hardback), xii+403 pages.
Author's Webpage: http://www.stats.ox.ac.uk/~ripley/
Book Webpage (Publisher): http://www.cup.cam.ac.uk/
Additional Information: The Webpage includes errata and additional
information, which hasn't been available at publishing time, for this book.
Brian Ripley's book is an excellent sequel to Bishop (1995). Ripley starts
up where Bishop left off, with Bayesian inference and statistical decision
theory, and then covers some of the same material on NNs as Bishop but at a
higher mathematical level. Ripley also covers a variety of methods that are
not discussed, or discussed only briefly, by Bishop, such as tree-based
methods and belief networks. While Ripley is best appreciated by people with
a background in mathematical statistics, the numerous realistic examples in
his book will be of interest even to beginners in neural nets.
Chapter headings: Introduction and Examples; Statistical Decision Theory;
Linear Discriminant Analysis; Flexible Discriminants; Feed-forward Neural
Networks; Non-parametric Methods; Tree-structured Classifiers; Belief
Networks; Unsupervised Methods; Finding Good Pattern Features; Statistical
Sidelines. 

Devroye, L., Györfi, L., and Lugosi, G. (1996), A Probabilistic Theory of
Pattern Recognition, NY: Springer, ISBN 0-387-94618-7, vii+636 pages.
This book has relatively little material explicitly about neural nets, but
what it has is very interesting and much of it is not found in other texts.
The emphasis is on statistical proofs of universal consistency for a wide
variety of methods, including histograms, (k) nearest neighbors, kernels
(PNN), trees, generalized linear discriminants, MLPs, and RBF networks.
There is also considerable material on validation and cross-validation. The
authors say, "We did not scar the pages with backbreaking simulations or
quick-and-dirty engineering solutions" (p. 7). The formula-to-text ratio is
high, but the writing is quite clear, and anyone who has had a year or two
of mathematical statistics should be able to follow the exposition.
Chapter headings: The Bayes Error; Inequalities and Alternate Distance
Measures; Linear Discrimination; Nearest Neighbor Rules; Consistency; Slow
Rates of Convergence; Error Estimation; The Regular Histogram Rule; Kernel
Rules; Consistency of the k-Nearest Neighbor Rule; Vapnik-Chervonenkis
Theory; Combinatorial Aspects of Vapnik-Chervonenkis Theory; Lower Bounds
for Empirical Classifier Selection; The Maximum Likelihood Principle;
Parametric Classification; Generalized Linear Discrimination; Complexity
Regularization; Condensed and Edited Nearest Neighbor Rules; Tree
Classifiers; Data-Dependent Partitioning; Splitting the Data; The
Resubstitution Estimate; Deleted Estimates of the Error Probability;
Automatic Kernel Rules; Automatic Nearest Neighbor Rules; Hypercubes and
Discrete Spaces; Epsilon Entropy and Totally Bounded Sets; Uniform Laws of
Large Numbers; Neural Networks; Other Error Estimates; Feature Extraction. 

The best books on neurofuzzy systems
------------------------------------

Brown, M., and Harris, C. (1994), Neurofuzzy Adaptive Modelling and
Control, NY: Prentice Hall, ISBN 0-13-134453-6.
Author's Webpage: http://www.isis.ecs.soton.ac.uk/people/m_brown.html
and http://www.ecs.soton.ac.uk/~cjh/
Book Webpage (Publisher): http://www.prenhall.com/books/esm_0131344536.html
Additional Information: Additional page at: 
http://www.isis.ecs.soton.ac.uk/publications/neural/mqbcjh94e.html and an
abstract can be found at: 
http://www.isis.ecs.soton.ac.uk/publications/neural/mqb93.html
Brown and Harris rely on the fundamental insight that that a fuzzy system is
a nonlinear mapping from an input space to an output space that can be
parameterized in various ways and therefore can be adapted to data using the
usual neural training methods (see "What is backprop?") or conventional
numerical optimization algorithms (see "What are conjugate gradients,
Levenberg-Marquardt, etc.?"). Their approach makes clear the intimate
connections between fuzzy systems, neural networks, and statistical methods
such as B-spline regression. 

The best comparison of NNs with other classification methods
------------------------------------------------------------

Michie, D., Spiegelhalter, D.J. and Taylor, C.C. (1994), Machine Learning,
Neural and Statistical Classification, Ellis Horwood. Author's Webpage:
Donald Michie: http://www.aiai.ed.ac.uk/~dm/dm.html
Additional Information: This book is out of print but available online at 
http://www.amsta.leeds.ac.uk/~charles/statlog/ 

Other notable books
+++++++++++++++++++

Introductory
------------

Anderson, J.A. (1995), An Introduction to Neural Networks, Cambridge,MA:
The MIT Press, ISBN 0-262-01144-1. 
Author's Webpage: http://www.cog.brown.edu/~anderson
Book Webpage (Publisher): 
http://mitpress.mit.edu/book-home.tcl?isbn=0262510812 or
http://mitpress.mit.edu/book-home.tcl?isbn=0262011441 (hardback)
Additional Information: Programs and additional information can be found at:
ftp://mitpress.mit.edu/pub/Intro-to-NeuralNets/
Anderson provides an accessible introduction to the AI and
neurophysiological sides of NN research, although the book is weak regarding
practical aspects of using NNs.
Chapter headings: Properties of Single Neurons; Synaptic Integration and
Neuron Models; Essential Vector Operations; Lateral Inhibition and Sensory
Processing; Simple Matrix Operations; The Linear Associator: Background and
Foundations; The Linear Associator: Simulations; Early Network Models: The
Perceptron; Gradient Descent Algorithms; Representation of Information;
Applications of Simple Associators: Concept Formation and Object Motion;
Energy and Neural Networks: Hopfield Networks and Boltzmann Machines;
Nearest Neighbor Models; Adaptive Maps; The BSB Model: A Simple Nonlinear
Autoassociative Neural Network; Associative Computation; Teaching Arithmetic
to a Neural Network. 

Hagan, M.T., Demuth, H.B., and Beale, M. (1996), Neural Network Design, 
Boston: PWS, ISBN 0-534-94332-2. 
It doesn't really say much about design, but this book provides formulas and

Section 1 of 5 - Prev - Next
All sections - 1 - 2 - 3 - 4 - 5

© allanswers.org | Terms of use