NOTES ON THE VERSION OF 2001-08-31

Changes in this version;

1) New sampling methods for neural networks were added that assist
   with coupling.  See net-mc.doc for details.

2) One can now specify that connections to certain inputs should be
   omitted for a hidden or output layer of a network, analogous to
   the "omit" option for Gaussian process models.  This is useful
   when explicitly setting up additive models.  The output format
   of net-spec and net-display has changed to accomodate display of 
   this information.  See net-spec.doc for details.  

3) The maximum number of hidden layers is now limited to seven, so that 
   the "omit" flags (see point above) will fit in a byte.

4) One can now specify that the activation function for hidden units in
   a particular layer should be h(u) = sqrt(2)*sin(sqrt(2)*u) or the 
   identity function, in place of the default of tanh.  See net-spec.doc 
   for details.

5) Formulas used by the 'dist' module (and by 'calc') can now refer
   to functions that are defined by external C programs.  Communication
   with these programs is done via pipes.  Currently, only function 
   evaluation is supported, not gradient evaluation or random number
   generation.  See formula.doc for details.

6) The "p" option to net-pred has now been implemented for survival models
   with piecewise-constant hazard. 

7) A "Q" option has been added to net-pred and gp-pred for displaying 1%
   and 99% quantiles.  See net-pred.doc and gp-pred.doc for details.

8) The number of sample points used to estimate medians and quantiles has
   been increased from 11 to 101 per iteration.  This gives more accurate 
   results, at the cost of extra computation and extra memory usage.  The
   way the random number seed is set has also changed.  See net-pred.doc
   and gp-pred.doc for details.

9) A "D" option for net-pred has been added, which finds the mean (over 
   multiple iterations) of the median response (using the network at a
   single iteration).  It's useful mainly as a way of accurately computing 
   the median response with one network for a survival analysis model
   (without using Monte Carlo as 'd' does).  See net-pred.doc for details.

10) The data and command files used to produce the examples for my talk on
    "Survival analysis using a Bayesian neural network", at the 2001 Joint
    Statistical Meetings in Atlanta, are included as a sub-directory of
    ex-surv.


Bug fixes.

1) Fixed a bug in dist-gen that led it to ignore any constant definitions
   in dist-spec (the constants were treated as zero).

2) Fixed a bug that prevented "Gaussian" from being a synonym for "Normal".

3) Fixed a bug that prevented coalescence in the first stage of xxx-circ
   when the coupled chains used dynamical sampling methods involving 
   momentum variables.

4) Fixed a bug (not initializing a variable to 0) in mc-spec that could
   cause spurious errors or other problems when "repeat/end" were used.

5) Fixed a bug in net-gen that caused hidden-output weights to always
   be generated as zeros when output unit "adjustments" were present.


Known bugs and other deficiencies.

1) The facility for plotting quantities using "plot" operations in xxx-mc
   doesn't always work for the first run of xxx-mc (before any
   iterations exist in the log file).  A work-around is to do a run of
   xxx-mc to produce just one iteration before attempting a run of
   xxx-mc that does any "plot" operations.

2) The CPU time features (eg, the "k" quantity) will not work correctly
   if a single iteration takes more than about 71 minutes.

3) The latent value update operations for Gaussian processes may recompute 
   the inverse covariance matrix even when an up-to-date version was 
   computed for the previous Monte Carlo operation.

4) Covariance matrices are stored in full, even though they are symmetric,
   which sometimes costs a factor of two in memory usage.