NET-SPEC:  Create a new network, or display specifications for existing net.

Net-spec creates a log file containing records describing the network
architecture and the associated priors.  The network architectures
that can be specified are described in net.doc and net-models.PDF.
When invoked with just a log file as argument, net-spec displays the
specifications of the network stored in that log file.

Usage:

    net-spec log-file N-inputs { N-hidden [ act-func ] } N-outputs
             / { group=prior [ omit-spec | config-spec ] } 

or: 

    net-spec log-file [ "sizes" ] [ "config" ]

The first form above is for specifying a network archicture. 

The second form is for displaying the network architecture stored in
the specified log file.  If the optional "sizes" argument is
specified, the numbers of parameters for each group are output.  The
optional "config" argument causes the detailed configurations fro
"config" specifications to be display, in fully-expanded form (as
individual triplets or pairs).


Specifying layer sizes and activation functions.

N-inputs and N-outputs are the numbers of units in the input and
output layers.  The sizes of zero or more hidden layers are specified
between these, with the activation function used for units in the
layer following the size.  Possible activation functions are:

    tanh  softplus  softplus0  identity

The default is tanh, the hyperbolic tangent function.  The softplus
activation function is h(u) = log(1+exp(u)).  The softplus0 function
is similar, h(u) = log(1+exp(u))-log(2), which has value zero when the
input is zero.  For the identity activation function, the output of a
hidden unit is identical to its summed input.


Specifying layer connections and priors.

Following a "/", the priors for groups of weights, biases, offsets,
and adjustments are specified.  For the syntax of a prior
specification (except for adjustments), see prior.doc.  If "-" is
given instead of a prior specification (or if a prior specification is
omitted entirely), the parameters in question do not exist at all,
which is equivalent to their being zero.  For adjustments, the prior
is specified by a single "alpha" value.  A value of "-" represents
infinity (effectively eliminating the adjustment).  Omitting a prior
for a set of adjustments also eliminates them.

The prior of a group of input-to-hidden or input-output weights may be
followed by a specification of inputs to be omitted, with the form:

    omit:[-]<input>{,<input>}

This specifies that connections from certain inputs are to be omitted
from those feeding into this layer.  If the "-" is placed before the
list of inputs, the list gives the inputs that are NOT omitted.
Inputs are numbered starting with 1.

The prior for a group of weights for input-hidden, input-output,
hidden-hidden, or hidden-output connections, or for hidden or output
biases may have the configuration of connections for the weights or
biases (with possible sharing) specified with an argument of the form:

    config:<file>

This may not be combined with an "omit" specification.  It is also not
allowed if the prior used specifies a value for Alpha-sub-group (see
prior.doc), since sub-groups by source unit are not well-defined when
weights may be shared.  The prior must also not have automatic width
scaling (no "x" option).  Finally, the prior for weights in a group
with a configuration file is not affected by the adjustments for its
destination layer

See net-config.doc for documentation on how a configuration is
specified in a file.  The named configuration files do not need to
exist when the configuration is specified; they are read (and checked
for errors) only when needed later.  A configuration file may be
specified with an initial "%", in which case the remainder is a
command that is run to output the "file" contents.

Groups of parameters are identified by name, sometimes with a hidden
layer number (from 0), as follows:

    ti      offsets for the inputs
    ih#     weights from inputs to hidden layer # (default for # is 0)
    bh#     biases for hidden layer # (default for # is 0)
    th#     offsets for hidden layer # (default for # is 0)
    h#h#    weights from the hidden layer identified by the first #
            to the hidden layer identified by the second # (which must
            be greater than the first)
    hh#     weights from hidden layer # minus 1 to hidden layer # 
            (default for # is 1)
    h#o     weights from hidden layer # to outputs (default for # is
            the last hidden layer)
    ho#     same as h#o
    io      weights from inputs to outputs
    bo      biases for outputs
    ah#     adjustments for hidden layer # (default for # is 0)
    ao      adjustments for outputs

The hidden layers are numbered starting with 0.

Note that a data model will also have to be specified (with model-spec)
if any learning is to be done.  

Two records are written to the newly created log file - one with type
'A' with a net_arch structure as data, and one with type 'P' with a
net_priors structure as data.  A record of type 'F' will be written as
well if there are "omit" or "config" specifications.  These records
have an index of -1.  When the second form of the command is used
(with just the log file as argument), these records are read and the
information they contain is displayed on standard output.

            Copyright (c) 1995-2022 by Radford M. Neal