NeuralWare's NeuralWorks Predict engine provides the technology foundation for the NeuralPower product suite. The Predict engine comprises a set of rigorously engineered software libraries written in C. NeuralPower Graphical User Interface components are written in Java.
NeuralPower Modeler creates forecasting models using genetic algorithm and neural network training capabilities contained in the Predict engine's development package. Modeler creates an ensemble of neural networks to produce hour-ending forecasts for the current day and one day in the future. Networks that produce day-ahead forecasts are also used to generate forecasts for all other future days.
NeuralPower Forecaster and Scheduler use the Predict engine's run-time package to execute models that produce forecast plots as well as structured files which can be easily imported by other software that requires accurate demand and/or price forecasts.
The following sections provide additional information about the technology that underlies NeuralPower forecasts.
An Overview of Neural Networks
Neural networks – or more precisely, artificial neural networks (ANNs) – comprise one branch of a family of technologies known as machine learning. Machine learning encompasses developing algorithms and procedures that allow computer systems (hardware and software) to both perform, and to improve their ability to perform, tasks which humans consider intelligent. A neural network learns patterns and relationships in data through an iterative process that in effect is adjusting coefficients in a complex, non-linear equation.
The development of artificial neural networks was initially motivated by insights into how biological brains – and in particular, mammalian brains (natural neural networks) – function. Put simply, mammalian brains learn as connections between neurons are strengthened – the result of electrochemical processes triggered by external or internal stimuli ("experiences"). Figure 1 depicts a biological neuron (the cell body, or Soma) and its connections. Together they comprise the basic building blocks for biological brains.
Similarly, Artificial Neural Networks consist of many interconnected artificial neurons – sometimes referred to as Processing Elements
(PEs). Artificial neural networks can be designed to implement either supervised or unsupervised learning. The neural networks created by NeuralPower employ supervised learning. With supervised learning, the learning (modeling) objective is to map a set of input (independent) data values to an associated 
output (dependent) data value. The supervised learning process is the equivalent of fitting a complex mathematical formula to a curve, yielding models which generalize well and which can be employed in a wide variety of prediction and classification tasks. With artificial neurons which implement supervised learning, thresholds are modeled by activation functions and reinforcement is modeled by non-linear transfer functions. A typical neural network uses sigmoid or hyperbolic transfer functions – which are non-linear in their response to inputs and differentiable. Figure 2 illustrates a generic artificial neuron (PEj) with one connection to an input node (PEi).
During supervised training, the back propagation algorithm uses calculus methods to distribute ("propagate") errors at the output layer backwards through the network until the input layer is reached. The learning (adaptation) power of neural networks stems from capabilities that emerge when many artificial neurons are connected. Figure 3 illustrates a typical neural network architecture comprising an input layer, one hidden layer and an output layer. The bias PE ensures that in the event an input record contains all zeroes some weights will still be updated.
The network depicted in Figure 3 is representative of networks which NeuralPower creates to generate electricity demand and price forecasts. Inputs for training the network are combinations of historic demand, weather, prices, and potentially other data. For a given objective (for example, hourly demand) the single output is predicted demand for a specific hour.
Using NeuralPower to Create Neural Networks
NeuralPower relies on the highly automated capabilities of the NeuralWorks Predict engine to construct robust neural networks. In general, all a modeler must do is specify the location (file names) and date spans for training data. NeuralPower performs basic validity checks on raw training data, then the Predict engine automatically handles the tedious and error-prone task of preparing validated data for use in training models. Predict first computes basic statistics for all potential input fields; it then identifies and applies appropriate mathematical transformations that yield better distributions of training data values for use in training models.
After raw data values have been validated and possibly transformed, the Predict engine processes the resulting pool of potential model inputs with a genetic algorithm (GA) to select the best subset to use as actual inputs for the neural network.
After input variables are selected by the GA, the Predict engine begins developing the neural network using a training method called Cascade Correlation, invented by Scott Fahlman of Carnegie Mellon University. Cascade Correlation automatically defines the network architecture by dynamically adding hidden units as needed – thus it is not necessary for the modeler to specify the number of hidden units or hidden layers.
Using NeuralPower to Evaluate Neural Networks
For every hour in the day, NeuralPower constructs multiple models (up to 18 per hour). While each model has different modeling parameters (for example, the number of transformations applied to raw data), all models are trained with the same data.
When all models for a forecast have been trained, it is a simple matter to rank them by a standard metric such as MAPE (Mean Absolute Percent Error). The best-performing models can then be examined in greater detail, using sensitivity analysis and by reviewing the specific data transformations applied, in order to better assess how well models will generalize when presented with new data. NeuralPower also permits viewing the effect of averaging the results from the top several models – which typically leads to better over-all performance.
The decision to deploy a given set of models over some other set of models is ultimately based on the performance of each set over the entire training dataset. The set of models which produces the lowest MAPE is deployed for use by NeuralPower Forecaster and Scheduler to generate daily forecasts.
