lnsc.lstm
Class ETLSTMNetwork1

java.lang.Object
  |
  +--lnsc.pmvf.AbstractFunctionalUnit2
        |
        +--lnsc.lstm.FastLSTMNetwork
              |
              +--lnsc.lstm.ETLSTMNetwork1

All Implemented Interfaces:

java.lang.Cloneable, FunctionalUnit, FunctionalUnit2, java.io.Serializable

public class ETLSTMNetwork1

extends FastLSTMNetwork

LSTM network using eligibility traces to find more rapidely association in time space.

Special memory blocks are used. These blocks maintain memory traces for their input and used them in derivatives instead of using raw input. Traces are build as if input where bound between [-1, 1]. For a given input x_t it traces e_t = bound(lamdba(e_t) + x_t,[-1,1]), unless e_t and x_t have opposite sign and OppSignResetTraces is true, in whic cases e_t = bound(x_t). This is a formed of bounded cumulated traces.

The implementation is totally transparent to the network, since it is the memoryblock parameters derivative that are properly constructed.

It is unclear whether each gate should have a different trace decay rate and whether the peepwhole should have trace at all. Right now, they are all the same.

Also note that gradient are defined recursively, so traces may be wrong.

Use factories to make it.

See Also:

ETLSTMMemoryBlock1, ETLSTMFactory1, Serialized Form

Nested Class Summary

Nested classes inherited from class lnsc.pmvf.FunctionalUnit2

FunctionalUnit2.ProcessPatternResult2

Nested classes inherited from class lnsc.FunctionalUnit

FunctionalUnit.ProcessPatternResult

Field Summary

Fields inherited from class lnsc.lstm.FastLSTMNetwork

m_Debug

Fields inherited from interface lnsc.FunctionalUnit

EMPTY_PATTERN

Constructor Summary

ETLSTMNetwork1(int newInputCount, int newBlockCount, int newCellperBlock, FunctionalUnit newg, FunctionalUnit newh, FunctionalUnit newInputGate, FunctionalUnit newForgetGate, FunctionalUnit newOutputGate, int newOutputCount, FunctionalUnit newSampleOutput, boolean newGateToGate, boolean newBiasToOutput, boolean newInputToOutput, boolean newGateToOutput, double newLambda, boolean newOppSignResetTraces)
Construct an LSTM network.

Method Summary

java.lang.String

toString()

Methods inherited from class lnsc.lstm.FastLSTMNetwork

clone, getParameters, processPattern, reset, setParameters

Methods inherited from class lnsc.pmvf.AbstractFunctionalUnit2

getInputCount, getOutputCount, getParameterCount, isDifferentiable, isParameterDifferentiable, isParameterTwiceDifferentiable, isStateless, isTwiceDifferentiable, processDataSet, processPattern

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ETLSTMNetwork1

public ETLSTMNetwork1(int newInputCount,
                      int newBlockCount,
                      int newCellperBlock,
                      FunctionalUnit newg,
                      FunctionalUnit newh,
                      FunctionalUnit newInputGate,
                      FunctionalUnit newForgetGate,
                      FunctionalUnit newOutputGate,
                      int newOutputCount,
                      FunctionalUnit newSampleOutput,
                      boolean newGateToGate,
                      boolean newBiasToOutput,
                      boolean newInputToOutput,
                      boolean newGateToOutput,
                      double newLambda,
                      boolean newOppSignResetTraces)

Construct an LSTM network.

Parameters:

newInputCount - Number of input to the network.

newBlockCount - Number of LSTM memory block.

newCellperBlock - Number of memory cells per block.

newg - Input squashing function (g)

newh - Output squashing function (h)

newInputGate - Input gate function

newForgetGate - Forget gate function

newOutputGate - Output gate function

newOutputCount - Number of output of the network

newSampleOutput - Sample of an output function (should have one input and one output, default LogisticUnit(1,0))

newGateToGate - Connects block gates to block (default false)

newBiasToOutput - Connects bias to output layer (default true)

newInputToOutput - Connects input to output layer (default true)

newGateToOutput - Connects block gates to output layer (default true)

newLambda - Eligibility traces decay rate.

newOppSignResetTraces - true to reset traces on opposite sign.

Method Detail

toString

public java.lang.String toString()

Overrides:

toString in class FastLSTMNetwork