lnsc.lstm
Class ETLSTMMemoryBlock1

java.lang.Object
  |
  +--lnsc.pmvf.AbstractFunctionalUnit2
        |
        +--lnsc.lstm.FastLSTMMemoryBlock
              |
              +--lnsc.lstm.ETLSTMMemoryBlock1

All Implemented Interfaces:

java.lang.Cloneable, FunctionalUnit, FunctionalUnit2, java.io.Serializable

public class ETLSTMMemoryBlock1

extends FastLSTMMemoryBlock

LSTM memory block using eligibility traces to find more rapidely association in time space.

Special memory blocks are used. These blocks maintain memory traces for their input and used them in derivatives instead of using raw input. Traces are build as if input where bound between [-1, 1]. For a given input x_t it traces e_t = bound(lamdba(e_t) + x_t,[-1,1]), unless e_t and x_t have opposite sign and OppSignResetTraces is true, in whic cases e_t = bound(x_t). This is a formed of bounded cumulated traces.

The implementation is totally transparent to the network, since it is the memoryblock parameters derivative that are properly constructed.

It is unclear whether each gate should have a different trace decay rate and whether the peepwhole should have trace at all. Right now, they are all the same.

Also note that gradient are defined recursively, so traces may be wrong.

Not to be used by itself

See Also:

ETLSTMNetwork1, Serialized Form

Nested Class Summary

Nested classes inherited from class lnsc.pmvf.FunctionalUnit2

FunctionalUnit2.ProcessPatternResult2

Nested classes inherited from class lnsc.FunctionalUnit

FunctionalUnit.ProcessPatternResult

Field Summary

Fields inherited from interface lnsc.FunctionalUnit

EMPTY_PATTERN

Constructor Summary

ETLSTMMemoryBlock1(int newInputCount, int newCellCount, FunctionalUnit newg, FunctionalUnit newh, FunctionalUnit newInputGate, FunctionalUnit newForgetGate, FunctionalUnit newOutputGate, double newLambda, boolean newOppSignResetTraces)
Construct an ETLSTM memory blocks with its input side weights.

Method Summary

FunctionalUnit2.ProcessPatternResult2	processPattern(double[] inputPattern, boolean computeDerivative, boolean computeSecondDerivative, boolean computeParameterDerivative, boolean computeParameterSecondDerivative, java.lang.String[] recordList) Processes an input pattern and returns its output pattern and derivatives (if requested).
void	reset() Reset internal transient state for non stateless functions.
java.lang.String	toString()

Methods inherited from class lnsc.lstm.FastLSTMMemoryBlock

clone, getParameters, setParameters

Methods inherited from class lnsc.pmvf.AbstractFunctionalUnit2

getInputCount, getOutputCount, getParameterCount, isDifferentiable, isParameterDifferentiable, isParameterTwiceDifferentiable, isStateless, isTwiceDifferentiable, processDataSet, processPattern

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ETLSTMMemoryBlock1

public ETLSTMMemoryBlock1(int newInputCount,
                          int newCellCount,
                          FunctionalUnit newg,
                          FunctionalUnit newh,
                          FunctionalUnit newInputGate,
                          FunctionalUnit newForgetGate,
                          FunctionalUnit newOutputGate,
                          double newLambda,
                          boolean newOppSignResetTraces)

Construct an ETLSTM memory blocks with its input side weights.

Parameters:

newInputCount - Number of feeding units.

newCellCount - Number of memory cells.

newg - Input squashing function (g)

newh - Output squashing function (h)

newInputGate - Input gate function

newForgetGate - Forget gate function

newOutputGate - Output gate function

newLambda - Eligibility traces decay rate.

newOppSignResetTraces - true to reset traces on opposite sign.

Method Detail

processPattern

public FunctionalUnit2.ProcessPatternResult2 processPattern(double[] inputPattern,
                                                            boolean computeDerivative,
                                                            boolean computeSecondDerivative,
                                                            boolean computeParameterDerivative,
                                                            boolean computeParameterSecondDerivative,
                                                            java.lang.String[] recordList)

Description copied from interface: FunctionalUnit2

Processes an input pattern and returns its output pattern and derivatives (if requested).

Specified by:

processPattern in interface FunctionalUnit2

Overrides:

processPattern in class FastLSTMMemoryBlock

Parameters:

inputPattern - The input pattern.

computeDerivative - Must be true if the derivative should be computed.

computeSecondDerivative - Must be true if the second derivative should be computed.

computeParameterDerivative - Must be true if the derivative with respect to the parameters should be computed.

computeParameterSecondDerivative - Must be true if be the derivative with respect to the parameters should be computed.

recordList - Extra data to be recorded.

Returns:

The output pattern and the derivative (if requested).

reset

public void reset()

Description copied from interface: FunctionalUnit

Reset internal transient state for non stateless functions.

Specified by:

reset in interface FunctionalUnit

Overrides:

reset in class FastLSTMMemoryBlock

toString

public java.lang.String toString()

Overrides:

toString in class FastLSTMMemoryBlock