lnsc.lstm
Class ETLSTMFactory1

java.lang.Object
  |
  +--lnsc.lstm.AbstractLSTMFactory
        |
        +--lnsc.lstm.LSTMFactory
              |
              +--lnsc.lstm.ETLSTMFactory1

All Implemented Interfaces:

FunctionalUnitFactory

public class ETLSTMFactory1

extends LSTMFactory

Factory for LSTM network using eligibility traces to find more rapidely association in time space.

Special memory blocks are used. These blocks maintain memory traces for their input and used them in derivatives instead of using raw input. Traces are build as if input where bound between [-1, 1]. For a given input x_t it traces e_t = bound(lamdba(e_t) + x_t,[-1,1]), unless e_t and x_t have opposite sign and OppSignResetTraces is true, in whic cases e_t = bound(x_t). This is a formed of bounded cumulated traces.

The implementation is totally transparent to the network, since it is the memoryblock parameters derivative that are properly constructed.

It is unclear whether each gate should have a different trace decay rate and whether the peepwhole should have trace at all. Right now, they are all the same.

Also note that gradient are defined recursively, so traces may be wrong.

See Also:

ETLSTMNetwork1, ETLSTMMemoryBlock1

Constructor Summary

ETLSTMFactory1(int newInputCount, int newBlockCount, int newCellperBlock, boolean newSquashInput, boolean newSquashOutput, int newOutputCount, FunctionalUnit newSampleOutput, boolean newGateToGate, boolean newBiasToOutput, boolean newInputToOutput, boolean newGateToOutput, double newOutputWeightsLocalGradientFactor, double newLambda, boolean newOppSignResetTraces)
Construct an LSTM network factory.

Method Summary

FunctionalUnit

createUnit() Creates a new functional unit.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ETLSTMFactory1

public ETLSTMFactory1(int newInputCount,
                      int newBlockCount,
                      int newCellperBlock,
                      boolean newSquashInput,
                      boolean newSquashOutput,
                      int newOutputCount,
                      FunctionalUnit newSampleOutput,
                      boolean newGateToGate,
                      boolean newBiasToOutput,
                      boolean newInputToOutput,
                      boolean newGateToOutput,
                      double newOutputWeightsLocalGradientFactor,
                      double newLambda,
                      boolean newOppSignResetTraces)

Construct an LSTM network factory.

Parameters:

newInputCount - Number of input to the network.

newBlockCount - Number of memory block.

newCellperBlock - Number of memory cells per block.

newSquashInput - true to squash input to cell (default) false without (use identity instead)

newSquashOutput - true to squash output of cell as in older papers false without squashing (default)

newOutputCount - Number of output of the network

newSampleOutput - Sample of an output function (should have one input and one output, default LogisticUnit(1,0))

newGateToGate - Connects block gates to block (default false)

newBiasToOutput - Connects bias to output layer (default true)

newInputToOutput - Connects input to output layer (default true)

newGateToOutput - Connects block gates to output layer (default true)

newOutputWeightsLocalGradientFactor - Scales the gradien of the output weigths internally.

newLambda - Eligibility traces decay rate.

newOppSignResetTraces - true to reset traces on opposite sign.

Method Detail

createUnit

public FunctionalUnit createUnit()

Description copied from interface: FunctionalUnitFactory

Creates a new functional unit.

Specified by:

createUnit in interface FunctionalUnitFactory

Overrides:

createUnit in class AbstractLSTMFactory