|
|||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | ||||||||
See:
Description
| Interface Summary | |
| Action | Defines an action as generally understand in RL and other AI problems. |
| ActionGenerator | Generates list of possible actions. |
| ActionRepresentation | Action representation converter, use to convert an Action into a real-vector. |
| Agent | Defines an agent as generally understand in RL and other AI problems. |
| Environment | Defines a state space and its controler. |
| State | Defines a state as generally understand in RL and other AI problems. |
| StateGenerator | Generates next (or observable) states. |
| StateRepresentation | State representation converter, use to convert a state into a real-vector. |
| Class Summary | |
| AbstractAction | Optional basis for class implementing Action interface. |
| AbstractActionGenerator | Optional basis for class implementing ActionGenerator interface. |
| AbstractActionRepresentation | Optional basis for class implementing ActionRepresentation interface. |
| AbstractAgent | Optional basis for class implementing Agent interface. |
| AbstractObservableAgent | Optional basis for class implementing Agent interface. |
| AbstractObservableEnvironment | Optional basis for class implementing Environment interface. |
| AbstractState | Optional basis for class implementing State interface. |
| AbstractStateGenerator | Optional basis for class implementing StateGenerator interface. |
| AbstractStateRepresentation | Optional basis for class implementing StateRepresentation interface. |
| AbstractStateWithGen | Optional basis for class implementing State interface. |
| DefaultActionsEnumerator | Creates an actions enumator based on an array of actions. |
| SingleAgentEnvironment | Run a single agent environement for a given number of step or until success. |
Classes with the Environment interface are considered as
environements or artificial worlds and classes with the Agent
interface are considered artificial animated objects (intelligent or not).
Agents are usually placed into an environement class and it is that class that
allows the agent to take actions into the world and that provides the agent with
the current state.
The environement must provide the agent with a class implementing the State interface as well as classes implementing the ActionGenerator interface and the Action
respectively. An environment usually also have a class with the StateGenerator interface to generate the next states once an agent as
choosen an action.
Since some agents are doing search internally (specially in deterministic
environment), an agent can also have its own sets of classes implementing
State, Action, StateGenerator, and ActionGenerator to do its
internal search. The state generator is first use to transform the environment
state into its internal state, and it can then use its own set of classes to
perform a search in its head. Those classes depend on the amount of
knowledge an agent might have of the environment as opposed to the environment
classes who have full knowledge of it. The environment itself may have two sets
of states and actions, one for its internal model of the world, and one with
only the information accessible to the agent.
StateRepresentation and ActionRepresentation
interfaces are for agents and/or algorithms who require real-vector state or
action representation.
The package also has some generic environement simulator that work on any state/action/generator environment with certain properties as well as some generic search and learning algorithms that work on any state/action/generator/representation agent.
The State and ActionGenerator interface
has been extended to better support long list of actions and depth search by
having a new method for returning an Enumerator of action instead of an
array of actions. Defaults implementation in abstract base classes creates
default enumerator reading the array of previous methods.
The State interface has been extended to better support
partially observable states (in code as !isObservable) and allow environment to
automatically convert them into states that can be observed by the agent. By
default, older packages states requiring this librairy (version 1) will
be considered observable and the default implementation provided by
AbstractState will provide the missing interface (return the
state itself). The format is such that if multiple agent can observe a state
at the same time, a list of state is returned. In the same spirit,
AbstractStateWithGen now requires an extra
StateGenerator for partially observable state.
{link lnsc.page.SingleAgentEnvironment} has been updated to automatically
handle such states space.
State and Agent interfaces now have a toDataSet method. Default null
implementation is provided in abstract classes. There are now new Observable
agents and environments, who can use that DataSet description to notify
obervers of the current agents and environment state.
SingleAgentEnvironment does it.
Older code should work without problem if it derives abstract base classes.
One changed that has do be accounted in older code is the renaming of getStates for getNextStates in State interface for semantical reason. Also some of the StateGenerator and ActionGenerator property have been duplicated into State and these addition may required some code updates.
The StateRepresentation interface has been extended to
better support representation that have internal state that affect their output
of the current state. This allows support for representation that deal with
time such as representaiton based on recurrent network, or on delay lines, etc.
Basic (@link Object#toString()} method where added.
The Agent, StateRepresentation, and
ActionRepresentation interface has been extended to force
support of Serialization. For AbstractObservableAgent,
obervers are not part of the serialization process (they are lost in the
serialization process). Also note that non-stateless information is also assumed
lost, i.e. that de-serialization will implied a call to newEpisode in agents,
and to reset in state representation.
The code of this library and the conceptual structure of this library are the intellectual property of Francois Rivest. This package was developed for research purpose and any commercial use, and non-educational use is strictly prohibited unless explicit written consent is given by the author Francois Rivest. Copyright 2003-2008.
|
|||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | ||||||||