Remote Proxies

Remote proxies provide a Pythonic way to communicate with the targets connected via InteractiveSession.

Basically the only requirement is that the remote end is capable of starting PythonShell in the interactive mode.

In this section the proxy management associated interfaces are described. Examples of proxies can be found from Examples of crl.interactivesessions usage.

AutoRecoveringTerminal

This module implements pseudo terminal based autorecovering terminal.

class crl.interactivesessions.autorecoveringterminal.AutoRecoveringTerminal

Automatically recovering terminal in case of listed exception occurs in run() or in initialize().

For running commands in the terminal of SSH host example.com just use:

>>> from crl.interactivesessions import (
...     AutorecoveringTerminal)
>>> from crl.interactivesessions import (
...     SshShell)
>>> term = AutoRecoveringTerminal()
>>> term.initialize(SshShell(
...     'example.com', 'username', 'password'))
>>> term.run('echo hello world!')

close()

Close terminal.

Note

Please remember to call initialize_terminal() after close(). in order to make the object usable again.

initialize(shells, broken_exceptions=<type 'exceptions.Exception'>, init_broken_exceptions=<type 'exceptions.Exception'>, sleep_between_tries=3, max_tries=3, prepare=<function <lambda>>, finalize=<function <lambda>>, verify=<function <lambda>>)

Initialize terminal topology information.

Note

This method does not open the connections. The connections are opened by initialize_terminal(). It is up to the use-case when to open the connection and thus connection opening is purposly separated from the topology setting.

The connection opening can be also left to retry_run(). It opens the connection in the first call. In this fashion the connection opening is not done without a real reason.

Args:

shells (Shell or list of Shells): List of objects derived from InteractiveSession.Shell which defines the desired state of the terminal.

broken_exceptions (Exception class or list of Exception classes): The exceptions which determines the broken session. If any of these exceptions occurs, the session is automatically recovered. By default, the occurence of any exception triggers the recovery process.

init_broken_exceptions (Eception class or list of exceptions): The exceptions which determines the broken session during init. By default all the exceptions are in this category.

sleep_between_tries (int): seconds to sleep between tries if the exception marking broken session/connection occurs.

max_tries (positive int): maximum number of tries for initialization or for run. If max_tries is 1, then initialization or run is tried only once even if the connection is broken.

prepare (callable): function to be called once initialization of the shell stack is ready.

finalize (callable): function to be called prior the terminal close. verify (callable): function to be called from auto_setup() for verifying session. Must raise an exception in broken_exceptions in case the session is broken.

initialize_if_needed()

Initialize the session in case verify fails or if the terminal is not initialized already.

initialize_terminal()

Initialize terminal connections.

set_verify(verify)

Set verify callable. If verify raises any broken_exceptions, the old session will be closed and the new session will be prepared in initialize_if_needed().

RunnerTerminal

class crl.interactivesessions.runnerterminal.RunnerTerminal

This is the Python terminal session wrapper for the transparent proxy instances: remoteproxies._RemoteProxy and remoteproxies._RecursiveProxy.

Note

This class has Python 2 dependencies.

add_handle_to_garbage(session_id, handle)

Adds crl.interactivesessions.remoteproxies._RemoteProxy handles (or the derivates of it) into garbage collection. The garbage is cleaned during executions when the treshold MAX_GARBAGE is exceeded.

assign_and_run_python(handle, cmd, timeout=None)

Runs a python expression cmd on the remote node and assigns then value to handle. If the value of the expression is an instance of any of the HANDLED_TYPES, then also the value is returned.

Note

The HANDLED_TYPES instances must be serializable and deserializable by picklers and unpicklers.

The return value of the remote call is returned as a python object with two attributes:

isobj: True if the value of the object is returned.

obj: The value of the expression.

If an exception is raised on the remote end, the traceback is logged, and the exception object is raised by this method.

close()

Close the terminal session.

Removes any handler modules from the remote end if necessary, and restores the terminal to the original state.

create_empty_recursive_proxy()

Creates remoteproxies._RecursiveProxy without handle or any other content. This proxy cannot be used but the content of it should be changed by calling remoteproxies._RemoteProxy.set_from_remote_proxy().

create_empty_remote_proxy()

Creates empty remoteproxies._RemoteProxy. See create_empty_recursive_proxy().

get_proxy_object(remote_object, local_spec)

Creates a proxy object for remote_object.

Args:

remote_object: the name of the remote object to proxy.

local_spec: a local class object, used to determine which methods

are available on the remote object. If None, the spec is determined dynamically.

Any method calls and attribute requests for the proxy object will be executed remotely on remote_object.

Calling as_recursive_proxy on the proxy object creates a new remoteproxies._RecursiveProxy for the proxied object.

Example:

>>> RunnerSession.run_python("a = open('test')")
>>> proxy = RunnerTerminal.get_proxy_object('a', file)
>>> proxy.readlines() # same as run_python("a.readlines()")

get_proxy_object_from_call(function_name, *args, **kwargs)

Calls a remote function, and proxies the return value.

The call is done as in run_python(), but the return value is a remoteproxies._RemoteProxy for the remote return value.

get_proxy_or_basic_from_call(function_name, *args, **kwargs)

Calls a remote function, and proxies the return value.

The call is done as in run_python(), but the return value is a remoteproxies._RemoteProxy for the remote return value in case of complex types. But in case the return value is of type int, float or is a string or None then the local copy is created and returned.

get_recursive_proxy(remote_object)

Creates a recursive proxy object for remote_object.

Args:

remote_object: the name of the remote object to proxy.

Any attributes, methods and return values produced by a recursive proxy will be recursive proxy objects wrapping that value. The proxied object can be retrieved by calling remoteproxies._RemoteProxy.as_local_value() on a remote proxy object. For more details, see remoteproxies._RecursiveProxy.

import_libraries(*imports)

Import the libraries given as arguments on the remote end.

initialize(session)

Initializes the terminal with session. The session has to be a wrapper to InteractiveSession.InteractiveSession which implements at least get_session method which returns the InteractiveSession.InteractiveSession instance.

iscallable(remote_object)

Determines whether remote_object is callable or not.

static isproxy(obj)

Returns True if obj is a remote proxy, else False.

run(cmd, timeout=-1, _rerun=False)

Run with error handling run of the session.

Note

It is recommended to use run_python() instead of this method.

run_and_return_handled_python(handle)

Get handle object associated with handle from the remote end in case the handle object is in HANDLED_TYPES

Note

The HANDLED_TYPES instances must be serializable and deserializable by picklers and unpicklers.

The return value of the remote call is returned as a python object with two attributes:

isobj: True if the value of the object is returned.

obj: The value of the expression.

If an exception is raised on the remote end, the traceback is logged, and the exception object is raised by this method.

run_python(cmd, timeout=None)

Runs a python expression on the remote node.

The return value of the remote call is returned as a python object.

If an exception is raised on the remote end, the traceback is logged, and the exception object is raised by this method.

run_python_call(function_name, *args, **kwargs)

Calls a python function on the remote end.

function_name: a string, describing what function to call.

Any additional arguments given will be passed as arguments to the function on the remote end.

The return value and any exceptions are handled as in run_python().

AutoRunnerTerminal

class crl.interactivesessions.autorunnerterminal.AutoRunnerTerminal

Bases: crl.interactivesessions.runnerterminal.RunnerTerminal

Python terminal session wrapper with the automated recovery feature for the session. This wrapper expects that the session is an instance of autorecoveringterminal.AutoRecoveringTerminal.

initialize(session, prepare=<function <lambda>>, finalize=<function <lambda>>)

This method initializes the terminal in the same fashion than initialize_with_shells() but instead of shells the underlying :class:.autorecoveringterminal.AutoRecoveringTerminal` instance should be given as a session.

Note

The autorecoveringterminal.AutoRecoveringTerminal.set_verify() must be called after this method as the verify is overridden by the default verify.

initialize_if_needed()

Initialize the terminal if necessary.

initialize_with_shelldicts(shelldicts, prepare=<function <lambda>>, finalize=<function <lambda>>)

This method initializes the terminal in the same fashion than initialize_with_shells() but instead of shells the dictionary of shells accepted by shells.shellstack.ShellStack.initialize() has to be to be given.

initialize_with_shells(shells, prepare=<function <lambda>>, finalize=<function <lambda>>)

Initialize the terminal with the InteractiveSession.Shell based shells stack or a single shell.

Note

The terminal session is not opened by this call. Either call initialize_if_needed() directly or use initially empty proxies. The empty proxies can be created via methods runnerterminal.RunnerTerminal.create_empty_remote_proxy() and runnerterminal.RunnerTerminal.create_empty_recursive_proxy()

The prepare callable should import any required modules and set the proxies. If the empty proxies are used in the initial setup then use remoteproxies._RemoteProxy.set_from_remote_proxy() to replace the content.

The finalize callable should do any special finalization of the proxies which are set in prepare. For example, if the proxy is managing a temporary file in the remote end, that could be removed in the finalize.

Note

Proxies are automatically invalidated after finalization without any need to do special operations in finalize method. After finalization runnerexceptions.InvalidProxySession will be raised in case the remote object is accessed via proxies.

The session is not guaranteed to be open when the finalize is called. That is why all possible exceptions are only logged.

RunnerTerminal Exceptions

exception crl.interactivesessions.runnerexceptions.InvalidProxySession

This exception is raised by remoteproxies._RemoteProxy methods in case the session identifier has been changed. This occurs for example in case the session is recovered but the proxy is not recovered.

exception crl.interactivesessions.runnerexceptions.RemoteTimeout(response_id)

Exception raised in case remote timeout occurs.

Args:

response_id: The response id to be used in remoteproxies._RemoteProxy.get_remote_proxy_response() call for retrieving the response later. The

response_wrap(callable): is called with the response and the return value of response_wrap is returned back to the remoteproxies._RemoteProxy.get_remote_proxy_response().

exception crl.interactivesessions.runnerexceptions.RunnerException

exception crl.interactivesessions.runnerexceptions.RunnerTerminalSessionBroken

Exception raised in case the RunnerTerminal.run() raises an exception. The original exception is embedded into the first argument of the new exception.

exception crl.interactivesessions.runnerexceptions.RunnerTerminalSessionClosed

Internal exception raised by RunnerTerminal in case RunnerTerminal.run_python() is called while the RunnerTerminal.close() is already called or alternatively if the session is not initialized.

exception crl.interactivesessions.runnerexceptions.RunnerTerminalUnableToDeserialize

Exception raised in case remote object returned is not deserialiazable by the local deserialization methods.

exception crl.interactivesessions.runnerexceptions.SessionInitializationFailed

This exception is raised by autorecoveringterminal.AutoRecoveringTerminal when the initialization of the terminal fails during autorecoveringterminal.AutoRecoveringTerminal.initialize_terminal(). The original exception is stored into the first argument of the exception.

Remoteproxies

class crl.interactivesessions.remoteproxies._RemoteProxy(session, remote_name, local_spec=None, parent=None, is_remote_owned=True)

Bases: object

Wrapper exposing a remote object as a local one.

Any attribute requests/assignments and method calls are transparently done on a remote object.

A new proxy object for remote_name is created.

Args:

session: a runnerterminal.RunnerTerminal instance.

remote_name: The name of the remote object to proxy.

local_spec: a local class object, used to determine which methods are available on the remote object. If None, the available methods are determined dynamically.

parent: a reference to parent proxy object of this proxy.

is_remote_owned: If True, then remote object is deleted as well when the proxy is removed from the Python interpreter.

as_local_value()

Returns a local copy of the proxied object.

as_recursive_proxy()

Returns a _RecursiveProxy for the proxied object.

get_proxy_handle()

Returns the remote name for the proxied object.

get_remote_proxy_response(remotetimeout, timeout=None)

Get remote respose when the call have been timed out. Blocks forever if no timeout in seconds is given. The remotetimeout is a handle which must be an instance of the exception runnerexceptions.RemoteTimeout.

remote_proxy_autoprepare()

Initialize the proxy session.

remote_proxy_use_asynchronous_response()

Set proxy to use asynchronous response without any timeout. In this mode, the proxy returns handle ( runnerexceptions.RemoteTimeout instance) which then should be passed to get_remote_proxy_response() to retrieve the actual value of the proxy call. Please see a complete example of the asynchronous mode usage from Running Commands In Background.

remote_proxy_use_synchronous_response()

Set proxy to use synchronous response and timeout

set_from_remote_proxy(proxy)

Set proxy content from proxy.

Warning

This method nulifies the proxy given as an argument. The reasoning is that it would be really dangerous to keep two proxies with the same handles simultaneously. The runnerexceptions.InvalidProxySession is raised when proxy is accessed after this call.

set_proxy_spec(local_spec)

Assigns a spec for this proxy object.

set_remote_proxy_timeout(timeout)

Set timeout for remote command execution. The negative timeout causes proxy to raise always runnerexceptions.RemoteTimeout. If None is set as timeout value, then the default timeout is used. The default timeout is 3600 seconds.

Note

The prompt_timeout of runnerterminal.RunnerTerminal is added automatically to timeout in case timeout is positive. If timeout is negative then the terminal uses prompt_timeout only.

class crl.interactivesessions.remoteproxies._RecursiveProxy(session, remote_name, parent=None, is_remote_owned=None)

Bases: crl.interactivesessions.remoteproxies._RemoteProxy

Wrapper exposing a remote object as a local one.

Any attributes, methods and return values produced by a recursive proxy will be recursive proxy objects wrapping that value. The exceptions are the basic types which are defined in runnerterminal.RunnerTerminal in HANDLED_TYPES attribute. The HANDLED_TYPES are returned as local value always. The proxied object can be retrieved by calling _RemoteProxy.as_local_value() on a remote proxy object.