Telerobot Software DocumentationJanuary 2004page 1
Description and Reference Information for Telerobot Software in Telelabs
James Trevelyan
January 2004
Updated August 2004 with movement logging functions
Summary
Andrew Babbage's thesis provides useful background information on the development of the telerobot, the various software systems that were developed to provide this world famous internet site, and finally the LabVIEW robot interface routines. In particular chapter 6 provides a detailed account of functions for handling frames and calculating wrist rotation angles. Chapter 7 provides a useful outline, particularly in the first part, to show how robot movements are generated. The section on Active-X event processing needed more details – see S4EventLogger.videscription below. Chapter 8 was presented in outline only and this detailed documentation provides what is missing there. Andrew's most lasting contribution was to sort out how Active-X controls can be used to link LabVIEW with the ABB Robot S4 controller package RobComm.
I took over at the final stage of commissioning the telelabs software system that Andrew had started: Andrew was too busy with final exam preparation. I had to revise the structure of the 'run hardware' level and had a frustrating time trying to work out why some status change events were never received. Once this was resolved the rest was routine and I largely followed the telelabs templates for the hardware and remote clients.
There are several significant deficiencies that need to be remedied in the short term.
When the robot was briefly made available for public use over the weekend of 19th December, one user managed to crash the robot into the table. I am not sure how. The main issue was that we had no log files that recorded the robot movements so they could be played back for checking.
We urgently need a system to make users responsible: a registration system, so that we can track users and deny access to troublesome ones.
Table of Contents
Upper Level Modules
Telerobot Client Master.vi
telerobot client Y Z controls.vi .
Internal Structure of Remote Client
Future Improvements
telerobot hw master.vi
Internal Structure
run hardware.vi
Event and Status Modules
S4EventLogger.vi
S4 Control States
S4 Operate States
S4CommandState.vi
S4CommandState.ctl
S4ResetState.ctl
S4Error.vi
Telerobot Support Modules
telerobot state translator.vi
Telerobot Exception Monitor.vi
Make Datalog Array.vi
telerobot set default controls.vi
record position.vi
record command.vi
Telerobot State Monitor Master.vi
Modules that have errors after editing on a machine with ABB RobComm
Run Hardware.vi
S4EventLogger.vi
S4RobPosData.vi
Other modules
Robot Control and Supporting Modules
LimitInputs.vi
Matrix2Quat.vi
Quat2Matrix.vi
GetRotAxisMatrix.vi
SpinTilt2Matrix.vi
Matrix2RPY.vi
InstructionList.vi
GridPlotX.vi and GridPlotY.vi
S4ActiveXControls.vi
S4Gripper.vi*
S4RobTarget.vi*
S4WriteMove.vi*
S4LoadAndRun.vi*
S4RobPosData.vi**
Movement Logging Functions
Open Movement Log.vi
Close Movement Log.vi
Record Movement Command.vi
Record Robot Position.vi
Record Movement Log.vi
Upper Level Modules
Telerobot Client Master.vi
The robot remote client (telerobot client master.vi) as shown here is an initial version to obtain some operating experience.
Spin is the rotation of the gripper about a local vertical axis. Given that the gripper is symmetrical, 180 degrees is sufficient for any desired gripper position. Care will be needed when handling blocks that are not gripped centrally, or have tilt angles. Multiple block manipulation may be necessary.
Gripper: Switch to set gripper for next move: gripper will open or close during the move. Therefore for precise control the gripper should be opened or closed, followed by pressing the MOVE button. The robot will not move, but the gripper will be opened or closed without the robot moving.
Move: Press to initiate a robot move.
Tilt is rotation about an axis running between the jaws of the gripper.
X: Movement left or right relative to the grid on the table as seen from main camera position.
Name to be attached to robot move: will be entered in records of movement.
Data reception error: data is broadcast from the DS server. If this data cannot be satisfactorily decoded this error will show. If the problem persists, QUIT and try again.
The panel goes with a companion telerobot client Y Z controls.vi which is a subVI to the telerobot client master.
telerobot client Y Z controls.vi .
This module provides Y and Z slider controls for the remote client in a separate window that can be positioned by the user alongside the remote video display of the robot. The front panel is displayed when the module is loaded and when called through the sub-VI node setup in the calling module.
As presently configured this module has a significant deficiency: the control values cannot be set except by user actions. This deficiency needs to be urgently addressed through correct design. The module was coded quickly in order to get the system working initially.
Y: Controls the Y position of the robot gripper: scale is in millimeters. Range is limited to prevent excursions beyond reasonable work table limits.
Z: Z coordinate range is restricted to minimize chance of damage to table or robot. Please be careful when bringing robot down on top of another block where the Z range will not prevent a collision.
Internal Structure of Remote Client
Refer to the manual 'Exploring Telelabs Clients' for a general discussion of the internal operation of both hardware and remote clients. The major simplification here is that there is no display of real time data on a moving chart display as there is with other typical experiments. This removes a number of important issues. However, the controls are more complicated, and will become more so in future editions of the client. I was quite satisfied that the basic telerobot client required very little work to implement within the framework of the telelabs standard remote client structure.
The current layout is rather cluttered and represents a hurried session to get something working – to be improved. Only non-standard features are described below.
[1] The sub-vi node setup for telerobot client Y Z controls.vi is important (right click icon) and specifies that the panel is to be opened on loading, and when called. It will always appear above the client master panel if the two overlap each other.
[2] For some reason spin and tilt have to be wired like this: tilt first, then spin, then zero. In Andrew's code the order was spin, tilt, zero. I am not sure why. Perhaps Andrew made an error. This needs to be understood by tracing through Andrew's robot control modules.
[3] The control settings (on a Move) are combined into a single cluster which is added to an array of clusters. This array is combined with the disable Boolean (a standard feature of the system) to make up the standard telelabs controls cluster that is passed to the hardware client through the LOL server when the user makes a move. This arrangement makes it possible to send (at some stage in future) a series of moves to be executed sequentially. The disable control is supposed to shut down communication with the robot. However it does not work, either from the remote client or using local controls in the hardware client. The reason was unclear but the analysis below reveals why.
[4] Display of current robot state in an understandable form.
[5] Here the front panel for the Y and Z sliders is closed: otherwise it would remain open after the client window closes.
[6] The simulation case (True) is blank: there is no provision for a practice task at the moment. A future challenge! It would be important here to avoid top-heavy code with too much complexity in the simulator. Or perhaps a separate down-loadable simulator application instead. To be decided.
Future Improvements
In the short term, the main improvement is to monitor the robot state and update the position and spin/tilt controls when Move is 'False' and the robot state is not 'Executing'. This should only be done once after each move….. to be implemented with great care and consideration.
As mentioned above, it should be possible to pre-set the Y and Z controls correctly. A 'mode' input could be used. When "set" the module takes values and assigns these external values to local variable copies of the Y and Z sliders. When "get", it operates as it does now.
Andrew's excellent set of functions need to be completed with proper forward and inverse kinematics modules. The former is relatively trivial. The latter requires careful thought and considerable experimentation. This is because the robot has multiple joint positions that satisfy some Cartesian space positions, partly because the robot can have different arm configurations, and partly because the wrist joints can move more than 360 degrees.
The next significant improvement is to provide an interlock mechanism for collaborating users. This will allow one user to take control of the robot and retain control, blocking out other users. Note that the user access type is available: a STAFF or TUTOR user could arbitrarily take control. I have designed, but not yet documented, a simple token algorithm to implement this without any internal changes to the telelabs system. One or two token keys needed to be added to the controls cluster, and a token display value to the state cluster.
Another improvement would be to implement an augmented reality display for the remote client along the lines suggested by Friz in 1998 and implemented by Dalton in the Java interface. This would require an image of the robot work area to be displayed: another significant implemention issue. So far we have side-stepped this by using the (very good) Axis network camera sending streaming video to an internet explorer client window.
A further improvement, based on this, would be to implement an augmented reality display of blocks along the lines suggested by Rowe Palmer and implemented by him in off-line form with his Mechatronics Design 310 group in 2003. This would require some form of centrally shared database of block positions.
All these extensions will make interesting student projects in years ahead.
telerobot hw master.vi
Internal Structure
Refer to the manual 'Exploring Telelabs Clients' for a general discussion of the internal operation of both hardware and remote clients. The major simplification here is that there is no display of real time data on a moving chart display as there is with other typical experiments. This removes a number of important issues. However, the controls are more complicated, and will become more so in future editions of the clients. I was quite satisfied that the basic telerobot client required very little work to implement within the framework of the telelabs standard hardware client structure.
The current layout is rather cluttered and represents a hurried session to get something working – to be improved. Only non-standard features are described below.
[1] Four Active-X control references are stored here for use elsewhere in the code. The active-X control RimbaseLib.Helper must be displayed on the front panel (not hidden) (note – it is OK to place it on an obscure pane of a tab control). Andrew did not realise he could handle the reference pointers like this. Having done this we could retrieve pointers using ActiveXControls.virather than wiring them through the code like we have done (so far).
[2] In the remote client I found it necessary to wire spin and tilt like this: tilt first, then spin, then zero. In Andrew's code the order was spin, tilt, zero and the hardware client reflects this. I am not sure why. Perhaps Andrew made an error. This needs to be understood by tracing through the robot control modules.
[3] The control settings (on a Move) are combined into a single cluster which is added to an array of clusters. This array is combined with the disable Boolean (a standard feature of the system) to make up the standard telelabs controls cluster that is passed to the hardware client through the LOL server when the user makes a move. This arrangement makes it possible to send (at some stage in future) a series of moves to be executed sequentially. The disable control is supposed to shut down communication with the robot. However it does not work, either from the remote client or using local controls in the hardware client. The reason was unclear but the analysis below reveals why this happens.
[4] Large 'moving' and 'overload' indicators.
The flow of the controls cluster is shown on the diagram below. I obtained this by carefully following the code in the hardware master and found the probable reason for the disable control not working: the arrangement is different to other hardware clients. The controls cluster returns to the local controls copy via a different route.
This must be changed. The reason is simple. If the controls cluster has one element, A and A has been zero for some time, then the monitor shift register (used to transfer data to the real time thread) will be zero. If the local user sets A to '1' (in local mode), then this value gets passed to the input shift register. However, at the same time, the zero value is retrieved from the output shift register and this is then recirculated back to the local controls through the local variable copy. This will erase the new '1' value.
Notice that the local front panel controls cluster in the hardware client serves as the current memory of control settings when no new settings are received from remote clients.
The only reason for setting out the code like this was space convenience in the telelabs hardware master diagram. By re-arranging the hardware client diagram the desired flow path can be restored.
Other features have been shown here.
First a command queue in the real time thread is used to store multiple commands to be passed to the robot controller when it is ready to accept them. Second a token lock module monitors user requests and generates a token status value in the state cluster (not shown). Only robot commands that have a token value matching the current token status will be placed in the command request queue. The current command queue could optionally be broadcast to all clients through the state cluster.
run hardware.vi
The robot code was based on original code by Andrew Babbage and modified by myself at the commissioning stage.
Notes:
[1] We retrieve the Active-X reference pointers from S4ActiveXControls for local use. The Active-X controls must be on the application front panel but they are needed here at the real time thread.
[2] We initialise S4CommandState.
[3] Here the interface state property is obtained from the Active-X Helper control. Note that if the code is edited on a computer without the ABB RobComm software installed the property value will have to be re-selected on a machine that does have RobComm before it can be used. The output Boolean 'Robot Interface OK' is retrieved here. Actually this should be retrieved in the default case so it is read every time step. It is inserted in the 'power' value of the state cluster. If this value is 'false' the system will revert to a power off and waiting for power state. The names might be confusing but the actions are appropriate.
[4] Here we retrieve the current robot position from the controller. However this seems to be a temporary display and not linked to anything in particular.
[5] Start automation handlers for two of the active X refnums. Why these two have to have this treatment and not the others I am not sure.
[6] We initialise the event queues for the Helper control: these event queues provide state change indications.
[7] On shutdown we close the Active-X controls – again why these two and not the others is not clear.
[8] Here we destroy the Active-X event queues.
[10] Here we call S4EventLogger to see whether a state change has happened: we can display the current S4 control state and operate state for reference, and pass these to S4CommandStatus to check for the current overall robot state. Status indications pass to Telerobot Exception Monitor for notifying faults and error conditions.
[11] Here a movement command is taken from the controls cluster, when the robot command state is 'S4Cmd Start': in this command state the software is judged to be ready to accept a new request. The sequence of S4 robot module calls is derived from Andrew Babbage's software exactly. The only addition is a call to a small module to record the command in circular buffer 3 (front panel text display).
[12] Here, when the robot command state is 'S4cmd Completed' we request the current robot position and orientation from the S4 controller and display this in the state cluster. The position, orientation and gripper status is also stored in local indicators. The sequence of S4 robot module calls is derived from Andrew Babbage's software exactly. The only addition is a call to a small module to record the position and orientation in circular buffer 3 (front panel text display).
[13] When the robot command state is not 'S4Cmd Completed' the stored values are retrieved from local variable copies of indicators and provided for the state cluster.