uwlab.devices

uwlab.devices#

Sub-package providing interfaces to different teleoperation devices.

Currently, the following categories of devices are supported:

Se3Keyboard: Standard keyboard with WASD and arrow keys.
RealsenseT265: Realsense cameras 6 degrees of freedom.
RokokoGlove: Rokoko gloves that track position and quaternion of the hand.

All device interfaces inherit from the isaaclab.devices.DeviceBase class, which provides a common interface for all devices. The device interface reads the input data when the DeviceBase.advance() method is called. It also provides the function DeviceBase.add_callback() to add user-defined callback functions to be called when a particular input is pressed from the peripheral device.

Classes

`Se3Keyboard`	A keyboard controller for sending SE(3) commands as absolute poses and binary command (open/close).
`RealsenseT265`
`RokokoGlove`	A Rokoko_Glove controller for sending SE(3) commands as absolute poses of hands individual part This class is designed to track hands and fingers's pose from rokoko gloves.

Keyboard#

class uwlab.devices.Se3Keyboard[source]#

Bases: Se3Keyboard

A keyboard controller for sending SE(3) commands as absolute poses and binary command (open/close).

This class is designed to provide a keyboard controller for a robotic arm with a gripper. It uses the Omniverse keyboard interface to listen to keyboard events and map them to robot’s task-space commands. Different from Isaac Lab Se3Keyboard that adds delta command to current robot pose, this implementation controls a target pose which robot actively tracks. this error corrective property is advantageous to use when robot is prone to drift or under actuated

The command comprises of two parts:

absolute pose: a 6D vector of (x, y, z, roll, pitch, yaw) in meters and radians.
gripper: a binary command to open or close the gripper.

Key bindings:

Description	Key (+ve axis)	Key (-ve axis)
Toggle gripper (open/close)	K
Move along x-axis	W	S
Move along y-axis	A	D
Move along z-axis	Q	E
Rotate along x-axis	Z	X
Rotate along y-axis	T	G
Rotate along z-axis	C	V

Real Sense#

class uwlab.devices.RealsenseT265[source]#

Bases: DeviceBase

Methods:

`__init__`(cfg[, device])
`advance`()	Advance the device state, read the device translational and rotational data, then transform them xyz rpy that makes intuitive sense in Isaac Sim world coordinates.
`__new__`(cls, args, *kwargs)

__init__(cfg: RealsenseT265Cfg, device='cuda:0')[source]#

advance()[source]#: Advance the device state, read the device translational and rotational data, then transform them xyz rpy that makes intuitive sense in Isaac Sim world coordinates.

static __new__(cls, *args: Any, **kwargs: Any) → Any#

Rokoko Gloves#

class uwlab.devices.RokokoGlove[source]#

Bases: DeviceBase

A Rokoko_Glove controller for sending SE(3) commands as absolute poses of hands individual part This class is designed to track hands and fingers’s pose from rokoko gloves. It uses the udp network protocol to listen to Rokoko Live Studio data gathered from Rokoko smart gloves, and process the data in form of torch Tensor. Addressing the efficiency and ease to understand, the tracking will only be performed with user’s parts input, and all Literal of available parts is exhaustively listed in the comment under method __init__.

available tracking literals:

LEFT_HAND:: leftHand, leftThumbProximal, leftThumbMedial, leftThumbDistal, leftThumbTip, leftIndexProximal, leftIndexMedial, leftIndexDistal, leftIndexTip, leftMiddleProximal, leftMiddleMedial, leftMiddleDistal, leftMiddleTip, leftRingProximal, leftRingMedial, leftRingDistal, leftRingTip, leftLittleProximal, leftLittleMedial, leftLittleDistal, leftLittleTip
RIGHT_HAND:: rightHand, rightThumbProximal, rightThumbMedial, rightThumbDistal, rightThumbTip rightIndexProximal, rightIndexMedial, rightIndexDistal, rightIndexTip, rightMiddleProximal, rightMiddleMedial, rightMiddleDistal, rightMiddleTip rightRingProximal, rightRingMedial, rightRingDistal, rightRingTip, rightLittleProximal, rightLittleMedial, rightLittleDistal, rightLittleTip

Methods:

`__init__`(cfg[, device])	Initialize the Rokoko_Glove Controller.
`reset`()	Reset Internal Buffer
`advance`()	Provides the properly scaled, ordered, selected tracking results received from Rokoko Studio.
`debug_advance_all_joint_data`()	Provides the properly scaled, all tracking results received from Rokoko Studio.
`__new__`(cls, args, *kwargs)

__init__(cfg: RokokoGlovesCfg, device='cuda:0')[source]#

Initialize the Rokoko_Glove Controller. Be aware that current implementation outputs pose of each hand part in the same order as input list, but parts come from left hand always come before parts from right hand.

Parameters:

UDP_IP – The IP Address of network to listen to, 0.0.0.0 refers to all available networks
UDP_PORT – The port Rokoko Studio Live sends to
left_hand_track – the tracking point of left hand this class will be tracking.
right_hand_track – the tracking point of right hand this class will be tracking.
scale – the overall scale for the hand.
proximal_offset – the inter proximal offset that shorten or widen the spread of hand.

reset()[source]#: Reset Internal Buffer

advance()[source]#

Provides the properly scaled, ordered, selected tracking results received from Rokoko Studio.

Returns:: A tuple containing the 2D (n,7) pose array ordered by user inputted joint track list, and a dummy truth value.

debug_advance_all_joint_data()[source]#

Provides the properly scaled, all tracking results received from Rokoko Studio. It is intended to use a debug and visualization function inspecting all data from Rokoko Glove.

Returns:: 0-21, right:21-42), and a dummy truth value.
Return type:: A tuple containing the 2D (42,7) pose array(left

static __new__(cls, *args: Any, **kwargs: Any) → Any#

`__init__`(cfg[, device])	Initialize the keyboard layer.
`advance`()	Provides the result from internal target command modified by keyboard event.
`__new__`(cls, args, *kwargs)