In custom charts you can use operators like + - * / to do basic math on different parts of module feedback. The example below creates a plot that calculates the position error, which can often be useful when tuning or evaluating the performance of a system:

The expression parser uses a Java compiler, so you may make use of math functions such as pow or sqrt. The example below calculates the magnitude of all three components of the accelerometer of a module.

Additional lines can be added with addititional <trace> children. You can also access the previous feedback to perform diffs over a single timestep. The example below numerically differentiates the commanded position and compares it to the commanded velocity.

Gains can be saved in a group-compatible gain format by selecting one or more devices and pressing the save to file button. The gains of multiple devices will be stored in the same order as they appear on the device pane.

Load file reads a gain file and loads the values into the corresponding input fields. Only one set of gains can be displayed at a single time.

Send File loads the gains of an arbitrary number of modules and immediately sends them to the selected devices, without populating the input fields. The order will again match the order in the device pane.

In many cases it is useful to reset or compare the active gains to the default values for an actuator. An easy way to do this is by using Load Defaults, which downloads the appropriate default gains for the selected hardware type and control strategy. Differences compared to the active gains are highlighted in yellow.

Windows Subsystem for Linux v2 (WSL2) lets Windows users run a GNU/Linux environment that is very tightly integrated with the host OS. While the Linux versions of the HEBI APIs and tools can run without modification, the environment runs inside a virtual machine that is on a separate network and prevents any devices from being seen.

Until WSL2 adds features to expose raw network interfaces, this limitation can be worked around by (1) launching the VPN Gateway on the Windows side,

and subsequently (2) adding the shown IP address on the WSL2 Linux side.

When you run code on WSL2 you can expect the frequency to track significantly better (green dots) at the cost of slightly higher latency (blue dots) due to the additional indirection. The charts below show latency plots running simultaneously on the same computer on Windows (left) and Ubuntu 20.04 on WSL2 (right):

HEBI Camera Modules use RTSP to stream the camera feed over the network connection. In order to visualize the camera stream, determine the IP Address of the RPi in the Camera Module through Scope as shown below.

Once you have located the RPi IP Address, use it with the following command:

Running this command will generate a .stream file that scope can use to view the stream with the View Stream button.

There are also switches in the app to enable/disable GPS location feedback and ARCore pose feedback.

In iOS the name and family of the mobile device can be set in the settings panel for the app, found going to iOS Settings, and scrolling down the app list until you see Mobile I/O. This name and family will be how the device appears in Scope and the APIs. You can also change the layout of the iOS app to an alternative portrait Sliders view, instead of the default landscape Joystick layout. Finally, you can select whether or not the acceleration feedback that is returned from the application includes the acceleration due to gravity, or filters this information out.

Where applicable, the reference frame for the feedback from the device follows the convention of the iOS Core Motion framework (see image below).

Devices that belong to the same system can be combined into a HebiGroup. Groups represent the basic way to send commands and retrieve feedback. They provide convenient ways to deal with devices, and handle high-level issues such as data synchronization and logging. Devices can be identified by user settable parameters, such as their name or family, or hardware constants, such as their serial number or mac address.

A comprehensive list of all available calls for selecting devices can be found in the HebiLookup API.

Position, velocity, and torque (effort) can be commanded by using a CommandStruct. Each field expects a 1xN vector where N is the number of devices within a group. Leaving a field empty or setting a value within a vector to NaN corresponds to turning the corresponding control loop off.

Commands are only valid for a limited CommandLifeTime, so even non-changing commands need to be re-sent before they expire. This safety measure may seem inconvenient at first, but it actually simplifies development due to the fact that motions automatically stop after halting a script (ctrl-c).

The following example shows an open-loop controller commanding sine waves with different frequencies on two actuators using simultaneous position and velocity control.

The IoCommandStruct serves as a similar structure to command the outputs on an I/O board. Note that for I/O boards we found it to be more maintainable to create a mapping and to access the individual pin fields by string as shown below.

Each group continuously gathers sensor feedback from all contained devices in a background thread, and synchronizes the data into a single struct. MATLAB can then asynchronously access the received feedback with getNextFeedback. Each call returns the next new (not previously accessed) synchronized feedback. Thus, a single feedback response will never be returned more than once. The default rate of 100Hz can be changed via setFeedbackFrequency or by modifying the hebi_config.m file.

Our devices tend to return a large number of sensors, so we have added different 'views' in order to enable advanced use cases without confusing new users. The default view returns feedback of the most used types of sensors, but there are alternative views in case you’d like to access additional information such as hardware timestamp data.

The below example shows a closed-loop controller that implements a virtual spring that controls torque (effort) to drive the output towards the origin (stiffness = 1 Nm / rad).

While groups are completely independent of each other, there may be more than one group at a time, and multiple groups may be used together in any way. For example, the following code combines feedback from one group (gyroscope input) to calculate commands for a second group (velocity output). This is similar to the demo shown in Controlling Robots Using Android and MATLAB.

Splitting a system into multiple groups often allows for a cleaner separation between the various elements, e.g., a 6-DoF arm (group 1) and a gripper (group 2).

The send method additionally supports various other commands such as programmatically setting names, resetting devices, setting safety limits, and setting LED colors.

LEDs can for example be used to provide live feedback to people standing next to a robot, or for synchronizing/debugging motions in combination with a high-speed camera.

The API has been developed with low-latency, and low memory usage in mind. For applications that require determinism or high control frequencies, we recommend the following additional steps.

Most configurations are serial chains such as the 6-DoF X-Series Manipulator kit A-2085-06G:

The file can be loaded from disk using the loadHRDF method. The resulting HebiKinematics object consists of multiple bodies that each represent either a rigid link or a dynamic joint element. The first body starts at the baseFrame and the last body represents the endEffector. The firstJointFrame is the combination of the baseFrame and all static bodies up until the first joint. Details about an existing configuration can be accessed at runtime using getBodyInfo for general body information, and getJointInfo for joint specific information.

Some kits such as Daisy (A-2049-01) consist of multiple limbs that are attached to a shared chassis but effectively act independent of each other:

Tree structures defined in a single HRDF can be parsed into a cell array of separate HebiKinematics objects representing each limb. Each one refers to the serial chain from the base frame to the desired endEffector indexed as they occur in the file.

The additionally returned jointIndices list the actuator indices that are used in the chain. For example, Daisy consists of 18 actuators total, and leg 4 uses actuators 10-12.

The getForwardKinematics method computes the poses of the chain of bodies in the base frame, using specified values for the joint parameters.

Poses are returned as a set of [4 x 4 x numBodies] homogeneous transforms, specified in the world frame of the kinematic configuration. Units of XYZ translation are in [m].

The getJacobian method calculates the partial derivatives of the kinematics equation, which relates the joint rates to the linear and angular velocity of each body in the kinematics configuration.

The Jacobian is returned as a [6 x numDoF x numBodies] set of matrices. Rows 1:3 of the Jacobian correspond to linear velocities [m/s] along the X-Y-Z axes in the world frame, while rows 4:6 correspond to rotational velocities [rad/s] about the X-Y-Z axes in the world frame.

The call parameters are the same as for forward kinematics shown above.

The getInverseKinematics method computes the joint positions associated to a desired end-effector configuration. The end effector is assumed to be the last body in the kinematic chain. There are a variety of optimization criteria that can be combined depending on the application. Available parameters include:

Dynamics and inverse dynamics are important concepts that describe the relationship between accelerations and forces. Modeling required torques appropriately can significantly improve tracking while also improving stability. Please take a look at the Motion Control tutorial for an in-depth explanation with visual examples.

Additionally, we provide HebiArmPlugin for abstracting commonly used functionality. Plugins are called at the end of each update and may modify the state in arbitrary ways. Below are some of the built-in examples:

Joint trajectories can be created using the newJointMove method. It computes minimum jerk trajectories that go through the specified waypoints (position, velocity, and acceleration) at the specified time. Unspecified velocity and acceleration constraints default to zero start and end conditions with unconstrained waypoints in between. The following code creates a sample trajectory between 6 position waypoints for a single actuator.

The trajectory can be evaluated using getState.

You can visualize the trajectory using the provided HebiUtils.plotTrajectory utility

Alternatively, you can create a custom visualization as shown below.

We additionally provide a heuristic for creating the time vector. Note that the current implementation is very basic and only takes into account velocity limits with an assumption that the velocity at each waypoint is zero.

The velocity limits can be specified by passing in velocity limits or a HebiKinematics object:

The getWaypointTime method provides access to the determined time vector:

Creating the time vector can be annoying, so we have added a heuristic based on velocity limits. Note that the current heuristic is very simple and only

In order to simplify the generation of the time vector, we have added a simple heuristic based on velocity limits.

Minimum-jerk joint trajectories generate smooth human-like motion, but the end effector generally does not move linearly through space.

Linear cartesian motions can be generated by using inverse kinematics to sample desired joint positions along a linear path between the points. For example, interpolating the above joint motion in 25mm intervals yields the approximately linear motion below:

The details depend on the kinematic configuration and are typically application specific. We provide newLinearMove to serve as a starting point.

Trajectories are designed to be executed/evaluated in a non-blocking manner so that users can dynamically react to changes in the environment such as hitting unforeseen obstacles or when following a moving target.

More information on various options, such as

can be found in the API documentation for HebiTrajectoryGenerator and newJointMove.

Logs are returned as structures where data is organized in vectors and matrices so that the resulting data interacts well with MATLAB’s built-in plotting tools.

Additionally, we provide HebiUtils.plotLogs to help with basic feedback plots.

All logging happens in the background and gets stored on disk in a binary streaming format. The default stopLog() call converts the binary format into another format that MATLAB can work with. We also provide the convertGroupLog() utility to load raw log files that were previously saved or generated using other APIs.

The same utility also useful for splitting logs into e.g. 1 hour chunks in a long running process. The conversion would be too expensive to do inside a loop, so instead we can call startLog to finish the previous log and to start a new one. This is comparatively very cheap and may be called inside a tight loop.

The resulting log files can then be loaded once a task is done, or from a different MATLAB instance while the task is still running.

We also added the newGroupFromLog utility which supports replaying log files via the standard group interface. After loading the log, each call to getNextFeedback on the group returns the next feedback in sequence (including the logged timestamp), regardless of the rate it is actually being called. This is useful, for example, for online algorithm development while working with offline data.

From the command line, you can download the API from pip:

Most modern IDEs (e.g., PyCharm) have pip integration by default. You can use the provided facilities to find the hebi-py package and install it.

Some Linux distributions prefer you to install numpy through their package manager. For example, Ubuntu uses the package python3-numpy; Red Hat/CentOS/Fedora use python3-numpy.

For some Linux environments, you may not be able to easily install matplotlib because you do not have a desktop environment (e.g., running a server Linux install on a Raspberry Pi). For such environments, you cannot use the plotting functions under hebi.util, but the rest of the API will not be affected.

A Lookup object can create a Group instance. A group is a collection of modules (often part of the same robotic system), and the group interface represents the basic way to send commands and retrieve feedback. They provide convenient ways to deal with modules and handle high-level issues such as data synchronization and logging. Modules can be identified by user settable parameters, such as name or family, or hardware constants, such as their mac address.

The Lookup class documentation provides a comprehensive set of functions used to create group objects.

Position, velocity, effort, and many other fields can be commanded by using the Command API. A GroupCommand object is used to store the commands to be sent to the group. We use types from the numpy library to update many of the fields with vectors of data.

For commands that are only sent once (such as setting gains on the modules), we recommend usage of Group.send_command_with_acknowledgement to verify that the command was received.

If you want to completely cancel out a previous command (especially when the Command Lifetime is not set!) and turn the control loops off, you can set the commanded value to nan. A few ways to get a nan in Python are:

The individual Command elements of a GroupCommand object can also be accessed and updated individually. Note that you cannot create an instance of Command - you only retrieve references through a GroupCommand. The Command object provides the same interface as a GroupCommand, but with all of the fields returning a scalar as opposed to a vector or matrix. As an example, updating the name of the module in the following example:

A group is responsible for gathering sensor feedback from all contained modules, and synchronizing these into a single GroupFeedback structure. By setting the group’s feedback frequency (set Group.feedback_frequency), a background thread will request and synchronize this feedback at a specified rate. The default feedback rate is 100Hz.

To access this feedback, there are three options:

The robot_model.RobotModel class is used to store how different rigid bodies and joints are attached together. There are helper functions to add standard HEBI components to the robot model.

The first body added to the RobotModel object represents the base and the last body represents the end (often an end effector) of the kinematic chain.

Robot configurations can be loaded from XML files using the HEBI Robot Description Format (HRDF) or programmatically using the API. It is strongly recommended that you use the HRDF toolset, and load your robot configuration in your code as follows:

You can also build your robot component by component in your code by specifying 3 types of components: Actuator, Link, and Bracket. A complete list of these methods is located here.

These methods and their named parameters correspond with kinematic types given in the Kinematic Body Types section. In addition to standard HEBI components, you can use the RobotModel.add_rigid_body and RobotModel.add_joint functions to add any desired masses and degrees of freedom to your system as well.

The get_forward_kinematics method computes the poses of each body in the RobotModel object, using specified values for the joint parameters. Using the frame_type parameter, one can choose either the center of mass or the output frame for each kinematic body.

The resulting 4x4 homogeneous transforms are specified in the world frame of the RobotModel object. This world frame defaults to the input frame of the base module, although a transformation can be prepended using the base_frame field. Units of translation are in meters.

The RobotModel.get_end_effector method is a shortcut for just obtaining the output transform of the last frame in the RobotModel object:

We can also use the RobotModel object to obtain the Jacobian, or the partial derivatives of the forward kinematics equation. This relates the joint velocities to the linear and angular velocity of each body in the system.

The Jacobian in returned as a set of matrices, one for each body in the RobotModel object. Each matrix is of size 6 x numDoF. Rows 1 - 3 of this matrix corresponds to this body’s linear velocities [m/s] along the X-Y-Z axes in the world frame, while rows 4 - 6 correspond to rotational velocities [rad/s] about the X-Y-Z axes in the world frame. The columns define the contributions to these linear and rotational velocities that would be a result of the respective degree of freedom instantaneously moving at unit velocity (e.g., a rotational joint being driven at 1 rad/s).

As with the forward kinematics, we provide versions for the end effector (which just returns a since 6 x numDoF matrix) as well as the entire system.

Full API documentation of these functions is available here.

The RobotModel object also provides a method for computing joint angles that solve the inverse kinematics, according to a number of objectives and constraints.

Note that the implementation of this function relies on a local optimization, and so is highly dependent on the initial joint angles that are provided as the seed for the search.

The desired objectives and constraints should be passed to the RobotModel.solve_inverse_kinematics function as a series of arguments - any number are permitted, but there are limitations in which can be used together.

As an example, the solve_inverse_kinematics function is used to find joint angles that move the end effector to a given position:

Full API documentation of these functions is available here.

Currently, computation of torques for gravity compensation is not supported directly in the API, but our examples repository provides a utility function and examples for adding this into your own system. This is subject to change and inclusion directly in the API in the future. This example in c++ demonstrates sending torques to compensate for the effect of gravity for a 3 DOF arm.

The Trajectory class is used to generate and access trajectory motions. To create a trajectory object, you must define the waypoints and time at which the robot should pass through each waypoints. Note that the waypoint velocity and acceleration constraints are optional, but it is often useful to at least define the starting and ending velocity and acceleration to be zero. If these are not given, it is left up to the internal implementation to determine values for these.

Below, we demonstate creation of a trajectory for 2 modules over 5 waypoints, using the unconstrained quadratic programming solver:

To play through the trajectory object, we query the position, velocity,' and acceleration at each point in time, and send appropriate commands to the robot:

Note that acceleration can not be (meaningfully) directly sent to the actuator, but with knowledge of the dynamics this can be converted to a torque or force to compensate for this acceleration. For improved control, we recommend adding effort commands that are the summation of torques/forces that compensate for gravity and torques/forces that compensate for this acceleration.

A Lookup class can create a hebi::Group instance. A group is a collection of modules (often part of the same robotic system), and the group interface represents the basic way to send commands and retrieve feedback. They provide convenient ways to deal with modules and handle high-level issues such as data synchronization and logging. Modules can be identified by user settable parameters, such as name or family, or hardware constants, such as their mac address.

The Lookup class documentation provides a comprehensive set of functions used to create group objects.

Position, velocity, effort, and many other fields can be commanded by using the Command API. A hebi::GroupCommand object is used to store the commands to be sent to the group. We use types from the open-source C++ matrix library Eigen to update many of the fields with vectors of data.

The individual hebi::Command elements of a GroupCommand object can also be accessed and updated individually. Note that you cannot create an instance of Command - you only retrieve references through a GroupCommand. The hebi::Command object provides a hierarchical message that provides access to all of the available options that can be sent through the API — for example, updating the name of the module in the following example:

The Command class documentation provides complete documentation of the Command class fields.

A group is responsible for gathering sensor feedback from all contained modules, and synchronizing these into a single Group Feedback structure. By setting the group’s feedback frequency (using Group::setFeedbackFrequencyHz), a background thread will request and synchronize this feedback at a specified rate. The default feedback rate is 100Hz.

To access this feedback, there are three options:

The hebi::robot_model::RobotModel class is used to store how different rigid bodies and joints are attached together. There are helper functions to add standard HEBI components to the robot model.

The first body added to the hebi::robot_model::RobotModel object represents the base and the last body represents the end (often an end effector) of the kinematic chain.

Robot configurations can be loaded from XML files using the HEBI Robot Description Format (HRDF) or programmatically using the API. It is strongly recommended that you use the HRDF toolset, and load your robot configuration in your code as follows:

You can also build your robot component by component in your code by specifying 3 types of components: Actuator, Link, and Bracket. A complete list of these methods is located here.

These methods and their named parameters correspond with kinematic types given in the Kinematic Body Types section. In addition to standard HEBI components, you can use the addRigidBody and addJoint functions to add any desired masses and degrees of freedom to your system as well.

The RobotModel::getForwardKinematics (or aliased RobotModel::getFK) method computes the poses of each body in the RobotModel object, using specified values for the joint parameters. Using the HebiFrameType parameter, one can choose either the center of mass or the output frame for each kinematic body.

The resulting 4x4 homogeneous transforms are specified in the world frame of the RobotModel object. This world frame defaults to the input frame of the base module, although a transformation can be prepended using the setBaseFrame function. Units of translation are in meters.

A hebi::robot_model::Matrix4dVector object is passed in to the getFK function, and is resized as necessary and filled in with the computed transforms (in order of the bodies in the RobotModel object). This type is simply a std::vector containing Eigen::Matrix4d elements; due to memory alignment considerations Eigen types in std containers must use particular allocators, and so we provide this convenience typedef. (See https://eigen.tuxfamily.org/dox-devel/group__TopicStlContainers.html for more information on this requirement)

The RobotModel::getEndEffector method is a shortcut for just obtaining the output transform of the last frame in the RobotModel object:

Full API documentation of these functions is available here.

We can also use the RobotModel object to obtain the Jacobian, or the partial derivatives of the forward kinematics equation. This relates the joint velocities to the linear and angular velocity of each body in the system.

The Jacobian in returned as a set of matrices, one for each body in the RobotModel object. Each matrix is of size 6 x numDoF. Rows 1 - 3 of this matrix corresponds to this body’s linear velocities [m/s] along the X-Y-Z axes in the world frame, while rows 4 - 6 correspond to rotational velocities [rad/s] about the X-Y-Z axes in the world frame. The columns define the contributions to these linear and rotational velocities that would be a result of the respective degree of freedom instantaneously moving at unit velocity (e.g., a rotational joint being driven at 1 rad/s).

As with the forward kinematics, we provide versions for the end effector (which just returns a since 6 x numDoF matrix) as well as the entire system. Note that the matrices are now of type Eigen::MatrixXd and MatrixXdVector.

Full API documentation of these functions is available here.

The RobotModel object also provides a method for computing joint angles that solve the inverse kinematics, according to a number of objectives and constraints. Note that the implementation of this function relies on a local optimization, and so is highly dependent on the initial joint angles that are provided as the seed for the search.

The desired objectives and constraints should be passed to the solveIK function as a series of arguments; any number are permitted, but there are limitations in which can be used together. In future API releases, the IKResult return structure will contain more information about the optimization result.

Here is an example of using the solveIK function to find joint angles that move the end effector to a given position:

Full API documentation of these functions is available here.

Currently, computation of torques for gravity compensation is not supported directly in the API, but our examples repository provides a utility function and examples for adding this into your own system. This is subject to change and inclusion directly in the API in the future. This example demonstrates sending torques to compensate for the effect of gravity for a 3 DOF arm; a simplified version is reproduced below.

The hebi::trajectory::Trajectory class is used to generate and access trajectory motions. To create a trajectory object, you must define the waypoints and time at which the robot should pass through each waypoints. Note that the waypoint velocity and acceleration constraints are optional, but it is often useful to at least define the starting and ending velocity and acceleration to be zero. If these are not given, it is left up to the internal implementation to determine values for these.

Below, we demonstate creation of a trajectory for 2 modules over 5 waypoints, using the unconstrained quadratic programming solver:

To play through the trajectory object, we query the position, velocity, and acceleration at each point in time, and send appropriate commands to the robot:

Note that acceleration can not be (meaningfully) directly sent to the actuator, but with knowledge of the dynamics this can be converted to a torque or force to compensate for this acceleration. For improved control, we recommend adding effort commands that are the summation of torques/forces that compensate for gravity and torques/forces that compensate for this acceleration (an example of this is coming soon).

Scope, as well as the C, C++, Python, and MATLAB APIs all have tools to read and write this gain format.

In the Scope GUI, the currently active gains can be saved to a file by using the "Save to File" button on the "Gains" tab, and the gains in a file can be written to a module by using the "Load File" button. The "Load Defaults" button loads the default gains for the selected module and control strategy. Differences between the input fields and the currently set values will be highlighted.

In the MATLAB API, these gains can be saved and loaded using function in the HebiUtils class:

In the C++ API, these gains can be saved and loaded by using the following code, assuming you have created a Group (see C++ documentation for more info).

The MATLAB API is currently the only API supporting reading and writing this format.

In the MATLAB API, these params can be saved and loaded using function in the HebiUtils class:

Families help to group modules together. Doing so correctly is especially useful and important when you have multiple robots connected to a network at the same time. Check our documentation on Groups to learn more.

All actuators involved in a given kit (Igor, Hexapod, etc.) should be assigned the same Family. As such, the Family assigned should represent the name of that kit.

For example, all actuators in an X-Series Hexapod kit would be have the family Daisy.

By convention, Family names should:

Our actuators look the same but can be used in multiple different ways; using our conventions can save you time and effort in identifying and debugging the right actuator.

Actuators in Serial Configuration

Here, J# indicates where the actuator exists along the chain (it is the #th actuator in the chain) and <type> describes what the actuator is acting as. For example J2_shoulder would be the second actuator in the serial chain and the shoulder joint in an arm.

Thus, the standard naming for a 6-DoF arm:

J1_base, J2_shoulder, J3_elbow, J4_wrist1, J5_wrist2, J6_wrist3

Special Conditions for Arms

Grippers

If your kit makes use of a standard HEBI parallel gripper, you will have a spool actuator that you command to change the state of the gripper. As per our convention, this actuator is named gripperSpool.

Mobile Base Wheels

Multi-limbed Robots

Here, L# indicates which limb we are looking at. Each standard robot kit created by HEBI Robotics has a predefined layout of limb numbers that can be found on that kit’s reference documentation.

Here is an example of how the "Daisy" Hexapod kit is named (see page 31 of Hexapod Manual for leg layout):

L1_J1_base, L1_J2_shoulder, L1_J3_elbow, L2_J1_base, L2_J2_shoulder, L2_J3_elbow, etc…​

Special Condition: Igor

For the Igor kit the arms and legs have been named in the following fashion:

Mobile devices

If you are using your mobile device to provide input to your program, your device should be named mobileIO. This can be set in the settings on your phone, or using Scope.

Single Actuators

Use the actuator’s serial number. For example: X-00542. This number is printed on the actuator, or can be seen on Scope.

IO Boards

Use the board’s serial number. For example: IO_00042. This number is printed on the board, or can be seen on Scope.

Our arm kits are numbered to make it faster for you to identify one arm from another. By looking only at an arm kit number, you should be able to discern which type (X-series or R-series) of actuators your arm is made of, the degrees of freedom in you arm, and the addition of any add-ons, such as a gripper, to your configuration. We use this numbering to name our kinematic (.hrdf) and gains (.xml) files.

Learn more about our arm kits at our Kit documentation or our Arms page.