Telekinesis：機器人、感知與實體AI的統一技能庫

We’ve been working on Telekinesis, a developer SDK aimed at reducing fragmentation in robotics, computer vision, and Physical AI.

This started from a recurring problem we ran into while building real systems: even relatively simple robotics applications often require stitching together a large number of incompatible components — classical robotics libraries, perception pipelines, learned models, and increasingly foundation models like LLMs and VLMs.

The problem we’re trying to address

Robotics software development is highly fragmented:

Perception, planning, control, and learning often live in separate ecosystems

Each library comes with its own APIs, assumptions, and data formats

Integration glue ends up dominating development time

Mixing classical robotics with modern AI workflows is still painful

As Physical AI and agent-based systems become more common, this gap between classical robotics workflows and modern AI tooling is becoming more acute.

What Telekinesis is

Telekinesis is a large-scale skill library for Physical AI, exposed through a single, consistent Python interface.

Instead of being another robotics framework, it’s designed as a modular, composable set of skills that can be combined into complete systems.

The SDK currently covers:

3D perception: detection, registration, filtering, clustering

The SDK will also cover:

2D perception: image processing, detection, segmentation

Synthetic data generation

Model training tools

Motion planning, kinematics, and control

Physical AI agents

Vision–Language Models (VLMs)

The idea is that roboticists and Computer Vision engineers can access these capabilities without spending most of their time integrating fragmented libraries, and instead focus on system behavior and iteration.

How it’s structured

From the developer’s perspective, Telekinesis provides:

A single Python interface

Hundreds of modular, composable skills

The ability to combine perception, planning, control, and AI components predictably

The skills are hosted on the cloud by default, but the same architecture can also be run on-premise for teams that need full control over data and computation.

This makes it possible to:

Prototype quickly

Reuse components across projects

Scale from experiments to more industry-grade systems without changing APIs

Who this is for

The SDK is intended for:

Robotics engineers working close to perception or control

Computer vision developers building systems, not just models

People experimenting with Physical AI and embodied agents

In particular, for those who feel they spend more time integrating components than evaluating system behavior.

What we’re still figuring out

This is still early, and we’re actively questioning parts of the design:

Where abstractions help vs. hide too much

Which components should never be unified

How to balance flexibility with predictability

How this compares to existing robotics + ML workflows in practice

Happy to hear critical perspectives from people who’ve built or maintained real systems.

Here the documentation (still evolving):
https://docs.telekinesis.ai/

Thanks for reading.