-
Notifications
You must be signed in to change notification settings - Fork 125
Add LM Link Technical Deepdive docs #197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kaydenlms
wants to merge
2
commits into
main
Choose a base branch
from
kayden/lmlink-docs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| title: "Technical Deepdive" | ||
| sidebar_title: "Technical Deepdive" | ||
| description: "A technical deepdive into LM Link" | ||
| index: 2 | ||
| --- | ||
|
|
||
| ## How does LM Link work? | ||
|
|
||
| LM Link was created in close technical collaboration with Tailscale and was implemented using Tailscale's tsnet library. | ||
|
|
||
| Tsnet: | ||
|
|
||
| * Runs fully in userspace, meaning it doesn't need kernel-level access or privileges, or touch your OS's core network settings. | ||
| * Has its own TCP stack — like its own mini-network engine — so it can make connections without messing with your routing table or change any network settings | ||
|
|
||
| Additionally, LM Link is an entirely separate and self-contained use of Tailscale's networking primitives. This means LM Link will not affect any existing networks you have with Tailscale. | ||
|
|
||
| ## What touches the LM Studio Server? | ||
|
|
||
| In the context of LM Link, LM Studio's backend server acts only as a discovery server for initial discovery and keepalive heartbeats. When an LM Studio instance with LM Link enabled comes online, it checks in with the server. The server then updates the Tailscale ACL (Access Control List) to accurately reflect which devices can talk to each other and which devices are removed from the network. | ||
|
|
||
| Once LM Studio instances discover and connect to each other, they use a websocket based protocol, transported on top of connections created by Tsnet. This protocol is composed of two layers: the underlying layer is open source and documented in the reference implementation: [https://github.com/lmstudio-ai/lmstudio-js](https://github.com/lmstudio-ai/lmstudio-js) | ||
|
|
||
| ## What goes through the Tailscale connection? | ||
|
|
||
| All communication between linked devices goes exclusively through the tailscale connection and never touches the LM Studio server. This includes inference (chats), model listing, hardware information, load parameters and configuration etc. | ||
|
|
||
| ## Where are chats stored when I use LM Link? | ||
|
|
||
| When you send a prompt through the encrypted link to a remote device, the prompt is processed on the remote machine and a response is returned to the client device. Chats will always be stored on the disk of the client device. | ||
|
|
||
| ## High level overview | ||
|
|
||
| 1. **Device Discovery**: First, LM Link uses LM Studio's server to facilitate initial device discovery | ||
| 2. **Link Setup**: Once initial device discovery is made, the Tailscale infrastructure is used to set up the VPN-style, end-to-end encrypted tunnel between devices | ||
| 3. **Device-to-device Communication**: Once the encrypted tunnel is set up, traffic goes directly between the devices over that tunnel | ||
| 4. **End-to-end encryption**: Neither Tailscale nor LM Studio has read access to any content moving through the tunnel. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.