This project, madbfs
(modern adb filesystem, formerly adbfsm
), aims to create a well-built filesystem abstraction over adb
using libfuse
that is fast, safe, and reliable while also have have well structured code according to modern C++ (20 and above) practices.
I want to manage my Android phone storage from my computer without using MTP (it's awful).
This project is inspired by the adbfs-rootless
project by Spion, available on GitHub. While adbfs-rootless
works as intended, I encoutered frequent crashes that affected its reliability. I initially considered contributing fixes directly into the codebase, but found it somewhat dated with practices that don't align well with modern practices. Consequently, I decided to rebuild the project from the ground up to create a more stable and modern solution.
TLDR: Full file and directory traversal with concurrent file streaming approach, partial read/write, and active caching.
-
No root access required
Non-rooted device will have standard file and directory access as regular user. Rooted device will have less constraints on file and directory access.
-
Full file and directory traversal
Browse the entire file tree of your Android device (subject to permission constraints).
-
Read and write support
Open, read, and write files (subject to permission constraints).
-
Create and delete files and directories
Support creating new files and directories as well as deleting them (subject to permission constraints).
-
Rename and move
Rename or move files and directory seamlessly (subject to permission constraints).
-
Modify file timestamps
Update file access and modification times.
-
FUSE integration
Seamlessly mount your Android device as a regular filesystem. Fully compatible with standard tools like
ls
,cp
,mv
,cat
,vim
, etc. -
Automatic in-memory caching
Recently accessed files are cached in memory using an LRU paging mechanism, allowing faster repeated access.
-
Streamed file access (partial read/write)
Read files on-demand without pulling the entire file first. Unlike MTP’s "pull-whole-file" model, this filesystem streams data over
adb
, enabling efficient access to large files or specific file segments. -
Efficient resource use
Loads only what you access, conserving memory usage and bandwidth. You can also control how much the cache stores file data.
-
Concurrent file access
Allows for concurrent access to files and directories without blocking. This ability comes out-of-the-box by virtue of using FUSE and
adb
. -
Flexible connection method
madbfs
offers two kinds of transport:-
adb
transportThe simplest transport. It executes all FUSE operations by running
adb shell
commands likedd
,stat
, andtouch
. No additional component is required. -
Proxy transport (optional)
Communicates with a lightweight TCP server running natively on the Android device via a custom RPC protocol. It requires a server binary compiled for the phone architecture but has better i/o throughput than the
adb
transport.
Both of the transport can work wired via USB or wireless in your local network.
madbfs
will automatically fall back toadb
transport if the server is not available. -
-
Resilient to disconnections
Stays mounted even when the device disconnects. Cached files and directories remain accessible, and full functionality resumes automatically when the connection is restored.
-
Built with modern C++
Developed in C++23 using coroutine-style asynchronous programming for clean, lightweight, high-performance I/O.
-
madbfs
Build dependencies
- CMake
- Conan
Library dependencies
- Boost (Asio, Process, and JSON component)
- fmt
- libfuse
- rapidhash
- spdlog
-
madbfs-server
(optional)Build dependencies
- Android NDK (must support C++20 or above)
- CMake
- Conan
Library dependencies
- Asio (non-Boost variant)
- fmt
- spdlog
Since the dependencies are managed by Conan, you need to make sure it's installed and configured correctly (consult the documentation on how to do it, here) on your system.
don't forget to run
conan profile detect
if you use Conan for the first time
-
madbfs
Navigate to the root of the repository, then you install the dependencies:
conan install . --build=missing -s build_type=Release
Then compile the project:
cmake --preset conan-release cmake --build --preset conan-release
The built binary will be in
build/Release/madbfs/
directory with the namemadbfs
. Since the libraries are statically linked to the binary you can place this binary anywhere you want (place it in PATH if you want it to be accessible from anywhere). -
madbfs-server
you may skip this process if you don't mind not having proxy transport support
Navigate to the root of the repository then go to
madbfs-server/
subdirectory. You then can proceed by editing theconan-android-profile.conf
file. This step is necessary to set the Android native app build system and its paramater to better suit your Android device(s). The parameters that you may change are- Android NDK path (
tools.android:ndk_path
), - Android ABI (
arch
), - Android API level (
os.api_level
), and - Compiler version (
compiler.version
).
The dependencies installation step is similar to previous one with some additional flags to the command like so:
conan install . --build missing -s build_type=Release --profile:build default --profile:host conan-android-profile.conf
The compilation process is the same:
cmake --preset conan-release cmake --build --preset conan-release
The binary will be in
build/Release/
directory relative tomadbfs-server/
with the namemadbfs-server
. You may want to placemadbfs-server
in the same directory asmadbfs
to allowmadbfs
find it automatically when you run it unless you don't mind runningmadbfs
with--server
flag (explained in the next section). - Android NDK path (
The help message can help you start using this program
usage: madbfs [options] <mountpoint>
Options for madbfs:
--serial=<s> serial number of the device to mount
(you can omit this [detection is similar to adb])
(will prompt if more than one device exists)
--server path to server file
(if omitted will search the file automatically)
(must have the same arch as your phone)
--log-level=<l> log level to use (default: warn)
--log-file=<f> log file to write to (default: - for stdout)
--cache-size=<n> maximum size of the cache in MiB
(default: 512)
(minimum: 128)
(value will be rounded to the next power of 2)
--page-size=<n> page size for cache & transfer in KiB
(default: 128)
(minimum: 64)
(value will be rounded to the next power of 2)
--port=<n> set port the server listens on
(default: 12345)
--no-server don't launch server
(will still attempt to connect to specified port)
(fall back to adb shell calls if connection failed)
(useful for debugging the server)
-h --help show this help message
--full-help show full help message (includes libfuse options)
To mount your device you only need to specify the mount point if there is only one device. If there are more than one device then you can specify the serial using --serial
option. If you omit the --serial
option when there are multiple device connected to the computer, you will be prompted to specify the device you want to mount.
$ ./madbfs mount
[madbfs] checking adb availability...
[madbfs] multiple devices detected,
- 1: 068832516O101622
- 2: 192.168.240.112:5555
[madbfs] please specify which one you would like to use: _
madbfs
respects the env variable ANDROID_SERIAL
(mimicking adb
behavior) so you can alternately use it to specify the device.
$ ANDROID_SERIAL=068832516O101622 ./madbfs
[madbfs] checking adb availability...
[madbfs] using serial '068832516O101622' from env variable 'ANDROID_SERIAL'
only relevant if you want proxy transport support
In order to use the proxy transport, madbfs
needs to be able to find the madbfs-server
binary. There are three approaches you can do in order for mabfs
be able to find the server file:
- Place it where you run the
madbfs
program, - Place it in the same directory as
madbfs
program, or - Specify explicitly the path of the file using
--server
flag.
If you want the filesystem to use adb
transport instead then you can use --no-server
flag. This flag prevents madbfs
from pushing the server into your phone and running it.
The proxy runs communicates with madbfs
over TCP enabled by port forwarding and by default it will listen on 12345
port number. If you find this port to be not suitable for your use you can always specify it with --port
flag.
madbfs
caches all the read/write operations on the files on the device. This cache is stored in memory. You can control the size of this cache using --cache-size
option (in MiB). The default value is 512
(512MiB).
$ ./madbfs --cache-size=512<mountpoint> # using 512MiB of cache
In the cache, each file is divided into pages. The --page-size
option dictates the size of this page (in KiB). Page size also dictates the size of the buffer used to read/write into the file on the device. You can adjust this value according to your use. From my testing, page-size
of value 128
(means 128KiB) works well when using USB cable for the adb
connection. You may want to decrease or increase this value for your use case. The default value is 128
(128KiB).
$ ./madbfs --page-size=128<mountpoint> # using 128KiB of page size
The default log file is stdout (which goes to nowhere when not run in foreground mode). You can manually set the log file using --log-file
option and set the log level using --log-level
.
$ ./madbfs --log-file=madbfs.log --log-level=debug <mountpoint>
As part of debugging functionality libfuse
has provided debug mode through -d
flag. You can use this to monitor madbfs
operations (if you don't want to use log file or want to see the log in real-time). If the debugging information is too verbose, you can use -f
instead to make madbfs run in foreground mode without printing fuse
debug information.
$ ./madbfs --log-file=- --log-level=debug -d <mountpoint> # this will print the libfuse debug messages and madbfs log messages
$ ./madbfs --log-file=- --log-level=debug -d <mountpoint> 2> /dev/null # this will print only madbfs log messages since libfuse debug messages are printed to stderr
see this python script for an example of an IPC client.
Filesystem parameters can be reconfigured and queried during runtime though IPC using unix socket. The supported operations are:
- help,
- invalidate cache,
- set/get page size, and
- set/get cache size.
The address of the socket in which you can connect to as client is composed of the name of the filesystem and the serial of the device. The socket itself is created in directory defined by XDG_RUNTIME_DIR
environment variable (it's usually set to /run/user/<uid>
). If the XDG_RUNTIME_DIR
is not defined, as fallback, the directory is set to /tmp
. The socket will be created when the filesystem initializes.
For example, the socket path for a device with serial 192.168.240.112:5555
:
/run/user/1000/[email protected]:5555.sock
If at initialization this socket file exists, the IPC won't start. This may happen if the filesystem is terminated unexpectedly (crash or kill signal). You need to remove this file manually if that happens.
The communication though the IPC is done using a simple Length-Value message protocol. The first 4 bytes of the message is the length of it (excluding itself) in network byte order, and the rest is the payload.
The payload must be a JSON object in the form that depends on the operation requested. The general form of the JSON is in the form of:
{
"op": <name>,
"value": <value>
}
Some operations only requires "op"
field, while some requires "value"
field. Below is the break down:
-
Help
{ "op": "help" }
-
Invalidate cache:
{ "op": "invalidate_cache" }
-
Get cache size
{ "op": "get_cache_size" }
-
Get page size
{ "op": "get_page_size" }
-
Set cache size:
{ "op": "set_cache_size", "value": { "mib": <uint> } }
- uint must be greater than or equal to 128
- the value will be rouded up to the nearest multiple of 2
-
Set page size:
{ "op": "set_page_size", "value": { "kib": <uint> } }
- uint must be between 64 and 4096
- the value will be rouded up to the nearest multiple of 2
The IPC will reply immediately after an operation complete. The reply is in a JSON in the form of
{
"status", <success|error>,
"<value|message>": <value>
}
The second field will be "value"
if the "status"
is "success"
, else the second field will be "message"
if the "status"
is "error"
.
The <value>
then will be different depending on the operation performed:
-
Invalidate cache:
{ "status": "success", "value": null }
-
Get cache size
{ "status": "success", "value": <uint> }
-
Get page size
{ "status": "success", "value": <uint> }
-
Set cache size:
{ "status": "success", "value": { "old_cache_size": <old_value>, "new_cache_size": <new_value> } }
size is in MiB
-
Set page size:
{ "status": "success", "value": { "old_cache_size": <old_value>, "new_cache_size": <new_value>, "old_page_size": <old_value>, "new_page_size": <new_value>, } }
cache size is in MiB page size is in KiB
- Make the codebase async using C++20 coroutines.
- IPC to talk to the
madbfs
to control the filesystem parameters like invalidation, timeout, cache size, etc. - Implement the filesystem as actual tree for caching the stat.
- Implement file read and write operation caching in memory.
-
Implement proper multithreading, (not needed, since it's using async now, though multiple executor might help). - Implement proper permission check.
- Implement versioning on each node that expires every certain period of time. When a node expires it needs to query the files from the device again.
- Periodic cache invalidation. Current implementation only look at the size of current cache and only invalidate oldest entry when newest entry is added and the size exceed the
cache-size
limit. - Eliminate copying data to and from memory when transferring/copying files within the filesystem.
- Use multiple threads backing the async runtime.
- Rewrite the server app in Kotlin, using the Android runtime instead as native binary.
- Use persistent TCP connection to the server instead of making connection per request.
- Fix open/close semantics.
- Add limit to open file descriptor (for adb query it using
ulimit -n
, for server query it usinggetrlimit
)