FFmpeg in .NET MAUI

[snpkhy]

Is it possible to use FFmpeg in .NET MAUI? Personally I have tried to use .so files, but not all are working correct... Also tried to build FFmpegkit, for linux it has built, for android no.. FFmpeg.Autogen I have also used. In WPF it worked. I also tried to write ffmpeg code in console program and using SignalR to communicate with MAUI project. In android nothing has worked... This message is my last hope!
PS: VLC and MediaElement do not recommend please, cause I wanna to use unmaneged code. Also it is not good looking and in MediaElemnt i can not render rtsp. For me it is the most important to recieve and decode frames from camera in live mode.

[AndrewFrancisSE]

I wanted to ask if you've managed to get it working, particularly on iOS? I'm facing similar issues and would really appreciate it if you could share any progress or experience you've had.

[ghost]

Thank you @vlad0syk .
This is currently outside of my knowldege scope. Trying to understand with CoPilot AI.

Using FFmpeg.AutoGen, we build ffmpeg to iOS, include it in MAUI project and then call it. Correct?

How did you call FFmpeg.Autogen library in Android?

[snpkhy]

Yeah. You build ffmpeg libs and then paste them and call functions from them. To call them I install FFmpeg.Autogen for possibility to use all structures and classes.

Methods I wrote in one class NativeFFmpeg.cs:

using FFmpeg.AutoGen;
using System.Runtime.InteropServices;

namespace MauiApp1.FFmpeg;

public static unsafe class NativeFFmpeg
{
    [DllImport("libavformat.so")]
    public static extern AVFormatContext* avformat_alloc_context();
    [DllImport("libavutil.so")]
    public static extern int av_dict_set(AVDictionary** opt, string key, string value, int flags);
    [DllImport("libavformat.so")]
    public static extern int avformat_open_input(AVFormatContext** ps, string url, AVInputFormat* fmt, AVDictionary** options);
    [DllImport("libavformat.so")]
    public static extern int avformat_find_stream_info(AVFormatContext* ic, AVDictionary** options);
    [DllImport("libavcodec.so")]
    public static extern AVPacket* av_packet_alloc();
    [DllImport("libavformat.so")]
    public static extern int av_find_best_stream(AVFormatContext* ic, AVMediaType type, int wanted_stream_db, int related_stream, AVCodec** decoder_ret, int flags);
    [DllImport("libavcodec.so")]
    public static extern int av_packet_unref(AVPacket* pkt);
    [DllImport("libavformat.so")]
    public static extern int av_read_frame(AVFormatContext* s, AVPacket* pkt);
    [DllImport("libavcodec.so")]
    public static extern AVPacket* av_packet_clone(AVPacket* src);
    [DllImport("libavutil.so")]
    public static extern int av_strerror(int errnum, byte* errbuf, ulong bufferSize);
    [DllImport("libavcodec.so")]
    public static extern void av_packet_free(AVPacket** pkt);
    [DllImport("libavformat.so")]
    public static extern void avformat_close_input(AVFormatContext** s);
    [DllImport("libavutil.so")]
    public static extern AVFrame* av_frame_alloc();
    [DllImport("libacodec.so")]
    public static extern AVCodec* avcodec_find_decoder(AVCodecID id);
    [DllImport("libavcodec.so")]
    public static extern AVCodecContext* avcodec_alloc_context3(AVCodec* codec);
    [DllImport("libavutill.so")]
    public static extern int av_hwdevice_ctx_create(AVBufferRef** device_ctx, AVHWDeviceType type, string device, AVDictionary* opts, int flags);
    [DllImport("libavcodec.so")]
    public static extern int avcodec_parameters_to_context(AVCodecContext* codec, AVCodecParameters* par);
    [DllImport("libavcodec.so")]
    public static extern int avcodec_open2(AVCodecContext* avctx, AVCodec* codec, AVDictionary** options);
    [DllImport("libavcodec.so")]
    public static extern string avcodec_get_name(AVCodecID id);
    [DllImport("libacodec.so")]
    public static extern void av_frame_free(AVFrame** frame);
    [DllImport("libavcodec.so")]
    public static extern void avcodec_free_context(AVCodecContext** avctx);
    [DllImport("libavutil.so")]
    public static extern void av_buffer_unref(AVBufferRef** buf);
    [DllImport("libavcodec.so")]
    public static extern int avcodec_send_packet(AVCodecContext* avctx, AVPacket* pkt);
    [DllImport("libavcodec.so")]
    public static extern int avcodec_receive_frame(AVCodecContext* avctx, AVFrame* frame);
    [DllImport("libavutil.so")]
    public static extern int av_hwframe_transfer_data(AVFrame* dst, AVFrame* src, int flags);
    [DllImport("libavutil.so")]
    public static extern AVFrame* av_frame_clone(AVFrame* src);
    [DllImport("libavutil.so")]
    public static extern void av_frame_unref(AVFrame* frame);
    [DllImport("libavcodec.so")]
    public static extern int avcodec_parameters_from_context(AVCodecParameters* par, AVCodecContext* codec);
    [DllImport("libavutil.so")]
    public static extern IntPtr av_version_info();
    [DllImport("libavcodec.so")]
    public static extern AVCodecParameters* avcodec_parameters_alloc();
    [DllImport("libswscale.so")]
    public static extern SwsContext* sws_getContext(int srcW, int srcH, AVPixelFormat srcFormat, int dstW, int dstH, AVPixelFormat dstFormat, int flags, SwsFilter* srcFilter, SwsFilter* dstFilter, double* param);
    [DllImport("libavutil.so")]
    public static extern int av_image_get_buffer_size(AVPixelFormat pix_fmt, int width, int height, int align);
    [DllImport("libavutil.so")]
    public static extern int av_image_fill_arrays(ref byte_ptrArray4 dst_data, ref int_array4 dst_linesize, byte* src, AVPixelFormat pix_fmt, int width, int height, int align);
    [DllImport("libswscale.so")]
    public static extern int sws_scale(SwsContext* c, byte*[] srcSlice, int[] srcStride, int srcSliceY, int srcSliceH, byte*[] dst, int[] dstStride);
}

Properties for each lib:

[snpkhy]

It's not really very well done, but it's the only way I've managed to get it up and running.

[snpkhy]

Oh and try to use Claude or Gemini. It is better. Also there is very cool feature called Cursor :). Use it all wisely!

[ghost]

OK thank you @vlad0syk . I think you made somebody life easier. For me, it made me understand the underlaying platform and connecting to native functions better. For now, I will stop here. Thank you @vlad0syk .

[landon-codes]

For desktop development you could bundle binaries for whatever operating system you are building for.

I made a Python script for Windows to automate some of my tasks; all I had to do was use the subprocesses module and call the program.

I would definitely take more effort to implement that for each system, and it likely would not support mobile at all. Maybe you could bundle a lightweight terminal like what Termux did for Android/IOS? That would help with writing less code for each OS, and you would only need to install Ffmpeg for one OS.