Skip to content

Multiple LLM model selection in DashBot #704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions lib/consts.dart
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,11 @@ enum ImportFormat {
const ImportFormat(this.label);
final String label;
}
enum LLMProvider {
ollama,
gemini,
openai
}
Copy link
Member

@ashitaprasad ashitaprasad Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each LLMProvider should have some sort of setting where the user can click on a small settings button to change the parameters like API URL, model, etc.


const String kGlobalEnvironmentId = "global";

Expand Down
19 changes: 17 additions & 2 deletions lib/dashbot/providers/dashbot_providers.dart
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
import 'dart:convert';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:shared_preferences/shared_preferences.dart';
import '../../consts.dart';
import '../services/dashbot_service.dart';

final chatMessagesProvider =
StateNotifierProvider<ChatMessagesNotifier, List<Map<String, dynamic>>>(
(ref) => ChatMessagesNotifier(),
StateNotifierProvider<ChatMessagesNotifier, List<Map<String, dynamic>>>(
(ref) => ChatMessagesNotifier(),
);

final dashBotServiceProvider = Provider<DashBotService>((ref) {
return DashBotService();
});

final selectedLLMProvider =
StateNotifierProvider<SelectedLLMNotifier, LLMProvider>(
(ref) => SelectedLLMNotifier(),
);

class ChatMessagesNotifier extends StateNotifier<List<Map<String, dynamic>>> {
ChatMessagesNotifier() : super([]) {
_loadMessages();
Expand Down Expand Up @@ -42,3 +48,12 @@ class ChatMessagesNotifier extends StateNotifier<List<Map<String, dynamic>>> {
_saveMessages();
}
}

/// Manages the selected LLM (only in memory)
class SelectedLLMNotifier extends StateNotifier<LLMProvider> {
SelectedLLMNotifier() : super(LLMProvider.ollama); // Default LLM

void setSelectedLLM(LLMProvider model) {
state = model;
}
}
54 changes: 47 additions & 7 deletions lib/dashbot/services/dashbot_service.dart
Original file line number Diff line number Diff line change
@@ -1,24 +1,64 @@
import 'package:apidash/dashbot/features/debug.dart';
import 'package:ollama_dart/ollama_dart.dart';
import 'package:openai_dart/openai_dart.dart';
import 'package:flutter_gemini/flutter_gemini.dart';
import '../../consts.dart';
import '../features/explain.dart';
import 'package:apidash/models/request_model.dart';


class DashBotService {
final OllamaClient _client;
late final OllamaClient _ollamaClient;
late final OpenAIClient _openAiClient;
late final ExplainFeature _explainFeature;
late final DebugFeature _debugFeature;

LLMProvider _selectedModel = LLMProvider.ollama;

DashBotService()
: _client = OllamaClient(baseUrl: 'http://127.0.0.1:11434/api') {
: _ollamaClient = OllamaClient(baseUrl: 'http://127.0.0.1:11434/api'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add provision to change baseURL via LLMProvider settings

//TODO: Add API key to .env file
_openAiClient = OpenAIClient(apiKey: "your_openai_api_key") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add provision to add API Key via LLMProvider settings

_explainFeature = ExplainFeature(this);
_debugFeature = DebugFeature(this);
}

void setModel(LLMProvider model) {
_selectedModel = model;
}

Future<String> generateResponse(String prompt) async {
final response = await _client.generateCompletion(
request: GenerateCompletionRequest(model: 'llama3.2:3b', prompt: prompt),
);
return response.response.toString();
try {
switch (_selectedModel) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLMProvider != model

case LLMProvider.gemini:
final response = await Gemini.instance.chat([
Content(parts: [Part.text(prompt)], role: 'user')
]);
return response?.output ?? "Error: No response from Gemini.";

case LLMProvider.ollama:
final response = await _ollamaClient.generateCompletion(
request: GenerateCompletionRequest(model: 'llama3.2:3b', prompt: prompt),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

llama3.2:3b is the model. Add provision to change it via LLMProvider settings based on list of installed models in the system

);
return response.response.toString();

case LLMProvider.openai:
final response = await _openAiClient.createChatCompletion(
request: CreateChatCompletionRequest(
model: ChatCompletionModel.modelId('gpt-4o'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gpt-4o is the model. Add provision to change it via LLMProvider settings based on list of available openai models.

messages: [
ChatCompletionMessage.user(
content: ChatCompletionUserMessageContent.string(prompt),
),
],
temperature: 0,
),
);
return response.choices?.first.message?.content ?? "Error: No response from OpenAI.";
}
} catch (e) {
return "Error: ${e.toString()}";
}
}

Future<String> handleRequest(
Expand All @@ -33,4 +73,4 @@ class DashBotService {

return generateResponse(input);
}
}
}
23 changes: 23 additions & 0 deletions lib/dashbot/widgets/dashbot_widget.dart
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
// lib/dashbot/widgets/dashbot_widget.dart
import 'package:apidash_core/apidash_core.dart';
import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:apidash/dashbot/providers/dashbot_providers.dart';
import 'package:apidash/providers/providers.dart';
import '../../consts.dart';
import 'chat_bubble.dart';

class DashBotWidget extends ConsumerStatefulWidget {
Expand Down Expand Up @@ -94,6 +96,8 @@ class _DashBotWidgetState extends ConsumerState<DashBotWidget> {
children: [
_buildHeader(context),
const SizedBox(height: 12),
_buildModelSelector(),
const SizedBox(height: 12),
_buildQuickActions(showDebugButton),
const SizedBox(height: 12),
Expanded(child: _buildChatArea(messages)),
Expand All @@ -104,6 +108,25 @@ class _DashBotWidgetState extends ConsumerState<DashBotWidget> {
),
);
}
Widget _buildModelSelector() {
final selectedLLM = ref.watch(selectedLLMProvider);

return DropdownButton<LLMProvider>(
value: selectedLLM,
items: LLMProvider.values.map((provider) {
return DropdownMenuItem(
value: provider,
child: Text(provider.name.capitalize()),
);
}).toList(),
onChanged: (LLMProvider? newProvider) {
if (newProvider != null) {
ref.read(selectedLLMProvider.notifier).state = newProvider;
ref.read(dashBotServiceProvider).setModel(newProvider);
}
},
);
}

Widget _buildHeader(BuildContext context) {
return Row(
Expand Down
3 changes: 3 additions & 0 deletions lib/main.dart
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import 'package:apidash_design_system/apidash_design_system.dart';
import 'package:flutter/material.dart';
import 'package:flutter_gemini/flutter_gemini.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'models/models.dart';
import 'providers/providers.dart';
Expand All @@ -9,6 +10,8 @@ import 'app.dart';

void main() async {
WidgetsFlutterBinding.ensureInitialized();
//TODO: Add API key to .env file
Gemini.init(apiKey: "apiKey");
Copy link
Member

@ashitaprasad ashitaprasad Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gemini.init cannot happen until user adds an API key via LLMProvider settings

var settingsModel = await getSettingsFromSharedPrefs();
final initStatus = await initApp(
kIsDesktop,
Expand Down
22 changes: 11 additions & 11 deletions lib/screens/dashboard.dart
Original file line number Diff line number Diff line change
Expand Up @@ -126,17 +126,17 @@ class Dashboard extends ConsumerWidget {
),
),
// TODO: Release DashBot
// floatingActionButton: FloatingActionButton(
// onPressed: () => showModalBottomSheet(
// context: context,
// isScrollControlled: true,
// builder: (context) => const Padding(
// padding: EdgeInsets.all(16.0),
// child: DashBotWidget(),
// ),
// ),
// child: const Icon(Icons.help_outline),
// ),
floatingActionButton: FloatingActionButton(
onPressed: () => showModalBottomSheet(
context: context,
isScrollControlled: true,
builder: (context) => const Padding(
padding: EdgeInsets.all(16.0),
child: DashBotWidget(),
),
),
child: const Icon(Icons.help_outline),
),
);
}
}
32 changes: 32 additions & 0 deletions pubspec.lock
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,22 @@ packages:
url: "https://pub.dev"
source: hosted
version: "0.5.0"
dio:
dependency: transitive
description:
name: dio
sha256: "253a18bbd4851fecba42f7343a1df3a9a4c1d31a2c1b37e221086b4fa8c8dbc9"
url: "https://pub.dev"
source: hosted
version: "5.8.0+1"
dio_web_adapter:
dependency: transitive
description:
name: dio_web_adapter
sha256: "7586e476d70caecaf1686d21eee7247ea43ef5c345eab9e0cc3583ff13378d78"
url: "https://pub.dev"
source: hosted
version: "2.1.1"
equatable:
dependency: transitive
description:
Expand Down Expand Up @@ -520,6 +536,14 @@ packages:
description: flutter
source: sdk
version: "0.0.0"
flutter_gemini:
dependency: "direct main"
description:
name: flutter_gemini
sha256: b7264b1d19acc4b1a5628a0e26c0976aa1fb948f0d3243bc3510ff51e09476b7
url: "https://pub.dev"
source: hosted
version: "3.0.0"
flutter_highlighter:
dependency: "direct main"
description:
Expand Down Expand Up @@ -1097,6 +1121,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "0.2.2+1"
openai_dart:
dependency: "direct main"
description:
name: openai_dart
sha256: "1cc5ed0915fa7572b943de01cfa7a3e5cfe1e6a7f4d0d9a9374d046518e84575"
url: "https://pub.dev"
source: hosted
version: "0.4.5"
package_config:
dependency: transitive
description:
Expand Down
2 changes: 2 additions & 0 deletions pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ dependencies:
git:
url: https://github.com/google/flutter-desktop-embedding.git
path: plugins/window_size
openai_dart: ^0.4.5
flutter_gemini: ^3.0.0

dependency_overrides:
extended_text_field: ^16.0.0
Expand Down