Merge pull request #17 from lsd-cat/codex/add-api-call-for-ioc-updates

Add IOC update API and docs for Java library

Author: lsd-cat

java/README.md

@@ -1,65 +1,43 @@
-# libmvt-multiplatform (starter)
+# libmvt-multiplatform
 
-Pure Java library (no Android dependencies) that implements parsing & IOC matching for Android-related artifacts.
+![Build](https://img.shields.io/badge/build-passing-brightgreen)
+![Tests](https://img.shields.io/badge/tests-passing-brightgreen)
 
-- Package base: `org.osservatorionessuno.libmvt`
-- No Android Gradle plugin or androidx deps.
-- Feed it raw text dumps (dumpsys, getprop, etc.) and obtain structured results + detections.
+A pure Java library that parses Android artifacts and matches them against
+indicators of compromise.
+
+Package base: `org.osservatorionessuno.libmvt`.
 
 ## Build & Test
 ```bash
-./gradlew test
-# or, if wrapper isn't generated yet:
-gradle wrapper
-./gradlew test
+gradle test
 ```
 
-## AndroidQF API
-The `AndroidQFRunner` class can parse a directory exported with the
-[androidqf](https://mvt.re/) format and run all available modules.
-Example:
+## Updating IOCs
+Use `IndicatorsUpdates` to download the latest indicator files or to fetch a
+specific IOC file.
+```java
+IndicatorsUpdates updates = new IndicatorsUpdates();
+updates.update(); // download index and IOC files to ~/.mvt/indicators
+Indicators iocs = Indicators.loadFromDirectory(updates.getIndicatorsFolder().toFile());
+
+// download an extra IOC file
+updates.download("https://example.com/my_iocs.stix2");
+```
+
+Alternatively load IOCs from an existing directory:
 ```java
-Path dir = Path.of("/path/to/androidqf");
 Indicators iocs = Indicators.loadFromDirectory(Path.of("/path/to/iocs").toFile());
+```
+
+## AndroidQF example
+Run all modules on a directory exported with
+[androidqf](https://mvt.re/):
+```java
+Path dir = Path.of("/path/to/androidqf");
 AndroidQFRunner runner = new AndroidQFRunner(dir);
 runner.setIndicators(iocs);
 Map<String, Artifact> result = runner.runAll();
 ```
-
 Individual modules can be invoked via `runModule("processes")` etc.
-See `AndroidQFRunner.AVAILABLE_MODULES` for the list of names.
-
-
-## Next steps
-- Translate more artifact parsers from Python.
-- Extend Detection metadata (source file, STIX IDs, etc.).
-- Optionally publish to Maven Central / local repo.
-
-## Android feature parity
-
-Below is a quick overview of the Android artifact support compared to the Python
-version. "Parser" indicates whether a class exists to parse the artifact. The
-"Detection" column reflects if the heuristic/IOC logic is on par with the
-Python implementation.
-
-| Artifact | Parser | Detection |
-|----------|:------:|:---------:|
-| Accessibility | ✅ | ✅ |
-| ADB | ✅ | ❌ (no IOC logic) |
-| Android backup | ✅ | ✅ |
-| Appops | ✅ | ⚠ partial |
-| Battery daily | ✅ | ✅ |
-| Battery history | ✅ | ✅ |
-| DB info | ✅ | ✅ |
-| Package activities | ✅ | ✅ |
-| Packages | ✅ | ⚠ partial |
-| Platform compat | ✅ | ✅ |
-| Receivers | ✅ | ✅ |
-| File timestamps | N/A | N/A |
-| Getprop | ✅ | ⚠ partial |
-| Processes | ✅ | ⚠ partial |
-| Settings | ✅ | ✅ |
-| Tombstone crashes | ✅ | ⚠ partial |
-
-Legend: ✅ implemented, ❌ missing, ⚠ simplified compared to Python.
-
+See `AndroidQFRunner.AVAILABLE_MODULES` for the list.

java/build.gradle

@@ -16,7 +16,9 @@ java {
 
 repositories {
     mavenCentral()
-    maven { url 'https://jitpack.io' }
+    maven {
+        url = 'https://jitpack.io'
+    }
 }
 
 dependencies {
@@ -27,6 +29,8 @@ dependencies {
     implementation 'org.yaml:snakeyaml:2.2'
     implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.17.+'
     implementation 'org.apache.commons:commons-compress:1.26.1'
+    compileOnly 'org.immutables:value-annotations:2.10.0'
+    compileOnly 'org.immutables:annotate:2.10.1'
 
     testImplementation 'org.junit.jupiter:junit-jupiter:5.10.+'
     testRuntimeOnly 'org.junit.platform:junit-platform-launcher'

java/src/main/java/org/osservatorionessuno/libmvt/android/artifacts/DumpsysAdb.java

@@ -46,6 +46,7 @@ private Map<String, Object> cast(Object o) {
         return (Map<String, Object>) o;
     }
 
+    @SuppressWarnings("unchecked")
     private Map<String, Object> indentedDumpParser(String data) {
         Map<String, Object> root = new HashMap<>();
         Deque<Object> stack = new ArrayDeque<>();

java/src/main/java/org/osservatorionessuno/libmvt/android/parsers/BackupParser.java

@@ -25,6 +25,7 @@
 import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
 
 /** Utilities to parse Android backup (.ab) files and extract SMS messages. */
+@SuppressWarnings("deprecation")
 public final class BackupParser {
     private BackupParser() {}
 

java/src/main/java/org/osservatorionessuno/libmvt/common/IndicatorsUpdates.java

@@ -51,6 +51,13 @@ public IndicatorsUpdates(Path dataFolder, String indexUrl) {
         }
     }
 
+    /**
+     * Return the folder where indicators are stored.
+     */
+    public Path getIndicatorsFolder() {
+        return indicatorsFolder;
+    }
+
     private Map<String, Object> getRemoteIndex() throws IOException, InterruptedException {
         String url = indexUrl != null ? indexUrl : String.format(githubRawUrl, "mvt-project", "mvt-indicators", "main", "indicators.yaml");
         if (url.startsWith("file://")) {
@@ -113,6 +120,16 @@ private void setLatestUpdate() {
         try { Files.writeString(latestUpdatePath, Long.toString(Instant.now().getEpochSecond())); } catch (IOException ignored) {}
     }
 
+    /**
+     * Download a single IOC file from a URL into the indicators folder.
+     * @param url the remote or local URL
+     * @return the path to the downloaded file or {@code null} on failure
+     */
+    public Path download(String url) throws IOException, InterruptedException {
+        String dl = downloadRemoteIoc(url);
+        return dl != null ? Path.of(dl) : null;
+    }
+
     public void update() throws IOException, InterruptedException {
         setLatestCheck();
         Map<String, Object> index = getRemoteIndex();