Skip to content

Commit 24b99a2

Browse files
pdamounepdamoune
authored
[FlashPoint] Implement parameter guess_relationships (#3920)
1 parent ca88992 commit 24b99a2

File tree

9 files changed

+76
-59
lines changed

9 files changed

+76
-59
lines changed

‎external-import/flashpoint/.env.sample

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,5 @@ FLASHPOINT_IMPORT_INDICATORS=true
1414
FLASHPOINT_IMPORT_ALERTS=true
1515
FLASHPOINT_ALERT_CREATE_RELATED_ENTITIES=false
1616
FLASHPOINT_IMPORT_COMMUNITIES=false
17-
FLASHPOINT_COMMUNITIES_QUERIES=cybersecurity,cyberattack
17+
FLASHPOINT_COMMUNITIES_QUERIES=cybersecurity,cyberattack
18+
FLASHPOINT_GUESS_RELATIONSHIPS_FROM_REPORTS=false

‎external-import/flashpoint/README.md

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -58,18 +58,19 @@ Priority: **YAML > .env > environment > defaults**.
5858

5959
### Connector extra parameters environment variables
6060

61-
| Parameter | config.yml | Docker environment variable | Default | Mandatory | Description |
62-
| -------------------------------------- | ----------------------------- | ------------------------------------------ | --------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
63-
| API access key | api_access_key | `FLASHPOINT_API_KEY` | / | Yes | Flashpoint API access key. |
64-
| Import interval (minutes) (Deprecated) | interval | `FLASHPOINT_INTERVAL` | 5 | No | Interval (in minutes) to import data from Flashpoint. This option option is deprecated. Please use 'duration_period' instead |
65-
| Import start date | import_start_date | `FLASHPOINT_IMPORT_START_DATE` | "P30D" | No | An ISO 8601 string specifying either a period of time or a start date for data import, e.g. 'P30D' (last 30 days) or '2025-05-01' (since May 1, 2025). |
66-
| Import reports | import_reports | `FLASHPOINT_IMPORT_REPORTS` | true | No | Import reports from Flashpoint. |
67-
| Indicators in reports | indicators_in_reports | `FLASHPOINT_INDICATORS_IN_REPORTS` | false | No | Include indicators in the reports imported from MispFeed. |
68-
| Import indicators | import_indicators | `FLASHPOINT_IMPORT_INDICATORS` | true | No | Import indicators of compromise (IoCs). |
69-
| Import alerts | import_alerts | `FLASHPOINT_IMPORT_ALERTS` | true | No | Import alert data from Flashpoint. |
70-
| Create alert related entities | alert_create_related_entities | `FLASHPOINT_ALERT_CREATE_RELATED_ENTITIES` | false | No | Create alert related Channel entity and Media-Content observable |
71-
| Import communities | import_communities | `FLASHPOINT_IMPORT_COMMUNITIES` | false | No | Import community data. |
72-
| Communities queries | communities_queries | `FLASHPOINT_COMMUNITIES_QUERIES` | "cybersecurity,cyberattack" | No | Comma-separated list of community queries to execute. |
61+
| Parameter | config.yml | Docker environment variable | Default | Mandatory | Description |
62+
| -------------------------------------- |----------------------------------|--------------------------------------------|-----------------------------| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
63+
| API access key | api_access_key | `FLASHPOINT_API_KEY` | / | Yes | Flashpoint API access key. |
64+
| Import interval (minutes) (Deprecated) | interval | `FLASHPOINT_INTERVAL` | 5 | No | Interval (in minutes) to import data from Flashpoint. This option option is deprecated. Please use 'duration_period' instead |
65+
| Import start date | import_start_date | `FLASHPOINT_IMPORT_START_DATE` | "P30D" | No | An ISO 8601 string specifying either a period of time or a start date for data import, e.g. 'P30D' (last 30 days) or '2025-05-01' (since May 1, 2025). |
66+
| Import reports | import_reports | `FLASHPOINT_IMPORT_REPORTS` | true | No | Import reports from Flashpoint. |
67+
| Indicators in reports | indicators_in_reports | `FLASHPOINT_INDICATORS_IN_REPORTS` | false | No | Include indicators in the reports imported from MispFeed. |
68+
| Import indicators | import_indicators | `FLASHPOINT_IMPORT_INDICATORS` | true | No | Import indicators of compromise (IoCs). |
69+
| Import alerts | import_alerts | `FLASHPOINT_IMPORT_ALERTS` | true | No | Import alert data from Flashpoint. |
70+
| Create alert related entities | alert_create_related_entities | `FLASHPOINT_ALERT_CREATE_RELATED_ENTITIES` | false | No | Create alert related Channel entity and Media-Content observable |
71+
| Import communities | import_communities | `FLASHPOINT_IMPORT_COMMUNITIES` | false | No | Import community data. |
72+
| Communities queries | communities_queries | `FLASHPOINT_COMMUNITIES_QUERIES` | "cybersecurity,cyberattack" | No | Comma-separated list of community queries to execute. |
73+
| Guess relationships | guess_relationships_from_reports | `GUESS_RELATIONSHIPS_FROM_REPORTS` | false | No | Enable or disable the guessing of relationships between entities. |
7374

7475
⚠️ Please be aware that `CONNECTOR_DURATION_PERIOD` default value takes precedence over `FLASHPOINT_INTERVAL` default value if none of them are set.
7576

‎external-import/flashpoint/docker-compose.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,4 +20,5 @@ services:
2020
- FLASHPOINT_ALERT_CREATE_RELATED_ENTITIES=false
2121
- FLASHPOINT_IMPORT_COMMUNITIES=false
2222
- FLASHPOINT_COMMUNITIES_QUERIES=cybersecurity,cyberattack
23+
- FLASHPOINT_GUESS_RELATIONSHIPS_FROM_REPORTS=false
2324
restart: always

‎external-import/flashpoint/src/config.yml.sample

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,4 +30,4 @@ flashpoint:
3030
alert_create_related_entities: false
3131
import_communities: false
3232
communities_queries: 'cybersecurity,cyberattack'
33-
33+
guess_relationships_from_reports: false

‎external-import/flashpoint/src/flashpoint_connector/config_loader.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,10 @@ class FlashpointConfig(ConfigBaseModel):
248248
description="List of community queries to execute.",
249249
default=["cybersecurity", "cyberattack"],
250250
)
251+
guess_relationships_from_reports: bool = Field(
252+
description="Whether to guess relationships between entities or not.",
253+
default=False,
254+
)
251255

252256

253257
class ConfigLoader(BaseSettings):

‎external-import/flashpoint/src/flashpoint_connector/connector.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ def _import_reports(self, start_date):
6464
for report in reports:
6565
try:
6666
stix_report_objects = self.converter_to_stix.convert_flashpoint_report(
67-
report
67+
report, self.config.flashpoint.guess_relationships_from_reports
6868
)
6969
bundle = self.helper.stix2_create_bundle(stix_report_objects)
7070
self._send_bundle(work_id=work_id, serialized_bundle=bundle)

‎external-import/flashpoint/src/flashpoint_connector/converter_to_stix.py

Lines changed: 51 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ def create_relation(self, source_id, target_id, relation):
7979
message = f"An error occurred while creating relation: {source_id} {relation} {target_id}, error: {ex}"
8080
self.helper.connector_logger.error(message)
8181

82-
def _guess_knowledge_graph(self, tags):
82+
def _guess_knowledge_graph(self, tags, guess_relationships_from_reports):
8383
"""
8484
:param tags:
8585
:return:
@@ -209,51 +209,57 @@ def _guess_knowledge_graph(self, tags):
209209
allow_custom=True,
210210
)
211211
)
212-
213-
for attack_pattern in elements["attack_patterns"]:
214-
threats = (
215-
elements["threat_actors"]
216-
+ elements["intrusion_sets"]
217-
+ elements["malwares"]
218-
)
219-
for threat in threats:
220-
relationship_uses = self.create_relation(
221-
source_id=threat.id,
222-
target_id=attack_pattern.id,
223-
relation="uses",
224-
)
225-
report_objects.append(relationship_uses)
226-
227-
for malware in elements["malwares"]:
228-
threats = elements["threat_actors"] + elements["intrusion_sets"]
229-
for threat in threats:
230-
relationship_uses = self.create_relation(
231-
source_id=threat.id, target_id=malware.id, relation="uses"
212+
if guess_relationships_from_reports:
213+
for attack_pattern in elements["attack_patterns"]:
214+
threats = (
215+
elements["threat_actors"]
216+
+ elements["intrusion_sets"]
217+
+ elements["malwares"]
232218
)
233-
report_objects.append(relationship_uses)
219+
for threat in threats:
220+
relationship_uses = self.create_relation(
221+
source_id=threat.id,
222+
target_id=attack_pattern.id,
223+
relation="uses",
224+
)
225+
report_objects.append(relationship_uses)
226+
227+
for malware in elements["malwares"]:
228+
threats = elements["threat_actors"] + elements["intrusion_sets"]
229+
for threat in threats:
230+
relationship_uses = self.create_relation(
231+
source_id=threat.id,
232+
target_id=malware.id,
233+
relation="uses",
234+
)
235+
report_objects.append(relationship_uses)
234236

235-
for tool in elements["tools"]:
236-
threats = elements["threat_actors"] + elements["intrusion_sets"]
237-
for threat in threats:
238-
relationship_uses = self.create_relation(
239-
source_id=threat.id, target_id=tool.id, relation="uses"
240-
)
241-
report_objects.append(relationship_uses)
237+
for tool in elements["tools"]:
238+
threats = elements["threat_actors"] + elements["intrusion_sets"]
239+
for threat in threats:
240+
relationship_uses = self.create_relation(
241+
source_id=threat.id, target_id=tool.id, relation="uses"
242+
)
243+
report_objects.append(relationship_uses)
242244

243-
victims = (
244-
elements["regions"] + elements["countries"] + elements["sectors"]
245-
)
246-
for victim in victims:
247-
threats = (
248-
elements["threat_actors"]
249-
+ elements["intrusion_sets"]
250-
+ elements["malwares"]
245+
victims = (
246+
elements["regions"]
247+
+ elements["countries"]
248+
+ elements["sectors"]
251249
)
252-
for threat in threats:
253-
relationship_uses = self.create_relation(
254-
source_id=threat.id, target_id=victim.id, relation="targets"
250+
for victim in victims:
251+
threats = (
252+
elements["threat_actors"]
253+
+ elements["intrusion_sets"]
254+
+ elements["malwares"]
255255
)
256-
report_objects.append(relationship_uses)
256+
for threat in threats:
257+
relationship_uses = self.create_relation(
258+
source_id=threat.id,
259+
target_id=victim.id,
260+
relation="targets",
261+
)
262+
report_objects.append(relationship_uses)
257263
report_objects = (
258264
report_objects
259265
+ elements["regions"]
@@ -267,7 +273,7 @@ def _guess_knowledge_graph(self, tags):
267273
)
268274
return report_objects
269275

270-
def convert_flashpoint_report(self, report):
276+
def convert_flashpoint_report(self, report, guess_relationships_from_reports):
271277
"""
272278
:param report:
273279
:return:
@@ -276,7 +282,9 @@ def convert_flashpoint_report(self, report):
276282
# Try to resolve
277283
tags = report["tags"]
278284
actors = report["actors"]
279-
report_objects = self._guess_knowledge_graph(tags + actors)
285+
report_objects = self._guess_knowledge_graph(
286+
tags + actors, guess_relationships_from_reports
287+
)
280288
object_refs = []
281289
for report_object in report_objects:
282290
objects.append(report_object)

‎external-import/flashpoint/tests/config.test.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,4 @@ flashpoint:
2020
indicators_in_reports: true
2121
import_communities: true
2222
communities_queries: "cybersecurity,cyberattack"
23+
guess_relationships_from_reports: false

‎external-import/flashpoint/tests/test_config_loader.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ def fake_config_dict() -> dict[str, dict[str, Any]]:
5353
"import_indicators": True,
5454
"import_communities": True,
5555
"communities_queries": "cybersecurity,cyberattack",
56+
"guess_relationships_from_reports": False,
5657
},
5758
}
5859

@@ -64,7 +65,7 @@ def fake_environ(config_dict: dict[str, dict[str, Any]]):
6465
environ = {}
6566
for key, value in config_dict.items():
6667
for sub_key, sub_value in value.items():
67-
if sub_value:
68+
if sub_value is not None:
6869
environ[f"{key.upper()}_{sub_key.upper()}"] = str(sub_value)
6970
return environ
7071

0 commit comments

Comments
 (0)