Skip to content

Add RDAP analyzer#3760

Open
thunderstornX wants to merge 2 commits into
intelowlproject:developfrom
thunderstornX:analyzer-rdap
Open

Add RDAP analyzer#3760
thunderstornX wants to merge 2 commits into
intelowlproject:developfrom
thunderstornX:analyzer-rdap

Conversation

@thunderstornX

Copy link
Copy Markdown

Adds an observable analyzer that queries the public RDAP bootstrap (https://rdap.org) for registration data of IPs, domains, and URLs. Closes #3759.

RDAP (RFC 9082 / 9083) is the IETF-standard, free, unauthenticated successor to WHOIS. The existing WHOIS analyzers are either paid (whoisxmlapi) or registry-specific (whoisripe), so a free, global lookup fills a gap.

  • Supports ip, domain, and url (resolved to its host) observables; maximum_tlp AMBER.
  • Returns the RDAP JSON; a 404 (no registration record) is surfaced as {"found": false} rather than an error.
  • No API key required.

Proof it works, against live rdap.org (the analyzer follows the bootstrap redirect via requests):

GET rdap.org/ip/1.1.1.1
{"objectClassName": "ip network", "handle": "1.1.1.0 - 1.1.1.255", "name": "APNIC-LABS", "country": "AU", "startAddress": "1.1.1.0", "endAddress": "1.1.1.255"}

GET rdap.org/domain/example.com    ->  objectClassName: domain, ldhName: EXAMPLE.COM (4 events, 2 nameservers)
GET rdap.org/domain/<unregistered> ->  HTTP 404  (returned as {"found": false})

tests/api_app/analyzers_manager/observable_analyzers/test_rdap.py mocks the HTTP layer and covers IP/domain/URL routing, the 404 path, and the unsupported-type and no-hostname error paths.

Adds an observable analyzer that queries the public RDAP bootstrap (rdap.org) for registration data of IPs, domains, and URLs. RDAP (RFC 9082/9083) is the free, unauthenticated, IETF-standard successor to WHOIS; the existing WHOIS analyzers are either paid (whoisxmlapi) or registry-specific (whoisripe).
Copilot AI review requested due to automatic review settings June 9, 2026 13:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new RDAP observable analyzer (and its registration) to query rdap.org for IP/domain/URL registration data, plus unit tests.

Changes:

  • Introduces Rdap observable analyzer that calls rdap.org and normalizes 404 into {"found": False}.
  • Adds Django migration to register the analyzer configuration (AnalyzerConfig) for RDAP.
  • Adds mocked unit tests covering IP/domain/URL behavior and error conditions.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
api_app/analyzers_manager/observable_analyzers/rdap.py Implements the RDAP observable analyzer logic and error handling.
api_app/analyzers_manager/migrations/0193_analyzer_config_rdap.py Registers the new analyzer/config via data migration.
tests/api_app/analyzers_manager/observable_analyzers/test_rdap.py Adds unit tests that mock HTTP calls and validate endpoint selection/results.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +103 to +104
python_path = plugin.pop("model")
Model = apps.get_model(*python_path.split("."))
Comment on lines +115 to +116
python_path = plugin.pop("model")
Model = apps.get_model(*python_path.split("."))
Comment on lines +26 to +27
def update(self) -> bool:
pass
Comment on lines +56 to +57
except requests.RequestException as e:
raise AnalyzerRunException(e)
@thunderstornX

thunderstornX commented Jun 12, 2026

Copy link
Copy Markdown
Author

Proof of a working analysis (per the new-analyzer checklist).

rdap_job_data rdap_job_gui

Ran the RDAP analyzer on 1.1.1.1 in a local IntelOwl v6.6.1 instance built from this branch. The job finished reported_without_fails and the Rdap analyzer report is SUCCESS (0.72s). GUI screenshots of the result are attached below.

I also added the analyzer to the FREE_TO_USE_ANALYZERS playbook (it is free, no API key) in migration 0067_add_rdap_to_free_to_use.

Raw JSON of the finished analysis (observable 1.1.1.1)
{
  "name": "APNIC-LABS",
  "type": "ASSIGNED PORTABLE",
  "found": true,
  "links": [
    {
      "rel": "self",
      "href": "https://rdap.apnic.net/ip/1.1.1.0/24",
      "type": "application/rdap+json",
      "value": "https://rdap.apnic.net/ip/1.1.1.1"
    }
  ],
  "events": [
    {
      "eventDate": "2011-08-10T23:12:35Z",
      "eventAction": "registration"
    },
    {
      "eventDate": "2023-04-26T22:57:58Z",
      "eventAction": "last changed"
    }
  ],
  "handle": "1.1.1.0 - 1.1.1.255",
  "port43": "whois.apnic.net",
  "status": [
    "active"
  ],
  "country": "AU",
  "notices": [
    {
      "title": "Source",
      "description": [
        "Objects returned came from source",
        "APNIC"
      ]
    },
    {
      "links": [
        {
          "rel": "terms-of-service",
          "href": "http://www.apnic.net/db/dbcopyright.html",
          "type": "text/html",
          "value": "https://rdap.apnic.net/ip/1.1.1.1"
        }
      ],
      "title": "Terms and Conditions",
      "description": [
        "This is the APNIC WHOIS Database query service. The objects are in RDAP format."
      ]
    },
    {
      "links": [
        {
          "rel": "inaccuracy-report",
          "href": "https://www.apnic.net/manage-ip/using-whois/abuse-and-spamming/invalid-contact-form",
          "type": "text/html",
          "value": "https://rdap.apnic.net/ip/1.1.1.1"
        }
      ],
      "title": "Whois Inaccuracy Reporting",
      "description": [
        "If you see inaccuracies in the results, please visit: "
      ]
    }
  ],
  "remarks": [
    {
      "title": "description",
      "description": [
        "APNIC and Cloudflare DNS Resolver project",
        "Routed globally by AS13335/Cloudflare",
        "Research prefix for APNIC Labs"
      ]
    },
    {
      "title": "remarks",
      "description": [
        "---------------",
        "All Cloudflare abuse reporting can be done via",
        "resolver-abuse@cloudflare.com",
        "---------------"
      ]
    }
  ],
  "entities": [
    {
      "links": [
        {
          "rel": "self",
          "href": "https://rdap.apnic.net/entity/AIC3-AP",
          "type": "application/rdap+json",
          "value": "https://rdap.apnic.net/ip/1.1.1.1"
        }
      ],
      "roles": [
        "administrative",
        "technical"
      ],
      "events": [
        {
          "eventDate": "2023-04-26T00:42:16Z",
          "eventAction": "registration"
        },
        {
          "eventDate": "2024-07-18T04:37:37Z",
          "eventAction": "last changed"
        }
      ],
      "handle": "AIC3-AP",
      "vcardArray": [
        "vcard",
        [
          [
            "version",
            {},
            "text",
            "4.0"
          ],
          [
            "fn",
            {},
            "text",
            "APNICRANDNET Infrastructure Contact"
          ],
          [
            "kind",
            {},
            "text",
            "group"
          ],
          [
            "adr",
            {
              "label": "6 Cordelia St South Brisbane QLD 4101"
            },
            "text",
            [
              "",
              "",
              "",
              "",
              "",
              "",
              ""
            ]
          ],
          [
            "tel",
            {
              "type": "voice"
            },
            "text",
            "+61 7 3858 3100"
          ],
          [
            "email",
            {},
            "text",
            "research@apnic.net"
          ]
        ]
      ],
      "objectClassName": "entity"
    },
    {
      "links": [
        {
          "rel": "self",
          "href": "https://rdap.apnic.net/entity/IRT-APNICRANDNET-AU",
          "type": "application/rdap+json",
          "value": "https://rdap.apnic.net/ip/1.1.1.1"
        }
      ],
      "roles": [
        "abuse"
      ],
      "events": [
        {
          "eventDate": "2011-04-12T17:56:54Z",
          "eventAction": "registration"
        },
        {
          "eventDate": "2025-11-18T00:26:57Z",
          "eventAction": "last changed"
        }
      ],
      "handle": "IRT-APNICRANDNET-AU",
      "remarks": [
        {
          "title": "remarks",
          "description": [
            "helpdesk@apnic.net was validated on 2021-02-09"
          ]
        }
      ],
      "vcardArray": [
        "vcard",
        [
          [
            "version",
            {},
            "text",
            "4.0"
          ],
          [
            "fn",
            {},
            "text",
            "IRT-APNICRANDNET-AU"
          ],
          [
            "kind",
            {},
            "text",
            "group"
          ],
          [
            "adr",
            {
              "label": "PO Box 3646\nSouth Brisbane, QLD 4101\nAustralia"
            },
            "text",
            [
              "",
              "",
              "",
              "",
              "",
              "",
              ""
            ]
          ],
          [
            "email",
            {},
            "text",
            "helpdesk@apnic.net"
          ],
          [
            "email",
            {
              "pref": "1"
            },
            "text",
            "helpdesk@apnic.net"
          ]
        ]
      ],
      "objectClassName": "entity"
    },
    {
      "links": [
        {
          "rel": "self",
          "href": "https://rdap.apnic.net/entity/ORG-ARAD1-AP",
          "type": "application/rdap+json",
          "value": "https://rdap.apnic.net/ip/1.1.1.1"
        }
      ],
      "roles": [
        "registrant"
      ],
      "events": [
        {
          "eventDate": "2017-08-08T23:21:55Z",
          "eventAction": "registration"
        },
        {
          "eventDate": "2023-09-05T02:15:19Z",
          "eventAction": "last changed"
        }
      ],
      "handle": "ORG-ARAD1-AP",
      "vcardArray": [
        "vcard",
        [
          [
            "version",
            {},
            "text",
            "4.0"
          ],
          [
            "fn",
            {},
            "text",
            "APNIC Research and Development"
          ],
          [
            "kind",
            {},
            "text",
            "org"
          ],
          [
            "adr",
            {
              "label": "6 Cordelia St"
            },
            "text",
            [
              "",
              "",
              "",
              "",
              "",
              "",
              ""
            ]
          ],
          [
            "tel",
            {
              "type": "voice"
            },
            "text",
            "+61-7-38583100"
          ],
          [
            "tel",
            {
              "type": "fax"
            },
            "text",
            "+61-7-38583199"
          ],
          [
            "email",
            {},
            "text",
            "helpdesk@apnic.net"
          ]
        ]
      ],
      "objectClassName": "entity"
    }
  ],
  "ipVersion": "v4",
  "endAddress": "1.1.1.255",
  "cidr0_cidrs": [
    {
      "length": 24,
      "v4prefix": "1.1.1.0"
    }
  ],
  "startAddress": "1.1.1.0",
  "objectClassName": "ip network",
  "rdapConformance": [
    "history_version_0",
    "nro_rdap_profile_0",
    "cidr0",
    "rdap_level_0"
  ]
}```
</details>

**LLM usage disclosure:** this PR was developed with LLM assistance. I have reviewed and verified all of the code, run it live end to end (the analysis above), and confirmed the output against the live RDAP service.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants