No data in the dashboard after running "docker compose run --rm amass enum -d owasp.org" successfully on clean build

[Columbus-droid]

Setup

Ubuntu 24.04.1 LTS

Docker Desktop 4.37.1 (please note that there seems to be an issue where Docker Desktop won't run on my current LTS. Therefore, I use this workaround: https://askubuntu.com/a/1512215)

I have to use bash since I get an error when using zsh.

I deleted all containers, images and volumes prior to this test.

I pulled the latest version of the repository (64069de)

Since I ran into unexpected behavior when following the instructions in the "Configure the Conpose Enviroment" as per the docs I omitted this step. It is all marked as a recommendation and as far as I can tell, it should default to the passwords set in the config files if not specifically set here. This did not help to fix the errors, but I hope it will at least make it easier to trace the source of the error

Expected Behavior

When running the command

docker compose run --rm amass enum -d owasp.org

in bash from the folder amass-docker-compose created by pulling the repository, docker should pull all nececary images, build and run them and execute

amass enum -d owasp.org

The results of this should be available through the graphana dashboard on the dashboard on: https://localhost:3000 as per this video: Attack Surface Mapping with Jeff Foley from OWASP Amass

Actual Behavior

Docker succsessfully pulls and builds all containers and runns the provided command. When opening "localhost:3000" however there is no data available in graphana. When opening the developer tools and looking at the network tab I see that all queries made to: http://localhost:3000/api/ds/query return a Status Code 400.
Here is a sample response of such a query:

{
    "results": {
        "domains": {
            "error": "db query error: pq: no PostgreSQL user name specified in startup packet",
            "errorSource": "",
            "status": 500,
            "frames": [
                {
                    "schema": {
                        "refId": "domains",
                        "meta": {
                            "typeVersion": [
                                0,
                                0
                            ],
                            "executedQueryString": "SELECT DISTINCT substring(zone.name from '(([a-zA-Z0-9]{1}|[_a-zA-Z0-9]{1}[_a-zA-Z0-9-]{0,61}[a-zA-Z0-9]{1})[.]{1}[a-zA-Z]{2,61}$)') AS name FROM ((SELECT entity_id, content-\u003e\u003e'name' as name FROM entities WHERE etype = 'FQDN' \nAND entity_id IN (SELECT from_entity_id FROM edges WHERE etype = 'SimpleRelation' AND content-\u003e\u003e'label' = 'node')) as zone \nINNER JOIN edges ON zone.entity_id = edges.from_entity_id) \nWHERE edges.etype = 'PrefDNSRelation' AND edges.content-\u003e\u003e'label' = 'dns_record' AND edges.content-\u003e'header'-\u003e'rr_type' = '15'"
                        },
                        "fields": []
                    },
                    "data": {
                        "values": []
                    }
                }
            ]
        }
    }
}

I can successfully log in to the database created using:

Host: localhost:55432
User: grafana
PW: grafana

As specified in config/grafana/grafana.ini in the section: [database].
This leads me to believe that the database has been set up correctly.

---

EDIT 1:

After checking to see if any tables were created using DataGrip, I cannot see any tables in the "Default schema"

---

START EDIT 2:

After running the same command against a different URL. I first checked the logs of the container "postal". Here I can see active logs that I assume relate to the query I ran:

IVACY++REDACTED+FOR+PRIVACY+CH resp_time=0.296973 ser_name=postal_server status=200
2025-01-13 15:20:05 2025-01-13T14:20:05Z INF Request client_ip=172.xx.x.x method=GET path=/parse?address=infomaniak+network+societe+anonyme+redacted+for+privacy+redacted+for+privacy+redacted+for+privacy+chemin resp_time=0.357176 ser_name=postal_server status=200
2025-01-13 15:20:05 2025-01-13T14:20:05Z INF Request client_ip=172.xx.x.x  method=GET path=/expand?address=Infomaniak+Network+SA+REDACTED+FOR+PRIVACY+REDACTED+FOR+PRIVACY++REDACTED+FOR+PRIVACY+CH resp_time=0.306961 ser_name=postal_server status=200
2025-01-13 15:20:05 2025-01-13T14:20:05Z INF Request client_ip=172.xx.x.x  method=GET path=/parse?address=infomaniak+network+societe+anonyme+redacted+for+privacy+redacted+for+privacy+redacted+for+privacy+chemin resp_time=0.356319 ser_name=postal_server status=200

(I redacted the IP addresses queried)

Then I looked at the logs of "assetdb". After the command I ran had finished executing as per the status bar in the CLI, the following logs were printed:

2025-01-13 04:38:41 2025-01-13 03:38:41.874 UTC [103] LOG:  checkpoint starting: time
2025-01-13 04:38:42 2025-01-13 03:38:42.368 UTC [103] LOG:  checkpoint complete: wrote 5 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.415 s, sync=0.026 s, total=0.494 s; sync files=5, longest=0.009 s, average=0.006 s; distance=4 kB, estimate=7004 kB; lsn=0/33C2FB0, redo lsn=0/33C2F58

Now when I once again checked the DB using DataGrip I could see the database "assetdb" had been created, and it contained five tables in its public schema. When I tried fetching these, I got an error:

[42501] ERROR: permission denied for table entity_tags

Changing from:
User: grafana
PW: grafana

to

User: amass
PS: amass4OWASP

I can now query the contents of the tables. As far as I can tell, they look much like what can be seen in this video: Attack Surface Mapping with Jeff Foley from OWASP Amass at about 1h30min.

Despite this, the dashboard still shows the same behavior as before.

END EDIT 2

---

When checking the logs of the container: "assetdb" I can see a series of entries like the following, every time I manually reload the dashboard, or when the dashboard queries the db:

2025-01-13 03:51:25 2025-01-13 02:51:25.607 UTC [1087] FATAL:  no PostgreSQL user name specified in startup packet

This seems to indicate that the frontend is trying to connect to the db but that there is a problem in how that happens. Sadly, I am not savvy enough with docker to troubleshoot the exact root cause of this.

Inspecting the logs of the "grafana" container yields a series of entries like this:

2025-01-13 03:57:17 logger=context userId=0 orgId=1 uname= t=2025-01-13T02:57:17.010285072Z level=info msg="Request Completed" method=POST path=/api/ds/query status=400 remote_addr=172.18.0.1 time_ms=4 duration=4.232089ms size=381 referer="http://localhost:3000/?from=now-7d&orgId=1&refresh=1m&timezone=browser&to=now&var-addresses=%24__all&var-domains=google.com&var-geocity=&var-geoips=%24__all&var-geolist=%24__all&var-organizations=Google+LLC&var-subs=%24__all" handler=/api/ds/query status_source=downstream

Any help in resolving this would be greatly appreciated.
Thank you

Steps to Reproduce the Problem

1. Start with a "clean slate" (remove all old images, containers and volumes from previous attempts. Use an unmodified version of, 64069de)

2. Navigate to the repository root ("amass-docker-compose" if name left unchanged)

3. Using bash run

docker compose run --rm amass enum -d owasp.org

4. Wait for all containers to download, build and execution to finish.

5. Open https://localhost:3000

6. There should be a warning in several tiles saying: "No data"

7. Open the DevTools and navigate to the "Network" tab

8. Inspect the queries returning an error (status 400)

9. Open Docker Desktop and inspect the logs of the container: "assetdb"

Disclaimer

It is absolutely possible that I simply did not read / misunderstood the documentation, and this is totally my fault. If so, I would be grateful for anyone to point this out to me and kindly reference the correct part of the documentation to read. I am fairly new to this, but I am willing to learn if given the chance.
Thanks

Changelog

13.01.24

• Corrected minor spelling mistakes
• Added an EDIT 1 under "Actual Behavior"
• Added an EDIT 2 under "Actual Behavior"

[Ben10R]

Hi, did you modifed the grafana ds.yaml file ?

[Columbus-droid]

Hi, did you modifed the grafana ds.yaml file ?

Hello,
No, as stated in "Steps to Reproduce the Problem" I started from a completely clean slate, i.e.: Removed all files from my disk, deleted all containers, deleted all images and deleted all volumes created by those images. Then downloading a fresh copy and without making any modifications, I started the whole thing up with the command:

docker compose run --rm amass enum -d owasp.org

I am using the latest version of docker available for my system, and all available updates have been installed.

[Ben10R]

Alright,
I'm also trying to run Amass compose but it seems that the documentation has not been updated.

The error 'no PostgreSQL user name specified in startup packet' that you're getting from Grafana is because the data sources file ds.yaml mounted in the container is using environnement variables, which are not set in the compose file making the username/password for data sources blank in Grafana.

You can add to the docker compose file the following environnement for Grafana service to solve the problem:

• AMASS_DB=${AMASS_DB}
• AMASS_USER=${AMASS_USER}
• AMASS_PASSWORD=${AMASS_PASSWORD}

But the provisioned Dashboards are set up to use an older version of the assetdb, you'll need to update the SQL queries in Grafana with the correct assetdb tables and column names.

That's my understanding of the project, there may be better fix.

[higoje]

I noticed that grafana data was removed on main branch, is that correct? How can I see the data now?