Skip to content

403 Forbidden when accessing filing endpoint programmatically #50

Description

@DhavalGojiya

Describe the bug
403 Forbidden when accessing filing endpoint programmatically.
Browser says it is 404 status code, while the python requests module says it is 403 for same URL

To Reproduce

# example.py

import fecfile

# 1954520 is paper filing fec file
filing1 = fecfile.from_http('1954520')

print("Successful +++++")

Now Run python example.py

Put a breakpoint() here inside .venv/lib/python3.12/site-packages/fecfile/__init__.py :: LINE - 81 and inspect the behaviour.

if r.status_code == 404:

Expected behavior
The filing1 = fecfile.from_http('1954520') work as expected

Environment
Python3.12

Additional context
(Browser says 404 for same url)

URL: https://docquery.fec.gov/dcdev/posted/1954520.fec
Image

-> if r.status_code == 404:
(Pdb) print(r)
<Response [403]>
(Pdb) url
'https://docquery.fec.gov/dcdev/posted/1954520.fec'
(Pdb) print(r.status_code)
403

Possible fix

See:

if r.status_code == 404:
    url = "https://docquery.fec.gov/paper/posted/{n}.fec".format(
        n=file_number
    )
    r = requests.get(url, headers=req_headers, stream=True)

If the requests module returns 404, then this fallback URL is used and everything works as expected.

However, in some cases the initial request returns 403 instead of 404.
Updating the condition to also handle 403 may resolve the issue:

if r.status_code in (403, 404):

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions