Docs might need updating

[jrichview]

I'm following along with this guide to get set up: https://github.com/codeforamerica/asap_pdf/blob/main/docs/windows_localdev.md (Yes, Windows. No cringing, please!)

I get through the Option 1 "Initial Setup" without any issue. The "Rails Application Setup" is probably targeted at a more experienced audience because Step 2 is vague. I added userid and password to config/database.yml in the development section. The command "rails db:migrate" does not error out but "rails db:setup" does.

[lkacenja]

Hi @jrichview, sorry for the slow response, just getting back from PTO. Glad to support our Windows friends, though I have a little less experience on that front. Your PostgreSQL service should have a user for the app. You should be able to follow standard PSQL user creation instructions for Windows. Something like this should work. After setting up a user in PSQL the credentials often have to be included in config/database.yml. You should be able to adapt the staging settings for development with your PSQL credentials hard coded. After that, the db:migrate and db:setup commands should work. If you can include the error you are seeing, I'd be happy to help debug.

Agreed that the docs need some updates. I don't think there is anything on updating the config. I'll try to get those updated. Thanks!

[jrichview]

Thanks for the response. I was finally able to get past the errors and run the web app, though I don't think I could accurately document how I did it. A lot of the issues seemed to derive from linux permissions issues, though not all.

Yes, a user was created for accessing the DB and an app-level admin user "admin@codeforamerica.org" was created, but there was no information how to log in as that user, or how to create your own ID/PW for the app.

Using chatGPT I managed to figure out how to do this, however. I guess some of this is just Ruby and Rails knowledge that I didn't have, but I took what chatGPT told me and corrected it by looking at the schema in pgAdmin4. Now I can log into the app.

Next I am trying to grasp the Python components readme. I believe it is building, but I need to get set up with appropriate Google Cloud / Gemini keys, and I'm working with my client to get that paid for.

Hope the PTO was good.

[lkacenja]

I'm glad you got the app up and running. Hopefully, the Python components are a little more straightforward. I'm working on making the PDF crawler easier to use in this PR. Once I get that merged, the crawling and classification steps might be something you could try without API credentials. You'd just need to add one or more of your sites to python_components/crawler/config.json and run the script.

If you hit any more snags, we'd be happy to set up a one hour tech assistance call if it would be helpful. Or I'm happy to help debug here. Thanks!

[jrichview]

The client I am evaluating this for is probably more interested in the piece with the actual identification of ADA compliance issues from the PDF more so than the crawling piece. I think many/most of their files are in a blob storage folder hive or otherwise fairly straightforward to access.

[lkacenja]

That makes sense. If you wanted to use the tool for anything beyond demo purposes, you'd need a csv of documents to import matching the format in db/seeds/site_documents.zip.

The app currently focuses on the audit process and doesn't have any tools for remediating the documents. It does have an LLM-driven feature that assesses whether a document might be exempt from the ADA rule. Maybe that would be helpful?

[jrichview]

The audit process is all I am concerned with at present. I think they just need a comprehensive list of what needs addressing.

I was looking at your Python code in document_inference/helpers.py (I am new to Python but have been a professional software developer since 1988) and it looks like the heart of it is in document_inference_summary() and document_inference_recommendation(), true? It seems to output a JPG-per-page of the PDF and then set the AI to analyze those? I'm wondering how reliable your experience has been with AI analysis? I've toyed with it a bit in a different situation and found it a lot less reliable than I would have expected, so I was just curious about your results.

FYI I have not actually run ASAP_PDF yet because I'm still going through red tape to get the Google Cloud account with access to Gemini API so I can try it.

[lkacenja]

Yes, that is all correct. We found that the image conversion approach allowed us to side-step OCR/text-extraction, which was convenient. We've been generally impressed by the reliability of the summary and analysis through our own usage. The two LLM-driven features behave relatively consistently on a small truthset we have for evaluation. Our partners are just getting going with the tool now, so we don't have aggregate data yet or their feedback, which would be more helpful in answering your question. Sorry not to be more helpful there.

[lkacenja]

I know it's a little late for your efforts, but I did deploy a big overhaul of the documentation today. Hopefully, it helps the next person. Thanks again for flagging.