Knex: Timeout acquiring a connection. The pool is probably full.

[betamax]

Describe the bug

I'm running a Ceramic node (on Clay testnet) which is being interacted with from ComposeDB. I figured as the output is coming from Ceramic this issue should go here, but if not please let me know and I can post it to the ComposeDB repo. In the logs I'm seeing some messages such as:

HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}

I had issues before with requests failing and then when I try and edit something later, getting CACAO Expired messages, due to the commits not being anchored, so I thought I should post this in case it causes that again.

To Reproduce

Not exactly sure how to reproduce as it seems rather dependent on the CAS.

Logs

Oct 30 19:06:30 discourse-graph-demo ceramic[31031]: [2022-10-30T19:06:30.471Z] WARNING: Anchor failed for commit bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q of stream kjzl6kcym7w8y6m7gc6xlapspgeo260wlsa6dhi8ryq4ajvljtnbx1fly28fj01: HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqceraxek6uqj3bxpvgbujwqdt7ao26w2unzoig5fzfmviyzzfwow5wf3q failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}
Oct 30 16:46:06 discourse-graph-demo ceramic[31031]: [2022-10-30T16:46:06.213Z] WARNING: Anchor failed for commit bagcqcera2jdme27qzuan5w3vn7tkzqsfk2dstnvl2tw3nmuvxub4d3bsrd4q of stream kjzl6kcym7w8y9c18xtd4e4tcm5m9qx263rql9tz7u5q59etcp0suxmiood5xwo: HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqcera2jdme27qzuan5w3vn7tkzqsfk2dstnvl2tw3nmuvxub4d3bsrd4q' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqcera2jdme27qzuan5w3vn7tkzqsfk2dstnvl2tw3nmuvxub4d3bsrd4q failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}
Oct 30 16:22:41 discourse-graph-demo ceramic[31031]: [2022-10-30T16:22:41.625Z] WARNING: Anchor failed for commit bagcqcera7sv7okibecewmyr32xx3uwvcj2j62e7anuats4hf32j54gynexna of stream kjzl6kcym7w8y5idp4mfqave7vs9ose3h76llhwmuwe6xnpzt8qhjpejvhpr3a4: HTTP request to 'https://cas-clay.3boxlabs.com/api/v0/requests/bagcqcera7sv7okibecewmyr32xx3uwvcj2j62e7anuats4hf32j54gynexna' failed with status 'Bad Request': {"error":"Loading request status for CID bagcqcera7sv7okibecewmyr32xx3uwvcj2j62e7anuats4hf32j54gynexna failed: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"}

Ceramic versions

2.8.0

Machine, OS, browser information (please complete the following information):

Ubuntu 18.04.6 LTS

[stbrody]

Hi @betamax
Are you using SQLite as your ComposeDB backing database, or PostgresQL? If SQLite, could you try switching to Postgres and see if that fixes things?