For Cockroach DB 20.2.5, getting 'Caused by: org.postgresql.util.PSQLException: ERROR: root: memory budget exceeded

,

After deployment of CDB v20.2.5 on servers, after some time we started getting this error at very frequent rate:

org.springframework.orm.jpa.JpaSystemException: could not extract ResultSet; nested exception is org.hibernate.exception.GenericJDBCException: could not extract ResultSet
Caused by: org.hibernate.exception.GenericJDBCException: could not extract ResultSet
Caused by: org.postgresql.util.PSQLException: ERROR: root: memory budget exceeded: 40960 bytes requested, 984028160 currently allocated, 984028160 bytes in budget

I guess, here ‘bytes in budget’ refers to cache memory of CDB, and according to error since its full, its not able handle new requests.

After these errors, we started getting connection timeouts for CDB.

My questions :

  • is that how cache should work, ideally it should replace data for any new requests
  • are Connection timeouts are consequence of these errors.?
  • Can Native JDBC queries impact this issue as hibernate cache is not present over there.

NOTE : Here cache memory is 0.25times of main memory as recommended by CDB

is that how cache should work, ideally it should replace data for any new requests

This error does not refer to any particular cache. It generally refers to the SQL module running against its limit, which is controlled by --max-sql-memory. Do you have really “big” queries?

are Connection timeouts are consequence of these errors.?

I’m surprised by this. I would expect errors instead of timeouts. If you could figure a good repro, we’d love to take a look.

Can Native JDBC queries impact this issue as hibernate cache is not present over there.

Don’t know, sorry. Or perhaps I don’t understand the question.