From: Tobias Hagelborn <tobiasha@axis.com>
To: <bitbake-devel@lists.openembedded.org>
Subject: [PATCH] Re-enable connection pooling with psycopg 3 driver
Date: Tue, 20 Feb 2024 07:27:05 +0100 [thread overview]
Message-ID: <20240220062705.3611869-1-tobiasha@axis.com> (raw)
Re-enable connection pooling in case `postgresql+psygopg` driver
is used. Async connection pooling is supported in psycopg 3 [psycopg]
driver in SQLAlchemy.
Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
---
Connection pooling was disabled since asyncpg does not support it.
With psycopg3 driver it is possible to re-enable connection pooling again.
I wanted to show that it is possible to get the performance back again
and avoid using other, more complex deployments such as pgbouncer to
achieve connection pooling.
Checking the driver url is possibly a bit hackish.
Having some command line parameter would result in a larger change
and also force the user to know which drivers do support pooling properly.
I tested this setup with some of my stress tests and with a batch of builds
and it seems to hold.
lib/hashserv/sqlalchemy.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/hashserv/sqlalchemy.py b/lib/hashserv/sqlalchemy.py
index b17a17621..e33527e80 100644
--- a/lib/hashserv/sqlalchemy.py
+++ b/lib/hashserv/sqlalchemy.py
@@ -93,7 +93,11 @@ class DatabaseEngine(object):
async def create(self):
self.logger.info("Using database %s", self.url)
- self.engine = create_async_engine(self.url, poolclass=NullPool)
+ if self.url.drivername == 'postgresql+psycopg':
+ # Psygopg 3 (psygopg) driver can handle async connection pooling
+ self.engine = create_async_engine(self.url)
+ else:
+ self.engine = create_async_engine(self.url, poolclass=NullPool)
async with self.engine.begin() as conn:
# Create tables
--
2.30.2
next reply other threads:[~2024-02-20 6:27 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-20 6:27 Tobias Hagelborn [this message]
2024-02-20 15:35 ` [bitbake-devel] [PATCH] Re-enable connection pooling with psycopg 3 driver Joshua Watt
2024-02-23 10:58 ` Tobias Hagelborn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240220062705.3611869-1-tobiasha@axis.com \
--to=tobiasha@axis.com \
--cc=bitbake-devel@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).