I am using airflow 1.10.9 and celery worker. I have dags which run whenever task comes and it spins up new ec2 instance and it connects to RDS on the basis of logic but ec2 holds the connection even when there no task is running and it keeps holding connection until Auto scaling scales down the instance.
RDS Details -
- Class : db.t3.xlarge
- Engine : PostgreSQL
I have checked the RDS logs but no luck.
LOG: could not receive data from client: Connection reset by peer
here is RDS connections.
state | wait_event | wait_event_type | count
--------+---------------------+-----------------+-------
| AutoVacuumMain | Activity | 1
| BgWriterHibernate | Activity | 1
| CheckpointerMain | Activity | 1
idle | ClientRead | Client | 525
| LogicalLauncherMain | Activity | 1
| WalWriterMain | Activity | 1
active | | | 1
All the connections are from celery workers.
Any help is appreciated.
