Troubleshooting Dying Database Connections in Java Tomcat

When working with Java applications hosted on Tomcat, a common issue developers encounter is the unexpected termination of database connections, particularly after periods of inactivity. This can manifest as errors in logs indicating that the last successful packet was sent a significant amount of time ago, forcing you to reconnect and causing potential downtime or application errors. In this blog post, we’ll explore this problem in detail and provide actionable solutions to prevent these connections from dying.

Understanding the Problem

Upon noticing a dying database connection, you may encounter an error like this in your logs:

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: 
The last packet successfully received from the server was68051 seconds 
ago. The last packet sent successfully to the server was 68051 seconds 
ago, which is longer than the server configured value of 
'wait_timeout'.

This error essentially indicates that MySQL has terminated the connection because it exceeded the server’s wait_timeout setting, which defines how long the server waits for activity on a connection before closing it. The default value can often be too low for applications that do not maintain a continuous stream of activity.

Solutions to Maintain Database Connection Health

Here are several strategies to help you address this issue and maintain stable database connections in your Java application running on Tomcat:

1. Adjusting the Database Configuration

You may want to consider modifying the wait_timeout setting on your MySQL database. While this isn’t always the most practical solution, it’s worth discussing with your database administrator. Here’s how to adjust it:

SET GLOBAL wait_timeout = 28800;  -- 8 hours
SET GLOBAL interactive_timeout = 28800;  -- 8 hours

2. Modify the context.xml Configuration

In the Tomcat’s context.xml file, take advantage of additional settings to enhance connection management. Below are a couple of useful properties that can prevent connections from dying:

  • Remove Abandoned Timeout: This setting ensures connections that are abandoned after a specified period are removed from the pool.
removeAbandonedTimeout="60"

This would make sure that if a connection is inactive for more than 60 seconds, it gets closed.

  • Test While Idle and Eviction Settings: Implement connection tests and define intervals for eviction runs. Try adding these properties in your context.xml:
testWhileIdle="true"
timeBetweenEvictionRunsMillis="300000"  <!-- Every 5 minutes -->

3. Utilize autoReconnect Option

Though you’ve already added autoReconnect=true in your JDBC URL, it’s important to note that this does not resolve the underlying issue of inactive connections and can mask further problems. Use it with caution while implementing the additional settings to ensure all bases are covered.

4. Establish Robust Connection Handling

Beyond configuration, consider these best practices for handling connections in your Java application:

  • Always Close Connections: Ensure every database connection is properly closed after use. Utilize try-with-resources in Java to automatically manage this.
  • Pooling Libraries: If applicable, leverage a robust connection pooling library like HikariCP, which provides excellent performance and reliability.

Conclusion

By understanding why your database connections are dying and implementing the suggested strategies, you can significantly enhance the stability of your application. Make sure to adjust the configurations in your context.xml, and review your database settings to optimize performance. Regular monitoring and maintenance of your connection settings can lead to a smoother operation and improved application experience.

By utilizing the collective power of these strategies, you can effectively prevent issues associated with dying database connections and ensure your Tomcat applications run uninterrupted.