Cluster monitoring failed after upgrade
Applies to
- Active IQ Unified Manager 9.7P1
Issue
In the /var/log/ocie/server_mega.log we see :
2021-01-11 15:14:46,338 ERROR [DefaultQuartzScheduler_Worker-2] c.o.s.s.sessions (UpdateTaskExecuteSessionEJB.java:41) - Fail execute task com.netapp.oci.netapp.originator.PruneTask: com.netapp.oci.server.UpdateTaskException [380-261-1594]
Failed to run PruneTask user::prune:
com.netapp.oci.server.UpdateTaskException [380-261-1594]
Failed to run pruneTask
com.netapp.oci.platform.common.table.TableException: Failed to execUpdate: DELETE FROM netapp_model.change_record WHERE updateTime < 1610374435209
at com.netapp.mega@9.7.0-2020.05.J3245//com.netapp.oci.platform.common.table.PreparedStatementWrapper.executeUpdateEx(PreparedStatementWrapper.java:380)
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.netapp.originator.PruneTask.prune(PruneTask.java:107)
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.netapp.originator.PruneTask.prune(PruneTask.java:99)
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.netapp.originator.PruneTask.run(PruneTask.java:83)
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.server.UpdateTask.runTask(UpdateTask.java:45)
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.server.UpdateTaskExecute.run(UpdateTaskExecute.java:51)
at deployment.mega-app.ear.server-core.jar//com.onaro.sanscreen.server.sessions.UpdateTaskExecuteSessionEJB.execute(UpdateTaskExecuteSessionEJB.java:39)
at jdk.internal.reflect.GeneratedMethodAccessor520.invoke(Unknown Source)
.
Caused by: java.sql.SQLException: (conn=169) Lock wait timeout exceeded; try restarting transaction
.
: com.netapp.oci.server.UpdateTaskException [380-261-1594]
Failed to run PruneTask user::prune:
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.server.UpdateTask.runTask(UpdateTask.java:51) [server-core.jar:9.7.0-2020.05.J3245]
at deployment.mega-app.ear.server-core.jar//com.netapp.oci.server.UpdateTaskExecute.run(UpdateTaskExecute.java:51) [server-core.jar:9.7.0-2020.05.J3245]
at deployment.mega-app.ear.server-core.jar//com.onaro.sanscreen.server.sessions.UpdateTaskExecuteSessionEJB.execute(UpdateTaskExecuteSessionEJB.java:39) [server-core.jar:9.7.0-2020.05.J3245]
In the cat /var/log/mysql/error.log | grep -i sql
[System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.22) starting as process 27110
[System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
[System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.22' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
[System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.22).
[System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.22) MySQL Community Server - GPL.
[Warning] [MY-010915] [Server] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
[System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.22) starting as process 27649
[System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
[System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.22' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
[System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.22).
[System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.22) MySQL Community Server - GPL.
[Warning] [MY-010915] [Server] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
[System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.22) starting as process 31129