OnCommand Insight polling issue for ONTAP datasource due to rapid postpone and resume
Applies to
- OnCommand Insight 7.3.10
Issue
- ONTAP data source occasionally stuck in postpone and has to be manually resumed.
- Errors showing that rapid postpone and resume within seconds via internal script was being taken place manually and causing a hang.
X:\SANscreen\wildfly\standalone\log\rest.log
shows:
rest-1.log:2020-12-02 04:44:39,834 INFO [default task-41700] ExceptionController (null:-1) - REST API call for [POST] /rest/v1/admin/datasources/250/poll returning code 400, error type: OPERATION_FAILED, message: Poll request denied. Datasource <datasource_name> is postponed.
rest.log:2021-01-28 17:26:43,814 INFO [default task-122240] ExceptionController (null:-1) - REST API call for [POST] /rest/v1/admin/datasources/250/poll returning code 400, error type: OPERATION_FAILED, message: Poll request denied. Datasource <datasource_name> is postponed.
X:\SANscreen\wildfly\standalone\log\rest-access.log
shows:
rest_access-1.log:2021-01-29 17:34:49,355 INFO [default task-125285] log (null:-1) - 128.161.186.86 - wfaadmin [29/Jan/2021:17:34:49 +0000] "POST /rest/v1/admin/datasources/250/postpone" 200 - "-" "null" 46 rest_access-1.log:2021-01-29 17:34:59,418 INFO [default task-125332] log (null:-1) - 128.161.186.86 - wfaadmin [29/Jan/2021:17:34:59 +0000] "POST /rest/v1/admin/datasources/250/resume" 200 - "-" "null" 47 rest_access-1.log:2021-01-29 17:39:54,501 INFO [default task-125350] log (null:-1) - 128.161.186.86 - wfaadmin [29/Jan/2021:17:39:54 +0000] "POST /rest/v1/admin/datasources/251/postpone" 200 - "-" "null" 63 rest_access-1.log:2021-01-29 17:40:04,548 INFO [default task-125301] log (null:-1) - 128.161.186.86 - wfaadmin [29/Jan/2021:17:40:04 +0000] "POST /rest/v1/admin/datasources/251/resume" 200 - "-" "null" 32 rest_access-2.log:2021-01-28 17:26:30,658 INFO [default task-122249] log (null:-1) - 128.161.186.86 - wfaadmin [28/Jan/2021:17:26:30 +0000] "POST /rest/v1/admin/datasources/250/postpone" 200 - "-" "null" 78 rest_access-2.log:2021-01-28 17:26:40,736 INFO [default task-122240] log (null:-1) - 128.161.186.86 - wfaadmin [28/Jan/2021:17:26:40 +0000] "POST /rest/v1/admin/datasources/250/resume" 200 - "-" "null" 47 rest_access-2.log:2021-01-28 17:26:42,658 INFO [default task-122239] log (null:-1) - 128.161.186.86 - wfaadmin [28/Jan/2021:17:26:42 +0000] "POST /rest/v1/admin/datasources/250/postpone" 200 - "-" "null" 47 rest_access-2.log:2021-01-28 17:26:52,705 INFO [default task-122240] log (null:-1) - 128.161.186.86 - wfaadmin [28/Jan/2021:17:26:52 +0000] "POST /rest/v1/admin/datasources/250/resume" 200 - "-" "null" 31 rest_access-2.log:2021-01-28 17:32:37,883 INFO [default task-122215] log (null:-1) - 128.161.186.86 - wfaadmin [28/Jan/2021:17:32:37 +0000] "POST /rest/v1/admin/datasources/251/postpone" 200 - "-" "null" 31 rest_access-2.log:2021-01-28 17:32:47,930 INFO [default task-122198] log (null:-1) - 128.161.186.86 - wfaadmin [28/Jan/2021:17:32:47 +0000] "POST /rest/v1/admin/datasources/251/resume" 200 - "-" "null" 47 rest_access-4.log:2021-01-22 05:47:46,692 INFO [default task-104489] log (null:-1) - 128.161.186.86 - wfaadmin [22/Jan/2021:05:47:46 +0000] "POST /rest/v1/admin/datasources/297/postpone" 200 - "-" "null" 47 rest_access-4.log:2021-01-22 05:47:58,032 INFO [default task-104490] log (null:-1) - 128.161.186.86 - wfaadmin [22/Jan/2021:05:47:57 +0000] "POST /rest/v1/admin/datasources/297/resume" 200 - "-" "null" 46 rest_access-4.log:2021-01-22 05:51:00,119 INFO [default task-104818] log (null:-1) - 128.161.186.86 - wfaadmin [22/Jan/2021:05:51:00 +0000] "POST /rest/v1/admin/datasources/298/postpone" 200 - "-" "null" 60 rest_access-4.log:2021-01-22 05:51:10,302 INFO [default task-104748] log (null:-1) - 128.161.186.86 - wfaadmin [22/Jan/2021:05:51:10 +0000] "POST /rest/v1/admin/datasources/298/resume" 200 - "-" "null" 31 rest_access-5.log:2021-01-20 10:41:35,436 INFO [default task-100015] log (null:-1) - 128.161.186.86 - wfaadmin [20/Jan/2021:10:41:35 +0000] "POST /rest/v1/admin/datasources/250/postpone" 200 - "-" "null" 63 rest_access-5.log:2021-01-20 10:41:45,655 INFO [default task-100032] log (null:-1) - 128.161.186.86 - wfaadmin [20/Jan/2021:10:41:45 +0000] "POST /rest/v1/admin/datasources/250/resume" 200 - "-" "null" 47 rest_access-5.log:2021-01-20 10:57:21,798 INFO [default task-100060] log (null:-1) - 128.161.186.86 - wfaadmin [20/Jan/2021:10:57:21 +0000] "POST /rest/v1/admin/datasources/251/postpone" 200 - "-" "null" 62 rest_access-5.log:2021-01-20 10:57:31,861 INFO [default task-100040] log (null:-1) - 128.161.186.86 - wfaadmin [20/Jan/2021:10:57:31 +0000] "POST /rest/v1/admin/datasources/251/resume" 200 - "-" "null" 31 rest_access-5.log:2021-01-20 11:02:46,258 INFO [default task-100077] log (null:-1) - 128.161.186.86 - wfaadmin [20/Jan/2021:11:02:46 +0000] "POST /rest/v1/admin/datasources/250/postpone" 200 - "-" "null" 78 rest_access-5.log:2021-01-20 11:02:56,319 INFO [default task-100074] log (null:-1) - 128.161.186.86 - wfaadmin [20/Jan/2021:11:02:56 +0000] "POST /rest/v1/admin/datasources/250/resume" 200 - "-" "null" 37 rest_access.log:2021-02-01 05:25:07,610 INFO [default task-132787] log (null:-1) - 128.161.186.86 - wfaadmin [01/Feb/2021:05:25:07 +0000] "POST /rest/v1/admin/datasources/297/postpone" 200 - "-" "null" 78 rest_access.log:2021-02-01 05:25:17,688 INFO [default task-132767] log (null:-1) - 128.161.186.86 - wfaadmin [01/Feb/2021:05:25:17 +0000] "POST /rest/v1/admin/datasources/297/resume" 200 - "-" "null" 47 rest_access.log:2021-02-01 05:27:46,559 INFO [default task-132760] log (null:-1) - 128.161.186.86 - wfaadmin [01/Feb/2021:05:27:46 +0000] "POST /rest/v1/admin/datasources/298/postpone" 200 - "-" "null" 47 rest_access.log:2021-02-01 05:27:59,341 INFO [default task-132767] log (null:-1) - 128.161.186.86 - wfaadmin [01/Feb/2021:05:27:57 +0000] "POST /rest/v1/admin/datasources/298/resume" 200 - "-" "null" 2110
X:\SANscreen\wildfly\standalone\log\webuistats.log
shows:
webuistats.log:[2020-12-17 09:07:10,581] [43265500@hsbc] page.adminDatasourceDetails.postpone.start:298:1|c:1608196023 webuistats.log:[2020-12-17 09:07:10,581] [43265500@hsbc] page.adminDatasourceDetails.postpone.period.3:298:1|c:1608196023 webuistats.log:[2020-12-17 09:13:10,558] [43265500@hsbc] page.adminDatasourceDetails.postpone.resume:298:1|c:1608196390 webuistats.log:[2021-01-01 13:01:49,809] [43265500@hsbc] page.adminDatasourceDetails.postpone.start:268:1|c:1609506107 webuistats.log:[2021-01-01 13:01:49,809] [43265500@hsbc] page.adminDatasourceDetails.postpone.period.3:268:1|c:1609506107 webuistats.log:[2021-01-01 13:04:49,831] [43265500@hsbc] page.adminDatasourceDetails.postpone.resume:268:1|c:1609506280