AIP Console - Troubleshooting

Created by James Hurrell on Mar 14, 2023

Summary: A list of issues and problems that may occur and how to solve them.

All releases

ZIP archive file is not processed

If your ZIP archive file is not accepted by CAST Console, or causes the onboarding action to not finish, then the ZIP may be corrupt or otherwise incorrectly compressed. CAST recommends unzipping the archive file and then re-zipping to ensure that the ZIP is created correctly.

≥ 2.x

Push Back Buffer is full - ≥ 2.x

This fatal error message can occur when running a fast scan (i.e. when using the onboarding with Fast Scan workflow - see Administration Center - Settings - Application Onboarding) on a very large ZIP file. This cause of this error message is due to a bug in the AIP Core engine itself and a fix has been provided in AIP Core 8.3.49. Therefore if you encounter this issue, please upgrade your Node instances to AIP Core ≥ 8.3.49.

1.x

Console or Node packages will not start

If the Console package or Node package(s) will not start (whether using a Windows Service or batch files to start the packages) the following log files should first be checked to see whether any relevant information has been recorded:

  • %PROGRAMDATA%\CAST\AipConsole\AipConsole\logs\webi_error.log
  • %PROGRAMDATA%\CAST\AipConsole\AipConsole\logs\webi.log
  • %PROGRAMDATA%\CAST\AipConsole\AipNode\logs\cmsapi_error.log
  • %PROGRAMDATA%\CAST\AipConsole\AipNode\logs\cmsapi.log

If anything relevant has been recorded and is listed in this page, please follow the remediation instructions below, otherwise, please open a ticket with CAST Support quoting the message from the log.

java.lang.IllegalArgumentException: Illegal character(s) in message header value - corrupt Extend API key

This error message indicates that the Extend API key (stored in the Node H2 Java database) providing access to CAST Extend is corrupt. To update the key, use the following instructions:

  • Ensure all Node services are stopped
  • On each Node, open a command prompt (CMD) using the right click "Run as Administrator" option.
  • Using the CMD window, move to the relevant location on disk where the run-h2-console.bat batch file is stored e.g.: cd %PROGRAMFILES%\CAST\AipConsole\AipNode\admin
  • Then run the batch file itself: run-h2-console.bat
  • This batch file will launch the following interface in the default browser:

Click to enlarge

The JDBC URL field should already be populated with the correct URL to access the H2 Java database, however, you can check this is correct by opening the following files and searching for the line spring.datasource.url:

  • %PROGRAMDATA%\CAST\AipConsole\AipNode\aip-node-app.properties

Click Connect to connect to the H2 Java database:

Click to enlarge

If the connection is refused, please remove AUTO_SERVER=TRUE from the end of the JDBC URL.

When connected, select the SETTINGS table:

Click to enlarge

Now edit the LICENSE entry to ensure that it is correct:

Click Save to store the correct value:

Close the H2 Console and restart the Node package.

java.lang.OutOfMemoryError: GC overhead limit exceeded

This error message indicates that the Java service for the package requires more RAM memory than has been allocated to it. To resolve the issue, see Configuring RAM for AIP Console front-end and AIP Nodes.

liquibase.lockservice.StandardLockService - Waiting for changelog lock....

Symptoms

The issue:

  • will cause the associated package to not start:
    • Console will not respond - a 404 "page not found" will be displayed when attempting to access the Console front-end 
    • the Node will appear to be "down" in Console.
  • will cause the following entries to be posted to the following log files:
    • %PROGRAMDATA%\CAST\AipConsole\AipConsole\logs\webi.log
    • %PROGRAMDATA%\CAST\AipConsole\AipNode\logs\cmsapi.log
15:47:45.771 [main] INFO  liquibase.executor.jvm.JdbcExecutor - SELECT LOCKED FROM PUBLIC.DATABASECHANGELOGLOCK WHERE ID=1
15:47:45.771 [main] INFO  liquibase.lockservice.StandardLockService - Waiting for changelog lock....
  • cannot be cleared by attempting to restarting the Console or Node package
Description

Both the Console and Node packages use a H2 Java database to store persistent data. These database are located here:

  • %PROGRAMDATA%\CAST\AipConsole\AipConsole\db
  • %PROGRAMDATA%\CAST\AipConsole\AipNode\db

During certain operations such as (but not limited to) upgrading to a new release of Console/Node, an issue may occur which causes the H2 Java database to remain locked. This will prevent the relevant package from starting.

Resolution

The first step is to ensure that the relevant package is shut down (either Console or Node) - stop the Windows Service or close the running CMD Windows.

Next, to resolve this issue, you need to run a query against the H2 Java database to remove the lock manually. To do this, CAST provides the H2 Java database client with both the Console and Node packages. This can be run using a batch file also provided in the package. Ensure you choose the appropriate batch file according to the package in which the error is occurring:

  • %PROGRAMFILES%\CAST\AipConsole\AipConsole\admin\run-h2-console.bat
  • %PROGRAMFILES%\CAST\AipConsole\AipNode\admin\run-h2-console.bat

You should run these batch files with elevated administrative permissions - in other words:

  • Open a command prompt (CMD) using the right click "Run as Administrator" option.
  • Using the CMD window, move to the relevant location on disk where the batch file is stored e.g.: cd %PROGRAMFILES%\CAST\AipConsole\AipConsole\admin or cd %PROGRAMFILES%\CAST\AipConsole\AipNode\admin
  • Then run the batch file itself: run-h2-console.bat

This batch file will launch the following interface in the default browser:

Click to enlarge

The JDBC URL field should already be populated with the correct URL to access the H2 Java database, however, you can check this is correct by opening the following files and searching for the line spring.datasource.url:

  • %PROGRAMDATA%\CAST\AipConsole\AipConsole\aipConsole.properties
  • %PROGRAMDATA%\CAST\AipConsole\AipNode\aip-node-app.properties

Click Connect to connect to the H2 Java database:

Click to enlarge

If the connection is refused, please remove AUTO_SERVER=TRUE from the end of the JDBC URL.

When connected, select the DATABASECHANGELOGLOCK table:

Click to enlarge

In the SQL statement tab, paste in the following query, removing any query that is already present in the tab:

UPDATE DATABASECHANGELOGLOCK set LOCKED=false, LOCKGRANTED=null, LOCKEDBY=null where ID=1;

Click to enlarge

Now run the query by clicking the Run tab:

Click to enlarge

Finally restart the AIP Console or AIP Node package.