What file storage requirements do I need?

Overview

CAST Imaging requires file storage as listed below:

Folder Default path on Microsoft Windows install Default path on Docker install Remarks
delivery On machine where analysis-node is installed: c:\cast-node\delivery On machine where analysis-node is installed: /shared/delivery Contains reference information about the application and the various source code deliveries that have occurred for a given application.

For a single machine install or where only one single analysis-node is installed, this can be left as is. For an enterprise/multi-node deployment, a mounted network folder that all nodes can access is required, e.g. S:\share\CAST\delivery (Microsoft Windows) /mnt/CAST/delivery (Linux via Docker).
deploy On machine where analysis-node is installed: c:\cast-node\deploy On machine where analysis-node is installed: /shared/deploy Used to store certain types of source code which require further extraction and processing before analysis, such as .castextraction files or PDS dump files.

For a single machine install or where only one single analysis-node is installed, this can be left as is. For an enterprise/multi-node deployment, a mounted network folder that all nodes can access is required, e.g. S:\share\CAST\deploy (Microsoft Windows) /mnt/CAST/deploy (Linux via Docker).
common-data On machine where analysis-node is installed: c:\cast-node\common-data On machine where analysis-node is installed: /shared/common-data A common location used for node related data (logs, backup, sherlock, source code upload and other folders used by the node).

For a single machine install or where only one single analysis-node is installed, this can be left as is. For an enterprise/multi-node deployment, a mounted network folder that all nodes can access is required, e.g. S:\share\CAST\common-data (Microsoft Windows) /mnt/CAST/common-data (Linux via Docker).
Logs Node: %PROGRAMDATA%\CAST\CAST\Logs Node (mapped container volume): /usr/share/CAST/CAST/Logs Used for storing all technical logs produced by the node with regard to code delivery/analysis activities. One sub-folder folder will be created per application onboarded in CAST Imaging. The log files will contain the path and the name of the source file. For some warning messages (e.g.: syntax error), the path of the source code involved in the warning is shown in the log.
LISA / LTSA Node: %PROGRAMDATA%/CAST/CAST/CASTMS/LISA and %PROGRAMDATA%/CAST/CAST/CASTMS/LTSA Node: /usr/share/CAST/CASTMS/LISA and /usr/share/CAST/CASTMS/LTSA (mapped container volumes) Location to store temporary files generated during the analysis process on each node:

- Large Intermediate Storage Area (LISA) - cleaned on completion of each analysis.
- Large Temporary Storage Area (LTSA) - is permanent and contains preprocessed source code files, archive files for analysis etc.

Some technology extensions need to preprocess the source code before performing the analysis. In this case, the source code that will be analyzed is stored in the LISA folder, not in the Deploy folder.
extensions Node: %PROGRAMDATA%/CAST/CAST/Extensions Node: /usr/share/CAST/Extensions (mapped container volume) Used for storing CAST technology extensions obtained by CAST Imaging Console from https://extend.castsoftware.com external link. Must always be located on the node machine and is required on each node.

Storage folder distribution strategy

The following can be configured (see deployment modes):

  • Local distribution only (only for single machine deployment)
  • Mixed local and network distribution (distributed deployment)
  • Network distribution only (distributed deployment)

We also describe the relationship between performance and scalability depending on the way you choose to deploy the storage folders. In addition, we suggest some required disk space recommendations (minimum recommendations and advised configuration for a large number of applications) depending on the chosen storage folder distribution strategy.

Local distribution (only for single machine deployment)

All storage folders are located on the local machine:

The limiting factor for this deployment is that all these folders can grow very quickly during an analysis, therefore, they can potentially overload the local disks on the machine.

Disk space recommendations

  • Minimum recommendation: 256GB
  • Run multiple applications in parallel: 1TB minimum

Mixed local and network distribution (distributed deployment)

The delivery/deploy/common-data and the logs folders are located on a shared network drive. Other folders are located on the local machine - i.e. the node:

The aim of using a shared network folder for the delivery/deploy/common-data and the logs folders is to accommodate the potentially large amount of storage that is required and to provide easy access to the files, particularly the logs folder. The size of these folders can grow very quickly during an analysis (in particular the delivery folder which stores multiple versions of a given application’s source code).

This configuration reduces the risk of overloading the local machine’s disk space, however, performance during analysis will be slightly reduced since the logs folder is located on the shared network drive, and an analyzer will require slightly more time to write to it during an analysis ((logs) can be located on the Node machine if required).

Disk space recommendations

  • Minimum recommendation:

    • Network storage (delivery/deploy/common-data/logs): 170GB
    • Local storage (LISA/LTSA/extensions): 86GB

  • Run multiple applications in parallel:

    • Network storage (delivery/deploy/common-data/logs): 700GB minimum
    • Local storage (LISA/LTSA/extensions): 300GB minimum

Network distribution only (distributed deployment)

All the storage folders, except “extensions” (which must always be located on the node), are located on a shared network drive:

This configuration will rarely overload the local disk space, however, performance during analysis is slower compared to other storage strategies for the following reasons:

  • The time required to load the source code in to memory from a network drive is inevitably longer.
  • Slower log file updates
  • Temporary files will be generated more slowly.

Disk space recommendations

  • Minimum recommendation:

    • Network storage (delivery/deploy/common-data/logs/LISA/LTSA): 256GB
    • Local storage (extensions): 50GB

  • Run multiple applications in parallel:

    • Network storage (delivery/deploy/common-data/logs/LISA/LTSA): 1TB minimum
    • Local storage (extensions): 50GB minimum