- Typical communication between CAST components
- Required components
- Optional components
- Storage considerations
- Source code replication
- RAM considerations for AIP Console (front-end) and AIP Nodes
- Windows Services user account considerations - Microsoft Windows only
- Proxy considerations
- Which deployment architecture should you use?
- Type of Applications being onboarded
Typical communication between CAST components
The following diagram shows the interaction between AIP Console and the main CAST Components. It also shows the flow from the source code delivery to the CAST Imaging data consumption:
Click to enlarge
Required components
| Package/Component | Description |
|---|---|
| CAST Imaging | CAST Imaging is the component used to display and investigate the analysis results:
|
| AIP Core | AIP Core is the "backbone" analysis engine:
|
| CAST Storage Service / PostgreSQL | This component provides the storage (i.e. database schemas) for the analysis result data. It can be installed:
CAST highly recommends the use of PostgreSQL on a Linux instance as this consistently gives the best performance. |
| AIP Node service (back end) | The AIP Node "back end" service contains the RESTful APIs that interact with AIP Core on the same host server and the remote CAST Storage Service/PostgreSQL, AIP Console and CAST Imaging instances. This package:
|
| AIP Console service (front end) | The AIP Console "front end" service connects and manages all the AIP Node(s). Users will connect (using a browser) directly to this package to manage the analyses of their Applications. This package:
|
Optional components
| Package/Component | Description |
|---|---|
| CAST Extend Offline or CAST Extend local server | These components are only required if (due to security concerns) your organization/AIP Console cannot interact over the public internet with CAST's Extend system for Extension management. The components provide CAST's Extend system as an "offline" on-premises component:
|
Storage considerations
As shown in the above diagram, there are five main storage folders (marked in yellow). All AIP nodes must have access to these folders:
| Folder | Location in v. 2.x | Location in v. 1.x | |
|---|---|---|---|
| delivery | Common shared folder (obligatory) | AIP Node (default) or Common shared folder if necessary | Used for storing successive and compressed versions of an application's source code produced during the source code delivery phase. This folder contains all the versions of the source that have been delivered. This folder is usually the largest of all storage folders. |
| deploy | Common shared folder (obligatory) | Used in Standard delivery mode for storing an application's source code in uncompressed format - this is the code that is analyzed. When the delivery is accepted and imported, the deploy folder is cleaned and the current version of the source code is deployed in the deploy folder. | |
| common-data | Common shared folder (obligatory) | N/A | A common location used for AIP Node related data (backup, sherlock, upload and other folders used by the AIP Node). |
| logs | AIP Node (default) or Common shared folder if necessary | AIP Node (default) or Common shared folder if necessary | Used for storing all logs produced by the AIP Node with regard to code delivery/analysis/snapshot activities. One sub-folder folder will be created per Application onboarded in AIP Console. The log files will contain the path and the name of the source file. For some warning messages (e.g.: syntax error), the path of the source code involved in the warning is shown in the log. |
| LISA / LTSA | Location to store temporary files generated during the analysis/snapshot process on each AIP Node:
Note that some analyzers/extensions, need to preprocess the source code before performing the analysis. In this case, the source code that will be analyzed is stored in the LISA folder, not in the Deploy folder. | ||
| extensions | Used for storing AIP extensions. Must always be located on the node. |
When an analysis is run, the analyzed source code files are uploaded into the Analysis schema (local).
Storage folder distribution strategy
This section explains the different ways that can be used to distribute the required storage folders:
- Local distribution only (v. 1.x only, in 2.x a shared folder is required for delivery/deploy/common-data (except where using an all on one machine deployment for demos and POCs))
- Local and network distribution
- Network distribution only
We also describe the relationship between performance and scalability depending on the way you choose to deploy the storage folders. In addition, we suggest some required disk space recommendations (minimum recommendations and advised configuration for a large number of applications) depending on the chosen storage folder distribution strategy.
This icon depicts the server on which the AIP Node service and AIP Core are installed.- CAST Storage Service / PostgreSQL can be installed on the AIP Node instance or on a separate and dedicated server (recommended).
Local distribution: better performance / lower scalability - v. 1.x only
All the storage folders are on the local server - i.e. the AIP Node:
The limiting factor for this deployment is that all these folders can grow very quickly during an analysis, therefore, they can potentially overload the local disks on the server.
Disk space recommendations
| Minimum recommendation | Local storage (all folders) | 256GB |
|---|---|---|
| Run multiple applications in parallel (minimum) | Local storage (all folders) | 1TB |
Balance between storage and performance: medium performance / medium scalability
The Delivery/Deploy/Common-data and the Logs folders are located on a shared network drive. Other folders are located on the local server - i.e. the AIP Node instance:
The aim of using a shared network folder for the Delivery/Deploy/Common-data and Logs folders is to accommodate the potentially large amount of storage that is required. The size of these folders can grow very quickly during an analysis (in particular the Delivery folder which stores multiple versions of a given Application's source code).
This configuration reduces the risk of overloading the local server's disk space, however, performance during analysis will be slightly reduced since the log file is located on a shared network drive, and an analyzer will require slightly more time to write to it during an analysis.
Disk space recommendations
| Minimum recommendation | Local storage (LISA and LTSA and Extensions folders) | 86GB |
|---|---|---|
| Network storage (Delivery/Deploy/Common-data and Log folders) | 170GB | |
| Run multiple applications in parallel (minimum) | Local storage (LISA and LTSA and Extensions folders) | 300GB |
| Network storage (Delivery/Deploy/Common-data and Log folders) | 700GB |
Network distribution: lower performance / better scalability
All the storage folders, except "extensions" (which must always be located on the node), are located on a shared network drive:
This configuration will never overload the local disk space, however, performance during analysis is slower compared to the first deployment scenario for the following reasons:
- The time required to load the source code in to memory from a network drive is inevitably longer.
- Slower log file updates
- Temporary files will be generated more slowly.
Disk space recommendations
| Minimum recommendation | Local storage (Extensions folders) | 50 GB |
|---|---|---|
| Network storage (All other folders) | 256GB | |
| Run multiple applications in parallel (minimum) | Local storage (Extensions folders) | 50GB |
| Network storage (All other folders) | 1TB |
Note that to analyze .NET applications with assemblies stored in a remote network location, you will need to alter the machine configuration by proceeding as described in the .NET Analyzer in the section Prerequisites.
Source code replication
The following diagram shows how your source code is replicated among the various different storage locations during the onboarding and analysis process:
RAM considerations for AIP Console (front-end) and AIP Nodes
See Configuring RAM for AIP Console front-end and AIP Nodes for more information.
Windows Services user account considerations - Microsoft Windows only
Both the AIP Console front-end and the AIP Node back-ends are run as Windows Services on the relevant Windows machine. CAST highly recommends that the Local System account is not used to run these Windows Service. This is particularly true if:
- you need to use a proxy service to access specific resources such as CAST Extend (see Complete start-up wizard - v. 1.x or Administration Center - Settings - Proxy)
- you are using shared network resources to store data such as Delivery and Deploy items (see Configure AIP Node storage folder locations - optional - v. 1.x)
In both these situations, the user running the Windows Service will be used to access the proxy/shared network resources.
Instead, CAST recommends using the login credentials that match the log in used to install AIP Console/AIP Node/AIP Core/set system proxy settings etc. - for example, this could be a specific "service account" that is created specifically for installing and running AIP Console/AIP Nodes/AIP Core/setting system proxy settings. This service account would also therefore have access to the shared network resources and would be able to use the system proxy settings.
Proxy considerations
If your organization requires the use of a proxy, please take the following into account:
Extension Downloader limitation
At the current time, the Extension Downloader (a tool present on each AIP Node which is used by AIP Console to download extensions) cannot be configured to obey a manual proxy configuration defined in AIP Console. Instead, if your organization uses a proxy, CAST recommends that:
- you define the required proxy configuration at system level (i.e. operating system level) on all AIP Nodes
- define a manual proxy configuration using the settings described below - this ensures that everything else will connect through the proxy
Windows Services
AIP Console and the AIP Nodes packages are configured to run through Windows Services, therefore it is important to ensure that the user login configured to run the Windows Services has permission to access any proxy that you define. If the user running the Windows Services cannot access the proxy, then AIP Console/AIP Nodes will not be able to access the required resources. See the sections regarding the configuration of the Windows Services in:
Which deployment architecture should you use?
| Deployment scenario | Component | Server Type | OS | Recommended deployment architecture |
|---|---|---|---|---|
| Multi-node (Large/Enterprise) | CAST Imaging | Physical / Virtual | Microsoft Windows or Linux | Installation CAST Imaging on a dedicated dedicated Windows or Linux server. |
| AIP Core | Physical / Virtual | Microsoft Windows | AIP Core/AIP Node package
| |
| AIP Node (back end) | ||||
| AIP Console (front end) | Physical / Virtual | Microsoft Windows or Linux | Installation of the "front-end" AIP Console package on a dedicated Windows or Linux server to manage all the AIP Nodes. | |
| CAST Storage Service (CSS) / PostgreSQL | Physical / Virtual | Microsoft Windows or Linux | Details of the CAST Storage Service/PostgreSQL instance will be required during the installation of each AIP Node package: all that is required is that the AIP Console installer can access all required CAST Storage Services/PostgreSQL. How you choose to deploy the CAST Storage Service/PostgreSQL generally depends on the types of Application you are onboarding using the AIP Console, however, CAST recommends the following as a minimum:
| |
| CAST Extend Offline or CAST Extend local server | Physical / Virtual | Microsoft Windows | If you need to deploy the CAST Extend Offline or CAST Extend local server, details of this service will be required during the initial start up of AIP Console: all that is required is that the wizard can access CAST Extend Offline/Proxy. CAST recommends using either:
| |
| Single server (Small/Simple) | CAST Imaging | Physical / Virtual | Microsoft Windows | If you have one single AIP Core installation that you would like to manage with the AIP Console, choose this option, which involves the installation of all packages/components on one single Windows server. Note that:
|
| AIP Core | ||||
| AIP Node (back end) | ||||
| AIP Console (front end) | ||||
| CAST Storage Service (CSS) / PostgreSQL | ||||
| CAST Extend Offline or CAST Extend local server |
Type of Applications being onboarded
The method and hardware chosen to deploy CAST components and packages depends on the type of the Application(s) you intend to onboard with AIP Console. Various factors related to the Application(s) you are analyzing must be considered:
- Number of lines of code (LOC) in the Application (in general, CAST considers an Application with over 3 million lines of code to be "large", however, this figure is not set in stone and smaller Applications may require dedicated resources).
- Number of objects after analysis
- Number of violations after analysis
- Technologies used
- Number of users accessing results
When onboarding "large" applications you should carefully consider how you deploy the required components and packages - particular attention should be paid to:
- AIP Core/AIP Node - dedicate one server for each large Application - or if this is not possible, run analyses of one single large application one at a time
- CAST Storage Service / PostgreSQL - dedicate one server and CAST Storage Service/PostgreSQL instance for each large Application - use Linux/PostgreSQL to improve performance
