SODALITE Use Cases

Abstract

This chapter describes the various use cases of the SODALITE project in more detail. Each use case is representative of a unique infrastructure and operational environment supported by SODALITE: Snow (Cloud/OpenStack), Clinical Trials (HPC/Torque), and Vehicle IoT (Cloud + Edge / Kubernetes). Each section includes an overview of the specific challenges faced by the use case, an overview of the use case system architecture, target infrastructure, and operational environment, followed by ways in which the SODALITE approach was successfully applied to address the unique challenges of the use case. Each section concludes with a brief review of the benefits that the use case achieved through the application of SODALITE.

7.1 Introduction

The SODALITE approach has been applied during the development of the project to three different case studies. They have shown to be quite complementary one with respect to the other and have allowed us to experiment with different aspects of the SODALITE platform.

The rest of this chapter is dedicated to the presentation of all the three cases. More specifically, Sect. 7.2 presents the Snow use cases, which has been already introduced in Chap. 5; Sect. 7.3 presents the in silico clinical trials use case, which shows strict QoS requirements and the need to be executed on HPC; Sect. 7.4 provides an overview of the Vehicle IoT case, which introduces a new challenging execution environment, that is, the edge. Finally, Sect. 7.5 concludes the paper.

7.2 Snow Use Case

The Snow use case has been briefly introduced in Chap. 2. This section provides further details on it and on how the usage of the SODALITE platform has been beneficial for modeling the use case deployment and for its execution.

The use case is about collection of images from multiple sources, their analysis and transformations to deduce from them information about the amount of snow available on mountains and, therefore, the available reserve of water. Application components are logically organized in a pipe and filter approach and are mostly executed on cloud resources. In this context, SODALITE offers proper mechanisms to model the deployment of the system and to control its operation and reconfiguration.

The following subsections will elaborate more on the motivation behind the use case, describe the main components of the use case, and provide more details on how SODALITE was used to deploy the use case and on the achieved benefits.

7.2.1 Background and Motivation

With climate change impacting the planet, the issue of water scarcity has entered political discourse globally. Even in countries that do not yet suffer water poverty, conservation has become a prominent goal. Making good use of water implies predicting its availability and planning usage in a way that satisfies the potentially conflicting objectives of industry, agriculture, and the common people.

In mountain regions, the water stock is essentially preserved in the form of snow and permafrost. The Alps are the highest and most extensive mountain range system that lies entirely in Europe. The climate changes have a deteriorating impact on the region, negatively affecting the ecosystem as well as a population of 14 million people spread across eight countries. The problem is worsening with increasing recorded annual average temperatures and decreasing precipitation levels. At the same time, ground monitoring infrastructures have not been updated due to high investment requirements. This calls for novel instruments for the low-cost, high resolution, high accuracy monitoring of the dynamics of environmental events and conditions in mountain regions, especially in the Alps.

The Snow Use Case plans to provide a solution by improving the capillarity of mountain environment monitoring. It applies advanced image processing workflows capable of extracting useful environment information from large collections of publicly available mountain related multimedia data.

7.2.2 The Snow Detection Pipeline

The solution pursued in the Snow Use Case is an original method to derive information on mountain snow coverage from a massive amount of public web content (user generated photos and images captured from touristic web cams). The method uses an image processing workflow that aligns the picture taken by the user or crawled from a touristic web cam to a synthesized rendered view of the terrain that should be seen from the camera point of view; the synthetic panorama is generated by a projection applied to a (publicly available) Digital Elevation Model (DEM) of the Earth.¹ This supports decision-making in a snow dominated mountain context, e.g., the prediction of water availability from snow coverage in mountain peaks.

Content acquisition and processing rely on crawling geo-located images from heterogeneous sources at scale. The proposed approach elaborates content by assessing the presence of mountains in each photo, identifying individual peaks, and extracting a snow mask from the portion of the image denoting an identified mountain. Architecturally, the system is distributed between several data centers. The alignment step is computationally intensive and can run on CPU, but is best executed on GPUs. The alignment operation must be executed for every acquired image. In particular, the image analysis workflow extracts the skyline for an image taken in uncontrolled conditions, and thus must implement a multi-stage, GPU-intensive sequence of steps, including image normalization, jitter compensation, occlusion detection, and skyline extraction. Given the massive amount of images to be processed to monitor the entire Alpine region over a long period of time (in the order of millions of images per year), the workflow must be deployed on a massively scalable, GPU-enabled architecture.

Figure 7.1 shows the different components of the pipeline. Two main image sources are used: touristic webcams in the Alpine area and geo-tagged user-generated mountain photos in a 300 $\times$ 160 km Alpine region. The subsections below describe the components of the pipeline in details.

7.2.2.1 User Generated Image Crawler

This component crawls Web image sharing sites for images representing mountains. It takes as input the coordinates of a rectangular geographical region and mountain-related keywords, then fetches images from the specified area whose metadata contain the keywords. To do so, it opens a connection to the query API of the image sharing site, submits queries formulated with the input keywords and search area, retrieves images that match the query, downloads the images, saves them on disk and stores their metadata in the database. Flickr is exploited as the data source for user-generated photographs, as it contains a large number of publicly available images, many of which have an associated geotag (GPS latitude and longitude position saved in the EXIF (Exchangeable Image File Format) container of the photograph). Figure 7.2 shows two examples of crawled images.

Fig. 7.1

Components of the Snow use case pipeline

Fig. 7.2

Examples of crawled images

7.2.2.2 Mountain Relevance Classifier

Images tagged with a location within a mountainous region may not portray mountain landscapes. For this reason, the probability of mountain presence is estimated and the non-relevant photographs are discarded. This component takes as input an image, computes a fixed-dimensional feature vector that summarizes its visual content, and feeds the vector to a multi-layer perceptron classifier that determines whether the image is relevant. A dataset of images annotated with mountain/no-mountain labels is used to train the model. Figure 7.3 exemplifies positively (left) and negatively (right) classified images.

Fig. 7.3

Crawled image of a mountain (left) and not a mountain (right)

7.2.2.3 Webcam Image Crawler

Outdoor webcams represent a valuable source of visual content. Their images must be filtered by weather conditions, because these can significantly impact the visibility of mountains. Since snow cover changes slowly over time, multiple webcam images of the same day can be aggregated so as to produce only one representative sample per day. Public webcams expose a URL which returns the most recent available image. The webcam crawler loads the list of all the webcams and starts an asynchronous loop for each of them. At each iteration, it checks the current webcam image and adds it to the dataset if it is changed w.r.t. the previous access. It then idles for 1 s and starts over again. The image check is performed only on the first 5KB of the image, which are hashed and compared to the previous hash of the same webcam: if the hash is different, it is saved as the new hash and the whole image is downloaded. The first image acquired from every webcam is discarded, because it may be stale (some webcams, due to failures, expose the same images for days).

7.2.2.4 Weather Condition Filter

Clouds, rains and snowfalls may hinder visibility significantly and thus only a fraction of the acquired images can be exploited for estimating snow cover. The weather condition filter is based on the assumption that, if the visibility is sufficiently good, the skyline mountain profile is not occluded. The component takes as input a webcam image and outputs a Boolean value indicating if a mountain skyline is visible or not.

7.2.2.5 Daily Median Image Aggregation

Good weather images might suffer from challenging illumination conditions (such as solar glare and shadows) and moving obstacles (such as clouds and persons in front of the webcam). Yet snow cover changes slowly over time, so one image per day is sufficient. This component aggregates the images collected by a webcam in a day and outputs a single image per webcam obtained by applying the median operator along the temporal dimension. Such a median image also removes temporary lighting effects and occlusions.

7.2.2.6 Skyline Extraction

To geo-reference the snow cover, it is necessary to determine which portions of the image represent which mountains and estimate for each pixel its content (terrain or sky) and the corresponding GPS position, altitude, and distance from the observer. Mountain image geo-registration is done by finding the correct overlap between the photograph and a 360-degree virtual panorama. To align the photo and the virtual panorama, the landscape skyline is extracted from the photo, by finding the pixels that represent the boundary between the terrain and the sky with a pixel-level binary classifier. The classifier is trained using a dataset of mountain images, annotated with the skyline boundary. The component takes as input an image, the camera field of view and shooting position, and outputs a skyline mask. Figure 7.4 shows a sample image (left) and the extracted skyline represented as a binary mask (right).

Fig. 7.4

Sample image (left) and extracted skyline represented as a binary mask (right)

7.2.2.7 360 Panorama Generation

The virtual panorama is a synthetic image of the visible mountain skyline generated by projecting the Digital Elevation Model (DEM) of the Earth provided by the NASA SRTM data set from the camera shooting position. The component is deployed as a service that takes as input the coordinates of a location and generates an image corresponding to the 360-degree panorama visible from that point. Figure 7.5 shows an example of a 360-degree virtual panorama with the skyline visible from a given location.

7.2.2.8 Panorama Alignment

The alignment between the skyline of the real image and of the virtual panorama can be seen as the search for the correct overlap between two cylinders: one containing the 360-degree virtual panorama and the other one containing the skyline extracted from the photo. The component takes as input the real and virtual panorama images and returns the real image annotated with a mask that represents the portion of the image that contains the mountain registered with the virtual panorama. Figure 7.6 shows an example of alignment between the 360-degree virtual panorama and the skyline extracted from the image.

Fig. 7.5

Example of 360-degree virtual panorama with the skyline visible from a given location

Fig. 7.6

Example of alignment between the 360-degree virtual panorama and the skyline extracted from the image

Fig. 7.7

Example of an image and of the snow mask generated from it

7.2.2.9 Snow Mask Computation

A snow mask is the output of a pixel-level binary classifier that, given an image and a mask M that represents the mountain area as input, produces a mask S that assigns each pixel of the mountain area a binary label denoting the presence of snow. Snow masks are computed using the Random Forest supervised learning classifier with spatio-temporal median smoothing of the output. The classifier is trained with images annotated at the pixel level indicating if the pixel corresponds to the snow area. Figure 7.7 shows an example of an image and its snow mask.

7.2.2.10 Snow Index Computation

The pipeline produces a pixel-wise snow cover estimation from images, along with a GPS position, camera orientation, and mountain peak alignment. Thanks to the image geo-registration with the DEM data it is possible to estimate physical and geographical properties of every pixel, including its type (snow/no snow) and altitude. Consequently, it is possible to compute the snow line altitude (the point above which snow and ice cover the ground) expressed in meters.

7.2.3 How SODALITE is Used

The Snow Use Case takes advantage from SODALITE in multiple ways. More specifically:

• to enable training of the skyline extractor, we exploit SODALITE ability to automate the deployment of application components on HPC resources and to optimize their execution;

• to manage the allocation of heterogeneous resources (i.e., CPUs and GPUs) at runtime, we used component Node Manager that allows to efficiently provision resources with the goal of minimizing SLA violations

• to automate the deployment of the whole pipeline on a cloud infrastructure, we rely on the definition of an articulated Abstract Application Deployment Model that containerizes application components and distributes them on different virtual machines;

• to inject images from data sources, e.g. webcam, we have used the integration between NiFi and SODALITE.

More details on these aspects are described in the following subsections.

7.2.3.1 Deployment on HPC

The training of the skyline extraction component has been optimized for the execution on HPC resources. The goal of this component is to obtain the landscape skyline of a photograph via a DL classification method run in TensorFlow. The dataset used for the training consists of 8,856 images with skyline annotations, from which 80% is used for training and validation and the remaining 20% for testing. The component was initially trained using TensorFlow 1.11. The training was performed with a baseline container taken from DockerHub (tensorflow/tensorflow:1.11.0-gpu-py3) and converged within approximately 7.2 h on one GPU node of the HPC testbed (using single core execution). The training executed until convergence was achieved and early stopping initiated at epoch 20.

The first step of the optimization process was the porting of the Python training code for TensorFlow 2.2, as it has been optimized by the developers to outperform the outdated TensorFlow 1.11. Therefore, we built an optimized Singularity container with TensorFlow 2.2. As a sanity check, we performed a run until convergence which finished within 8056 s (approximately 2.3 h) and 20 epochs. As training times converge across epochs within 2-3 epochs, we trained the skyline extractor for 5 epochs across every further optimization we considered. For the initial Singularity container with TensorFlow 2.2, that took 3473 s, of which 872 s constitute training time, while the rest includes data batching time. This is a well-known bottleneck for DL applications involving massive datasets. To account for this, we optimized the Python code to perform batch dataset prefetching to the GPU via the TensorFlow Data API. This shortens the execution pipeline by performing training and data input concurrently. The training time thus improved to 2181 s, of which 514 s constitute training time. As a final optimization, we optimized the data movement by staging the dataset on an SSD attached to the GPU node. The dataset was moved to the SSD, and the dataset directory passed to Singularity via file binding. This optimization improved the training time to 424 s, of which 236 s constitute training time. This yields an 8.2x speedup improvement over the initial TensorFlow 2.2 run. We tested additional optimizations such as using XLA and various combinations of SSD, GPU prefetching, and XLA, but these did not yield significant improvements. Finally, we executed the optimized container up to convergence. It takes 2042 s with 21 epochs. Overall, this is a 12.7x speedup.

MODAK (see Sect. 4.4) can be used to automate the process of choosing an optimal container, thus returning the best possible container, in this case one that stages the dataset to an SSD.

7.2.3.2 Resource Management

As shown in Sect. 5.6 component Node Manager is able to manage a set of applications that share a cluster of heterogeneous resources. Users set SLAs for each applications and the Node Manager (i) deploys containers for the applications using Kubernetes, (ii) schedules incoming requests on either GPUs or CPUs according to application needs, (iii) dynamically allocates CPU cores to avoid SLA violations. In the context of Snow UC we configured Node Manager to control application skyline extractor along with other third party apps (ResNet, GoogLeNet, and VGG16) on a shared cluster deployed on the Azure public cloud. The cluster was composed of three virtual machines: one VM of type HB60rs with a CPU with 60 cores and 240GB of memory, and two VMs of type NV 6 equipped with a NVIDIA Tesla M60 GPU and a CPU with 6 cores and 56GB of memory. An additional instance of type HB60rs was used for generating the client workload. Different shaped, highly varying synthetic workloads were tested in all the experiments run and the different applications were run in random combinations concurrently on the servers.

The first type of experiment conducted was about varying either the input workload or the set-point of the system to test the ability of Node Manager to rapidly adapt the resource allocation to the new state. Results showed that the Node Manager is able to efficiently adapt to different unforeseen conditions. Figure 7.8 shows how Node Manager quickly reconfigures the resources when the SLA is changed (around second 80) for application skyline extractor to avoid SLA violations.

Fig. 7.8

Skyline extractor controlled by Node Manager

Fig. 7.9

Comparison

Node Manager was compared with a rule-based approach that schedules incoming requests using a round-robin approach on available CPUs and GPUs and dynamically scales the resources using a rule-base engine. Different synthetic workloads were tested and the Node Manager outperformed the baseline in all the experiments obtaining overall 96% fewer SLA violations while using 15% fewer resources. Figure 7.9 shows the different behavior of the systems ( with the same workload when all the four applications were running concurrently. While Node Manager can quickly react to changes, rule-based approach often violates the SLAs for applications ResNet and VGG16.

By adopting the Node Manager, use case owners can deploy components that exploit heterogeneous resources, set constraints on their response times and have the platform automatically managed for optimizing resource allocation and fulfill the desired goal. Node Manager is able to control multiple applications at the same time and to govern potential resource contention scenarios among concurrent applications. As clearly shown in Fig. 7.9, Node Manager outperforms a rule-based approach by order of magnitude. The SLA violations are minimized (96% improvement) and the resources are precisely allocated to the different containers (15% improvement).

7.2.3.3 Deployment on the Cloud

The Snow Use Case was modeled using SODALITE AADM to facilitate its deployment on the SODALITE testbed. Figure 7.10 shows how components were deployed onto two VMs provisioned using OpenStack. The first VM contains more components but they are generally less demanding of resources than the four deployed on the second virtual machine. Listing 7.1 reports an extract of the AADM. Lines 1–3 declare the snow module and import the necessary dependencies where Docker and OpenStack types are declared. Lines 5–10 define the inputs (partially omitted) of the AADM. These values must be prompted by the user before starting a deployment. Lines 13–22 show how the security rules for the OpenStack VMs are defined. In particular ports between 8080 and 8084 are opened so that Snow components can be used by clients after the deployment. Lines 24–41 declare the two VMs. These nodes use some of the inputs declared above (e.g., ssh-key-name) and are protected by the defined security rules (e.g., lines 36–37). Lines 43–53 define Docker related nodes, namely a Docker host, registry and network. Finally lines 55–82 report the definition of component snow-webcam-crawler. The node is declared as a container with an image that is assumed to be stored in the Docker registry defined at lines 47–49. The component has some key requirements: a docker host (lines 68–69), a MySQL database (lines 70–71) which definition is omitted herein, a Docker network (lines 72–73) and registry (lines 74–75).

Fig. 7.10

Deployment of the Snow UC on SODALITE testbed

7.2.3.4 Data Management

One of the tasks of Snow UC component - WebCam Crawler - is to pull images from public webcams and store the images in either local file system or in a MinIO (S3 compatible) object storage. Data Management component is an alternative to WebCam Crawler with the benefits for scalable data movement and support for various heterogeneous data sources. Data Management component creates a data pipeline, consisting of HTTP consumer and S3 Publisher - shown in Fig. 7.11, that allows to periodically get the images from the public webcam and store them in S3 storage.

Fig. 7.11

Data pipeline, connecting HTTP Consumer and S3 Publisher

Listing 7.2 shows an AADM for such data pipeline. The webcam-consumer component is a subpipeline that receives the JSON list of objects (List), containing URL of the webcams and any arbitrary data that can be used e.g. to name the images in a specific format in Filename, and downloads the images. These arbitrary data can be extracted through AttributeMap. The ListHTTPSchedulingPeriodTimer property sets the scheduling of pulling the images, e.g. once a minute. The connectToPipeline requirement connects the subpipeline to another subpipeline - S3 Publisher, which saves the downloaded images into the specific S3 bucket. Additional properties of the S3 Publisher include access and secret keys and the endpoint to the S3 storage.

7.2.4 Benefits of Using SODALITE

The Snow Use Case represents a typical scenario in which a complex data processing pipeline comprising components with very different non-functional requirements and resource usage profiles should be deployed and administered over a long period of time. The application targets public administrations, such as environment protection agencies and water bodies regulators, which makes simplicity of maintenance and cost effectiveness prominent requirements.

In such a scenario, the first benefit of SODALITE is the advantage of a model-driven approach. In fact, the definition of the Abstract Application Deployment Model proven to be an effective tool to raise to the conceptual level of architecture design the problem of defining the details related to the pipeline deployment, so as to enable an effective communication between the operation manager and the application owners. More specifically, an analysis of the pipeline from the deployment and operation perspective has highlighted a number of design and implementation issues concerning, among the others, the approach adopted to coordinate the pipeline components, the way to manage data transfer between components, the importance of clearly identifying all software layers to be wrapped in each component container, as well as the need to parametrize the application software for what concerns information such as IP addresses and user credentials. The importance of the AADM definition increases even further when considering the case in which the Snow system is hosted by a third party, a frequent case in PAs. In this case, the explicit definition of the AADM simplifies the migration from one provider to the other compared to the case when the deployment and configuration of the system is manually handled.

A second benefit is the monitoring of complex runtime conditions possibly spanning multiple components and measured by means of declarative rules. Such a capability goes well beyond the alarm triggers of commercial cloud platforms and is essential in such an articulated and heterogeneous pipeline as the one of the Snow Use case, where many things can get critical at the same time.

Other benefits, not even foreseen in advance, concern the possibility to take advantage not only of the cloud but also of HPC clusters for specific operations (the skyline extraction in our case), to exploit the Node Manager for increasing the efficiency of execution and to exploit SODALITE-ready mechanisms for data transfer.

7.3 In-Silico Clinical Trials Use Case

This section describes the in-silico clinical trials use case and how the usage of the SODALITE platform is beneficial for the use case execution. The use case reproduces real clinical trials in biomecanics by means of simulation to determine optimal bone implant systems for patients with spinal conditions. The workflow of the use case was originally executed on a specific HPC infrastructure, however, with SODALITE the hybrid execution of the workflow was achieved, additionally targeting cloud environments. Moreover, SODALITE provided a user interface, which assists in the development of deployment model for the use case workflow, and helped to optimise the runtime execution of certain components of the use case.

The following subsections will elaborate more on the motivation behind the use case and describe the simulation chain of the use case, as well as provide more details on how SODALITE was used to deploy the use case and what were the benefits of SODALITE usage for the use case.

7.3.1 Background and Motivation

The in-silico clinical trials for spinal operations use case targets the development of a simulation process chain supporting in-silico clinical trials of bone-implant-systems in Neurosurgery, Orthopedics, and Osteosynthesis. It deals with the analysis and assessment of screw-rod fixation systems for instrumented mono- and bi-segmental fusion of the lumbar spine by means of continuum mechanical simulation methods. As a novelty, we consider the uncertainty inherent in the computation by means of probabilistic programming. The simulation chain consists of a number of steps that need to be fulfilled in order and can be thought of as a pipeline. The output of each step serves as input to the next step.

The use case addresses one of the most prevalent health problems experienced by the populations of developed nations resulting in enormous losses of productivity and costs for ongoing medical care. The simulation process developed within this use case will optimize the screw-rod fixation systems based on clinical imaging data recorded during standard examinations and consequently target the lowering of the reported rates of screw loosening and revisions, enhance safety, expand the knowledge of the internal mechanics of screw-rod fixation systems applied to the lumbar spine and finally reveal optimization potential in terms of device application and design.

7.3.2 The Simulation Chain of Clinical Trials Use Case

The individual steps for the simulation chain can be seen in Fig. 7.12. First, the Image Processing and Filtering component receives three data sets reconstructed in different image planes and generates a high resolution image dataset. The Extraction component then takes this enhanced imaging dataset and extracts a geometry for the vertebral bodies. The de-facto standard for doing this is the marching cubes algorithm, for which many implementations exist.

Fig. 7.12

The simulation chain of the Virtual Clinical Trial use case

Next, the Discretization component generates a volume-mesh inside the surface geometry. This enables one to treat the mesh as a set of finite elements and to use the existing finite elements methodology. The Density Mapping component takes the original image data and maps it onto the volume-mesh. In doing this, each element inside the mesh is assigned a density value.

This enhanced meshed geometry is then fed into the Probabilistic Mapping component. Here the values for density are transformed into values for elasticity, as this is what is actually needed for the simulation. Because of the uncertainty that is inherent in this transformation, we use a probabilistic programming approach. Eventually, boundaries for the 95% highest density interval as well as the mode are computed.

These data are used in the input decks for the last step, the Solver component. Here, the finite element method is actually used in computing a solution that describes the structural mechanics inside the vertebral bodies.

7.3.2.1 Image Processing and Filtering

The image processing and filtering component is composed out of several processing steps, integrated into a pipeline via the Visualization Toolkit² (VTK), which provides all steps as ready-to-use algorithms with corresponding python bindings. As input the implemented pipeline takes three image series in VTK file format,³ which are reconstructed in different image planes i.e. whose resolution differs along the three coordinate axes as described in the Extraction section below. As output it delivers an integrated dataset in VTK file format.

Since the data are represented as image data, i.e. as a rectilinear grid, each dataset’s reconstruction plane is originally aligned along the xy-coordinate plane. Due to that the first step is to rotate them back into their original scanner coordinate system as specified in the header of the original DICOM image series. Subsequently all three datasets are resampled and interpolated to a rectilinear grid which provides high resolution in each of the three coordinate planes.

7.3.2.2 Extraction

In the original pipeline implementation the Extraction step was directly based on the clinical imaging datasets, which were first analyzed with respect to their content as well as their quality using 3D Slicer.

The datasets used for the developments were provided by the clinic for neurosurgery at the university medical center Knappschaftskrankenhaus Bochum, Germany as DICOM (Digital Imaging and Communications in Medicine) datasets. While there is no standard on how DICOM datasets have to be stored physically, they are organized logically by header information stored within each image. In Fig. 7.13 it can be seen that the datasets are logically organized firstly by patient and secondly by so called studies, which in turn can contain several series.

Even though the datasets contain several imaging modalities like full body X-ray images, dual energy X-ray absorptiometry (DXA) scans, magnetic resonance imaging (MRI) data, as well as computer tomography (CT) images, we currently concentrate on the CT-images as they are the basis for the three dimensional (3D) reconstruction of the spine’s bone structures.

Fig. 7.13

Logical organization of patient datasets

The analysis of the datasets revealed that, for each patient, three studies contain 5-8 series with CT-data, which are recorded pre-operatively (without the implant), directly after surgery (with the implant), and after several weeks during clinical control examination. Further on, it was found that out of the 5-8 series, each study contains at least three CT-series each with a different reconstruction plane. The remaining CT-series contained different information like dose reports, or in some cases additional CT-series treated with different smoothing kernels.

The left subfigures of Fig. 7.14 show an isosurface along with two cutting planes and resulting contour lines. As a basis for this feature extraction we took the CT-series from the pre-operative study, reconstructed in the x-y-plane. In the bottom-right subfigure the contour line of a vertebra in the x-y-plane is shown in detail. It can be seen that the vertebra’s contour was extracted smoothly. In the top-right subfigure the contour line of the same vertebra, taken on the y-z cutting plane is shown. Here it can be seen that the vertebra’s contour could not be extracted smoothly due to the low resolution of this CT-series in the y-z- direction.

Fig. 7.14

y-z cut contour line and x-y cut contour line - x-y reconstructed dataset

For the results shown in Fig. 7.15 the same feature extraction as shown in Fig. 7.14 was applied to the CT-series from the pre-operative study reconstructed in the y-z plane. In contrast to Fig. 7.14, in the top right subfigure of Fig. 7.15 it can be seen that the contour line on the y-z cutting plane in that case reproduces the vertebra’s contour smoothly. In the lower right subfigure of Fig. 7.15, however it can be seen that the contour in the x-y plane is no longer reproduced smoothly once the feature extraction is based on the CT-series reconstructed in the y-z plane.

Fig. 7.15

y-z cut contour line and x-y cut contour line - y-z reconstructed dataset

From these analyses it was concluded that exploiting only one CT-series for geometry extraction is not feasible and additional image processing steps have to be introduced as explained in the previous subsection.

7.3.2.3 Discretization

Based on the three surface meshes volume meshing of the bone geometries is performed. Targeted libraries for 3D volume mesh generation are currently Netgen or NGSolve. Additionally the boundary conditions i.e. the supports and the loadings have to be discretized and structural model features like muscle strands, tendons and cartilage have to be attached to the modeled bone geometries. As a result of this step the completed models will be written out as so called solver input decks still with a homogeneous material distribution.

7.3.2.4 Density Mapping

In this step, the three input decks as well as the three CT data sets are taken as input. By means of direct geometrical mapping, the grayscale distribution of the improved imaging data produced by the Image Processing and Filtering step is mapped onto the volume mesh provided by the discretization. After the mapping, each element in the volume mesh holds a density value. The Density Mapping step is written in Fortran 2003. Its initial implementation was started in 2007 and is documented by Schneider [1]. The algorithmic principles are published by Schneider et al. [2].

Since the original implementation relied on an internal data format and was not able to load DICOM datasets with the given complex structure, the data input part had to be modified. 3D Slicer, which was used for the analysis of the datasets, has a very advanced DICOM reader implemented and is also able to export the loaded data as VTK STRUCTURED_POINTS.⁴ Because of these features and since the Image Processing and Filtering stepis also based on VTK, it was decided to add the capability to load VTK STRUCTURED_POINTS data in VTK XML format to the Density Mapping component. By this extension, the Density Mapping step is now able to map the density distribution from the CT-data on the Meshed Geometry. This results basically in one density value per element.

To give an impression of the resulting density distribution, the final mapping result is visualized side by side with the original data in Fig. 7.16. Since the probabilistic mapping step acts locally on each element, no topological information has to be passed on between the density mapping component and the probabilistic mapping component. Everything that has to be passed is a list that contains one integer value per element representing its averaged density value. These values have to be passed as a file containing binary 64-bit integer values.

Fig. 7.16

Density Mapping component - Left: Input data - Right: Mapping result

7.3.2.5 Probabilistic Mapping

Based on the input from the Density Mapping component, the Probabilistic Mapping component produces three probability distributions per element, one for each of the three elastic moduli of an orthotropic stiffness matrix. This is done by means of the transfer functions between density and orthotropic elastic moduli given in Schneider et al. [2]. The low and high bounds and the mode of 95% confidence interval are computed from the aforementioned probability distributions. This finally results in the output of three files where each file contains three elastic moduli for each element.

Since these outputs have to be extended by shear moduli and poisson ratios to form complete, orthotropic material distributions and then become integrated with the Meshed Geometry to form a complete Input Deck, i.e. the Enhanced Meshed Geometry, the data path in the simulation process chain was changed from the original layout. Instead of treating the Enhanced Meshed Geometry directly with the Probabilistic Mapping component and passing its output directly to the Solver component, its output is now sent to the Applying Boundary Conditions component. This change in the data flow was decided during the parallel development of the Probabilistic Mapping component and the prototype of the Code_Aster⁵ model. During the development, it was recognized that it would amount to less effort to add the integration of the results from the Probabilistic Mapping component to the Applying Boundary Conditions component than to implement the treatment of the Enhanced Meshed Geometry by the Probabilistic Mapping component. This was due to the fact that in the Density Mapping component, from which we derived the Applying Boundary Conditions component, the algorithmic part was already implemented and only the output part had to be changed.

7.3.2.6 Applying Boundary Conditions

The Applying Boundary Conditions component is derived from the original implementation of the Density Mapping component. The component integrates each of the three output files of the Probabilistic Mapping component with the Meshed Geometry in Code_Aster med file format. It generates three Enhanced Meshed Geometries in Code_Aster’s med file format which include inhomogeneously distributed material Information needed by Code_Aster.

7.3.2.7 Solver

In the Solver component, a reference model prototype had been developed for ABAQUS and transferred to Code_Aster format. Initial results of this development are displayed in Fig. 7.17.

Fig. 7.17

Density Mapping component - Left: Input data - Right: Mapping result

Additionally, the coupling of the Applying Boundary Conditions component to the Solver component was implemented. The implementation of the coupling required basically the implementation of the mesh output in Code_Aster’s HDF5-based MED-format and the implementation of the output of Code_Aster’s input parameter file to the Applying Boundary Conditions component.

7.3.3 How SODALITE is Used

The in-silico Clinical Trials use case reproduces real clinical trials in biomecanics by means of simulation to determine an optimal fixation and function of bone implant systems for patients with spinal conditions (e.g. disk displacement or prolapse). The use case is being developed by HLRS and was originally strictly HPC driven, i.e. it was realised as a workflow executed on a particular HPC infrastructure provider. This limits the adoption of the developed methodology in biomechanical clinical trials for medical device manufacturers or medical research institutes due to the specifics of the target execution environment of the use case.

Therefore, Clinical UC expects SODALITE to be beneficial in moving the process towards production-like environments with the following improvements:

• 1. Increase the effectiveness and productivity of component deployment.

• 2. Ease the adaptation to different IT-infrastructures (supercomputers, Clouds, on-premise) and different hardware.

• 3. Lower the efforts for component integration.

• 4. Lower the efforts for data management.

The rest of the subsection presents how SODALITE platform was used in the scope of Clinical UC.

7.3.3.1 Clinical UC Workflow Orchestration

SODALITE produced a set of reference TOSCA libraries and Ansible playbooks⁶ for provisioning of VMs in Cloud (AWS EC2 and OpenStack) as well as execution of batch jobs on HPC clusters, managed by resource manager, such as PBS Torque and Slurm. Clinical UC validated these libraries by executing the workflow on the single and multiple HPC and Cloud targets using the SODALITE orchestrator. Additionally, SODALITE developed TOSCA libraries for data movement with GridFTP, a data transfer protocol widely offered by HPC infrastructure providers, thus enabling multitarget workflow execution of Clinical UC components with data dependencies.

Clinical UC benefited from this development as it allowed the distribution of the workflow execution from a single infrastructure target into multiple targets, utilising the capabilities offered by various providers. For example, less capable but more available virtual resources can be used for UC components that do not demand a lot of computation resources, while bare-metal and more capable resources can be used for more compute-demanding tasks. Possible setups for the workflow execution of the Clinical UC are presented later in Sect. 7.3.4.

7.3.3.2 Clinical UC Workflow Optimization

Together with optimization experts (Quality Experts), we focused on optimization of the Code_Aster Solver component, which is contributing the most to the total execution of the workflow. For Code_Aster, we compared the performance of the MODAK optimised container with that of the official Code_Aster container.⁷ The experiment was performed on the MC partition of the Cray XC50 “Piz Daint” supercomputer at the Swiss National Supercomputing Centre (CSCS). Each node of the system is equipped with a dual-socket CPU Intel Xeon E5-2695 v4 @2.10GHz (18 cores/socket, 64/128 GB RAM). On a single thread, the MODAK containerised application had an average execution time of 725 s, while the official Code_Aster container executed for 745 s. This is a 3% speed-up.

Furthermore, with the build of parallel Code_Aster, additional optimised container images are being prepared for parallel execution of the Solver, which is proven to reduce the execution time significantly. The runtime tests were carried out on the HLRS system Vulcan using computing nodes with Intel Xeon Gold 6138 @2.0GHz Skylake processors and 192 GB of main memory, and different underlying solver libraries (MUMPS⁸ and PETSc⁹) with and without METIS¹⁰ partitioner were applied. The results of the runtime tests are given in Table 7.1.

7.3.3.3 Clinical UC Workflow Modeling

SODALITE IDE offers a context-assistance and models validation that was useful during the modeling of the Clinical UC deployment. Moreover, SODALITE IDE offers optimization specification, which allows to create optimization recipes (with the help of Quality Experts) and apply optimizations to a particular application component, which in turn is executed using optimized container runtime.

Using SODALITE IDE, we modeled and deployed the single and multi-targets use case workflow executions on HLRS HPC testbed with the optimization applied to the Probabilistic Mapper for parallel execution, as described in Sect. 7.3.4. Resource Experts have provided HPC resource models and models for workflow execution. Clinical UC developers acted as Application Ops Experts and created an AADM (Abstract Application Deployment Model) for the use case deployment, utilizing provided resource models. Quality Experts helped to develop optimization recipes to enable MPI pluralization for Probabilistic Mapper, during runtime of which, it was built and executed within optimized container runtime.

During the development of AADM for Clinical UC, SODALITE IDE assisted with modeling, e.g. offering available node types and resolving requirements for node templates. Inconsistencies in deployment models (e.g. mismatch of node types in requirements) were also checked and reported back instantly at the development time. Since initially we started deployment modeling using TOSCA with a simple YAML editor, we found the usage of the SODALITE IDE extremely convenient, error reducing and effort saving for failure resolution and component integration.

Table 7.1 Results of bare metal runtime tests of parallel Code_Aster

7.3.4 Deployment Architecture

The components of the Clinical Trials UC can be deployed with SODALITE in various Cloud platforms and HPC systems. We validated the following deployment cases:

• 1. Single target HPC system managed by either PBS Torque or Slurm

• 2. Multiple HPC targets with data transfers using GridFTP

• 3. Cloud and HPC targets with data transfers using GridFTP.

The PBS Torque target is a bare-metal cluster and has MPICH as an MPI implementation, whereas the Slurm target is virtual (deployed via EGI EC3¹¹) and uses OpenMPI as an MPI implementation. As Cloud target, OpenStack and AWS EC2 is used. Both HPC and Cloud targets are modelled in an AADM that is reused across different cases.

To ensure portability of application components deployed over these targets, Singularity container runtime was used. In case of HPC, the compatibility should also be respected at the level of communication between processes with MPI. As such, the version and implementation variant of MPI (e.g. OpenMPI and MPICH) included in the container image must match with the MPI setup of the host. In the Clinical UC workflow, the Probabilistic Mapper component can be run in parallel with MPI. Optimization model is associated with this component, and this helps MODAK to select a container compatible with the MPI of the target host, i.e. either OpenMPI or MPICH will be selected, depending on what the target host supports.

In the first case, the partial workflow of Clinical UC is executed in a single HPC target, as depicted in Fig. 7.18. The AADM for this case consists of definition of workflow components, executed in the specific order defined with dependency relationship between the components. To define where the component should be deployed, the host relationship between the component and the target is used. In this case, we specify either PBS Torque or Slurm cluster for all the components.

Fig. 7.18

Deployment of Clinical UC over a single HPC target, either PBS Torque (used in HLRS HPC Testbed) or Slurm (used in EGI EC3 Virtual Cluster)

In the second case, the partial workflow of Clinical UC is distributed across multiple HPC targets, as depicted in Fig. 7.19. Density Mapper and Boundary Condition components are hosted in the Slurm cluster, whereas the Probabilistic Mapper is hosted in the PBS Torque cluster. Additionally, a GridFTP client is deployed in OpenStack, such that a 3rd party data transfer between the clusters is performed, thus allowing to transfer intermediate results of the workflow. Compared to the first case, the workflow order was extended by adding the data transfer steps between the component execution, to ensure that a dependent component will first receive the needed data from the previous step before its execution.

Fig. 7.19

Deployment of Clinical UC over multiple HPC targets with the third-party data transfers between the GridFTP servers, initiated by GridFTP client

The third case, depicted in Fig. 7.20 is similar to the second case with the difference that Density Mapper and Boundary Condition components are hosted in a VM (either AWS or OpenStack). The GridFTP client is co-located in the VM and moves the data to and from the HPC cluster after the respective component execution is finished.

Fig. 7.20

Deployment of Clinical UC over Cloud and HPC targets with the data transfers between the GridFTP client and server

7.3.5 Benefits of Using SODALITE

The workflow of Clinical UC was executed with the SODALITE platform. Modelling, optimisation and orchestration aspects of the platform were validated and evaluated in the context of the Clinical UC, showing the use case improvements in the following:

• 1. Originally, Clinical UC workflow was HPC-driven and executed in a single HPC cluster. With SODALITE, the workflow execution is distributed across multiple infrastructure targets.

• 2. The execution of the Solver (Code_Aster) component of Clinical UC in a container was optimised. Furthermore, parallel build and execution became possible, promising further reduced execution time.

• 3. The workflow was modelled in SODALITE IDE, which reduced the effort for development of the deployment code, compared to TOSCA, as it became easier to integrate new components and avoid possible deployment errors. Additionally, optimised container runtime can be automatically provided based on the optimisation options specified in SODALITE IDE.

7.4 Vehicle IoT Use Case

This section describes the Vehicle IoT use case and how the usage of the SODALITE platform is beneficial for the use case execution. The use case is centered around the development of services useful to support the management of car fleets. As discussed in the following subsections, the use case is composed of a significant number of components and services that run on different operational environments. End-user apps are typically executed on mobile devices of various kinds. Special services with strict latency requirements are executed at the edge, either on special-purpose devices installed on the cars or at the periphery of the system. Specific tasks that require a significant amount of resources, such as those related to model training activities, are executed on HPC resources or on the cloud. In this context, SODALITE offers proper mechanisms to control the deployment, operation and reconfiguration of the whole Vehicle IoT system.

The following subsections elaborate more on the motivation behind the use case, describe the main components of the use case, and provide more details on how SODALITE was used to deploy the use case and on the achieved benefits.

7.4.1 Background and Motivation

Through the combination of vehicle telemetry, instrumentation, and behavioural data, insurance companies are able to shape a more holistic view of an individual driver’s overall risk profile based on empirical analysis of driving data (referred to as usage based-insurance, or UBI) - areas that have traditionally relied upon static data points over which the individual has little control, and which have been more focused on risk probability than empirical analysis (these factors include, e.g. age, gender, marital status, make/model of vehicle, etc.). While UBI models have been successfully engaged in markets with a more relaxed and homogeneous regulatory environment, European industry (and citizens) have been hesitant to pursue this model without adequate safeguards for personal data protection and privacy rights - a situation remedied in part by the coming into force of the GDPR (General Data Protection Regulation).

The growth of Connected Car data and concerns over data usage are further compounded by: (1) Individual expectation of contextualised service offerings that respect personal preferences and privacy expectations; (2) Service providers aiming to deploy service offerings across an increasingly dynamic environment; and (3) growing trend of drivers seeking to analyse and benefit from their own driving data directly.

These growing expectations, both from individuals and businesses, lead to an enormous increase in the volume and rate of the sensor data, its aggregation, and its analysis, at various hierarchical levels. This data, in turn, must be processed in line with the relevant privacy constraints and regulatory restrictions it is subject to - aspects subject to dynamic change, while also being highly latency-sensitive.

This leads to two key architectural demands: (1) an increasing amount of in-vehicle data processing and intelligence at the network edge, and (2) increased computational capacity to process large amounts of data in a timely manner - at varying levels of granularity (e.g. device-local, vehicle-local, fleet-wide) - including both fleet-wide big data analytics, as well as periodic online retraining of machine learning models that support the deployment.

Thanks to this case study, it is possible to demonstrate the ability of SODALITE to deploy applications through the whole continuum from the cloud to the edge, to refactor the deployment depending on the system configuration at runtime and to ensure privacy preservation.

7.4.2 The Vehicle IoT Case Architecture

Figure 7.21 presents the structure of the VehicleIoT system. It includes end-user apps running on IoT devices such as smartphones and watches, a special-purpose device installed on cars and a number of components at the edge that enables the connection with specific microservices, including License Plate Recognition, Reverse Geocoding, and Drowsiness Detection. Many of these microservices, in turn, leverage trained machine learning models, and are able to quickly provide results with minimal computational overhead, providing the opportunity to re-deploy and run these services at different hierarchical levels (backend, in-vehicle edge gateway, smartphone, etc.).

Fig. 7.21

Overview of the VehicleIoT architecture

The edge part of this case study is managed through Kubernetes and is assumed to be located partly on each car and partly on the premises of some fleet management company, as show in Fig. 7.22.

Fig. 7.22

The VehicleIoT edge part configuration

The following subsections present in detail the most relevant features offered by the system.

7.4.2.1 License Plate Detection

Within the Vehicle IoT Use Case, individuals may, at various times, submit license plate images for recognition. This can be done for various purposes, including the initial registration of the vehicle with the mobile app (as one possible registration mechanism - of particular interest in countries which provide open access to their vehicle registration databases), evidence to support claims preparation (in the case of a collision), etc. In order to benefit from improved plate recognition, the license plate recognition service relies on a machine learning model that is trained by leveraging appropriate resources (Cloud or HPC). This is envisioned across a number of steps:

• Inclusion of user-generated images in the training data set;

• Plate extraction from uncropped training data (Bulk processing);

• Re-training model on suitable backend resource (e.g. GPU cluster);

• Validating control set against the new model (regression detection);

• Re-deployment / update of plate recognition microservice backed by the new model.

This process is summarised in Fig. 7.23.

Fig. 7.23

The License Plate Recognition Pipeline

7.4.2.2 Drowsiness Detection

Drowsiness Detection aims to determine when a Driver is at risk of falling asleep at the wheel and taking evasive actions (e.g., playing a loud noise, triggering a vibration, etc.) in order to alert the driver to the problem before a more serious incident occurs. Drowsiness detection is typically carried out using a couple of different methods, with differing levels of accuracy and invasiveness. While the gold standard (and most accurate method) for drowsiness detection remains ECG monitoring, ECG measurement itself is invasive and requires active participation by the individual under monitoring, making it a poor fit for passive observation of a driver. The most common non-invasive methods, on the other hand, are PERCLOS (Percentage of eyelid closure) - measuring the proportion of time that the eyelids are between 80-100% closed, and blink detection (Blink detection methods further being split between blink frequency and duration detection). While the PERCLOS method is fairly well established, it has also been found to generate false positives in scenarios that are compatible with the driving activity. For this reason, blink detection is investigated in this case.

It is important to note that drowsiness detection is highly latency-sensitive and must be done in real-time in order to be as accurate as possible and to alert the driver at the time they need to be alerted. Blink duration can be summarized as awake (<400 ms blink duration), drowsy (400–800 ms blink duration) and sleepy (blink duration>800 ms).

With current wireless technologies demonstrating round-trip latencies near 50–200 ms (for 4G) and 500 ms (for 3G) with good connectivity, a backend-deployed monitoring service cannot be expected to reliably identify and respond to drowsiness events in time - necessitating a push-down of the service delivery to the Vehicle itself.

7.4.2.3 Intrusion and Theft Detection (Face Recognition)

Intrusion detection builds on the face detection model developed in the drowsiness detector and defines a face recognition model capable of identifying the authorized driver’s face. In the case where someone other than the designated driver is found to be driving the vehicle, further actions can be taken by the system. These may include notifying the authorized driver and seeking confirmation of a driver switch, notifying a fleet manager, streaming vehicle telemetry to a third party, etc.).

In contrast with drowsiness detection, intrusion and theft detection is not directly latency-sensitive, and as it does not require real-time access to the driver’s video stream, is suitable for backend deployment as a long-lived microservice. While the authorized driver may indeed wish to know if someone is stealing their vehicle as quickly as possible, the added round-trip latency associated with mobile communications is unlikely to have a measurable impact on any asynchronous notifications that may result from the analysis.

A unique characteristic of this service is that custom classifiers must be modelled and trained in order to provide value for the Driver (that is, SVC models capable of identifying the authorized Driver’s face - which the driver may take with them). This may involve dynamic training of vehicle-restricted classification models or may be open for sharing across a fleet of vehicles, or any other vehicle the end-user may use, dependent upon their individual privacy preferences and sharing settings.

Notably, the infrequent nature of the invocation also makes this an ideal candidate for serverless deployments.

7.4.3 How SODALITE is Used

The vehicle IoT use case makes effective use of the SODALITE platform.

• SODALITE IDE and deployment workflow is used to automate the deployment of the application over the cloud and edge infrastructure.

• Kubernetes support is used to model the Edge cluster and to deploy the services on the Edge nodes.

• Platform discovery service is used to discover the changes to the Edge cluster, for example, appearing and disappearing of Edge nodes (CPU and GPU), and to atomically update the knowledgebase to reflect the current state of the Edge testbed.

• Image Builder is employed to create the container images of the service variants optimized for specific Edge accelerator hardware devices.

• The monitoring and refactoring components are employed to collect various metrics from Edge devices, to generate alerts based on the collected metrics, and to perform the necessary changes to the application deployment at runtime.

7.4.3.1 Deployment Architecture

Figure 7.24 shows the deployment of the vehicle IoT case study in the SODALITE cloud and Edge testbeds. Some services (e.g., drowsiness detector, the MySQL storage, and the reverse geocoder) are deployed on edge nodes, while other services (e.g., region router and echo services) are deployed on cloud VMs. Three edge devices are Raspberry Pi 4, Google Coral AI Dev Board, and NVIDIA Jetson Xavier NX. Their accelerators are NCS2 (Neural Compute Stick 2), EdgeTPU, and NVDLA x2.

Fig. 7.24

Vehicle IoT case study deployment architecture

7.4.3.2 Monitoring and Refactoring

The vehicle IoT use case employs the SODALITE runtime layer to implement two deployment adaptation scenarios: location-aware redeployment and alert-driven redeployment. In the first scenario, the application deployment is adapted in response to changes in legal jurisdiction, helping the application to maintain both service continuity and meet its compliance requirements as vehicles travel between countries. The second scenario uses edge-based monitoring and alerting to throttle an application deployment that has exceeded thermal tolerances to mitigate the risks of rising temperature inducing inference failure. The EdgeTPU run-time libraries are provided in -max and -std versions. Using the image builder, the two different variants of the inference application containers, each linked against one version of the run-time library, using an appropriate accelerator-specific base container were created. The deployment refactorer can switch between these two variants as thermal trip points of the devices change.

List 7.3 shows two adaptation rules used by the case study. The first rule reacts to the event LocationChangedEvent by replacing the hosting VM and the second rule reacts to the alert TempCritical by replacing the hosting edge node. The predicates over the TOSCA node properties, for example, location and variant are used to find the correct TOSCA node templates.

7.4.4 Benefits of Using SODALITE

The SODALITE platform improved the vehicle IoT use case in the following aspects:

• The ability of the SODALITE platform to seamlessly orchestrate applications over hybrid heterogeneous infrastructure eased the deployment of the vehicle IoT application. The SODALITE IDE and the guidance provided by the SODALITE Knowledge Base simplified the development of the deployment model for the application.

• The image builder and the application optimization components helped to utilize heterogeneous edge devices by enabling creating device-specific variants of application services.

• The SODALITE runtime improved the responsiveness of the vehicle IoT application by enabling monitoring of the application and the hosting infrastructure resources and codifying and enforcing of deployment adaptation decisions.

7.5 Conclusion

This chapter demonstrated the usage and benefits of SODALITE in the context of the three case studies, each of which required a specific execution and operational environment in Cloud, HPC and Edge platforms.

As such, using SODALITE platform, Snow use case has successfully managed the lifecycle of its execution pipeline in private Cloud, trained its deep learning component in HPC resources using optimized container runtime and utilized runtime optimization of resource allocation. Clinical use case managed to extend its workflow execution from a single HPC infrastructure to multiple HPC and Cloud platforms and utilize optimized container runtimes. Vehicle IoT use case leveraged SODALITE to deploy its application components in Cloud and Edge infrastructures, to build container image variants optimized for specific Edge accelerator hardware devices and to handle dynamics of the Edge infrastructure, such as appearance of new hardware in edge nodes.

The experiments performed by the case studies demonstrate that SODALITE is able to handle complex use cases in various domains and execution platforms and achieves its objectives to enable simpler and faster development of IaC and deployment and execution of heterogeneous apps in HPC, Cloud and other software defined computing environments.