Microsoft launches Azure confidential VMs with NVIDIA Tensor Core GPUs

Microsoft Launches Azure Confidential VMs with NVIDIA Tensor Core GPUs

Microsoft has officially rolled out its Azure confidential virtual machines (VMs), known as the NCC H100 v5 SKU, which are equipped with NVIDIA Tensor Core GPUs. This announcement marks a significant milestone in cloud computing, blending hardware-based data protection courtesy of 4th-generation AMD EPYC processors with high-performance computing resources.

The recent general availability (GA) of these VMs follows a preview phase that began last year. By incorporating confidential computing capabilities on GPUs, Azure enhances the options available to its customers, allowing them to run workloads securely and efficiently within the cloud. These newly launched VMs are particularly well-suited for tasks like inferencing, fine-tuning, and training models, catering to both small and medium-sized computational needs. Examples of these models include Whisper, Stable Diffusion and its variants (such as SDXL and SSD), as well as language models like Zephyr, Falcon, GPT-2, MPT, Llama2, Wizard, and Xwin.

The NCC H100 v5 VM SKUs feature a Trusted Execution Environment (TEE) based on hardware, which bolsters the security of guest virtual machines. This environment is designed to shield VM memory and state from potential access by the hypervisor and other host management code, thereby protecting against unauthorized operator interactions. Customers using these VMs have the option to initiate attestation requests, which verify that the virtual machines are operating in a properly configured TEE. This step is crucial before deploying keys and launching sensitive applications.

Commentary from industry professionals highlights ongoing discussions about the robustness of TEE attestation. In a LinkedIn post, Vikas Bhatia, head of product for Azure confidential computing, engaged with a comment from Drasko Draskovic, founder and CEO of Abstract Machines, who pointed out that attestation remains a contentious aspect of TEEs in cloud service provider VMs. Draskovic noted that current attestation mechanisms require a level of trust with the cloud provider that may seem counterintuitive to the aims of confidential computing. He suggested that, for now, a bare-metal approach might be more viable, although this could negate the need for TEEs by nature.

Various companies have already embraced the Azure NCC H100 v5 GPU virtual machine for diverse applications, including confidential audio-to-text inference leveraging Whisper models, video analysis aimed at incident prevention, data privacy through confidential computing, and projects involving stable diffusion with sensitive design data in the automotive industry.

Microsoft’s advancement in this domain is mirrored by its competitors, with AWS and Google among the major hyperscalers also offering NVIDIA H100 Tensor Core GPUs. AWS, for instance, provides access to H100 GPUs through its EC2 P5 instances, which are optimized for high-performance computing and artificial intelligence applications.

In a detailed whitepaper, NVIDIA elaborated on its H100 Tensor Core GPU, which is based on the Hopper architecture. This 9th-generation data centre GPU is engineered to deliver significant performance improvements for large-scale AI and high-performance computing (HPC) workloads, marking an evolutionary step from its predecessor, the NVIDIA A100 Tensor Core GPU.

At present, the Azure NCC H100 v5 virtual machines are only made available in select regions, specifically East US2 and West Europe, offering insights into Microsoft’s strategic deployment approach.

Source: Noah Wire Services