Microsoft has unveiled a sneak peek into Windows AI Studio, an early preview of its AI development environment tailored for desktops. This platform allows developers to construct compact language models compatible with PCs and mobile devices.
During the developer-centric segments of Ignite 2023, Microsoft disclosed a suite of AI development tools. While Azure AI Studio caters to extensive AI development for cloud-based applications using Azure OpenAI models, Copilot Studio enhances the capabilities of the Power Virtual Agents low-code AI tools with OpenAI-powered enhancements.
Adding to this lineup is Windows AI Studio, a tool that has recently made its debut on developers’ PCs. Let’s delve into the details.
Introduction to Windows AI Studio Windows AI Studio aims to bring a repository of AI models from Microsoft and its partners to personal computers, initially leveraging GPUs and eventually extending to on-board AI accelerators like Arm and Intel NPUs found in Microsoft’s latest Surface hardware. The integration of DirectML support for Intel NPUs in devices like the Surface Laptop Studio 2, slated for early 2024, makes this prospect even more appealing for developers and users alike.
The purpose of Windows AI Studio is to facilitate model training and customization, preparing them for integration into code. Once trained, models can be converted to run using the ONNX (Open Neural Network Exchange) cross-platform runtime, suitable for deployment in desktop and mobile applications. Delivered as an extension for Visual Studio Code, Windows AI Studio consolidates various tools and AI models into a unified environment, allowing developers to refine and build models seamlessly into .NET applications.
Windows AI Studio operates as a hybrid solution, employing both Windows and Linux tools across CPU and GPU, utilizing the Windows Subsystem for Linux (WSL) for hosting and running models. Notably, this approach requires robust hardware with ample memory and a recent GPU. Without a discrete GPU, be it a workstation-grade card or an external GPU via Thunderbolt, Windows AI Studio cannot function.
Installation and Prerequisites for Windows AI Studio Installing Windows AI Studio is a straightforward process. It can be downloaded from the Visual Studio Marketplace, with accompanying quick start instructions. Developers should be aware that the default view in Visual Studio Code may need to be switched to pre-release versions to access Windows AI Studio. After making this adjustment, the installation is quick and hassle-free.
Certain prerequisites must be met, such as having an Nvidia GPU and WSL running at least the Ubuntu 18.4 release as its default Linux. Additionally, Windows AI Studio checks for Conda and CUDA support in the WSL environment during installation, with a one-click option available to ensure all necessary libraries are in place.
Your First Foray into Windows AI Studio Once installed, Windows AI Studio integrates a chip-like icon into the Visual Studio Code extensions sidebar. Clicking on this icon launches the Windows AI Studio development environment, which conducts checks to verify that the development environment meets prerequisites. After passing these checks and updating the WSL configuration, the extension presents a “What’s New” page and populates its actions pane with current features. Although more features are planned, only the model fine-tuning action is currently functional in the latest preview release.
Future features include Retrieval Augmented Generation (RAG), a platform for Microsoft’s Phi-2 foundation models, and access to a library of ready-to-use models from services like Hugging Face. Incorporating RAG support will enable developers to build and train custom language models without relying on cloud-hosted services. This feature allows the utilization of external data sources, enhancing the model’s context and accuracy.
Quantizing Models with QLoRA The current preview release primarily focuses on fine-tuning existing AI models for conversion to ONNX, suitable for embedding in WinML projects. This feature is crucial for custom machine learning products where local hardware deployment is preferred over cloud-based solutions.
To set up a model tuning environment, users select a local folder and a model from a limited initial selection. Microsoft utilizes the QLoRA tuning methodology, Quantized Low Rank Adapters, which has demonstrated impressive results. QLoRA is an approach developed at the University of Washington that achieves 99.3% of ChatGPT’s performance with just 24 hours of tuning on a single GPU.
Choosing a model prompts the configuration of the project in both Windows and WSL. An access token for Hugging Face may be required, and Windows AI Studio presents tuning parameters to refine the model’s performance. Initial tests can be conducted with default settings, and users have the option to incorporate additional datasets for improved tuning.
Fine-Tuning with Olive Upon generating the model, users are guided to relaunch the Visual Studio Code window in a Windows AI Studio workspace. This transition from Windows to WSL enables the use of tools installed during setup. As part of the workspace initialization, Windows AI Studio installs a Prompt Flow extension.
With the model workspace open, users can employ the Visual Studio Code terminal to activate the Conda environment for model tuning. Olive, utilizing QLoRA on either default content or a custom dataset, can be run, albeit with a considerable time investment. Tuning, even on high-end graphics cards, may take several hours.
After completing the tuning process, a Gradio web interface facilitates testing the trained model before packaging and integration into applications. While Windows AI Studio is in its early stages and serves as a complex tool, Microsoft’s efforts to simplify AI model work and tuning tools are evident. However, a solid understanding of language models and tuning variables is crucial for effective utilization.
In its current form, Windows AI Studio may be more suited for AI experts. Nonetheless, its potential is significant, and with ongoing evolution and feature additions, it could become an integral part of the Windows development workflow, particularly if AI accelerators become commonplace in the next generation of PCs.
Exploring Windows AI Studio: A Novel Approach to AI Development
Microsoft has unveiled a sneak peek into Windows AI Studio, an early preview of its AI development environment tailored for desktops. This platform allows developers to construct compact language models compatible with PCs and mobile devices.
During the developer-centric segments of Ignite 2023, Microsoft disclosed a suite of AI development tools. While Azure AI Studio caters to extensive AI development for cloud-based applications using Azure OpenAI models, Copilot Studio enhances the capabilities of the Power Virtual Agents low-code AI tools with OpenAI-powered enhancements.
Adding to this lineup is Windows AI Studio, a tool that has recently made its debut on developers’ PCs. Let’s delve into the details.
Introduction to Windows AI Studio Windows AI Studio aims to bring a repository of AI models from Microsoft and its partners to personal computers, initially leveraging GPUs and eventually extending to on-board AI accelerators like Arm and Intel NPUs found in Microsoft’s latest Surface hardware. The integration of DirectML support for Intel NPUs in devices like the Surface Laptop Studio 2, slated for early 2024, makes this prospect even more appealing for developers and users alike.
The purpose of Windows AI Studio is to facilitate model training and customization, preparing them for integration into code. Once trained, models can be converted to run using the ONNX (Open Neural Network Exchange) cross-platform runtime, suitable for deployment in desktop and mobile applications. Delivered as an extension for Visual Studio Code, Windows AI Studio consolidates various tools and AI models into a unified environment, allowing developers to refine and build models seamlessly into .NET applications.
Windows AI Studio operates as a hybrid solution, employing both Windows and Linux tools across CPU and GPU, utilizing the Windows Subsystem for Linux (WSL) for hosting and running models. Notably, this approach requires robust hardware with ample memory and a recent GPU. Without a discrete GPU, be it a workstation-grade card or an external GPU via Thunderbolt, Windows AI Studio cannot function.
Installation and Prerequisites for Windows AI Studio Installing Windows AI Studio is a straightforward process. It can be downloaded from the Visual Studio Marketplace, with accompanying quick start instructions. Developers should be aware that the default view in Visual Studio Code may need to be switched to pre-release versions to access Windows AI Studio. After making this adjustment, the installation is quick and hassle-free.
Certain prerequisites must be met, such as having an Nvidia GPU and WSL running at least the Ubuntu 18.4 release as its default Linux. Additionally, Windows AI Studio checks for Conda and CUDA support in the WSL environment during installation, with a one-click option available to ensure all necessary libraries are in place.
Your First Foray into Windows AI Studio Once installed, Windows AI Studio integrates a chip-like icon into the Visual Studio Code extensions sidebar. Clicking on this icon launches the Windows AI Studio development environment, which conducts checks to verify that the development environment meets prerequisites. After passing these checks and updating the WSL configuration, the extension presents a “What’s New” page and populates its actions pane with current features. Although more features are planned, only the model fine-tuning action is currently functional in the latest preview release.
Future features include Retrieval Augmented Generation (RAG), a platform for Microsoft’s Phi-2 foundation models, and access to a library of ready-to-use models from services like Hugging Face. Incorporating RAG support will enable developers to build and train custom language models without relying on cloud-hosted services. This feature allows the utilization of external data sources, enhancing the model’s context and accuracy.
Quantizing Models with QLoRA The current preview release primarily focuses on fine-tuning existing AI models for conversion to ONNX, suitable for embedding in WinML projects. This feature is crucial for custom machine learning products where local hardware deployment is preferred over cloud-based solutions.
To set up a model tuning environment, users select a local folder and a model from a limited initial selection. Microsoft utilizes the QLoRA tuning methodology, Quantized Low Rank Adapters, which has demonstrated impressive results. QLoRA is an approach developed at the University of Washington that achieves 99.3% of ChatGPT’s performance with just 24 hours of tuning on a single GPU.
Choosing a model prompts the configuration of the project in both Windows and WSL. An access token for Hugging Face may be required, and Windows AI Studio presents tuning parameters to refine the model’s performance. Initial tests can be conducted with default settings, and users have the option to incorporate additional datasets for improved tuning.
Fine-Tuning with Olive Upon generating the model, users are guided to relaunch the Visual Studio Code window in a Windows AI Studio workspace. This transition from Windows to WSL enables the use of tools installed during setup. As part of the workspace initialization, Windows AI Studio installs a Prompt Flow extension.
With the model workspace open, users can employ the Visual Studio Code terminal to activate the Conda environment for model tuning. Olive, utilizing QLoRA on either default content or a custom dataset, can be run, albeit with a considerable time investment. Tuning, even on high-end graphics cards, may take several hours.
After completing the tuning process, a Gradio web interface facilitates testing the trained model before packaging and integration into applications. While Windows AI Studio is in its early stages and serves as a complex tool, Microsoft’s efforts to simplify AI model work and tuning tools are evident. However, a solid understanding of language models and tuning variables is crucial for effective utilization.
In its current form, Windows AI Studio may be more suited for AI experts. Nonetheless, its potential is significant, and with ongoing evolution and feature additions, it could become an integral part of the Windows development workflow, particularly if AI accelerators become commonplace in the next generation of PCs.
Archives
Categories
Archives
Rust 1.84 Unveils Enhanced Strict Provenance APIs
January 16, 2025Google Introduces Jules: A New Contender in the AI Coding Assistant Space
December 29, 2024Categories
Meta