Generative AI

A guide for building your own AI applications

​​Table of Contents

00 / 04 /
Introduction Building a generative AI
03 Intended audience development team
04 Executive Summary

16 AI engineer
17 Data professional
17 Domain SME
01 / 18 Data scientist/machine learning
(ML) professional
Comparing approaches: Generative
AI-enabled software development
vs. standard software development
05/
Azure AI Studio:
A comprehensive platform
02 /
Five common applications 22 Deploy generative AI responsibly
with Azure AI Studio
of building with generative AI
28 Integrated data security and privacy

03 / 06/
Select the right model Key insights and
for your use case next steps
 3

Intended audience
This e-book is specifically geared
towards IT decision makers (ITDMs),
such as Chief Technology Officers (CTOs)
or Chief Information Officers (CIOs)
at companies of all sizes, including
Independent Software Vendors (ISVs),
who are interested in building an AI
application using a foundation model.

The approximate reading

time for this e-book is
30-45 minutes.
 4

Executive Summary
Generative AI is transforming Microsoft provides a variety of courses
organizations and the people who drive designed to help data and development
them. It allows software to be more professionals enhance their AI skills.
These can be found in our
intuitive and helpful, raising employee
AI learning hub.
productivity and user satisfaction. At the
same time, generative AI poses a new set
of challenges and risks.

This e-book delves into the strategic What’s the difference

considerations for those who want to build between custom
their own AI applications using foundation machine learning
and generative AI models, or those that
(ML) models and
want to add generative AI to their existing
applications. It covers details on generative
generative AI models?
AI, offering guidance tailored for ITDMs on
Custom ML models
harnessing this technology to fulfill specific
business needs and achieve competitive Built for a specific purpose

advantages. You’re probably the model builder

Key topics covered in Useful to make predictions on future outcomes

this e-book include: based on pre-existing data (past outcomes)

1. Understanding generative AI:

Differentiating generative AI from Generative AI models
traditional software approaches, Built as “general-purpose”
highlighting the ability to create
dynamic and contextually relevant You’re probably not the model builder
responses beyond predetermined
Useful when you want to create new content
outputs.
or data that resembles/mimics patterns from
2. Building a generative AI team: pre-existing data
Detailing the roles within a generative
AI development team to optimize the
deployment processes.
 5

3. Developing with responsible AI practices: Learn about the tools and best
practices that mitigate risk and support AI safety, quality, and compliance.

4. Utilizing a comprehensive AI development platform: Guidelines for selecting

generative AI platforms and tools that align with specific business objectives.
This includes learning about Azure AI Studio, an essential tool for developing,
deploying, and managing generative AI applications. The platform supports the
entire AI lifecycle with advanced tools for model selection, data integration, and
enterprise-grade production at scale.

This e-book serves as an introductory guide to navigating generative AI and

facilitating informed decisions that drive efficiency and innovation.

Learn the basics of generative AI with

Microsoft Azure AI Fundamentals: Generative AI.
 6

01 | Comparing approaches: Generative

AI-enabled software development vs.
standard software development
Integrating generative AI into applications, when compared to standard
software development, is different in three key ways.

1. Ownership and data control

Standard software development: Microsoft
In standard software development, data Azure Security
control and ownership are clearly defined.
Developers maintain complete control
Regulated industries still require
over their data, which remains within
demonstration to risk managers and
their own data estate. This ownership
regulators that data remains private
provides straightforward management and
throughout the development process. This is
compliance with data regulations, ensuring
why security and compliance are so important
security and privacy without the need for
when choosing a platform for the responsible
third-party involvement.
development and deployment of generative AI
Generative AI development: applications at scale.

Alternatively, generative AI development With Microsoft Azure Security, gain multi-

involves less control over data, as it often layered security across physical datacenters,
requires sending data outside one’s own infrastructure, and operations. Azure cloud
data estate to interact with AI models. is built with customized hardware with
This could potentially raise security and integrated security controls and firmware
regulatory concerns. Although the data components, as well as added protections
still legally belongs to the developer, against threats such as Distributed Denial-of-
the dependency on external AI models Service (DDoS) attacks.
and services managed by third parties
introduces complexity in ensuring data Learn more about Azure Security.

privacy and compliance.

 7

2. Development process defining the scope and specifications

and problem solving of applications, but they also play a
decisive role in continually assessing the
Standard software development: AI-generated outputs for metrics such
Developers write explicit instructions as accuracy and relevance. The iterative
to solve explicit problems, leading to nature of this process, along with the need
predictable and consistent outputs. for frequent evaluations and adjustments,
underscores the importance of the
Generative AI development: human-in-the-loop development process.
In contrast, generative AI operates on
probabilistic principles, using patterns
and contextual data to generate outputs
that may vary with the same inputs. This
uncertainty makes the development
process more complex and less
predictable. It requires extensive testing
and iteration to achieve a production-
ready state. The variability in outputs often
necessitates a significant involvement from
One way to evaluate applications
domain-specific experts who assess the
is for human experts to rate individual
relevance and accuracy of the responses
answers. However, this is time
from AI.
consuming, error prone, and not feasible
at scale or for ongoing monitoring of
3. Evaluation
applications in production.
Standard software development:
To help expedite and remediate
The developer uses unit tests to determine this process, Microsoft researchers
if the application is returning the correct are developing tools for automating
answer and working as intended. assessment by using models to evaluate
the output of other models. Human
Generative AI development: experts remain necessary even in this
Generative AI requires a deeper, automated assessment to provide some
continuous involvement of domain subject ground-truth answers, which can be
matter experts (SMEs) throughout the a significant investment in time.
development process considering the
multiple answer outputs that need to be
evaluated if they are all correct answers.
These experts not only contribute to
 8

02 | Five common
applications
of building with
generative AI
Generative AI-enabled applications
can be developed from scratch,
but often a faster way to gain
generative AI benefits is integrating
it into existing applications. This
can allow users to interact with
applications in new, more intuitive
ways, contextualize assistance,
and provide more relevant
information. Generative AI is, as
a result, improving applications
across the value chain from
customer contact to financial
and operational analysis.
Below are five types of applications organizations are building with generative 9

AI. They combine intuitive interfaces, links to proprietary knowledge bases,

and enterprise systems:

Many organizations possess a wealth of proprietary knowledge in documents

and websites. To unlock this value, they are developing chat applications that
Chat with allow employees and customers to query this data in plain language and
receive direct answers instead of just links to source documents. Generative
your own AI tools should be equipped with built-in responsible AI capabilities so that
data responses are derived solely from internal data, safeguarding against off-brand
experiences and any exposure to malware through untrusted data sources. In
doing so, information remains secure and compliant with company standards.

Software developers use generative AI to integrate advanced helpers into their

Create systems, aimed at reducing the monotony of repetitive tasks and minimizing
disruptions that hinder professional focus. These AI copilots can offer enhanced
personalized in-context support, automatically draft responses tailored to the requester’s
copilot identity and the context of their inquiry and compile comprehensive end-of-day
assistants or weekly summaries detailing activities and statuses. This automation aids
professionals in maintaining productivity without sacrificing accuracy
or attention to detail.

Tailor interactions to individual users at scale. Hyper-personalization

can range from customizing marketing messages and adjusting website
Add hyper-
interfaces in real-time based on user actions, to personalizing product
personalization recommendations at an individual level and enhancing customer
engagement and satisfaction.

Generative AI and speech AI are allowing enterprises to create sophisticated,

multi-lingual brand ambassador avatars where they can engage with the public
in the brand’s look and tone via website, kiosk, or smartphone. While early
Build customer
versions provided general information, avatars are now increasingly able to
service avatars interact with business systems to provide specific information, make informed
recommendations, check inventory, and add items to an e-commerce cart,
making them an exciting new way to create high-value branded experiences.

When important information is scattered across incompatible systems,

analysts can spend hours just assembling the data for routine reports,
such as end-of-quarter performance and prospect profiles. With new agent
Access
architecture, organizations are now using generative AI to pull information
insights from multiple systems, assemble it in standard visualizations, and draft a
narrative. This enables analysts to spend their time building and adding
value since the computer has already done the rote work.
 10

03 | Select the right model for your use case

The number and diversity of generative AI models is growing rapidly, creating
confusion for would-be users. Here are some considerations for selecting the
right model for your application:

Model power
Evaluate the model’s computational Small Language
power and sophistication. This determines Models (SLMs)
its capability to handle complex datasets
and produce nuanced outputs. Higher With SLMs, which are trained on smaller
power might be required for intricate amounts of data with fewer parameters, you
tasks, impacting operational demands get the benefit of a model that can work with
and associated costs. limited computing resources, while allowing
for increased specificity. SLMs can be run
locally—an advantage for regulated industries
Time efficiency/latency
and in cases where lag time is extremely
The model’s speed is vital, especially important—and can be fine-tuned to specific
for tasks that need fast data processing tasks and work with narrow context. This can
or instant decision-making. Efficiency be an excellent choice for specific industries

not only affects performance but looking to complete a specific task. For
instance, a financial services organization can
also influences user satisfaction and
benefit from SLMs when processing claims but
operational efficiency.
wants to determine the precise training to use.

Cost effectiveness Learn more about Microsoft’s recent

advancements with SLMs.
Analyze all associated costs, including
initial setup, ongoing operations, and
maintenance. It’s essential to strike a
balance between the model’s capabilities
and your budget.
 11

Fine-tuneability/extensibility
Take into consideration the ability to tweak the model to your specific data, as this
can be important for the extensibility and modifications necessary for your model
and the intended purposes. However, if your use case requires playing with the
weights, then choosing an open model is more appropriate. Keep in mind that
not all open models will have permissive license to build commercial applications.

Licensing and availability of LLMs

To use LLMs for commercial purposes, consider the license of a particular model.
Remember that availability is not always straightforward, considering that some
models are closed source. Another availability consideration is that models are
dependent on the data center and not all models are available in all data centers,
creating challenges around restrictions with data crossing certain boundaries.

Generality vs. specificity

Choose between general-purpose and specialized models based on the breadth or
specificity of your needs. General models offer flexibility, while specialized models
provide high efficiency in specific contexts. For example, some models specialize in
specific tasks, such as chat completion or summarization, or are purpose-built for
specific data types such as code, images, video, or text, or a specific industry
such as healthcare.
Microsoft evaluation and monitoring
12

metrics for generative AI

Automated metrics for

output evaluation
To enable evaluation at scale,
Microsoft is developing tools to
have LLMs evaluate the output
of generative AI applications.

Groundedness
Azure AI Content Safety-based Prompt-only-based
groundedness groundedness
The model’s generated answers are Measures how well the model's generated
evaluated based on their alignment with answers align with information from the
information from the source data, such as source data (user-defined context).
retrieved documents in RAG, question
and answering, or documents used for
summarization. The evaluation process
flags output that lacks grounding.

Relevance Coherence
The extent to which the model's How well the language model can
generated responses are pertinent produce output that flows smoothly,
and directly related to the given reads naturally, and resembles
questions. human-like language.

Fluency GPT-similarity
How well written and easy to How close the answer is to a
understand the answer is. user-provided ground truth and only
applies when you’ve supplied ground
truth and are using a generative AI
model to compare them.
 13

By incorporating these considerations,

organizations can more effectively
align generative AI capabilities with
strategic goals. The nuanced approach
required for selecting and managing
generative AI highlights the importance
of expert involvement and iterative
testing, ensuring the technology not
only performs well but also integrates
seamlessly into organizational processes
to meet business needs. Further details
on the roles of domain SMEs and other
team members will be explored later in
this e-book, providing deeper insights
into the collaborative and dynamic
nature of generative AI development.
 14

04 | Building a generative AI
development team
Developing with generative AI
requires a hybrid approach— Preparing your data
standard software development
blended with AI expertise. Roles
for generative AI
like the AI engineer have emerged
development
to provide this connection between
Organizations can effectively prepare for
software development and AI. generative AI projects by making sure their
data is ready. Generative AI runs on data, so
At the same time, there is a partnership
it’s essential to ensure its quality so the output
between technical and business teams,
is more accurate.
with business needs driving development.
Non-technical business leaders and Prepare your relevant data

other stakeholders play focal roles in Your organization needs robust data
determining the utility and trustworthiness infrastructure to handle large volumes of
of AI applications. Their decisions are vital data and high-performance computing
in deciding whether to continue using and tasks. Data must be accessible, well-

investing in the AI system. organized, and secure to facilitate

effective AI training and operations.

Ensure data quality

Clean and prepare your data to ensure that

it is trusted and suitable for using prebuilt
foundation models. In addition to data
cleaning, consider if data is normalized and
if biases are mitigated. Quality is essential, as
it affects how precise and dependable the AI
models’ results are.
 15

Building a generative AI development team

From the visual below, we can see that the AI engineer leads the development team,
while time investment may be greater for the domain SMEs.

Overall project Picks the tech Develops Iterates the Hardens for
lead stack the app prompt(s) deployment

AI Engineer

a gene
ng
Iterates processing,
Builds data i r
pipelines d chunking, and
at

indexing
l

Data
Bui

ive

Data Pro Science

Validates Pro Leads evaluation
AI d

data inputs (in conjunction with

ev
ea

tt
and outputs the domain SME)
elo
p men

Domain SME

Partners to identify Gathers Evaluates iterative Leads UAT,

business goal documents versions of the output roll out
 16

AI engineer
AI engineers are typically central,
orchestrating the development initiative. AI engineer learning
Often, they are software developers
resources
who have been informally upskilled to
fill this role on the team. Consequently, Looking for specific AI engineer educational
resources? Get started with:
the AI engineer assumes the dual role of
a leader and an integrator, bridging the
• AI engineer certification path
gap between AI capabilities and business
• Azure AI Fundamentals
needs without the necessity to delve
deeply into data science, which is typically • Introduction to Azure AI Studio

handled by data science professionals.

The AI engineer also takes information
from the domain SME and adjusts the
application based on their testing.
 17

Data professional
A critical component of building with generative AI is pulling data from business
systems and getting it to the model. The data professional is the one who builds
the pipelines that make this happen safely and efficiently. Therefore, expertise in the
nuances of data sources can be critical to create quality generative AI applications.
The data professional must also navigate the complexities of data privacy regulations
to ensure all data collection and usage comply with legal standards.

Domain SME
The domain subject matter expert (SME) collaborates closely with the AI engineer to
ensure precise data collection. They also continually monitor key performance metrics
set by the AI engineer. In generative AI development, domain SMEs tend to play a
bigger role in than in standard software or machine learning because the system
outputs are not easily validated.  

Domain SMEs take on the role of evaluating the correctness of responses, which
can be time-consuming. Their role is pivotal in bridging the technical and business
aspects of AI implementations. To streamline their input, domain SMEs can work with
the rest of the team to create test prompts and ground truth answers. AI engineers
can then use Microsoft’s automated evaluation metrics to assess the application’s
response quality over time and compare performance across different models.
 18

Data scientist/
machine learning (ML)
professional
Not every generative AI team has a data
scientist, but they are often present on
Looking for specific
complex and business-critical projects.
The data scientist plays an advisory role,
AI data science
responsible for developing formalized
educational resources?
test plans and setting evaluation criteria, Get started with:
which alleviates some of the domain • Microsoft Certified:
SME’s workload. Azure Data Scientist Associate

A common approach is Retrieval

Augmented Generation (RAG), which
combines the reasoning capabilities
of models with custom data to generate
responses. This involves testing various
document chunking and annotation
techniques, assessing retrieval quality,
and evaluating responses across
different prompts.

As the need for generative AI within

organizations grows, so does the
demand for a centralized platform to
develop and deploy these applications
responsibly. This underscores the
increasing significance of the data
science professional’s role in ensuring
these technologies are implemented
effectively and ethically.
 19

05 | Azure AI Studio:
A comprehensive
platform
McDonald’s
China transforms
Organizations need tools that simplify AI
development, allowing them more time
its operations,
to focus on big-picture business needs.
elevates service
Azure AI Studio, Microsoft’s generative AI levels with Azure AI
platform, is designed to democratize the
What was their goal?
AI development process for developers,
bringing together the models, tools, McDonald’s China needed to uplevel
services, and integrations necessary to their customer service, quality, and
begin quickly and efficiently developing operations to accommodate the growing
number of locations and the rapid pace
your own AI applications.
of innovation. They needed a way to
• 87% of organizations believe adhere to their brand mission through
AI will give them a competitive edge1 increased digital transformation.

• 66% of surveyed board members

How did they achieve it?
report pressure to accelerate
AI adoption2 With the support of Microsoft, McDonald’s
China established the AI Lab, allowing them
to integrate AI into their existing operations,
predominately by leveraging large language
models (LLMs). Through the AI Lab, AI has
proliferated their entire operations, from
supply chain operations to even supporting
marketing campaigns.

¹ Expanding AI’s Impact With Organizational Learning

2
CEO decision-making in the age of AI
 20

As a comprehensive platform for

developing and deploying generative
AI applications, Azure AI Studio emerges
Siemens and
as a game-changing tool for both
Microsoft partner
seasoned developers and those new to drive cross-industry
to AI. With drag-and-drop functionality, AI adoption
visual programming environments, and
What was their goal?
prebuilt templates, Azure AI Studio makes
it easier for users to prototype, build, Multinational technology conglomerate
and refine AI applications without deep Siemens needed to simplify virtual
technical knowledge of the underlying collaboration of design engineers, frontline

algorithms. This accessibility accelerates workers, and other teams across key business
functions. They were looking to utilize
the development process and helps
generative AI to rapidly generate, optimize,
users quickly translate their creative
and debug complex automation code.
and business ideas into fully operational
AI solutions.
How did they achieve it?

With the introduction of Siemens Industrial

Copilot, a generative AI-powered assistant,
Siemens enhanced human-machine
collaboration and boosted productivity.
They also significantly shortened simulation
times, reducing task time from weeks to
minutes. Maintenance staff could now use
the power of natural language and gain
assistance with detailed repair instructions
and provide engineers with quick access to
simulation tools.
 21

Key features of Azure AI Studio that provide substantial value to users include:

Responsible AI tools and best practices:

Azure AI Studio empowers developers to safely, securely, and responsibly innovate and shape the
future with AI. The comprehensive platform accelerates the development of production-ready
copilots to support enterprise chat, content generation, data analysis, and more. Developers use
their protected data to build custom models and solutions with collaborative, responsible AI tools
and best practices.

API and model choice:

Users can access and deploy the latest models as APIs, facilitating rapid, serverless, and fine-
tuned model deployment. This reduces development time and resource costs, accelerating time
to market for developers and providing end-users with state-of-the-art AI capabilities quickly. The
Assistants API makes it easier for developers to create automated applications with sophisticated,
agent-like experiences that sift through data, suggest solutions, and automate tasks using
advanced tools like code.

Complete AI toolchain:
Azure AI Studio offers tools to ground models on specific data, orchestrate complex AI workflows,
and evaluate model outputs for quality and safety, ensuring robust end-to-end management.
Developers find it easier to integrate and manage AI projects, enhancing productivity and
operational efficiency, while users experience more reliable and effective AI applications.

Enterprise-grade production:
The platform facilitates scalable deployment of models, flows, and apps, incorporating continuous
monitoring and fine-tuning capabilities within a secure and governed environment. Organizations
can scale their AI solutions as needed without compromising on security or performance,
providing developers with a flexible and robust infrastructure and ensuring that users enjoy
consistent, reliable AI services.

The centralized nature of the Azure AI Studio platform provides a collaborative development
environment so generative AI teams can efficiently work together and stay in sync. Azure
AI Studio helps users build AI solutions faster with prebuilt capabilities and templates to
ultimately accelerate solution development.
 22

Deploy responsible AI
with Azure AI Studio
Responsible generative AI refers to
the development and deployment of
generative AI systems in a manner that
is safe, transparent, and accountable. ASOS uses Azure
Microsoft offers numerous tools and AI Studio to surprise
controls that help with the responsible and delight young
deployment of generative AI. fashion lovers
Azure AI services: Developers can
What was their goal?
utilize pre-built and customizable APIs
and models to rapidly build cutting-edge, With the prevalence of generative AI
platforms, ASOS, a United Kingdom-based
responsible applications. These services
fashion and cosmetic retailer, saw an
provide detailed Transparency Notes
opportunity to expand its business model,
and fairness assessments, such as Face
enrich its technology infrastructure, and
and Speech, to support customer choice meet customers’ modern tech expectations.
and transparency.
Responsible use of AI with How did they achieve it?
Azure AI services
ASOS used Azure OpenAI Service and prompt
Azure AI Content Safety: Developers can flow, part of Azure AI Studio, to quickly
get support in detecting and mitigating streamline their development and testing
risky content, including prompt attacks, cycles, helping the customer and the solution

and the generation of ungrounded or interact effectively. ASOS helped developers

onboard quickly with built-in prompt flow
copyright material. By using Azure AI
resources, limiting the need for custom code.
Content Safety as a built-in safety system,
develops can build more trustworthy
applications.
Build AI applications responsibly with
Azure AI Content Safety
 23

Evaluation and Monitoring in Azure AI Studio: In addition to mitigating problematic inputs

and outputs, developers can continuously measure the effectiveness of their mitigations during
development and in production environments using pre-built evaluation and monitoring metrics
for risks and safety. As data and user behavior change over time, these measurement tools help
developers and domain SMEs understand their application’s behavior and intervene quickly when
performance degrades, or an end user is attempting to manipulate the application to behave
outside its prescribed purpose.
Evaluation and monitoring metrics for generative AI

Security: Microsoft provides the multilayered security backbone of your applications, using built-
in security controls and unique threat intelligence to help identify and secure against evolving
threats.

• A focal point of Microsoft security is Microsoft Defender for Cloud—Microsoft’s cloud-

native application protection platform (CNAPP)—which provides comprehensive security to
proactively mitigate risks and identify threats to AI applications from code to cloud.
Learn more about Microsoft Defender for Cloud
 24

Mitigation layers Using a layered mitigation plan can

help developers identify and remediate
in Azure AI Studio potential risks earlier in the development
process. Here is a brief overview of Azure
AI Studio’s robust safety mechanisms,
designed to secure and optimize your
generative AI deployments:
System Message
• System message layer: This layer
provides hidden instructions to your
model with every user prompt, so
Safety System you can guide the model’s behavior
and data retrieval to generate higher
quality responses by default.

Model • Safety system layer: This added layer

goes beyond the basic safety fine-
tuning that is part of the model. Azure
AI Content Safety provides this extra
layer by running both the prompt and
completion of your model through
classification models. These models
are designed to detect and prevent
the output of harmful content across a
range of categories and severity levels.

• Model layer: With Azure AI Studio’s

model catalog, you can explore
benchmarks and model cards for
Azure OpenAI Service, Meta, Hugging
Face, and other model developers,
all organized by collection and task.
 25

Safety feature highlights in Azure AI Studio

Azure AI Content Filters

Prompt shields:
Detect and block prompt injection attacks, including
direct and indirect prompt attacks.

Groundedness detection:
Detect “hallucinations” in model outputs, for better
accuracy and reliability of the content generated by AI.

Configurable harmful content filters:

Detect four categories of harmful content
(violence, hate, sexual, and self-harm) at four severity
levels respectively (safe, low, medium, and high), and
optional binary classifiers for detecting jailbreak risk,
existing text, and code in public repositories.
 26

Safety system message:

Steer your model’s behavior toward safe, responsible outputs using system message
templates developed by Microsoft Research.

Safety evaluations:
Assess an application’s vulnerability to jailbreak attacks and content risks using your
own test dataset or a test dataset generated with AI assistance.

Risk and Safety Monitoring:

Understand what model inputs, outputs, and end-users are triggering content filters
to inform mitigations.
• System message and grounding layer:
This is where retrieval augmented generation (RAG) comes in. Instead of using the model as
a source of information, the model serves as a reasoning engine over data sources that are
relevant to the query. The system message helps guide the model to use grounding data
effectively and can help steer overall behavior for more predictable, responsible
model outputs.

• User experience layer: At this level, there is a litany of user-centered interventions,

guidance, and best practices that can be provided to users to ensure the system
is used as intended.

These enhancements to Azure AI Studio’s safety features demonstrate Microsoft’s

commitment to responsible AI, ensuring that your applications are not only effective
but also align with the highest standards of data integrity and security.
Read in more detail: Harms mitigation strategies with Azure AI
 27

Another key component of responsible

AI is the evaluation and monitoring of
your application. We’ve discussed the
Trailblazing AI
generation of quality metrics that Azure
answer engine
AI Studio provides, but there are three Perplexity.
overarching topics for consideration: AI doubles
throughput,
Manual evaluations:
This is the form of evaluation we
cuts cost with
recommend starting with before moving
Azure AI Studio
to automated evaluations. This can be a
What was their goal?
useful way of tracking progress on a small
set of priority issues. Startup Perplexity.AI is the creator of
Perplexity Ask, a revolutionary AI-based
Automated evaluations: conversational answer engine that combines
On the other hand, automated evaluations large language models with a robust semantic
can be more useful for measuring quality search engine. To better support Perplexity
and safety on a larger scale and can gain Ask, they needed a platform that would

more comprehensive results. support faster time to market, serve as a

force multiplier for their lean staff, scale to
Monitoring: support millions of users, and deliver security
Monitoring models that are deployed and reliability at a cost-effective price.
in production is an essential part of
the generative AI application lifecycle. How did they achieve it?
Changes in data and consumer behavior
Perplexity.AI used Azure AI Studio to
can change your applications performance
develop their first prototype in just hours.
over time. Azure AI Studio makes it easy
They were able to try out large language
for you to monitor your applications in models available with Azure OpenAI Service,
production for ongoing safety and quality. getting going “with just a few clicks.” Overall,
this resulted in running twice as many
Learn more: Evaluation of generative
experiments, in parallel, before they adopted
AI applications Azure, enabling them to retrain the new
version of the model twice as fast.
 28

Integrated data security

and privacy
Microsoft is obligated to defend customers
against certain third-party intellectual
property claims relating to Output
To help ensure data security and privacy,
Content. For Azure OpenAI Service and any
Microsoft makes several commitments Configurable Generative AI Service, Customer
to its customers. For one, Microsoft also must implement all mitigations required
Azure is built on security, privacy, and by the Azure OpenAI Service documentation
compliance. When data is stored in Azure, in the offering that delivered the Output
your data remains yours – it is never used Content that is the subject of the claim.

for marketing, advertising, or foundation Learn more: Introducing the Microsoft

model training purposes without your Copilot Copyright Commitment
permission. The prompts and the
completions back are still your data.

Microsoft Fabric, Microsoft’s unified AI-

powered analytics platform, can help
reshape and improve how you access,
manage, and act on data. On a single
platform, Fabric unites data and services
to simplify data integration into a single,
multicloud data lake for your organization
to work from the same data across
analytics engines and languages.
 29

In addition, Azure AI Studio contains

enterprise security configurations,
including:
• Azure Policy integrations

• Role-based access control (RBAC)

• Network isolation and security

• Data protection and encryption

• Vulnerability management

To tie it all together, when managing

data security and privacy, Fabric has a
native integration with Microsoft Purview,
which enables organizations to govern,
protect, and manage their data across
their entire data estate. When integrated
together, Fabric and Purview enhances
AI capabilities through secure data
integration.
 30

06 | Key insights
As the demand for generative AI within
organizations grows, the need for a
centralized platform to develop and
and next steps deploy these technologies responsibly
and effectively becomes critical.

Azure AI Studio provides a robust solution,

offering a comprehensive platform that
caters to developers at all levels, including
those without data science expertise.
This platform not only simplifies the
development process by providing access
to a comprehensive model catalog from
leading providers but also supports the
full lifecycle of AI development with tools
for advanced data integration, workflow
orchestration, and interactive app
development.

Get started with Get started with

Azure AI: Azure AI Studio:

• Microsoft Certified: Azure • ai.azure.com

AI Engineer Associate
• Azure AI Studio Overview
• Microsoft Certified: Azure
• Introduction to Azure AI Studio
AI Fundamentals
- Training | Microsoft Learn
• GitHub repo for building
a RAG application

• Template Library on GitHub

©2024 Microsoft Corporation. All rights reserved. This document is provided “as-
is.” Information and views expressed in this document, including URL and other
Internet website references, may change without notice. You bear the risk of using
it. This document does not provide you with any legal rights to any intellectual
property in any Microsoft product. You may copy and use this document for your
internal, reference purposes.