Lecturer: Ray Li

Unlocking the Power of Large Language

Models: From ChatGPT to Real-World
 Topic
• LLM
s • Prompt Engineering
• Transformers • AI Detector
• Attention Mechanism • Python Interpretor
• LLM Applications • OpenCV Lecturer: Ray Li
 What is LLM?
• Advanced AI systems designed to understand and generate human-like text
• Trained on vast amounts of text data from books, articles, websites, etc
• They have millions or even billions of parameters (the parts of the model that get adjusted during
training).
• Examples include GPT-3, GPT-4, and BERT
 Transformers
• A type of neural network architecture
• Designed for handling sequential data
efficiently
• They can consider the entire context of a
sentence at once rather than word by word
• Relies on a mechanism called “self-attention”
to understand the context of words in a sentence

Add a little bit of https://www.youtube.com/watch?v=n9TlOhRjYocbody text

 Attention Mechanism
Imagine you are reading a sentence, and to understand a particular word, you might need to look
back at the previous words. Transformers do this efficiently by weighing the importance of each
word in the context of the entire sentence.

https://medium.com/@vimal.parakhiya/decoding-large-language-models-a-deep-dive-into-the-self-attention-mechanism-967d6207d4ae
 There are 6 key parts:

1. Input Representation 4. Applying Softmax Function

2. Query, Key, and Value 5. Weighted Sum of Value vectors

Vectors
6. Multi-Head Attention
3. Calculating Attention
Scores
 1. Input Representation / Embedding
• Each word in a sentence is converted into a vector (a list of numbers) representing its meaning
 2. Query, Key, and Value Vectors
• For each word, three different vectors are created:
• Query (Q), Key (K), Value (V)
• Obtained by multiplying the input vector by three different weight matrices
 3. Calculating Attention Scores
• For each word, calculate how much attention it should pay to every other word in the sentence
• This is done by taking the dot product of the Query vector of one word with the Key vectors of all
words
• The result is a set of scores indicating the importance of each word
 4. Applying Softmax Function
• Apply the softmax function to the attention scores to get attention weights
• These weights are probabilities that sum up to 1, indicating how much attention to give to each word
 5. Weighted Sum of Value Vectors
• Multiply each Value vector by the corresponding attention weight
• Sum these weighted Value vectors to get the final attention output for each word
• Note that weighted Value vectors play an important role in a neutral network
 6. Multi-Head Attention
• Multiple sets of Query, Key, and
Value vectors (heads) are used to
capture different aspects of the
relationships between words

• The outputs of these heads are

concatenated and transformed to
produce the final result
 Applications
1. Text Generation 2. Translation 3.Summarisation
4. Sentimental Analysis 5. Chatbots

https://www.techopedia.com/12-practical-large-language-model-llm-applications
 1.Text Generation
• Creating articles, stories, or any written content.
 2.Translation
• Translating text from one language to another
 3.Summarisation
• Condensing long articles or papers into shorter summaries
• Generating a summary based on multiple data sources
 4.Sentimental Analysis
• Analysing the sentiment of text, like determining if a review is positive or negative
 5.Chatbots
• Building intelligent chatbots that can hold conversations with humans
 Monica
• Go to Monica and Choose a Chatbot

https://monica.im/zh_TW
 Prompt Engineering
• To develop and optimise prompts to efficiently use language models (LMs) for a wide
variety of applications and research topics

https://promptingguide.azurewebsites.net/
 1.Text Summarization
• Summarize articles and concepts into quick and easy-to-read summaries
 2. Information Extraction
• Perform natural language generation, classification and a range of other natural language
processing (NLP) tasks, e.g. extract information from a given paragraph
 3. Question Answering
• A more structured prompt - combining instructions, context, input, and output indicators
to get improved results
 4. Text Classification
• Elements used in the prompt: input data / examples
 5. Conversation (Role Prompting)
• Instructing the LLM system on how to behave, its intent, and its identity
• This is particularly useful when you are building conversational systems like customer
service chatbots
 6. Code Generation
• Notice that you didn't even need to specify the language to use
• If you are trying to debug a code, the LLM system will automatically switch to the
corresponding programming language
• A great example of such application is Copilot
 7. Reasoning
• One of the most difficult tasks for LLM
• One of the most interesting areas due to the types of complex applications that can
emerge from LLMs
• Basic examples: show arithmetic capabilities
 AI Content Detectors
• Analysing the content's linguistic and structural features (semantic meaning, sentence
structure, language choices, etc)
• Comparing the text to existing datasets of content created by artificial intelligence or
humans to differentiate between the two.

https://www.cnet.com/tech/services-and-software/chatgpt-detectors-are-biased-and-easy-to-fool-research-shows/
 AI Detectors - Gimmick or Gambit?
• Detectors sometimes suggest that human-generated contents are produced by AI
• E.g. OpenAI tested well-known, human-written texts such as Shakespeare's literature and
the Declaration of Independence in their content detector, but they were labeled as AI-
generated by the detector
• News:

https://www.cnet.com/tech/services-and-software/chatgpt-detectors-are-biased-and-easy-to-fool-research-shows/
If you have not downloaded Anaconda Distribution, please
download it during the break!
 Online Python
• If you have not installed Python on your computer, you may go to Online Python by
visiting the link below

https://www.online-python.com/
 Python Interpreter

1. Hello, Python! 4. Control Flow (If Statements and

Loops)
2. Variables and Types
5. Functions
3. Basic Operations

https://www.w3schools.com/python/python_intro.asp
 Hello, Python!
• Let's begin with the classic "Hello, World!" program
• Open your Python environment and type:

print("Hello, World!")

• This line introduces you to the print function, which displays text on screen
 Variable and Types
• In Python, you can store information in variables without declaring their type. For
instance:

name = "Alice"
age = 25
height = 5.9

• Here, name is a string, age is an integer, and height is a float (decimal number)
 Basic Operations
• Python allows easy manipulation of numbers and strings
• Example:
a=5
b=3

#Arithmetic
sum_result = a + b
difference_result = a - b
product_result = a*b
quotient_result = a/b

#String Concatenation
greeting = "Hello, "
name = "World!"
full_greeting = greeting + name
 Control Flow
(If Statements and Loops)
• Make decisions with if statements and repeat actions with loops: x = 10

#Conditional statement
if x > 5:
print("x is greater than 5")
else:
print("x is not greater than 5")

#For loop
for i in range(5):
print(i)
 Functions
• Functions help organize your code
• Example:

#Function definition
def greet(name):
return "Hello, " + name + "!"

#Function call
message = greet("Bob")
print(message)
 Challenge - Sum and Min
• Write a program to use a while loop to read the numbers from the user repeatedly until
he/she inputs 0, then report the sum of these numbers and the minimum value among
them
• Note: You don’t need to store all the numbers
• If you are unable to get the desired output, you may ask Monica for debug
 Sample Code
num = int(input())
total = num
minimum = num

while num != 0:
num = int(input())
total += num
if num != 0 and num < minimum:
minimum = num

print("sum =", total)

print("min =", minimum)
 OpenCV
• An open-source computer vision and machine learning software library
• It was originally developed by Intel and is now supported by Willow Garage and Itseez
• Written in C++ and provides a Python interface
 Features and Capabilities

• Image and video processing • 3D reconstruction

• Object detection and recognition • Camera calibration and 3D vision
• Facial recognition • Machine learning and deep learning
• Motion analysis and tracking
 Applications
• Robotics and automation • Medical imaging
• Surveillance and security • Automotive applications
• Augmented reality • Industrial automation
 Tutorial - Face Detection
1. Go to Anaconda Distribution and Create a virtual environment
2. Launch VS Code
3. Follow the following commands:
#change to virtual environment
conda activate virtual

#install OpenCV
pip install opencv-python

#import libraries
import
# Load cv2
the image
import
image =numpy as np
cv2.imread('image.jpg')

https://medium.com/jimmy-wang/opencv-%E5%9F%BA%E7%A4%8E%E6%95%99%E5%AD%B8%E7%AD%86%E8%A8%98-with-python-d780f571a57a
# Convert the image to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

#Use a pre-trained Haar Cascade Classifier to detect people in the image

people_detector = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_fullbody.xml')
people = people_detector.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
#Draw rectangles around the detected people and display the image
for (x, y, w, h) in people:
cv2.rectangle(image, (x, y), (x+w, y+h), (0, 255, 0), 2)

cv2.imshow('People in Image', image)

cv2.waitKey(0)
cv2.destroyAllWindows()

# Count the number of detected peoplenum_people = len(people) print(f"Number of people detected:

{num_people}")
 Tutorial - Face Detection
1. Go to Anaconda Distribution
2. Launch VS Code
3. Follow the following commands:
#install OpenCV
pip install opencv-python

#import libraries
import cv2
import numpy as np

# Load the image

image = cv2.imread('image.jpg')

https://medium.com/jimmy-wang/opencv-%E5%9F%BA%E7%A4%8E%E6%95%99%E5%AD%B8%E7%AD%86%E8%A8%98-with-python-d780f571a57a
# Convert the image to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

#Use a pre-trained Haar Cascade Classifier to detect people in the image

people_detector = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_fullbody.xml')
people = people_detector.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
#Draw rectangles around the detected people and display the image
for (x, y, w, h) in people:
cv2.rectangle(image, (x, y), (x+w, y+h), (0, 255, 0), 2)

cv2.imshow('People in Image', image)

cv2.waitKey(0)
cv2.destroyAllWindows()

# Count the number of detected peoplenum_people = len(people) print(f"Number of people detected:

{num_people}")
OpenCV + ChatGPT
直接變大師
(Become a Master Directly)