1

SMU Classification: Restricted

SMU-IRB: Participant Information Sheet and Informed Consent Form (Online)

Title of Research Study as it will be stated on the Informed Consent Form provided to participants:
Evaluation of Culture Understanding Skills of Multimodal LLMs

Principal Investigator, Title, and Affiliation: Professor NGO Chong Wah, SCIS, SMU

__________________________________________________________________________________

1. Purpose of Research Study:

The purpose of this research study is to evaluate the ability of large-scale multimodal language models to
understand and accurately represent cultural elements from Southeast Asian countries. Specifically, we
aim to explore how these models identify and label culturally significant artifacts, practices, and concepts
in images. This study involves collaboration with local annotators to ensure cultural accuracy and relevance
in the data used for training and evaluating these models.

2. Study Procedures and Duration:

As a participant, you will be involved in the following tasks:

Select relevant images and label the polygon:

You will select relevant images that contain humans or human parts from an image pool associated with a
cultural keyword, and label them by drawing polygons around these areas. Each image is estimated to take
approximately 1 minute to label.

The total time required to complete labeling for your country is estimated to be roughly 40 hours. However,
you have the right to stop the task at any time. If you choose to withdraw, you will still receive prorated
compensation based on the number of images you have accurately completed. Please note that each label
will be reviewed for accuracy before it is considered complete.

Review and Feedback: Your contributions will be cross-checked with those of other annotators and
reviewed by researchers to ensure accuracy. If discrepancies or unclear annotations are found, feedback
will be provided, and you may be asked to make adjustments to ensure the data meets the study's quality
standards.

If you disagree with the feedback provided, you are encouraged to discuss the differences with the research
team. We value your expertise and cultural knowledge, and any clarifications or additional insights you
provide will be taken into account. In cases where there are significant differences between annotators, the
research team will consult with multiple annotators and experts to understand the cultural context better
and reach a consensus.

If consensus cannot be reached, the final decision will be made by the project PI, who will carefully consider
all perspectives and feedback before proceeding.

The research team recognizes the importance of cultural expertise and will ensure that all differences are
handled respectfully, aiming to incorporate as much cultural accuracy as possible.

3. Benefits of Study:

By participating in this study, you will contribute to advancing the understanding of cultural representation
in artificial intelligence, especially for Southeast Asia. Your input will help ensure that AI models are trained
/ evaluated with culturally accurate data, leading to more inclusive and culturally aware AI technologies.

SMU-IRB: Participant Information Sheet and Informed Consent Form (Online) dated 1 Jul 2016
 2
SMU Classification: Restricted

This research may also provide insights into how AI can better serve diverse communities by recognizing
and respecting cultural differences.

As a token of appreciation for your time and effort, you will be compensated at $5 / 50 images accurately
labelled, or at a rate agreed between you and the researcher. If you withdraw mid-way you will not be
penalised, your approved work will still be eligible to receive a prorated compensation.

4. Possible Risks of Study:

The risks associated with this study are minimal. You may experience some discomfort if you encounter
images or concepts that do not accurately reflect your culture or if the process of categorizing and labeling
images is time-consuming. However, you are free to skip any tasks or questions that make you
uncomfortable and can withdraw from the study at any time without penalty.

5. Confidentiality and Privacy of Research Data :

Your privacy and confidentiality are of utmost importance to us. All data collected during this study will be
kept strictly confidential. Your responses and annotations will be anonymized, and no personal identifying
information will be linked to your data. The data will be stored securely and will only be accessible to the
research team. Results from the study will be reported in aggregate form, and individual responses will not
be identifiable. If any data is shared for academic or research purposes, it will be in a de-identified format
to protect your privacy.

6. Contact Details:
• For questions/ clarifications on this study, please contact the Principal Investigator, NGO Chong
Wah, at email address [email protected], and/or office/mobile number: +65 68264818.

• If you have any questions or concerns regarding your rights as a participant in this research study
and wish to contact someone unaffiliated with the research team, please contact the SMU
Institutional Review Board Secretariat at [email protected] or + 65 68281925. When contacting SMU
IRB, please provide the title of the Research Study and the name of the Principal Investigator, or
quote the IRB approval number (IRB-24-154-A113(924)).

• Please bookmark or save a copy of this information sheet and informed consent form for your
records.

__________________________________________________________________________________

SMU-IRB: Participant Information Sheet and Informed Consent Form (Online) dated 1 Jul 2016
 3
SMU Classification: Restricted

SMU-IRB: Participant Information Sheet and Informed Consent Form (Online)

Title of Research Study as it will be stated on the Informed Consent Form provided to participants:
Evaluation of Culture Understanding Skills of Multimodal LLMs

Principal Investigator, Title, and Affiliation: Professor NGO Chong Wah, SCIS, SMU

__________________________________________________________________________________

Principal Investigator’s Declaration:

I have explained and defined in detail the research procedures in which the participant (or legal
representative) has consented to participate.

I also declare that the data collected for this research study will be handled as stated above.

____________________________________
Principal Investigator’s Name and Date: Ngo Chong Wah, 3-Sep-2024

Participant’s Declaration:

I understand that participation is voluntary. Refusal to participate will involve no penalty.

I declare that I am at least 18 years of age.

If I am affiliated with Singapore Management University, my decision to participate, decline, or withdraw

from participation will have no adverse effect on my status at or future relations with Singapore
Management University.

I have read and fully understood the contents of this form, and hereby give consent to the Singapore
Management University research team and its affiliates for this project to collect and/or use my data for the
purpose(s) described in this form.

By replying “I have read and fully understood the contents of this form, and would like to participate”, I
consent to participate in this study and agree to all of the above.

If you do not wish to participate in the study, you may close the doc now to exit.

SMU-IRB: Participant Information Sheet and Informed Consent Form (Online) dated 1 Jul 2016