<style>
.leftSmall {
text-align: left;
font-size: 40%;
}
.reveal blockquote {
width: 100%;
text-align: ;
box-shadow: none;
color: #411900;
font-size: 70%;
}
.reveal a {
color: #00A7E1;
}
.reveal table {
font-size: 50%;
margin: inherit;
}
.reveal table th, .reveal table td {
border: none;
}
p, .reveal ul {
color: #65463e;
}
.smallLeft {
text-align: left;
font-size: 25%;
}
.smallImage img{
max-width: 50%;
}
</style>
## Survey Design,
## Data Management
## &
## Qualtrics
---
Mathew Vis-Dunbar
Data & Digital Scholarship Librarian
[mathew.vis-dunbar@ubc.ca](mailto:mathew.vis-dunbar@ubc.ca)
[csc.ok.ubc.ca](https://csc.ok.ubc.ca)
[https://tinyurl.com/igs-515b](https://hackmd.io/@vdunbar/BkOgteOaj)
[https://tinyurl.com/igs-515b-pdf](https://drive.google.com/file/d/1wWbIaSIYNMRnG0hxicDOe5vGJjgQ81A7/view?usp=sharing)
---
## Plan of Action
* Qualtrics platform
* Planning ahead
* Implementing the plan
* Pulling data
* Analyzing data
---
# Qualtrics Platform
----
## Create an account
[UBC IT Resource Page](https://it.ubc.ca/services/teaching-learning-tools/survey-tool)
[ubc.qualtrics.com](https://ubc.qualtrics.com)
----
## Landing page & projects
Frequently accessed items and notifications.
Note:
Overview of what's presented to you. Head into the projects area. Organize projects with folders if you have a lot of surveys you're collecting!
----
## Creating a New Survey
Projects > Create project > Survey > Get started
Note:
Start a new blank survey. Expand the options pane.
* The builder is where we construct questions.
* The survey flow is where we can govern how people move through our survey and where we can handle re-directions. For example, if we want to end a survey if a respondent indicates a particular preference, we can do that.
* Look and feel allows us to customize the overall appearance; this will be audience specific and can be rather important. [show csc as an example]
* Survey options lets you customize messages, collect data without identifiers etc.
* Translations allows you to offer a single survey in multiple languages; this is manual!
----
## Survey Actions
* Workflows
* Distributions
* Data & Analysis
* Results
* Reports
Note:
Across the top bar, we have another series of options, mostly dealing with survey and data management. Many of these options are designed for non-academic research audiences.
* Workflows can automatically trigger events, like sending an email based on a specific response.
* Distributions allows you to control how people come to your survey.
* Data and analysis gives you access to your responses, results presents you with. We'll look briefly at distributions and exporting data.
* Results provides in house analysis.
* Reports allows custom, reproducible reports to be generated from data.
----
## Survey Design & Build
* Builder
* Flow
* Look & Feel
* Options
* Translations
---
# Planning Ahead
----
## DMPs & Dictionaries
Note:
Before diving into the survey builder, and all that other jazz, I'm going to suggest organizing everything outside of Qualtrics. Running a survey is about collecting data, so we have to have a plan for what we're going to do with that data - we don't want to collect data that we don't need -- which means we should know why we're asking what we're asking -- we want to be able to clearly communicate to our participants how their data will be handled, and we want to be set up to ensure that the data is organized, accessible, and stored in a way that is conducive to whatever analysis we intend to follow up with.
We also want to keep things as simple as possible for both us and our respondents. Too many options can get confusing, especially when it comes to processing the data, and being able to work through these complexities and simplify them in advance is advantageous.
----
## Data Management Plans
* Collection tools & Sources
* Roles & Responsibilities
* Privacy & Security
* Storage locations & Requirements
* Distribution
* Preservation
* [Organization]
Note:
We'll talk about the data aspect in terms of a data management plant, which is something we should have. We'll then look at how this interfaces with building content outside of Qualtrics.
Things that a DMP should articulate include: data **sources** and **collection tools**, **roles and responsibilities** related to the data, if there are considerations related to **privacy and security**, where data will be **stored**, how data might be distributed or **shared**, and what **preservation** or destruction looks like.
Additionally, a DMP might address how data will be **organized**,
----
## DMP Assistant
[Portage DMP Assistant](https://assistant.portagenetwork.ca/)
----
## Collection tools
> Data will be collected using UBC's survey tool, Qualtrics. More information about this tool is available [here](https://it.ubc.ca/services/teaching-learning-tools/survey-tool).
>
> The survey will be distributed to participants via email with a customized link to track responses.
Note:
Articulating all of this in advance in a succinct way can help facilitate implementing the actual data collection process as well as increase the transparency of the research process.
----
## Data sources
> Participants will be sourced from an after-school care program in Kelowna. Participation will be voluntary; the researcher has active involvement with this group and will solicit volunteers during regular meetings.
>
> Participants must be 18 or over.
----
## Responsibilities
> The researcher will be responsible for implementing the study, data collection, and data management. The researcher's supervisor will handle post project requirements to maintain data on UBC supported services.
>
> Throughout the project, URAs will support aspects of the analysis and will have access to de-identified data only.
| Resource |
| --- |
| [UBC Policy SC6 - Scholarly Integrity Policy](https://universitycounsel.ubc.ca/policies/scholarly-integrity-policy/) |
| Support |
| --- |
| csc.ok@ubc.ca |
Note:
The CIO is responsible for drafting these policies and standards. The emphasis tends to be on administrative data, not research data, which can make interpretation difficult.
----
## Privacy & Security
> Data is being collected with personal identifiers. These will be replaced with codes. The key will only be available to the researcher and their supervisor, and will be encrypted at the file level.
>
> All data will be stored on UBC administered services. All personal devices used in the conduct of the project (phone, tablet, personal computer, external storage, etc) will be encrypted at the device level.
| Resource |
| --- |
| [Information Security Standard U1 - Security Classification of UBC Electronic Information](https://cio.ubc.ca/information-security-standards/U1) |
| [Information Security Standard U5 - Encryption Requirements](https://cio.ubc.ca/information-security-standards/U5) |
| Example |
| --- |
| [Microsoft OneDrive and Teams for Research](https://arc.ubc.ca/microsoft-onedrive-and-teams-research) |
| Support |
| --- |
| csc.ok@ubc.ca |
----
## Storage
> The primary storage device will be the researcher's personal laptop.
>
> Regular back-ups will be made, syncing to UBC's OneDrive and the researcher's supervisor's P Drive. Back-ups will be run on a weekly basis, end of day each Friday.
| Resources |
| --- |
| [https://researchdata.library.ubc.ca/plan/](https://researchdata.library.ubc.ca/plan/) |
| [UBC IT Knowledge Base](https://knowit.ok.ubc.ca/article/list-of-okanagan-campus-file-storage-shares-508.html) |
| [OneDrive](https://it.ubc.ca/services/web-servers-storage/microsoft-onedrive) |
| [Teams](https://it.ubc.ca/services/email-voice-internet/microsoft-teams) |
| [ARC Resources](https://arc.ubc.ca/compute-storage) |
| Support |
| --- |
| csc.ok@ubc.ca |
----
## Distributing Data
> Participants will be provided with the opportunity to allow de-identified data to be shared at the unaggregated level. Additionally, participants will be made aware of all data that may need to be publicly deposited to support funder and publisher requirements.
>
> Anonymized, unaggregated data will be deposited in UBC Dataverse. Data deposit supporting publication will either be deposited in UBC Dataverse or on Dryad unless otherwise specified by publisher requirements.
| Resource |
| --- |
| [https://researchdata.library.ubc.ca/deposit/](https://researchdata.library.ubc.ca/deposit/) |
| Support |
| --- |
| csc.ok@ubc.ca |
----
## [Organization]
> All project related content will be stored in a single project folder. There will be a dedicated folder to hold research data.
>
> Documentation will be kept through file naming conventions, readmes, and data dictionaries. This documentation will, among other things, describe the nature of the project's data, including it's provenance (who, when, where, how, etc), file formats, and any necessary variable level descriptions.
>
> A code book detailing all questions and answer ranges will be prepared to accompany the survey.
| Resource |
| --- |
| [https://researchdata.library.ubc.ca/plan/](https://researchdata.library.ubc.ca/plan/) |
| Support |
| --- |
| csc.ok@ubc.ca |
----
## Data Dictionaries
Note:
Also known as a code book, a data dictionary should, at the very least, list the questions that we're asking, and the options available.
----
## Data Dictionaries
Note:
But ideally has more detail. In fact, the more detail it contains, the more it will begin to mirror exactly what we need to put into Qualtrics to implement our survey.
----
## Data Dictionaries
---
# Implementing the Plan
----
## Tell them what to expect
----
* What is the study
* Who are you
* What kinds of questions will you be asking
* How long will this take
* Can answers be saved
* Is there a downloadable version of the form
* What will you do with the data
* How do they do if they have questions
Note:
* What is the study
* Who are you
* What kinds of questions will you be asking
* How long will this take
* Can answers be saved
* Is there a downloadable version of the form
* What will you do with the data
* How do they do if they have questions
----
<div class = 'smallLeft'>
<strong>Background</strong>
This survey is part of the research project 'Questioning the Quest: Exploring Monty Python's Holy Grail'. More information about this research project, including the team behind the study, is available at https://qqmt.org
<strong>Survey Flow</strong>
This survey is comprised of upwards of 7 questions and should take approximately 5 minutes to complete.
The survey does not need to be completed in one sitting; you may save your answers and return to the survey, however you must have browser cookies enabled and be returning to the survey on the same browser. Alternatively, you may download a copy of the questions to draft answers in advance. Download a copy here.
<strong>Data Collection and Storage</strong>
All data is being collected for the explicit purposes of research.
You may opt to consent to have your de-identified data shared. Otherwise, the data you provide will be reported on only at a summary level and may appear in conference proceedings, journal publications, and the media as part of knowledge dissemination activities. If you consent, direct quotes may also be used.
All data will be stored within Canada on UBC approved storage solutions. Data directly collected from this study will be stored for a minimum of 5 years post study completion. The researcher (Mathew Vis-Dunbar) will coordinate all management of data during the study; upon study completion, the Faculty Supervisor (Robert Janke) will steward the data until its destruction. Some data may be deposited for longer term storage.
The project's data management plan, which explicitly outlines how data will be stored and accessed is available at this link.
<strong>Privacy and Security</strong>
All data is being collected with personal identifiers. Identifiers will be stripped to de-identify data before analysis, but a key to re-identify will be maintained. Only the researcher and their supervisor will have access to this key. All of their data may be reviewed by the rest of the research team. Please see https://qqmt.org/about/team for more details on the research team.
<strong>Follow up and Questions</strong>
Upon completion of this survey, you have the option to download a pdf copy of your responses.
If you have any questions about this survey, before proceeding, please contact one of:
Mathew Vis-Dunbar (PI) | mathew.vis-dunbar@ubc.ca
Robert Janke (Supervising Faculty) | robert.janke@ubc.ca
By clicking the next arrow, you consent to participate in this survey.
</div>
----
## Migrate the dictionary
----
## Get Help
[Qualtrics Support](https://www.qualtrics.com/support/)
---
# Pulling data
----
## The Survey
[https://tinyurl.com/igs-survey](https://tinyurl.com/igs-survey)
---
# Analyzing data
----
## Questions & Data Types
* Open ended questions are challenging to analyze!
* Questions may be posed language and converted to numeric data
* Categorical data may be ordered or unordered
----
## Getting Help
<div class = 'smallImage'>
<img src="https://i.imgur.com/MlJvYiq.png">
</div>
[https://tinyurl.com/csc-consults](https://tinyurl.com/csc-consults)
---
{"metaMigratedAt":"2023-06-17T20:33:41.734Z","metaMigratedFrom":"YAML","title":"IGS 515B","breaks":true,"slideOptions":"{\"theme\":\"serif\"}","contributors":"[{\"id\":\"ac08882d-85e9-4611-a1dd-24f635bf6517\",\"add\":33709,\"del\":19893}]"}