LibGuides: Research Data and Reproducibility: Processing Data

What is data processing?

Data processing activities include cleaning, transformation, enrichment, validation, analysis, and visualization. Here you will find a variety of tools and resources to facilitate these activities.

Cleaning, Transforming, and Validating Data

Tidy Data
This article in the Journal of Statistical Software by data scientist Hadley Wickham outlines a standard method for displaying multivariate data in the form of a data matrix where each variable is a column, each observation is a row, and each type of observational unit is a table. It makes it easy to tidy messy datasets and to develop tools for data analysis.
Frictionless Data
The Frictionless Data project aims to make it easier to work with data by reducing common data workflow issues. You can use Frictionless to describe your data (add metadata and schemas), validate your data, and transform your data. You can also write custom data standards based on the Frictionless specifications.
CSV on the Web: A Primer
The CSV on the Web Working Group has developed standard ways to express useful metadata about CSV files and other kinds of tabular data. This primer takes you through the ways in which these standards work together, covering what we mean by "tabular data" and "CSV", where files that provide metadata about CSV live, how to create a schema to validate the content of a CSV file, how to specify how a CSV file should be converted to RDF or JSON, and how to provide other documentation and metadata about a CSV file.
Great Expectations
Great Expectations is a shared, open standard for data quality. It helps data teams eliminate pipeline debt, through data testing, documentation, and profiling.
Data Cleaning with Excel
This guide from the Galter Health Sciences Library at Northwestern's Feinberg School of Medicine offers tricks and tips for cleaning and organizing tabular data in Microsoft Excel.
OpenRefine
OpenRefine (previously Google Refine) is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.
Cleaning Spreadsheet Data with OpenRefine
This guide from the Galter Health Sciences Library at Northwestern's Feinberg School of Medicine can be used to learn the basic functions of OpenRefine.
Essential Spreadsheet Data Cleaning with OpenRefine
This guide from the Galter Health Sciences Library at Northwestern's Feinberg School of Medicine offers guidance on some of the more advanced features of OpenRefine.

Exploring and Analyzing Data

Datasette
Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size, analyze and explore it, and publish it as an interactive website and accompanying API.
VisiData
VisiData is a free, open-source tool that lets you quickly open, explore, summarize, and analyze datasets in your computer’s terminal. VisiData works with CSV files, Excel spreadsheets, SQL databases, and many other data sources.

ATLAS.ti Qualitative Data Analysis Software
- This link opens in a new window
ATLAS.ti is a qualitative data analysis and management tool that helps users to systematically arrange, reassemble, and manage large bodies of textual, graphical, audio and video data.
more...less...
At this time, ATLAS.ti is available on one computer on the 4th floor of the OHSU Library at the BICC (the 24-Hour Space). This workstation may be reserved up to 90 days in advance and for up to 6 hours per day.
Click the calendar icon to visit our reservation page.
NVivo Qualitative Data Analysis Software
- This link opens in a new window
NVivo (produced by QSR International) is a software program used for qualitative and mixed-methods research. Specifically, it is used for the analysis of unstructured text, audio, video, and image data, including (but not limited to) interviews, focus groups, surveys, social media, and journal articles.
more...less...
At this time, NVivo is available on one computer on the 4th floor of the OHSU Library at the BICC (the 24-Hour Space). This workstation may be reserved up to 90 days in advance and for up to 6 hours per day.
Click the calendar icon to visit our reservation page.
Taguette
Taguette is a free and open-source tool for qualitative research. You can import your research materials, highlight and tag quotes, and export the results!

Posit Workbench (formerly RStudio Statistics Software)
Posit Workbench is the premier development experience for data scientists who use R and Python.
SAS Statistics Software
SAS Statistics is a software package used for statistical analysis. OHSU licenses the Windows version only. It is available for OHSU employees and students. The software is licensed only for academic purposes. Visit the link to learn how to obtain a copy of SAS from the Library.
more...less...
This program is also available for use on general computers in both the BICC and the Learning Resources Center(LRC) at the RLSB.
SPSS Statistics Software
SPSS Statistics is a software package used for statistical analysis. It is available for OHSU students and employees for both work and personal computers. The software is licensed only for academic purposes. Visit the link to learn how to obtain a copy of SPSS from the Library.
more...less...
This program is also available for use on general computers in both the BICC and the Learning Resources Center(LRC) at the RLSB.

Data Scavenger Hunt: Data Exploration of NHANES
Materials from the OHSU BioData Club workshop, which introduced data exploration using the publicly available NHANES (National Health and Nutritional Examination Survey) to learn about answering questions about diabetes and depression.
A Dive into Logistic Regression Modeling
Materials from the OHSU BioData club workshop, which introduced the mathematics of logistic regression for binary classification modeling, and how to prevent the harms of overfitting with cross validation in R.
LinkedIn Learning: Data Analysis
A collection of data analysis videos and courses, curated by OHSU Career and Workplace Enhancement (CWE) Staff.
more...less...
All OHSU employees and students have free access to LinkedIn Learning's extensive subscription library thanks to a partnership between ITG, OHSU/AFSCME Labor Management Committee (LMC) and the Career and Workplace Enhancement (CWE) Center.

Activate your membership

Go to this link: https://lnkd.in/ge-cddd
Enter your OHSU username and password.
If you are not on campus and not using VPN, use DUO to authenticate your log in.
If this is your first time logging in: choose to connect your LinkedIn profile or skip this step. You do not need a LinkedIn profile to use LinkedIn Learning. If you have a LinkedIn profile, connecting it will provide a more personalized learning experience.
Start exploring!

Logging in after membership activation

Go to this link: https://lnkd.in/ge-cddd
Enter your OHSU username and password
If you are not on campus or using VPN, use DUO to authenticate your log in.
IF YOU CHOSE connect your LinkedIn Profile to your LinkedIn Learning account, enter your LinkedIn Profile password. If YOU DID NOT connect your LinkedIn Profile to your LinkedIn Learning account, SKIP THIS STEP.
Begin learning!

To learn more, visit the CWE Center O2 site or email cwecenter@ohsu.edu.

Visualizing Data

Tableau
Tableau is an application designed to connect to many different data resources for the purpose of creating meaningful graphs and charts and present them in reports or dashboards. This link describes Tableau licensing options available to OHSU members facilitated by OHSU's Business Intelligence community.
Data Exploration and Visualization with Tableau
Materials from the OHSU BioData Club workshop, which provided a hands-on introduction to creating interactive visualizations with Tableau Public, the free version the software. Instructor Connor Smith covered how to load and prepare data, the Tableau interface, and creating basic visualization types and interactive dashboards.
Data Storytelling
Materials from the OHSU BioData Club workshop, which covered techniques for creating figures that communicate findings clearly. Discusses the role of color, annotations, and simplifying figures to communicate effectively.
A Practical Introduction to Data Visualization with R and ggplot2
Materials from the OHSU BioData Club workshop, which introduced beginners to practical hands-on examples using R and ggplot2, tools for making impactful dat visualizations, to one's own data analyses, and provided experienced users an opportunity to test their knowledge and learn new tricks and techniques.
LinkedIn Learning: Data Visualization
A collection of data visualization videos and courses, curated by OHSU Career and Workplace Enhancement (CWE) Staff.
more...less...
All OHSU employees and students have free access to LinkedIn Learning's extensive subscription library thanks to a partnership between ITG, OHSU/AFSCME Labor Management Committee (LMC) and the Career and Workplace Enhancement (CWE) Center.

Activate your membership

Go to this link: https://lnkd.in/ge-cddd
Enter your OHSU username and password.
If you are not on campus and not using VPN, use DUO to authenticate your log in.
If this is your first time logging in: choose to connect your LinkedIn profile or skip this step. You do not need a LinkedIn profile to use LinkedIn Learning. If you have a LinkedIn profile, connecting it will provide a more personalized learning experience.
Start exploring!

Logging in after membership activation

Go to this link: https://lnkd.in/ge-cddd
Enter your OHSU username and password
If you are not on campus or using VPN, use DUO to authenticate your log in.
IF YOU CHOSE connect your LinkedIn Profile to your LinkedIn Learning account, enter your LinkedIn Profile password. If YOU DID NOT connect your LinkedIn Profile to your LinkedIn Learning account, SKIP THIS STEP.
Begin learning!

To learn more, visit the CWE Center O2 site or email cwecenter@ohsu.edu.