Scientific Computing Training Series

Instructor: Chris Myers

Description: This lecture will introduce the Python programming language, the Python software ecosystem, some key concepts in computer programming, and how those concepts are implemented in Python. The Python ecosystem contains a rich set of packages and tools to support research and data analysis in several different application areas; being able to use the Python programming language to customize computing workflows that leverage those tools enhances researcher productivity and capability. The material is intended both for people new to programming or new to Python who want to get started, and for more experienced Python programmers who would like to get a different perspective on how Python supports a variety of programming tasks.

Level: Introductory

Prereqs: None

• Introduction to Python Slides (Requires WCM CWID to access)
• Introduction to Python Slides  (Requires CU NetID to access)
• Introduction to Python recording 

Instructor: Christopher Cameron

Description: Jupyter is the most common Python interface used by researchers. Cloud computing providers, like Amazon and Google, offer their own Jupyter-alike interfaces. This lecture provides an overview of the JupyterLab interface and cloud-based derivatives. It is designed to familiarize new users with common Jupyter-like interfaces, features and best practices.

Level: Introductory

• Introduction to Python Slides (Requires WCM CWID to access)
• JupyterLab (in the Cloud) for Python Slides (Requires CU NetID to access)
• JupyterLab (in the Cloud) for Python Recording 

Instructor: Chris Myers

Description: This lecture will provide an overview of select core components of the Python software ecosystem for scientific computing and data science, with a particular focus on numpy, scipy, pandas, and matplotlib. The lecture will include both descriptions of the overall design and structure of those packages and their key components, and numerous code examples that demonstrate some of the important functionality. Opportunities for live, hands-on exercises using these packages will be integrated throughout the lecture, all as part of a Jupyter notebook that will include both the lecture content and the hands-on exercises. The Python ecosystem for scientific computing and data science ecosystem enables researchers to use proven and widely-used tools that are easily customized for specific problems using the Python programming language.

Level: Intermediate

Prereqs: Some familiarity with the Python programming language or other languages used for scientific computing (e.g., R, MATLAB) would be useful, but is not required.  The hands-on exercises will be coordinated through the use of an online cloud environment providing support for running Jupyter notebooks, although participants should feel free to use their own local machines if they are familiar with running Jupyter and installing whatever additional packages might be necessary.  Instructions about these details will be circulated in advance of the lecture, but participants should be prepared to set up and/or sign up for accounts in those environments before the lecture so that they are ready to run the hands-on exercises during the allotted time.

• Scientific Computing with Python (hands-on) Slides (Requires WCM CWID to access)
• Scientific Computing with Python (hands-on) Slides (Requires CU NetID to access)
• Scientific Computing with Python (hands-on) Recording 

Instructor: Christopher Cameron

Description: RStudio is a common R interface used by researchers. This overview is designed to familiarize new users with the interface, features and best practices so they are ready to delve into conducting their own analyses. New content includes an overview of Quarto — an evolution of Rmarkdown — for documenting and sharing data analysis and research.

• Scientific Computing with Python (hands-on) Slides (Requires WCM CWID to access)
• Scientific Computing with Python (hands-on) Slides (Requires CU NetID to access)

• Getting Started with R Recording

Instructor: Christopher Cameron

Description: This lecture presents several examples of data analysis and visualization in R. It will demonstrate a variety of analyses intended to help researchers determine if learning R is a good investment for their research, including new data analysis examples drawn from the WCM community.

Level: Introductory/Intermediate

Prereqs: Some familiarity with R

• Data Analyis with R Slides (Requires WCM CWID to access)
• Data Analysis with R Slides (Requires CU NetID to access)

• Data Analysis with R Recording 

Instructor: Adam Brazier

Description: An overview of managing data workflows for scientific computing, starting with data collection and aggregation, through processing and storing in an accessible form. We will cover some issues relating to security policy, integration of Identity and Access Management and retention policy (but this is not a security policy workshop!), possible storage venues and formats, models for aggregating and distributing data such as the pub/sub model, and modes of data storage such as relational database, file system, cloud, noSQL, Data Lake, etc.

Level: Introductory/Intermediate

Prereqs: Some knowledge of software and data processing

Time: 9am-10am EST

• Data Management in Science Research Slides (Requires WCM CWID to access)
• Data Management in Science Research Slides (Requires CU NetID to access)
• Data Management in Science Research Recording

Instructor: Christopher Cameron

Description: Learn to read R analysis scripts in this introduction to the R language. We will examine language fundamentals like built-in in data types, conditional execution, flow control, and indexing, then look at some basic data summary and modeling functions with an emphasis on how R is meant to be used. 

Level: Introductory

Prereqs: Some experience with statistical concepts and tabular data formats or spreadsheet software. 

• R Basics Slides (Requires CU NetID to access)
• R Basics Slides (Requires WCM CWID to access)
• R Basics Recording 

Instructor: Chris Myers

Description: An examination of the core components of the Python software ecosystem for scientific computing and data science, with a particular focus on numpy, scipy, and pandas. This lecture will describe the overall design and structure of these packages and some of their components, complemented by code examples that demonstrate some of the key functionality. Also addressed will be issues of performance and the integration of these core packages in the larger Python ecosystem.

Level: Intermediate

Prereqs: Some familiarity with the Python language would be useful but is not required.

Time: 9am-10am EDT

• Python for Scientific Computing and Data Science Slides (Requires CU NetID to access)
• Python for Scientific Computing and Data Science Slides (Requires WCM CWID to access)
• Python for Scientific Computing and Data Science Recording

Instructor: Christopher Cameron

Description: Humans generate messy data. While statistics-focused environments like R and Stata are great for data analysis, these specialized tools can be difficult to use with data that defies tabular representation. Human data, like written language, social relationships, images, and social media content, require flexible tools that can handle complexity. In this talk, we will provide an overview of Python, highlight how this free and open-source programming language supports digital humanities and social science research, and discuss Cornell and web-based resources to help you get started using Python in your research.

Level: Introductory

Prereqs: Some experience working with tabular data formats or spreadsheet software is helpful but not required.

Time: 12:15pm-1:15pm EDT

• Python for Digital Humanities and Social Science Slides (Requires CU NetID to access)
• Python for Digital Humanities and Social Sciences Slides (Requires WCM CWID to access)
• Python for Digital Humanities and Social Sciences Recording 

Instructor: Ben Trumbore

Description: An introduction to choosing the best type of chart to use for the data you have and the message you want to convey. Includes a breakdown of the different types of data you might have and descriptions of the main types of 2D data visualization. Does not include instruction for any particular visualization tool.

Level: Introductory

Prereqs: None

Time: 9am-10am EDT

• Creating the Best Visualizations for your Data Slides (Requires CU NetID to access)
• Creating the Best Visualizations for your Data Slides (Requires WCM CWID to access)
• Creating the Best Visualizations for your Data Recording

Instructor: Steve Lantz

Description: Git is a widely used tool for revision tracking and collaborative code development. The talk introduces Git and how to use it effectively in conjunction with a repository hosting service like GitHub.

Level: Intermediate

Prereqs: Programming ability and activity at a level to warrant revision tracking and (possibly) collaborative development of codes.

• Revision Control with Git Slides (Requires CU NetID to access)
• Revision Control with Git Slides (Requires WCM CWID to access)
• Revision Control with Git Recording

Instructor: Chris Myers

Description: An examination of some of the Python packages that support data visualization for various use cases, providing both a general discussion of capabilities and multiple code examples demonstrating specific functionality. This lecture will address the generation of both static images suitable for inclusion in publications and presentations, and interactive data visualizations useful for exploring complex datasets and steering computations. Packages examined include matplotlib, pandas, seaborn, plotnine, bokeh, plotly, and possibly others.

Level: Introductory/Intermediate

Prereqs: Some familiarity with the Python language would be useful but is not required.

• Python for Data Visualization Slides (Requires CU NetID to access)
• Python for Data Visualization Slides (Requires WCM CWID to access)
• Python for Data Visualization Recording 

Instructor: Adam Brazier

Description: While producing long-lasting software in academic research domains shares many of the same problems as commercial development, the environment is often different. In particular, the number of coders is often smaller, the people writing code may be learning as they go, development of software is often not their main career goal, and the funding model is different. This means that industry approaches to producing, maintaining, and operating software may not apply, or may have to be modified for the research environment. In this talk we will see some ideas, based on experience of research software at a variety of scales, to suit the different situations in which researchers develop software.

• Research Project Software Continuity Slides (Requires CU NetID to access)
• Research Project Software Continuity Slides (Requires WCM CWID to access)
• Research Project Software Continuity Recording

Instructor: Ben Trumbore

Description: An introduction to working with Excel spreadsheets from within computer programs and scripts. Python and C# examples will be given for reading Excel files and accessing their contents, as well as populating, formatting, and writing new Excel files.

Level: Intermediate

Prereqs: Familiarity with Excel files and modest experience programming in Python, C# or Java.

• Working with Excel Files in Python and C# Slides (Requires CU NetID to access)
• Working with Excel Files in Python and C# Slides (Requires WCM CWID to access)
• Working with Excel Files in Python and C# Recording

Instructor: Christopher Cameron

Description: Do you have a workflow with elements that can be automated? Sometimes the hardest part is knowing what might be possible. This case study involves using Python to process multichannel confocal microscopy images with ImageJ and then organize the output into PowerPoint slides.

Level: Introductory

Prereqs: None

• Case Study - Scripting ImageJ and PowerPoint with Python Slides (Requires CU NetID to access)
• Case Study - Scripting ImageJ and PowerPoint with Python Slides (Requires WCM CWID to access)
• Case Study - Scripting ImageJ and PowerPoint with Python Recording

Instructor: Steve Lantz

Description: Parallel processing is no longer just a concern for supercomputers--these days, it takes place in nearly all computing devices down to laptops and cell phones. This presentation describes parallel computing capabilities that are found within single processors and how applications can access them through techniques such as multithreading and vectorization.

Level: Introductory/Intermediate

Prereqs: Familiarity with programming in any language and with using a command-line interface 

• Slides - Case Study - Using the Whole Processor (Requires CU NetID to access)
• Slides - Case Study - Using the Whole Processor (Requires WCM CWID to access)
• Recording - Using the Whole Processor 

Instructor: Adam Brazier

Description: An introduction to the use of relational (SQL) databases, with a brief overview of database structure then covering SQL queries, some information on best practices, and development tools. We will mostly deal with ANSI SQL which will run on most Relational Database Management Systems (RDBMs), noting some important inter-RDBMS differences. Covered will be SQL queries for data retrieval, insertion and deletion, correlated subqueries and how to construct a complicated query. We will also discuss the interface between the database and the code, including the use of Object-Relational Model tools and stored procedures. 

Level: Introductory/Intermediate

Prereqs: This session does not require having knowledge of how to write SQL queries for data extraction, insertion, and deletion, but is a convenient companion to such a workshop or pre-existing knowledge.

• Slides - Using Relational Databases (Requires CU NetID to access)
• Slides - Using Relational Databases (Requires WCM CWID to access)
• Recording - Using Relational Databases

Instructor: Ben Trumbore

Description: An introduction to the numerous tools that can be used to transfer data between computer systems and to cloud storage. Different tools offer different transfer speeds, security, and ease of use, and some automatically recover from failures and allow syncing between systems. Tools covered in this session include FTP, SCP, SFTP, rsync, rclone and Globus.

Prereqs: Familiarity with basic Linux commands.

Time: 9am-10am EST

Slides + Recording:

• Data Transfer Tools Slides (requires WCM CWID to access)
• Data Transfer Tools Slides (requires CU Net ID to access)
• Data Transfer Tools Recording 

Instructor: Christopher Cameron

Description: Overview of R Studio interfaces (scripts, console and Rmarkdown notebooks) and popular modern data manipulation and visualization packages in R (especially ggplot and dplyr). This is primarily useful for people seeking an entry point into the R ecosystem for research and data analysis tasks.

Prereqs: Some experience with statistical concepts and tabular data formats or spreadsheet software.

Time: 9am-10am EST

• Introduction to Modern R Data Analysis Slides (requires WCM CWID to access)
• Introduction to Modern R Data Analysis Slides (requires CU Net ID to access)
• Introduction to Modern R Data Analysis Recording

Instructor: Christopher Cameron

Description: Jupyter Notebooks are a popular format for scientific communication that intermingles descriptive text, code, statistical analysis, results, and visualizations in a single document. This workshop showcases the features of Jupyter notebooks, demonstrates how to use and share notebooks effectively, and explains how to address common pain points. This workshop is useful for people who send, receive or use Jupyter notebooks. 

Time: 9am-10am EST

• Introduction to Jupyter Lab for Python Slides (requires WCM CWID to access)
• Introduction to Jupyter Lab for Python Slides (requires CU Net ID to access)
• Introduction to Jupyter Lab for Python Recording

Instructor: Chris Myers

Instructor: Steve Lantz

Description: Presents an introduction to using Linux operating systems. Includes practical techniques for working with the file system, descriptions of common commands and information about customizing a user’s environment. Can be tailored to a specific flavor of Linux.

Prereqs: Some familiarity with hierarchal file systems and a modern computer operating system (macOS, Linux, or Windows).

Time: 9am-10am EST

• Linux for Researchers Slides (Requires WCM CWID to access)
• Linux for Researchers Slides (Requires CU NetID to access)
• Linux for Researchers Recording