Tutorials.

GTN Tutorials aim to follow best practices in course design [23] and are typically constructed around a real-world research story, e.g., a published journal article describing an analysis workflow or dataset. The tutorial starts by introducing the relevant scientific background, and then proceeds to describe the analysis, providing details on the relevant scientific and computational concepts involved at each step. The tutorials are composed of alternating theory and practical sections, interspersed with formative assessment questions and exercises to enhance learning [24], and may be supported by a set of introductory slides (Fig 3). Our tutorials are self-contained; everything needed to complete them is bundled with the tutorial; this includes all the datasets, lecture slides, videos, the workflows, and the list of public Galaxy servers supporting the tutorial. No software installation is needed to follow these tutorials; the only technical requirement is access to internet.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Structure of a GTN tutorial and its features.

(A) Screenshots for parts of a GTN tutorial to show its structure. Title and authors are listed, followed by an overview box containing metadata about the tutorial (target audience, learning objectives, prerequisites, supporting materials, time estimate, etc). The tutorial content itself is a mix of theory (text) and so-called “hands-on” boxes describing practical steps to be performed in Galaxy. Question and answer boxes may be added at any point in the tutorial to allow students to self-evaluate as they progress. The end of a tutorial provides a box with take-home messages, references, and suggestions for further reading and follow-up tutorials. Additionally, a feedback form is embedded at the end of every tutorial. Access to FAQs, support channels, page view statistics, and translated versions of the materials are provided via the top menu of the webpage. (B) Features of a GTN tutorial grouped in 4 categories: rich metadata, structured content, strong (technical) support, and community oriented.

https://doi.org/10.1371/journal.pcbi.1010752.g003

A survey of trainers identified a desire of increased interactivity between tutorials and Galaxy. We have developed a new Galaxy feature “Tutorial Mode” that allows following the training materials directly in Galaxy. With that mode activated, hands on steps are now interactive and clicking tools brings you directly to the tool in Galaxy.

Recording and automated video lectures.

In order to further expand the formats in which our training materials are available, several tutorials have been recorded by community members, are available on GTN Training Video Library [25] and accessible directly from the tutorial itself. Recording a video is a laborious and time-intensive activity. To save a significant amount of instructor time, we automatically generate videos based on lecture slide decks. The slides are narrated using automated text-to-speech (TTS), and the script is based on the speaker’s slide notes, which makes for an extremely easy video update process—updating the slides automatically re-records the videos.

Coding tutorials.

Jupyter and RStudio [26] can both be run within Galaxy in the form of an interactive tool, allowing for interactive tutorials that offer a combination of Galaxy-based analysis and coding-based R or Python analysis steps. In addition to covering a wide range of analyses in Galaxy, the GTN also supports coding-oriented tutorials in the form of Jupyter [27] and RMarkdown [28] notebooks that are automatically generated from the tutorial content. A large set of introductory coding tutorials have recently been developed in a new “Foundations of data science” topic.

FAIR training.

The GTN infrastructure has been developed in accordance with the FAIR principles for training materials [16] (Table 1) and following the 10 simple rules for collaborative lesson development as defined by Deveny and colleagues [29] (Table 2). Following these principles enables trainers and trainees to find, reuse, adapt, and improve the available tutorials.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Implementation of the “Ten simple rules for making training materials FAIR” [16] in the GTN.

https://doi.org/10.1371/journal.pcbi.1010752.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Implementation of the “Ten simple rules for collaborative lesson development” [29] in the GTN.

https://doi.org/10.1371/journal.pcbi.1010752.t002

Train-the-trainer (TtT) tutorials.

To support teachers and trainers, we have developed a dedicated topic in the GTN for TtT tutorials, covering all aspects of using the GTN materials in education. This includes best practices providing pedagogical, technical and logistical recommendations, accessible on the website with a series of tutorials (available in “Teaching and Hosting Galaxy training” category). For first-time instructors, the GTN has come together to collect different instructors’ experiences (“Training Philosophies” [31]).

Preparing a workshop.

During preparation of a workshop, organizers and instructors work together to identify relevant tutorials for their event. Our tutorials are roughly divided into “topics” such as Transcriptomics or Climate Science, with a search function available on the GTN website (https://training.galaxyproject.org). Within topics, multiple tutorials can be found. Each tutorial starts with a list of metadata such as learning objectives following Bloom’s taxonomy [32], prerequisites, time estimate, or questions addressed by the tutorial (Fig 3). This but also tags of tutorials enables instructors to identify the best tutorials for their audience. Reusable slide decks with speaker notes are also available to introduce the topics prior to a tutorial.

To ensure the success of practical sessions, the tutorials rely on specific datasets (included in the tutorial metadata) and tools that the GTN analyzes in order to provide a list of compatible public Galaxy servers. Automated workflow testing [33] provides reassurances to instructors, letting them know that a given tutorial keeps working on their selected server.

We have added support for FAQs to be added to tutorials, in order to help instructors prepare their lesson. These FAQs typically cover common questions or frequently observed trainee mistakes encountered during a particular tutorial.

Preparing workshop infrastructure.

A major challenge in computational workshop organization is the identification of affordable and reliable compute infrastructure. The needs of training infrastructure are significantly different from regular research; analysis steps must complete in relatively short time periods in order to not disrupt the flow of the lesson. The GTN uses small (sub-sampled) training datasets for tutorials in order to reduce the run time of the analysis step, but a second factor to consider is the waiting time of the compute infrastructure. In order to support training events, we developed Training Infrastructure as a Service (TIaaS) [34]. This free service provides a dedicated job queue for the participants of training events, in order to reduce waiting times on the cluster and ensure courses run smoothly and efficiently. Furthermore, the TIaaS service also provides instructors with a dashboard, enabling them to monitor the progress of the participants. To date, TIaaS has been widely used: 17,000+ students were taught over the course of 330+ events between June 2018 and May 2022, with 65% using the GTN materials (Fig 1B and 1C).

Teaching a workshop.

During the workshop, instructors can introduce the topic using the available slide decks, supported by detailed speaker notes. After this theoretical introduction, instructors may either use a live-demo approach, guiding students through the tutorials in a step-by-step fashion, or alternatively let students work through the tutorials at their own pace while providing support. The tutorials are a fully self-contained teaching resource as described in Fig 3, and therefore, support both these training modalities.

Feedback from instructors.

In January 2020, we conducted a survey asking how trainers used the available training material and infrastructure. We received answers from 33 trainers, 88% of which had conducted a training event in the last 3 years. This sample consisted of a continuum of trainers, from occasional to seasoned trainers. Of those recently giving training events, 79% of them used GTN resources, and some even developed new GTN materials for the occasion.

The GTN tutorials are well liked and shared; 91% of respondents indicating they would recommend them. The use of GTN material allows them to have more time to focus on the fundamental principles of analyses and the specific needs of the trainees. Of the surveyed trainers, 69% are contributors to the GTN, and among those who are not, 55% declare that they plan to become one in the future.

Tutorials about contributing.

To get acquainted with the tutorial development process, a contributor can start by consulting a set of dedicated tutorials in the Contributing topic in the GTN. These tutorials provide a step-by-step guide to creating a tutorial in the GTN framework, covering everything from technical guides about the framework to pedagogical best practices and submission of tutorials to GitHub.

Fig 4 depicts the typical process a tutorial contributor will go through when developing a new tutorial. The first step is usually to develop the analysis workflow in Galaxy. The challenge is to identify or create suitable input datasets. The selected data must be informative enough to illustrate the meaning of a given analysis, but not too large as to require long waiting times for upload or processing during a training event. The selected data could be a toy dataset created from scratch, or (preferably) an informative subset of a real-life dataset, for example using a single chromosome of a whole-genome dataset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Typical process to create a new tutorial.

Authors usually start by identifying suitable input datasets and developing the workflow in Galaxy. Thus, a workflow can be automatically converted into a tutorial skeleton using the PTDK described in the next section. The tutorial is then tested, reviewed, and finally merged into the GTN.

https://doi.org/10.1371/journal.pcbi.1010752.g004

Planemo Tutorial Development Kit (PTDK).

To simplify the tutorial development process, we have created a Tutorial Development Kit within Planemo, the Galaxy development kit [35]. This automatically generates a tutorial skeleton from a Galaxy workflow as starting point for tutorial authors. This skeleton contains the overall structure of the tutorial: metadata section, example question boxes for formative assessments, instructions on how to proceed with adding scientific and pedagogical content, and more important auto-generated hands-on boxes for every analysis tool in the workflow with the parameters to select. This process greatly reduces the development time and allows tutorial creators to focus on the scientific content of the tutorial, rather than the technical details and style guidelines.

In order to further lower the contribution barrier, this Planemo Tutorial Development Kit has been encapsulated into a web service (https://ptdk.apps.galaxyproject.eu/), where tutorial authors can provide a link to a public workflow on one of the UseGalaxy.* servers and obtain the tutorial skeleton based on this workflow in Markdown, but also the workflow and data library file if a Zenodo link is provided. This removes the need for contributors to install and run Planemo locally. We have found that this approach saves time for not only the contributors but also for reviewers, since the tutorials contributed using this method adhere well to the GTN style guide.

Content reuse.

To avoid duplication of content between tutorials, we have developed a set of modular tutorial components, called snippets, which can be easily reused across tutorials. For example, common tasks in Galaxy such as starting a workflow or creating a new history will be a part of most tutorials. Contributors can include these snippets at any place in a tutorial with a simple import statement. If instructions for this common task change, e.g., due to changes in the Galaxy user interface, the changes need to only be made in the snippet itself and will automatically propagate to any tutorial using them.

Tutorial preview and testing.

GTN tutorials are written in Markdown format for ease of contribution and subsequently converted to HTML web pages automatically by our framework. The HTML web page can be locally previewed if desired by using simple commands. Additionally, contributors who do not wish to install and run the GTN locally, can also generate a preview of their in-development tutorials online using GitPod [36]. We have integrated the GTN GitHub repository with GitPod, enabling contributors to obtain an online tutorial development environment, complete with online preview, with a single click of a button. GitPod has proven particularly useful for collaboration between multiple lesson developers with varying degrees of familiarity with git or GitHub.

In addition to a visual inspection of the generated web pages, our framework offers a suite of testing tools, allowing contributors to check that their tutorial meets the technical and style guidelines. These tests include checks of whether all required metadata is present, whether links within the tutorial are valid, and whether files are correctly formatted. This helps contributors and reviewers to quickly identify and correct potential problems with a tutorial.

Peer review.

Once a contributor is happy with their tutorial, they can create a pull request (PR) to the GTN GitHub repository (https://github.com/galaxyproject/training-material), where each contribution will then undergo automatic quality assurance tests and a peer-review process.

This review process is completely open, and any volunteers from the community may participate. For each topic within the GTN, we have encouraged several prominent community members who are regular contributors to act as topic maintainers that help safeguard the quality of the content in the topic and are empowered to review, approve, and merge contributions. Maintainers and other reviewers will check the proposed contribution, both in terms of formatting and scientific content. They can make suggestions for updates and start a discussion with the contributor(s). Typically, there will be 2 or 3 reviewers for any given PR. Based on these reviews, the contributor(s) will update the tutorial, and this cycle of update and review will continue until both contributor and reviewer are happy with the result. In case of disagreements, topic maintainers will decide on the path forward.

This open strategy for content creation and updates is paying off. Since the beginning of the project in the middle of 2016, over 2,500 PRs have been created to add or update tutorials.

The review process is quite fast: it takes 20 days between the opening of a pull request and its merge. Thanks to a fast turnaround, tutorials are regularly updated and iteratively improved over time, keeping the material relevant and of a high quality. Indeed, each tutorial undergoes (on average) a PR every quarter to update it. Some highly used tutorials, like the “Reference-based RNA-seq” tutorial, have received 100+ PRs over the last 3 years, on average one every 3 weeks.

Maintenance.

Tutorials are dynamic entities; underlying analysis tools receive updates, new tools are developed, and the Galaxy interface itself changes regularly. As a result, tutorials must undergo regular updates as well to reflect the state-of-the-art in the scientific domain and correctly reflect the latest available version of the tools and the Galaxy interface. Instructors preparing for their next workshop check the tutorials as preparation, and in the process, identify the places where the tutorial should be updated. If the instructors feel comfortable making these changes themselves, they can open a PR proposing the changes. If not, they can create an issue on GitHub to request the changes.

Feedback.

Learner feedback is one of the most valuable resources to inform training improvements [37]. An anonymous feedback form is embedded at the end of every tutorial, where users can provide feedback and make suggestions for enhancement. All feedback is transparently collected on the GTN website, enabling the community to view and address the feedback (Fig 2D).

GTN in higher education.

Galaxy and the GTN are an integral part of Bioinformatics programs at the undergraduate level (e.g., Clermont Auvergne University, Texas A&M University, Avans Hogeschool, University of Freiburg, University of Frankfurt).

In these undergraduate courses, Galaxy is often used to teach the concept of data analysis, pipelines/workflows, and also reproducibility. Students learn the importance of the tool versions, as tools can change subtly or significantly over time and this may impact their analyses. Teachers can likewise showcase the evolution of algorithms over time using different versions of tools (e.g., bowtie and bowtie2) to help students understand the advances in the field. Without the need to spend resources on teaching command line skills and tool installation, the time can be fully used to introduce each step (data cleaning, data analysis, and visualization) and to go into details of each tools and algorithms. Galaxy is also convenient to explain how data is structured (data types, metadata) and its connection to tools. Finally, Galaxy can be used to illustrate complete analytical workflows paying attention to input and output data.

GTN for research scientists.

Postgraduate courses (e.g., Agrocampus Ouest, Rennes University, Brest University, Clermont Auvergne University, Station Biologique de Roscoff, Melbourne University, University of Freiburg) as well as internal bioinformatics short-format training sessions (e.g., Friedrich Miescher Institute, French Bioinformatics Institute, Erasmus MC, University of Freiburg) are relying on Galaxy as the main teaching tool. These courses are often directed at learners who are not yet comfortable working with the command line. However, we often notice that after getting familiar with each analysis step and understanding the meaning of parameters of each tool, some may feel confident to learn command line or scripting (possibly within Galaxy using RStudio or Jupyter notebook) and to move to command line environments.

GTN materials are also often used to provide supplemental training to later-career research scientists, for example, at the Erasmus Medical Center (EMC), Earlham Institute, and as part of the Gallantries project [18]. These trainings often consist of short-running workshops, usually aimed at familiarizing researchers with novel analysis techniques. Since many of the GTN tutorials are centered around an analysis pipeline described in a recently published article, they are ideally suited to provide researchers with an update on the latest state-of-the art analysis pipelines in their domain. GTN tutorials are also frequently created as part of scientific publications by authors presenting a novel analysis method, as an additional form of documentation for the readers [38–43].

Training in underrepresented communities.

Reliable internet access is often taken for granted by researchers. Bottlenecks in infrastructure may, however, cause significant issues for bioinformatics training events, as many students try to upload or download data at once. This is especially problematic in low- or middle-income countries (LMICs) where internet access may be intermittent, restricted, or completely unavailable.

Trainers are often brought in from afar, meaning that the teaching takes places in—for the trainer—an unfamiliar setting and on a strict time schedule limited by the return trip(s) of those involved. It is therefore important to avoid or minimize unforeseen delays caused by incompatibilities with local infrastructure, connectivity failures or unforeseen updates forcing new software to be downloaded or queries to remote servers to fail.

Based on these needs, the eBioKit [44] was developed: it is an assembly of open source or free-for-academic-use software, along with key databases and selected material for bioinformatics. It can be installed beforehand, brought on a portable server to the local training, and made available on the local network. Students can then access it through the network and work directly on the server, which avoids installation issues for the students, unforeseen updates of web services, or other failures. Galaxy has been a part of the eBioKit for most of its existence, specially in the eB3Kit that includes a Galaxy-based bioinformatics platform with a specialized workflow interface [45].

Various versions of the eBioKit has been used for over a decade by organizations such as EMBnet, H3Abionet, SANBio, and BECA/ILRI to train hundreds of researchers and bioinformatics trainers in LMICs.

Teaching based on Galaxy is also valuable in LMICs with limited internet access. The training datasets are all available online, either on the Galaxy server, in shared data libraries, or in a third-party service such as Zenodo. Thus, the learners own (poor) internet connections are bypassed for these bandwidth-heavy tasks. The internet is only used to launch the analysis steps and for the teacher to monitor the progress of the analysis in Galaxy via TIaaS.

For cases where internet access is frequently absent, the GTN offers Docker [46] images, allowing for completely offline training. Every topic has its own Docker image, preconfigured with all the necessary tools, workflows, tours, and data-libraries to complete the tutorials within that topic. A drawback of this solution is that all required compute resources to complete the tutorials must be available locally.

Citizen science and education.

Thanks to its graphical user interface, Galaxy can also be used to introduce bioinformatics to a general audience and include them into scientific projects.

The Street Science Community [47] offers workshops to introduce biology, genomic sciences, and bioinformatics to the public. For example, participants extract yeast DNA out of beer bottles and sequence it using a MinION, the Oxford Nanopore sequencing device. The generated sequencing data are then processed by participants inside Galaxy: uploading data, running a metagenomic analysis workflow, and visualizing results. Combining lab work and bioinformatics data analysis with Galaxy vividly demonstrates the challenges and possibilities that genomics brings to our society.

The GTN has also provided an opportunity for the ecology community to experiment with several ways to extend citizen science schemes, for example, through 2 monitoring schemes from Vigie-Nature [48], French citizen science programs about common birds (Suivi Temporel des Oiseaux Communs) and bats (Vigie-Chiro). Volunteers use Galaxy to analyze data they gathered, as researchers would do, through user-friendly tools and workflows, documented in GTN. Such solutions increase the motivation of participants, especially volunteers, as they can analyze and visualize data about species and ecological systems they monitor, sometimes for several years.

Training in the COVID-19 era: Remote learning.

Due to the global COVID-19 pandemic, many educators have found themselves forced to change the modality of their training activities to completely virtual events. Galaxy and the GTN cater to remote learners and teachers with a set of features to facilitate the online learning process [9]. It provides easy access to data and the possibility to share the progress and achievements, both student-to-student and student-to-instructor [6]. This has been extensively tested during the COVID-19 pandemic. For example, the GTN community organized the GTN Smörgåsbord events [49], global, 5 day, 24/7 events. These were fully asynchronous events, where instructors prerecorded GTN tutorials, which the 2,000+ registered participants could work through at their own pace, with 120+ instructors available online for support on Slack across all time zones. This approach has been repeated for the Galaxy Community Conference training week [50], as well as various other virtual training events, including (but not limited to):

1. A 5-day workshop on “Machine Learning using Galaxy” organized by the Galaxy Europe team in June 2020, with 400+ registrations and 200+ participants on the first day [51].
2. A 5-day plant transcriptomics workshop organized by the Galaxy Europe team [52].
3. A Microbiome Informatics 5-day workshop organized by the Galaxy-P team with Galaxy India [53].
4. A SARS-CoV-2 data analysis workshop [54].
5. A case study in COVID-19 data analysis at the Great Lakes Bioinformatics Conference (GLBIO 2021) [55].
6. Metatranscriptomics analysis using microbiome RNA-seq data in Galaxy [56].
7. The Spanscriptomics workshop, a pilot study into using Spanish-translated GTN tutorials to teach single-cell analysis to native Spanish speakers [57].

Hybrid learning in geographically sparse locations.

Even before the pandemic, learners in remote areas have faced significant barriers to traveling to in-person training events. In such areas, a so-called hybrid training [19] approach offers a solution. In such an approach, learners gather in classrooms in several geographically distinct locations. The training is live-broadcast to each of these locations and communication with the instructor happens in real-time.

GTN resources have been successfully used for such hybrid training events, for example, by Galaxy Australia, which organized a large number of hybrid training events, with up to 11 satellite classrooms across the region participating, and reaching over 800 learners [19]. The Gallantries project [18] has successfully applied a similar approach in the European region.