Hello Quarto

BioC 2022

mine çetinkaya-rundel

mine.quarto.pub/hello-quarto-bioc

Thank you so much, we’re excited to be here! I have been using, teaching, and contributing to the development of Quarto over the last year. As I’ve woven it into my work I realized that I love it, and I’m very thrilled to talk to you about it today.

Acknowledgements

Julia Stewart Lowndes, Phd

Artwork by Allison Horst

Hello, Quarto?

You might be wondering – What is Quarto? Why is Quarto? And how does it relate to R Markdown and the workflows you already have?

–> Well

Quarto unifies and extends
the R Markdown ecosystem

Quarto unifies & extends the R Markdown ecosystem.

Quarto unifies and extends
the R Markdown ecosystem

unifies for people who love R Markdown

extends for people who don’t know R Markdown

It unifies it for people who love R Markdown by reducing little points of friction, so that websites, books, and slides all have a common approach.

And it extends it for people who don’t know RMarkdown by being a friendly way to work reproducibly and publish documents that have text and code in the same place

Poll

slido.com / #quarto

Quarto is a new, open-source,
scientific and technical
publishing system

Quarto is a new open-source scientific and technical publishing system.

Quarto is a new, open-source,
scientific and technical
publishing system

the goal is to make the process of creating
and collaborating dramatically better

The goal is to make the process of creating and collaborating dramatically better.

–> It handles inputs in multiple languages, and outputs in multiple formats, all going through the same machinery.

First Mine will focus on how Quarto unifies the R Markdown ecosystem through stories of sharing and teaching

I’m going to introduce Quarto in four acts – Share, Collaborate, Teach, and Reimagine. Each with a story and a demo. So, let’s get started.

Share

Julie: We’re excited to get started! So lets starts off with how Quarto enables sharing. Over to you Mine

Mine: Thanks Julie! I’m very excited to have this opportunity to talk to you about Quarto, which I’ve had the pleasure to live and breathe over the last year.

I work at the intersection of education, research, and development.

The word educator might make you think I’m in front of students lecturing 24/7 but, in reality, I spend most of my time in front of my computer authoring and creating materials to share with people in various ways. Whatever it is that I create, I share them all openly - that is, with an open license that allows for free use, reuse, and adaptation. More often than not I’m thinking “web first” for wider reach, but I also still live in the confines of traditional journal publishing and academia, which means I regularly need to create PDF versions of my work as well.

Therefore an authoring system that allows for multiple types of outputs from a single source that contains both code and prose is not only a time saver, but it’s basically essential for me to carry on my existence. And for the last decade, that tool for me has been R Markdown.

In fact, I remember vividly the first time I’ve heard of knitr, now 10 years ago, at Yihui and JJ’s 2012 useR keynote, when they presented on a “new” tool for reproducible authoring. It must have been no more than 3 slides into their talk that I started to converting my teaching materials to use knitr. And by the end of that keynote I was already brainstorming ways of teaching it to my intro stats students.

So, I’ll call this talk a success if some of you are already firing up Quarto right now, instead of listening to me, to see what all the hype is about!

mine.quarto.pub/hello-quarto-bioc

But I hope you’ll continue to listen with one ear and take in the beautiful illustrations by Allison Horst. And if you just want to enjoy the talk, feel free to go to this link.

sit back,

and I’ll advance the slides for you.

The R Markdown ecosystem

Julie mentioned unifying the R Markdown ecosystem so let’s dig a bit more into what that means…

Think of all the packages from the R Markdown universe that you’ve come to love and rely on over the years. Each package addresses a different need and offers a different output: for example, you can use blogdown or distill for creating websites and blogs, bookdown for writing a book, rticles for writing journal articles, etc.

Quarto: Next generation R Markdown

Quarto unifies the functionality from these packages,

building on Pandoc on the technical side

and on the human side – over a decade of experience developing, maintaining, and tweaking these packages as well as community feedback R Markdown extension packages.

Such a unification effort presents a fantastic opportunity to identify and address gaps functionality and inefficiencies in user experience, and the process of building Quarto has been no different.

Quarto highlights

Consistent implementation of attractive and handy features across outputs: tabsets, code-folding, syntax highlighting, etc.

More accessible defaults as well as better support for accessibility

Guardrails, particularly helpful for new learners: YAML completion, informative syntax errors, etc.

Support for other languages like Python, Julia, Observable, and more via Jupyter engine for executable code chunks.

Some highlights of these improvements include

–> consistent implementation of attractive and handy features across all outputs, like tabsets, code-folding, and syntax highlighting,

–> more accessible defaults as well as better support for creating accessible documents,

–> guardrails that are particularly helpful for new learners, like YAML completion and informative syntax errors,

–> and perhaps most excitingly for those who are not coming from the R ecosystem, Quarto offers support for other languages like Python, Julia, Observable, and more via the Jupyter engine for executable code chunks. And it’s designed to be expandable to more engines and languages, even those that might not exist today.

[pause] So by now I’m assuming many of you have already started playing with Quarto, or you’re waiting for me to do that. So, let’s dive in and see what all the Quarto fuss is about!

• Open hello.Rmd with source editor -> knit
• Change to visual editor -> knit
• Close file -> change file type to .qmd -> open + render
• Change output: html_document to format: html -> render
• Click on Render on save
• Change to format: pdf -> render -> then change back
• Add chunk option echo: false to the first chunk
• Add execute > echo: false to the yaml
• Add alt text to figure in second code chunk -> save to render
• Change to format: revealjs -> save to render
• Add H2 to break slides
• Change to format: html
• Add index.qmd with File > New Quarto Document
• Add _quarto.yml for website
• Relaunch project
• Show new Build tab
• Render site, show _site folder
• Add freeze to _quarto.yml
• From terminal: quarto publish

Quarto CLI orchestrates
each step of rendering

Now that you’ve all had a chance to see Quarto in action, you might be wondering, “how are R Markdown and Quarto different”?

The main difference is that with R Markdown, the R package rmarkdown does the heavy lifting of going from the source Rmd file to the the output, using knitr for evaluating the code chunks.

–> With Quarto, on the other hand, the Quarto command line interface, or the Quarto CLI, does the orchestration of processing executable code chunks with either knitr or jupyter and then converting the resulting markdown file to the desired output.

While this is technically impressive, I’ll be honest, it’s not exactly what sparked my interest in Quarto in the first place.

Quarto makes moving between formats straightforward

Document

lesson-1.qmd

title: "Lesson 1"
format: html

Presentation

lesson-1.qmd

title: "Lesson 1"
format: revealjs

Website

_quarto.yml

project:
  type: website

website: 
  navbar: 
    left:
      - lesson-1.qmd

What did spark my interest was how how straightforward it is with Quarto to move between output formats.

As an educator, two things are of utmost importance to me about the tools I use to create my materials: reproducibility and ease of transition between output formats, like documents to slides to websites to books.

Over the last year of using Quarto for pretty much everything, I’ve felt like I’ve finally found the tool that lets me go from one output type to the other with minimal, if any, futzing around with my source code beyond the yaml. For example, here are common things I produce

–> a lesson in document form
–> the same content in presentation form
–> the same content on a page in a website,

and you can see that all that needed to change going between these formats is a few lines in the yaml. Nothing in the content part of my document. No slide breaks to remove, no citation style to change, no headings to re-level. This ease of transition has freed up time to focus my time on content, and that, folks, is the dream!

Quarto facilitates open sharing

Tooling that makes it easy to share what you do
with the world on the web is incredibly useful

It’s useful not just to you, but to the world
as it facilitates open sharing of knowledge
for learning and for reuse

Ultimately, easily creating reproducible artifacts is great.

But for the sharing story to be complete, the tool also needs to support easily sharing those artifacts, like with a simple command like quarto publish, and hopefully soon a button RStudio that does the same thing.

–> I think that tooling that makes it easy to share what you do with the world on the web is incredibly useful.

–> And it’s not just useful to you, the creator, but to the world, as it facilitates open sharing of knowledge for learning and for reuse.

[Pause]

I’ve been talking just about me and my work so far, but I, like many of you, don’t operate in a silo disconnected from others.

[Mine continue on next slide]

Collaborate

Mine: Much of my work, and I’m sure much of yours is collaborative.

Many scientists
use Python
😬

Many scientists use python and jupyter notebooks. So your collaborations might involve going across languages, but you likely are not looking to learn Python from the ground-up for a collaboration alone. Additionally, if you’re an R Markdown user, you might already either be running Python code via reticulate to keep your work in R Markdown, or feel like you need to give up on the R Markdown authoring experience in your collaborations across languages. Jupyter notebooks are an obvious place for reproducible authoring, and a place where Python users particularly feel at home. And when you feel “at home” in a computing environment, you rarely want to leave it just for the purpose of collaboration, and instead you just with your collaborator would leave theirs and come to yours.

So, let’s take a pulse first.

Poll

slido.com / #quarto

Quarto enables collaborating
across coding languages

_quarto.yml

contents:
  - section: in-development/index.qmd
    contents:
      - in-development/earthdata-access-demo.ipynb
      - in-development/nsidc/icesat2-cmr-onprem-vs-cloud.ipynb
      - in-development/lpdaac/lpdaac_ecostress_lste_kerchunk.ipynb
      - in-development/matlab-aws.qmd
      - in-development/earthdata-python-r-handoff.rmd  
  - section: contributing/onboarding.qmd
    contents: 
      - contributing/quarto-website-tutorial.md

Quarto enables collaborating across coding languages. This is a simplified view of the yaml it’s collection of different file types. It’s mostly ipynbs Jupyter notebooks.

And, as we work with more research teams we’ve found that there is indeed a substantial R community using NASA Earthdata Cloud data. And we’re positioned to support R users too

Quarto enables contributing
from our current tools

With Quarto, you’re able to contribute from whatever tool you’re already comfortable with,whether that’s the RStudio IDE, JupyterLab, VS Code, or directly editing documents on GitHub.

This is so important because doing science is already challenging and and we don’t also need people to shift their software tools to be able to participate in the work.

transtion: Let me show you how to use Quarto with Jupyter with ipynb notebooks.

• Orient to JupyterLab. This is a JupyterHub managed by our collaborators at 2i2c
• Open Mine’s example site, quarto preview
• add .ipynb
• Add .ipynb to _quarto.yml
• View website - new page is added to website
• Add fig captions
• margin plots: a style preferred by many now in jupyter notebook

Native multi-language support has implications for teaching as well. So I’ll now shift gears to talk a bit about teaching next.

Teach

Over the last decade I have thoroughly loved teaching R Markdown. It’s amazing when students hit that Knit button, see their code and their prose come together – you can often hear a “wow, that’s so cool” in the classroom the first time that happens. I’ve also loved that this is a tool that they can pick up easily on the first day of an intro course, and it grows with them as their statistical and computational expertise grows. However, as students’ expertise in these areas grow, the computation gets more resource intensive (and even sometimes involving multiple languages) and they end up needing different output formats to support their needs, like PDFs in specific journal styles or a thesis with multiple chapters. A pleasant surprise in teaching with Quarto has been how much less steep the learning curve is for making this transition from single, simple document to a multi-document projects. Let me tell you the story of one of my thesis students.

Quantifying
Art Historical
Narratives

Holland Stam

Art History +
Visual and Media Studies

Class of 2022

Duke University

Last semester I had an art history student, Holland Stam, working on a pretty quantitative research project. Her thesis explored the representation of artists from different genders and ethinicities in foundational art history books and how this representation has changed, if at all, through editions. If you’ve ever taken an art history course you might be familiar with them: Garner’s Art Through The Ages and Janson’s History of Art. She meticulously collected data from numerous editions of each of the textbooks and measured, page by page, how much space each featured artwork was given and joined that with demographic information on the artists. When I met Holland in my regression course she had only one semester of data science as a background, where she learned R Markdown, and then she started using Quarto as part of coursework in my class. In both classes her experience with computational documents was limited to a single document per assignment, writing a bit of code to analyze data then writing some text to interpret findings. In a sense, this is exactly what her thesis entailed as well, but of course there was more. Multiple chapters, millions of citations in a very specific style that art historians use, captioned figures and tables, and not having to rerun the whole thesis code with each minor edit. With a bit of Googling and a bit of my help, she pretty quickly figured out how to bring chapters of her thesis written in Quarto together with a yaml file, and voiola! She had a thesis written in Quarto. Sure, this took a bit of trial and error and at times frustration, but the fact that she could caption her figures while creating them with code and insert citations without manual text styling, she was sold! It is always so heartwarming to see your students push what you’re teaching them and use them to solve their problems, it’s especially heartening when this happens with a new thing you’re teaching for the first time while the thing is still in active development!

Quantifying
Art Historical
Narratives

Holland Stam

Art History +
Visual and Media Studies

Class of 2022

Duke University

with Sara Lemus

And, by the way, if you’re interested in the data, Holland and another student of my, Sara Lemus, packaged it up as “arthistory”. Sara is a stats major who had a lot more experience with R and was working with me for updating some data packages I maintain to learn about package development. She got interested in Holland’s thesis during our lab meetings and it was clear the two of them, with their newly acquired skills and enthusiasm, should team up and make the data available to more researchers, and packaged it up and even got it on CRAN!

But, back to Quarto… like for Holland, Quarto has provided simple but effective solutions for so many of my problems as an instructor too. Instead of telling you about them, let me show you!

For learning:

• RStudio comes with everything, no additional install needed
• visual editor – Julie will say more
• code-link
• yaml errors

For teaching:

• revealjs
• Show hamburger menu outline and print to PDF
• chalkboard
• multiplex: Slides / scroll for audience
• execute > echo: true
• Add code line highlighting to plotting chunk

Features that are useful for research:

• quarto journals

Quarto flattens the learning curve

So, in regards to teaching, I’ve talked about a much less steep hill students need to climb as their projects grow. I’ve also said this was a pleasant surprise as it was not something I had thought about when I started teaching with Quarto. But, in hindsight, this was no coincidence. Quarto’s features like consistent citation syntax, more intentional control over when computations should run or rerun, and a single, minimal yaml that controls inputs, layouts, theming, etc. all contribute to flattening this learning curve.

Quarto can grow with learners in a profound way

And beyond learners growing and expanding the scope of their work easily with Quarto, Quarto can grow with them in a profound way as well.

–> The fact that Quarto is language and authoring tool independent offers an opportunity for folks to take Quarto with them on their data science journey, wherever that journey takes them. I mean.. I’m almost tempted to finally learn Python to see it in action in Quarto. But who I know is learning Python, and Julia, and other languages are my students in their Computer Science courses. To imagine that they could learn R in RStudio using Quarto in my course, and then continue to use Quarto in Jupyter in their CS courses… That sounds like such a bright future to me!

[Pause] And a bright future includes things we can’t even imagine yet.

[Mine continue on next slide]

Reimagine

So now let’s take one more leap to thinking about reimagining with Quarto.

And as I invite you to reimagine a future of open science, I want to give all a huge shoutout to my friend and colleague Julia Stewart Lowndes, who launched Openscapes along with her partner Erin Robinson in 2018. The flagship activity of Openscapes is mentoring research teams in open science. They help people move from the sad lonely animals like the foreground in the bottom right so that they start to explore and navigate this landscape safely, without too much trampling. It’s difficult for me to explain the fantastic and deep work Openscapes does in just a few soundbytes, so instead I’ll share with you a quote from Ileana Fenwick, who is a marine ecologist who participated in Openscapes last year.

Because of Openscapes, I have renewed hope in how we can bring about a kinder future in science. I see a commitment in the open science community to bridging the gaps where we’ve left members of our community behind, and it inspires me.

Ileana Fenwick
University of North Carolina, Openscapes

thefaylab.github.io/lab-manual/

Here is a practical example of how we can bring about a kinder future in science. This is the Fay Lab Manual – a site created by Gavin Fay’s Group UMass Dartmouth, a quantitative fisheries ecology lab group. This site includes information on lab culture, code of conduct, info about how they name files, how to learn the software they use. They transparently and clearly outline their practices, and don’t assume that people come in knowing this stuff or that they can figure them out on their own. It makes the implicit explicit and onboards learners as contributors. This is great practice because formal training in data science is uncommon among environmental scientists, and people are bound to come in with a range of skills. So let’s showcase one more way Quarto can play a role in this approach of onboarding learners as contributors.

Quarto enables collaborating
across coding comfort levels

Quarto enables collaborating across coding comfort levels, which means it’s a friendly tool and that we can broaden our idea of collaborators substantially and earlier.

One way Quarto and the RStudio IDE facilitates this is via the visual editor, which I’ll do one final demo of.

I’d like to note that while the visual editor is friendly for new learners, it is also a great tool for experienced users. I love using the visual editor for authoring and I’ll show you in a second why! As you watch the demo I encourage you to think about how you might use this yourself and also teach it to a new student or a colleague.

• i can write and paste formatted text
• Add image, caption, alt text
• source editor
• citations 2 ways: @ from my Zotero, forward-slash citation for doi
• Also: Call-outs
• Add extension

Thank you

mine.quarto.pub/hello-quarto-bioc

Thank you so much for joining this morning. This was an introduction to Quarto from the perspective of sharing, collaborating, teaching, and reimagining. I’ve only managed to scratch the surface of Quarto’s features so I invite you to dig deeper into the documentation and tutorials and play with it yourself. The slides for the talk are at the link above. And if you go there and proceed to the next slide you’ll find many useful resources, including the source code for the site we built in the demos. While Quarto is stable and ready for use, it’s also a baby that you too can be a part of its growth. So start using Quarto, and get involved by asking questions, reporting issues, providing feedback, and even building extensions for it. We can’t wait to see what you create!

Say hello to Quarto!

• Pick up where we left off:

  • Site: mine.quarto.pub/hello-quarto-demo
  • Source code: github.com/mine-cetinkaya-rundel/hello-quarto-demo

• Source code for these slides: github.com/mine-cetinkaya-rundel/hello-quarto-bioc

• Quarto documentation: quarto.org

• Video: Reproducible authoring with Quarto

• Video: Openscapes: Hello Quarto with NASA Openscapes, RLadies Santa Barbara, and JJ Allaire

• Tutorial: rstudio-conf-2022.github.io/get-started-quarto/

• Quarto gallery: quarto.org/docs/gallery

• Quarto tip a day: rstd.io/quartotip