rOpenSci | Learn All About HTTP Testing: Book Update

Learn All About HTTP Testing: Book Update

More and more R packages access resources on the web, and play crucial roles in workflows. Examples from the rOpenSci suite of packages include rromeo, GSODR, qualtRics, rnassqs, and many, many others. Like for all other packages, appropriate unit testing can make them more robust. However, unit testing of these packages can bring special challenges: dependence of tests on a good internet connection, testing in the absence of authentication secrets, etc. Having tests fail due to resources being down or slow, during development or on CRAN, means a time loss for everyone involved (slower development, messages from CRAN). Although many packages accessing remote resources are well tested, there used to be a lack of resources around best practices… But now there is one great information source, our HTTP testing in R online book! 🎉

🔗 Motivation and support for the book

The best guidance about HTTP testing for R packages used to be an rOpenSci forum entry by Noam Ross that pre-dated the development of relatively new packages for HTTP testing that have now been released on CRAN: vcr and webmockr by Scott Chamberlain, httptest by Neal Richardson, webfakes by Gábor Csárdi. These packages help make HTTP testing smooth to set up and resilient to internet connection failures, which means faster tests, tests running well even when on a spotty internet connection, and less burden on the remote resources.

We have updated the HTTP testing in R online book to make it a free, central reference for developers of R packages accessing web resources, to help them have a faster and more robust development. Scott created the book a while back, as central documentation for his packages crul, vcr and webmockr. Maëlle got funding from the R Consortium to expand the scope of the book. Thank you, R Consortium! Thanks also to Scott, Adam Sparks, Matthias Grenié and Neal Richardson for contributing to the project proposal. Thanks also to Gábor Csárdi who was also consulted… even before he created webfakes! Last but not least, thanks to recent readers of the book like Dave Parr, Lluís Revilla Sancho and Hugo Gruson for useful feedback.

🔗 What’s between the covers?

The book still has advanced content about webmockr and vcr, but now this comes after new content.

The section also features a comparison of the two packages using mock files / cassettes for storing API responses or interactions (vcr and httptest), and a comparison of all packages.

🔗 Other technical details

Whilst writing chapters Maëlle filed several docs and feature requests to vcr, httptest, webfakes. All three maintainers were very kind and receptive.

  • There was a recent tech note about vcr release, part of which is a consequence of the book update.
  • In httptest the new httptest::with_mock_dir() function was contributed as a result of Maëlle’s work.
  • In webfakes Maëlle suggested and helped develop the OAuth2.0 apps.

Besides, the book itself is an early adopter of a brand-new bookdown template by Hadley Wickham, bookdown::bs4_book(), which you might want to check out for your own bookish endeavors! It’s only available in the development version of bookdown at the moment of writing, and there quite a few features still in the work. As for deployment, we’d recommend looking at the GitHub Actions workflow for the R packages book that we drew inspiration from.

🔗 Further work and feedback request!

While we are happy with our “HTTP testing in R” book, we are committed to updating and improving it! Have a look at the book issue tracker, and feel free to give us feedback there or on rOpenSci forum. Comments by experts and newbies and any expertise level in-between are most welcome! Happy HTTP testing!